主机配置
| 主机名 | IP | 角色 |
|---|---|---|
| ooovooo.org | 172.25.254.100 | Harbor仓库 |
| k8s-master.org | 172.25.254.200 | Master主机 |
| k8s-node1.org | 172.25.254.10 | Node主机 |
| k8s-node2.org | 172.25.254.20 | Node主机 |
Master主机、Node主机、Harbor仓库
部署Docker-Ce
vim /etc/yum.repos.d/docker.repo
[docker]
name=docker
baseurl=https://mirrors.aliyun.com/docker-ce/linux/rhel/9/x86_64/stable/
gpgcheck=0dnf install docker-ce -y
添加主机解析
vim /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.25.254.100 ooovooo.org
172.25.254.10 k8s-node1.org
172.25.254.20 k8s-node2.org
172.25.254.200 k8s-master.org
禁用Swap、SeLinux、FireWalld
# 注释参数
vim /etc/fstab
# /dev/mapper/rhel-swap none swap defaults 0 0# 锁定 swap 功能
systemctl mask swap.target
swapoff -a# 无内容显示，即配置成功
swapon -s
# 建议重启主机,并查看是否开机启动 swap 功能# 关闭防火墙
systemctl stop firewalld
# 锁死防火墙
systemctl mask firewalld# 关闭SeLinux
grubby --update-kernel ALL --args selinux=0
# 关闭SELinux,修改成disabled
vi /etc/selinux/config
22 SELINUX=disabled
Harbor仓库
下载地址：https://github.com/goharbor/harbor/releases
生成认证 和 证书
mkdir /data/docker/certs/ -p
openssl req -newkey rsa:4096 \
-addext "subjectAltName = DNS:ooovooo.org" \
# Harbor仓库域名,且/etc/hosts有解析
-x509 -days 365 -out /data/docker/certs/ovo.org.crt \
-nodes -sha256 -keyout /data/docker/certs/ovo.org.key...
# 其他随意填写,此项要注意
Common Name (eg, your name or your server's hostname) []:ooovooo.org
# 与"subjectAltName = DNS:ooovooo.org" 域名保持一致
...# 进行认证
mkdir /etc/docker/certs.d/ooovooo.org/ -p
cp /data/docker/certs/ovo.org.crt /etc/docker/certs.d/ooovooo.org/ca.crt
# ca.crt 固定名字# 若没有认证成功,可能显示报错
Error response from daemon: Get "https://ooovooo.org/v2/": tls: failed to verify certificate: x509: certificate signed by unknown authority
安装Harbor
tar zxf harbor-offline-installer-v2.5.4.tgz
# 修改 Harbor.yml 文件cd /root/harbor/
cp harbor.yml.tmpl harbor.ymlvim harbor.yml
5 hostname: ooovooo.org
17 certificate: /data/docker/certs/ovo.org.crt
18 private_key: /data/docker/certs/ovo.org.key
34 harbor_admin_password: aaa
# Username：admin
# Password：aaa./install.sh --with-chartmuseum
# kubernetes的helm工具
登录Harbor仓库
Master主机、Node主机、Harbor仓库 均需要登录
# 登录
docker login ooovooo.org
Username: admin
Password: aaa# 登出
docker logout ooovooo.org
自定义镜像拉取目录
Master主机、Node主机、Harbor仓库 均需要配置
cd /etc/docker/
vim daemon.json
{"registry-mirrors":["https://ooovooo.org"]
}
systemctl restart docker
# 从Harbor仓库的 library 目录中拉取镜像
Master主机
配置Cri-Docker
# 上传 cri-dockerd 和 libcgroup
scp libcgroup-0.41-19.el8.x86_64.rpm cri-dockerd-0.3.14-3.el8.x86_64.rpm root@172.25.254.10:/root/
scp libcgroup-0.41-19.el8.x86_64.rpm cri-dockerd-0.3.14-3.el8.x86_64.rpm root@172.25.254.20:/root/dnf install libcgroup-0.41-19.el8.x86_64.rpm cri-dockerd-0.3.14-3.el8.x86_64.rpm -y
systemctl enable --now cri-dockervim /lib/systemd/system/cri-docker.service
# 指定网络插件及基础容器镜像
# 根据Harbor仓库中 k8s 目录下 k8s/pause 的Tags来填写pause的版本
10 ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=cni --pod-infra-container-image=ooovooo.org/k8s/pause:3.9
配置kubernetes
vim /etc/yum.repos.d/k8s.repo
[k8s]
name=k8s
baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.30/rpm/
gpgcheck=0dnf install kubeadm kubectl kubelet -y
Kubectl 命令补齐
dnf install bash-completion -y
echo "source <(kubectl completion bash)" >> ~/.bashrc
source ~/.bashrc
上传镜像到Harbor仓库
kubeadm config images pull \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.30.0 \
--cri-socket=unix:///var/run/cri-dockerd.sock# Harbor仓库需要有 k8s 目录且公开
docker images | awk '/google/{ print $1":"$2}' \
| awk -F "/" '{system("docker tag "$0" ooovooo.org/k8s/"$3)}'docker images | awk '/k8s/{system("docker push "$1":"$2)}'
Master主机初始化
kubeadm config print init-defaults # 可查看初始化选项systemctl enable --now kubelet.service
systemctl daemon-reload
systemctl restart cri-dockerkubeadm init --pod-network-cidr=10.244.0.0/16 \
# 集群使用的网络,不可修改
--image-repository ooovooo.org/k8s \
# 指定启动容器集群时,从哪里拉取容器镜像
--kubernetes-version v1.30.0 \
# kubernetes版本
--cri-socket=unix:///var/run/cri-dockerd.sock
# 指定sock文件
-------------------------------------------------
# 若初始化失败,需要执行
kubeadm reset --cri-socket=unix:///var/run/cri-dockerd.sock
# 才能再次重新初始化# 初始化完毕后,获取Master节点状态
kubectl get nodes
# 此时Master未处于 Ready 状态,缺乏网络插件
安装Flannel网络插件
# 拉取Flannel
wget https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
docker pull docker.io/flannel/flannel:v0.25.5
docekr pull docker.io/flannel/flannel-cni-plugin:v1.5.1-flannel1# 上传至Harbor仓库,仓库需要有 flannel 目录
docker tag flannel/flannel:v0.25.5 ooovooo.org/flannel/flannel:v0.25.5
docker push ooovooo.org/flannel/flannel:v0.25.5docker tag flannel/flannel-cni-plugin:v1.5.1-flannel1 ooovooo.org/flannel/flannel-cni-plugin:v1.5.1-flannel1
docker push ooovooo.org/flannel/flannel-cni-plugin:v1.5.1-flannel1vim kube-flannel.yml
146 image: flannel/flannel:v0.25.5
173 image: flannel/flannel-cni-plugin:v1.5.1-flannel1
184 image: flannel/flannel:v0.25.5
Node主机
配置Cri-Docker
dnf install libcgroup-0.41-19.el8.x86_64.rpm cri-dockerd-0.3.14-3.el8.x86_64.rpm -y
systemctl enable --now cri-dockervim /lib/systemd/system/cri-docker.service
# 指定网络插件及基础容器镜像
# 根据Harbor仓库中 k8s 目录下 k8s/pause 的Tags来填写pause的版本
10 ExecStart=/usr/bin/cri-dockerd --container-runtime-endpoint fd:// --network-plugin=cni --pod-infra-container-image=ooovooo.org/k8s/pause:3.9
配置kubernetes
kubeadm 和 kubelet 即可
vim /etc/yum.repos.d/k8s.repo
[k8s]
name=k8s
baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.30/rpm/
gpgcheck=0dnf install kubeadm kubelet -y
kubernetes 主机扩容
# Master重新生成集群token
kubeadm token create --print-join-command# Node节点加入集群
kubeadm join 172.25.254.200:6443 --token imt60l.2fcqnep75us948g3 --discovery-token-ca-cert-hash sha256:f536dbcbf4412bd1592fe32fff1470b241989ba1e38a8afc52a8d2459f52b81c --cri-socket=unix:///var/run/cri-dockerd.sock
# 在生成的token,加上--cri-socket=unix:///var/run/cri-dockerd.sock
------------------------------------------------------------------
# 若显示
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
# 即加入集群成功# 加入失败,需要重置
kubeadm reset --cri-socket=unix:///var/run/cri-dockerd.sock
kubernetes 测试
# 查看node状态,均处于Ready
kubectl get nodes# 运行任意镜像
kubectl run ono --image nginx
# 查看Pod信息,关注在哪个Node节点运行
kubectl get pods -o wide
# 在对应节点查看
docker ps
# 删除测试用例
kubectl delete pod ono