Kubernetes集群实践（10）滚动更新和回滚

Kubernetes集群的证书有效期是1年，若是超过有效期，kublet服务会无效，此时继续使用kubelet命令，将会获得相似这样的提示“Unable to connect to the server: x509: certificate has expired or is not yet valid.”所以，咱们须要对其进行维护。node

1. 登录Master节点查看证书有效期

kubeadm alpha certs check-expiration

2. 备份现有证书

mkdir -p $HOME/k8s-old-certs/pki
cp -p /etc/kubernetes/pki/*.* $HOME/k8s-old-certs/pki
ls -l $HOME/k8s-old-certs/pki/

结果：api

total 56
-rw-r--r-- 1 root root 1261 Sep  4  2019 apiserver.crt
-rw-r--r-- 1 root root 1090 Sep  4  2019 apiserver-etcd-client.crt
-rw------- 1 root root 1679 Sep  4  2019 apiserver-etcd-client.key
-rw------- 1 root root 1679 Sep  4  2019 apiserver.key
-rw-r--r-- 1 root root 1099 Sep  4  2019 apiserver-kubelet-client.crt
-rw------- 1 root root 1679 Sep  4  2019 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 Sep  4  2019 ca.crt
-rw------- 1 root root 1675 Sep  4  2019 ca.key
-rw-r--r-- 1 root root 1038 Sep  4  2019 front-proxy-ca.crt
-rw------- 1 root root 1675 Sep  4  2019 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 Sep  4  2019 front-proxy-client.crt
-rw------- 1 root root 1679 Sep  4  2019 front-proxy-client.key
-rw------- 1 root root 1675 Sep  4  2019 sa.key
-rw------- 1 root root  451 Sep  4  2019 sa.pub

3. 备份配置文件

cp -p /etc/kubernetes/*.conf $HOME/k8s-old-certs
ls -ltr $HOME/k8s-old-certs

结果：服务器

total 36
-rw------- 1 root root 5451 Sep  4  2019 admin.conf
-rw------- 1 root root 5595 Sep  4  2019 kubelet.conf
-rw------- 1 root root 5483 Sep  4  2019 controller-manager.conf
-rw------- 1 root root 5435 Sep  4  2019 scheduler.conf
drwxr-xr-x 2 root root 4096 Dec 19 21:21 pki

4. 备份Home目录的配置文件

mkdir -p $HOME/k8s-old-certs/.kube
cp -p ~/.kube/config $HOME/k8s-old-certs/.kube/.
ls -l $HOME/k8s-old-certs/.kube/.

结果：ide

-rw------- 1 root root 5451 Sep  4  2019 config

5. 更新Kubernetes证书

kubeadm alpha certs renew all

结果rest

certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healtcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed

6. 在次检查证书有效期是不是364天后无效

kubeadm alpha certs check-expiration

7. 确保kubelet服务器正常，同时work和master间通讯正常。
8. 等待几分钟，使用以下命令确保work可用

kubectl get nodes

若是输出如下信息code

The connection to the server 9.37.21.119:6443 was refused - did you specify the right host or port?

则须要执行下面步骤继续进行修复server

9. 检查比/etc/kubernetes/kubelet.conf文件

diff $HOME/k8s-old-certs/kubelet.conf /etc/kubernetes/kubelet.conf

若是没有任何输出，则更新证书操做没有影响此文件，继续下面的步骤手动修复。ci

10. 更新/etc/kubernetes/kubelet.conf文件

cd /etc/kubernetes
sudo kubeadm alpha kubeconfig user --org system:nodes --client-name system:node:$(hostname) > kubelet.conf
diff $HOME/k8s-old-certs/kubelet.conf /etc/kubernetes/kubelet.conf

更新后能够查看与备份文件的区别.get

11. 检查对比~/.kube/config文件

diff ~/.kube/config $HOME/fcik8s-old-certs/.kube/config

若是没有任何输出，则该文件包含过时的key和证书，继续下面的步骤手动修复。kubernetes

12. 使用当前更新后的/etc/kubernetes/kubelet.conf中的'client-certificate-data '和'client-key-data'的值更新~/.kube/config文件中对应的值。
13. 重启kubelet服务

systemctl daemon-reload&&systemctl restart kubelet

14. 再次查看节点和Pod是否正常

kubectl get nodes
...
kubectl get pods

参考资料：https://www.ibm.com/docs/en/fci/1.1.0?topic=kubernetes-renewing-cluster-certificates