kubernetes 인증서 만료되어 kubectl 실행 안될 때

Unable to connect to the server: x509: certificate has expired or is not yet valid: current time 2023-10-18T14:59:15+09:00 is after 2023-10-17T01:26:03Z

위와 같은 에러 발생하며 kubectl 명령을 사용할 수 없는 상황

대충 영문 경고 해석해보면 느낌이 오겠지만, 인증서 기간이 만료된 것임

[root@Minhang02 pki]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[check-expiration] Error reading configuration from the Cluster. Falling back to default configuration

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Oct 17, 2023 01:26 UTC   <invalid>       ca                      no      
apiserver                  Oct 17, 2023 01:26 UTC   <invalid>       ca                      no      
apiserver-etcd-client      Oct 17, 2023 01:26 UTC   <invalid>       etcd-ca                 no      
apiserver-kubelet-client   Oct 17, 2023 01:26 UTC   <invalid>       ca                      no      
controller-manager.conf    Oct 17, 2023 01:26 UTC   <invalid>       ca                      no      
etcd-healthcheck-client    Oct 17, 2023 01:26 UTC   <invalid>       etcd-ca                 no      
etcd-peer                  Oct 17, 2023 01:26 UTC   <invalid>       etcd-ca                 no      
etcd-server                Oct 17, 2023 01:26 UTC   <invalid>       etcd-ca                 no      
front-proxy-client         Oct 17, 2023 01:26 UTC   <invalid>       front-proxy-ca          no      
scheduler.conf             Oct 17, 2023 01:26 UTC   <invalid>       ca                      no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Oct 14, 2032 01:26 UTC   8y              no      
etcd-ca                 Oct 14, 2032 01:26 UTC   8y              no      
front-proxy-ca          Oct 14, 2032 01:26 UTC   8y              no     

-> kubeadm certs check-expiration 명령어를 통해 인증서 만료 일자를 확인해보니 이미 기간이 만료되어 있었음

[root@Minhang02 pki]# kubeadm certs renew all
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[renew] Error reading configuration from the Cluster. Falling back to default configuration

certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed

Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates.

-> kubeadm certs renew all 명령어를 통해 인증서를 갱신할 수 있음

갱신하고 다시 인증서 정보를 확인해보면

[root@Minhang02 pki]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[check-expiration] Error reading configuration from the Cluster. Falling back to default configuration

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Oct 17, 2024 06:21 UTC   364d            ca                      no      
apiserver                  Oct 17, 2024 06:21 UTC   364d            ca                      no      
apiserver-etcd-client      Oct 17, 2024 06:21 UTC   364d            etcd-ca                 no      
apiserver-kubelet-client   Oct 17, 2024 06:21 UTC   364d            ca                      no      
controller-manager.conf    Oct 17, 2024 06:21 UTC   364d            ca                      no      
etcd-healthcheck-client    Oct 17, 2024 06:21 UTC   364d            etcd-ca                 no      
etcd-peer                  Oct 17, 2024 06:21 UTC   364d            etcd-ca                 no      
etcd-server                Oct 17, 2024 06:21 UTC   364d            etcd-ca                 no      
front-proxy-client         Oct 17, 2024 06:21 UTC   364d            front-proxy-ca          no      
scheduler.conf             Oct 17, 2024 06:21 UTC   364d            ca                      no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Oct 14, 2032 01:26 UTC   8y              no      
etcd-ca                 Oct 14, 2032 01:26 UTC   8y              no      
front-proxy-ca          Oct 14, 2032 01:26 UTC   8y              no 

-> 인증서 갱신되어 이제 364일 남았다고 뜸

[root@Minhang02 pki]# kubectl get pod -o wide
NAME                        READY   STATUS    RESTARTS   AGE    IP            NODE        NOMINATED NODE   READINESS GATES
bookstack-77df4879c-6z2v2   1/1     Running   0          28d    10.244.1.42   minhang03   <none>           <none>
bookstack-77df4879c-hqwgp   1/1     Running   0          28d    10.244.2.45   minhang04   <none>           <none>
jupyter-66477949-jvvqk      1/1     Running   0          307d   10.244.2.31   minhang04   <none>           <none>

-> 이제 kubectl도 다시 정상적으로 사용됨


Revision #1
Created 18 October 2023 06:32:00 by 신민항
Updated 18 October 2023 06:32:45 by 신민항