[Kubernetes] pod原地升级

kubelet原地升级实现

在实际的业务需求中，有很多需要能够实现原地升级。只改变镜像名称而不改变其他的配置。在现有的deployment中无法做到，而在kubelet层面其实是能做到原地升级，此时可以自定义operator来实现原地升级。

kubelet内部实现

在kubelet.Run函数中，启动各种manager，并执行kl.syncLoop(updates, kl)。在syncLoopIteration中将根据config的类型进行相应的处理。传入参数有

1
2
3
4
5


// 1.  configCh:       a channel to read config events from
// 2.  handler:        the SyncHandler to dispatch pods to
// 3.  syncCh:         a channel to read periodic sync events from
// 4.  housekeepingCh: a channel to read housekeeping events from
// 5.  plegCh:         a channel to read PLEG updates from

这里主要看handler.HandlePodUpdates。通过kl.dispatchWork最终由syncPod完成pod的更新。

1. 计算sandbox和container

通过computePodActions来检查pod spec是否发生变化。传入的参数是pod和上一时刻pod的状态。这里先调用podSandboxChanged来判断是否需要重建pod。当podStatus中存在SandboxStatus并且状态为Ready时，不会重建。

如果需要重建，这里也会先判断Restart状态是否为Never，然后将需要重启的容器加入changes
如果不需要重建。查看是否有initContainer失败，如果有则会将对应的容器加入changes。如果不是则会计算普通Container。如果容器不处于Running状态则会调用PostStopContainer执行postStop。如果容器不处于Running状态则会将容器加入ContainersToStart。如果容器处于运行状态则会计算容器的hash值来判断容器是否发生变化，如果根据新的container计算出来的值和status中的hash不一样，也会将container加入ContainerToStart。

2. 根据需要重建pod

如果上面返回的需要重建则会killpod

3. kill container

如果上面返回的结果中有需要重建的容器，则会调用killContainer把对应的container删除。

4. 根据需要创建Sandbox

如果createSandbox为true，则创建新的sandbox。

5. 启动 ephemeral container

ephemeral container这个功能在1.16之后才有，参考文档。

6. 启动 init container

这里根据podContainerChanges中的NextInitContainerToStart来启动对应的InitContainer

7. 启动业务 container

根据podContainerChanges中的ContainersToStart启动对应的container。

kube-apiserver

kube-apiserver在修改一个已经存在的pod，并不是所有的pod字段都能更新。只有如下几个字段能修改。

1

`spec.containers[*].image`, `spec.initContainers[*].image`, `spec.activeDeadlineSeconds` or `spec.tolerations`

测试修改spec.schedulerName字段会得到如下错误信息。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
# pods "test-pod" was not valid:
# * spec: Forbidden: pod updates may not change fields other than `spec.containers[*].image`, `spec.initContainers[*].image`, `spec.activeDeadlineSeconds` or `spec.tolerations` (only additions to existing tolerations)
# {"Volumes":[{"Name":"default-token-v7scp","HostPath":null,"EmptyDir":null,"GCEPersistentDisk":null,"AWSElasticBlockStore":null,"GitRepo":null,"Secret":{"SecretName":"default-token-v7scp","Items":null,"DefaultMode":420,"Optional":nu
ll},"NFS":null,"ISCSI":null,"Glusterfs":null,"PersistentVolumeClaim":null,"RBD":null,"Quobyte":null,"FlexVolume":null,"Cinder":null,"CephFS":null,"Flocker":null,"DownwardAPI":null,"FC":null,"AzureFile":null,"ConfigMap":null,"VsphereV
olume":null,"AzureDisk":null,"PhotonPersistentDisk":null,"Projected":null,"PortworxVolume":null,"ScaleIO":null,"StorageOS":null,"CSI":null}],"InitContainers":null,"Containers":[{"Name":"test","Image":"r.addops.soft.360.cn/library/cen
tos-74-el7-official-addops:latest","Command":null,"Args":null,"WorkingDir":"","Ports":null,"EnvFrom":null,"Env":null,"Resources":{"Limits":null,"Requests":null},"VolumeMounts":[{"Name":"default-token-v7scp","ReadOnly":true,"MountPath
":"/var/run/secrets/kubernetes.io/serviceaccount","SubPath":"","MountPropagation":null,"SubPathExpr":""}],"VolumeDevices":null,"LivenessProbe":null,"ReadinessProbe":null,"Lifecycle":null,"TerminationMessagePath":"/dev/termination-log
","TerminationMessagePolicy":"File","ImagePullPolicy":"Always","SecurityContext":null,"Stdin":false,"StdinOnce":false,"TTY":false}],"RestartPolicy":"Always","TerminationGracePeriodSeconds":30,"ActiveDeadlineSeconds":null,"DNSPolicy":
"ClusterFirst","NodeSelector":null,"ServiceAccountName":"default","AutomountServiceAccountToken":null,"NodeName":"stark09.add.bjyt.qihoo.net","SecurityContext":{"HostNetwork":false,"HostPID":false,"HostIPC":false,"ShareProcessNamespa
ce":null,"SELinuxOptions":null,"RunAsUser":null,"RunAsGroup":null,"RunAsNonRoot":null,"SupplementalGroups":null,"FSGroup":null,"Sysctls":null},"ImagePullSecrets":null,"Hostname":"","Subdomain":"","Affinity":null,"SchedulerName":"
#
# A: scheduler","Tolerations":[{"Key":"node.kubernetes.io/not-ready","Operator":"Exists","Value":"","Effect":"NoExecute","TolerationSeconds":300},{"Key":"node.kubernetes.io/unreachable","Operator":"Exists","Value":"","Effect":"NoExec
ute","TolerationSeconds":300}],"HostAliases":null,"PriorityClassName":"","Priority":null,"DNSConfig":null,"ReadinessGates":null,"RuntimeClassName":null,"EnableServiceLinks":true}
#
# B: default-scheduler","Tolerations":[{"Key":"node.kubernetes.io/not-ready","Operator":"Exists","Value":"","Effect":"NoExecute","TolerationSeconds":300},{"Key":"node.kubernetes.io/unreachable","Operator":"Exists","Value":"","Effect"
:"NoExecute","TolerationSeconds":300}],"HostAliases":null,"PriorityClassName":"","Priority":null,"DNSConfig":null,"ReadinessGates":null,"RuntimeClassName":null,"EnableServiceLinks":true}

总结

以上就是pod实现原地升级背后的相关知识，如果想通过类似deployment实现相关功能可以通过自定义CRD来实现。

文章目录