展开查看详情
1 .Safely upgrading
Kubernetes clusters
1
2 .廖宇 Yu Liao (yliaog@github)
Staff Software Engineer at Google Kubernetes team. Active contributor to Kubernetes.
Confidential + Proprietary Confidential + Proprietary
3 .徐超 Chao Xu (caesarxuchao@github)
Software Engineer at Google Kubernetes team. Active contributor to Kubernetes.
Confidential + Proprietary Confidential + Proprietary
4 .Death, taxes, and upgrades
4
5 .Death, taxes, and upgrades
5
6 .Death, taxes, and upgrades
6
8 .Scope of cluster upgrades
• Kubernetes binaries
8
10 .Before you begin...
10
11 .Before you begin...
etcdctl snapshot save backup.db
11
12 .Before you begin...
etcdctl snapshot save backup.db
etcdctl snapshot status backup.db
12
13 .Before you begin...
13
14 .Before you begin...
14
15 .Before you begin...
15
16 .Before you begin...
16
18 .Reverse engineering `kubeadm upgrade`
18
19 .Reverse engineering `kubeadm upgrade`
• kubectl drain
• kubectl uncordon
19
20 .Reverse engineering `kubeadm upgrade`
20
22 .Uninterpretable data in etcd
store at
encoded in batch/v1/job registry/jobs/<namespace>/<name>
CREATE batch/v1/job deserialize as batch/v1/job
convert to extensions/v1beta1/job
read from
encoded in batch/v1/job registry/jobs/<namespace>/<name>
GET batch/v1/job deserialize as extensions/v1beta1/job
convert to batch/v1/job
22
23 . Uninterpretable data in etcd
store at
encoded in batch/v1/job registry/jobs/<namespace>/<name>
Created today
CREATE batch/v1/job deserialize as batch/v1/job
convert to extensions/v1beta1/job
read from
500 internal server error registry/jobs/<namespace>/<name>
Read 1 year later...
GET batch/v1/job Failed to deserialize as extensions/v1beta1/job.
The apiserver doesn’t have the schema.
23
24 .Workarounds & solutions
1.KEP
2.https://github.com/kubernetes-sigs/kube-storage-version-migrator
24
25 .Clients are outdated
25
26 .Clients are outdated
26
27 .Policy breaks after upgrade
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
webhooks:
- name: enforce-image-policy.kubernetes.io
rules:
- apiGroups:
- "batch"
apiVersions:
- "v1"
resources:
- jobs
27
28 .Policy breaks after upgrade
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
webhooks:
- name: enforce-image-policy.kubernetes.io
rules:
- apiGroups:
- "batch"
apiVersions:
- "v1"
resources:
- jobs
28
29 .Policy breaks after upgrade
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
webhooks:
- name: enforce-image-policy.kubernetes.io
rules:
- apiGroups:
- "batch"
apiVersions:
- "*"
resources:
- jobs
29
热门城市
