Kustomize:针对 Kubernetes 的无模板配置自定义- Jeffrey Regan, Google

在 Kubernetes 中管理配置是一项常见的任务,并且随着应用程序的扩展,这通常非常具有挑战性。已经开发出许多工具,但很少利用本地 Kubernetes API 的功能。Kustomize 是一种以声明方式自定义 Kubernetes 应用程序清单的新工具。在本次演讲中,与会者将学习 Kustomize 和工作流的基本概念,从而为不同的场景定制像 MySQL-Wordpress 这样的实际应用程序。与会者还将学习到管理 Kubernetes 清单的最佳实践
展开查看详情

1.kubernetes configuration customization a composable, template-free approach Nov 2018 kubecon 上海 jeff regan jregan@google monopole@github

2.Kubernetes 配置的用户定制 声明式的途径 Nov 2018 kubecon 上海 jeff regan jregan@google monopole@github

3.~60 k8s configuration tools k8s的配置工具 spreadsheet recent - automation broker maintained by Brian Grant

4. app app descriptor discovery package cluster manager dashboard customization lifecycle management

5. app app descriptor discovery Description, maintainer, version, ... package cluster manager dashboard customization lifecycle management

6. app app descriptor discovery Browse, search, download package cluster manager dashboard customization lifecycle management

7. app app descriptor discovery package cluster manager dashboard Bundling, plus dependencies customization lifecycle management

8. app app descriptor discovery package cluster manager dashboard What apps are running? Are they healthy? customization lifecycle management

9. app app descriptor discovery package cluster manager dashboard customization lifecycle management Rollouts, rollbacks, upgrades.

10. app app descriptor discovery package cluster manager dashboard customization lifecycle management Given config - adapt it to my needs.

11.kustomize Command line tool for k8s customization. k8s 用户化的命令行工具 Closes several old kubectl issues. 解决了一些kubectl的老问题 Composes with other tools. customization 可以和别的工具一起使用 sponsored by sig-CLI per this proposal

12.$ kustomize build helloWorld | \ kubectl apply -f -

13.$ tree helloWorld al helloWorld ← norm ├── configMap.yaml norm al ← ├── deployment.yaml o p p ed in r ←d ├── kustomization.yaml al └── service.yaml ← norm

14.service.yaml kustomization.yaml /dev/stdout kind: Service kind: Service metadata: resources: metadata: name: wordpress - service.yaml name: demo-wordpress kustomize spec: build spec: ports: ports: - port: 389 namePrefix: demo- - port: 389 selector: selector: app: wordpress app: wordpress

15.This is k8s-aware patching.

16. operands (things to include) kustomization.yaml = operations (ways to patch operands)

17.operands operations result service.yaml namePrefix: deployment.yaml demo- YAML Stream configMap.yaml

18.operands operations result YAML Stream A result can be used as an operand.

19. kustomize input is plain kubernetes yaml. You can kubectl apply that yaml without kustomize. To start customizing, just add a kustomization.yaml file.

20.Use Case #1 Variants 不同环境 dev (development, staging and production) image: hourly build staging common resources image: passed QA 1% of prod traffic r1.yaml, r2.yaml, ... production image: v2.3.8 replicas: 3000 cpu: 100

21.Use Case #1 Variants 不同环境 kustomization.yaml dev (development, staging and production) namePrefix: dev- bases: - ../../base kustomization.yaml kustomization.yaml staging resources: - r1.yaml namePrefix: staging- r2.yaml newTag: qa ... bases: - ../../base kustomization.yaml production namePrefix: prod- newTag: v2.3.8 bases: - ../../base

22.Use Case #1 Variants 不同环境 $ tree wordpress (development, staging and production) wordpress ├── base File layout: │ ├── kustomization.yaml │ ├── deployment.yaml │ ├── env.txt kustomization.yaml service.yaml deployment.yaml │ └── service.yaml └── overlays kind: Service ├── production kind: Deployment commonLabels: │ └── kustomization.yaml metadata: metadata: app: wordpress │ ├── replica_count.yaml name: wordpress name: wordpress resources: spec: │ └── cpu_count.yaml spec: - deployment.yaml ports: └── staging replicas: 1 - service.yaml - port: 389 template: ... │ ├── kustomization.yaml configMapGenerator: │ └── ... - name: wordpress-map └── dev files: ├── kustomization.yaml - env.txt └── ...

23.Use Case #1 Variants 不同环境 $ tree wordpress (development, staging and production) wordpress ├── base │ ├── kustomization.yaml │ ├── deployment.yaml │ ├── env.txt kustomization.yaml replica_count.yaml cpu_count.yaml │ └── service.yaml └── overlays namePrefix: prod- kind: Deployment kind: Deployment ├── production commonLabels: metadata: metadata: │ ├── kustomization.yaml variant: prod name: wordpress name: wordpress │ ├── replica_count.yaml commonAnnotations: spec: spec: │ └── cpu_count.yaml note: I’m Prod! replicas: 80 template: └── staging bases: spec: │ ├── kustomization.yaml - ../../base containers: │ └── ... patchesStrategicMerge: - name: my-container └── dev - replica_count.yaml resources: ├── kustomization.yaml - cpu_count.yaml limits: └── ... cpu: 7000m

24. $ tree wordpress Deploy production: wordpress ├── base │ ├── kustomization.yaml │ ├── deployment.yaml │ ├── env.txt $ kustomize build \ │ └── service.yaml wordpress/overlays/production |\ └── overlays ├── production kubectl apply -f - │ ├── kustomization.yaml │ ├── replica_count.yaml │ └── cpu_count.yaml └── staging │ ├── kustomization.yaml │ └── ... └── dev ├── kustomization.yaml └── ...

25. $ tree wordpress Deploy staging: wordpress ├── base │ ├── kustomization.yaml │ ├── deployment.yaml │ ├── env.txt $ kustomize build \ │ └── service.yaml wordpress/overlays/staging |\ └── overlays ├── production kubectl apply -f - │ ├── kustomization.yaml │ ├── replica_count.yaml │ └── cpu_count.yaml └── staging │ ├── kustomization.yaml │ └── ... └── dev ├── kustomization.yaml └── ...

26.$ kustomize build target 1 load universal k8s object descriptions 2 read kustomization.yaml from target 3 kustomize bases (recurse 2-5) 4 load and/or generate resources 5 apply target’s kustomization operations 6 fix name references 7 emit yaml

27.Use Case #2 Feeding customized names to containers 把用户化的名字放入容器 patch.yaml kind: Deployment /dev/stdout kustomization.yaml metadata: name: wordpress apiVersion: v1 vars: spec: kind: Deployment - name: MYSQL_SERVICE template: ... objref: spec: spec: kind: Service initContainers: initContainers: name: mysql - name: init-command - command: apiVersion: v1 image: debian fieldref: - curl demo-mysql command: fieldpath: metadata.name containers: - "curl $(MYSQL_SERVICE)" patchesStrategicMerge: - env: - patch.yaml containers: - name: WORDPRESS_DB_HOST - name: wordpress env: value: demo-mysql - name: WORDPRESS_DB_HOST value: $(MYSQL_SERVICE)

28.Use Case #3 ConfigMaps generated from multiple sources 归并属性 base production overlay /dev/stdout kustomization.yaml kind: ConfigMap kustomization.yaml bases: metadata: configMapGenerator: - ../../base name: prod-myCMap-b5m75cxc - name: myCMap namePrefix: prod- data: files: configMapGenerator: color=blue - common.properties - name: myCMap height=10m behavior: merge dbpassword=foo files: common.properties - secret.properties color=blue height=10m secret.properties … so on for staging and development variants. dbpassword=foo Properties can be owned by different teams.

29.It’s all patching. kustomize is just a means to manage k8s-targeted patching.