刨根问底：使用秘钥来安全地管理 Kubernetes 秘钥

1 .Turtles all the way down: securely managing Kubernetes secrets with secrets Alexandr Tcherniakhovski, Google Cloud Maya Kaczorowski, Google Cloud Nov 14 2018 

2 .Turtles all the way down 

3 .Turtles all the way down 

4 . Maya Kaczorowski Security PM, Google Cloud @MayaKaczorowski Alex Tcherniakhovski Security Engineer, Google Cloud 

5 .Protecting secrets 

6 .What’s a secret? Credentials, configurations, API keys, and other small bits of information needed by applications at build or run time 

7 .Why protect secrets? ● Attractive target ○ Controls access or use of sensitive resources ● Common attack vector ○ Checked into Github ○ Accessible by users who shouldn’t have access, e.g., CEO ○ Stored in public storage buckets 

8 .Secret management requirements Identity Auditing Require strong Verify the use of identities and least individual secrets privilege Encryption Rotation Isolation Always encrypt Change a secret Separate where before writing to regularly in case of secrets are used vs disk compromise managed 

9 . Encryption at different layers (or turtles) etcd file system disks Recommendation: Use two-layers of encryption, e.g., full-disk & application-layer 

10 .… then tries to decrypt it https://xkcd.com/538/, https://xkcd.com/license.html 

11 .Key rotation 

12 .“Keys are analogous to the combination of a safe. If a safe combination is known to an adversary, the strongest safe provides no security against penetration. Similarly, poor key management may easily compromise strong algorithms.” NIST SP 800-57, Recommendation for Key Management 

13 .Keys get old 

14 .Key rotation ● Key rotation is meant to limit the ○ ‘Blast radius’ if a single key is compromised ○ Time available for attempts to penetrate physical, procedural, and logical access ○ Time available for computationally intensive cryptanalytic attacks ● A cryptoperiod is the time during which a key is used to encrypt data 

15 .Key rotation: cryptoperiod There are lots of factors that influence the choice of cryptoperiod From NIST SP 800-57: ○ Strength of cryptographic ○ Number of key copies algorithms used ○ Personnel turnover ○ Implementation ○ Threat model ○ Operating environment ○ New and disruptive ○ Volume of data technologies, e.g., ○ Re-keying method quantum computers 

16 . Key rotation: compliance “ PCI DSS v3.2.1 3.5 Document and implement procedures to protect keys used to secure stored cardholder data against disclosure and misuse. 3.6 Fully document and implement all key-management processes and procedures for cryptographic keys used for encryption of cardholder data, including the following: 3.6.4 Cryptographic key changes for keys that have reached the end of their ” cryptoperiod (for example, after a defined period of time has passed and/or after a certain amount of cipher-text has been produced by a given key) https://www.pcisecuritystandards.org/documents/Prioritized-Approach-for-PCI-DSS-v3_2_1.pdf 

17 .Re-encrypting data is hard 

18 .Envelope encryption 

19 .Envelope encryption Data 

20 .Envelope encryption Data Data encryption key (DEK) 

21 .Envelope encryption Data Data encryption key Key encryption key (DEK) (KEK) 

22 .Envelope encryption: benefits Easier to manage 

23 .Envelope encryption: best practices Managing DEKs: Managing KEKs: ● Generate DEKs locally ● Store KEKs centrally ● Use a strong cryptographic algorithm ● Set the granularity of ● For easy access, store the DEK near the data that it the DEKs encrypted encrypts based on use case ● Ensure DEKs are encrypted at rest ● Rotate keys regularly, ● Don’t use the same DEK to encrypt data from two and also after a different apps/users suspected incident ● Generate a new DEK every time you write the data. This means you don't need to rotate the DEKs 

24 .Kubernetes secrets 

25 .Kubernetes secrets Master etcd ● Secrets are stored in etcd ○ base64 encoded SECRET ● A pod can access secrets via the filesystem, as an environment variable, or via Kubernetes kube-apiserver API call ● Operations with secrets are audit logged 

26 .Kubernetes secrets: 1.7 EncryptionConfig Master etcd ● Encrypt secrets with a locally managed key ● EncryptionConfig for secrets DEK SECRETDEK ● Multiple provider options ○ aesgcm kube-apiserver ○ aescbc ○ secretbox 

27 .Kubernetes secrets: 1.7 EncryptionConfig kind: EncryptionConfig apiVersion: v1 resources: - resources: - secrets providers: - identity: {} - aesgcm: keys: - name: key1 secret: c2VjcmV0IGlzIHNlY3VyZQ== - aescbc: {} - secretbox: {} 

28 .Kubernetes secrets: 1.10 KMS plugins Master KMS etcd ● Encrypt secrets with a locally managed key, which is then DEKKEK SECRETDEK KEK encrypted with a centrally managed key kube-apiserver ● EncryptionConfig uses aescbc with a KMS provider ● Sidecar pod for the KMS plugin kms-plugin 

29 .Terminology and Notation DEK Data encryption key KEK Key encryption key {SECRET}DEK Secret is encrypted with DEK {DEK}KEK DEK is encrypted with KEK {SECRET}DEK + {DEK}KEK Envelope Source for crypto notation: https://en.wikipedia.org/wiki/Security_protocol_notation