Runtime VM Protection By Intel Multiple Key Total Memory Encrypt

1 . Runtime VM Protection By Intel Multi-Key Total Memory Encryption ® (MKTME) Kai Huang @ Intel Corporation LINUXCON + CONTAINERCON + CLOUDOPEN Beijing, China, 2018 1 

2 .Legal Disclaimer No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document. Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade. This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps. The products and services described may contain defects or errors known as errata which may cause deviations from published specifications. Current characterized errata are available on request. Copies of documents which have an order number and are referenced in this document may be obtained by calling 1-800-548- 4725 or by visiting www.intel.com/design/literature.htm. Intel and the Intel logo are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others © Intel Corporation. 2 

3 .Agenda • MKTME Introduction • MKTME Use Cases • MKTME Enabling 3 

4 .Background: Trusted VM in Cloud VM protection by using encryption ❶ At-rest Cloud Orchestrator • VM encrypted ‘at-rest’, ‘in-transit’ and ‘runtime’. VM Image • There has been existing technologies for ‘at- Repo rest’ and ‘in-transit’ encryption • Qemu TLS support for live migration • Qemu encrypted image support Cloud Cloud Agent Agent • VM runtime encryption requires hardware memory encryption support ❷ Runtime ❸ In-transit • AMD® SME/SEV VM VM Live Migration Launch Launch • Intel® MKTME Launch VM on ‘Trustiness Verified’ Host • Trusted hardware/location, etc. Intel Arch ® Intel Arch ® • Trusted Cloud SW stack. Compute Node Compute Node Typical VM Lifecycle in Cloud 4 

5 .Trusted VM w/ OpenCIT -- OpenStack as Example Enterprise Data Center Cloud Service Provider OpenStack App App App Trust Glance Director OS OS OS OpenStack OpenStack Horizon Nova KB Proxy Policy Enforcement Key Broker Attestation Trust Agent hub Reporting Verifier Measurement OpenStack tbootxm Barbican Attestation Server TPM Intel Arch w/ TXT Intel OpenStack Compute Node Intel Open CIT helps on Host trustiness verification ® 5 

6 .TME & MKTME Introduction • New AES-XTS engine in data path to external memory bus. • Data encrypted/decrypted on-the-fly when entering/leaving memory. • AES-XTS uses physical address as “tweak” • Same plaintext, different physical address -> different ciphertext. • TME (Total Memory Encryption) • Full memory encryption by TME key (CPU generated). • Enabled/Disabled by BIOS. • Transparent to OS & user apps. • MKTME (Multi-key Total Memory Encryption) • Memory encryption by using multiple keys. • Use upper bits of physical address as keyID (see next) 6 

7 .MKTME KeyIDs • Repurpose upper bits of physical address as KeyID as shown below. • Reduces useable physical address bits. • Creates “aliases” of physical memory locations: different keyIDs can refer to the same page. • Cache-coherency is not guaranteed for the same page that accessed by different keyIDs. • CPU caches are unaware of keyID (still treat keyID as part of PA) • Architecturally upto 2^15-1 keyIDs (15 keyID bits). • Reported by MSR. Configured by BIOS. • KeyID 0 is reserved as TME’s key (not useable by MKTME). • New PCONFIG instruction to program keyID w/ associated key (see next) 7 

8 .MKTME KeyID Programming Overview New Ring-0 instruction PCONFIG to program the KEYID and associated key • Package scoped • Supports programming keyID to 4 modes: • Using CPU generated random ephemeral key (invisible to SW) • Using SW provided key (tenant’s key) • No encryption – plaintext domain • Clearing a key (using TME’s key effectively) • Allows SW to specify crypto algorithms • Only AES-XTS-128 for initial server intercept 8 8 

9 .VM Protection & Isolation With MKTME • Protection • Use keyID to encrypt VM memory at runtime • Isolation • Use different keyIDs for different VMs • Software Enabling • For CPU access, SW sets keyID at PTEs • IA page table (host) • EPT (KVM) • For Device access (DMA) • w/ IOMMU: Set keyID to IOMMU page table • Physical DMA: Apply keyID to PA directly 9 

10 .Highlights of MKTME Guests continue to run “without modifications” in MKTME domains: • Encrypted with 1) CPU-generated ephemeral key, or 2) the one provided by API (“tenant- controlled keys”) • Virtio, including optimization (direct access to guest memory by kernel) continues to work • Direct I/O (including accelerators, FPGA) assignment (including SR-IOV VFs) is available • Live migration can be supported (among platforms that support MKTME) • vNVDIMM can be supported w/ limitation (because of physical address “tweak”) • Host DIMM configuration cannot be changed cross reboots. • Qemu DIMM & vNVDIMM configuration cannot be changed cross VM reboots. 10 

11 .Agenda • MKTME Introduction • MKTME Use Cases • MKTME Enabling 11 

12 .MKTME Enabled Use Cases 1. Launch Tenant VMs with in-use protection (CPU generated keys) • Let CSP handle the keys • VM image provided by CSP 2. Launch Tenant VMs with at-rest and in-use protection with full tenant-control • VM image encrypted @rest with tenant-specific keys • Keys in tenant control • VM memory isolation with tenant-specific keys • Trustiness verified host • Additional: integrity verification of VM image Use-case Extension: KeyID Sharing for all VMs launched by single tenant with the same tenant-key (or CPU generated key). 12 

13 . VM Launch w/ CPU Generated Keys MKTME ephemeral key Image TME key Repo Cloud 1 Cloud VM Launch w/ Manager VM Launch Agent • CPU generated key 2 Policy Enforcement Keyid 3 • CSP provided VM image Policy Keyid 3 Security Properties Agent VM • w/ VM runtime protection 3 VM Launch w/ KeyID Keyid 3 • w or w/o at-rest and in-transit protection HYPERVISOR • No Host Trustiness Verification (Qemu/KVM) Keyid 0 HARDWARE Keyid 0 Intel® TXT / TPM Intel® MKTME DRAM* CSP Controlled 13 

14 . VM Launch w/ Tenant Controlled Keys 0 Upload VM image encrypted w/ tenant key Wrapped tenant key MKTME ephemeral Image Repo key Tenant key TME key VM Launch w/ • Tenant provided key 2 Cloud Cloud • Tenant provided encrypted VM image Manager VM Launch Agent • Tenant controlled key server 3 Policy Enforcement KeyID 3 Trustiness • Trustiness verified host Verification 1 • VM image integrity verified Policy KeyID 3 • Use TPM to wrap/unwrap tenant-key Agent VM 4 VM Launch w/ KeyID KeyID 3 Security Properties HYPERVISOR • w/ VM runtime protection Key (Qemu/KVM) KeyID 1 • w/ VM at-rest protection Server Attestation • w/ or w/o in-transit protection Service HARDWARE Intel® TXT / TPM KeyID 1 • w/ Host trustiness verification Intel® MKTME • w/ VM image integrity verification Request Tenant-key DRAM* Return wrapped key Tenant Controlled CSP Controlled 14 

15 .KeyID Sharing Among VMs New VM Launch MktmePolicy { w/ MktmePolicy tenant_id: <UUID>, Cloud SW makes decision whether to share or not. key_type: “ephemeral” | “persistent”, key_server: https://..., allow_to_share: “yes” | “no” } KeyIDPolicy KeyID VMs Policy1: <tenant1, “ephemeral”> keyID1 VM1, VM2.. Cloud SW Policy2: <tenant2, “persistent”, xxxxxx> keyID2 VM3 Launch VM w/ keyID Example: KeyID sharing is based on KeyIDPolicy: <tenant_id, key_type, tenant_key> Cloud SW: Qemu • Maintains ‘KeyIDPolicy-to-KeyID’ table Apply keyID to • Makes keyID sharing decision according to the table VM memory Launch VM • Updates the table on VM launch and teardown Compute Node mKey API: MKTME key management API mKey API KVM 15 

16 .Agenda • MKTME Introduction • MKTME Use Cases • MKTME Enabling 16 

17 . MKTME Enabling QEMU High Level Cloud SW Launch VM w/ KeyID Guest memory with KeyID VM Guest Memory Direct I/O Virtio/Vhost Live Migration • Host kernel • mkey APIs mkey APIs • Key/KeyID Management VFIO/IOMMU KVM • Core-MM KeyID support with KeyID Core-MM code • VFIO/IOMMU KeyID support with KeyID MMU Notifier KeyIDs EPT • DMA KeyID support Vhost-kernel for guest • KVM DMA with Key/KeyID • KeyID setup in EPT KeyIDs KeyID IA-PT Management Setting KeyIDs for Host in EPT • Qemu KeyIDs VT-d KeyIDs • Receive KeyID from Cloud SW PCONFIG for DMA for DMA • Apply KeyID to guest memory New Code Encrypted Device MKTME Engine Memory with (NIC, SCSI, etc) KeyIDs KeyID 17 

18 .MKTME Enabling Current Status • Specification has been published [1] • Core kernel enabling status • Some preliminary patches have been upstreamed • Feature emulation (CPUID, MSR); PCONFIG • Proposal of some components have been sent to community for feedback • Key management API: Using kernel key retention service • Other components WIP internally • Core-MM keyID support; IOMMU keyID support; DMA keyID support; … • KVM/Qemu enabling status • PoC has been done to prove MKTME actually works. • Depending on core kernel parts ready for formal patches. [1] https://software.intel.com/sites/default/files/managed/a5/16/Multi-Key-Total-Memory-Encryption-Spec.pdf 18 

19 .THANKS