Retrofitting Memory Protection in the Zephyr OS

1 .Retrofitting Zephyr Memory Protection Wayne Ren, Sensor Software Engineer, Synopsys 

2 .Acknowledgements These slides are based on Andrew Boie (Intel)’s “Retrofitting Zephyr Memory Protection” Thank you, Andrew! 

3 . 01 Background Design and Implementation in CONTENTS 02 Zephyr 03 ARC Specific Implementation 04 Future Work 

4 .Background 

5 .What is Zephyr • Zephyr: a modular RTOS and a complete solution stack – RTOS for use on connected resource-constrained and embedded devices – Focused on safety, security, connections with Bluetooth support and a full native networking stack – Apache 2.0 license, hosted at Linux Foundation – Support diverse use cases and architectures: ARC, ARM, RISC-V, X86 … – Web site: https://www.zephyrproject.org/ • More Zephyr related events: – “Introduction to the Zephyr Project” - Ryan Qian, NXP & Kate Stewart, The Linux Foundation, Tuesday, June 26 • 11:20 - 12:00 – “License Information Management ” – Kate Stewart, The Linux Foundation, Tuesday, June 25 • 15:50 - 16:30 – IoT Meetup, Tuesday, June 26 • 18:00 - 21:00, 北京海淀区科学院南路2号, 融科资讯中心C座南楼1层 南极洲会议室 

6 .Why Required • Security concern of IoT devices – More and more “things” are connected, traditionally offline->online • Secure communication • Trusted execution (secure boot) • Data protection • …. • Zephyr uses systematic approaches for security – Static: high quality design, code review, tests, certification – Dynamic: secure communication, cryptography, memory protection • Memory protection – One important approach for more secure, reliable and safe system – 1st step to implement security 

7 .Memory Protection Hardware Memory Protection Unit(MPU) Memory Management Unit(MMU) • Popular in low-end device, ARM Cortex M4, ARC • Popular in Application processors, x86, ARM Cortex EM A series, ARC HS series • Fixed number of configurable regions, each with • Address space divided into equal sized pages their own access policy (typically 4K). • No virtualization, physical memory addresses • Configuration for caching and access, policy for each page set in page tables • Typically have constraints on region specification, • MPU-like behavior with identity page table, but e.g. region sizes must be power of two, aligned to Optional support for virtual memory their size 

8 .Memory Protection in Zephyr • Zephyr had no means of preventing unwanted memory access before • Joint effort with most contributions from Linaro (ARM), Synopsys (ARC), and Intel (x86), initial efforts targeting MPU-based systems • Milestones – 1.9 release (7/2017): MMU/MPU enabled, stack overflow protection on ARM/x86 – 1.10 release (11/2017): user mode support on x86 MMU – 1.11 release (3/2018): user mode support on ARC/ARM MPU – 1.12 release (6/2018): more tests, refinement • Future work – Additional CPU architecture support – Flesh out APIs and iterative refinement – Support of TEE (Trusted Execution Environment), e.g., secure and non-secure world (1.13 or later) 

9 .Use Cases • Protect against unintentional programming errors – Stack overflows – Writing to bad memory – Data corruption • Sandbox complex data parsers and interpreters – Network stacks/protocols – File systems – Reduce likelihood of third-party data compromising the system • Support the notion of multiple logical isolated applications 

10 .Comparison with Other RTOSes • FreeRTOS-MPU • NuttX Protected Build • not default configuration of • Supports ARM MPU and MMU (with FreeRTOS identity page table) • Unprivileged "User" threads with • Unprivileged threads similar to configurable memory access, system FreeRTOS-MPU calls for privileged operations • Separately loaded applications • Not well maintained, often doesn't • Many features proposed but still compile WIP • ThreadX Modules • Zephyr • MPU or MMU Virtualized address • Thread-level protection spaces for separately loaded • Support for lost of different CPUS modules with thread-level memory • Same kernel & driver APIs for kernel protection features and user mode threads • Support for lots of different CPUs • Free, Apache 2.0 License • Not free. Royalty-free license with • More features in the future significant upfront cost, modules feature costs extra 

11 .Design and Implementation in Zephyr 

12 .Threat Model • User thread – Untrusted – Isolated from the kernel and each other User Kernel threads threads • Kernel thread and kernel MPU hardware separation – Trusted, privilege to access all CPU mode separation Unprivileged/User thread 1 thread 2 • A flawed or malicious user thread thread 3 cannot: – Leak or modify private data of another thread 4 thread 5 thread 6 thread unless specifically granted permission Zephyr Kernel – Interfere with or control another thread except through designed thread Privileged/Kernel communication APIs (pipes, semaphores, etc.) 

13 .User Mode • Control access to kernel objects and device drivers – Per-object and per-thread basis • Maintain compatibility with existing Zephyr APIs • Implement system calls for privilege elevation • Arch-specific code to enter user mode • Validate system call parameters including kernel object pointers • Do not require changes to individual drivers • Manage user mode access to memory 

14 .High-Level Policy • User threads are by default granted only – Read/write access to their own stack memory and application memory – Read-only/execute access to program text and ROM – Memory domain APIs to configure access to additional regions with child thread inheritance • User threads cannot use device drivers or kernel objects without being granted permission – Permission granted by other threads with sufficient permission or inherited • System call API parameters are rigorously checked • User mode stack overflows are safely caught 

15 .Permission Model • Each kernel object has a bitfield indicating what user threads have access to it Kernel object • Kernel threads can grant object access to any user thread 01100110 0 • User threads may grant object access to Thread another user thread if the calling thread has permissions on both the object and the Thread target user thread Thread • Newly created user threads may optionally inherit object permissions of the parent thread. 

16 .Kernel Object • Three main types of kernel-private data structures – Kernel API data structures - k_thread, k_sem, k_mutex, k_pipe, etc. – All device driver instances – All thread stacks, instead of individual structs, these are arrays of a special typedef to character data • To preserve Zephyr API compatibility, all are referenced by memory address – Act as a handle for user threads, object memory not accessible – Need a system for validating object pointers passed to system calls 

17 .Kernel Object Permissions • Kernel threads can access all objects – Permissions still tracked, because • Thread drops to user mode • Creates child user threads with object permission inheritance enabled – May designate some objects as "public" and usable by all threads • User threads – If created with permission inheritance, gain access to all parent thread's permissions except parent thread object – k_object_access_grant() calls must have permission on both the target thread and the object being granted permission to 

18 .Kernel Object: New Type • Creating new kernel object types is easy! – Add the name of the associated data structure to the build • Struct name itself for new kernel APIs • API struct name for new device driver subsystem types – Small modifications to some lists in two C files • Could eventually be automated • Recognizing instances of kernel objects and providing a validation function for them is all handled automatically at build time 

19 .Kernel Object: Constraints • Must be declared as a top-level global – Needs to appear in the kernel's ELF symbol table – OK to declare with static scope – May be embedded as members of larger data structures • Memory for an object must be exclusive to that object – Can't be part of a union data type • Must be in the kernel data section • Objects that do not meet these constraints will not be accessible from user mode • Future work: support runtime allocation use-cases from slabs/kernel heap 

20 .Kernel Object: Definition • perms: permission bitfield indicating thread permissions for that object • type: object type information struct _k_object { enum, K_OBJ_THREAD, char *name; K_OBJ_UART_DRIVER, ... u8_t perms[CONFIG_MAX_THREAD_BYTES]; u8_t type; • flags: initialization state, u8_t flags; public/private, others as needed u32_t data; } • data: extra data in some cases – Stack object size extern struct _k_object * – Build-time assigned thread ID _k_object_find(void *obj); 

21 .How to Get Kernel Object Info? • Problem: need to find all the kernel objects – Map object memory addresses to instantiations of struct _k_object containing metadata – Validate kernel object pointers passed in from user thread • Solution: – gen_kobject_list.py • Use pyelftools to unpack ELF binary and fetches all the DWARF debug information, and does object identification – gperf • a GNU tool for creating perfect hash tables • Generate the hash table of kernel objects for efficiency 

22 .Kernel Object: Flow zephyr pre-built.elf gen_kobject_list.py kobject_hash.gperf kobject_hash_prep gperf tool (1st build) rocess.c zephyr.elf kobject_hash.c 2nd build preprocess tool 

23 .System Call • Typical OS mechanism for allowing user threads to perform operations they can’t do User mode User threads • On all arches, API ID and parameters are marshaled into registers and a software interrupt/exception is triggered System call wrapper – Up to six registers used; additional args passed in via SW irq/exception irq/exception return struct and stack call checker clean up • Common landing site for system calls on kernel side Kernel mode – Validate API ID, execute the handler function – Clean general purpose registers on exit to prevent Zephyr Kernel private data leakage • Use build-time logic to make adding new system calls as painless as possible 

24 .System Call: Components • Very easy for developers to define • Created by developer for each system call: – System call header prototype __syscall void k_sleep(s32_t duration) – Handler function for argument validation Z_SYSCALL_HANDLER(k_sleep, duration) • Verify caller permissions on provided memory buffers or data passed via pointer • Copy any parameter data passed in via pointer to local memory • Verify object pointers, permission, initialization state • Verify parameter values which are otherwise left to assertions or simply un-checked – Implementation function void _impl_k_sleep(s32_t duration) • Kernel object API code under kernel/ • Driver subsystem API functions defined at the subsystem level • Auto-generated for each system call: – System call ID enumerated type – Handler function prototypes – _k_syscall_table entry mapping ID to handler function – __weak handler function for system calls excluded from kernel config – System call invocation function 

25 .System Call: Flow k_oops() N User Y Marshal Valid Y Lookup handler in API Call Mode args, Trigger call dispatch table ? SW IRQ ID? N Return to Marshal Return Implementation Y Handler Caller Value, exit IRQ Function Checks N k_oops() 

26 .System Call: Build-Time Magic • Limited parsing of kernel header files, looking for function prototypes prefixed with "__syscall“ • Parsing limited to determining return value and argument types to generate additional functions – Some minor limitations in parsing with array/function pointer argument types which can be easily worked around • Generated headers contains implementation of API as an inline function - invokes system call trap or direct call to implementation as appropriate • Some generated C code for default handler and dispatch table entry 

27 .Memory Domain • User threads by default can't look at any RAM except their own stacks • Need a flexible way to designate additional memory areas that a thread has access to • Limited number of total MPU regions needs to be taken into consideration • Grant access to top-level data or BSS section globals defined and used by the thread, or application data that needs to be shared between threads • Memory Domain APIs exist to handle re-programming the MPU for the incoming thread's memory access policy on context switch 

28 .Memory Domain: Implementation • Memory domain APIs are kernel-access only, no system calls mem_domain 0 • Implemented as an object struct k_mem_domain – Contains some number of memory partitions (struct mem partition 0 k_mem_partition) • Up to the maximum number of regions supported by MPU hardware, no limit for MMU • Each partition is a starting address, size, and access policy mem partition n • Hardware dictates alignment and size constraints – APIs to add/remove partitions to an initialized memory domain object • Any thread may be added/removed to a particular memory domain to implement an access policy for mem_domain 1 that thread • MPU region registers or MMU page tables updated mem partition 0 upon context switch to activate policy for incoming thread • Special Case: Application Memory mem partition n – Shared to all threads, CONFIG_APPLICATION_MEMORY in Kconfig – All top level globals in non-kernel object files (libs, application code) placed in user read/writable section by linker and access policy configured in MMU/MPU at boot • Facilities for grouping data by the linker (WIP) 

29 .ARC Specific Implementation