Building DPDK Unikernels with Unikraft

1 . Unikraft DPDK Sharan Santhanam What we saw Introduce Building DPDK Unikernels with Unikraft Unikraft Unikraft meets DPDK Sharan Santhanam Unikraft within DPDK Synergy NEC Laboratories Europe GmbH between Unikraft and sharan.santhanam@neclab.eu DPDK 24th June 2019 This work has received funding from the European Unions Horizon 2020 research and innovation program under grant agreements no.825377 (UNICORE).This work reflects only the authors views and the European Commission is not responsible for any use that may be made of the information it contains. . . . . . . . . . . . . . . . . . . . . ⃝NEC c Corporation 2019 . . . . . . . . . . . . . . . . . . . . 

2 . Overview Unikraft DPDK Sharan Santhanam 1 What we saw What we saw Introduce Unikraft 2 Introduce Unikraft Unikraft meets DPDK Unikraft within DPDK 3 Unikraft meets DPDK Synergy between Unikraft and 4 Unikraft within DPDK DPDK 5 Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 2 / 16 . . . . . . . . . . . . . . . . . . . . 

3 . VNF with DPDK Ecosystem Unikraft DPDK Sharan Santhanam What we saw Introduce Unikraft Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 3 / 16 . . . . . . . . . . . . . . . . . . . . 

4 . VNF with DPDK Ecosystem Unikraft DPDK Sharan Santhanam What we saw Introduce Unikraft Unikraft meets DPDK Can we do better? Unikraft within DPDK ⇝ Guest OS specialization Synergy between ⇝ Boot Time Unikraft and DPDK ⇝ Isolation within the guest . . . . . . . . . . . . . . . . . . . . 3 / 16 . . . . . . . . . . . . . . . . . . . . 

5 . VNF with DPDK Ecosystem Unikraft DPDK Sharan Santhanam What we saw Introduce Unikraft Unikraft meets DPDK Can we do better? Unikraft within DPDK ⇝ Guest OS specialization Synergy between ⇝ Boot Time Unikraft and DPDK ⇝ Isolation within the guest Let’s discuss Unikernel... . . . . . . . . . . . . . . . . . . . . 3 / 16 . . . . . . . . . . . . . . . . . . . . 

6 . Unikernel - Do One Thing and Do It Well Unikraft DPDK Sharan Santhanam What we saw Virtual Machine Introduce Unikraft Unikernel Unikraft meets DPDK App A App B Unikraft App A App B Lib A Lib B within DPDK Lib A Lib B Kernel Kernel Synergy between Unikraft and DPDK Hypervisor Hypervisor Hardware Hardware . . . . . . . . . . . . . . . . . . . . 4 / 16 . . . . . . . . . . . . . . . . . . . . 

7 . Unikernel - Do One Thing and Do It Well Unikraft DPDK Sharan Santhanam What we saw Introduce Unikraft ⇝ Unikraft are purpose built App A App B - Thin kernel layer Unikraft meets DPDK - Single monolithic binary Lib A Lib B Unikraft within DPDK Hypervisor Synergy between Hardware Unikraft and DPDK Unikernel . . . . . . . . . . . . . . . . . . . . 5 / 16 . . . . . . . . . . . . . . . . . . . . 

8 . Unikernel - Do One Thing and Do It Well Unikraft DPDK Sharan Santhanam What we saw Introduce Unikraft ⇝ Unikraft are purpose built App A App B - Thin kernel layer Unikraft meets DPDK - Single monolithic binary Lib A Lib B Unikraft ⇝ No isolation within the Unikernel within DPDK needed Hypervisor - Flat address space Synergy between Hardware Unikraft and DPDK Unikernel . . . . . . . . . . . . . . . . . . . . 5 / 16 . . . . . . . . . . . . . . . . . . . . 

9 . Unikernel - Do One Thing and Do It Well Unikraft DPDK Sharan Santhanam What we saw Introduce Unikraft ⇝ Unikraft are purpose built App A App B - Thin kernel layer Unikraft meets DPDK - Single monolithic binary Lib A Lib B Unikraft ⇝ No isolation within the Unikernel within DPDK needed Hypervisor - Flat address space Synergy between Hardware Unikraft and ⇝ Full Stack Specialization DPDK Unikernel . . . . . . . . . . . . . . . . . . . . 5 / 16 . . . . . . . . . . . . . . . . . . . . 

10 . ”Really Unikernels!” Unikraft DPDK Sharan Santhanam ⇝ Fast instantiation, destruction and migration times - 10 milliseconds or less What we saw (LightVM [Manco SOSP 2017], Jitsu [Madhvapeddy, NSDI 2015]) Introduce Unikraft Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 6 / 16 . . . . . . . . . . . . . . . . . . . . 

11 . ”Really Unikernels!” Unikraft DPDK Sharan Santhanam ⇝ Fast instantiation, destruction and migration times - 10 milliseconds or less What we saw (LightVM [Manco SOSP 2017], Jitsu [Madhvapeddy, NSDI 2015]) Introduce ⇝ Low memory footprint Unikraft - Few MBs of RAM or less (ClickOS [Martins NSDI 2014]) Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 6 / 16 . . . . . . . . . . . . . . . . . . . . 

12 . ”Really Unikernels!” Unikraft DPDK Sharan Santhanam ⇝ Fast instantiation, destruction and migration times - 10 milliseconds or less What we saw (LightVM [Manco SOSP 2017], Jitsu [Madhvapeddy, NSDI 2015]) Introduce ⇝ Low memory footprint Unikraft - Few MBs of RAM or less (ClickOS [Martins NSDI 2014]) Unikraft meets DPDK ⇝ High Deployment Density Unikraft - 8k guests on a single x86 server within DPDK (LightVM [Manco SOSP 2017]) Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 6 / 16 . . . . . . . . . . . . . . . . . . . . 

13 . ”Really Unikernels!” Unikraft DPDK Sharan Santhanam ⇝ Fast instantiation, destruction and migration times - 10 milliseconds or less What we saw (LightVM [Manco SOSP 2017], Jitsu [Madhvapeddy, NSDI 2015]) Introduce ⇝ Low memory footprint Unikraft - Few MBs of RAM or less (ClickOS [Martins NSDI 2014]) Unikraft meets DPDK ⇝ High Deployment Density Unikraft - 8k guests on a single x86 server within DPDK (LightVM [Manco SOSP 2017]) Synergy ⇝ High Performance between Unikraft and - 10-40Gbit/s Ethernet throughput with a single guest CPU DPDK (ClickOS [Martins NSDI 2014], Elastic CDNs [Kuenzer VEE 2017]) . . . . . . . . . . . . . . . . . . . . 6 / 16 . . . . . . . . . . . . . . . . . . . . 

14 . ”Really Unikernels!” Unikraft DPDK Sharan Santhanam ⇝ Fast instantiation, destruction and migration times - 10 milliseconds or less What we saw (LightVM [Manco SOSP 2017], Jitsu [Madhvapeddy, NSDI 2015]) Introduce ⇝ Low memory footprint Unikraft - Few MBs of RAM or less (ClickOS [Martins NSDI 2014]) Unikraft meets DPDK ⇝ High Deployment Density Unikraft - 8k guests on a single x86 server within DPDK (LightVM [Manco SOSP 2017]) Synergy ⇝ High Performance between Unikraft and - 10-40Gbit/s Ethernet throughput with a single guest CPU DPDK (ClickOS [Martins NSDI 2014], Elastic CDNs [Kuenzer VEE 2017]) ⇝ Reduced attack surface - Small trusted compute base - Strong isolation by hypervisor . . . . . . . . . . . . . . . . . . . . 6 / 16 . . . . . . . . . . . . . . . . . . . . 

15 . La-la Land Unikraft DPDK Sharan Santhanam So, Unikernel High Performance What we saw Isolation and reduced attack surface. Introduce Faster Instantiation Time Unikraft Smaller image size Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 7 / 16 . . . . . . . . . . . . . . . . . . . . 

16 . La-la Land Unikraft DPDK Sharan Santhanam So, Unikernel High Performance What we saw Isolation and reduced attack surface. Introduce Faster Instantiation Time Unikraft Smaller image size Unikraft meets DPDK Unikraft The problem with Unikernel development: within DPDK Building take several months or longer Synergy Potentially repeat the process for each target application between Unikraft and ”Specialization” is hard to build DPDK Ooops!! Thats not an effective way of doing things! . . . . . . . . . . . . . . . . . . . . 7 / 16 . . . . . . . . . . . . . . . . . . . . 

17 . What is Unikraft? Unikraft DPDK Sharan Santhanam Objectives What we saw ⇝ Support wide range of use cases Introduce ⇝ Simplify building and optimiz- Unikraft ing Unikraft meets ⇝ Common and shared code DPDK base Unikraft ⇝ Support different hypervisors within DPDK ⇝ CPU architectures Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 8 / 16 . . . . . . . . . . . . . . . . . . . . 

18 . What is Unikraft? Unikraft DPDK Sharan Santhanam Objectives ⇝ Support wide range of use Unikraft What we saw cases ⇝ Everything is a library Introduce ⇝ Simplify building and optimiz- Unikraft ing ⇝ Decomposed OS functionality Unikraft meets ⇝ Common and shared code ⇝ Unikrafts two components: DPDK base - Library Pool Unikraft ⇝ Support different hypervisors - Build Tool within DPDK ⇝ CPU architectures Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 8 / 16 . . . . . . . . . . . . . . . . . . . . 

19 . What is Unikraft? Unikraft DPDK Sharan Santhanam Objectives ⇝ Support wide range of use Unikraft What we saw cases ⇝ Everything is a library Introduce ⇝ Simplify building and optimiz- Unikraft ing ⇝ Decomposed OS functionality Unikraft meets ⇝ Common and shared code ⇝ Unikrafts two components: DPDK base - Library Pool Unikraft ⇝ Support different hypervisors - Build Tool within DPDK ⇝ CPU architectures Synergy between Unikraft and DPDK Unikraft says Hi!! Source is BSD-licensed Kconfig based build system . . . . . . . . . . . . . . . . . . . . 8 / 16 . . . . . . . . . . . . . . . . . . . . 

20 . libukforest - Unikraft System Overview Unikraft DPDK Sharan Santhanam Appln l2fwd pyapp ⇝ Take an existing application What we saw Introduce - For example, a Python application or a l2fwd Unikraft Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 9 / 16 . . . . . . . . . . . . . . . . . . . . 

21 . libukforest - Unikraft System Overview Unikraft DPDK Sharan Santhanam Appln l2fwd pyapp ⇝ Take an existing application What we saw drv memalloc runtime tracing Introduce blkdev mempool rust etc.. - For example, a Python consdev heap golang trace application or a l2fwd Unikraft netdev buddy python debug Unikraft meets Main nw fs sched libc ⇝ Pick Unikraft DPDK libpool dpdk 9pfs rt etc.. functionality ssl ramfs prempt musl lwip vfscore coop newlib Unikraft - Pool of drivers and within DPDK standard libraries Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 9 / 16 . . . . . . . . . . . . . . . . . . . . 

22 . libukforest - Unikraft System Overview Unikraft DPDK Sharan Santhanam Appln l2fwd pyapp ⇝ Take an existing application What we saw drv memalloc runtime tracing Introduce blkdev mempool rust etc.. - For example, a Python consdev heap golang trace application or a l2fwd Unikraft netdev buddy python debug Unikraft meets Main nw fs sched libc ⇝ Pick Unikraft DPDK libpool dpdk 9pfs rt etc.. functionality ssl ramfs prempt musl lwip vfscore coop newlib Unikraft - Pool of drivers and within DPDK standard libraries Plat container solo5 baremetal* Synergy ⇝ Pick a platform and libpool linuxu kvm xen firecracker between architecture Unikraft and Arch DPDK - Pool of drivers and libpool x86 64 arm 32 arm 64 standard libraries . . . . . . . . . . . . . . . . . . . . 9 / 16 . . . . . . . . . . . . . . . . . . . . 

23 . libukforest - Unikraft System Overview Unikraft DPDK Sharan Santhanam Appln l2fwd pyapp ⇝ Take an existing application What we saw drv memalloc runtime tracing Introduce blkdev mempool rust etc.. - For example, a Python consdev heap golang trace application or a l2fwd Unikraft netdev buddy python debug Unikraft meets Main nw fs sched libc ⇝ Pick Unikraft DPDK libpool dpdk 9pfs rt etc.. functionality ssl ramfs prempt musl lwip vfscore coop newlib Unikraft - Pool of drivers and within DPDK standard libraries Plat container solo5 baremetal* Synergy ⇝ Pick a platform and libpool linuxu kvm xen firecracker between architecture Unikraft and Arch DPDK - Pool of drivers and libpool x86 64 arm 32 arm 64 standard libraries Build ⇝ Build Unikraft unikernel pyapp kvm x86 64 pyapp xen x86 64 pyapp linuxu x86 64 application binary l2fwd kvm x86 64 l2fwd xen x86 64 l2fwd linuxu x86 64 . . . . . . . . . . . . . . . . . . . . 9 / 16 . . . . . . . . . . . . . . . . . . . . 

24 . Unikraft - DPDK Target Arch? Unikraft DPDK Sharan Santhanam l2fwd libukdpdk What we saw libpthread intrinsics Introduce Unikraft libvfscore libnetdev Unikraft meets libramfs libvirtio DPDK Unikraft libkvmplat libx86 64 uk dpdk kvm x86 64 within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 10 / 16 . . . . . . . . . . . . . . . . . . . . 

25 . Unikraft - DPDK Target Arch? Unikraft DPDK Sharan Santhanam l2fwd libukdpdk What we saw libpthread intrinsics Introduce Unikraft libvfscore libnetdev Unikraft meets libramfs libvirtio DPDK Unikraft libkvmplat libx86 64 uk dpdk kvm x86 64 within DPDK Synergy between Unikraft and DPDK Challenges!! ⇝ Build System Integration ⇝ Specialization of Guest OS ⇝ Minimize modification to DPDK library . . . . . . . . . . . . . . . . . . . . 10 / 16 . . . . . . . . . . . . . . . . . . . . 

26 . Build DPDK as an Unikraft Library Unikraft DPDK Sharan Santhanam Unikraft Build system DPDK Build System What we saw ⇝ Config.uk (Kconfig based) ⇝ Automatic config generation Introduce - Handles dependencies across library - CPU feature flags Unikraft - Enable/Disable Function Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 11 / 16 . . . . . . . . . . . . . . . . . . . . 

27 . Build DPDK as an Unikraft Library Unikraft DPDK Sharan Santhanam Unikraft Build system DPDK Build System What we saw ⇝ Config.uk (Kconfig based) ⇝ Automatic config generation Introduce - Handles dependencies across library - CPU feature flags - Enable/Disable Function Unikraft ⇝ Makefile (gmake) ⇝ Makefile.uk (make based) - SRCS Unikraft meets - INCLUDE - [LIBNAME] SRCS - CFLAG DPDK - [LIBNAME] CFLAG - DIRS - CFLAG Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 11 / 16 . . . . . . . . . . . . . . . . . . . . 

28 . Build DPDK as an Unikraft Library Unikraft DPDK Sharan Santhanam Unikraft Build system DPDK Build System What we saw ⇝ Config.uk (Kconfig based) ⇝ Automatic config generation Introduce - Handles dependencies across library - CPU feature flags - Enable/Disable Function Unikraft ⇝ Makefile (gmake) ⇝ Makefile.uk (make based) - SRCS Unikraft meets - INCLUDE - [LIBNAME] SRCS - CFLAG DPDK - [LIBNAME] CFLAG - DIRS - CFLAG Unikraft within DPDK ⇝ version map ⇝ exportsyms.uk Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 11 / 16 . . . . . . . . . . . . . . . . . . . . 

29 . Build DPDK as an Unikraft Library Unikraft DPDK Sharan Santhanam Unikraft Build system DPDK Build System What we saw ⇝ Config.uk (Kconfig based) ⇝ Automatic config generation Introduce - Handles dependencies across library - CPU feature flags - Enable/Disable Function Unikraft ⇝ Makefile (gmake) ⇝ Makefile.uk (make based) - SRCS Unikraft meets - INCLUDE - [LIBNAME] SRCS - CFLAG DPDK - [LIBNAME] CFLAG - DIRS - CFLAG Unikraft within DPDK ⇝ version map ⇝ exportsyms.uk Synergy between Unikraft and DPDK libukdpdkbuild ⇝ Process DPDK Makefile. . . . . . . . . . . . . . . . . . . . . 11 / 16 . . . . . . . . . . . . . . . . . . . .