DPDK Unikernel with Unikraft

Unikernel具有巨大的性能潜力(例如,整个范围在10-40 Gb / s的范围内,启动时间仅为几毫秒,图像大小仅为数百KB)。 但是,其中大多数都是手动构建的,并使用了相当晦涩或研究原型软件(例如Click模块化路由器)来处理数据包。

在本次演讲中,我们将介绍如何立即解决这两个问题。 首先,我们将介绍Unikraft,这是一个Linux Foundation项目,可大大减少开发新unikernel的时间。 第二,我们将向我们展示DPDK的端口,据我们所知,这是第一个Unikernel,它完全专门用于运行仅DPDK的工作负载。 最后,我们将显示运行此Unikernel的性能数字,并讨论未来的工作。

展开查看详情

1. Unikraft DPDK Sharan Santhanam What we saw Introduce DPDK Unikernels with Unikraft Unikraft Unikraft meets DPDK Sharan Santhanam Unikraft within DPDK Synergy NEC Laboratories Europe GmbH between Unikraft and DPDK 12th Nov 2019 This work has received funding from the European Unions Horizon 2020 research and innovation program under grant agreements no.825377 (UNICORE).This work reflects only the authors views and the European Commission is not responsible for any use that may be made of the information it contains. . . . . . . . . . . . . . . . . . . . . ⃝NEC c Corporation 2019 . . . . . . . . . . . . . . . . . . . .

2. Overview Unikraft DPDK Sharan Santhanam 1 What we saw What we saw Introduce Unikraft 2 Introduce Unikraft Unikraft meets DPDK 3 Unikraft meets DPDK Unikraft within DPDK Synergy between 4 Unikraft within DPDK Unikraft and DPDK 5 Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 2 / 17 . . . . . . . . . . . . . . . . . . . .

3. VNF with DPDK Ecosystem Unikraft DPDK Sharan Santhanam What we saw Introduce Unikraft Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 3 / 17 . . . . . . . . . . . . . . . . . . . .

4. VNF with DPDK Ecosystem Unikraft DPDK Sharan Santhanam What we saw Introduce Unikraft Unikraft meets Can we do better? DPDK Unikraft within DPDK ⇝ Guest OS specialization Synergy ⇝ Boot Time between Unikraft and DPDK ⇝ Isolation within the guest . . . . . . . . . . . . . . . . . . . . 3 / 17 . . . . . . . . . . . . . . . . . . . .

5. VNF with DPDK Ecosystem Unikraft DPDK Sharan Santhanam What we saw Introduce Unikraft Unikraft meets Can we do better? DPDK Unikraft within DPDK ⇝ Guest OS specialization Synergy ⇝ Boot Time between Unikraft and DPDK ⇝ Isolation within the guest Let’s discuss Unikernel... . . . . . . . . . . . . . . . . . . . . 3 / 17 . . . . . . . . . . . . . . . . . . . .

6. Unikernel - Do One Thing and Do It Well Unikraft DPDK Sharan Santhanam What we saw Virtual Machine Introduce Unikraft Unikernel App A App B Unikraft meets DPDK App A App B Lib A Lib B Lib A Lib B Unikraft Kernel Kernel within DPDK Synergy between Hypervisor Hypervisor Unikraft and DPDK Hardware Hardware . . . . . . . . . . . . . . . . . . . . 4 / 17 . . . . . . . . . . . . . . . . . . . .

7. Unikernel - Do One Thing and Do It Well Unikraft DPDK Sharan Santhanam What we saw ⇝ Unikernel are purpose built Introduce App A App B Unikraft - Thin kernel layer Lib A Lib B - Single image with application, specific kernel Unikraft meets DPDK primitives Unikraft Hypervisor within DPDK Hardware Synergy between Unikernel Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 5 / 17 . . . . . . . . . . . . . . . . . . . .

8. Unikernel - Do One Thing and Do It Well Unikraft DPDK Sharan Santhanam What we saw ⇝ Unikernel are purpose built Introduce App A App B Unikraft - Thin kernel layer Lib A Lib B - Single image with application, specific kernel Unikraft meets DPDK primitives Unikraft Hypervisor ⇝ No isolation within a Unikernel within DPDK Hardware - Flat address space Synergy between Unikernel Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 5 / 17 . . . . . . . . . . . . . . . . . . . .

9. Unikernel - Do One Thing and Do It Well Unikraft DPDK Sharan Santhanam What we saw ⇝ Unikernel are purpose built Introduce App A App B Unikraft - Thin kernel layer Lib A Lib B - Single image with application, specific kernel Unikraft meets DPDK primitives Unikraft Hypervisor ⇝ No isolation within a Unikernel within DPDK Hardware - Flat address space Synergy between Unikernel ⇝ Software Stack Specialization Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 5 / 17 . . . . . . . . . . . . . . . . . . . .

10. ”Really Unikernels!” Unikraft DPDK Sharan ⇝ Fast instantiation, destruction and migration times Santhanam - 10 milliseconds or less (LightVM [Manco SOSP 2017], Jitsu [Madhvapeddy, NSDI 2015]) What we saw Introduce Unikraft Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 6 / 17 . . . . . . . . . . . . . . . . . . . .

11. ”Really Unikernels!” Unikraft DPDK Sharan ⇝ Fast instantiation, destruction and migration times Santhanam - 10 milliseconds or less (LightVM [Manco SOSP 2017], Jitsu [Madhvapeddy, NSDI 2015]) What we saw ⇝ Low memory footprint Introduce Unikraft - Few MBs of RAM or less (ClickOS [Martins NSDI 2014]) Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 6 / 17 . . . . . . . . . . . . . . . . . . . .

12. ”Really Unikernels!” Unikraft DPDK Sharan ⇝ Fast instantiation, destruction and migration times Santhanam - 10 milliseconds or less (LightVM [Manco SOSP 2017], Jitsu [Madhvapeddy, NSDI 2015]) What we saw ⇝ Low memory footprint Introduce Unikraft - Few MBs of RAM or less (ClickOS [Martins NSDI 2014]) Unikraft meets ⇝ High Deployment Density DPDK - 8k guests on a single x86 server Unikraft (LightVM [Manco SOSP 2017]) within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 6 / 17 . . . . . . . . . . . . . . . . . . . .

13. ”Really Unikernels!” Unikraft DPDK Sharan ⇝ Fast instantiation, destruction and migration times Santhanam - 10 milliseconds or less (LightVM [Manco SOSP 2017], Jitsu [Madhvapeddy, NSDI 2015]) What we saw ⇝ Low memory footprint Introduce Unikraft - Few MBs of RAM or less (ClickOS [Martins NSDI 2014]) Unikraft meets ⇝ High Deployment Density DPDK - 8k guests on a single x86 server Unikraft (LightVM [Manco SOSP 2017]) within DPDK ⇝ High Performance Synergy between - 10-40Gbit/s Ethernet throughput with a single guest CPU Unikraft and (ClickOS [Martins NSDI 2014], Elastic CDNs [Kuenzer VEE 2017]) DPDK . . . . . . . . . . . . . . . . . . . . 6 / 17 . . . . . . . . . . . . . . . . . . . .

14. ”Really Unikernels!” Unikraft DPDK Sharan ⇝ Fast instantiation, destruction and migration times Santhanam - 10 milliseconds or less (LightVM [Manco SOSP 2017], Jitsu [Madhvapeddy, NSDI 2015]) What we saw ⇝ Low memory footprint Introduce Unikraft - Few MBs of RAM or less (ClickOS [Martins NSDI 2014]) Unikraft meets ⇝ High Deployment Density DPDK - 8k guests on a single x86 server Unikraft (LightVM [Manco SOSP 2017]) within DPDK ⇝ High Performance Synergy between - 10-40Gbit/s Ethernet throughput with a single guest CPU Unikraft and (ClickOS [Martins NSDI 2014], Elastic CDNs [Kuenzer VEE 2017]) DPDK ⇝ Reduced attack surface - Small trusted compute base - Strong isolation by hypervisor . . . . . . . . . . . . . . . . . . . . 6 / 17 . . . . . . . . . . . . . . . . . . . .

15. La-la Land Unikraft DPDK So, Unikernel Sharan High Performance Santhanam Isolation and reduced attack surface. What we saw Faster Instantiation Time Introduce Smaller image size Unikraft Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 7 / 17 . . . . . . . . . . . . . . . . . . . .

16. La-la Land Unikraft DPDK So, Unikernel Sharan High Performance Santhanam Isolation and reduced attack surface. What we saw Faster Instantiation Time Introduce Smaller image size Unikraft Unikraft meets DPDK The problem with Unikernel development: Unikraft Building take several months or longer within DPDK Potentially repeat the process for each target application Synergy between ”Specialization” is hard to build Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 7 / 17 . . . . . . . . . . . . . . . . . . . .

17. La-la Land Unikraft DPDK So, Unikernel Sharan High Performance Santhanam Isolation and reduced attack surface. What we saw Faster Instantiation Time Introduce Smaller image size Unikraft Unikraft meets DPDK The problem with Unikernel development: Unikraft Building take several months or longer within DPDK Potentially repeat the process for each target application Synergy between ”Specialization” is hard to build Unikraft and DPDK Ooops!! Thats not an effective way of doing things! . . . . . . . . . . . . . . . . . . . . 7 / 17 . . . . . . . . . . . . . . . . . . . .

18. What is Unikraft? Unikraft DPDK Sharan Santhanam Objectives What we saw ⇝ Support wide range of use cases Introduce ⇝ Simplify building and optimizing Unikraft ⇝ Common and shared code base Unikraft meets ⇝ Support different hypervisors DPDK ⇝ CPU architectures Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 8 / 17 . . . . . . . . . . . . . . . . . . . .

19. What is Unikraft? Unikraft DPDK Sharan Santhanam Objectives Unikraft What we saw ⇝ Support wide range of use cases ⇝ Everything is a library Introduce ⇝ Simplify building and optimizing ⇝ Decomposed OS functionality Unikraft ⇝ Common and shared code base ⇝ Unikrafts two components: Unikraft meets ⇝ Support different hypervisors DPDK - Library Pool - Build Tool ⇝ CPU architectures Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 8 / 17 . . . . . . . . . . . . . . . . . . . .

20. What is Unikraft? Unikraft DPDK Sharan Santhanam Objectives Unikraft What we saw ⇝ Support wide range of use cases ⇝ Everything is a library Introduce ⇝ Simplify building and optimizing ⇝ Decomposed OS functionality Unikraft ⇝ Common and shared code base ⇝ Unikrafts two components: Unikraft meets ⇝ Support different hypervisors DPDK - Library Pool - Build Tool ⇝ CPU architectures Unikraft within DPDK Synergy between Unikraft and Unikraft says Hi!! DPDK Source is BSD-licensed Kconfig based build system . . . . . . . . . . . . . . . . . . . . 8 / 17 . . . . . . . . . . . . . . . . . . . .

21. libukforest - Unikraft System Overview Unikraft DPDK Sharan Appln l2fwd pyapp Santhanam ⇝ Take an existing application What we saw Introduce - For example, a Python application or a l2fwd Unikraft Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 9 / 17 . . . . . . . . . . . . . . . . . . . .

22. libukforest - Unikraft System Overview Unikraft DPDK Sharan Appln l2fwd pyapp Santhanam tracing ⇝ Take an existing application drv memalloc runtime blkdev mempool rust etc.. What we saw consdev heap golang trace Introduce netdev buddy python debug - For example, a Python application Main nw fs sched libc or a l2fwd Unikraft dpdk 9pfs rt etc.. Unikraft meets libpool ssl ramfs prempt musl ⇝ Pick Unikraft functionality lwip vfscore coop newlib DPDK - Pool of drivers and standard Unikraft libraries within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 9 / 17 . . . . . . . . . . . . . . . . . . . .

23. libukforest - Unikraft System Overview Unikraft DPDK Sharan Appln l2fwd pyapp Santhanam tracing ⇝ Take an existing application drv memalloc runtime blkdev mempool rust etc.. What we saw consdev heap golang trace Introduce netdev buddy python debug - For example, a Python application Main nw fs sched libc or a l2fwd Unikraft dpdk 9pfs rt etc.. Unikraft meets libpool ssl ramfs prempt musl ⇝ Pick Unikraft functionality lwip vfscore coop newlib DPDK Plat container solo5 baremetal* - Pool of drivers and standard Unikraft libraries linuxu kvm xen firecracker within DPDK libpool ⇝ Pick a platform and architecture Synergy between - Pool of drivers and standard Unikraft and libraries DPDK . . . . . . . . . . . . . . . . . . . . 9 / 17 . . . . . . . . . . . . . . . . . . . .

24. libukforest - Unikraft System Overview Unikraft DPDK Sharan Appln l2fwd pyapp Santhanam tracing ⇝ Take an existing application drv memalloc runtime blkdev mempool rust etc.. What we saw consdev heap golang trace Introduce netdev buddy python debug - For example, a Python application Main nw fs sched libc or a l2fwd Unikraft dpdk 9pfs rt etc.. Unikraft meets libpool ssl ramfs prempt musl ⇝ Pick Unikraft functionality lwip vfscore coop newlib DPDK Plat container solo5 baremetal* - Pool of drivers and standard Unikraft libraries linuxu kvm xen firecracker within DPDK libpool ⇝ Pick a platform and architecture Arch Synergy x86 64 arm 32 arm 64 between libpool - Pool of drivers and standard Unikraft and libraries DPDK ⇝ Build Unikraft application . . . . . . . . . . . . . . . . . . . . 9 / 17 . . . . . . . . . . . . . . . . . . . .

25. libukforest - Unikraft System Overview Unikraft DPDK Sharan Appln l2fwd pyapp Santhanam tracing ⇝ Take an existing application drv memalloc runtime blkdev mempool rust etc.. What we saw consdev heap golang trace Introduce netdev buddy python debug - For example, a Python application Main nw fs sched libc or a l2fwd Unikraft dpdk 9pfs rt etc.. Unikraft meets libpool ssl ramfs prempt musl ⇝ Pick Unikraft functionality lwip vfscore coop newlib DPDK Plat container solo5 baremetal* - Pool of drivers and standard Unikraft libraries linuxu kvm xen firecracker within DPDK libpool ⇝ Pick a platform and architecture Arch Synergy x86 64 arm 32 arm 64 between libpool - Pool of drivers and standard Unikraft and libraries DPDK Build ⇝ Build Unikraft application unikernel pyapp kvm x86 64 pyapp xen x86 64 pyapp linuxu x86 64 binary l2fwd kvm x86 64 l2fwd xen x86 64 l2fwd linuxu x86 64 . . . . . . . . . . . . . . . . . . . . 9 / 17 . . . . . . . . . . . . . . . . . . . .

26. Unikraft - DPDK Target Arch? Unikraft DPDK Sharan l2fwd Santhanam libukdpdk What we saw libpthread intrinsics Introduce Unikraft libvfscore libnetdev Unikraft meets libramfs libvirtio uk dpdk kvm x86 64 DPDK libkvmplat libx86 64 Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 10 / 17 . . . . . . . . . . . . . . . . . . . .

27. Unikraft - DPDK Target Arch? Unikraft DPDK Sharan l2fwd Santhanam libukdpdk What we saw libpthread intrinsics Introduce Unikraft libvfscore libnetdev Unikraft meets libramfs libvirtio uk dpdk kvm x86 64 DPDK libkvmplat libx86 64 Unikraft within DPDK Synergy between Unikraft and DPDK Challenges!! ⇝ Build System Integration ⇝ Specialization of Guest OS ⇝ Minimize modification to DPDK library . . . . . . . . . . . . . . . . . . . . 10 / 17 . . . . . . . . . . . . . . . . . . . .

28. Build DPDK as an Unikraft Library Unikraft DPDK Unikraft Build system DPDK Build System Sharan ⇝ Config.uk (Kconfig based) ⇝ Automatic config generation Santhanam - Handles dependencies across library - CPU feature flags - Enable/Disable Function What we saw Introduce Unikraft Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 11 / 17 . . . . . . . . . . . . . . . . . . . .

29. Build DPDK as an Unikraft Library Unikraft DPDK Unikraft Build system DPDK Build System Sharan ⇝ Config.uk (Kconfig based) ⇝ Automatic config generation Santhanam - Handles dependencies across library - CPU feature flags - Enable/Disable Function ⇝ Makefile (gmake) What we saw ⇝ Makefile.uk (make based) - SRCS - [LIBNAME] SRCS - INCLUDE Introduce - CFLAG - [LIBNAME] CFLAG - DIRS Unikraft - CFLAG Unikraft meets DPDK Unikraft within DPDK Synergy between Unikraft and DPDK . . . . . . . . . . . . . . . . . . . . 11 / 17 . . . . . . . . . . . . . . . . . . . .