Transparent Container Solution for DPDK Applications

1 . x Transparent Container Solution for DPDK Applications TANYA BROKHMAN SHAHAR BELKAR 

2 .Agenda  Introduction to containers ‒ Overview and background ‒ Docker Containers Networking Model (CNM) ‒ Existing solutions for DPDK  Extending CNM capabilities ‒ Container networking over SRIOV ‒ DPDK networking for containers  Evaluation  Conclusions and future work 2 

3 .Introduction & Market Overview 

4 . Telco Evolution Classical Network: Virtualized Network: Appliance Approach Network Function Virtual Appliance Approach Virtualization (NFV) Firewall SDN Session Border Message Controller Message CDN Router Router Orchestrated, automatic & remote install. Session Border Firewall Controller Standard High Volume Servers Disadvantages of the Classical approach: Advantages of the Virtualized approach:  Network functions are based on specific HW&SW  Network functions are based SW-based  Physical install per appliance per site  Multiple roles over same hardware 4 

5 . Application Evolution Monolithic to distributed  Increase in Complexity of interactions  Distributed applications introduces Security,  Information grows exponentially Management Issues  Compounded by Web-Based communications  Balancing flexibility with control is becoming more difficult  SOA/Web Services bring exponential increase in interactions  Latency became important 5 

6 .Application Shift to Microservices  Application Design is moving to Microservices: ‒ Loosely couples distributed application ‒ Private datastore ‒ Owned by different teams We need a packaging unit to deploy a Microservice… 6 

7 .Application / MicroServices over a VM App App VM1 VM1 VM1 App App Hypervisor Libraries Infrastructure Kernel Disadvantages of VMs: Machine Virtualization  Additional memory and storage footprint  Performance overheads due to additional SW layers  Can’t be easily migrated Is there an alternative?  Slow spinup time  Low density 7 

8 .Containers are Better Match for Microservices Container is a standard unit of software that packages up code and all of its dependencies Deploy with Containers  Low footprint App App Kernel Kernel  Easy to deploy Libraries Libraries  Isolate software from its environment App App Kernel Kernel  Portable across hybrid infrastructures Libraries Libraries  Fast spin-up Kernel  High density 8 

9 .World Trend – Shifting to Containers 9 

10 .World Trend – Shifting to Containers 50% 10 

11 .World Trend – Cloud is Shifting to Containers 11 

12 .Next Wave of Service Deployment Running within the container == Running directly on host/VM How we achieve that? Containers Direct! 12 

13 .Docker & CNM 

14 .What is Docker? Docker is platform for developers and sysadmins to develop, deploy and run applications with containers Flexibility Decentralized Scalability User Friendly Cross Platform Support 14 

15 .Docker Basic Workflow Container Staging Server Container Container Network Docker Docker Image Docker file Hub Project Code Container Docker Production Container Server Container Container Docker Networking is a communication channel between the independent containers 15 

16 . Container Networking Model (CNM) Network Sandbox Network Sandbox Network Sandbox Container Networking Model Container Container Container formalizes the steps required to Endpoint Endpoint Endpoint Endpoint provide networking for containers, Network Network while providing an abstraction that Docker Engine can be used to support multiple Network driver network drivers. Network Infrastructure Architecture of container Networking model 16 

17 . Container Network Model Objects Network Network Controller Controller Provides an entry point Driver into the libnetwork Provides an actual Driver implementation of the CNM (libnetwork) network Network Network An implementation of CNM:Network Endpoint Provides the connectivity Endpoint for services exposed by a container network Sandbox Sandbox Represents container’s network configuration 17 

18 .CNM Lifecycle  Drivers register with Network Container Sandbox controller Network 2.create Network  Network created by the controller Controller and bound to driver 3.bind 1.register  Driver create endpoints for the network and configures it 4.create Driver Endpoint  The container will attach to the endpoint and a sandbox will be created for it. 18 

19 .CNM: Available Network Drivers Docker Engine Container Network Model (libnetwork) Bridge Host 3rd Party plugin Overlay MACVLAN Remote Drivers Native Drivers 19 

20 .Existing Solutions for DPDK Container Networking  Based on virtio/vhost with virtual switch ‒ vhost consumes a lot of CPU cycles ‒ Redundant data copies between host and container  All examples target connectivity between containers running on same host 20 

21 . ~ZERO PERFORMANCE Container Direct IMPACT WHEN RUNNING WITHIN A CONTAINER 

22 .Container Networking over SRIOV  Leverage HW mechanisms Container for security and isolation Engine Container 1 Container 2  Dedicated NIC Tx and Rx queues App App per container CNM (libnetwork) Libraries Libraries ‒ No contention with other containers! hisriov  Native access to PCIe based network DPDK RoCE plugin interface RDMA networking devices with no overheads PF VF1 VF2  No redundant SW layer copies! 22 

23 .hisriov Driver – SW Components  go-plugin-helpers/network ‒ Go handler to create external network extensions for Docker  netlink Docker framework ― An interface to the Linux kernel for network manipulations go-plugins-helpers/ Legend:  sriovnet New component added and network described by this document ― Go library for configuring SRIOV networking devices Existing package/library docker-hisriov-plugin  hilogger ― Logging framework hilogger netlink sriovnet 23 

24 .Usage Example – Create a New Network $ docker network create –d hisriov –subnet=40.0.0.0/24 –o netdevice=eth2 –o nvf=5 mynet Docker CreateNetwork() hisriov Engine libnetwork driver Locate hisriov 1. Enable SRIOV driver 2. Enable and configure 5 VFs 3. Save new network configuration 24 

25 .DPDK Networking for Containers  Enables SRIOV based DPDK networking for containers  Allocate a VF(s) for the container  Handle DPDK setup/teardown transparently to the user  Works on top of a hisriov managed network docker_run_dpdk is a Cobra based utility to enable running DPDK applications inside a container (*) Cobra library provides us with a simple interface for creating new applications 25 

26 .docker_run_dpdk Utility – SW Components Legend:  Cobra library New component added ‒ Create a new CLI interface Existing package/library  dpdkmap Cobra library ‒ Handles bind/unbind to/from DPDK ‒ Utilizes dpdk-devbind.py script from the DPDK build dpdkmap docker-run-dpdk  hilogger hilogger  sriovnet sriovnet docker framework 26 

27 . Usage Example – Start a DPDK Bound Container $ docker_run_dpdk run --net=mynet --name=dpdkC --numVf=2 –it centos-hiofed:latest bash docker run… Docker docker_run_dpdk Engine 1. Bind allocated VF to DPDK 2. Generate a new docker run command hisriov exposing UIO char device of the VF driver 3. Save container details in local DB Allocate VF for the new container 27 

28 .Usage Example – Stop a DPDK Bound Container $ docker_run_dpdk stop --name=dpdkC docker container rm –f dpdkC Docker docker_run_dpdk Engine 1. Load container details from local DB 2. Unbind container interfaces from DPDK 3. Stop the container 28 

29 . LETS SEE SOME Evaluation & NUMBERS, AND Wrap up NICE GRAPHS…