Accelerate Network Protocol Stack Perforamance and Adoption

1 .Accelerate Network Protocol Stack Performance and Adoption in the Cloud Networking via DMM Waterman Cao Senior Researcher Cloud Networking Lab, Huawei 

2 . AGENDA 01 02 03 Overview What we face DMM Overview Use Cases 2 

3 . Why we need a new way to implement protocol stack? Diversified and Ultimate QoS req. Ossified Kernel Networking Stack • Diversity: e.g. >2 million mobile apps with various QoS • General-purpose design • Performance: e.g. <1ms, >1Gbps, 10M concurrent , @5G • General performance tradeoff • App Density: e.g. >30 of APP @ smart-phone, 100s • Hard to customization containers @ cloud host • Long protocol/algorithm release cycle… Kernel The history of transport protocol version released in Linux Kernel ? MPTCP Mismatched v0.2.6.33 UDP-Lite v0.2.5.20 TCP/IP v0.98 SCTP 1992 2002 2007 ？ time • More than 25 years, <5 transport protocols are released in the Linux kernel • It has takes 8 years after MPTCP was firstly proposed, but MPTCP is still not released in the Linux kernel 3 

4 . Challenges in Future Transport Protocol Design • Extremely high performance  Video – orders of magnitudes higher bandwidth  VR/AR – very low latency and jitter  IoT – orders of magnitudes more concurrent connections • Diversified network QoS/SLA  Applications with different QoS/SLA requirements exist simultaneously on the same platform  Any optimization is tradeoff between factors • Heterogeneous network environments  Cloud computing and mobile internet turn the network into an extremely complicated system  Network environment might change significantly due to network participants’mobility 4 

5 . Trends in Future Transport Protocol Design • Alternative transport protocols  Google’s QUIC  IBM’s FASP • User-space network stack  Improving performance  Protecting intellectual property 5 

6 . AGENDA 01 02 03 Overview What we face DMM Overview Use Cases 6 

7 . DMM Project : Re-design the Protocol Stack - DMM (Dual mode Multi-protocol Multi- instance) is DMM is an open source network Online Game Video Web stack framework under FD.io project, licensed as Apache , which enables: POSIX SOCKET - Dual mode :Support Kernel Space and User DMM Space Stack Orchestrator Kernel User - Multi-protocol: Simplify new protocols Space Space adoptions and Integrations with flexible framework TCP/IP High throughput Low delay stack stack - Multi-instance: concurrent stack instances and Enable “protocol routing” in Cloud Networking - DMM aims to provide the capability to have VPP multiple protocol and multiple stack Network I/O Engine instances running simultaneously in the same platform. 7 

8 .DMM in the Overall stack Application Layer/App Server Dataplane Management Agent Orchestration Honeycomb hc2vpp Network Controller Packet Processing DMM Data Plane Services NSH_SFC ONE TLDK Dataplane Packet Network VPP Management Processing IO CICN odp4vpp Agent Sandbox VPP Operation System Network IO Hardware deb_dpdk rpm_dpdk 8 

9 .Protocol Routing Workflow 1 Application server and client calls Application server socket interface. socket(), bind() Application client socket() 9 

10 .Protocol Routing Workflow 1 Application server and client calls Application server socket interface. socket(), bind() LD_PRELOAD 2 Socket APIs are hijacked to DMM nSocket APIs. DMM nSocket API DMM nSocket API DMM LD_PRELOAD Application client socket() 10 

11 .Protocol Routing Workflow Application server 1 Application server and client calls socket(), bind() socket interface. LD_PRELOAD L-RD 2 Socket APIs are hijacked to DMM nSocket APIs. DMM nSocket API 3 Server call listen() triggers L-RD to negotiate protocol policies. L-RD: manage local DMM Policies and Protocol Configure. DMM nSocket API DMM LD_PRELOAD Application client socket() 11 

12 .Protocol Routing Workflow Application server 1 Application server and client calls socket(), bind() socket interface. LD_PRELOAD L-RD 2 Socket APIs are hijacked to DMM nSocket APIs. DMM nSocket API 3 Server call listen() triggers L-RD to negotiate protocol policies. L-RD: manage local DMM Policies and Protocol Configure. 4 Server call accept() and client call connect() trigger L-RD to retrieve DMM nSocket API and resolve protocol stack DMM LD_PRELOAD mapping. Application client socket() 12 

13 .Protocol Routing Workflow Application server 1 Application server and client calls socket(), bind() socket interface. LD_PRELOAD L-RD 2 Socket APIs are hijacked to DMM nSocket APIs. DMM nSocket API vppcom_session_...() 3 Server call listen() triggers L-RD to negotiate protocol policies. VPP Host Stack L-RD: manage local DMM Policies and Protocol Configure. vppcom_session_...() 4 Server call accept() and client call connect() trigger L-RD to retrieve DMM nSocket API and resolve protocol stack DMM LD_PRELOAD mapping. Application client 5 According to the mapping, the socket() socket is instantiated to one protocol stack 13 

14 .Protocol Routing Workflow Application server 1 Application server and client calls socket(), bind() socket interface. LD_PRELOAD L-RD 2 Socket APIs are hijacked to DMM nSocket APIs. DMM nSocket API stackx-socket_...() 3 Server call listen() triggers L-RD to negotiate protocol policies. Another protocol L-RD: manage local DMM Policies and stack Protocol Configure. stackx-socket_...() Server call accept() and client call 4 connect() trigger L-RD to retrieve DMM nSocket API and resolve protocol stack DMM LD_PRELOAD mapping. Application client 5 According to the mapping, the socket() socket is instantiated to one protocol stack or Another. 14 

15 .Protocol Routing Workflow 1 Application server and client calls socket interface. 2 Socket APIs are hijacked to DMM Application server nSocket APIs. socket(), bind() LD_PRELOAD Server call listen() triggers L-RD to L-RD 3 DMM nSocket API negotiate protocol policies. L-RD: manage local DMM Policies and stackx-socket_...() Protocol Configure. vppcom_session_...() Another protocol 4 Server call accept() and client call VPP Host Stack connect() trigger L-RD to retrieve stack and resolve protocol stack vppcom_session_...() stackx-socket_...() mapping. DMM nSocket API 5 According to the mapping, the DMM LD_PRELOAD socket is instantiated to one protocol stack or Another. Application client socket() 6 Dual mode(kernel or user-space), Multiple protocols, Multiple instances can exist simultaneously. 15 

16 .Protocol Routing Workflow (with Centeral RD) 16 

17 . AGENDA 01 02 03 Overview What we face DMM Overview Use Cases 17 

18 . Use Case 1 : Protocol Routing File Sync Application Using Kernel stack • 3 Clients --> Server Network Setting • Internet (Client #1) • Intra DataCenter (Client #2) • Inter DataCenter (Client #3) Comparison scheme Client #3 Using FillP Using RDMA Client #2 • Default: the kernel TCP/IP stack • DMM: support kernel TCP/IP stack, RDMA, FillP (home-grown stack) By adaptively negotiating stacks according to the network environments, DMM achieves significant performance improvement comparing with the kernel stack by default 18 For more detail of this demo, please go to Huawei Demo Booth. 

19 .Use Case #2: Dual mode support for Nginx Server Nginx application Nginx • kernel stack vs user-space stack ? Server SBR DM DMM nRD Policy M nRD Kernel User • Internet connection ---> kernel stack TCP/IP Space • LAN connection ---> user-space stack Stack TCP/IP Stack LAN Interne t Using DMM, Nginx server could switch between kernel stack and user-space stack adaptively to use their advantages respectively under different scenario 19 

20 .Demo: Protocol Routing for Multi-network Client- Server Application • No one stack/protocol fits all scenario, but by adaptively negotiating stack according to the network environment, DMM achieves significant performance improvement. Almost no Framework overhead Reduced by Kernel Stack Reduced by 97% 69% Customized Userspace TCP/IP RDMA 20 

21 . Key Takeaway 21 

22 . DMM: Key takeaways  Flexibility to dynamically choose different protocols according to performance and/or functional requirements  End-to-end orchestration to maintain stack instances and the app/socket-to-stack mappings  Extendable transport protocol plug-in framework to host multiple stack instances simultaneously  Let stack developers concentrate on user space protocol innovation 22 

23 . DMM: Benefits to application developers/end-users • Friendly Acceleration:  Acceleration w/ backward compatible API, friendly to the legacy • Adaptive and customized Acceleration:  ‘Protocol Routing’ based on network env, application requirements and host information ‘Protocol Routing’ workflow 23 

24 . DMM: Benefits to stack/protocol researchers/developers • Friendly interfaces to integrate new protocol stacks  Flexible Pkt I/O NIC/L2/L3/L4 (EAL)  Simplified API (SBR) • Accelerate innovation of new protocol stacks  Modular and reusable function blocks w/ high perf.  Integration w/ both kernel and user space (VPP) 24 

25 . DMM project roadmap 18.04 18.07 18.10 • Fd.io open source • User space LWIP Stack announcement (DPDK) is ready for example. • F-stack into DMM • User Guide • Initial support for rSocket • Support “Fork”example • Manual nRD • Initial support for VPP host • DMM Performance • Dual mode support stack optimization • DMM Whitepaper ( Stack • Enhance migration in into DMM the“contactless” for APP framework • Package release on rpm and deb. 25 

26 . Welcome to join in us • FD.io DMM Web Site • https://wiki.fd.io/view/DMM • Code • https://git.fd.io/dmm • Contact us • Mail list: dmm-dev@lists.fd.io • IRC: #fdio-dmm 26 

27 . Thank You. Copyright©2018 Huawei Technologies Co., Ltd. All Rights Reserved. The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any time without notice.