UPF (User Plane Function) is the main data path element in 3gpp architecture for 5G.
Several carriers have announced their plans to place UPF in edge locations as part of their 5G deployment plans.
Carriers are looking for HW acceleration for UPF, as compute resources at edge locations are limited.
There’s a need to define a standard interface between the UPF application (UPF-C) and the SmartNICs (UPF-U).
We suggest using DPDK APIs as the interface between UPF-C and UPF-U.
The presentation will also list the missing APIs we need to add to DPDK for fully offloading UPF functionality.
1. x Using DPDK APIs as the I/F Between UPF-C and UPF-U BARAK PERLMAN & BRIAN KLAFF ETHERNITY NETWORKS
2. 5G System Architecture • UPF is a 5G architecture NSSF NEF NRF PCF UDM AF data plane element Nnssf Nnef Nnrf Npcf Nudm Naf • Replaces the user plane of SGW and PGW Nausf Namf Nsmf • Control and User Plane AUSF AMF SMF SCP Separation (CUPS) N2 N4 User Plane Function UE (R)AN N3 UPF N6 DN User Equipment (Radio) Access Network Data Network N9 2
3. Accelerating UPF in 5G Access Endpoints Core Edge • Many operators are now moving UPF to the edge AMF • Optimal UPF at aggregation N1 N2 N11 locations • Used for local breakout N4 PFCP SMF • Partial/complete data plane offloading over FPGA-based SmartNICs N3 N6 UPF UPF UPF SmartNIC • Programmable Acceleration (R)AN Data Network • Scalable •SMF – Session Management Function • Open APIs UE •UPF – User Plane Function •PFCP – Packet Forwarding Control Protocol 3
4.Packet Processing Flow in UPF PDR – Packet Detection Rule FAR – Forwarding Action Rule QER – QoS Enforcement Rule URR – Usage Reporting Rule BAR – Buffering Action Rule MAR – Multi-Access Rule 4
5. Separation of UPF to UPF-C and UPF-U SMF N4 PFCP VM/container #m UPF-C DPDK Hypervisor UPF x86 Server PCIe Access N3 GTP IP N6 Data UPF-U Network PM counters Network 25G/40G/100G 25G/40G/100G SmartNIC 5
6. Partial Offload SMF N4 PFCP VM/container #m UPF-C PM Counters Control DPDK Packets Hypervisor UPF x86 Server PCIe N3 GTP UPF-U IP N6 Data Access Network Network 25G/40G/100G 25G/40G/100G SmartNIC 6
7. Full Offload SMF N4 PFCP VM/container #m UPF-C DPDK Hypervisor UPF x86 Server PCIe N3 GTP UPF-U IP N6 Access Data Network PM counters Network 25G/40G/100G 25G/40G/100G SmartNIC 7
8. FPGA SmartNIC Accelerates UPF Features S-GW/UPF P-GW/UPF GW ACL Filters UPF-U features offloaded by SmartNIC H-QoS Lawful ➢ Packet routing forwarding HandOver I/F DPI Profile Mgmt Interception ➢ GTP termination (if needed) UPF-C ➢ Gating, redirection & traffic steering X86 ➢ QoS Server PCIe ➢ Packet buffering Access FPGA Network ➢ Packet duplication UPF-U ➢ ACL GTP IP SmartNIC ➢ Lawful interception ➢ PM counters collection for billing ➢ IPsec encryption & decryption (for N3IWF) SmartNIC Forwarding Engine 8
9. I/F Between UPF-C and UPF-U SMF N4 PFCP VM/container #m UPF-C DPDK Hypervisor ? UPF x86 Server PCIe N3 GTP UPF-U IP N6 Access Data Network PM Counters Network 25G/40G/100G 25G/40G/100G SmartNIC 9
10.Options for UPF-C to UPF-U I/F • Control Plane Messages • Send in-band control packets between UPF-C and UPF-U • Option 1: dedicated control packets for this purpose, new standard • Option 2: use an existing SDN I/F (for example, OpenFlow, P4 & P4 run-time) • Use DPDK HW offload APIs • Use existing DPDK methods for HW offload 10
11. Control Plane Messages In-band SMF Approach N4 PFCP VM/container #m UPF-C DPDK Hypervisor UPF x86 Server PCIe N3 GTP UPF-U IP N6 Access Data Network PM Counters Network 25G/40G/100G 25G/40G/100G SmartNIC 11
12. Dedicated Control Plane Messages • Benefit: good performance • Control packets consume a small portion of the large data plane packets • Dedicated Control Plane Messages • Need to define a spec for control plane message content for all UPF-U features • Need to implement a specific design in both UPF-C and UPF-U • Need to update the spec and implementation for new UPF-U features • Need to address error reporting and retransmission • Existing SDN I/F • Need to adapt existing I/F to cover all UPF features not easily covered by existing SDN protocols • For example: cover policies, billing reports, etc. 12
13. Using DPDK HW Offload APIs Suggested SMF Approach N4 PFCP VM/container #m UPF-C DPDK flow APIs rte_flow DPDK Hypervisor UPF x86 Server PCIe N3 GTP UPF-U IP N6 Access Data Network PM Counters Network 25G/40G/100G 25G/40G/100G SmartNIC 13
14. DPDK-Based APIs for UPF-C to UPF-U • Most UPF applications are already implemented in DPDK • For example, 5G UPF based on VPP: https://github.com/travelping/vpp • rte_flow is the natural choice for DPDK applications • UPF is flow based, maps nicely to DPDK rte_flow offload APIs (generic flow API) • Avoids vendor lock-in • Supported by a large variety of vendors • Becoming a de-facto standard • Futureproof: maintained and enhanced by the DPDK community • Provides methods for handling flow validation • Flexible enough to cover almost all UPF-U features 14
15. SUGGESTED DPDK rte_flow IMPROVEMENTS REQUIRED FOR UPF OFFLOAD
16.Improve DPDK rte_flow APIs Performance • UPF requires a large number of flows (e.g., 1M flows) • Need to improve the rte_flow configuration rate • Add burst write configurations • Batching of rte_flow entries and then committing the batch to the HW offload • Use shared memory and DMA for flow data structures • Provide pointers to complete rte_flow data structures • This is required for delivering PM counters for a large number of rte_flows • Create a single rte_flow template, then populate just a few variable fields • Avoids configuration of repeated fields in the same rte_flow template 16
17. GTP Header • GTP header match is already supported in rte_flow • Need to add GTP-U encap/decap • GTP-U header encapsulation and decapsulation rte_flow actions • Very similar to other tunnel headers that are already supported: VxLAN, NVGRE, MPLS and raw_encap/decap • Should include optional support for 5GS GTP-U extension header ➢ The GTP-U Extension Header for 5GS is called "PDU Session Container" 17
18.GTP-U Extension Header for 5GS 18
19. Summary • UPF is the 5G element implementing user plane data path • UPF can be placed at the edge • There is a need for UPF HW offload • UPF can be split into UPF-C and UPF-U • Need to define an I/F between UPF-C and UPF-U • DPDK rte_flow APIs are a good option for implementing this I/F • Need to implement some enhancements in rte_flow for optimal support of UPF 19
20. x Thank You BARAK PERLMAN, CTO BRIAN KLAFF, MARKETING DIRECTOR BARAK@ETHERNITYNET.COM BRIANK@ETHERNITYNET.COM