Stabilizing the DPDK ABI and What it Means for You

1 .Stabilizing the DPDK ABI and what it means for you Stephen Hemminger (Microsoft Azure) Ray Kinsella (Intel) DPDK Summit – Mountain View - 2019 

2 . Legal Notices & Disclaimers  This document contains information on products, services and/or processes in development. All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest forecast, schedule, specifications and roadmaps.  Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer. No computer system can be absolutely secure.  Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit http://www.intel.com/performance.  Cost reduction scenarios described are intended as examples of how a given Intel-based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction.  Statements in this document that refer to Intel’s plans and expectations for the quarter, the year, and the future, are forward-looking statements that involve a number of risks and uncertainties. A detailed discussion of the factors that could affect Intel’s results and plans is included in Intel’s SEC filings, including the annual report on Form 10-K.  The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.  No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.  Intel does not control or audit third-party benchmark data or the web sites referenced in this document. You should visit the referenced web site and confirm whether referenced data are accurate.  Intel, the Intel logo, Atom, Xeon, Xeon Phi, 3D Xpoint, Iris Pro and others are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. © Intel Corporation. 

3 .3 Technology Adoption Curve Constant ABI churn limits the reach of latest How do we encourage the silent majority, to releases – to Enthusiasts and Visionaries. adopt new releases of DPDK? Increasing conservativism resulting in slower release adoption ABI Breakages discourages new technology adoption by the silent majority. 

4 .Comparative ABI Churn Gstreamer Backward Compat. DPDK Backward Compat. 100.00% 100% 80.00% 80% 60.00% 60% 40.00% 40% 20% 20.00% 0% 0.00% 12/18/2014 12/18/2015 12/18/2016 12/18/2017 12/18/2018 7/1/2016 7/1/2017 7/1/2018 GStreamer Application Binary Interface DPDK Application Binary Interface • 100% backward compatible within Major Versions • 8.7% median ABI churn between quarterly releases. (1.x). • LTS release is API stable for 2 years, however limited • Stable since 1.4.5, typically < 1% change between backporting of new features or HW. Major Versions. https://abi-laboratory.pro/index.php?view=timeline&l=gstreamer https://abi-laboratory.pro/index.php?view=timeline&l=dpdk Thanks to ABI Laboratory for providing the ABI analysis tools. 

5 .Linux Kernel ABI Stability We care about user-space interfaces to an insane degree. We go to extreme lengths to maintain even badly designed or unintentional interfaces. Breaking user programs simply isn't acceptable. Linus Torvalds, 2005 Linux Tux, by Larry Ewing 1996. Public Domain 

6 .6 What is an ABI? • API: Application Programming Interface • ABI: Application Binary Interface • What a programmer expects to see. • What the application expects to see. • Programing specification comprised of: functions, • Runtime binary interface comprised of: datatypes, headers … Public symbols and symbols versions • Typically managed very carefully by software • Typically managed very carefully by DevOPs engineers. Engineers, OSVs, CSPs etc Understanding ABI is key part of deploying a stable application 

7 .What is an ABI? Application function calls Instruction set ABI Executable & Linker Format Programming Calling Conventions. Language … Functions Datatypes Return Types API Constants … DPDK 

8 .Proposals on ABI Stability  Major ABI versions are declared every year and are then supported for one year, typically aligned with the LTS release. ABI / API remains stable for one  The ABI version is managed at a project level in DPDK, with year the ABI version reflected in all library’s soname.  The ABI should be preserved and not changed lightly. ABI changes must follow the outlined deprecation process. New platform, features &  The addition of symbols is generally not problematic. The improvement continue to be modification of symbols is managed with ABI Versioning. released.  The removal of symbols is considered an ABI breakage, once approved these will form part of the next ABI version.  Libraries or APIs marked as Experimental are not considered part of an ABI version and may change without constraint. Experimental libraries or APIs may  Updates to the minimum hardware requirements, which change without restriction. drop support for hardware which was previously supported, should be treated as an ABI change. [dpdk-dev] [PATCH v7 0/4] doc: changes to abi policy introducing major abi versions 

9 . ABI Stability: addition and modification Old ABI depreciated rte_foo() rte_foo() rte_foo(uint8_t bar) ABI ABI v 20 { … rte_foo … } v 20 { … rte_foo … } Release v19.11 (LTS) v 21 { … rte_foo …} Release v20.08 rte_foo() rte_foo(uint8_t bar) rte_foo(uint8_t bar) ABI ABI v 21 { … rte_foo …} Release v20.02 v 20 { … rte_foo … } Release v20.11 (LTS) v 21 { … rte_foo …} • The addition of symbols is generally not • New platform, features & improvements problematic. The modification of symbols should continue to be released. be managed with ABI versioning. 

10 .Where the ABI policy applies In Scope Out of Scope • lib • lib (experimental) • drivers (with public API's) • api (annotated experimental) • examples • usertools • devtools • doc 

11 . ABI Stability: major ABI versions Minor ABI Version LTS Release Release v 20 ABI is declared aligned with v 19.11 LTS v 21 symbols are added and v 20 symbols are modified, support for v 20 ABI continues. v 21 ABI is declared aligned with v 20.11 LTS, remaining v 20 symbols are remov ed. v 23 v 22 √ √ √ ABI Version Compatibility √ √ √ √ √ √ √ v 21 v 20 √ √ √ √ DPDK Releases v 19.11 v 20.02 v 20.05 v 20.08 V20.11 v 21.02 v 21.05 v 21.08 ABI Versions v 20 v 21 

12 . Next Steps • Agreement at the DPDK Technical Board following Userspace Bordeaux. • Supporting work - preparation for ABI stability • Build Changes: Global ABI Version & Build Symbol Check. • [dpdk-dev] [PATCH v5] Implement the new ABI policy and add helper scripts - Anatoly Burakov & Marcin Baran. • [dpdk-dev] [RFC 0/6] Add ABI compatibility checks to the meson build – Kevin Laatz • Supporting infrastructure to test ABI stability in the UNH lab (WiP). • Ensure testing of ABI Compatibility as part of the RC Test Cycle (WiP). What does this mean for your adoption of DPDK releases? 

13 . Stephen Hemminger Questions? stephen@networkplumber.org