The Last Mile On Democratizing Ai

Democratizing AI is important to enable everyone to be able to utilize one of the great industry innovation of the century. However only open sourcing the frameworks is not going to achieve the goal. While the user could have many open source frameworks at their disposal (Tensorflow, PyTorch, MXNet,...), they have to run it on proprietary public clouds where you have virtually zero control. In this talk, we will present how Kubernetes working with OpenStack Cyborg project could help truly democratizing AI by providing an open source infrastructure for deep learning. We will also show how the user could achieve better control and performance by correlating hardware accelerator resource with the deep learning graphs.

1.Last Mile On Democratizing AI Howard Huang, Huawei Jianfeng Ding, Intel

2.Outline ➔ Diffuse The Hype ➔ Introduce Cyborg Project ➔ Intel’s Recent Effort in AI ➔ Look Into The Future

3.Everyone is talking about democratizing AI

4.But it can’t be truly done without an open cloud infrastructure ● Tensorflow, CNTK, Pytorch, Caffe, MXNET, … Basically everything you can Application find now about major AI related open source projects ● Same goes to majority of the research papers Infrastructure ???

5.Interesting comparison on Blockchain and AI Most hyped AI technologies Blockchain Level of Good Sense Few Sense understanding on Infrastructure Few work Good work

6.Define a Cloud Infrastructure For AI PaaS Cloud Infrastructure IaaS Hardware Heterogeneous Physical Resource (CPU,GPU,FPGA,ASIC,...)

7.Asking the right question Can we have an AI cloud infrastructure software which (1) Provides nice abstraction and management of the heterogeneous resources (2) Is open source and driven by an open community (3) Fascilates the e2e AI development

8.Outline ➔ Diffuse The Hype ➔ Introduce Cyborg Project ➔ Intel’s Recent Effort in AI ➔ Look Into The Future

9.Cyborg is a general management framework for accelerators Proud OpenStack Official Project since 2017.09 ( )

10.Cyborg Project Overview ● Subteams: release, driver, doc ● Active Chinese Dev wechat group (48 members) from companies like Huawei, China Mobile, Intel, Lenovo, ZTE, Tencent, Nokia, Unionpay, 99Cloud, Xilinx, Inspur, iFlyTech, UC Berkeley, UIUC, CMU ● Lots of gifs ...

11.Cyborg Pike Release cyborg-api Legend cyborg-conductor cyborg-db Pike Finished Out-of-scope cyborg-agent vendor-a-driver vendor-b-driver vendor-a-acc vendor-b-acc

12.Cyborg Queens Release cyborg-api Legend cyborg-db cyborg-conductor Pike Finished (resource provider) Queens Finished Out-of-scope cyborg-agent report SPDK driver cyborg-generic-driver Intel FPGA driver NVMe SSD vendor-acc-test Intel FPGA

13. Cyborg Rocky Release Planning (OpenStack) pythonclient-cyborg cyborg-api os-acc Legend Progr cyborg-db cyborg-conductor Quota Pike Finished aming (resource provider) Queens Finished Rocky Planned cyborg-agent report Out-of-scope SPDK driver Intel FPGA driver NVIDIA GPU driver Xilinx FPGA driver NVMe SSD Intel FPGA NVIDIA GPU Xilinx FPGA

14.Cyborg Rocky Release Planning (Kubernetes) ● Align Cyborg data model with DPI before 1.13 release ● Cyborg DPI Plugin releady when DPI GA ● Consider the possibility of a CRD Acc controller ● Could be utilized by Kubeflow containerized

15.Cyborg is forming

16.Outline ➔ Diffuse The Hype ➔ Introduce Cyborg Project ➔ Intel’s Recent Effort in AI ➔ Look Into The Future

17.Intel AI Compute Continuum Cloud/Data Center Edge Devices Large scale data centers Small scale data centers, User-touch endpoint such as public cloud or small business IT devices with lower power comms service providers, infrastructure, to few requirements such as gov’t and academia, large on-premise server racks and laptops, tablets, smart enterprise IT workstations home devices, drones


19.Intel® Xeon® Scalable Processors

20.Intel® Deep Learning Inference Accelerators

21.Intel® Omni-Path Architecture

22.Intel® Next Generation High Performance Storage

23.Intel® Optimized AI Libraries

24.Intel® Optimized AI Libraries (continue)

25.Deep Learning Frameworks Many Popular DL Frameworks are now optimized for CPU

26.Intel® Open AI Cloud Reference Architecture

27.Outline ➔ Diffuse The Hype ➔ Introduce Cyborg Project ➔ Intel’s Recent Effort in AI ➔ Look Into The Future

28.Future #1 : AI Native Open Infrastructure Infrastructure As Code - Programmable Framework Infrastructure As Model - Learnable Framework

29.Future #1 : AI Native Open Infrastructure Software 1.0 : Write a complex fixed cyborg-api scheduler cyborg-conductor cyborg-db Where is the scheduler ? cyborg-agent