Microsoft Azure in Autonomous Driving

1 .Microsoft Azure in Autonomous Driving 

2 .Microsoft’s approach to automotive We complement OEMs and suppliers – not compete We ensure your data is always under your control We guarantee that the brand and customer experience belongs to you 

3 .

4 .OEM System Engineering Process Engineering Development Product Validation Manufacturing & Service Feasibility Operations Regional Changes Retirement/ Study/concept and Architecture (s) and Upgrades replacement Exploration Maintenance Concepts of System Validation Plan System Lifecycle Processes Operations Validation System Verification Plan (System Acceptance) System System Verification & Requirements Deployment Subsystem Verification Plan (Subsystem Acceptance) High-Level Subsystem Design Verification Unit/Device Test Plan Unit/Device Detailed Design Testing Document/Approval Software/Hardware Development Field Installation Implementation Development Processes Time Line 

5 . Process, Sensor/ Algorithm Control Logic Software Train Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x) Tag Replay Performance Hardware Simulate Simulation in the loop Render/Convert Test-Drive DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE 

6 .• Cloud and Analytics partner for • Open source AD Platform • OpenADx ACM, an autonomous and smart • 200+ Member Consortium • Interoperable Eclipse Framework mobility test facility • Microsoft is the cloud provider for • Leveraged by all OEMs, tier ones and Project Apollo worldwide with the technology start ups exception of China • Engaged with ACM to influence standards Industry Academia Government 

7 .

8 .LG’s autonomous vehicle program had unique requirements, including portability, security, and fast turnaround time; Data Box Disk was the perfect solution. “We needed a way to transfer massive amounts of data for our autonomous vehicle projects, based all around the world. The solution needed to be portable, simple to use, cost-effective and, of course, very secure. The Azure Data Box Disk met all of those criteria.” – Hyoyuel (Andy) Kim, Senior Manager, LG Electronics. Products and services Organization size Industry Country Azure Data Box Disk 91,000 Automotive Worldwide Azure Blob Storage electronics 

9 . Process, Sensor/ Algorithm Control Logic Software Train Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x) Tag Replay Performance Hardware Simulate Simulation in the loop Render/Convert Test-Drive DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE Azure Express Route Azure Compute Databricks & HDInsight Azure IoT Edge Azure Storage, Data Lake Gen2 Azure Batch AI Azure Machine Learning Container and Kubernetes , Visual Studio Team Services Azure Data Box Cycle Cloud Cognitive Services Azure Key Vault, AAD 

10 . SOFTWARE TOOLS AND DEVOPS PLATFORM CI/CD flows Machine Learning Azure Source Container Container Control Registry Service DATA INGEST Data Factory Virtual machine Machine Learning Cycle Cloud HDInsight, Azure GPU Azure VM Cosmos DB In-car Databricks Logic App data Async storage upload Kubernetes Azure blob Service Simplygon storage (cool, ML PowerBI archive) CosmosDB Data Fast Advanced HDInsight reduction, upload Stream Analytics Analytics Batch Service Bus Databricks redaction IoT Edge ADLS Gen 2 and Azure blob storage Hot Tier 

11 . Process, Sensor/ Algorithm Control Logic Software Train Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x) Tag Replay Performance Hardware Simulate Simulation in the loop Render/Convert Test-Drive DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE 

12 . Offline Data Transfer Online Data Transfer Data Box Data Box Disk Data Box Heavy Data Box Gateway Data Box Edge • Capacity: 100 TB • Capacity: 8TB ea.; 40TB/order • Capacity: 1000TB • Virtual device provisioned in • Local Cache Capacity: ~12 TB • Weight: ~50 lbs • Secure, ruggedized USB • Weight 500+ lbs your hypervisor • Includes Data Box Gateway • Secure, ruggedized drives orderable in packs of • Secure, ruggedized • Supports storage gateway, and Azure IoT Edge. appliance 5 (up to 40TB). appliance SMB, NFS, Azure blob, files • Currently in Preview • Currently in Preview • Currently in preview • Currently in preview • Data Box enables bulk migration to Azure when • Perfect for projects that • Same service as Data Box, • Virtual network transfer • Data Box Edge manages network isn’t an option. require a smaller form factor, but targeted to petabyte- appliance (VM), runs on your uploads to Azure and can e.g., autonomous vehicles. sized datasets. choice of hardware. pre-process data prior to upload. Network Data Transfer Edge Compute Order Send Fill Return Upload Cloud to Edge Edge to Cloud Pre-processing ML Inferencing 

13 .Massive Data Ingest – ExpressRoute Ingest up to PB of data daily with Azure Networking Express Route Service Private, high speed network connections Connecting your data center and Azure Predictable performance and Secure 100 + Express Route partners 

14 .

15 .Data Storage – Azure Blob Storage Store up to Exabytes of data in massively scalable object storage for unstructured data • Foundational service for Microsoft Scalable • >40 million transactions per second • Multi-PB accounts • 50+Gbps account throughput Performant • Continued enhancements under Frequently Less frequently Rarely development accessed data accessed data accessed data • Client & Service Encryption Secure & • AAD Integration + ACLs Lowest transaction Lower capacity Lowest capacity Compliant • Broad & deep compliance portfolio cost cost cost • Multiple redundancy options Immediate (ms) Immediate (ms) Hours Durable • Strong consistency, data integrity • Policy: Versioning & WORM locks • Integrated storage tiers Cost • Lifecycle management Effective • Rich metrics 

16 . Identity & Data Network Threat Security access protection security protection management management Encr yption Azure Active Director y VNET, VPN, NSG Azure Security Center (Disks, Storage, SQL) Multi-Factor Application Gateway Microsof t Antimalware Azure Key Vault Azure Log Analytics Authentication (WAF), Azure Firewall for Azure Role Based Access DDoS Protection Control Standard Azure Active Director y (Identity Protection) ExpressRoute + Partner Solutions 

17 . Partner-based Solutions Sensor data fusion tools Data Data Annotation Data extraction Extraction Preparation Labeling and annotation Microsoft Services Data analysis Blob Cognitive Services Services Logic Apps Functions HDInsight Search Databricks Cosmos DB Batch Data anonymization Data selection and transformation, PII data Workflow orchestration and redaction, annotation and training data managed services preparation 

18 .Analyze your Data in one place Big Data Use Cases Ingest & ETL Streaming Analytics & Machine Learning Data Aggregation Presentation Functions Monitor Event Hubs App Insights Log Analytics Stream Machine Search Power BI Analytics Data Lake Learning Data Warehouse Analytics IoT Hub Data Factory CDN Batch Azure HDInsight Blob Storage Pillars Open & Manageable & Scalable & Secure & Durable & Interoperable Cost Efficient Performant Compliant Available Manageable & Scalable & Secure & Cost Efficient Performant Compliant 

19 .Deploy AI models in Azure for auto- labeling images to optimize for speed of labeling and possibly cost Semantic Segmentation Each Pixel of the image is assigned a category Object Detection & Classification Bounding box drawn around each object of interest 3D Point Cloud Labeling Objects of interest as assigned a category in 3D LIDAR point cloud 

20 . Process, Sensor/ Algorithm Control Logic Software Train Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x) Tag Replay Performance Hardware Simulate Simulation in the loop Render/Convert Test-Drive DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE 

21 . Partnership Opportunities Open loop testing (e.g. ADTF) Simulation tools for sensor and algorithm Scene Reconstruction Test and Replay Algorithm scoring Results Analysis validation repository Microsoft Services Workflow Management Services Cosmos Cycle Cloud Comprehensive Validation framework Blob CPU Logic Apps Batch Key Vault Service Bus PowerBI DB Compute Verification and validation of training algorithms and sensors via open loop testing 

22 .

23 .

24 .Build and deploy deep learning models Azure databricks Caffe2 Keras MS cognitive toolkit Azure Scale out clusters databricks TensorFlow Notebooks Batch AI Azure ML Services Scale out clusters Streamline Scale compute Quickly evaluate AI development efforts resources in any environment and identify the right model Leverage popular deep learning toolkits Choose VMs for your modeling needs Run experiments in parallel Develop your language of choice Process video using GPU-based VMs Provision resources automatically Leverage your favorite deep learning frameworks TensorFlow MS Cognitive Toolkit PyTorch Scikit-Learn ONNX Caffe2 MXNet Chainer 

25 .Operationalize and manage models with ease Azure Azure databricks ML services AKS ACI Containers Model MGMT, experimentation, Train and evaluate models and run history IoT edge Bring models Proactively manage Deploy models to life quickly model performance closer to your data Build and deploy models in minutes Identify and promote your best models Deploy models anywhere Iterate quickly on serverless infrastructure Capture model telemetry Scale out to containers Easily change environments Retrain models with APIs Infuse intelligence into the IoT edge 

26 . Simulation is the fastest way to “drive” the billions of miles needed to discover the breadth of real-world anomalies required to develop AD solutions. Sensor Simulation Scenario Simulation Engineering Simulation Photo-realistic rendering of modeled sensors Scenario definition and simulation for path Mathematical based simulation for (Lidar, radar, RGB cameras) for the training planning and navigational maneuvers Structural CAD design, Fluid dynamics, and validation of Perception AI algorithms. (overtake, parallel parking, etc) using post Thermo dynamics, material properties perception object level simulation. Render once, test many times Design sensor properties and placement Run 7200 simulated scenarios covering ~5000 on vehicle options at scale before miles per day/per node hardware prototyping is needed. Simulate accidents and AD navigation to a safe stop 

27 . Simulation/Reinforcement Learning Centric Model Simulation Render/convert Engine Test Vehicle Fleet Hardware Control Logic in the loop Sensor Fusion, Validation Process, Sample, Ingest/Store Training Algorithm Generate Reduce, Tag, QA Train Dataset Validation Engine Build Integrate Code Performance Software Production Simulation in the loop Vehicle Fleet F(x) Test-Drive “Drive Millions, Train on Billions” 

28 . Process, Sensor/ Algorithm Control Logic Software Train Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x) Tag Replay Performance Hardware Simulate Simulation in the loop Render/Convert Test-Drive DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE 

29 . Control Logic Validation Software in the loop Partner-based Solutions Generate Code F(x) Comprehensive test management framework Performance Hardware Simulation in the loop HIL solutions Test-Drive System validation tools Workflow management services Microsoft Services Managed services Blob storage Batch GPU VM Active Directory Container service Embedded system validation via hardware-in- loop and software-in-loop