Applications of Deep Learning in Telematics

1 .Applications of Deep Learning in Telematics Wayne Zhang, Uber #UnifiedAnalytics #SparkAISummit 

2 .Stand for Safety We want Uber to be the safest transportation platform on the planet. Safety should be our number-one priority. We have to, as a company, stand for safety.” Dara Khosrowshahi (2018) 

3 .Example Driving Safety Products Driving Hours Limit Speed Limit Alert Ride Check 

4 . Telematics - Wide availability - Lower quality Source: Smartphone-based Vehicle Telematics - A Ten-Year Anniversary - Cheap - Measure phone motion - Short upgrade cycle 4 

5 .Sensor Data (Driver Device) ● GPS ○ Absolute location, velocity and time ○ Low frequency (~0.5Hz) ● IMU ○ Relative motion of phone ○ Accelerometer: 3D linear acceleration ○ Gyroscope: 3D angular velocity ○ High frequency (~25Hz) 

6 .Motivation ● High-frequency signals ○ Intricate and diverse patterns ○ Dynamic over time Driving Handling Driving Walking On Train Phone 

7 .Motivation 

8 .Sequence Classification Classify whole sequence to certain events: ● Crash ● Driving events (brake, turn, speeding) ● Phone handling ● Rider complaint ● ... 

9 .Sequence-Sequence Prediction Input sequence Output sequence (Phone sensor data) Vehicle sensor Align to other sensor phone => vehicle Turn event (binary encoding) Pinpoint telematics events (turn, activity) 

10 .Design Choice ● High-frequency data result in huge # time steps ○ Pre-filtering: identify specific time window of interest ○ Window segmentation: divide input sequence into small windows Pre-filtering Window Segmentation window0 window2 windowT window1 

11 .Window Prediction Embedding LSTM [optional] Raw data 

12 .Feature Extraction Raw data window 1 window 2 window t window T Sample Summary 1-D CNN - Time domain stats (min, max, mean, sd) - Frequency domain feature (FFT) New Feature Vector LSTM LSTM 

13 .Data Augmentation ● Sensor readings depend on phone orientation ● Create augmented data by artificially rotating phone ○ New sensor readings ○ Label stays the same 

14 . Model Dev Pipeline Data Label/Feature Model Score Non-DL - Sensor (Driver) - Telematics - SparkML Transformer - Score and classify - Map - Trip - XgBoost - Trip - xM trips in training - Other - xM trips in validation - Saved model pipeline Data Label/Feature Model Score - Sensor (Driver) - Event definition - Multi-layer LSTM - Sensor embedding DL - Feature - xM trips in training - Saved protocol buffer 

15 .Horovod ● Open source library developed at Uber ● Distributed training for TensorFlow, Keras & PyTorch ● Uses bandwidth-optimal communication protocols & makes use of advanced networking ● Seamlessly installs via pip install horovod 

16 .Petastorm ● Open source library developed at Uber ATG Apache Parquet as a dataframe with ● Enables deep learning directly from Parquet tensors ● Supports Tensorflow, PyTorch, and PySpark Hedgehog Fog Horse nd-arrays, scalars (e.g. images, Apache Parquet lidar point store clouds) 

17 .Performance DL model DL model 

18 .Proprietary and confidential © 2019 Uber Technologies, Inc. All rights reserved. No part of this document may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval systems, without permission in writing from Uber. This document is intended only for the use of the individual or entity to whom it is addressed and contains information that is privileged, confidential or otherwise exempt from disclosure under applicable law. All recipients of this document are notified that the information contained herein includes proprietary and confidential information of Uber, and recipient may not make use of, disseminate, or in any way disclose this document or any of the enclosed information to any person other than employees of addressee to the extent necessary for consultations with authorized personnel of Uber. 

19 .Thank You! Wayne Zhang actuaryzhang@uber.com