Continuous Deployment for Deep Learning

播放视频

视频文档

Continuous Deployment for Deep Learning

下载 1

Spark开源社区

发布于

3656

人观看

#信息技术

Continuous integration and deployment has become an increasingly standard and common practice in software development. However, doing this for machine learning models and applications introduces many challenges. Not only do we need to account for standard code quality and integration testing, but how do we best account for changes in model performance metrics coming from changes to code, deployment framework or mechanism, pre- and post-processing steps, changes in data, not to mention the core deep learning model itself?

In addition, deep learning presents particular challenges:

model sizes are often extremely large and take significant time and resources to train
models are often more difficult to understand and interpret making it more difficult to debug issues
inputs to deep learning are often very different from the tabular data involved in most ‘traditional machine learning’ models
model formats, frameworks and the state-of-the art models and architectures themselves are changing extremely rapidly
usually many disparate tools are combined to create the full end-to-end pipeline for training and deployment, making it trickier to plug together these components and track down issues.

We also need to take into account the impact of changes on wider aspects such as model bias, fairness, robustness and explainability. And we need to track all of this over time and in a standard, repeatable manner. This talk explores best practices for handling these myriad challenges to create a standardized, automated, repeatable pipeline for continuous deployment of deep learning models and pipelines. I will illustrate this through the work we are undertaking within the free and open-source IBM Model Asset eXchange.

展开查看详情

1 .Continuous Deployment for Deep Learning — Nick Pentreath Principal Engineer @MLnick

2 .About – @MLnick on Twitter & Github – Principal Engineer, IBM CODAIT (Center for Open-Source Data & AI Technologies) – Machine Learning & AI – Apache Spark committer & PMC – Author of Machine Learning with Spark – Various conferences & meetups IBM Developer / © 2019 IBM Corporation 2

3 .Center for Open Source Data & AI Technologies CODAIT aims to make AI solutions dramatically easier to create, deploy, and manage in the CODAIT enterprise. We contribute to and advocate for the open-source technologies that are foundational to IBM’s AI offerings. Improving the Enterprise AI Lifecycle in Open Source 30+ open-source developers! IBM Developer / © 2019 IBM Corporation 3

4 .Agenda – Overview of Continuous Integration & Deployment – The Machine Learning Workflow – How is CI/CD for ML Different & Challenges – Model Asset Exchange – Conclusion IBM Developer / © 2019 IBM Corporation 4

21 .Deep learning pipeline Input image Image pre-processing Inference Post-processing Prediction beagle: 0.82 Decode image [0.2, 0.3, … ] Resize Label map basset: 0.09 Normalization (label, prob) bluetick: 0.07 Convert types / format Sort ... PIL, OpenCV, tf.image, Custom … Python IBM Developer / © 2019 IBM Corporation 21 * Logos trademarks of their respective projects

22 .Pipelines, not Models – Deploying (and testing) just the model – Pipelines in frameworks part of the workflow is not enough • scikit-learn – Entire pipeline must be taken into • Spark ML pipelines account • TensorFlow Transform • Data transforms • pipeliner (R) • Feature extraction & pre-processing • DL / ML model • Prediction transformation – Even ETL is part of the pipeline! IBM Developer / © 2019 IBM Corporation 22

24 .Source of changes Build Test Merge Deploy Changes come from: • Our code • Internal dependencies Data Model Dependencies • 3rd party dependencies • Data • Model • Time IBM Developer / © 2019 IBM Corporation 24

26 .Models – Size of models – Resource requirements – Hardware • CPU, GPU, TPU, Mobile, Edge – Need to manage and bridge many different languages, frameworks – Formats – State of the art is changing very rapidly IBM Developer / © 2019 IBM Corporation 26 * Logos trademarks of their respective projects

28 .Monitoring Traditional software monitoring Latency, throughput, resource usage, etc Model performance metrics Traditional ML evaluation measures Software (accuracy, prediction error, AUC etc) Business metrics Impact of predictions on business outcomes Monitoring • Additional revenue - e.g. uplift from recommender • Cost savings – e.g. value of fraud prevented Performance Business • Metrics implicitly influencing these – e.g. user engagement IBM Developer / © 2019 IBM Corporation

29 .Feedback Adapt An intelligent system must automatically Data learn from & adapt to the world around it Continual learning Retraining, online learning, reinforcement learning Feedback Transform Feedback loops Explicit: models create or directly influence their own training data Implicit: predictions influence behavior in longer-term or indirect ways Deploy Train Humans in the loop IBM Developer / © 2019 IBM Corporation

6点赞

2收藏

1下载