申请试用
HOT
登录
注册
 
Distributed Deep Learning Training and Inference on Apache Spark

Distributed Deep Learning Training and Inference on Apache Spark

Spark开源社区
/
发布于
/
3763
人观看
Big data and AI are joined at the hip: the best AI applications require massive amounts of constantly updated training data to build state-of-the-art models. AI has always been one of the most exciting applications of big data. Project Hydrogen is a major Apache Spark initiative to bring the best AI and big data solutions together. It introduced barrier execution mode to Spark 2.4.0 release to help distributed model training, and it explores optimized data exchange to accelerate distributed model inference. In this talk, we will explain why barrier execution mode is needed, how it works, and how to use it to integrate distributed DL training on Spark. We will demonstrate HorovodRunner, the first Spark+AI integration powered by Project Hydrogen. It is based on the Horovod framework
10 点赞
3 收藏
4下载
确认
3秒后跳转登录页面
去登陆