申请试用
HOT
登录
注册
 
SPIP: Accelerator-aware scheduling

SPIP: Accelerator-aware scheduling

Spark开源社区
/
发布于
/
3638
人观看
GPUs and other accelerators have been widely used for accelerating special workloads, e.g., deep learning and signal processing. While users from the AI community use GPUs heavily, they often need Apache Spark to load and process large datasets and to handle complex data scenarios like streaming. YARN and Kubernetes already support GPUs in their recent releases. Although Spark supports those two cluster managers, Spark itself is not aware of GPUs exposed by them and hence Spark cannot properly request GPUs and schedule them for users. This leaves a critical gap to unify big data and AI workloads and make life simpler for end users.
1点赞
0收藏
0下载
确认
3秒后跳转登录页面
去登陆