申请试用
HOT
登录
注册
 
Accelerating Spark MLlib and DataFrame with Vector Processor “SX-Aurora TSUBASA”

Accelerating Spark MLlib and DataFrame with Vector Processor “SX-Aurora TSUBASA”

Spark开源社区
/
发布于
/
8552
人观看
NEC has recently released new vector system “SX-Aurora TSUBASA”. This system is usually used for HPC, but is also designed for data analytics by building the vector processor as a PCIe-attached accelerator. In comparison with GPGPU, it suits for memory intensive workloads, often see at statistical machine learning and data frame processing. To accelerate data analytics on Spark, we have created acceleration framework “Frovedis” for SX-Aurora TSUBASA. It supports several machine learning algorithms on MLlib and Data Frame processing that are fully optimized for the vector processor. It is also optimized for distributed systems with multiple vector processors, and has API that is mostly the same with Spark MLlib and Data Frame. These features enables Spark developers to use multiple vector processors seamlessly from Spark and get a huge performance improvement. The performance evaluation shows that the “Frovedis” on the vector processor shows 10x to 50x speedup on several machine learning and data frame kernels compared with a Spark on Xeon Gold.
1 点赞
0 收藏
2下载
确认
3秒后跳转登录页面
去登陆