申请试用
HOT
登录
注册
 
Fast and Reliable Apache Spark SQL Engine
Fast and Reliable Apache Spark SQL Engine

Fast and Reliable Apache Spark SQL Engine

Spark开源社区
/
发布于
/
7456
人观看
Building the next generation Spark SQL engine at speed poses new challenges to both automation and testing. At Databricks, we are implementing a new testing framework for assessing the quality and performance of new developments as they produced. Having more than 1,200 worldwide contributors, Apache Spark follows a rapid pace of development. At this scale, new testing tooling such as random query and data generation, fault injection, longevity stress, and scalability tests are essential to guarantee a reliable and performance Spark later in production. By applying such techniques, we will demonstrate the effectiveness of our testing infrastructure by drilling-down into cases where correctness and performance regressions have been found early. In addition, showing how they have been root-caused and fixed to prevent regressions in production and boosting the continuous delivery of new features.
4点赞
2收藏
44下载
确认
3秒后跳转登录页面
去登陆