申请试用
HOT
登录
注册
 
Productizing Structured Streaming Jobs
Productizing Structured Streaming Jobs

Productizing Structured Streaming Jobs

Spark开源社区
/
发布于
/
9287
人观看
“Structured Streaming was a new streaming API introduced to Spark over 2 years ago in Spark 2.0, and was announced GA as of Spark 2.2. Databricks customers have processed over a hundred trillion rows in production using Structured Streaming. We received dozens of questions on how to best develop, monitor, test, deploy and upgrade these jobs. In this talk, we aim to share best practices around what has worked and what hasn’t across our customer base. We will tackle questions around how to plan ahead, what kind of code changes are safe for structured streaming jobs, how to architect streaming pipelines which can give you the most flexibility without sacrificing performance by using tools like Databricks Delta, how to best monitor your streaming jobs and alert if your streams are falling behind or are actually failing, as well as how to best test your code.”
7 点赞
4 收藏
2下载
确认
3秒后跳转登录页面
去登陆