申请试用
HOT
登录
注册
 
The Internals of Stateful Stream Processing in Spark Structured Streaming
The Internals of Stateful Stream Processing in Spark Structured Streaming

The Internals of Stateful Stream Processing in Spark Structured Streaming

Spark开源社区
/
发布于
/
3644
人观看

Let’s talk about state management in Spark Structured Streaming. During this talk you will learn the streaming concepts that are particularly relevant for stateful stream processing in Structured Streaming, e.g. watermark and output modes, but also GroupState and GroupStateTimeout. We will be exploring simple stateful processing (with groupBy operator) and more advanced use cases with KeyValueGroupedDataset.mapGroupsWithState and the most advanced KeyValueGroupedDataset.flatMapGroupsWithState operator. In other words, you will learn how to use the stateful streaming API and understand the internals.

6 点赞
2 收藏
0下载
确认
3秒后跳转登录页面
去登陆