郑鲁-类SQL在实时流处理上的探索及应用final

注脚

展开查看详情

1.

2.第七届 全球软件案例研究峰会 AI/ AIOps DevOps AI 2018 11 30 -12 3 | 100+

3. 上海精品公开课 管理3.0认证课程 时间:12月22-23日 | 地点:上海 | 讲师:林伟丹 大数据及AI挖掘技术 时间:12月22-23日 | 地点:上海 | 讲师:风清扬 高可用架构与设计 时间:01月12-13日 | 地点:上海 | 讲师:沈老师 K8S与service mesh 时间:01月12-13日 | 地点:上海 | 讲师:Jim 备注:扫码查看课程详情,两人以上报名有优惠,详情咨询:15802217295

4.Ø Ø Ø Ø Ø

5.01 02 03 04

6.RD • • •

7.SQL

8.01 02 03 & 04

9.l Apache Beam l Apache Samza ü l Apache Apex ü l Apache Flink ü l Apache Strom ü SQL l Apache Strom trident l Apache Spark

10. Native Micro-batching Micro-batching Native At-least-once Exactly-once Exactly-once Exactly-once ACKS RDD based Checkpointing Checkpointing 1 Lineage Chandy-Lamport 2 Checkpoint Metadata operator DStream operator l mapWithState l updateStateBykey l Keyed State l Operator State Process time SQL Spark sql table api SQL java Java/Scala Java/Scala

11.Why Choose Flink? Lambda Kappa

12.Why Choose Flink? Event Time 3 Ingestion Time Kafka Flink Processing Time eg: <<Flink >> l === Event Time l == Ingestion Time l == Processing Time

13.Why Choose Flink? Tumbling Window Sliding Window Session Window ,( 1 ) 1 30 30 1

14.Why Choose Flink? ü Event-driven Appliacations ü Watermark ü Keyed State Operator State ü Checkpoint

15.01 & 02 03 04

16. SQL Ø ETL Flink Mysql Kakfa Eagle OLAP Ø

17.

18.SQL ETL

19.SQL ETL

20.SQL ETL

21. SQL ETL Flink JSONSource POJO Row JSON Table Flink

22. SQL ETL Ø json Ø SQL UDF Ø kafka mysql Eagle SnappyData

23.SQL ETL- 8 7 6 5 4 3 2 1 0 DSL

24.SQL ETL -

25.SQL

26.V1 Flink +siddhi V2: Flink SQL match recognize

27.CEP Complex Event Processing Storm+Esper Storm+avi Apache Flink CEP + ator Eagle esper query matching Storm+Siddhi cpu

28.•Multi-threading •Queues and use of pipelining •Nested queries and chaining streams

29.V1