展开查看详情
1. Flink Streaming SQL 2018
ݪلғ data Artisans
ᘳ֖ғ Software Engineer
ᄍᦖᘏғ Piotr Nowojsk
2. About data Artisans
ىԭ data Artisans
PLATFORM
Original Creators of Real-Time Stream Processing
Apache Flink® Enterprise Ready
Apache Flink ܻত֢ᘏ մӱᕆਫၞୗ॒ቘ
3.Flink Streaming SQL 2018
4. ፓ୯
$JHQGD
ԅ֜ᭌೠ 64/Ҙ
Why SQL?
ᭌೠ 6WUHDPLQJ64/ਖ਼ᥝᶎጱ
Challenges in Streaming SQL
ࣁ 6WUHDPLQJ64/Ӿᬳളᤒጱӧݶොୗ
Various ways to join tables in Streaming SQL
ཛྷୗᦩڦ
Pattern recognition
ٌ՜ᬪ๗౮ຎ
Other recent improvements.
5. ԅ֜ᭌೠ SQL
Why SQL?
ռಅޮᎣጱളݗ
:HOONQRZQLQWHUIDFH
෫ᵱᖫᑕńńฃԭӤಋ
1RSURJUDPPLQJLVUHTXLUHG HDVLHUWROHDUQ
ኩᦫୗᤒᬡ֦ጱࠟӱ᭦ᬋ
'HFODUDWLYHZD\WRH[SUHVV\RXUEXVLQHVVORJLF
ٖୌս۸
2XWRIWKHER[RSWLPL]DWLRQ
6.ᭌೠ Streaming SQL ਖ਼ᥝᶎጱ
Challenges in Streaming SQL
7. ಢེ॒ቘਫֺ
Batch example
SELECT
a.id
FROM
A a, B b
WHERE
a.id = b.id
8. SELECT a.id FROM A a, B b WHERE a.id = b.id
Table A Table B Result
1 42 1
42 7 42
2 3 3
3 1
6
9. ୭ଚᬳളᓒဩ
Sort-Merge Join
Table A Table B
1 42
42 7
2 3
3 1
6
10. ᒫӞྍ ń ړᔄ
First step - Sort
Table A Table B
1 1
2 3
3 7
6 42
42
11. ᒫԫྍ ݳଚ݊ᬳള
Second step Merge and Join
Table A Table B Result
1 1 1
2 3 3
3 7 42
6 42
42
12. ग़ᤒᬳളᓒဩ
Hash Join
Table A Table B Result
1 42 1
42 7 42
2 3 3
3 1
6
13. ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
... 42 ...
...
14. ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
1 42 ...
... ...
15. ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
1 42 42
42 ... ...
...
16. ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
1 42 42
42 7 ...
... ...
17. ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
1 42 42
42 7 ...
... 3
...
18. ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
1 42 42
42 7 1
... 3 ...
1
...
19. ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
1 42 42
42 7 1
2 3 3
3 1 ...
6 ...
...
20. ᳵᑻݗᬳള
Time-windowed Join
21. ֖ᕚ
Watermarks
Ԫկᳵ౿
Ԥၞ Event timestamp
Stream (out of order)
21 19 20 17 22 12 14 12 9 15 11 7
֖ᕚ ֖ᕚ
Watermark 17 Watermark 11 Ԫկ
Event
22. ᳵᑻݗᬳള
Time-windowed Join
SELECT
*
FROM
Orders o, Shipments s
WHERE
o.id = s.orderId AND
s.shiptime BETWEEN o.ordertime AND o.ordertime + INTERVAL '4' HOUR
23.ᳵᑻݗᬳള
Time-windowed Join
24. ܲݥᤒ
Temporal Tables
