展开查看详情
1 . Flink Streaming SQL 2018
ݪلғ data Artisans
ᘳ֖ғ Software Engineer
ᄍᦖᘏғ Piotr Nowojsk
2 . About data Artisans
ىԭ data Artisans
PLATFORM
Original Creators of Real-Time Stream Processing
Apache Flink® Enterprise Ready
Apache Flink ܻত֢ᘏ մӱᕆਫၞୗ॒ቘ
3 .Flink Streaming SQL 2018
4 . ፓ୯
$JHQGD
ԅ֜ᭌೠ 64/Ҙ
Why SQL?
ᭌೠ 6WUHDPLQJ64/ਖ਼ᥝᶎጱ
Challenges in Streaming SQL
ࣁ 6WUHDPLQJ64/Ӿᬳളᤒጱӧݶොୗ
Various ways to join tables in Streaming SQL
ཛྷୗᦩڦ
Pattern recognition
ٌ՜ᬪ๗౮ຎ
Other recent improvements.
5 . ԅ֜ᭌೠ SQL
Why SQL?
ռಅޮᎣጱളݗ
:HOONQRZQLQWHUIDFH
෫ᵱᖫᑕńńฃԭӤಋ
1RSURJUDPPLQJLVUHTXLUHG HDVLHUWROHDUQ
ኩᦫୗᤒᬡ֦ጱࠟӱ᭦ᬋ
'HFODUDWLYHZD\WRH[SUHVV\RXUEXVLQHVVORJLF
ٖୌս۸
2XWRIWKHER[RSWLPL]DWLRQ
6 .ᭌೠ Streaming SQL ਖ਼ᥝᶎጱ
Challenges in Streaming SQL
7 . ಢེ॒ቘਫֺ
Batch example
SELECT
a.id
FROM
A a, B b
WHERE
a.id = b.id
8 . SELECT a.id FROM A a, B b WHERE a.id = b.id
Table A Table B Result
1 42 1
42 7 42
2 3 3
3 1
6
9 . ୭ଚᬳളᓒဩ
Sort-Merge Join
Table A Table B
1 42
42 7
2 3
3 1
6
10 . ᒫӞྍ ń ړᔄ
First step - Sort
Table A Table B
1 1
2 3
3 7
6 42
42
11 . ᒫԫྍ ݳଚ݊ᬳള
Second step Merge and Join
Table A Table B Result
1 1 1
2 3 3
3 7 42
6 42
42
12 . ग़ᤒᬳളᓒဩ
Hash Join
Table A Table B Result
1 42 1
42 7 42
2 3 3
3 1
6
13 . ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
... 42 ...
...
14 . ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
1 42 ...
... ...
15 . ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
1 42 42
42 ... ...
...
16 . ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
1 42 42
42 7 ...
... ...
17 . ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
1 42 42
42 7 ...
... 3
...
18 . ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
1 42 42
42 7 1
... 3 ...
1
...
19 . ᬳളᬳᖅັᧃ
Join in continuous queries
Table A Table B Result
1 42 42
42 7 1
2 3 3
3 1 ...
6 ...
...
20 . ᳵᑻݗᬳള
Time-windowed Join
21 . ֖ᕚ
Watermarks
Ԫկᳵ౿
Ԥၞ Event timestamp
Stream (out of order)
21 19 20 17 22 12 14 12 9 15 11 7
֖ᕚ ֖ᕚ
Watermark 17 Watermark 11 Ԫկ
Event
22 . ᳵᑻݗᬳള
Time-windowed Join
SELECT
*
FROM
Orders o, Shipments s
WHERE
o.id = s.orderId AND
s.shiptime BETWEEN o.ordertime AND o.ordertime + INTERVAL '4' HOUR
23 .ᳵᑻݗᬳള
Time-windowed Join
24 . ܲݥᤒ
Temporal Tables
