李剑--Flink在阿里巴巴电商业务中的应用

1 .Application of Flink in Alibaba E-Commerce Business Flinkࣁᴨ᯾૬૬ኪࠟӱ‫ۓ‬Ӿጱଫአ Corperation: ALIBABA ‫ݪل‬ғᴨ᯾૬૬ Position Technical experts ᘳ֖ғದ๞ӫਹ Speaker: Qiu Qi ᄍᦖᘏғᐾ॰ 

2 .Ø Business background ӱ‫ۓ‬ ᙧว Ø Typical scenario َࣳ࣋ว Ø Data development හഝ୏‫ݎ‬ 

3 .Offline Computing Real-time Computing ᐶᕚᦇᓒ ਫ෸ᦇᓒ 

4 .Application DataSource Computing Storage Bussiness ଫአ හഝრ ਂ‫ؙ‬ ӱ‫ۓ‬ ᦇᓒ 

5 .Index Computing ೰ຽᦇᓒ 

6 .

7 . Deal Table Deal Screen of Double 11 ౮Իᤒ ‫݌‬౮Իय़੽ orderId day userId sellerId price 1 20171111 A 100 0.01 2 20171111 B 101 128.25 3 20171111 C 100 45.2 4 20171111 A 102 100.00 6 20171110 D 100 300.00 select day,sum(price) as total from deal_table group by day select day,sellerId,sum(price) as sellerTotal from deal_table group by day,sellerId 

8 . 10:00 search 10:00 ൤ᔱ User clicks on the search results page and there is an order transaction on that day አಁࣁ൤ᔱᕮຎᶭ‫ݎ‬ኞᅩ‫҅ڋ‬ 10:01 click on search ଚӬ୮ॠํᦈ‫ܔ‬౮Ի result 10:01 Direct lead transaction of search channel ൤ᔱჀ᭲ፗള୚੕౮Ի᷐ 22:30 Deal done 22:30 

9 . Search click table: A Deal table: B ൤ᔱᅩ‫ڋ‬ᤒғA ౮ԻᤒғB day userId itemId day userId itemId price 20181111 A 1 20181111 A 1 0.01 20181111 A 1 20181111 B 2 128.25 20181111 B 3 20181111 B 3 45.2 20181111 C 4 20181111 A 4 100.00 create viewA as select day as left_day, userId as left_userId, itemId as left_itemId from A group by day,userId, itemId create viewB select day as right_day, userId as right_userId, itemId as direct lead turnover of search channel right_itemId, price from B ൤ᔱჀ᭲ፗള୚੕౮Ի᷐ select day, sum(price) as total from (select left_day as day, price from viewA join viewB on left_day =right_day and left_userId and right_userId and left_itemId = right_itemId)t group by day 

10 .,QWHUDFWLYH5HDOWLPH%LJ 'DWD6HUYLFH Ի԰ୗਫ෸य़හഝ๐‫ۓ‬ 

11 .Data Synchronization හഝ‫ྍݶ‬ 

12 . MySQL HBase Data integration 20+ Heterogeneous data source හഝᵞ౮ ୑຅හഝრ HDFS Insert into target_table select *** from source_table OSS ODPS 

13 .Monitoring and Alarm ፊഴಸᦄ 

14 .Search without result ൤ᔱ෫ᕮຎ Tmall&TaobaoCo & mmodity share ॠሞ Ⴃਪࠟߝ ‫ྲܛ‬ 

15 .

16 .shortage early unmarketable warning control Supply chain replenishment 

17 .Scene Reproduction ࣋ว॔ሿ 

18 . Full-link debug platform ‫ق‬᱾᪠debugଘ‫ݣ‬ replay debug validate Personalized snapshot User action Log collection Data analysis Storage Reproduction አಁ඙֢ ෭பතᵞ හഝᥴຉ ਂ‫ؙ‬ ॔ሿ 

19 .

20 .Stream computing development platform ၞᦇᓒ୏‫ݎ‬ଘ‫ݣ‬ 

21 .

22 .

23 .DAG development mode DAG୏‫ݎ‬ཛྷୗ 

24 .DAG\SQL mode transformation DAGཛྷୗ԰᫨ 

25 .Real-time Debug ਫ෸᧣ᦶ 

26 .UDF online development UDFࣁᕚ୏‫ݎ‬ 

27 .

28 .

29 .