李剑--Flink在阿里巴巴电商业务中的应用

Flink在阿里巴巴电商业务中的应用
展开查看详情

1.Application of Flink in Alibaba E-Commerce Business Flinkࣁᴨ᯾૬૬ኪࠟӱ‫ۓ‬Ӿጱଫአ Corperation: ALIBABA ‫ݪل‬ғᴨ᯾૬૬ Position Technical experts ᘳ֖ғದ๞ӫਹ Speaker: Qiu Qi ᄍᦖᘏғᐾ॰

2.Ø Business background ӱ‫ۓ‬ ᙧว Ø Typical scenario َࣳ࣋ว Ø Data development හഝ୏‫ݎ‬

3.Offline Computing Real-time Computing ᐶᕚᦇᓒ ਫ෸ᦇᓒ

4.Application DataSource Computing Storage Bussiness ଫአ හഝრ ਂ‫ؙ‬ ӱ‫ۓ‬ ᦇᓒ

5.Index Computing ೰ຽᦇᓒ

6.

7. Deal Table Deal Screen of Double 11 ౮Իᤒ ‫݌‬౮Իय़੽ orderId day userId sellerId price 1 20171111 A 100 0.01 2 20171111 B 101 128.25 3 20171111 C 100 45.2 4 20171111 A 102 100.00 6 20171110 D 100 300.00 select day,sum(price) as total from deal_table group by day select day,sellerId,sum(price) as sellerTotal from deal_table group by day,sellerId

8. 10:00 search 10:00 ൤ᔱ User clicks on the search results page and there is an order transaction on that day አಁࣁ൤ᔱᕮຎᶭ‫ݎ‬ኞᅩ‫҅ڋ‬ 10:01 click on search ଚӬ୮ॠํᦈ‫ܔ‬౮Ի result 10:01 Direct lead transaction of search channel ൤ᔱჀ᭲ፗള୚੕౮Ի᷐ 22:30 Deal done 22:30

9. Search click table: A Deal table: B ൤ᔱᅩ‫ڋ‬ᤒғA ౮ԻᤒғB day userId itemId day userId itemId price 20181111 A 1 20181111 A 1 0.01 20181111 A 1 20181111 B 2 128.25 20181111 B 3 20181111 B 3 45.2 20181111 C 4 20181111 A 4 100.00 create viewA as select day as left_day, userId as left_userId, itemId as left_itemId from A group by day,userId, itemId create viewB select day as right_day, userId as right_userId, itemId as direct lead turnover of search channel right_itemId, price from B ൤ᔱჀ᭲ፗള୚੕౮Ի᷐ select day, sum(price) as total from (select left_day as day, price from viewA join viewB on left_day =right_day and left_userId and right_userId and left_itemId = right_itemId)t group by day

10.,QWHUDFWLYH5HDOWLPH%LJ 'DWD6HUYLFH Ի԰ୗਫ෸य़හഝ๐‫ۓ‬

11.Data Synchronization හഝ‫ྍݶ‬

12. MySQL HBase Data integration 20+ Heterogeneous data source හഝᵞ౮ ୑຅හഝრ HDFS Insert into target_table select *** from source_table OSS ODPS

13.Monitoring and Alarm ፊഴಸᦄ

14.Search without result ൤ᔱ෫ᕮຎ Tmall&TaobaoCo & mmodity share ॠሞ Ⴃਪࠟߝ ‫ྲܛ‬

15.

16.shortage early unmarketable warning control Supply chain replenishment

17.Scene Reproduction ࣋ว॔ሿ

18. Full-link debug platform ‫ق‬᱾᪠debugଘ‫ݣ‬ replay debug validate Personalized snapshot User action Log collection Data analysis Storage Reproduction አಁ඙֢ ෭பතᵞ හഝᥴຉ ਂ‫ؙ‬ ॔ሿ

19.

20.Stream computing development platform ၞᦇᓒ୏‫ݎ‬ଘ‫ݣ‬

21.

22.

23.DAG development mode DAG୏‫ݎ‬ཛྷୗ

24.DAG\SQL mode transformation DAGཛྷୗ԰᫨

25.Real-time Debug ਫ෸᧣ᦶ

26.UDF online development UDFࣁᕚ୏‫ݎ‬

27.

28.

29.