【主会场06-罗李】滴滴实时计算平台架构与实践 部分2

1 . UDFय़ଏಘ઀ Map॒ቘ‫ڍ‬හ ෭๗॒ቘ‫ڍ‬හ ᑮᳵ֖ᗝ‫ڍ‬හ    ਁᒧԀ॒ቘ‫ڍ‬හ   JSON॒ቘ‫ڍ‬හ UDFय़ଏಘ઀ 

2 . ඪ೮चԭTTLጱ‫݌‬ၞJoin ၾ௳ᴚ‫ڜ‬ StreamSQL ၾ௳ᴚ‫ڜ‬ topic1 source1 JoinOperator Sink topic3 topic2 source2 TTLState1 TTLState2 Ø inner join Ø Left join Ø Full join 

3 . ᖌᤒJOIN ᖌᤒਂ‫ؙ‬ HBase/MySQL) ၾ௳ᴚ‫ڜ‬ StreamSQL ၾ௳ᴚ‫ڜ‬ ୑ྍ JoinOperator IO topic1 source2 Sink topic2 TTLState Cache 

4 .SQL IDE ØStreamSQL Editor ØSQLཛྷ຃ ØUDF‫ڍ‬හ᧔ก Ø᧍ဩ༄ၥ ØDEBUG Øᇇ๜ᓕቘ 

5 . ਫ෸ᦇᓒଘ‫ݣ‬ຝ຅ Stream Platform Infrastructure 

6 . ਫ෸ᦇᓒଘ‫ݣ‬ຝ຅ ձ‫ۓ‬ ᦬ෙ ᤅᖭ ፊഴ ग़ᑄಁ ଘ‫੶ݣ‬ SQL IDE WEB IDE ᓕഴ ֛ᔮ ‫ى‬ᔮ ಸᦄ ֛ᔮ ਫ෸ᥢ‫܃ڞ‬ᯈ StreamSQL DataStream ҁCEP ୚ක੶ Flink Streaming ਫ෸ ਂ‫ؙ‬ Hadoop ବଷ ‫૲ړ‬ୗ෈կᔮᕹғHDFSᕹӞᩒრ᧣ଶғYARN 

7 . ӣᐿၞᦇᓒ୏‫ݎ‬ཛྷୗ Web۸ձ‫ۓ‬ᓕഴ ๜ࣈIDE୏‫ݎ‬ ၾᴻਮಁ๢ Web IDE୏‫ݎ‬ ൉‫ف׀‬ᳪཛྷ຃ SQL IDE୏‫ݎ‬ ٖᗝ݇හ᧣ս 

8 . ٖᗝ೰ຽӤಸ ᤅᖭ‫ى‬ᔮ ᤅᖭӤಸ Druid ୊෸༄ၥ ೰ຽӤಸ Kafka Flink App ೰ຽፊഴ ෭பӤಸ ES ෭ப༄ᔱ 

9 . හഝᤅᖭ‫ى‬ᔮ ᤅᖭ‫ى‬ᔮ ᤅᖭӤಸ Druid ୊෸༄ၥ ೰ຽӤಸ Kafka Flink App ೰ຽፊഴ ෭பӤಸ ES ෭ப༄ᔱ 

10 . Hadoopବଷ Router Ø ᇿᒈୌᦡਫ෸ᦇᓒᵞᗭ҅᭿‫ع‬ᐶᕚձ‫ۓ‬ଗಟ Ø ਫ෸ᵞᗭੱᰁᶌᬪӱ‫҅ۓ‬ᴳ֗ᗑᕶ෸୊ Ø ਧ‫᧣ګ‬ଶ࢏ RM1 ਫ෸᧣ଶ࢏ RM2 ਫ෸᧣ଶ࢏ Ø ᧣ଶ࣐ᤍ Ø चԭ&38᧣ଶ Ø चԭ1RGH /DEHO޾&*URXS‫݌‬᯿ᵍᐶ๢‫ګ‬ NM1 NM2 NM3 NM4 NM5 NM6 Ø ग़๢಄᮱ᗟ Lable1 Lable1 Lable2 Lable3 Lable4 Lable4 HDFS1 HDFS1 DC1 DC2 

11 .௛ᕮӨᥢ‫ښ‬ Future plans 

12 . ௛ᕮ Summary Ø 6WUHDP64/ य़ଏᴳ֗ၞᦇᓒ୏‫ݎ‬ᳪད Ø StreamSQL makes the Streaming develop much easier Ø ਫ෸ᥢ‫܃ڞ‬ᯈҁ&(3҂ ࣁӻ௔۸០ᲀ̵ਫ෸ᷚഴᒵᶾऒ៧ࣈ Ø Real-time CEP very useful in the risk control & Personalized marketing Ø ਫ෸ᦇᓒଘ‫ ݣ‬Ӟᒊୗၞᦇᓒ୏‫ݎ‬ଘ‫ݣ‬ Ø A Big Shared Stream Processing platform 

13 . )OLQNᥢ‫ښ‬ Flink Plans Ø ਠ౮6SDUN 6WUHDPLQJ‫ک‬6WUHDP64/ጱද᭜ Ø Streaming Job Spark Streaming -> StreamSQL upgrade Ø ,R7ᶾऒ‫ێݎ‬ Ø In IoT (Cars/bicye/Monitor in Cars) Ø 6WUHDP0/ୌᦡ Ø StreamML 

14 .