确定删除吗?
1.ᄆᄆਫᦇᓒଘݣຝӨਫ᪢ The Application and Architecture of the Real-Time Computing Platform in DiDi ݪلғᄆᄆڊᤈ Company: DiDi Chuxing ᘳ֖ғದᎸᑪާ Title Director of BigData Infrastructure Department ᄍᦖᘏғᗔ Lecturer Luo Li
2. Agenda Ø ᙧว݊ሿᇫ Ø Background Ø ਫᥢ܃ڞᯈਫ᪢ Ø The CEP practice Ø 6WUHDP64/Օᕨ Ø StreamSQL Ø ਫᦇᓒଘݣຝ Ø Stream Platform Infrastructure Ø ᕮᥢښ Ø Future plans
3. ӱۓᇙᅩ characteristic of business
4. Ø ᄆᄆฎӞӻਫԻฃක Ø DiDi is a realtime Trade Engine Ø ᄆᄆጱහഝ࣋วॠᆐฎਫጱ Ø Didi's data and scenes are naturally real-time Special Shared /X[XU\ ள ӫ սՁ ᨟ … ڊᑄ Fast Cars … Taxi Cars Cars &DUV Transaction Collection ᭃහഝ Իฃහഝ ईᅩහഝ ෭பහഝ Trace Data Log Data Data point Data ਫᵞ ਫᦇᓒ Real-time ETC & Collection Real-time process
5. ၞᦇᓒದᄍᬰ Didi’s Real-time develop steps ᒫӞᴤྦྷғӱۓොᛔୌੜᵞᗭ First Step A lot of some Clusters ᒫԫᴤྦྷғᵞӾୗय़ᵞᗭ̵ଘݣ۸ Second Step Single Big Shared Cluster ᒫӣᴤྦྷғၞᦇᓒݎ64/۸ Third Step SQL On Stream process चԭFlinkጱStreamSQLӤᕚ StreamSQL On Flink चԭFlink CEPጱਫᥢ܃ڞ ᯈ๐ۓӤᕚ Flink CEP Flink Streaming ಭفଫአ Flink Streaming Spark Streaming Storm/Jstorm/ ๐ۓ۸ଘݣ۸ Storm/Jstorm/ Spark Streaming Spark/Samza Spark/Samza ӱۓොᛔୌੜᵞᗭ some clusters
6. ଫአ࣋ว Scenarios Ø ਫ̵ڭݎਫᷚഴ̵ Ø හഝၾ௳ᴚکڜHBase Ø Ø HBase ਫଉ༄ၥᒵ Druid ESᒵਂؙ Druid ES - -- Ø Իฃፊഴ̵๐ۓፊ Ø ਫԙਮᇙ̵ݪᇙ̵ Ø Ø ഴ̵ૡܔፊഴᒵ ᦈܔᇙᒵ
7. ଫአᥢཛྷ Data Scale ๐ۓӱۓᕚ ᵞᗭᥢཛྷ ၞᦇᓒձۓහ ྯ෭॒ቘහഝᰁ Applications Cluster Scale Streaming Jobs Data Scale per day
8.ਫᥢ܃ڞᯈҁCEP҂ਫ᪢ The CEP practice
9. ਫᥢ܃ڞᯈҁCEP҂ Pattern .begin(s1) ӻ۸០ᲀ .where(key=bubble && city=166) Ø ʼnउ૱ԙਮٛုݸᑁဌํӥܔŊ .notFollowedBy(s2) .where(key=order_send && city = 166) Ø ʼnउ૱ӥݸܔᑁဌՈളܔŊ .timeEnd(s3) Ø શ꧊ᥢڞ .timeShiftMore(s1, Time.ms(5000)) ᥢڜڞᤒ$3, ଉૡܔ༄ၥ Ø ᦈܔᇫாᳩᳵړᰦ؊ኸࣁݪ૪ಶܔŊ හഝრ ᶼ॒ቘ DD-CEP ܃ᯈ Øʼnᦈܔᇫாᳩᳵੜ؊ኸࣁতᦇᩇŊ ਫᥢ܃ڞᯈ๐ۓ Øʼn೪ӨGRVᦈܔᇫாӧӞᛘݪ૪ಶܔŊ
10.CEPۑᚆ ᚆදᬰ
11.StreamSQL
12. StreamSQLୌᦡ ಘ''/ ӿ8') ၞ-2,1 ᖌᤒ-2,1 64/ ,'(
13. ಘ''/ Ø ᭗ᄆᄆٖ᮱Ԇၞၾ௳ᴚڜਫਂؙ Ø ᭗ᬦಘ''/ݙൈᬿ6RXUFH6LQN MQ Ø ٖᗝ-621̵ຽٵ෭ப̵3%໒ୗᥴຉ Kafka Druid Kafka HBase StreamSQL Phoenix MQ ElaticSearch MYSQL KVStore