Apache Flink+Apache Kafka在Vipkid的实践

1 .Apache Kafka x Apache Flink· 

2 .

3 .

4 .

5 .

6 . Yarn Kafka HDFS Mysql Hadoop Spark Tidb Hbase ES Flink Kafka 

7 .l l l l 

8 .l l 

9 .l l 

10 .

11 .l l 

12 .l l l l l 

13 .l l 

14 . PROCESS SINK MYSQL LOG KAFKA SOURCE DB MQ FLINK ES DATA HDFS 

15 .

16 .

17 .

18 .

19 .{"id":1,"stu":123,"status":"book"} (true,123,1) ---------- INSERT {"id":2,"stu":123,"status":"book"} {"id":1,"stu":123,"status":"finish"} (false,123,1) ---------- DELETE UPDATE {"id":2,"stu":123,"status":"finish"} (true,123,2) ---------- INSERT 

20 .

21 .

22 .{"id":1,"stu":123,"status":"book"} (true,123,1) ---------- INSERT {"id":2,"stu":123,"status":"book"} {"id":1,"stu":123,"status":"finish"} {"id":2,"stu":123,"status":"finish"} (false,123,1) ---------- DELETE UPDATE (true,123,2) ---------- INSERT (false,123,2) ---------- DELETE (true,123,1) ---------- INSERT UPDATE (false,123,1) ---------- DELETE DELETE 

23 .

24 .

25 .