Apache Kafka在京东的演进和实践

1 .Apache Kafka x Apache Flink· 

2 .

3 .

4 .

5 .

6 .

7 .

8 .

9 . p oi ( c 01 d d 02 03 a& n m t ) t d 

10 .

11 . 2 + 10 CS Z K b c 2 . ) Z - ( - Z - ( - a 

12 .K K 

13 .• • • • • • • • • • • • • • • • • • • • 

14 .

15 . (Raid10) Kafka Kafka 0.10.1.X V1 700w/s 1400w/s 0.11 0.8 CPU 78% 200MB/s Crc32 CPU80% Crc32 kafka Crc32 hadoop Crc32c jdk8 Crc32 jdk9 Crc32c JMH 1kb jdk8 crc32 

16 . (Raid10) Kafka Kafka 2.1.X V2 2400w/s 3500w/s 0.11 CPU 96% 600MB/s CPU CPU V2 kafka V2 CPU 

17 . JD Record byteBuffer 2kw 2kw GC 1. key compact 2. ts value key 3. inPlaceAssignment key value 3500w/s CPU 15% 1100MB/s 3500w/s CPU 58% 1100MB/s CPU GC 

18 . IL S K A nb 1 P S C 2 5 Dc g 3 N g 4 o f kr 1 

19 . & & fP K nk f &tU CSM M ) OL p e ( c otU Z 8 Vg a i Vg k fr l n e e &o tU c 

20 .