Pulsar 简介和用户案例-郭斯杰

郭斯杰 | Apache Pulsar 和 Apache BookKeeper 的 PMC 成员和 Committer
摘要:权威解读 Apache Pulsar 及其发展历史、现状和用户案例。

展开查看详情

1.Apache Pulsar Introduction and community

2. 我是谁 我来自哪里 我要到哪里去

3.What is Apache Pulsar?

4.What is Apache Pulsar? 2003 2010 2012 2006 2011

5.Built for Cloud

6. A brief history of Pulsar • 2012 Pulsar idea started • 5+ years on production, 100+ applications, 10+ data centers • 2016/09 Yahoo open sourced Pulsar • 2017/06 Yahoo donated Pulsar to ASF • 2018/09 Pulsar graduated as a Top-Level project • 25 committers, 139 contributors, 843 forks, 3700+ stars

7.Pulsar

8.1.x - PubSub Messaging Multi language clients Messaging Stream Storage Low Latency Storage

9. 1.x - PubSub Messaging Multi language clients Messaging Stream Storage Low Latency Storage

10. 1.x - PubSub Messaging Multi language clients Messaging Stream Storage Low Latency Storage

11. 1.x - PubSub Messaging Multi language clients Messaging Stream Storage Low Latency Storage

12. 1.x - PubSub Messaging Multi language clients Messaging Stream Storage Low Latency Storage

13. 1.x - PubSub Messaging Multi language clients Messaging Stream Storage Low Latency Storage

14. 1.x - PubSub Messaging Multi language clients Messaging Stream Storage Low Latency Storage

15. 1.x - PubSub Messaging Multi language clients Messaging Stream Storage Low Latency Storage

16. 1.x - PubSub Messaging Multi language clients Messaging Stream Storage Low Latency Storage

17.1.x - PubSub Messaging Multi language clients Messaging Stream Storage Low Latency Storage

18. 2.x - Tiered Storage Multi language clients Messaging Stream Storage Low Latency Cloud Native Storage Storage

19. 2.x - Processing Multi language clients Pr g in oc g es sa s in s Me g Stream Storage Low Latency Cloud Native Storage Storage

20.2.x - Interactive Queries Interactive Queries Multi language clients Pr g in oc g es sa s in s Me g Stream Storage Low Latency Cloud Native Storage Storage

21.2.x - Unified Data Processing Interactive Queries Multi language clients Pr g in oc Stream & Batch g es sa Processing s in s Me g Stream Storage Low Latency Cloud Native Storage Storage

22.2.x - Pulsar Functions Interactive Queries Multi language clients Pr g in oc Stream & Batch g es sa Processing s in s Me g Function Processing Stream Storage { } Low Latency Cloud Native Storage Storage

23. 2.x - Pulsar IO Interactive Queries Multi language clients Pr g Pulsar IO in oc Stream & Batch g es sa Processing s in s Me g Function Processing Stream Storage { } Low Latency Cloud Native Storage Storage

24. 2.x - Pulsar CDC Interactive Queries Multi language clients Pr g Pulsar IO in oc Stream & Batch g es sa Processing s in s Me g Change Data Capture Function Processing Stream Storage { } Low Latency Cloud Native Storage Storage

25.Event Streaming Platform Interactive Queries Multi language clients Pr g Pulsar IO in oc Stream & Batch g es sa Processing s in s Me g Change Data Capture Function Processing Stream Storage { } Low Latency Cloud Native Storage Storage

26.Use cases & Community

27. 案例 - Yahoo • 使用 Pulsar 作为核心的 Event Data Bus • 整合不同的技术和集群成统一的解决 方案 • 全球多机房( 8+) 复制 • 每日处理千亿量级的消息,多达 230 万个 topics

28. 案例 - Twitter • 整个消息系统 (EventBus)EventBus) 的存储方案 • Search 、 Ads 、 Stream computing 、 Manhattan Database • 1000+ Bookies per datacenter • 17PB/day, 1.5 trillion records/second

29. 案例 - 智联招聘 Pulsar based Event Center POC 3 billion msgs/day 1 billion msgs/day 2018/07 2018/09 2018/10 2018/11 2018/12 2019/02 Win the best innovative 6 billion msgs/day Pulsar on Production platform award at Zhaopin • 统一: 替换 RabbitMQ + Kafka • 0 数据丢失 • 50+ NameSpaces • 60 亿 消息 / 天 • 3TB Storage/ 天 • 20+ 核心应用