Building Zhaopin's enterprise event center on Apache Pulsar

Apache Pulsar 如何满足智联招聘的需求、智联招聘如何使用 Pulsar,以及将 Pulsar 用作企业事件总线的最佳实践。重点介绍了 Apache Pulsar 的优势、高可用、高吞吐量和低延迟等特性,以及为何 Apache Pulsar 是企业事件总线的理想选择。

展开查看详情

1. Building Zhaopin’s enterprise Event Center on Apache Pulsar Penghui Li Jia Zhai (@lipenghui6) (@Jia_Zhai)

2.Zhaopin.com Zhaopin.com is the biggest online recruitment service provider in China Zhaopin.com provides job seekers a comprehensive resume service, latest deployment, and career development related information, as well as in-depth online job search for positions throughout China. Zhaopin.com provides professional HR services to over 2.2 million clients and its average daily pageviews are over 68 millions.

3.Who we are ❏ Penghui Li ❏ Tech lead of Infrastructure team at Zhaopin ❏ 5+ years of experiences in messaging and microservices ❏ Apache Pulsar Committer

4.Who we are ❏ Jia Zhai ❏ Pulsar PMC Member / Committer ❏ BookKeeper PMC Member / Committer ❏ Funding engineer at StreamNative

5.Agenda ❏ Why building an Event Center ❏ Why Apache Pulsar ❏ Apache Pulsar at Zhaopin ❏ Streaming Platform ❏ Zhaopin’s contributions to Apache Pulsar

6.Why building an Event Center Data Silos -> Unified Platform

7.Data Silos Pain Points To Enterprises Data Processing ❏ High Maintenance Cost MSMQ Kafka ❏ Extremely hard to scale data cross teams ❏ Inconsistency between data silos ❏ Doesn’t scale To End Users ❏ No consistent SLA RabbitMQ

8.Data Silos Pain Points To Enterprises Data Processing ❏ High Maintenance Cost MSMQ Kafka ❏ Extremely hard to scale data cross teams ❏ Inconsistency between data silos ❏ Doesn’t scale To End Users ❏ No consistent SLA RabbitMQ

9.Unification - MQService Problem Solved ❏ Simplified Operations ❏ Scale-out Service ❏ High Availability Problem Unsolved ❏ Keep messages for longer period ❏ Data rewind ❏ Order guarantee

10.Unification - MQService Online Services Data Processing MQService Kafka

11.Why building an Event Center

12.Why building an Event Center

13.Why building an Event Center

14.Why building an Event Center

15.Why Apache Pulsar Pulsar == Messaging + Storage

16.Why Apache Pulsar? Flexible Pub/Sub Messaging backed by scalable log storage

17.Why Apache Pulsar / Multi Tenancy

18.Why Apache Pulsar / Queuing + Streaming

19.Why Apache Pulsar / Cloud Native Architecture

20.Why Apache Pulsar

21. Apache Pulsar at Zhaopin 20+ core services, 20 billions events/day

22.Unification - MQService Problem Solved ❏ No Data Silos ❏ Queue + Streaming ❏ Disaster Recovery ❏ Infinite Stream Storage (via Tiered Storage) ❏ Data rewind

23.Milestones

24.Core Metrics ❏ 50+ Namespaces ❏ 5000+ Topics ❏ 20+ billions events/day ❏ 5TB storage per day ❏ 20+ core services

25.System Metrics

26.Pulsar at Zhaopin ❏ One copy of data, single source-of-truth ❏ Don’t worry about data consistency between RabbitMQ and Kafka ❏ Multi-tenancy makes topic management easier ❏ Strong data durability allows us to stop worrying about message loss

27.Event Streaming Platform Beyond Pub/Sub Messaging

28.Event Streaming Platform

29.Event Streaming Platform ❏ Pulsar Functions: lightweight computing ❏ Flink: streaming-first, unified data processing ❏ Pulsar SQL (presto): interactive queries on both historic and real-time data