展开查看详情
1 . Building Zhaopin’s enterprise
Event Center on Apache Pulsar
Penghui Li Jia Zhai
(@lipenghui6) (@Jia_Zhai)
2 .Zhaopin.com
Zhaopin.com is the biggest online recruitment service
provider in China
Zhaopin.com provides job seekers a comprehensive resume service, latest
deployment, and career development related information, as well as in-depth
online job search for positions throughout China.
Zhaopin.com provides professional HR services to over 2.2 million clients and its
average daily pageviews are over 68 millions.
3 .Who we are
❏ Penghui Li
❏ Tech lead of Infrastructure
team at Zhaopin
❏ 5+ years of experiences in
messaging and microservices
❏ Apache Pulsar Committer
4 .Who we are
❏ Jia Zhai
❏ Pulsar PMC Member / Committer
❏ BookKeeper PMC Member /
Committer
❏ Funding engineer at
StreamNative
5 .Agenda
❏ Why building an Event Center
❏ Why Apache Pulsar
❏ Apache Pulsar at Zhaopin
❏ Streaming Platform
❏ Zhaopin’s contributions to Apache Pulsar
6 .Why building an Event Center
Data Silos -> Unified Platform
7 .Data Silos
Pain Points
To Enterprises Data Processing
❏ High Maintenance Cost
MSMQ Kafka
❏ Extremely hard to scale data cross
teams
❏ Inconsistency between data silos
❏ Doesn’t scale To End Users
❏ No consistent SLA
RabbitMQ
8 .Data Silos
Pain Points
To Enterprises Data Processing
❏ High Maintenance Cost
MSMQ Kafka
❏ Extremely hard to scale data cross
teams
❏ Inconsistency between data silos
❏ Doesn’t scale To End Users
❏ No consistent SLA
RabbitMQ
9 .Unification - MQService
Problem Solved
❏ Simplified Operations
❏ Scale-out Service
❏ High Availability
Problem Unsolved
❏ Keep messages for longer period
❏ Data rewind
❏ Order guarantee
10 .Unification - MQService
Online Services Data Processing
MQService Kafka
11 .Why building an Event Center
12 .Why building an Event Center
13 .Why building an Event Center
14 .Why building an Event Center
15 .Why Apache Pulsar
Pulsar == Messaging + Storage
16 .Why Apache Pulsar?
Flexible Pub/Sub Messaging
backed by scalable log storage
17 .Why Apache Pulsar / Multi Tenancy
18 .Why Apache Pulsar / Queuing + Streaming
19 .Why Apache Pulsar / Cloud Native Architecture
21 . Apache Pulsar at Zhaopin
20+ core services, 20 billions events/day
22 .Unification - MQService
Problem Solved
❏ No Data Silos
❏ Queue + Streaming
❏ Disaster Recovery
❏ Infinite Stream Storage
(via Tiered Storage)
❏ Data rewind
24 .Core Metrics
❏ 50+ Namespaces
❏ 5000+ Topics
❏ 20+ billions events/day
❏ 5TB storage per day
❏ 20+ core services
26 .Pulsar at Zhaopin
❏ One copy of data, single source-of-truth
❏ Don’t worry about data consistency between RabbitMQ and
Kafka
❏ Multi-tenancy makes topic management easier
❏ Strong data durability allows us to stop worrying about
message loss
27 .Event Streaming Platform
Beyond Pub/Sub Messaging
28 .Event Streaming Platform
29 .Event Streaming Platform
❏ Pulsar Functions: lightweight computing
❏ Flink: streaming-first, unified data processing
❏ Pulsar SQL (presto): interactive queries on both historic and
real-time data
热门城市
