深入理解 Pulsar 跨地域复制:GEO-Replication
展开查看详情
1.Geo-Replication 翟佳
2.⾃自我介绍 • 开源项⽬目爱好者: • Pulsar, BookKeeper, DistributedLog的Committer & PMC成员 • EMC -> StreamNative • StreamNative Core Engineer • 华中科⼤大 -> 中科院计算所
3.• Pulsar 简介 • Pulsar Geo-Replication概览 • ⼯工作原理理 • 管理理和实践 • ⼯工作模式
4.Apache Pulsar是什什么 2003 2010 2012 2006 2011
5. Apache Pulsar是什什么 “Flexible Pub/Sub messaging backed by durable log/stream storage”
6.Pulsar统⼀一模型 队列列 + 流
7.存储和服务分离 分层架构: Brokers & Bookies • 独⽴立扩展 • 灵活容错 • 快速扩容
8.企业级特性
9.• Pulsar 简介 • Pulsar Geo-Replication概览 • ⼯工作原理理 • 管理理和实践 • ⼯工作模式
10.概念:同步/异步
11.Pulsar集群 ZooKeeper
12.Pulsar多集群 ConfigStore ZooKeeper Brokers Brokers Brokers ZooKeeper ZooKeeper ZooKeeper BookKeeper BookKeeper BookKeeper
13.Pulsar的Geo-replication Beijing Shanghai Producer Producer (P1) Topic (T1) Topic (T1) (P2) Consumer Subscrip/on (S1) Subscrip/on (S1) Consumer (C1) (C2) Broker原⽣生 Producer Topic (T1) Pub/Sub⼀一体 (P3) 管理理简单 Guangzhou 配置灵活
14.• Pulsar 简介 • Pulsar Geo-Replication概览 • ⼯工作原理理 • 管理理和实践 • ⼯工作模式
15.• ⼯工作原理理 • 数据通路路 • 元数据
16.原理理 Beijing Shanghai Producer Producer (P1) Topic (T1) Topic (T1) (P2) Consumer Subscrip/on (S1) Subscrip/on (S1) Consumer (C1) (C2) Producer Topic (T1) (P3) Guangzhou
17.Topic
18.Producer, Consumer Producer Consumer
19.订阅
20.订阅 — Cursor Broker Subscription1 Producer Topic … Write Ack Subscription Consumer readEntries ManagedLedger + Cursor Ack/N-Ack 1 2 3 4 5 6 7 1 2 3 4 5 6 7 -/+ BookKeeper 1 2 3 4 5 6 7 A B C D E Ledger stores Data Ledger stores Cursor changes
21.机房间数据通路路 Beijing Replicator 1 Producer Topic1 2 Cursor 1 2 3 4 5 6 7 3 Consumer Subscription Producer 4.1 4 Replicator Cursor Topic1 Producer 1 2 3 4 5 6 7 5 6 Consumer Subscription Producer Shanghai
22.• ⼯工作原理理 • 数据通路路 • 元数据
23.ZooKeeper ZooKeeper ConfigStore Beijing Shanghai Guangzhou Brokers Brokers Brokers ZooKeeper ZooKeeper ZooKeeper BookKeeper BookKeeper BookKeeper
24.Configuration Storage
25.ZooKeeper
26.多租户
27.原理理⼩小结 Beijing Shanghai Producer Producer (P1) Topic (T1) Topic (T1) (P2) Consumer Subscrip/on (S1) Subscrip/on (S1) Consumer (C1) (C2) Producer Topic (T1) (P3) Guangzhou
28.• 跨地域复制 • 管理理,backlog, dispatch
29.• Pulsar 简介 • Pulsar Geo-Replication概览 • ⼯工作原理理 • 管理理和实践 • ⼯工作模式