- 快召唤伙伴们来围观吧
- 微博 QQ QQ空间 贴吧
- 文档嵌入链接
- 复制
- 微信扫一扫分享
- 已成功复制到剪贴板
TGIP-001:Pulsar Basics(Sijie G)
第一期 TGIP 演讲文件
展开查看详情
1 .Pulsar Concepts TGIP-CN Episode 001
2 .Pulsar with legos ❏ We will now demonstrate how Pulsar works with legos ❏ Concepts are shown ❏ Publish / Subscribe ❏ Topics ❏ Partitions ❏ Brokers ❏ Producers / Consumers / Readers
3 .Producers Producer 0 Producer 1 Producers output the Producers send all events events to Pulsar. What as key/value pairs. each producer does is Pulsar can make a topic use case specific. conform to a schema.
4 .Topics Producer 0 Producer 1 Topic Partition 0 Partition 1 Partition 2 Data is divided into All events are sent and partitions. Partitions received on topics. are both logical and Topic is the logic unit. physical divisions.
5 .Brokers Producer 0 Producer 1 Topic Partition 0 Partition 1 Partition 2 Broker X Broker Y Broker Z The brokers are responsible for receiving and dispatching events. Data is stored elsewhere
6 . Producer 0 Producer 1 Topic Partition 0 Partition 1 Partition 2 Broker X Broker Y Broker Z A consumer with an exclusive Consumer (P012) subscription receives events from all topic partitions it subscribes to, and connects to broker X, Y, Z Subscription A
7 .Subscription Producer 0 Producer 1 Modes Topic Partition 0 Partition 1 Partition 2 Broker X Broker Y Broker Z Consumer (P012) Consumer (P012) Consumer (P012) Consumer (P) Consumer (P012) Consumer (P012) Failover Subscription Shared Subscription Key_Shared Subscription In failover, all partitions are In shared, messages are In key_shared, messages with consumed by one consumer and sent in a round robin the same key are consistently will fail over to hot spare on fail. way to all consumers. routed to the same consumer.
8 .Topic (Partition) Event streams are append- only sequences of data. Streams can be truncated for space. ❏ Infinite Event Stream Event Stream Segment 1 Segment 2 Segment 3 Streams are made up of individual segments. New segments are Entry 1 Entry 2 Entry 3 Entry 4 rolled when events are appended to streams Msg 1 Msg 2 Msg 3 Msg 4 Segments are made up of individual entries. This is how pieces of data are stored. Entries are made up of individual messages. This is how Pulsar messages are stored.
9 .Segment - Apache BookKeeper Event Stream Segment 1 Segment 2 Segment 3 Entry 1 Entry 2 Entry 3 Entry 4 Msg 1 Msg 2 Msg 3 Msg 4
10 .Message ❏ Data*: bytes ❏ Key: string, optional ❏ Publish Time: long, timestamp generated by the producer ❏ Event Time: long, specified by the producer application ❏ Properties: Map<String, String>, a key/value property map specified by applications ❏ MessageId: the position of a message within a topic ❏ SchemaVersion: schema version of the message
11 .Message ID Topic Partition 1 Partition 2 Partition 0 Segment 1 Segment 2 Segment 3 Entry 1 Entry 2 Entry 3 Entry 4 Msg 1 Msg 2 Msg 3 Msg 4 ledger-id, entry-id, batch-index, partition-index
12 .Message ID Examples ❏ 100, 10, 2, 0: ptn 0, segment 100, entry 10, batch 2. ❏ 100, 10, -1, 0 ❏ 100, 10, -1, -1 ❏ 100, 10, 0, -1 ❏ ...
13 .Cursor: Subscription State Subscription B (2, 2) Subscription C (3, 2) Subscription A (1, 1) Partition (Event Stream)
14 .Acknowledge Cumulatively Subscription B (2, 2) acknowledgeCumulative (4, 3) Subscription C (3, 2) Subscription A (1, 1) Partition (Event Stream)
15 .Acknowledge Cumulatively Subscription B (2, 2) acknowledgeCumulative (4, 3) Subscription C (3, 2) Subscription A Subscription A (1, 1) (4, 3) Partition (Event Stream)
16 .Acknowledge moves cursors Subscription B (2, 2) acknowledgeCumulative (4, 3) Subscription C (3, 2) Subscription A (4, 3) Partition (Event Stream)
17 .Seek / Reset-Cursor Reset Cursor Subscription B (1, 1) (2, 2) Subscription C (3, 2) Subscription A (4, 3) Partition (Event Stream)
18 .Seek / Reset-Cursor Reset Cursor Subscription B (1, 1) (2, 2) Subscription C (3, 2) Subscription A Subscription A (1, 1) (4, 3) Partition (Event Stream)
19 .Seek / Reset-Cursor Reset Cursor Subscription B (1, 1)c (2, 2) Subscription C (3, 2) Subscription A (1, 1) Partition (Event Stream)
20 .Reader: Non-durable Cursor Subscription B (2, 2) Subscription A Subscription C (1, 1) (3, 2) Partition (Event Stream) Non-durable Exclusive Subscription (1, 100) Reader (1, 100)
21 .Reader: Non-durable Cursor Subscription B (2, 2) Subscription A Subscription C (1, 1) (3, 2) Partition (Event Stream) Close Reader will trigger delete non-durable subscription at broker side Non-durable Exclusive Subscription (1, 100) Reader (1, 100)
22 .Tenant & Namespace persistent :// my-tenant / my-namespace / my-topic Topic Domain Tenant Namespace Topic
23 .Fully Qualified Name ❏ persistent://my-tenant/my-namespace/my-topic ❏ p://my-tenant/my-namespace/my-topic ❏ my-tenant/my-namespace/my-topic ❏ my-topic => persistent://public/default/my-topic