展开查看详情
1 .Apache Pulsar
in 2021
翟佳
2 .Pulsar 之前
streamnative.io
3 .架构 Pulsar 的坚实的基础
云原⽣架构
• 存储和计算分离
• 节点对等
• 独⽴扩展
• 灵活扩容
• 快速容错
streamnative.io
4 . 企业级流存储层： Apache BookKeeper
• 逻辑分区
• 分区分⽚
分布式⽇志/流存储
• Size&Time
• 低延时、⾼吞吐、持久化
• 存储均衡
• 强⼀致
• ⾼可⽤
• I/O隔离
streamnative.io
5 .基础之上：企业级特性：⼤集群、多租户
streamnative.io
6 .基础之上：跨地域复制
Beijing Shanghai
Producer Producer
(P1) Topic (T1) Topic (T1) (P2)
Consumer Subscrip/on (S1) Subscrip/on (S1) Consumer
(C1) (C2)
Broker原⽣
Pub/Sub⼀体
管理简单
Producer Topic (T1)
(P3)
配置灵活
Guangzhou
streamnative.io
7 .Pulsar github star
streamnative.io
8 .国内第⼀场 Meetup： Pulsar Beijing Meetup
streamnative.io
9 .第⼀场 Pulsar Asia Meetup
streamnative.io
10 .Pulsar 在亚洲的旅程
2019年8⽉:
First Pulsar Asia Meetup
2020年11⽉:
First Pulsar Summit Asia
2018年9⽉:
第⼀场 Beijing Meetup 2020年6⽉:
First Pulsar Summit NA
streamnative.io
11 .Asia Adoption @ 2020
streamnative.io
12 .所属⾏业
Internet E-commerce Retail
Finance Education Security
Telecom IoT Live Streaming
streamnative.io
13 .Pulsar in 2021
streamnative.io
14 .Pulsar as a converged messaging solution
Pulsar Clients
Kafka Clients AMQP Clients MQTT Clients
(queue + stream)
AMQP Protocol
Pulsar Protocol Handler Kafka Protocol Handler MQTT Protocol Handler
Handler
Apache Pulsar
streamnative.io
15 .Kafka on Pulsar (KoP)
• Broker 插件，Client ⽆须改动
• 共享访问
• ⽀持Kafka 0.10—2.x 版本
• Pulsar 优势
改进：
• 连续Offset： PIP 70
• Performance：
协议解析 broker=> client
• Envoy/Schema
https://streamnative.io/en/blog/tech/2020-03-24-bring-native-kafka-protocol-support-to-apache-pulsar streamnative.io
16 .AMQP on Pulsar（AoP）
• 改进： AMQP-1.0
streamnative.io
https://streamnative.io/en/blog/tech/2020-06-15-announcing-aop-on-pulsar
17 .MQTT on Pulsar（MoP）
• 改进： QoS level 2
streamnative.io
https://streamnative.io/en/blog/tech/2020-09-28-announcing-mqtt-on-pulsar
18 . Delayed Message
Delayed index memory limitation
Rebuilding delayed index
https://github.com/apache/pulsar/wiki/PIP-26%3A-Delayed-Message-Delivery streamnative.io
19 .ReadOnly Broker
• 唯⼀ writer
• 多 Read Broker
https://github.com/apache/pulsar/wiki/PIP-63%3A-Readonly-Topic-Ownership-Support streamnative.io
20 .Transaction
https://github.com/apache/pulsar/wiki/PIP-31%3A-Transaction-Support streamnative.io
21 . Rest Protocol Handler
POST /topics/(string: tenant_name)/(string: namespace_name)/(string: topic_name)
GET /topics/(string: tenant_name)/(string: namespace_name)/(string: topic_name)/partitions/(int:
partition_id)/messages
POST /topics/(string: tenant_name)/(string: namespace_name)/(string: topic_name)/subscription/
(string: subscription)
GET /topics/(string: tenant_name)/(string: namespace_name)/(string: topic_name)/subscription/
(string: subscription)/cursor
https://github.com/apache/pulsar/wiki/PIP-64%3A-Introduce-REST-endpoints-for-producing%2C-consuming-and-reading-messagesstreamnative.io
22 .Pulsar with No ZooKeeper
➔ PIP 45: Pluggable metadata interface
➔ BP-29: Metadata API module
➔ BP-28: use etcd as metadata store
➔ BP-30: BookKeeper Table Service
https://github.com/apache/pulsar/wiki/PIP-45%3A-Pluggable-metadata-interface streamnative.io
23 .Pulsar as unified batch and streaming platform
Segment
Readers
Producer
Segmented
Segment0 Segment1 Segment2 Segment3 Segment4 Segment 5
Stream
Consumer
Time
➔ Streaming Offloader
➔ Columnar Offloader
➔ Pulsar + Flink work Unify
streamnative.io
24 .Pulsar Functions
➔ Function Mesh < == 欢迎申请试⽤
➔ WebAssembly rust runtime
Function Mesh 介绍： https://www.bilibili.com/s/video/BV1Ly4y1S7RL streamnative.io
25 .StreamNative Cloud
➔ GCP、AWS、Azure
➔ Alibaba、Tencent
streamnative.io
26 .社区活动
➔ TGIP
➔ Meetup
➔ Contributor Club
➔ Pulsar Summit
➔ Pulsar Summit Asia
➔ Apache Con Asia
➔ Apache Con
streamnative.io
27 . 社区资源
• 微信公众号:
• ApachePulsar / StreamNative
• B站：https://space.bilibili.com/391380821
• 邮件列表
• dev@pulsar.apache.org
• users@pulsar.apache.org
• Slack
• https://apache-pulsar.slack.com (#china)
• register:
• https://apache-pulsar.herokuapp.com/
• Github
• https://github.com/apache/pulsar
• https://github.com/apache/bookkeeper
• https://github.com/streamnative
streamnative.io
热门城市
