热门城市
- 快召唤伙伴们来围观吧
- 微博 QQ QQ空间 贴吧
- 视频嵌入链接 文档嵌入链接
- 复制
- 微信扫一扫分享
- 已成功复制到剪贴板
TGIP-CN 028: Apache Pulsar 延时队列的设计与实现
本周分享简介
本周将由 StreamNative 李鹏辉为大家带来「Apache Pulsar 延时队列的设计与实现」主题分享，大家可以在直播时留言和互动。
关于 TGIP
TGIP 就是 Thank God It’s Pulsar 的缩写，灵感来自于谷歌团队的 Thank God It’s Friday，表示工作周快结束了，很快又会有一个欢快的周末。TGIP 主要是为了向大家科普关于 Pulsar 的一些基础概念，不管是它的架构模式，还是消息之间的传递动态等，你都可以从 TGIP 的直播中获取到。TGIP 的直播通常会安排在周日，不会占用大家太多的时间。利用周末来了解一下 Apache Pulsar，岂不快哉？
关于 Apache Pulsar
Apache Pulsar 是 Apache 软件基金会顶级项目，是下一代云原生分布式消息流平台，集消息、存储、轻量化函数式计算为一体，采用计算与存储分离架构设计，支持多租户、持久化存储、多机房跨区域数据复制，具有强一致性、高吞吐、低延时及高可扩展性等流数据存储特性。
GitHub 地址：http://github.com/apache/pulsar/
关于组织方 StreamNative
StreamNative 是一家开源基础软件公司，由 Apache 顶级项目 Apache Pulsar 、Apache BookKeeper 原生核心开发者组建而成，围绕 Apache Pulsar 及 Apache BookKeeper 打造下一代流数据平台。SteamNative 专注于开源生态和社区构建，致力于前沿技术领域的创新，创始团队成员曾就职于 Yahoo、Twitter、EMC 等知名大公司。
公司创建于 2019 年，创立初期即获得红杉中国种子基金天使轮投资，在近期已完成数百万美元的 Pre-A 轮融资，本轮融资由源码资本领投、红杉中国种子基金跟投。
展开查看详情
1 .TGIP-CN - 28 Delayed Message Delivery Penghui Li 2020/12/13 streamnative.io
2 .Community 1. 2.6.3 Coming soon 2. PIP 70 Lightweight broker entry metadata 3. PIP 73 Configurable data source priority for message reading streamnative.io
3 .Delayed Message Delivery 4 3 2 1 0 4 3 2 1 0 4 3 2 1 0 1 4 2 3 0 streamnative.io
4 .Delayed Message Delivery 4 3 2 1 0 4 3 2 1 0 4 3 2 1 0 1 4 2 3 0 streamnative.io
5 .Delayed Message Delivery 4 3 2 1 0 4 3 2 1 0 4 3 2 1 0 1 4 2 3 0 streamnative.io
6 .Delayed Message Delivery 4 3 2 1 0 4 3 2 1 0 4 3 2 1 0 1 4 2 3 0 streamnative.io
7 .Delayed Message Delivery 4 3 2 1 0 4 3 2 1 0 4 3 2 1 0 1 4 2 3 0 streamnative.io
8 .Delayed Message Delivery 4 3 2 1 0 4 3 2 1 0 4 3 2 1 0 1 4 2 3 0 streamnative.io
9 .Delayed Message Delivery Different delay granularity on a partition Delayed messages and Non-delayed messages in a partition streamnative.io
10 .Usage producer.newMessage() .deliverAt(long timestamp) .sendAsync(); producer.newMessage() .deliverAfter(long delay, TimeUnit unit) .sendAsync(); streamnative.io
11 . M0 streamnative.io M1 | Delay 5 min M2 How it works? M3 | Delay 2 min M4 | Delay 10 min M5 M6 M7 M8 | Delay 1 min M9
12 . streamnative.io M0 M0 M1 | Delay 5 min M2 How it works? M3 | Delay 2 min M4 | Delay 10 min M5 M6 M7 M8 | Delay 1 min M9
13 .How it works? Delayed Message Tracker M1 Timestamp | LedgerId | EntryId M9 M8 | Delay 1 min M7 M6 M5 M4 | Delay 10 min M3 | Delay 2 min M2 M1 | Delay 5 min M0 M0 streamnative.io
14 . streamnative.io M0 M2 Timestamp | LedgerId | EntryId M5 Delayed Message Tracker M6 M7 M8 M0 M3 M1 | Delay 5 min M1 M2 M4 How it works? M3 | Delay 2 min M4 | Delay 10 min M5 M6 M7 M8 | Delay 1 min M9
15 . streamnative.io M0 M2 Timestamp | LedgerId | EntryId M5 Delayed Message Tracker M6 M7 M8 M9 M3 M0 M1 M1 | Delay 5 min M4 M2 How it works? M3 | Delay 2 min M4 | Delay 10 min M5 M6 M7 M8 | Delay 1 min M9
16 .Disadvantages Delayed index memory limitation Rebuilding delayed index The index only available for a subscription streamnative.io
17 .PIP 26 Hierarchical Timing Wheels Time Partition Time Partition Time Partition Time Partition Time Partition Tailer ... 2 1 0 http://www.cs.columbia.edu/~nahum/w6998/papers/sosp87-timing-wheels.pdf https://blog.acolyer.org/2015/11/23/hashed-and-hierarchical-timing-wheels/ streamnative.io
18 . streamnative.io M0 Build delayed index M0 M1 | Delay 5 min M2 Subscription How it works? M3 | Delay 2 min M4 | Delay 10 min M5 M6 M7 M8 | Delay 1 min M9
19 . streamnative.io M0 M2 Time Partition 0 M1 M3 M0 M1 | Delay 5 min M2 How it works? M3 | Delay 2 min M4 | Delay 10 min M5 M6 M7 M8 | Delay 1 min M9
20 . streamnative.io M0 M2 Time Partition 0 M1 M3 Time Partition 1 M4 M0 M1 | Delay 5 min M2 How it works? M3 | Delay 2 min M4 | Delay 10 min M5 M6 M7 M8 | Delay 1 min M9
21 . streamnative.io M0 M2 M5 M6 Time Partition 0 M7 M1 M3 M8 Time Partition 1 M4 M0 M1 | Delay 5 min M2 How it works? M3 | Delay 2 min M4 | Delay 10 min M5 M6 M7 M8 | Delay 1 min M9
22 . streamnative.io M0 M2 M5 M6 Time Partition 0 M7 M1 M3 M8 M8 M9 Time Partition 1 M4 M0 M1 | Delay 5 min M2 How it works? M3 | Delay 2 min M4 | Delay 10 min M5 M6 M7 M8 | Delay 1 min M9
23 . streamnative.io M0 M2 M5 M6 M7 M8 M9 M3 Time Partition 1 M4 M1 M0 M1 | Delay 5 min M2 How it works? M3 | Delay 2 min M4 | Delay 10 min M5 M6 M7 M8 | Delay 1 min M9
24 .Advantages Load delayed indexes more efficiently Reduce the memory usage for delayed indexes Avoid rebuilding indexed by replaying too many entries streamnative.io
25 .Challenges Data cleaning Safe position for subscription to start read Too much individual acks introduced much memory overhead streamnative.io
26 .Wechat Apache Pulsar / StreamNative Bilibili https://space.bilibili.com/391380821 Email dev@pulsar.apache.org users@pulsar.apache.org Slack https://apache-pulsar.slack.com (#general / #china) register: https://apache-pulsar.herokuapp.com/ Github https://github.com/apache/pulsar https://github.com/apache/bookkeeper https://github.com/streamnative streamnative.io
27 . Thank You streamnative.io
3秒后跳转登录页面
去登陆
