TGIP_CN_025:Apache Pulsar 问答专场

本周直播,我们收集了近一个月内 Pulsar 交流群中用户的提问(已解答+未解答的问题),通过直播的方式进行梳理和回答,希望可以将用户心中的疑惑进行全面的答疑解惑。

展开查看详情

1.TGIP-CN 025 Pulsar Q&A Penghui Li & Xiaolong Ran

2.Features and bug fixes A. Add ability to specify the subscription init position for functions #789 B. Fix IndexOutOfBoundsException in the Key_Shared subscription when dispatch batch message to consumers #802 C. Support set max consumers per subscription on the topic level #800 D. Support set max consumers on the topic level #796 E. Support set message key in pulsar-perf #798 F. Support key based message batching in the CPP Client #799 G. Transaction message ack abort on subscriptions #7979 4 9 8 6 3 1

3.Data deletion in Pulsar Topic Ledger - 2 (open) Ledger - 1 (closed) Ledger - 0 (closed) Fragment F - 1 F - 0 Fragment E-3 E-2 E-1 E-0 E-3 E-2 E-1 E-0 E-3 E-2 E-1 E-0

4.Data deletion in Pulsar Current active in the topic Removed from the topic Ledger - 2 (open) LedgerTopic - 1 (closed) Ledger - 0 (closed) Not removed from the topic

5.Data deletion in Pulsar

6.Data deletion in Pulsar

7.Data deletion in Pulsar L-0 E-0 L-1 E-0 L-2 E-0 L-0 E-0 L-0 E-0 L-1 E-0 L-1 E-1 Checkpoint L-1 E-0 L-1 E-1 L-1 E-2 L-1 E-1 L-2 E-0 L-0 E-1 L-2 E-0 Journal Entry Log - 0 Index DB Entry Log - 1

8.Data deletion in Pulsar minorCompactionThreshold=0. majorCompactionThreshold=0. minorCompactionInterval=360 majorCompactionInterval=86400 0 5 2

9.Question 1 最近刚刚接触pulsar项⽬,之前⼀直关注confluence的kafka项⽬,对pulsar很感兴趣。⽤ pulsar对数据做永久保存怎么样,是否是个好主意?我看到对接presto时候是直接去bookie⾥⾯ 取历史数据,那么当做批处理的存储层也可以吧,会对broker的消息吞吐产⽣影响么?

10.Question 2 最请问有没有pulsar性能压测的指导,不知道咋测试pulsar的性能,求⼤佬指导指导,谢谢! https://github.com/apache/pulsar/tree/master/pulsar-testclient https://github.com/openmessaging/openmessaging-benchmark

11.Question 3 通过bookkeeper shell metaformat 操作后,是不是整个bookie集群的元数据都格式化了? ⽽不是单个bookie节点。然后broker消费不了数据(⽣产没问题),异常ERROR org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleC onsumers - [persistent://public/default/topic_aep_order / aep_order_consumer_group] Error reading entries at 110260:190 : No such ledger exists, Read Type Normal - Retrying to read in 55.281 seconds。请问 这种情况怎么快速恢复?topic⽐较多

12.Question 4 Pulsar 如何保证 Bookie 节点的负载均衡呢 ⽐如原来有三个 Bookie 节点, 现新加了⼀个 bookie节点, 在写⼊数据时会提⾼新加 Bookie 节点的选中率来实现均衡吗

13.Question 5 要验证pulsar⽀持百万级topic数量,请问各位有什么建议吗?需要多少个租户/名称空间?创建 百万topic后,如何判断集群的稳定性?

14.Question 6 ⼤家有尝试过在function中动态创建topic,并根据不同参数指定不同分区数的经验么?除了⽤ pulsar-admin,还有其他便利的⽅式么?现在在function中有啥姿势可以尝试?之前我再 issue上提了,但回复是⽤pulsar-admin接⼝。觉得有点重,不知道有没有更便捷的⽅式?

15.Question 7 请问下,新增了⼏个functionWorker 节点,怎么让任务做⼀次 rebalance ? https://github.com/apache/pulsar/pull/7388 https://github.com/apache/pulsar/pull/7449

16.扫码⼊群! Pulsar 中国社区等你加⼊ 添加好友后,回复“进群”即可

17.Thanks

StreamNative 是一家围绕 Apache Pulsar 和 Apache BookKeeper 打造下一代流数据平台的开源基础软件公司。秉承 Event Streaming 是大数据的未来基石、开源是基础软件的未来这两个理念,专注于开源生态和社区的构建,致力于前沿技术。