StreamNative 是一家围绕 Apache Pulsar 和 Apache BookKeeper 打造下一代流数据平台的开源基础软件公司。秉承 Event Streaming 是大数据的未来基石、开源是基础软件的未来这两个理念，专注于开源生态和社区的构建，致力于前沿技术。
确定删除吗？
1.Pulsar 2.3.0 @sijieg
2. Agenda • Bookie & Broker • Schema • Security • Functions • Connectors • Clients
3.Bookie & Broker
4. Memory Management • Direct Memory & Netty Buffer Pool • Problems: • -Xms2g -Xmx2g -XX:MaxDirectMemorySize=4g • io.netty.util.internal.OutOfDirectMemoryError
5. MM - ByteBufAllocator • ByteBufAllocator • PoolingPolicy • UnpooledHeap, PooledDirect • OutOfMemoryPolicy • ThrowException, FallbackToHeap • LeakDetectionPolicy • Disabled, Simple, Advanced, Paranoid
6.ReadAhead Amplification
7.Sticky Read
8.Sticky Read
9. Performance & Metadata • Performance • CPU-Affinity • JCTools - Lock-free concurrent data structures • Metadata • BookKeeper: Support Etcd as a metadata store • Reverse Index : Ledger => Topic, Subscription, Cursor
10.Schema
11. New Schemas • KeyValue Schema • AUTO_PRODUCE Schema • AUTO_CONSUME Schema
12.Support in new language clients • C++ • Python • JSON • Avro • Primitives: str, bytes
13.Security
14. Before 2.3.0 • TLS • Athenz • Password
15. JWT Authentication • JSON Web Tokens
16.Functions
17. Pulsar Functions in K8S • Demo
18.Python Function State
19.Connectors
20. New Connectors • CDC • Debezium: MySQL, Postgres, MongoDB, and … • Canal: MySQL • MongoDB • HBase • Elastic Search • Local Filesystem
21. New Connectors • CDC • Debezium: MySQL, Postgres, MongoDB, and … • Canal: MySQL • MongoDB Thanks! • HBase • Elastic Search - @tuteng - @ambition119 • Local Filesystem - @murong00
22.Clients
23. Java • Multi-Hosts Service Url • pulsar://host1:6650,host2:6650,host3:6650/ • Auto Partitions Change Discovery • GetPartitionsForTopic • Consumer: pauseMessageListener, resumeMessageListener
24. C++ • Consumer: • SubscriptionInitialPosition • receiveAsync • Producer: Flush • Hostname verifications on TLS connections • Avro Schema
25. Go • Caught up a lot of features in CGo client
26. 2.4.0 and more • PIP-30: Kerberos Authentication • PIP-32: Go Function, More languages to support • PIP-31: Transactional Streaming • System Topic, Replicated Subscription, Key Ordered Dispatching, … • NodeJS Client, Native Go Client, … • More Connectors, Integrations and Larger ecosystem
27. Community ✓ Twitter: @apache_pulsar ✓ Wechat Subscription: ApachePulsar ✓ Mailing Lists dev@pulsar.apache.org, users@pulsar.apache.org ✓ Slack https://apache-pulsar.slack.com (#china) ✓ Localization https://crowdin.com/project/apache-pulsar ✓ Github https://github.com/apache/pulsar https://github.com/apache/bookkeeper
28.
