展开查看详情
1. Fast, Durable, Flexible Pub/Sub based on Segment-Oriented Architecture 演讲者/streamlio 翟佳
2.What is Apache Pulsar? Durability Ordering Delivery Guarantees Data replicated and Guaranteed ordering At least once, at most synced to disk once and effectively once Geo-replication Multi-tenancy Low Latency Out of box support for A single cluster can Low publish latency of geographically support many tenants 5ms at 99pct distributed and use cases applications Unified messaging High throughput Highly scalable model Can reach 1.8 M Can support millions of Support both Topic & messages/s in a topics Queue semantic in a single partition single model !2
3.Architecture
4. Architecture view - Separate layers between brokers bookies 4
5.Brokers
6.Bookies - Apache BookKeeper \ Durable and Consistent - I/O Isola3on ! High Throughput 2 Low Latency !6
7.Bookies - Apache BookKeeper !7
8.Architecture view • Unbounded topic partition storage • Instant scaling without data rebalance • Independent scalability
9.A Compare
10.Benefits
11.Seamless - broker failure
12.Seamless - bookie failure
13.Seamless - cluster expand
14. Conclusion • Unbounded topic partition storage • Instant scaling without data rebalance • Seamless - broker failure recovery • Seamless- bookie failure recovery • Seamless - cluster expansion • Independent scalability
15. Benchmark https://github.com/openmessaging/openmessaging-benchmark
16.Throughput
17.Latency
18.Pulsar Functions
19. Pulsar Functions - Lightweight stream processing - New in Pulsar 2.0 - Currently supports Java and Python Python def process(input): return input.replace(“jia”, “anonymous”) Java import java.util.function.Function; public class Anon implements Function<String,String> { @Override public String apply(String input) { return input.replace(“jia”, “anonymous”); } }
20. Pulsar Functions Input Function Output Topic Topic # pulsar-admin functions create \ —py anon.py --className anon \ --fqfn lc3-tenant/demo/anony \ Python --inputs persistent://lc3-tenant/demo/input \ --output persistent://lc3-tenant/demo/output # pulsar-admin functions create \ —jar anon.jar --className Anon \ --fqfn lc3-tenant/demo/anony \ Java --inputs persistent://lc3-tenant/demo/input \ --output persistent://lc3-tenant/demo/output
21. Curious to Get More • Apache Pulsar : http://pulsar.incubator.apache.org • Apache BookKeeper : http://bookkeeper.apache.org • Technical Blog : https://streaml.io/blog/ • Twitter: @apache_pulsar @asfbookkeeper • slack: • https://apache-pulsar.herokuapp.com/ • https://apachebookkeeper.herokuapp.com/
22.Thanks!