2019-06-22_ Serverless Event Streaming with Pulsar Functions
展开查看详情
1.Serverless Event Streaming with Pulsar Functions Sijie Guo (@sijieg) 2019.06.22
2. Who am I ● Apache Pulsar PMC Member ● Apache BookKeeper PMC Chair ● Twitter, Yahoo Alumni ● Founder of StreamNative ● Interested in technologies around Event Streaming
3. Agenda ● What is Apache Pulsar? ● Event Stream - Pulsar view on Data ● When Event Streaming meets serverless ○ Programming Model ○ Architecture ○ Use cases
4.What is Apache Pulsar?
5.What is Apache Pulsar? “Flexible pub/sub messaging backed by a durable stream storage”
6.Pulsar - Pub/Sub
7.Pulsar - Multi Tenancy
8.Pulsar - Flexible Messaging
9. Pulsar - Flexible Messaging ● One data, different ways to consume ● Queuing (aka stateless messaging) ○ Shared (* RabbitMQ) ● Streaming (aka stateful messaging) ○ Exclusive ○ Failover (* Kafka) ○ Key_Shared
10.Pulsar - Cloud Native Architecture Layered Architecture ❏ Independent scalability ❏ Instance failure recovery ❏ Balance-free on cluster expansions
11.A Pulsar view on Data
12.Pulsar View - Topic
13.Pulsar View - Partition
14.Pulsar View - Segment
15.Pulsar View - Event Stream
16.Event Stream is the right foundation for your data
17.M(essaging), S(torage), P(rocessing)
18.MSP - Interactive Queries
19.MSP - Stream & Batch Processing
20.MSP - What is next? What is next?
21.When Event Streaming meets Serverless
22.Introduce Pulsar Functions
23. Pulsar Functions ● A serverless event streaming framework ● Lightweight computation ● Event-first, Stream-first ● Multi languages ● Multi runtimes ● SDK-less & SDK
24. Function Elements ● Input Topics ● Output Topics ● Function ● State ● Log Topics
25.API - Native Java / Python / Go Function Java Python Golang
26. API - Function Context ● Logger ● State ● Metrics ● Security / Secrets ● ...
27.Context - Logger
28.Context - State
29. Context - State ● Global Managed State * ● Mutable by functions and admin-cli ● Queryable by functions and admin-cli ● State are stored at storage layer ● State are implemented using streams + snapshot