展开查看详情
1 .Serverless Event Streaming
with Pulsar Functions
Sijie Guo (@sijieg)
2019.06.22
2 . Who am I
● Apache Pulsar PMC Member
● Apache BookKeeper PMC Chair
● Twitter, Yahoo Alumni
● Founder of StreamNative
● Interested in technologies around Event Streaming
3 . Agenda
● What is Apache Pulsar?
● Event Stream - Pulsar view on Data
● When Event Streaming meets serverless
○ Programming Model
○ Architecture
○ Use cases
4 .What is Apache Pulsar?
5 .What is Apache Pulsar?
“Flexible pub/sub messaging
backed by a durable stream storage”
7 .Pulsar - Multi Tenancy
8 .Pulsar - Flexible Messaging
9 . Pulsar - Flexible Messaging
● One data, different ways to consume
● Queuing (aka stateless messaging)
○ Shared (* RabbitMQ)
● Streaming (aka stateful messaging)
○ Exclusive
○ Failover (* Kafka)
○ Key_Shared
10 .Pulsar - Cloud Native Architecture
Layered Architecture
❏ Independent scalability
❏ Instance failure recovery
❏ Balance-free on cluster
expansions
11 .A Pulsar view on Data
13 .Pulsar View - Partition
14 .Pulsar View - Segment
15 .Pulsar View - Event Stream
16 .Event Stream is the right foundation for your data
17 .M(essaging), S(torage), P(rocessing)
18 .MSP - Interactive Queries
19 .MSP - Stream & Batch Processing
20 .MSP - What is next?
What is next?
21 .When Event Streaming
meets Serverless
22 .Introduce Pulsar Functions
23 . Pulsar Functions
● A serverless event streaming framework
● Lightweight computation
● Event-first, Stream-first
● Multi languages
● Multi runtimes
● SDK-less & SDK
24 . Function Elements
● Input Topics
● Output Topics
● Function
● State
● Log Topics
25 .API - Native Java / Python / Go Function
Java
Python
Golang
26 . API - Function Context
● Logger
● State
● Metrics
● Security / Secrets
● ...
29 . Context - State
● Global Managed State *
● Mutable by functions and admin-cli
● Queryable by functions and admin-cli
● State are stored at storage layer
● State are implemented using streams + snapshot
