展开查看详情
1 .Serverless Event Streaming
with Pulsar Functions
Xiaolong Ran (@wolf4j1)
09/12/2019
2 . Who am I
● Go programmer
● PingCAP > Bitmain > StreamNative
● Pulsar committer
○ pulsar-client-go
○ Go Functions
○ …
● https://github.com/wolfstudy
3 . Agenda
● What is Apache Pulsar?
● Event Stream - Pulsar view on Data
● When Event Streaming meets serverless
○ Programming Model
○ Architecture
○ Use cases
4 . Agenda
● What is Apache Pulsar?
● Event Stream - Pulsar view on Data
● When Event Streaming meets serverless
○ Programming Model
○ Architecture
○ Use cases
7 .Pulsar View - Partition
9 .Pulsar View - Event Stream
10 .Event Stream is the right foundation for your data
11 .M(essaging), S(torage), P(rocessing)
12 .MSP - Interactive Queries
13 .MSP - Stream & Batch Processing
14 .MSP - What is next?
What is next?
15 .When Event Streaming
meets Serverless
17 .Introduce Pulsar Functions
18 . Pulsar Functions
● A serverless framework for data processing
● Lightweight computation
● Event-first, Stream-first
● Support both stateless and stateful computation
● Multi languages
● Multi runtimes
● SDK-less & SDK
19 . Function Elements
● Input Topics
● Output Topics
● Function
● State
● Log Topics
20 .API - Native Java / Python / Go Function
Java
Python
Golang
21 . API - Function Context
● Logger
● State
● Metrics
● Security / Secrets
● ...
24 . Context - State
● Global Managed State *
● Mutable by functions and admin-cli
● Queryable by functions and admin-cli
● State are stored at storage layer
● State are implemented using streams + snapshot
25 . Context - State API
● Key/Value State API
○ putState
○ getState
● Counter State API
○ getCounter
○ incrCounter
26 . Context - Metrics
● API - recordMetric(String metricName, double value)
● Exposed in prometheus format
● Collected by prometheus
27 . Flexible Runtime
● Colocate with Broker - Thread & Process
● Managed Function Workers - Thread & Process
● External Schedulers - Container
○ Kubernetes
28 .Colocate with Brokers
29 .Managed Function Workers
热门城市
