展开查看详情
1 . StreamNative
TGIP 019
Interactive Querying Of Streams Using Apache Pulsar
Gaoran 2020.8.2
2 .Community
PR 7646 - Support Set Backlog Quota On Topic Level
Http Rest API
- GET: {tenant}/{namespace}/{topic}/backlogQuotaMap
- POST: {tenant}/{namespace}/{topic}/backlogQuota
- DELETE: {tenant}/{namespace}/{topic}/backlogQuota
- PIP 39 - Namespace Change Events
- PR 4955 - Introduce system topic and topic policies service
3 .Community
PR 7647 - Allow Ability To Specify Retain Key Ordering In Functions
- FunctionCon ig: retainKeyOrdering
f
4 .Community
PR 7655 - Fix Backward Compatibility Issues
With Batch Index Acknowledgment
- Consumer: batchIndexAckEnabled
- Broker: acknowledgmentAtBatchIndexLevelEnabled
5 .Pulsar SQL
Presto
Distributed SQL Query Engine for Big Data
What Presto Is Not ？
What Presto Is ?
6 .Pulsar SQL
Pulsar SQL Based On Presto
- PulsarConnector -> Connector
- PulsarSplitManager -> ConnectorSplitManager
- PulsarSplit -> ConnectorSplit
- PulsarRecordCursor -> RecordCursor
- PulsarMetadata -> ConnectorMetadata
- PulsarColumnMetadata -> ColumnMetadata
7 .Pulsar SQL
Namespace Retention
- bin/pulsar-admin namespaces set-retention --size 100M --time 7d public/default
- bin/pulsar-admin namespaces get-retention public/default
8 .Pulsar SQL
Produce Messages To Pulsar
- Primitive Type: BOOLEAN, INT16, DOUBLE, STRING, etc.
- Complex Type: KeyValue, Struct ( Avro, Json)
10 .Pulsar SQL
1. Use JsonSchema
2. Set batching max messages
3. Set batching max publish delay
4. Set compressionType
5. Send in async way
12 .Pulsar SQL
bin/pulsar sql
13 .Pulsar SQL
Presto JDBC
14 .Pulsar SQL
Presto Http API
http://ip:port/v1/statement
http header: X Presto-User
-
15 .Pulsar SQL
Metabase
Metabase is the easy, open source way
for everyone in your company to
ask questions and learn from data.
20 .Pulsar SQL
Query Performance Optimized
21 .Pulsar SQL
Add Topic Partition Number
22 .Pulsar SQL
Use Meta Property __publish_time__
23 .Pulsar SQL
How Pulsar SQL Use __publish_time__
isLowerUnbounded = true
24 . Pulsar SQL
Pulsar SQL Con ig
${PULSAR_HOME}/conf/presto/con ig.properties
coordinator=true
node-scheduler.include-coordinator=true
${PULSAR_HOME}/conf/presto/catalog/pulsar.properties
Of load related con igurations
pulsar.target-num-splits
pulsar.bookkeeper-throttle-value = 0
f
f
f
f
25 .Pulsar SQL
localhost:8081
26 .Pulsar SQL
New Feature
Pulsar SQL Read Compaction Data
热门城市
