TGIP-CN 017:What's new in Pulsar 2.6.0?

展开查看详情

1.TGIP-CN 017 New features in 2.6.0 Penghui Li

2.1. PIP-68 Exclusive producer 2. 2.6.1 Release

3.[PIP-37] Large message size support

4.[PIP-38] Namespace change events NS-A NS-B NS-C __change_events __change_events __change_events

5.[PIP-38] Namespace change events

6.[PIP-54] Acknowledgement at batch index 3 4 5 0 1 2 Consumer-1

7.[PIP-54] Acknowledgement at batch index 0 1 2 3 4 5 Consumer-1 Redeliver message 0 1 2 0 1 2 3 4 5 Consumer-1

8.[PIP-54] Acknowledgement at batch index AckSet - Bitset AckSet - Bitset Filter out the acked batch index 3 4 5 0 1 2 Consumer-1

9.[PIP-58] Support setting custom retry delay

10.[PIP-61] Advertise multiple addresses Broker-A Connect to owner broker Service url : 172.33.56.32:6650 advertised address : 172.33.56.30:6650 Topic : my-topic Broker-B Producer advertised address : 172.33.56.31:6650 Lookup Owner: 172.33.56.30:6650 Broker-C advertised address : 172.33.56.32:6650

11.[PIP-61] Advertise multiple addresses Broker-A Service url : pulsar://ip:port Connect to owner broker internal:pulsar://ip:port Topic : my-topic external:pulsar://ip:port Listener name : external Broker-B Producer internal:pulsar://ip:port Lookup external:pulsar://ip:port Broker-C Broker-A - external:pulsar://ip:port internal:pulsar://ip:port external:pulsar://ip:port

12.[PIP-61] Advertise multiple addresses # Used to specify multiple advertised listeners for the broker. # The value must format as <listener_name>:pulsar://<host>:<port>, # multiple listeners should separate with commas. # Do not use this configuration with advertisedAddress and brokerServicePort. # The Default value is absent means use advertisedAddress and brokerServicePort. # advertisedListeners= # Used to specify the internal listener name for the broker. # The listener name must contain in the advertisedListeners. # The Default value is absent, the broker uses the first listener as the internal listener. # internalListenerName=

13.[Key_Shared] consistent hashing consumers distribution m1 m3 Consumer 1 Consumer 2 m2

14.[Key_Shared] Fix ordering issue in KeyShared dispatcher when adding consumers Consumer-1 9 8 7 6 5 4 3 2 1 0 Consumer-2

15.[Key_Shared] Fix ordering issue in KeyShared dispatcher when adding consumers 4 3 2 1 0 Consumer-1 9 8 7 6 5

16.[Key_Shared] Fix ordering issue in KeyShared dispatcher when adding consumers 4 3 2 1 0 Consumer-1 Consumer-2 9 8 7 6 5

17.[Key_Shared] Fix ordering issue in KeyShared dispatcher when adding consumers 9 8 7 4 3 2 6 5 1 0

18.[Key_Shared] Fix ordering issue in KeyShared dispatcher when adding consumers 4 3 2 1 0 Mark delete Consumer-1 9 8 7 6 5 4 3 2 1 0 Read position Consumer-2 Consumer-2

19.[Key_Shared] Fix ordering issue in KeyShared dispatcher when adding consumers 4 3 2 1 0 Mark delete Consumer-1 9 8 7 6 5 4 3 2 1 0 Read position Consumer-2 Consumer-2 9 8 7 6 5

20.Support triggering ledger rollover when maxLedgerRolloverTimeMinutes is met Topic Ledger - 2 (open) Ledger - 1 (closed) Ledger - 0 (closed) Ledger rollover Fragment F - 1 F - 0 Fragment Bookie failover E-3 E-2 E-1 E-0 E-3 E-2 E-1 E-0 E-3 E-2 E-1 E-0

21.Support triggering ledger rollover when maxLedgerRolloverTimeMinutes is met # Max number of entries to append to a ledger before triggering a rollover # A ledger rollover is triggered on these conditions # * Either the max rollover time has been reached # * or max entries have been written to the ledged and at least min-time # has passed managedLedgerMaxEntriesPerLedger=50000 # Minimum time between ledger rollover for a topic managedLedgerMinLedgerRolloverTimeMinutes=10 # Maximum time before forcing a ledger rollover for a topic managedLedgerMaxLedgerRolloverTimeMinutes=240

22.Expose check delay of new entries in broker.conf # New entries check delay for the cursor under the managed ledger. # If no new messages in the topic, the cursor will try to check again after the delay time. # For consumption latency sensitive scenario, can set to a smaller value or set to 0. # Of course, use a smaller value may degrade consumption throughput. Default is 10ms. managedLedgerNewEntriesCheckDelayInMillis=10

23.[Pulsar Client] 1. Support waiting for inflight messages while closing a producer 2. Support returning sequence ID when throwing an exception public PulsarClientException(String msg, long sequenceId) { super(msg); this.sequenceId = sequenceId; }

24.[Pulsar SQL] 1. Support key value schema 2. Support multiple Avro schema versions

25.Precise topic publish rate limiting

26.Optimize consumer fetch messages in case of batch message

StreamNative 是一家围绕 Apache Pulsar 和 Apache BookKeeper 打造下一代流数据平台的开源基础软件公司。秉承 Event Streaming 是大数据的未来基石、开源是基础软件的未来这两个理念,专注于开源生态和社区的构建,致力于前沿技术。