1.MongoDB 6.0新特性-唐峰

1 . WELCOME What's New in MongoDB 6.0 Feng Tang Manager , Professional Service 

2 .Agenda ● MongoDB Atlas: Developer Data Platform ● Simplified development ○ Time Series improvements ○ Change Streams updates ● Additional data security ○ Queryable Encryption ● Operational efficiency ○ Cluster-to-Cluster Sync ○ Initial Sync 

3 .Typically, organizations have hundreds, even thousands of tables built up over years 

4 .Adding niche “NoSQL” datastores addresses relational shortcomings, but adds complexity Application Application Application Application Caching Caching Layer Layer This is a tax on innovation. RDBMS RDBMS RDBMS RDBMS ETL A Data & Innovation Recurring Tax ETL ETL ETL ETL (DIRT). Key-Value DB ETL Wide Column Sync Time Series DB Graph DB ETL Search Engine Mobile DB Search Engine DocumentDB ETL ETL ETL Search Engine Data Lake Edge DB Sync Enterprise Data Warehouse 

5 .The most resilient, Data Lake scalable, and secure platform available Before After DIRT { ⤱ developer experience Unified developer experience ∞ operational models Repeatable operational / security model ↑ data integration Automated / transparent data movement ↑ data duplication Reduced data duplication 

6 .The evolution of MongoDB 2015 2016 2017 2018 2019 2020 2021-2022 2022-2023 3.0 & 3.2 3.4 3.6 4.0 4.2 4.4 5.x 6.x Doc-Level Concurrency Views Graph Processing Zones Change Streams Replica set transactions Distributed Transactions Union Time Series collections Queryable encryption (preview) RAFT / Fast Failover ++Aggregation ++ Retryable Writes Atlas global clusters Client-Side Field Level Encryption Custom Agg Expressions Clustered indexes Atlas Serverless GA $lookup Auto-balancing ++ Linearizable Schema Validation Expressive Materialized Views Refinable Shard Keys Window functions Column store indexes 40% Faster Shard Migrations Ops Manager Reads Decimal Intra-cluster $lookUp Wildcard Indexes Compound Hashed Shard Keys Live resharding Cluster-to-cluster Sync Atlas HIPAA Compression Compression Log Redaction Query Expressivity Global PIT Reads Mirrored Reads Client-Side FLE KMIP & cloud KMS Independent analytic nodes scaling Atlas LDAP ≤50 replicas Spark Connector ++ BI Causal Consistency Consistent Large Transactions Hedged Reads Atlas Serverless (preview) Flexible sync GA Atlas Audit Aggregation ++ Connector ++ Sharded Secondary Reads Mutable Shard Key Values Resumable Initial Sync Atlas Search fast facets, function Asymmetric sync preview Atlas Enc. Storage Engine Encrypted and In-Memory storage Query Advisor Atlas Data Lake (Beta) Time-Based Oplog Retention scores, synonyms Kotlin and Flutter SDKs for Realm Atlas Backup Snapshots engines End to End Compression Atlas Auto Scaling (Beta) Simultaneous Indexing Long running snapshot reads $lookup perf improvements Type Conversions BI Connector WiredTiger 1m+ Collections Atlas Search (Beta) Hidden Indexes Sharded $lookup & $graphlookup Atlas CLI GA Snapshot Reads Compass MongoDB BI Connector ++ Multi-CAs Streaming Replication Majority write concern default Atlas Data API GA Non-Blocking Sec. Reads R Driver Expressive Updates Global Read/Write Concerns 4x faster initial sync Secondary index on measurements SHA-2 & TLS 1.1+ Charts (post GA) Apache Kafka Connector Rust & Swift Drivers GA Schema validation diagnostics for time series Compass Agg Pipeline Builder Atlas X-Region Replication Atlas MongoDB Charts GA TLS 1.3 & Faster Client Auth New MongoDB Shell GA Time series read perf + Compass Export to Code Auto Storage Scaling Retryable Reads & Writes OCSP Stapling Resumable index builds Atlas Data Lake with fully managed Free Monitoring Cloud Service New Index Builds Kerberos Utility Rewritten Swift driver storage Ops Manager K8s Beta 10x Faster stepDown Atlas Online Archive Rewritten C# LINQ provider and Atlas Data Federation Storage Node Watchdog Auto-Scaling .NET Analyzer Atlas Charts dashboard embedding Zstandard Compression Schema Recommendations PyMongoArrow API Atlas SQL interface AWS IAM Auth & Atlas x509 New accumulator operators Encrypted audit log Federated Queries Charts on Data Lake Change streams improvements Multi-cloud clusters x509 certificate rotation Announcement of relational to Auditing ++ MongoDB migrator Atlas K8S controller Ops Manager migration wizard WiredTiger (Acquisition + Integration) ACID Stable API & Rapid Developer Data MongoDB Atlas Multi-Cloud (Transactions) Releases Platform 

7 .Simplified Development Time Series improvements Change Streams Updates 

8 .Time Series Improvements 

9 . Launched with 5.0 Making it Faster, Easier Native Support and Less Expensive to Build and Run Time Series Workloads with MongoDB Reduced Cost Fast Setup 

10 . Improved Support for MongoDB Time Series 5.0 5.1 5.2 5.3 6.0 Time Series Sharding Time Series Gap filling Sort collections support (added columnar performance in 5.0.6) compression Top-N & bottom-N optimizations Window accumulators Secondary Functions Delete-many on Densification metadata indexing on Temporal support (added Min-N & max-N measurements Expressions in 5.0.6) accumulators Secondary Atlas Online First-N & last-N indexing on Archive support metadata accumulators (added in 5.0.0+) 

11 .Improved Support for MongoDB Time Series Time Series Collections updates Query updates Secondary Indexes on Sort Performance Measurements Optimizations Sharding support TopN/BottomN Columnar Compression Densification & Gap Multi-Delete Support Filling 

12 .Improved Support for MongoDB Time Series Time Series Collections updates Query updates Secondary Indexes Sort Performance on Measurements Optimizations Sharding support TopN/BottomN Columnar Compression Densification & Gap Multi-Delete Support Filling 

13 .Improved Support for MongoDB Time Series Since MongoDB 5.0, there have been a significant number of features and capabilities added to improve Time Series collections Time Series Secondary Indexes on Measurements (Launched with 6.0) Collections updates Time Series Collections now support creating single and Secondary Indexes on compound indexes on any field within the document including the Measurements ability to geo-index measurements for geospatial analysis on moving sensors Sharding support Columnar Compression Multi-Delete Support 

14 .Improved Support for MongoDB Time Series Since MongoDB 5.0, there have been a significant number of features and capabilities added to improve Time Series collections Time Series Collections updates Sharding (Launched with 5.0.6) Time series collections can now take advantage of MongoDB Secondary Indexes on Measurements native sharding to horizontally distribute massive data sets Sharding support Increases throughput & cardinality handling Columnar Compression Multi-Delete Support 

15 .Improved Support for MongoDB Time Series Since MongoDB 5.0, there have been a significant number of features and capabilities added to improve Time Series collections Columnar Compression (Launched with 5.2) Time Series Collections updates Dramatically reduce database storage footprint Columnar compression adds a number of innovations that work Secondary Indexes on Measurements together to significantly improve practical compression, reduce your data's overall storage on disk, and improve read performance Sharding support Increases Cache efficiency fitting more data in memory and using Columnar Compression less I/O Multi-Delete Support 

16 .Improved Query Support for MongoDB Time Series Query updates Sort Performance Optimizations (Launched with 6.0) Time series collections can now take advantage of indexes to sort Sort Performance data by the timeField and/or metaField Optimizations “Lastpoint” queries, e.g. fetch the latest temperature for each unique sensor, can now take advantage of indexes to optimize Densification & Gap Filling performance TopN/BottomN 

17 .Change Streams Updates 

18 . Launched with 4.0 Native Support Change Streams enable monitoring events in the database and reacting to changes near real time Elegant Architecture Improved Scalability 

19 .Improved Support for MongoDB Change streams 4.0 5.0 6.0 Specify a start time, Optimized user Point-in-time Pre- and or start after option pipeline for filter and Post-Images projection Resume tokens Reporting DDL events Record array Additional truncations New ‘wallTime’ field aggregation stages Expressions for Timestamp Timestamp conversion to Date 

20 . New in 6.0 Update Events Can include pre-image and/or post-image Point in time pre- at the time of event, or current post-image and post-images Replace Events Can include pre-image in addition to post-image at the time of event Delete Events Can include pre-image at the time of event Options Choose to return error if image is not available or return null 

21 . New in 6.0 Enabled per collection Point in time pre- Off by default and post-images Optional retention config New cluster-level parameter preAndPostImages.expireAfterSeconds To include in Change Stream event Options fullDocument and fullDocumentBeforeChange on opening change stream 

22 .Example // enable pre/post images on a collection db.runCommand( { collMod: 'orders', changeStreamPreAndPostImages: {enabled: true} } ) // open change stream that includes both pre and post image cursor = db.watch( [], { fullDocument: "required", fullDocumentBeforeChange: "required" } ) 

23 . Change Stream Event for Update operation { _id: {... }, operationType: 'update', clusterTime: Timestamp({ t: 1651694083, i: 1 }), ... fullDocument: { _id: 0, customer: 'A', datePlaced: ISODate("2022-05-04T19:44:30.497Z"), status: 'SHIPPED' }, documentKey: { _id: 0 }, updateDescription: { updatedFields: { status: 'SHIPPED' }, ... }, fullDocumentBeforeChange: { _id: 0, customer: 'A', datePlaced: ISODate("2022-05-04T19:44:30.497Z"), status: 'PROCESSING' } } 

24 . New in 6.0 New events createIndexes, dropIndexes; collection create*, drop*, modify, Reporting DDL rename, shardCollection. events To enable Option showExpandedEvents:true when opening change stream * including views 

25 .Example // open change stream on the database and include DDL events cursor = db.watch( [], { showExpandedEvents: true } ) 

26 .Change Stream Event for Index Creation { _id: {...}, operationType: 'createIndexes', clusterTime: Timestamp({ t: 1651759748, i: 6 }), collectionUUID: UUID("b04242db-dccd-4203-b15a-9469e0b7fdc5"), wallTime: ISODate("2022-05-05T14:09:08.728Z"), ns: { db: 'test', coll: 'orders' }, operationDescription: { indexes: [ { v: 2, key: { customer: 1 }, name: 'customer_1' } ] } } 

27 . MISC Few other additions New field in change event 'wallTime': ISODate("2022-05-04T20:06:33.283Z") Timestamp can be converted to Date $toDate/$convert Expressions for Timestamp $tsSeconds, $tsIncrement to extract parts of the timestamp 

28 .Additional Data Security: Queryable Encryption 

29 .Option#1 – No encryption of data from client side Query to find ssn = “901-10-4312” {payer: “Acme Corp”, ssn: “901-10-4312”} {payer: “Jones Inc”, ssn: “901-10-4312”} … {payer: “Baker Co”, ssn: “901-10-4312”} 10 records fetched with ssn = “901-10-4312” 1 million records total 10 records with ssn = “901-10-4312” ● Fast querying ● But data is not secure in-use