Gain a MongoDB Advantage with the Percona Memory Engine

1 . Gain a MongoDB Advantage with the Percona Memory Engine Webinar December 6th, 2017 Vadim Tkachenko CTO, Percona 1 © 2017 Percona 

2 . Me • CTO at Percona • Leading Software Engineering at Percona • Technology & (database, hardware, filesystems) Performance Enthusiast • Current areas of interest: MySQL, MongoDB and ClickHouse • @VadimTk 2 © 2017 Percona 

3 .Memory Engine • https://www.percona.com/software/mongo-database/percona- memory-engine-for-mongodb • Storage Engine in Percona Server For MongoDB • Based on wiredTiger • but without writes to the storage • High Write Throughput • High Read Throughput • Predictable, low latencies 3 © 2017 Percona 

4 .Memory Engine use cases 4 © 2017 Percona 

5 .Application cache Replace services such as memcached and custom application-level data structures with the full power of MongoDB features. 5 © 2017 Percona 

6 .Typical architecture Caching layer Key-value store (memcached or Limited capabilities Redis) Database layer (MongoDB, MySQL, etc) 6 © 2017 Percona 

7 .Better way • Document store • Flexible query language • Replication / sharding Caching layer • Familiar MongoDB deployment Percona Memory Engine Database layer (MongoDB, MySQL, etc) 7 © 2017 Percona 

8 .Session management Decrease application response times by keeping active user sessions in memory. 8 © 2017 Percona 

9 .Transient runtime state Store application stateful runtime data that doesn't require on-disk storage. 9 © 2017 Percona 

10 .Multi-tier object sharing Facilitate sharing of data in multi-tier/multi-language applications. 10 © 2017 Percona 

11 .Application testing Reduce turnaround time for automated application tests. 11 © 2017 Percona 

12 .Sophisticated data manipulation Increase performance for data manipulation operations such as aggregation and map reduction. 12 © 2017 Percona 

13 .Real-time Analytics Uses in-memory computing in situations where response time is more critical than persistence. 13 © 2017 Percona 

14 .Memory Engine deployments 14 © 2017 Percona 

15 .Standalone • For testing and development • Useful for sizing and quick resets • No Redundancy PSMDB Memory • No Persistence • Limited to server size 15 © 2017 Percona 

16 .Standalone config ./mongod --storageEngine inMemory -–dbpath /data/db -–inMemorySizeGB 150 PSMDB Memory 16 © 2017 Percona 

17 .Replica Set - Memory • For production primary secondary • Failover capability • Data redundancy PSMDB Memory PSMDB Memory • No Persistence secondary • Read scaling PSMDB Memory 17 © 2017 Percona 

18 .Replica Set - config ./mongod --storageEngine inMemory -–dbpath /data/db -–inMemorySizeGB 150 --replSet repSet1 primary secondary PSMDB Memory PSMDB Memory secondary PSMDB Memory 18 © 2017 Percona 

19 .Replica Set – with Persistent Option • For production primary secondary • Failover capability • Data redundancy PSMDB Memory PSMDB Memory • Data Persistence secondary • Read scaling Permanent storage • Configure permanent member as a hidden member (i.e. hidden: true PSMDB wiredTiger or MongoRocks and priority: 0) 19 © 2017 Percona 

20 .Replicate set with permanent option • Only the mongod instances running with the Memory storage engine can become the primary. • Clients connect only to the Memory storage engine mongod instances. • Even if both mongod instances running Memory storage engine crash and restart, they can sync from the member running permanent storage engine. • The hidden mongod instance running with permanent storage persists the data to disk, including the user data, indexes, and replication configuration information. 20 © 2017 Percona 

21 .Sharding – scaling for more memory mongos Shard by id PSMDB Memory PSMDB Memory 21 © 2017 Percona 

22 .Sharding – scaling mongos • For production • Failover capability • Data redundancy Shard by id • Data Persistence primary secondary primary secondary • Write scaling PSMDB Memory secondary PSMDB Memory PSMDB Memory secondary PSMDB Memory PSMDB wiredTiger or PSMDB wiredTiger or MongoRocks MongoRocks 22 © 2017 Percona 

23 .PSMDB extras 23 © 2017 Percona 

24 .PSMDB – secondary indexes { "_id“ : “Vadim”, • Data retrieval by “visits“ : 45, document fields ”created“ : ISODate("2016-01-07T15:46:32.085Z”) } > db.coll.ensureIndex({visits:1}); > db.coll.find({visits : {$gte :40}}); 24 © 2017 Percona 

25 .PSMDB – TTL indexes { "_id“ : “Vadim”, • To remove data “visits“ : 45, automatically ”created“ : ISODate("2016-01-07T15:46:32.085Z”) } > db.coll.createIndex({"created": 1}, {expireAtferSeconds: 7200 }); 25 © 2017 Percona 

26 .More features • Authentication • Including External SASL Authentication • Role-Based Access Control • Audit logging • Geospatial Indexes • Text Search 26 © 2017 Percona 

27 .PSMDB Memory Engine - summary Store Type Document Store Failover / HA Replica Set Scaling Sharding Persistence Extra Replica with wiredTiger or MongoRocks CRUD Insert, find, update, remove Expire data TTL Concurrency Multi-threading 27 © 2017 Percona 

28 .Scalability – Multi Threading • Intel(R) Xeon(R) CPU E5-2683 v3 @ 2.00GHz with 28 cores / 56 threads • 256GB RAM • Sysbench 1.0 with oltp_mongo.lua • Available at https://github.com/Percona-Lab/sysbench-mongodb-lua • 100GB dataset • User threads from 1 to 128 • Operation: OLTP event 28 © 2017 Percona 

29 .29 © 2017 Percona