Scaling MongoDB

下载 0

空白

发布于

1755

人观看

#信息技术

在本次网络研讨会中，我们将讨论如何将MongoDB扩展到每秒数千次写入和读取。在扩展MongoDB时，常见的问题是什么？最好是分片还是增加更多的二级片？
我们将介绍许多常见的扩展情况，以及部署分片集群所需的步骤：从单个实例到分片环境。我们还将讨论公司在扩展数据库时的常见错误/陷阱，以及如何避免这种情况。

展开查看详情

7 .The adamo.com company story adamo.com is a startup that started using MongoDB as a single instance and was using a single machine on a cloud provider. In this webinar, I’m going to tell you the story of how the company evolved from one instance to a sharded environment. Single instance I 8 GB RAM, 2 processors - WiredTiger as Storage Engine 7 © 2017 Percona

8 .The adamo.com company story After a few months, the database started answering queries very slowly. The company noticed high processor usage and a very active disk, so they decided to increase the instance type. Single instance I ETL process 16 GB RAM, 4 processors - WiredTiger as Storage Engine 8 © 2017 Percona

9 .The adamo.com company story Even after increasing the instance type, the company still noticed slow reads and writes. Processor usage was still high and the company still noticed high disk I/O. To make matters worse, the database failed after a few weeks. 9 © 2017 Percona

11 .The adamo.com company story So, why was this database still slow even after increasing the machine resources? What could be done to avoid outages? Now the adamo.com company had learned both how to use replica-sets and the advantages of having multiple instances with High Availability. The HA was solved, the single instance became a replica-set with 3 members, but they were still having issues on the primary instance. The same problem as they had faced before. 11 © 2017 Percona

12 .The adamo.com company story A new environment. The replica-set was working properly but still facing issues while ETL was running, and the application auto fail-over was not working properly. CACHE ETL Free memory ETL 12 © 2017 Percona

14 .The adamo.com company story A new environment. The replica-set was working properly but still facing issues while ETL was running, and the application auto fail-over was not working properly. CACHE ETL Free memory ETL 14 © 2017 Percona

15 .The adamo.com company story A few names you should know for the next steps: ● Read Preference: Most application drivers feature the read preference option. This is, in simple words, where the application will get data from. All writes will be routed to the primary, but you can change the read preference option to: - Primary, secondary_prefered, secondary_only, nearest, and tags. 15 © 2017 Percona

16 .The adamo.com company story A few names you should know for the next steps: ● Write Concern: MongoDB offers eventual consistency, but we can tune it up and have more than one instance confirming a write command. This is very useful to guarantee data consistency across the replica-set and/or datacenters. The write concerns are: 1,2,<n> and majority, where majority means ½ + 1 instance from the replica-set. 16 © 2017 Percona

17 .The adamo.com company story A few names you should know for the next steps: ● Replication Lag: MongoDB replication between instances doesn't occur in real time. It is asynchronous. A secondary instance with fewer resources than the primary can easily fall behind to receive new writes and updates. The most common issues are slow disks, limited bandwidth, and slow processors. Replication lag is a critical metric when you run queries on secondaries. 17 © 2017 Percona

18 .The adamo.com company story A few names you should know for the next steps: ● Oplog Window: Per design, the replication is asynchronous and all the primary commands are saved on the oplog.rs collection. The oplog collection is a fixed-size collection that saves such operations, so that the secondaries can pull and apply the same commands. As it is a fixed-size collection, the old documents will be replaced by new ones when it gets full. The period of time between the first and the last command in this collection is called oplog window. 18 © 2017 Percona

20 .The adamo.com company story The new environment seemed to be really robust, but after a while the company started facing new issues. The understanding of the concepts below is required to discuss the next issues. ● Hidden Instance ● Priority ● Votes ● Arbiters 20 © 2017 Percona

23 .The adamo.com company story The hidden secondary was completely hidden from the replica-set, which means that the application was not connected to the instance to perform reads. In the case above, only the ETL process read data from this secondary. If we need more reads, it is ok to add new secondaries. However, we must make sure we don't add too many of them, as issues such as replication delay may occur. 23 © 2017 Percona

25 .The adamo.com company story The replication delay can occur for several reasons. A replica-set may, for example, perform chained replication, which in other words means the database is replicating to a secondary, and that secondary is replicating to another secondary. The chained replication can be disabled on the replica-set configuration. ● https://docs.mongodb.com/manual/tutorial/manage-chained-replication/ 25 © 2017 Percona

27 .The adamo.com company story Not all the instances will be eligible to vote. Replica-sets can only have up to 7 instances voting. Therefore, if there are more than 7 instances in the replica-set, only 7 can vote. As good practice, keep those close to the "primary" and choose the best ones. Sometimes we need a mongodb instance that will only check whether the other instances are available. These are called arbiters and arbiters don’t keep data, they only perform votes. 27 © 2017 Percona

28 . The adamo.com company story PRIMARY Votes : 1 Priority : 1 P ETL Votes : 1 S S HS Votes : 0 Priority : 1 Votes : 1 Priority : 0 Priority : 1 S S Votes :0 Priority :0 Votes : 0 28 Priority : 0 © 2017 Percona

29 .The adamo.com company story It is time to consider sharding. Even with the replica-sets, the number of writes we need as well as the working set are making it too expensive to keep everything in a single replica-set. Sharding will create a virtual database among replicas and split the load between those replica sets, which will be called shards. 29 © 2017 Percona

0点赞

0收藏

0下载