AWS EC2上的快速、可靠、安全且经济实惠的MongoDB

AWS EC2上的自我管理MongoDB为您提供了领先的开源数据库和领先的云计算平台。我将解释在EC2中建立和维护快速、可靠、安全和经济的MongoDB环境所需了解的AWS概念。主题包括性能(计算、磁盘和网络)、容错、网络安全、备份和成本。您将收到一个工作部署模板,您可以根据自己的需要进行自定义。
欢迎大家。在任何平台上创建或管理MongoDB复制集的人,或者在EC2工作过并且现在想部署MongoDB的人,都将立即受益。

展开查看详情

1.Fast, Reliable, Secure, Affordable MongoDB on AWS EC2 Paul Marcelin

2.Why are we here today? • Among open-source databases, MongoDB is a complete production-ready solution • Self-managing MongoDB is worthwhile, for the best AWS performance at the lowest cost • A few simple but not widely-understood AWS tips prevent most AWS performance, cost, and security problems 2

3.Compute for MongoDB • Choose between 3 major instance families... • Memory-optimized • Compute-optimized • General-purpose • And one with a twist: • Burstable performance • Use the latest generation • Better performance, lower unit price 3

4.Disk for MongoDB • Network storage as a service: Elastic Block Store • Affordable: from 10¢ per gigabyte per month • Reliable: multiple copies; decoupled from compute • Convenient: snapshots; online volume enlarge • Don’t use Provisioned IOPS SSD (io1 volumes) or local SSD (i3, i3en instances) before you: • Optimize for EBS general-purpose SSD (gp2) volumes • Study CloudWatch metrics data Check with AWS for official prices. 4

5.Optimize for EBS General-Purpose SSD • Use latest-generation instances • More EBS bandwidth • Less EBS overhead • Use larger instances • Even more bandwidth! • Enlarge your general-purpose (gp2) volumes • More operations per second at a much lower cost than Provisioned IOPS (io1) gp2 IOPS: Base = 3 × volume size in GB Max = 16,000 IOPS (5.3 to 16 TB volume size) 5

6.Back Up MongoDB with EBS Snapshots • Create at least two separate EBS volumes 1. Operating system + software 2. Data + journal • Enable journaling • Keep journal on same volume as data • Take frequent snapshots • You pay only for changed disk blocks • AWS Backups: every 12 or 24 hours • github.com/sqlxpert/aws-tag-sched-ops: up to every 10 minutes 6

7.Basic Fault Tolerance • Each AWS region has multiple availability zones, in separate locations • Distribute your MongoDB replicaSet across availability zones Region Availability Zone Availability Zone Availability Zone MongoDB MongoDB MongoDB replicaSet replicaSet replicaSet member member member • Plan for a 2-zone minimum in some regions • Multi-region replicaSets are possible (but consider network latency) 7

8.More Fault Tolerance • Distribute your application layer, not just your database! • Always use replicaSet connection strings for application traffic Region Availability Zone Availability Zone Availability Zone MongoDB MongoDB MongoDB replicaSet replicaSet replicaSet member member member Application Application Application server server server Application Load Balancer 8

9.Yet More Fault Tolerance • Detect and replace failed replicaSet members • Make this automatic, or at least quick and convenient • Potential components: • An EC2 Fleet automatically replaces failed instances • A configuration management system (AWS OpsWorks, Chef, Ansible, SaltCloud, etc.) configures each new instance upon first boot • A pipeline updates a base Amazon Machine Image (AMI) 9

10.Basic Security Elements • AWS Key Management System (KMS) customer-managed key • Encrypts disks and snapshots • TLS certificates (not from AWS) • Encrypt application and replication traffic • Validate server identity • AWS security groups • Enforce network firewall rules • Also validate server identity (within the same AWS region) 10

11.Understand Disk Encryption (EBS + KMS) • Use customer-managed keys, not your default EBS service key! • Create a separate key for every MongoDB replicaSet • Encrypt both your data volume and your OS volume • Snapshots of encrypted volumes are necessarily encrypted • Edit the key policy to limit the people who can: • Attach encrypted volumes to instances • Create volumes from snapshots • Copy snapshots 11

12.Take Full Advantage of TLS Certificates • Every replicaSet member needs its own TLS certificate (and DNS record) • Underlaying an AWS Route 53 DNS private hosted zone may help! • The private key should live only as long as the replicaSet member • Obtain certificates from a third party; don’t self-sign • My personal favorites: • Let’s Encrypt (free) • DigiCert’s duplicate certificate feature + a wildcard certificate (worth the cost) 12

13.Understand AWS Security Groups • A security group is a set of network firewall rules • These rules can only allow, not block • Traffic that’s not allowed is blocked, but... • If you send out a request, the response is always allowed in (“stateful”) • Never use a default security group! • An instance can be a member of multiple security groups • Never reference same-region instances by IP address; instead, identify source and destination instances by their security groups • If you only police inbound traffic, put all instances in an all-outbound group and delete the default all-outbound rule from all other groups 13

14.Define Security Groups by Membership VPC all-out Security group: all-out SG: mongodb-server • All traffic out to 0.0.0.0/0 MongoDB MongoDB mongodb-server replicaSet replicaSet member member TCP 27017 in from: • mongodb-server (replication) • mongodb-client (application data) mongodb-client Application server No rules; just identifies clients https-server SG: mongodb-client • TCP 443 in from 0.0.0.0/0 SG: https-server 14

15.Advanced Security Elements • AWS Identity and Access Management EC2 instance role • Authorizes AWS API calls from an instance (including calls made by the Systems Manager agent) • No AWS API keys to rotate, distribute, and hold on disk • Task-specific IAM roles • Grant specific people shell access to specific instances • AWS Systems Manager – Session Manager • Provides shell access, with no SSH key pairs to manage 15

16.Summary: MongoDB on AWS EC2 • Three t3 instances, in multiple availability zones • Instance role with AmazonEC2RoleforSSM policy • AWS Systems Manager – Session Manager for shell access • Security groups: all-out, mongodb-server, mongodb-client • KMS key to encrypt only this replicaSet’s disks • For each instance: • Large additional EBS gp2 volume for data + journal • Public + private DNS records and a TLS certificate 16

17.We’re Almost Done • Thanks for coming! • Keep in touch at marcelin@alumni.cmu.edu • Try my template, github.com/sqlxpert/mongodb-percona-live • Or try Amazon’s, aws.amazon.com/quickstart/architecture/mongodb/ • Don’t forget to rate this session... 17

18.Rate This Session 18

19.Compute for MongoDB • Choose between 3 major instance families... • Memory-optimized from $92 per month 16 GB RAM • Compute-optimized $63 4 GB • General-purpose $71 8 GB • And one with a twist: • Burstable performance $31 4 GB • Use the latest generation • Better performance, lower unit price Smallest (t3: smallest reasonable) instance size, with 2 vCPUs in each case. Reserve, or use spot, for even lower prices. Check with AWS for official prices. 19

20.MongoDB + AWS Loose Ends • Swap • MongoDB documentation recommends it! • Put it on a third EBS volume • Encrypt that volume (of course) • Extra work is required to put swap on local (instance store) volumes, for instance types (e.g., m5d) that offer local storage • Customary configuration changes for MongoDB • Transparent Huge Pages: disable • File descriptor and process limits (ulimit): increase • Data volume mount options: add noatime 20