16/06 - Running Cassandra on Amazon's ECS

展开查看详情

1.Running Cassandra on Amazon’s ECS    Anirvan Chakraborty   @anirvan_c  

6. Docker history • Came out of dotCloud, a PaaS company • Was originally written in Python • Got re-written in Golang in Feb, 2013 • Docker 0.1 was released on Mar, 2013 • Docker 1.10 is the latest release © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0

8. Docker tag line • Build: package your application in a container • Ship: move it between machines • Run: execute that container with your application • Any application: as long as it runs on Linux • Anywhere: local VM, bare metal, cloud instances © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0

11. What is Apache Cassandra? • Fast distributed database • High Availability • Linear Scalability • Predictable performance • No single point of failure • Multi-DC • Easy to manage • Can use commodity hardware • Not a drop in replacement for RDBMS © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0

12. Hash ring • Data is partitioned around the ring • Location of data in ring is determined by partition key • Data is replicated to N servers based on RF • All nodes hold data and can answer read or write queries source: https://academy.datastax.com/courses/ds101-introduction-cassandra/introduction-cassandra-overview © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0

13. CAP Tradeoff • During network partition it is impossible to be both consistent and highly available • Latency between data centres also makes consistency impractical • Cassandra chooses Availability & Partition tolerance over Consistency © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0

14. Replication • Choose “replication factor” or RF • Data is always replicated to each replica • If node is down, missing data is replayed via hinted handoff source: https://academy.datastax.com/courses/ds101-introduction-cassandra/introduction-cassandra-overview © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0

15. Consistency level • Per query consistency • ALL, QUORUM, ONE • How many replicas to respond OK for query to succeed source: https://academy.datastax.com/courses/ds101-introduction-cassandra/introduction-cassandra-overview © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0

17. Dockerize C* Dev Environment • Make it run as slow, but as stable as possible! • Super low memory settings in cassandra-env.sh • MAX_HEAP_SIZE=“128M” • HEAP_NEWSIZE=“24M” • Remove caches in dev mode in cassandra.yml • key_cache_size_in_mb: 0 • reduce_cache_sizes_at: 0 • reduce_cache_capacity_to: 0 © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0

18. Dockerize C* Production • Use host networking (—net=host) for better network performance • Put data, commitlog and saved_caches in volume mount folders to the underlying host • Run cassandra on the foreground using (-f) • Tune JVM heap for optimal size • Tune JVM garbage collector for your workload © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0

20. What is ECS? …is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0

27. Task { "containerDefinitions": [ { "name": "wordpress", "links": [ "mysql" ], "image": "wordpress", "essential": true, Docker Container "portMappings": [ { "containerPort": 80, "hostPort": 80 } ], "memory": 500, "cpu": 10 }, { "name": "mysql", "image": "mysql", "cpu": 10, Docker Container "memory": 500, "essential": true } ], Task definition } "family": "hello_world" http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0