1.Running Cassandra on Amazon’s ECS
 
 Anirvan Chakraborty 
 @anirvan_c 
 

2. Agenda • Motivation • Docker • Cassandra • Cassandra on Docker best practices • EC2 Container Service ECS • Cassandra on ECS © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

3. Motivation © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

4. Motivation • Ease of development • Support polyglot languages, frameworks and components • Operational simplicity • Quick feedback loop © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

5. Docker © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

6. Docker history • Came out of dotCloud, a PaaS company • Was originally written in Python • Got re-written in Golang in Feb, 2013 • Docker 0.1 was released on Mar, 2013 • Docker 1.10 is the latest release © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

7. Docker tag line Build, ship and run any app, anywhere © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

8. Docker tag line • Build: package your application in a container • Ship: move it between machines • Run: execute that container with your application • Any application: as long as it runs on Linux • Anywhere: local VM, bare metal, cloud instances © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

9. Why Docker? • Deploy reliably & consistently • Execution is fast and light weight • Simplicity • Developer friendly workflow • Fantastic community © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

10. Apache Cassandra © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

11. What is Apache Cassandra? • Fast distributed database • High Availability • Linear Scalability • Predictable performance • No single point of failure • Multi-DC • Easy to manage • Can use commodity hardware • Not a drop in replacement for RDBMS © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

12. Hash ring • Data is partitioned around the ring • Location of data in ring is determined by partition key • Data is replicated to N servers based on RF • All nodes hold data and can answer read or write queries source: https://academy.datastax.com/courses/ds101-introduction-cassandra/introduction-cassandra-overview © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

13. CAP Tradeoff • During network partition it is impossible to be both consistent and highly available • Latency between data centres also makes consistency impractical • Cassandra chooses Availability & Partition tolerance over Consistency © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

14. Replication • Choose “replication factor” or RF • Data is always replicated to each replica • If node is down, missing data is replayed via hinted handoff source: https://academy.datastax.com/courses/ds101-introduction-cassandra/introduction-cassandra-overview © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

15. Consistency level • Per query consistency • ALL, QUORUM, ONE • How many replicas to respond OK for query to succeed source: https://academy.datastax.com/courses/ds101-introduction-cassandra/introduction-cassandra-overview © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

16. Cassandra on Docker © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

17. Dockerize C* Dev Environment • Make it run as slow, but as stable as possible! • Super low memory settings in cassandra-env.sh • MAX_HEAP_SIZE=“128M” • HEAP_NEWSIZE=“24M” • Remove caches in dev mode in cassandra.yml • key_cache_size_in_mb: 0 • reduce_cache_sizes_at: 0 • reduce_cache_capacity_to: 0 © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

18. Dockerize C* Production • Use host networking (—net=host) for better network performance • Put data, commitlog and saved_caches in volume mount folders to the underlying host • Run cassandra on the foreground using (-f) • Tune JVM heap for optimal size • Tune JVM garbage collector for your workload © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

19. Amazon EC2 Container Service © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

20. What is ECS? …is a highly scalable, fast, container management service that makes it easy to run, stop, and manage Docker containers on a cluster of Amazon EC2 instances. http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

21. What is ECS? Amazon Docker as a Service https://www.expeditedssl.com/aws-in-plain-english © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

22. How does ECS work? http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

23. Cluster http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

24. Container Instance http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

25. ECS Agent http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

26. Task http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

27. Task { "containerDefinitions": [ { "name": "wordpress", "links": [ "mysql" ], "image": "wordpress", "essential": true, Docker Container "portMappings": [ { "containerPort": 80, "hostPort": 80 } ], "memory": 500, "cpu": 10 }, { "name": "mysql", "image": "mysql", "cpu": 10, Docker Container "memory": 500, "essential": true } ], Task definition } "family": "hello_world" http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

28. Service http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0 

29. ECS Service http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html © 2016 Anirvan Chakraborty CC BY-NC-SA 4.0