Building and running cloud-native Cassandra

1 .Building and Running Cloud Native Cassandra Vinay Chella, Joey Lynch Distributed Database Engineers Netflix NGCC 2019 

2 .Speakers Vinay Chella Joey Lynch Distributed Systems Engineer Distributed Systems Engineer Focusing on Apache Cassandra and Data Distributed system addict and data wrangler Abstractions Cloud Data Engineering Cloud Data Engineering Netflix Netflix 

3 .Outline — What is Cloud Native? Goals to help align the project cloud principles. — Cassandra’s Rough Edges ◆ Development ◆ Packaging ◆ Starting a Cluster ◆ Running a Cluster — Our Proposed Solutions Agenda 

4 .What even is “Cloud Native”? 

5 .

6 .Any hardware configuration Any operating system Cattle not Pets Will die constantly 

7 .Clouds provide mappings on top of “datacenters” Zones will fail Regions will fail “Durable” storage is not so durable 

8 .Develop Package Starting Scaling 

9 .Develop Package Starting Scaling 

10 .Friction for New Contributions docs Discovery, building mental model code Understand and modify build Create artifacts (jar, pkg, container) test Confidence they haven’t broken something 

11 . First interactions? — I don’t know how the database works — Where can I run v4.0? 

12 .How Netflix Does This 

13 .How Netflix Does This 

14 .How Netflix Does This Markdown Pull Request Immediate Deploy 

15 .Proposal docs Move svn website to git branch in main repo code Replace sphinx with markdown build (pandoc)? test Automatically build and publish docs (jenkins + docker?) 

16 .Proposal docs code build test 

17 .Non standard layout 2260 line XML build.xml Checked in library .jars Checked in python library 

18 .

19 .

20 .Can use Gradle for Cassandra too Import build.xml Add any jars we want at build time 

21 .Can use Gradle for Cassandra too Build debs directly 

22 .Proposal docs Start by importing build.xml code Gradually modernize build system via gradle multi-project builds build Use out of the box dependency test locking 

23 .Proposal docs code build test 

24 .Jenkins 10 hour builds Who can run? 

25 .CircleCI 20 minute builds Anyone can run them 

26 .Proposal docs Keep moving with the CircleCI integration for testing (artifacts code stay on jenkins) build Run unit tests on every pull request (5 minutes, doable with free tier) test 

27 .Develop Distribute Launch Scale 

28 .Cloud Native Distribution source Building from source must be easy packages Integration with package managers apt, yum, brew ... containers Having Docker containers for testing, as well as production 

29 .Cloud Native Distribution source Already talked about this, we can improve in this area with previous packages proposals containers