Building and running cloud-native Cassandra

Building and running cloud-native Cassandra from Netflix


1.Building and Running Cloud Native Cassandra Vinay Chella, Joey Lynch Distributed Database Engineers Netflix NGCC 2019

2.Speakers Vinay Chella Joey Lynch Distributed Systems Engineer Distributed Systems Engineer Focusing on Apache Cassandra and Data Distributed system addict and data wrangler Abstractions Cloud Data Engineering Cloud Data Engineering Netflix Netflix

3.Outline — What is Cloud Native? Goals to help align the project cloud principles. — Cassandra’s Rough Edges ◆ Development ◆ Packaging ◆ Starting a Cluster ◆ Running a Cluster — Our Proposed Solutions Agenda

4.What even is “Cloud Native”?


6.Any hardware configuration Any operating system Cattle not Pets Will die constantly

7.Clouds provide mappings on top of “datacenters” Zones will fail Regions will fail “Durable” storage is not so durable

8.Develop Package Starting Scaling

9.Develop Package Starting Scaling

10.Friction for New Contributions docs Discovery, building mental model code Understand and modify build Create artifacts (jar, pkg, container) test Confidence they haven’t broken something

11. First interactions? — I don’t know how the database works — Where can I run v4.0?

12.How Netflix Does This

13.How Netflix Does This

14.How Netflix Does This Markdown Pull Request Immediate Deploy

15.Proposal docs Move svn website to git branch in main repo code Replace sphinx with markdown build (pandoc)? test Automatically build and publish docs (jenkins + docker?)

16.Proposal docs code build test

17.Non standard layout 2260 line XML build.xml Checked in library .jars Checked in python library



20.Can use Gradle for Cassandra too Import build.xml Add any jars we want at build time

21.Can use Gradle for Cassandra too Build debs directly

22.Proposal docs Start by importing build.xml code Gradually modernize build system via gradle multi-project builds build Use out of the box dependency test locking

23.Proposal docs code build test

24.Jenkins 10 hour builds Who can run?

25.CircleCI 20 minute builds Anyone can run them

26.Proposal docs Keep moving with the CircleCI integration for testing (artifacts code stay on jenkins) build Run unit tests on every pull request (5 minutes, doable with free tier) test

27.Develop Distribute Launch Scale

28.Cloud Native Distribution source Building from source must be easy packages Integration with package managers apt, yum, brew ... containers Having Docker containers for testing, as well as production

29.Cloud Native Distribution source Already talked about this, we can improve in this area with previous packages proposals containers