Cloud Native Databases

1 .Cloud Native Databases 11.10.2017 Michael Mueller @michmueller_ info@container-solutions.com container-solutions.com container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

2 . Who am I? ■ Dad ■ CTO - Container Solutions AG ■ CNCF Ambassador ■ Kubernetes expert ■ All about Cloud (Native) stuff! ■ Feeling a bit awkward to talk about DBs at a conference of DB experts - Hint: Not a DBA, but interesting to be here container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

3 . What is Cloud Native, and should I care? container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

4 . What is Cloud Native? According to Cloud Native Computing Foundation ■ Container packaged ■ Running applications and processes in containers ■ achieve high levels of resource isolation ■ simplify operations for cloud native applications. ■ Dynamically managed ■ By a scheduler ■ Micro-services oriented ■ Loosely coupled ■ increase the overall agility and maintainability of applications container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

5 . What is Cloud Native (Michael’s Version) ■ Horizontally scalable ■ Build to handle failures ■ No single point of failure ■ Self-healing ■ Minimal operational overhead ■ Decoupled from underlying infrastructure container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

6 .Design Principles of Cloud Native Applications ■ Design for Performance responsiveness, concurrency, efficiency ■ Design for Automation automate dev & ops tasks ■ Design for Resiliency fault-tolerance, self-healing ■ Design for Elasticity automatic scaling ■ Design for Delivery minimise cycle-time, automate deployment ■ Design for Observability cluster-wide logs, traces & metrics container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

7 . Should I care? container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

8 .■ Easier to use ■ Easier to manage ■ Easier to scale ■ Less downtime ■ Made for cloud infrastructures ■ Harder to debug container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

9 . Example Application container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

10 . Sock Shop Architecture container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

11 . Deploy the sock-shop container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

12 . Kubernetes https://docs.google.com/presentation/d/17TxRTHYNaZ3l5ZmBLXAwpBUksrp8XiM-aN9GEhHF6aI/edit#slide=id.g1e639c415b_0_56 container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

13 . Kubernetes Core Principles ■ Pods ■ Services ■ Replication Controllers ■ Labels / Selectors apiVersion: v1 apiVersion: v1 apiVersion: v1 kind: Pod kind: Service kind: ReplicationController metadata: metadata: metadata: name: nginx name: nginx-service name: nginx labels: spec: spec: app: nginx ports: replicas: 3 spec: - port: 8000 selector: containers: targetPort: 80 app: nginx - name: nginx protocol: TCP template: image: nginx:1.7.9 selector: metadata: ports: app: nginx name: nginx - containerPort: 80 labels: app: nginx spec: containers: - name: nginx image: nginx ports: - containerPort: 80 All Resources can be expressed as YAML or JSON files! With a combination of this, an application can be described in k8s container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

14 . Here’s the sock-shop container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

15 . (Auto)Scaling the Sock-Shop ■ We can use autoscaling for the stateless parts --- apiVersion: extensions/v1beta1 kind: HorizontalPodAutoscaler metadata: name: front-end namespace: sock-shop spec: scaleRef: kind: Deployment name: front-end subresource: scale minReplicas: 1 maxReplicas: 10 cpuUtilization: targetPercentage: 50 ■ Most likely the DB needs also to be scaled if the FE get pressure, but how? ■ Vertically ■ Horizontally container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

16 . ContainerPilot ■ Self-aware and self-operating containers promoted by Joyent ■ Agent inside each container that recognize changes of the infrastructure ■ Automation is inside the application Wrapper-script which start the agent and the agent the binary root@993acf351cd9:/# ps axo uid,pid,ppid,stime,cmd UID PID PPID STIME CMD root 1 0 19:02 /bin/containerpilot mysql 94 1 19:02 |_ mysqld --console --gtid-mode=ON... root 107 1 19:04 |_ python /bin/triton-mysql.py health root 109 1 19:04 | |_ /usr/bin/innobackupex --no-timestamp... root 120 1 19:06 |_ python /bin/triton-mysql.py health root 121 1 19:06 |_ mysql -u repl -p... container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

17 . ContainerPilot - Typical use case Operating MySQL with asynchronous replication: ■ Clients make queries to the replica, or make writes to the primary. Concerns for the deployment: ■ How does the replica know where to find the primary? ■ How does the primary tell the replicas where to start replication? ■ How does the client know where to find nodes and which nodes accept writes? And then after deployment we have another set of ongoing concerns: ■ How to do backups? ■ How to promote a replica if the primary fails? ■ How do the other replicas know where to find a new primary during failover? ■ How does the client know that we failed over? container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

18 . Stateful applications in Kubernetes - Operators An operator is a piece of software that: ■ knows a lot of the operational best practices for a specific software ■ can deploy that piece of software in a good configuration on top of Kubernetes ■ And keep it healthy over time container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

19 . Use case ■ Using one of them in the e.g. SockShop and apply an auto-scaling policy to the DB ■ If load increases a new node gets added and load will get distributed ■ Works great within a datacenter or region, but... container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

20 .container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

21 . Sock-Shop goes global ■ Usually traffic peaks during lunch-break and after work until 10pm ■ In a global shop, traffic is following the sun ■ So we deploy Kubernetes Clusters across the globe ■ Apply cluster-autoscaling and pod-autoscaling with ContainerPilot or an K8s Operator and we’re fine, or? container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

22 . How to guarantee consistency over multiple regions/ datacenters and independent Kubernetes clusters? container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

23 . Cloud Native DBs - Global replication? container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

24 . Google Cloud Spanner ■ fully managed ■ mission-critical relational database ■ tested for transactional consistency ■ high availability ■ global scale ■ ACID 1 https://en.wikipedia.org/wiki/Atomic_clock#/media/File:Atomic_Clock-Louis_Essen.jpg ■ is “technically CP” but “effectively CA”. Nice to consume as a service, but hard to build by yourself. 1 Louis Essen (right) and Jack Parry (left) standing next to the world's first caesium-133 atomic clock. container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

25 . Google Cloud Spanner What does it need for something like cloud spanner: ■ Atomic clock and GPS clock ■ really accurate wall time synchronization (own scientific area) ■ clock offset between any two nodes in the system ■ Needed for external consistency Not what you usually have on your shelf container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

26 . CockroachDB CockroachDB is a distributed SQL database (ACID) ■ built on a transactional and strongly-consistent key-value store ■ it scales horizontally ■ survives ■ disk ■ machine ■ rack ■ even datacenter failures with minimal latency disruption and no manual intervention ■ provides a familiar SQL API for structuring, manipulating, and querying data container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_ 

27 . Questions? container-solutions.com info@container-solutions.com Cloud Native Databases@XLDB @michmueller_