AntsDB Introduction

1 .hosted by AntsDB MySQL Compatibility for HBase August 17,2018 

2 . hosted by Content About Us 01 02 Scenario Architecture 03 Complementary to 04 HBase 

3 . hosted by 01 About Us 

4 . hosted by Water Guo  Founder of AntsDB  Love Hadoop and HBase but a heavy RDB user in the past  Love programming for extreme performance  Previously Software Architecture at Informatica, DataMirror, founder of BDI Systems. 

5 . hosted by AntsDB  Open source project at https://github.com/waterguo/antsdb  Problem to resolve: NoSQL is taking over the world but majority of the data applications are still using RDB  Database virtualization software  Attempt to bring MySQL compatibility to HBase 

6 . hosted by 02 Scenario 

7 . hosted by Virtual Telephone Number  Telecom industry  Rigid latency requirement: 20ms max  Data is growing fast  Built on top of relational database - MySQL 

8 . hosted by Solution Architecture User Batch Application Analytics Spark SQL AntsDB HBase 

9 . hosted by Lessons Learned What we like HBase What we were struggling  Linear scalability  Latency spike  Hadoop ecosystem  Transaction  Good random access  Join performance performance 

10 . hosted by Why don’t cache hot data  Operational data = hot data  Historical data = majority of big data  Daily hot data = 1-5%  Weekly hot data = 5-10%  Monthly hot data = 10-15% 

11 . hosted by Silver bullet - caching ✧ Large amount local caching using SSD – 500 GB ✧ Local data fetch -> sub-millisecond latency ✧ Distributed transaction -> local transaction ✧ Distributed joins -> local joins 

12 . hosted by 03 AntsDB Architecture 

13 . hosted by Layers MySQL Network JDBC, ODBC, PHP, Python, Native 1 SQL Parser/Executor 2 MySQL syntax, functions, data types Transaction MVCC, row locking 3 4 Caching HBase Connectivity Data in and out from HBase 

14 . hosted by Architecture Subtitle Text AntsDB SQL Parser Application MySQL Driver Query Executor Query Result Transaction Manager Local Transactions Cache HBase HBase Connector Query Cluster Result 

15 . hosted by Fight the GC - Java Unsafe - skip list, cache, WAL, row locking, cursor - Atomic operation for row locks - Small heap size - less than 4G - Escape analysis - Scan performance : 1 million rows / thread / second - Insert performance: 200K inserts / thread / second 

16 . hosted by Performance - YCSB write 

17 . hosted by Performance - TPCC 

18 . hosted by 04 Complementary to HBase 

19 . hosted by MySQL Compatibility  MySQL protocol – JDBC, ODBC, PHP, Python, Native  Core SQL syntax  MySQL data types  MySQL functions – 45  Low latency transactions  Low latency joins  Indexes - Unique, non-unique, full-text 

20 . hosted by MySQL Behaviour Simulation - Locking - Commit and rollback - MVCC - Quirks such as 0000-00-00 00:00:00 datetime, zerofill integers - Millisecond latency 

21 . hosted by Compatible applications  MySQL command line utilities  MySQL Workebench  DBeaver  SquirrelSQL  MediaWiki  SonarQube  PHPMyAdmin  BechmarkSQL – TPC-C benchmarking  YCSB 

22 . hosted by Next step High availability Cluster Low latency CP system 

23 .hosted by Thanks