Democratizing HBase

1 .Democratizing HBase Josh Elser HBaseCon West June 2017 

2 . Engineer at Hortonworks, Member of the Apache Software Foundation Various roles and involvement in… Top-Level Projects ASF Incubator • Accumulo • Fluo • Calcite • Gossip • HBase • Rya • Phoenix • Slider These Apache project names are trademarks or registered trademarks of the Apache Software Foundation. 2 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 

3 .Scope Apps Middleware HBase HDFS Hypervisor Linux Hardware 3 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 

4 .A Retrospective on Adoption 1 User Proof of Concept Single Org Transition Single Org Sim-Prod 2+ Users Multi-Org Transition Multi-Org Sim-Prod Company-Wide Shift 4 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 

5 .Optimal Utilization à Tight Budgets à Expensive Hardware Acquisition à Easily Accessible to Clients Bad Good Bad 5 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 

6 .Multi-tenancy à Out of many workloads, one system. • Low Latency (ms) • High Latency (min-hr) • Always Up • Scheduled/Regular • Guarantees of Multiple 9’s • High Throughput 6 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 

7 .Our Goals à User isolation à Sharing of resources à Flexibility in resource partitioning à Willful developer ignorance Image: Me, South Philadelphia Graffiti 7 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 

8 .Security side-step à Strong authentication à No impersonation à No unauthenticated users 8 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 

9 .Execution Throttles User-Driven Tasks à RPC Quotas à RPC Priorities à Space Quotas HBase Internals à MemStore Flushes à Compactions Image: 3DOM Graffiti 9 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 

10 .Workload Partitioning à RegionServer Groups – Benefits workload per table à Anti-pattern with present value à Beware external systems à Signal for required improvements Image: Me, South Philadelphia Graffiti 10 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 

11 .Insight and Metrics à Categorize system metrics à Validate configured limits à Understand HBase/HDFS limitations à Verify application use Image: Me, South Philadelphia Graffiti 11 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 

12 .Takeaways à Quantify application needs à Calculate system ability à Meet needs with throttling and partitioning à Understand non-HBase/HDFS processes à Monitor, Interpret, Act 12 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 

13 .Thanks! Email: elserj@apache.org Twitter: @joshelser 13 © Hortonworks Inc. 2011 – 2017. All Rights Reserved