Democratizing HBase

Democratizing_HBase
展开查看详情

1.Democratizing HBase Josh Elser HBaseCon West June 2017

2. Engineer at Hortonworks, Member of the Apache Software Foundation Various roles and involvement in… Top-Level Projects ASF Incubator • Accumulo • Fluo • Calcite • Gossip • HBase • Rya • Phoenix • Slider These Apache project names are trademarks or registered trademarks of the Apache Software Foundation. 2 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

3.Scope Apps Middleware HBase HDFS Hypervisor Linux Hardware 3 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

4.A Retrospective on Adoption 1 User Proof of Concept Single Org Transition Single Org Sim-Prod 2+ Users Multi-Org Transition Multi-Org Sim-Prod Company-Wide Shift 4 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

5.Optimal Utilization à Tight Budgets à Expensive Hardware Acquisition à Easily Accessible to Clients Bad Good Bad 5 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

6.Multi-tenancy à Out of many workloads, one system. • Low Latency (ms) • High Latency (min-hr) • Always Up • Scheduled/Regular • Guarantees of Multiple 9’s • High Throughput 6 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

7.Our Goals à User isolation à Sharing of resources à Flexibility in resource partitioning à Willful developer ignorance Image: Me, South Philadelphia Graffiti 7 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

8.Security side-step à Strong authentication à No impersonation à No unauthenticated users 8 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

9.Execution Throttles User-Driven Tasks à RPC Quotas à RPC Priorities à Space Quotas HBase Internals à MemStore Flushes à Compactions Image: 3DOM Graffiti 9 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

10.Workload Partitioning à RegionServer Groups – Benefits workload per table à Anti-pattern with present value à Beware external systems à Signal for required improvements Image: Me, South Philadelphia Graffiti 10 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

11.Insight and Metrics à Categorize system metrics à Validate configured limits à Understand HBase/HDFS limitations à Verify application use Image: Me, South Philadelphia Graffiti 11 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

12.Takeaways à Quantify application needs à Calculate system ability à Meet needs with throttling and partitioning à Understand non-HBase/HDFS processes à Monitor, Interpret, Act 12 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

13.Thanks! Email: elserj@apache.org Twitter: @joshelser 13 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

为了让众多HBase相关从业人员及爱好者有一个自由交流HBase相关技术的社区,阿里巴巴、小米、华为、网易、京东、滴滴、知乎等公司的HBase技术研究人员共同发起了组建中国HBase技术社区。