Ke.com大数据平台架构,OLAP应用场景,Hbase性能优化等基础。

注脚

展开查看详情

1. hosted by Hbase Practice At ke.com Guoxian Zhao

2. hosted by Content 01 Architecture Evolution 02 Olap Scenario 03 Monitor Scenario 04 Data Mining Scenario 05 Reatime ETL Scenario 06 Hdic Scenario

3. hosted by 01 Architecture Evolution

4. hosted by bigdata architecture for ke Evolution

5. hosted by 02 Olap Scenario

6. hosted by Olap Scenario Requirement List Billion data set High concurrency subsecond response times sql interface

7. hosted by Olap Scenario kylin

8. hosted by Olap Scenario Olap data architecture map product Cube Metrics api redis query execute redis permission adhoc Meta Statistics manager queryengine Kylin middleware monitor Sparksql presto zk nginx Offline Spark cluster kylin meta build build query query Alluxio hadoop, warehouse + compute Hbase scheduler load … …

9. hosted by Data For Kylin  800+cube、16+business  200T storage、160000 million data、1 cube 6000 million  Query/day million, <500ms(95%), <1s (99%)

10. hosted by Data Product Metrics platform • roll-up drill • multidimensional analysis • self-service configuration report • standard indices

11. hosted by Performance Optimizations For Hbase SSD-FIRST Clien t OR H D S S D H D D • Read remote ssds first D DN1 DN2 DN3

12. hosted by Performance Optimizations For Hbase others  HDFS short-circuit  Data hedged ,multi thread read  Test stale datanodes, avoid the slow datanode  Shutdown balance  Multiwal, Improve write performance

13. hosted by 03 Monitor Scenario

14. hosted by Monitor senario grafana TeleGraf TSD Task metrics HAProxy HBASE Hadoopmetrics TSD

15. hosted by Monitor Scenario

16. hosted by 04 Data Mining Scenario

17. hosted by Data Mining Scenario

18. hosted by 05 RealTime ETL Scenario

19. hosted by RealTime ETL Scenario Subtitle Text RealTime DB ETL RealTime log ETL fast write bigdata storage HBASE 3 fast read 4 change record

20. hosted by 06 Hdic Scence

21. hosted by Hdic Scenario Subtitle Text 3 4

22. hosted by Data For Ke

23.hosted by Thanks

user picture
为了让众多HBase相关从业人员及爱好者有一个自由交流HBase相关技术的社区,阿里巴巴、小米、华为、网易、京东、滴滴、知乎等公司的HBase技术研究人员共同发起了组建中国HBase技术社区。

相关文档