展开查看详情
2 .HBase At Tencent
Andrew Cheng | 程广旭
Tencent | HBase Committer
3 .Content
01. HBase Service In Tencent
02. Applications
03. Practices & Optimization
4 .01. HBase Service
In Tencent
5 .HBase Story in Tencent
l Began using since 2013
l Used version
l 0.94.17 -> 0.98.6 -> 1.2.5 -> 2.2.0 (ing)
l Largest cluster more than 500 nodes
90+ 4000+ 10PB+ 3Tri+
Clusters Nodes Data RPD
6 .Overview
HBase Users come from 6 groups , more than 100+ different applications
7 .Architecture
Phoenix Advertiseme Deploy Center
Tenpay Wepay Game …
nt
OpenTSDB Doss
TDBank Spark Tookit RestServer
S2Graph Lhotse HBase Api ThriftServer monitoring
Kylin Tencent HBase Zookeeper TNM2
9 .Tencent Ads – Real-Time Logjoin System
Data Source Mixer Exposure Click …
Transport TDBank
Logical LogJoin LogJoin LogJoin LogJoin
Association Table
Storage Tencent HBase
Flow Table
Consumer Model learning Freshness Budget control Report
10 .Tenpay - Transaction record
Data Source MySQL
Application Application
Binlog Paser DBSync C++ JAVA
Read
Cache Hippo
Read
Thrift Server
Write TDSort
Read
Write
Storage Tencent HBase
11 .03. Practices &
Optimization
12 .Practices–Data migration
Business-insensitive data migration
Cluster A Cluster B
add_peer delete_snapshot Client switch to new cluster
disable_peer enable_peer Check Data
Set REPLICATION_SCOPE => '1' Set REPLICATION_SCOPE => '0'
snapshot clone_snapshot
ExportSnapshot
13 .Practices–Table
l Create table per day
l Large amount of data
l TTL is short
l Benefit
l Reduce the amount of data in compaction
l Easy to delete expired data
14 .Optimization - Bandwidth
Input Data Input Data Input Data ① Input Data
② RS2 and RS3 Wal data
①
② ③ RS2 and RS3 Flush data
Wal
Flush ③
Small compact ④ ④ RS2 and RS3 Small compact
RS1 RS2 RS3
Large compact ⑤
⑤ RS2 and RS3 Large compact
15 .Optimization - Bandwidth
l Enable compressing of CellBlocks
l Wal compressor
l Increase the size of memstore
l Reduce the number of threads about compaction
l Turn off major compaction
l create tables by day
16 .Optimization - Online filtering of dirty data
l A large amount of data which have the same Rowkey
l How to find filter rowkeys?
Input Data
l ResponseTooSlow
Yes
l How to set filter rowkeys?
Filter Yes
l hbase.hregion.filter.rowkeys Filter
Enable
l How to refresh filter rowkeys? No
l update_config No
Write
17 .Optimization - Prefix Bloom Filter（HBASE-20636）
l ROWPREFIX_FIXED_LENGTH
Create Table:
l ROWPREFIX_DELIMITER
uin ts action
Prefix
File info:
Bloom Filter
18 .Optimization - Prefix Bloom Filter（HBASE-20636）
Write Read
Input Data
Get prefix key by
Scan
prefix_length
Rowkey
Get prefix key by prefix_length
Computer hash
{StartKey,EndKey} value
Computer hash value
No
Same No
prefix? Hit
Filter StoreFile
BloomFilter?
Set BloomFilter
Yes
Prefix length Yes Yes
No
>=
Last line? prefix_length
Yes No
Not Filter StoreFile
Write BloomFilter information to StoreFile metadata
19 .Optimization - RestServer
User
Nginx
RestServer A RestServer B RestServer C RestServer D
Cluster A Cluster B Cluster C
20 .Optimization - RestServer
User
Nginx
Mysql RestServer A RestServer B RestServer C
Cluster A Cluster B Cluster C
21 .Optimization - RestServer
l Only maintain one configuration
l use effectively resources
l User-friendly access
22 .HBase Community
l 1 Committer, 2 Contributor
l Total commits: 80+
l Feature
l HBASE-20636 Introduce two bloom filter type : ROWPREFIX_FIXED_LENGTH and ROWPREFIX_DELIMITED
l HBASE-19799 Add web UI to rsgroup
l HBASE-20243 [Shell] Add shell command to create a new table by cloning the existent table
l HBASE-19483 Add proper privilege check for rsgroup commands
l ………
23 .Join Us
Dept. Wechat Personal Wechat
