HBase基本架构

Meta表

Meta表位置

通过zkCli.sh 查看 Meta 表位置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[zk: localhost:2181(CONNECTED) 0] get /hbase/meta-region-server
�regionserver:16020��ܻ>6�rPBUF

node3�}�����0
cZxid = 0x900000029
ctime = Tue Apr 26 11:45:39 CST 2022
mZxid = 0x900000029
mtime = Tue Apr 26 11:45:39 CST 2022
pZxid = 0x900000029
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 58
numChildren = 0

Meta表结构

1
2
3
4
5
6
hbase(main):058:0> desc 'hbase:meta'
Table hbase:meta is ENABLED
hbase:meta, {TABLE_ATTRIBUTES => {IS_META => 'true', REGION_REPLICATION => '1', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}
COLUMN FAMILIES DESCRIPTION
{NAME => 'info', BLOOMFILTER => 'NONE', VERSIONS => '3', IN_MEMORY => 'true', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', COMPRESSION => 'NONE', CACHE_DATA_IN_L1 => 'true', MIN_VERSIONS => '0', BLO
CKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}

Meta表内容

1
hbase(main):063:0> scan 'hbase:meta'

Region查找

数据存储与读取

image-20220426140159206

MemStore

手动刷新MemStoreStoreFile

1
2
hbase(main):001:0> flush 'student_a'
0 row(s) in 5.0000 seconds

StoreFile

Store Implementation

HFile

HFile结构

HFile内容

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
[zhangsan@node1 ~]$ hbase hfile
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/bigdata/hbase/hbase-1.4.13/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/bigdata/hadoop/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
usage: HFile [-a] [-b] [-e] [-f <arg> | -r <arg>] [-h] [-k] [-m] [-p]
[-s] [-v] [-w <arg>]
-a,--checkfamily Enable family check
-b,--printblocks Print block index meta data
-e,--printkey Print keys
-f,--file <arg> File to scan. Pass full-path; e.g.
hdfs://a:9000/hbase/hbase:meta/12/34
-h,--printblockheaders Print block headers for each block.
-k,--checkrow Enable row order check; looks for out-of-order
keys
-m,--printmeta Print meta data of file
-p,--printkv Print key/value pairs
-r,--region <arg> Region to scan. Pass region name; e.g.
'hbase:meta,,1'
-s,--stats Print statistics
-v,--verbose Verbose output; emits file and meta data
delimiters
-w,--seekToRow <arg> Seek to this row and print all the kvs for this
row only
1
[zhangsan@node1 ~]$ hbase hfile -v -p -m -f hdfs://node1:9000/hbase/data/default/student_region/32c6c6066478baacf6df565787ba07e5/Info/7066087eb9bb4244b9bc7effbb4f5c18

HFile合并

压实

https://forum.huawei.com/enterprise/en/hcia-big-data-hbase-key-processes-and-highlights/thread/839289-893

Minor

合并前的HFile被标记为删除;

Major

会删除被Minor标记为删除的HFile

1
2022-04-27 15:35:59,711 INFO  [node1,16000,1651036083810_ChoreService_6] balancer.StochasticLoadBalancer: Skipping load balancing because the cluster is balanced. Total cost: 10.445108454847286, Sum multiplier: 602.0, Minimum cost needed for balance: 0.05

HLog

WAL

Region管理

集群管理

节点管理

数据管理

数据导出
1
[zhangsan@node1 ~]$ hbase org.apache.hadoop.hbase.mapreduce.Import 'student_a' /backup

禁用、删除表。

数据导入
  • 创建表
1
[zhangsan@node1 ~]$ hbase org.apache.hadoop.hbase.mapreduce.Import 'student_a' /backup
数据迁移

故障处理

文件检测修复

HFile查看工具