可以简单点 2018-07-17 02:04 采纳率: 80%
浏览 6826
已采纳

使用HbaseTemplate对hbase数据库进行增删改查速度缓慢,效率不高

hbase我的是单机版的,不上集群,版本是1.2.6
使用Java代码对HBASE数据库进行查询期间没有报错,但就是不管为查询多少条数据都是花了20s
在 hbase-env.sh文件里面加了下面配置:
export JAVA_HOME=/usr/local/jdk1.8
export HBASE_MANAGES_ZK=true

在hbase-site.xml文件加了下面配置:

hbase.rootdir
file:///usr/local/app/hbase-1.2/data

下面是HBASE的一些log日志

2018-07-16 20:09:19,899 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x164a2feb4680001 type:create cxid:0x7 zxid:0xa7 txntype:-1 reqpath:n/a Error Path:/hbase/flush-table-proc/acquired Error:KeeperErrorCode = NodeExists for /hbase/flush-table-proc/acquired
2018-07-16 20:09:19,909 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x164a2feb4680001 type:create cxid:0xa zxid:0xa8 txntype:-1 reqpath:n/a Error Path:/hbase/online-snapshot/acquired Error:KeeperErrorCode = NodeExists for /hbase/online-snapshot/acquired
2018-07-16 20:09:19,910 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x164a2feb4680000 type:create cxid:0x2a zxid:0xa9 txntype:-1 reqpath:n/a Error Path:/hbase/flush-table-proc/acquired Error:KeeperErrorCode = NodeExists for /hbase/flush-table-proc/acquired
2018-07-16 20:09:19,911 INFO [master:37346.activeMasterManager] procedure.ZKProcedureUtil: Clearing all procedure znodes: /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort
2018-07-16 20:09:19,915 INFO [RS:0;master:51571] regionserver.MemStoreFlusher: globalMemStoreLimit=97.5 M, globalMemStoreLimitLowMark=92.6 M, maxHeap=243.7 M
2018-07-16 20:09:19,919 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x164a2feb4680000 type:create cxid:0x30 zxid:0xaa txntype:-1 reqpath:n/a Error Path:/hbase/online-snapshot/acquired Error:KeeperErrorCode = NodeExists for /hbase/online-snapshot/acquired
2018-07-16 20:09:19,919 INFO [RS:0;master:51571] regionserver.HRegionServer: CompactionChecker runs every 10sec
2018-07-16 20:09:19,925 INFO [master:37346.activeMasterManager] procedure.ZKProcedureUtil: Clearing all procedure znodes: /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort
2018-07-16 20:09:19,938 INFO [RS:0;master:51571] regionserver.RegionServerCoprocessorHost: System coprocessor loading is enabled
2018-07-16 20:09:19,938 INFO [RS:0;master:51571] regionserver.RegionServerCoprocessorHost: Table coprocessor loading is enabled
2018-07-16 20:09:19,940 INFO [RS:0;master:51571] regionserver.HRegionServer: reportForDuty to master=master,37346,1531742959208 with port=51571, startcode=1531742959688
2018-07-16 20:09:19,951 INFO [master:37346.activeMasterManager] master.MasterCoprocessorHost: System coprocessor loading is enabled
2018-07-16 20:09:19,964 INFO [master:37346.activeMasterManager] procedure2.ProcedureExecutor: Starting procedure executor threads=5
2018-07-16 20:09:19,964 INFO [master:37346.activeMasterManager] wal.WALProcedureStore: Starting WAL Procedure Store lease recovery
2018-07-16 20:09:19,990 INFO [master:37346.activeMasterManager] wal.WALProcedureStore: Lease acquired for flushLogId: 105
2018-07-16 20:09:19,999 INFO [master:37346.activeMasterManager] zookeeper.RecoverableZooKeeper: Process identifier=replicationLogCleaner connecting to ZooKeeper ensemble=localhost:2181
2018-07-16 20:09:19,999 INFO [master:37346.activeMasterManager] zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=10000 watcher=replicationLogCleaner0x0, quorum=localhost:2181, baseZNode=/hbase
2018-07-16 20:09:20,000 INFO [master:37346.activeMasterManager-SendThread(localhost:2181)] zookeeper.ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
2018-07-16 20:09:20,001 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxnFactory: Accepted socket connection from /0:0:0:0:0:0:0:1:52389
2018-07-16 20:09:20,001 INFO [master:37346.activeMasterManager-SendThread(localhost:2181)] zookeeper.ClientCnxn: Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
2018-07-16 20:09:20,001 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.ZooKeeperServer: Client attempting to establish new session at /0:0:0:0:0:0:0:1:52389
2018-07-16 20:09:20,002 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x164a2feb4680005 with negotiated timeout 10000 for client /0:0:0:0:0:0:0:1:52389
2018-07-16 20:09:20,003 INFO [master:37346.activeMasterManager-SendThread(localhost:2181)] zookeeper.ClientCnxn: Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x164a2feb4680005, negotiated timeout = 10000
2018-07-16 20:09:20,007 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x164a2feb4680005 type:create cxid:0x1 zxid:0xad txntype:-1 reqpath:n/a Error Path:/hbase/replication/rs Error:KeeperErrorCode = NodeExists for /hbase/replication/rs
2018-07-16 20:09:20,018 INFO [master:37346.activeMasterManager] master.ServerManager: Waiting for region servers count to settle; currently checked in 0, slept for 0 ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval of 1500 ms.
2018-07-16 20:09:20,078 INFO [B.defaultRpcServer.handler=5,queue=2,port=37346] master.ServerManager: Registering server=master,51571,1531742959688
2018-07-16 20:09:20,088 INFO [RS:0;master:51571] hfile.CacheConfig: Allocating LruBlockCache size=97.47 MB, blockSize=64 KB
2018-07-16 20:09:20,097 INFO [RS:0;master:51571] hfile.CacheConfig: Created cacheConfig: blockCache=LruBlockCache{blockCount=0, currentSize=60888, freeSize=102149056, maxSize=102209944, heapSize=60888, minSize=97099448, minFactor=0.95, multiSize=48549724, multiFactor=0.5, singleSize=24274862, singleFactor=0.25}, cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false
2018-07-16 20:09:20,123 INFO [RS:0;master:51571] wal.WALFactory: Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.DefaultWALProvider
2018-07-16 20:09:20,123 INFO [master:37346.activeMasterManager] master.ServerManager: Waiting for region servers count to settle; currently checked in 1, slept for 105 ms, expecting minimum of 1, maximum of 2147483647, timeout of 4500 ms, interval of 1500 ms.
2018-07-16 20:09:20,139 INFO [RS:0;master:51571] wal.FSHLog: WAL configuration: blocksize=32 MB, rollsize=30.40 MB, prefix=master%2C51571%2C1531742959688.default, suffix=, logDir=file:/usr/local/app/hbase-1.2/data/WALs/master,51571,1531742959688, archiveDir=file:/usr/local/app/hbase-1.2/data/oldWALs

2018-07-16 20:09:24,981 INFO [master:37346.activeMasterManager] master.RegionStates: Transition {d196d9d271d0ab1b388757dd95e061c9 state=OPEN, ts=1531742964981, server=master,40181,1531740756070} to {d196d9d271d0ab1b388757dd95e061c9 state=OFFLINE, ts=1531742964981, server=master,40181,1531740756070}
regionserver.HRegionServer: master,51571,1531742959688-MemstoreFlusherChore requesting flush of hbase:meta,,1.1588230740 because info has an old edit so flush to free WALs after random delay 30410ms

ree WALs after random delay 42302ms
2018-07-16 20:18:15,580 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 1/1 column families, memstore=3.89 KB
2018-07-16 20:18:15,612 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed, sequenceid=207, memsize=3.9 K, hasBloomFilter=false, into tmp file file:/usr/local/app/hbase-1.2/data/data/hbase/meta/1588230740/.tmp/e4ad28e038974eed91b4ce9b9b3ddb0a
2018-07-16 20:29:20,097 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=1.30 MB, freeSize=96.18 MB, max=97.47 MB, blockCount=27, accesses=58, hits=31, hitRatio=53.45%, , cachingAccesses=58, cachingHits=31, cachingHitsRatio=53.45%, evictions=119, evicted=0, evictedPerRun=0.0

2018-07-16 20:29:57,015 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x164a2feb4680007 with negotiated timeout 40000 for client /192.168.204.1:53452
2018-07-16 20:30:06,968 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x164a2feb4680007
e018-07-16 20:30:06,970 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxn: Closed socket connection for client /192.168.204.1:53452 which had sessionid 0x164a2feb4680007

  • 写回答

6条回答

  • 可以简单点 2018-07-17 02:15
    关注

    真正耗时的地方就是连接hbase的那一段代码

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
查看更多回答(5条)

报告相同问题?

悬赏问题

  • ¥15 如何在3D高斯飞溅的渲染的场景中获得一个可控的旋转物体
  • ¥88 实在没有想法,需要个思路
  • ¥15 MATLAB报错输入参数太多
  • ¥15 python中合并修改日期相同的CSV文件并按照修改日期的名字命名文件
  • ¥15 有赏,i卡绘世画不出
  • ¥15 如何用stata画出文献中常见的安慰剂检验图
  • ¥15 c语言链表结构体数据插入
  • ¥40 使用MATLAB解答线性代数问题
  • ¥15 COCOS的问题COCOS的问题
  • ¥15 FPGA-SRIO初始化失败