hbase启动几十秒后,HMaster和HRegionserver自动关闭

采用hbase自带的zookeeper。
hbase-site.xml配置如下:


hbase.rootdir
hdfs://master:9000/hbase
hadoop集群地址



hbase.cluster.distributed
true
是否启动集群模式


hbase.tmp.dir
/root/hbase-1.2.4/tmp


hbase.master #指定hbase集群主控节点
master:60000


hbase.zookeeper.quorum
slave1,slave2,slave3
zookeeper集群主机名列表


hbase-root-master-master.log部分日志如下:
2016-12-26 09:23:53,703 INFO [main-SendThread(slave2:2181)] zookeeper.ClientCnxn: Socket connection established to slave2/192.168.241.237:2181, initiating session
2016-12-26 09:23:53,707 INFO [main-SendThread(slave2:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
2016-12-26 09:23:54,117 INFO [main-SendThread(slave3:2181)] zookeeper.ClientCnxn: Opening socket connection to server slave3/192.168.241.238:2181. Will not attempt to authenticate using SASL (unknown error)
2016-12-26 09:23:54,123 INFO [main-SendThread(slave3:2181)] zookeeper.ClientCnxn: Socket connection established to slave3/192.168.241.238:2181, initiating session
2016-12-26 09:23:54,125 INFO [main-SendThread(slave3:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
2016-12-26 09:23:56,025 INFO [main-SendThread(slave1:2181)] zookeeper.ClientCnxn: Opening socket connection to server slave1/192.168.241.236:2181. Will not attempt to authenticate using SASL (unknown error)
2016-12-26 09:23:56,027 INFO [main-SendThread(slave1:2181)] zookeeper.ClientCnxn: Socket connection established to slave1/192.168.241.236:2181, initiating session
2016-12-26 09:23:56,030 INFO [main-SendThread(slave1:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
2016-12-26 09:23:56,133 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper create failed after 4 attempts
2016-12-26 09:23:56,134 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster.
at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2426)
at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:231)
at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:137)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2436)
Caused by: org.apache.hadoop.hbase.ZooKeeperConnectionException: master:160000x0, quorum=slave1:2181,slave2:2181,slave3:2181, baseZNode=/hbase Unexpected KeeperException creating base node
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:206)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.(ZooKeeperWatcher.java:187)
at org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:585)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:381)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2419)
... 5 more
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:565)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:544)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1204)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1182)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:194)
... 13 more

3个回答

已解决,/etc/hosts配置错误
每台虚拟机上都进行如下配置
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.241.235 master
192.168.241.236 slave1
192.168.241.237 slave2
192.168.241.238 slave3

Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
其他相关推荐
Hbase开启后hmaster闪退!Hregionserver可以开启无法关闭!
2015-04-27 10:22:22,023 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, conn ectString=slave1:2181,master:2181,slave2:2181 sessionTimeout=180000 watcher=master:600000x0 2015-04-27 10:22:22,848 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to serv er slave2/10.200.0.232:2181. Will not attempt to authenticate using SASL (无法定位登录配置) 2015-04-27 10:22:22,889 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to slave2/10.200.0.232:2181, initiating session 2015-04-27 10:22:22,939 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server slave2/10.200.0.232:2181, sessionid = 0x34cf8ad10320000, negotiated timeout = 40000 2015-04-27 10:22:23,198 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server Responder: starting 2015-04-27 10:22:23,223 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: s tarting 2015-04-27 10:22:23,645 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 60000: starting 2015-04-27 10:22:23,646 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60000: starting 2015-04-27 10:22:23,648 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 60000: starting 2015-04-27 10:22:23,649 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 60000: starting 2015-04-27 10:22:23,807 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 60000: starting 2015-04-27 10:22:23,850 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 60000: starting 2015-04-27 10:22:23,871 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 60000: starting 2015-04-27 10:22:23,874 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 60000: starting 2015-04-27 10:22:23,903 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 60000: starting 2015-04-27 10:22:23,920 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 60000: starting 2015-04-27 10:22:23,973 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 1 on 60 000: starting 2015-04-27 10:22:24,016 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 0 on 60 000: starting 2015-04-27 10:22:24,017 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 2 on 60 000: starting 2015-04-27 10:22:24,142 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=Master, sessionId=master,60000,1430101339946 2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: revision 2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsUser 2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsDate 2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsUrl 2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: date 2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsRevision 2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: user 2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsVersion 2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: url 2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: MetricsString added: version 2015-04-27 10:22:24,284 INFO org.apache.hadoop.hbase.metrics: new MBeanInfo 2015-04-27 10:22:24,288 INFO org.apache.hadoop.hbase.metrics: new MBeanInfo 2015-04-27 10:22:24,288 INFO org.apache.hadoop.hbase.master.metrics.MasterMetrics: Initialized 2015-04-27 10:22:24,827 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org. mortbay.log) via org.mortbay.log.Slf4jLog 2015-04-27 10:22:25,014 INFO org.apache.hadoop.http.HttpServer: Added global filtersafety (clas s=org.apache.hadoop.http.HttpServer$QuotingInputFilter) 2015-04-27 10:22:25,075 INFO org.apache.hadoop.http.HttpServer: Port returned by webServer.getC onnectors()[0].getLocalPort() before open() is -1. Opening the listener on 60010 2015-04-27 10:22:25,077 INFO org.apache.hadoop.http.HttpServer: listener.getLocalPort() returne d 60010 webServer.getConnectors()[0].getLocalPort() returned 60010 2015-04-27 10:22:25,077 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 60010 2015-04-27 10:22:25,077 INFO org.mortbay.log: jetty-6.1.26 2015-04-27 10:22:25,625 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:60010 2015-04-27 10:22:25,705 INFO org.apache.hadoop.hbase.master.ActiveMasterManager: Deleting ZNode for /hbase/backup-masters/master,60000,1430101339946 from backup master directory 2015-04-27 10:22:25,746 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /hbas e/backup-masters/master,60000,1430101339946 already deleted, and this is not a retry 2015-04-27 10:22:25,746 INFO org.apache.hadoop.hbase.master.ActiveMasterManager: Master=master, 60000,1430101339946 2015-04-27 10:22:26,154 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Star ting shutdown. org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client vers ion 4 at org.apache.hadoop.ipc.Client.call(Client.java:1070) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225) at $Proxy10.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379) at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:238) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:203) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:8 9) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187) at org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:674) at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:112) at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:573) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:433) at java.lang.Thread.run(Thread.java:662) 2015-04-27 10:22:26,155 INFO org.apache.hadoop.hbase.master.HMaster: Aborting 2015-04-27 10:22:26,155 DEBUG org.apache.hadoop.hbase.master.HMaster: Stopping service threads 2015-04-27 10:22:26,155 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60000 2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 60000: exiting 2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60000: exiting 2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 60000: exiting 2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 60000: exiting 2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 60000: exiting 2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 60000: exiting 2015-04-27 10:22:26,156 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 60000: exiting 2015-04-27 10:22:26,157 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 60000: exiting 2015-04-27 10:22:26,157 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 60000: exiting 2015-04-27 10:22:26,157 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 60000: exiting 2015-04-27 10:22:26,157 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 0 on 60 000: exiting 2015-04-27 10:22:26,157 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 1 on 60 000: exiting 2015-04-27 10:22:26,157 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 2 on 60 000: exiting 2015-04-27 10:22:26,158 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server listener on 60000 2015-04-27 10:22:26,160 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder 2015-04-27 10:22:26,160 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder 2015-04-27 10:22:26,161 INFO org.apache.hadoop.hbase.master.HMaster: Stopping infoServer 2015-04-27 10:22:26,170 INFO org.mortbay.log: Stopped SelectChannelConnector@0.0.0.0:60010 2015-04-27 10:22:26,224 INFO org.apache.zookeeper.ZooKeeper: Session: 0x34cf8ad10320000 closed 2015-04-27 10:22:26,224 INFO org.apache.hadoop.hbase.master.HMaster: HMaster main thread exitin g 2015-04-27 10:22:26,224 ERROR org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to star t master java.lang.RuntimeException: HMaster Aborted at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.jav a:160) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:104) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2129) 2015-04-27 10:22:26,224 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
关于hbase HMaster启动后自动关闭的问题
最近在学习hbase,按照教程配置好后,发现使用hbase-daemon.sh start master 启动后,HMaster进程一会自动退出,查看日志如下: 2015-10-25 04:38:32,322 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase 2015-10-25 04:38:32,322 ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 3 retries 2015-10-25 04:38:32,322 ERROR org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to start master java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2115) at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:104) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2129) Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1069) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:220) at org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:1111) at org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:1101) at org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:1085) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:164) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:157) at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:348) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2110) ... 5 more 请高手看看是什么原因,在网上找了很多解决办法,但还是无法解决。
hbase配置启动后,HMaster在jps出现几秒后消失
集群共1、2、3、4机器 zookeeper配置在2、3、4机器上,hbase的Master设置为1机器,regionserver设置为2、3、4. 启动后发现regionserver正常启动,但是Hmaster出现几秒后消失,启动失败。 查看日志发现是拒绝连接。 ``` java.net.ConnectException: Call From node01/192.168.216.101 to node01:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ``` 有没有碰见过这种情况的,怎么解决啊。
hbase启动后hmaster进程消失的问题
hadoop-2.4.1+zookeeper-3.4.5(非hbase自带)+hbase-0.96.2-hadoop2 无论执行什么命令,都报一下错: hbase(main):001:0> status ERROR: Can't get master address from ZooKeeper; znode data == null 日志文件 hbase-root-master-master.log: 2014-09-05 10:09:59,705 FATAL [master:master:60000] master.HMaster: Unhandled exception. Starting shutdown. java.net.UnknownHostException: Invalid host name: local host is: (unknown); destination host is: "master":9000; java.net.UnknownHostException; 2014-09-05 10:10:00,707 ERROR [main] master.HMasterCommandLine: Master exiting java.lang.RuntimeException: HMaster Aborted 配置文件 hbase-site.xml <configuration> <property> <name>hbase.rootdir</name> <value>hdfs://master:9000/hbase</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>hbase.master</name> <value>master:60000</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>master,slave1,slave2</value> </property> </configuration> 配置文件 zoo.cfg # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/root/SoftWare/zkdata # the port at which the clients will connect clientPort=2181 server.1=master:2888:3888 server.2=slave1:2888:3888 server.3=slave2:2888:3888 # # Be sure to read the maintenance section of the # administrator guide before turning on autopurge. # # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance # # The number of snapshots to retain in dataDir #autopurge.snapRetainCount=3 # Purge task interval in hours # Set to "0" to disable auto purge feature #autopurge.purgeInterval=1 配置文件 core-site.xml <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://master:9000</value> </property> <property> <name>hadoop.tmp.dir</name> <value>file:///home/hadoop/tmp</value> </property> <property> <name>io.file.buffer.size</name> <value>131072</value> </property> </configuration>
hbase 启动后 Jps, HMaster出现又消失
WARN org.apache.zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, cl osing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) 2015-08-09 19:50:02,232 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server Slave1.hadoop /10.37.44.122:2181. Will not attempt to authenticate using SASL (Unable to locate a login configuration) 2015-08-09 19:50:02,237 WARN org.apache.zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, cl osing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
hmaster 启动以后自动关闭,快疯了。
先启动hadoop , 2739 JobHistoryServer 2454 NameNode 2630 NodeManager 5019 Jps 2508 DataNode 2573 ResourceManager 2685 ApplicationHistoryServer 再启动hbase hmaster 启动以后,马上就自动关闭了,什么方法都试过了就是不行。疯快了 hadoop 重装,hbase 重装能想的招都使了,就是不行 at org.apache.hadoop.hbase.master.handler.ServerShutdownHandler.process(ServerShutdownHandler.java:183) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2015-07-25 02:39:31,159 DEBUG [master:CentOS002:60000] catalog.CatalogTracker: Stopping catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@fbf7eda 2015-07-25 02:39:31,159 INFO [master:CentOS002:60000] client.HConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x14ec49496dc0003 2015-07-25 02:39:31,161 INFO [master:CentOS002:60000] zookeeper.ZooKeeper: Session: 0x14ec49496dc0003 closed 2015-07-25 02:39:31,161 INFO [master:CentOS002:60000-EventThread] zookeeper.ClientCnxn: EventThread shut down 2015-07-25 02:39:31,169 INFO [CentOS002,60000,1437817155951.splitLogManagerTimeoutMonitor] master.SplitLogManager$TimeoutMonitor: CentOS002,60000,1437817155951.splitLogManagerTimeoutMonitor exiting 2015-07-25 02:39:31,172 INFO [master:CentOS002:60000] zookeeper.ZooKeeper: Session: 0x14ec49496dc0001 closed 2015-07-25 02:39:31,172 INFO [master:CentOS002:60000] master.HMaster: HMaster main thread exiting 2015-07-25 02:39:31,172 INFO [main-EventThread] zookeeper.ClientCnxn: EventThread shut down 2015-07-25 02:39:31,172 ERROR [main] master.HMasterCommandLine: Master exiting java.lang.RuntimeException: HMaster Aborted at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:201) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:135) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3047)
hbase 启动后自动退出
日志 如下 2017-11-22 15:59:03,632 INFO [main] util.ServerCommandLine: env:CVS_RSH=ssh 2017-11-22 15:59:03,632 INFO [main] util.ServerCommandLine: env:HBASE_MANAGES_ZK=false 2017-11-22 15:59:03,632 INFO [main] util.ServerCommandLine: env:G_BROKEN_FILENAMES=1 2017-11-22 15:59:03,632 INFO [main] util.ServerCommandLine: env:HBASE_NICENESS=0 2017-11-22 15:59:03,632 INFO [main] util.ServerCommandLine: env:HBASE_REST_OPTS= 2017-11-22 15:59:03,632 INFO [main] util.ServerCommandLine: env:NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) at com.sun.proxy.$Proxy15.getFileInfo(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hbase.fs.HFileSystem$1.invoke(HFileSystem.java:279) at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1977) at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1118) at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1114) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1114) at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1400) at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:436) at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:153) at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:128) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:693) at org.apache.hadoop.hbase.master.HMaster.access$600(HMaster.java:189) at org.apache.hadoop.hbase.master.HMaster$2.run(HMaster.java:1803) at java.lang.Thread.run(Thread.java:745) 2017-11-30 15:30:29,684 INFO [master1:16000.activeMasterManager] regionserver.HRegionServer: STOPPED: Unhandled exception. Starting shutdown. 2017-11-30 15:30:32,555 INFO [master/master1/172.17.153.117:16000] ipc.RpcServer: Stopping server on 16000 2017-11-30 15:30:32,555 INFO [RpcServer.listener,port=16000] ipc.RpcServer: RpcServer.listener,port=16000: stopping 2017-11-30 15:30:32,556 INFO [master/master1/172.17.153.117:16000] regionserver.HRegionServer: Stopping infoServer 2017-11-30 15:30:32,557 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2017-11-30 15:30:32,557 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2017-11-30 15:30:32,565 INFO [master/master1/172.17.153.117:16000] mortbay.log: Stopped SelectChannelConnector@0.0.0.0:6123 2017-11-30 15:30:32,565 INFO [master/master1/172.17.153.117:16000] regionserver.HRegionServer: stopping server master1,16000,1512027026091 2017-11-30 15:30:32,565 INFO [master/master1/172.17.153.117:16000] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x15fe8b9c7df000a 2017-11-30 15:30:32,570 INFO [master/master1/172.17.153.117:16000-EventThread] zookeeper.ClientCnxn: EventThread shut down 2017-11-30 15:30:32,570 INFO [master/master1/172.17.153.117:16000] zookeeper.ZooKeeper: Session: 0x15fe8b9c7df000a closed 2017-11-30 15:30:32,572 INFO [master/master1/172.17.153.117:16000] regionserver.HRegionServer: stopping server master1,16000,1512027026091; all regions closed. 2017-11-30 15:30:32,573 INFO [master/master1/172.17.153.117:16000] hbase.ChoreService: Chore service for: master1,16000,1512027026091 had [] on shutdown 2017-11-30 15:30:32,578 INFO [master/master1/172.17.153.117:16000] ipc.RpcServer: Stopping server on 16000 2017-11-30 15:30:32,587 INFO [main-EventThread] zookeeper.ClientCnxn: EventThread shut down 2017-11-30 15:30:32,587 INFO [master/master1/172.17.153.117:16000] zookeeper.ZooKeeper: Session: 0x45fe8b9c7260005 closed 2017-11-30 15:30:32,587 INFO [master/master1/172.17.153.117:16000] regionserver.HRegionServer: stopping server master1,16000,1512027026091; zookeeper connection closed. 配置文件 <configuration> <property> <name>hbase.rootdir</name> <value>hdfs://172.17.153.117:8020/hbase</value> </property> <property> <name>hbase.tmp.dir</name> <value>file:/root/app/hbase-1.0.0/tmp</value> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>172.17.153.117,master2,slave1,slave2,slave3</value> </property> <property> <name>hbase.zookeeper.property.clientPort</name> <value>2181</value> </property> <property> <name>hbase.zookeeper.property.dataDir</name> <value>/root/app/zookeeper-3.4.5/var/data</value> </property> <property> <name>hbase.master.info.port</name> <value>6123</value> <description>The port the HBase Master should bind to.</description> </property> </configuration>
Hbase安装出现错误Hmaster节点无法启动
以下错误日志输出详情: java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:3048) at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:193) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:135) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3062) Caused by: java.lang.NoClassDefFoundError: com/amazonaws/AmazonServiceException at java.lang.Class.getDeclaredConstructors0(Native Method) at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) at java.lang.Class.getConstructor0(Class.java:3075) at java.lang.Class.newInstance(Class.java:412) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2563) at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2574) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) at org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:943) at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:533) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:3043) ... 5 more Caused by: java.lang.ClassNotFoundException: com.amazonaws.AmazonServiceException at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 27 more 2015年 12月 09日 星期三 19:11:35 CST Stopping hbase (via master) 求大师解答,在线等!!小弟急
hbase 启动,查看日志文件出现错误 这是为什么?
ERROR org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to start master java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2115) at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:104) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2129) Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for / hbase at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
hbase启动报错问题,求解
hbase集群启动通过ssh启动会爆下面这个错 java.lang.RuntimeException: Failed construction of Regionserver: class org.apache.hadoop.hbase.regionserver.HRegionServer at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2706) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:64) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2721) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:408) at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2704) ... 5 more Caused by: java.io.IOException: No FileSystem for scheme: hdfs at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2799) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2810) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:100) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2849) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2831) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:389) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356) at org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:1003) at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:579) 通过自定义脚本或者 bin/start-hbase.sh 启动会报错,但通过$ bin/hbase-daemon.sh start master $ bin/hbase-daemon.sh start regionserver 这命令单独在各个机子上运行则不报,求解!!!!
hbase启动报错 hbase shell
请大神帮帮忙 hbase启动 OK hbase shell有下问题 # ./bin/hbase shell 2016-04-05 08:53:06,328 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts 2016-04-05 08:53:06,331 WARN [main] zookeeper.ZKUtil: hconnection-0x1f6917fb0x0, quorum=salve1:2181,master:2181,salve2:2181, baseZNode=/hbase Unable to set watcher on znode (/hbase/hbaseid) org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:221) at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:482) at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:833) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:623) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450) at org.jruby.javasupport.JavaMethod.invokeStaticDirect(JavaMethod.java:362) at org.jruby.java.invokers.StaticMethodInvoker.call(StaticMethodInvoker.java:58) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169) at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57) at org.jruby.ast.InstAsgnNode.interpret(InstAsgnNode.java:95) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169) at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302) at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:148) at org.jruby.RubyClass.newInstance(RubyClass.java:822) at org.jruby.RubyClass$i$newInstance.call(RubyClass$i$newInstance.gen:65535) at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrNBlock.call(JavaMethod.java:249) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135) at usr.local.hadoop.hbase_minus_1_dot_0_dot_3.bin.hirb.__file__(/usr/local/hadoop/hbase-1.0.3/bin/hirb.rb:118) at usr.local.hadoop.hbase_minus_1_dot_0_dot_3.bin.hirb.load(/usr/local/hadoop/hbase-1.0.3/bin/hirb.rb) at org.jruby.Ruby.runScript(Ruby.java:697) at org.jruby.Ruby.runScript(Ruby.java:690) at org.jruby.Ruby.runNormally(Ruby.java:597) at org.jruby.Ruby.runFromMain(Ruby.java:446) at org.jruby.Main.doRunFromMain(Main.java:369) at org.jruby.Main.internalRun(Main.java:258) at org.jruby.Main.run(Main.java:224) at org.jruby.Main.run(Main.java:208) at org.jruby.Main.main(Main.java:188) 2016-04-05 08:53:06,338 ERROR [main] zookeeper.ZooKeeperWatcher: hconnection-0x1f6917fb0x0, quorum=salve1:2181,master:2181,salve2:2181, baseZNode=/hbase Received unexpected KeeperException, re-throwing exception org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1045) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:221) at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:482) at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:833) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:623) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:422) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450) at org.jruby.javasupport.JavaMethod.invokeStaticDirect(JavaMethod.java:362) at org.jruby.java.invokers.StaticMethodInvoker.call(StaticMethodInvoker.java:58) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169) at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57) at org.jruby.ast.InstAsgnNode.interpret(InstAsgnNode.java:95) at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104) at org.jruby.ast.BlockNode.interpret(BlockNode.java:71) at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74) at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169) at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302) at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:148) at org.jruby.RubyClass.newInstance(RubyClass.java:822) at org.jruby.RubyClass$i$newInstance.call(RubyClass$i$newInstance.gen:65535) at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrNBlock.call(JavaMethod.java:249) at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292) at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135) at usr.local.hadoop.hbase_minus_1_dot_0_dot_3.bin.hirb.__file__(/usr/local/hadoop/hbase-1.0.3/bin/hirb.rb:118) at usr.local.hadoop.hbase_minus_1_dot_0_dot_3.bin.hirb.load(/usr/local/hadoop/hbase-1.0.3/bin/hirb.rb) at org.jruby.Ruby.runScript(Ruby.java:697) at org.jruby.Ruby.runScript(Ruby.java:690) at org.jruby.Ruby.runNormally(Ruby.java:597) at org.jruby.Ruby.runFromMain(Ruby.java:446) at org.jruby.Main.doRunFromMain(Main.java:369) at org.jruby.Main.internalRun(Main.java:258) at org.jruby.Main.run(Main.java:224) at org.jruby.Main.run(Main.java:208) at org.jruby.Main.main(Main.java:188) HBase Shell; enter 'help<RETURN>' for list of supported commands. Type "exit<RETURN>" to leave the HBase Shell Version 1.0.3, rf1e1312f9790a7c40f6a4b5a1bab2ea1dd559890, Tue Jan 19 19:26:53 PST 2016
HMaster每天都自动挂掉,求大神指点
最近遇到一个比较头疼的问题,HBase每天都会自动挂掉一次,时间大概在5:30-5:45之间,做了几种尝试 1. 检查host配置。 2. 检查时钟同步。 3. 设置会话超时时间为60s #####HMaster的出错日志如下:##### ``` 2015-09-21 05:32:20,463 INFO [main-SendThread(132.37.5.197:29184)] zookeeper.ClientCnxn: Socket connection established to 132.37.5.197/132.37.5.197:29184, initiating session 2015-09-21 05:32:20,465 FATAL [main-EventThread] master.HMaster: Master server abort: loaded coprocessors are: [] 2015-09-21 05:32:20,465 INFO [main-SendThread(132.37.5.197:29184)] zookeeper.ClientCnxn: Unable to reconnect to ZooKeeper service, session 0x24f1f7bb79103a9 has expired, closing socket connection 2015-09-21 05:32:20,465 FATAL [main-EventThread] master.HMaster: master:60900-0x24f1f7bb79103a9, quorum=132.37.5.196:29184,132.37.5.195:29184,132.37.5.197:29184, baseZNode=/hbase master:60900-0x24f1f7bb79103a9 received expired from ZooKeeper, aborting org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:417) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:328) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) 2015-09-21 05:32:20,466 INFO [main-EventThread] regionserver.HRegionServer: STOPPED: master:60900-0x24f1f7bb79103a9, quorum=132.37.5.196:29184,132.37.5.195:29184,132.37.5.197:29184, baseZNode=/hbase master:60900-0x24f1f7bb79103a9 received expired from ZooKeeper, aborting 2015-09-21 05:32:20,466 INFO [main-EventThread] zookeeper.ClientCnxn: EventThread shut down 2015-09-21 05:32:20,466 INFO [master/pkgtstdb2/132.37.5.194:60900] regionserver.HRegionServer: Stopping infoServer 2015-09-21 05:32:20,468 INFO [master/pkgtstdb2/132.37.5.194:60900] mortbay.log: Stopped SelectChannelConnector@0.0.0.0:60910 2015-09-21 05:32:20,570 INFO [master/pkgtstdb2/132.37.5.194:60900] regionserver.HRegionServer: stopping server pkgtstdb2,60900,1442548707194 2015-09-21 05:32:20,570 INFO [master/pkgtstdb2/132.37.5.194:60900] client.ConnectionManager$HConnectionImplementation: Closing master protocol: MasterService 2015-09-21 05:32:20,570 INFO [master/pkgtstdb2/132.37.5.194:60900] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x24f1f7bb79103ad 2015-09-21 05:32:20,572 INFO [master/pkgtstdb2/132.37.5.194:60900] zookeeper.ZooKeeper: Session: 0x24f1f7bb79103ad closed 2015-09-21 05:32:20,573 INFO [master/pkgtstdb2/132.37.5.194:60900-EventThread] zookeeper.ClientCnxn: EventThread shut down 2015-09-21 05:32:20,573 INFO [master/pkgtstdb2/132.37.5.194:60900] regionserver.HRegionServer: stopping server pkgtstdb2,60900,1442548707194; all regions closed. 2015-09-21 05:32:20,573 INFO [CatalogJanitor-pkgtstdb2:60900] master.CatalogJanitor: CatalogJanitor-pkgtstdb2:60900 exiting 2015-09-21 05:32:20,574 WARN [master/pkgtstdb2/132.37.5.194:60900] zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=132.37.5.196:29184,132.37.5.195:29184,132.37.5.197:29184, exception=org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/master 2015-09-21 05:32:20,574 INFO [pkgtstdb2:60900.oldLogCleaner] cleaner.LogCleaner: pkgtstdb2:60900.oldLogCleaner exiting ``` HBase的配置文件如下: ``` <configuration> <property> <name>hbase.rootdir</name> <value>hdfs://gxuweg3tst2:8920/wa</value> </property> <property> <name>hbase.master.port</name> <value>60900</value> <description>The port the HBase Master should bind to.</description> </property> <property> <name>hbase.cluster.distributed</name> <value>true</value> <description>The mode the cluster will be in. Possible values are false for standalone mode and true for distributed mode. If false, startup will run all HBase and ZooKeeper daemons together in the one JVM. </description> </property> <property> <name>hbase.tmp.dir</name> <!-- <value>/tmp/hbase-${user.name}</value> --> <value>/uniiof/users/devdpp01/hbase/tmp</value> <description>Temporary directory on the local filesystem. Change this setting to point to a location more permanent than '/tmp' (The '/tmp' directory is often cleared on machine restart). </description> </property> <property> <name>hbase.master.info.port</name> <value>60910</value> <description>The port for the HBase Master web UI. Set to -1 if you do not want a UI instance run. </description> </property> <property> <name>hbase.regionserver.port</name> <value>60920</value> <description>The port the HBase RegionServer binds to. </description> </property> <property> <name>hbase.regionserver.info.port</name> <value>60930</value> <description>The port for the HBase RegionServer web UI Set to -1 if you do not want the RegionServer UI to run. </description> </property> <!-- The following three properties are used together to create the list of host:peer_port:leader_port quorum servers for ZooKeeper. --> <property> <name>hbase.zookeeper.quorum</name> <value>132.37.5.195,132.37.5.196,132.37.5.197</value> <description>Comma separated list of servers in the ZooKeeper Quorum. For example, "host1.mydomain.com,host2.mydomain.com,host3.mydomain.com". By default this is set to localhost for local and pseudo-distributed modes of operation. For a fully-distributed setup, this should be set to a full list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in hbase-env.sh this is the list of servers which we will start/stop ZooKeeper on. </description> </property> <property> <name>hbase.zookeeper.peerport</name> <value>29888</value> <description>Port used by ZooKeeper peers to talk to each other. See http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperStarted.html#sc_RunningReplicatedZoo Keeper for more information. </description> </property> <property> <name>hbase.zookeeper.leaderport</name> <value>39888</value> <description>Port used by ZooKeeper for leader election. See http://hadoop.apache.org/zookeeper/docs/r3.1.1/zookeeperStarted.html#sc_RunningReplicatedZoo Keeper for more information. </description> </property> <!-- End of properties used to generate ZooKeeper host:port quorum list. --> <property> <name>hbase.zookeeper.property.clientPort</name> <value>29184</value> <description>Property from ZooKeeper's config zoo.cfg. The port at which the clients will connect. </description> </property> <!-- End of properties that are directly mapped from ZooKeeper's zoo.cfg --> <property> <name>hbase.rest.port</name> <value>8980</value> <description>The port for the HBase REST server.</description> </property> </configuration> ```
Ambari2.4.2 Hbase启动失败
HBase Master Process Connection failed: [Errno 111] Connection refused to bi.corp.mama.cn2:16000 启动过程是没有报错的,都是Success,但是alter提示这个,过一会儿本来Start的HBase Master就自动Stop,关请教一下有遇到同样问题的朋友吗?
Hbase Regionserver启动出错
replace_conf_dir + find /var/run/cloudera-scm-agent/process/469-hbase-REGIONSERVER -type f '!' -path '/var/run/cloudera-scm-agent/process/469-hbase-REGIONSERVER/logs/*' '!' -name '*.log' '!' -name '*.keytab' '!' -name '*jceks' -exec perl -pi -e 's#{{CMF_CONF_DIR}}#/var/run/cloudera-scm-agent/process/469-hbase-REGIONSERVER#g' '{}' ';' **Can't open /var/run/cloudera-scm-agent/process/469-hbase-REGIONSERVER/supervisor.conf: Permission denied. **+ acquire_kerberos_tgt hbase.keytab + '[' -z hbase.keytab ']' + '[' -n '' ']' + export 'HBASE_OPTS=-Djava.net.preferIPv4Stack=true ' + HBASE_OPTS='-Djava.net.preferIPv4Stack=true ' + locate_hbase_script + '[' 5 -ge 5 ']' + export BIGTOP_DEFAULTS_DIR= + BIGTOP_DEFAULTS_DIR= + HBASE_BIN=/opt/cloudera/parcels/CDH-5.7.2-1.cdh5.7.2.p0.18/lib/hbase/../../bin/hbase + '[' upgrade = regionserver ']' + '[' region_mover = regionserver ']' + '[' toggle_balancer = regionserver ']' + '[' shell = regionserver ']' + '[' hfileCheck = regionserver ']' + '[' remoteSnapshotTool = regionserver ']' + '[' '' '!=' '' ']' + '[' regionserver = regionserver -a -n '' ']' + '[' start = start -a 5 -gt 4 ']' + '[' regionserver = regionserver -o master = regionserver ']' + export HBASE_ZNODE_FILE=/var/run/cloudera-scm-agent/process/469-hbase-REGIONSERVER/znode59836 + HBASE_ZNODE_FILE=/var/run/cloudera-scm-agent/process/469-hbase-REGIONSERVER/znode59836 + exec /opt/cloudera/parcels/CDH-5.7.2-1.cdh5.7.2.p0.18/lib/hbase/../../bin/hbase --config /var/run/cloudera-scm-agent/process/469-hbase-REGIONSERVER regionserver start + znode_cleanup regionserver ++ date + echo 'Thu Aug 10 15:01:12 CST 2017 Starting znode cleanup thread with HBASE_ZNODE_FILE=/var/run/cloudera-scm-agent/process/469-hbase-REGIONSERVER/znode59836 for regionserver' ++ replace_pid -Djava.net.preferIPv4Stack=true ++ sed 's#{{PID}}#59836#g' ++ echo -Djava.net.preferIPv4Stack=true + HBASE_OPTS=-Djava.net.preferIPv4Stack=true + '[' '' '!=' '' ']' + LOG_FILE=/var/run/cloudera-scm-agent/process/469-hbase-REGIONSERVER/logs/znode_cleanup.log + set +x
java连hbase永久等待问题
只有一个有价值信息就是,Session establishment complete on server tb86-es01/172.18.1.86:2181, sessionid = 0x15b79d49b158594, negotiated timeout = 40000 代码如下: public class extract_hbase_job { private static Configuration hconf = null; static{ Configuration conf = new Configuration(); conf.set("hbase.zookeeper.quorum", "tb86-es01,tb87-es02,tb88-es03"); conf.set("hbase.zookeeper.property.clientPort", "2181"); System.setProperty("hadoop.home.dir", "D:\\work\\apache\\hadoop\\hadoop-2.6.0"); hconf = HBaseConfiguration.create(conf); } public static void getRecord(String tableName,String rowKey) throws Exception{ HTable table = new HTable(hconf, tableName.getBytes()); Get get = new Get(rowKey.getBytes()); Result rs = table.get(get); for (KeyValue kv : rs.raw()) { System.out.println(new String(kv.getQualifier()) + " " ); } @Test public void test() throws Exception{ extract_hbase_job.getRecord("default:relation_graph", "00:global:cp:10878983"); } } hbase中也有报错: java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1867) at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:198) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:139) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:1881) ed by: java.net.BindException: 地址已在使用 at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:444) at sun.nio.ch.Net.bind(Net.java:436) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:214) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2366) at org.apache.hadoop.hbase.ipc.RpcServer$Listener.<init>(RpcServer.java:524) at org.apache.hadoop.hbase.ipc.RpcServer.<init>(RpcServer.java:1896) at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:788) at org.apache.hadoop.hbase.master.MasterRpcServices.<init>(MasterRpcServices.java:197) at org.apache.hadoop.hbase.master.HMaster.createRpcServices(HMaster.java:401) at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:487) at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:271) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:1862) ... 5 more
hbase shell 启动不了,求帮忙
安装了hbase 1.3.1 hadoop 2.8.2 jdk 9.0.1 hbase 启动没问题,但当启动 hbase shell 错误如下: root@Master bin]# ./hbase shell Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.jruby.java.invokers.RubyToJavaInvoker (file:/usr/local/hadoop/hbase-1.2.6/lib/jruby-complete-1.6.8.jar) to method java.lang.Object.registerNatives() WARNING: Please consider reporting this to the maintainers of org.jruby.java.invokers.RubyToJavaInvoker WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release ArgumentError: wrong number of arguments (0 for 1) method_added at file:/usr/local/hadoop/hbase-1.2.6/lib/jruby-complete-1.6.8.jar!/builtin/javasupport/core_ext/object.rb:10 method_added at file:/usr/local/hadoop/hbase-1.2.6/lib/jruby-complete-1.6.8.jar!/builtin/javasupport/core_ext/object.rb:129 Pattern at file:/usr/local/hadoop/hbase-1.2.6/lib/jruby-complete-1.6.8.jar!/builtin/java/java.util.regex.rb:2 (root) at file:/usr/local/hadoop/hbase-1.2.6/lib/jruby-complete-1.6.8.jar!/builtin/java/java.util.regex.rb:1 require at org/jruby/RubyKernel.java:1062 (root) at file:/usr/local/hadoop/hbase-1.2.6/lib/jruby-complete-1.6.8.jar!/builtin/java/java.util.regex.rb:42 (root) at /usr/local/hadoop/hbase-1.2.6/bin/hirb.rb:38 而hirb.rb 第38行是 include JAVA, 而我的环境变量都配好的,不知道还有什么出错了。
hbase 的datanode老师挂掉,求解决
hbase启动之后datanode就挂掉,而且我的hbase启动之后无法新创建pid文件,因此无法关闭,启动完hbase之后 ,使用hbase shell之后用status 会报错![图片说明](https://img-ask.csdn.net/upload/201712/07/1512636854_641235.png)
Hbase 正常启动后,运行status命令的错误
client.HConnectionManager$HConnectionImplementation:The node /hbase is not in Zookeeper![运行status命令时的错误](https://img-ask.csdn.net/upload/201505/13/1431504319_531298.png)如图中所示的错误异常,我再hbase-site.xml中已经配置了如下图![在hbase-site.xml中的配置](https://img-ask.csdn.net/upload/201505/13/1431504468_206666.png)我的HBase是按单机模式配置的,其大神帮助啊
Kafka实战(三) - Kafka的自我修养与定位
Apache Kafka是消息引擎系统,也是一个分布式流处理平台(Distributed Streaming Platform) Kafka是LinkedIn公司内部孵化的项目。LinkedIn最开始有强烈的数据强实时处理方面的需求,其内部的诸多子系统要执行多种类型的数据处理与分析,主要包括业务系统和应用程序性能监控,以及用户行为数据处理等。 遇到的主要问题: 数据正确性不足 数据的收集主要...
volatile 与 synchronize 详解
Java支持多个线程同时访问一个对象或者对象的成员变量,由于每个线程可以拥有这个变量的拷贝(虽然对象以及成员变量分配的内存是在共享内存中的,但是每个执行的线程还是可以拥有一份拷贝,这样做的目的是加速程序的执行,这是现代多核处理器的一个显著特性),所以程序在执行过程中,一个线程看到的变量并不一定是最新的。 volatile 关键字volatile可以用来修饰字段(成员变量),就是告知程序任何对该变量...
Java学习的正确打开方式
在博主认为,对于入门级学习java的最佳学习方法莫过于视频+博客+书籍+总结,前三者博主将淋漓尽致地挥毫于这篇博客文章中,至于总结在于个人,实际上越到后面你会发现学习的最好方式就是阅读参考官方文档其次就是国内的书籍,博客次之,这又是一个层次了,这里暂时不提后面再谈。博主将为各位入门java保驾护航,各位只管冲鸭!!!上天是公平的,只要不辜负时间,时间自然不会辜负你。 何谓学习?博主所理解的学习,它是一个过程,是一个不断累积、不断沉淀、不断总结、善于传达自己的个人见解以及乐于分享的过程。
程序员必须掌握的核心算法有哪些?
由于我之前一直强调数据结构以及算法学习的重要性,所以就有一些读者经常问我,数据结构与算法应该要学习到哪个程度呢?,说实话,这个问题我不知道要怎么回答你,主要取决于你想学习到哪些程度,不过针对这个问题,我稍微总结一下我学过的算法知识点,以及我觉得值得学习的算法。这些算法与数据结构的学习大多数是零散的,并没有一本把他们全部覆盖的书籍。下面是我觉得值得学习的一些算法以及数据结构,当然,我也会整理一些看过...
有哪些让程序员受益终生的建议
从业五年多,辗转两个大厂,出过书,创过业,从技术小白成长为基层管理,联合几个业内大牛回答下这个问题,希望能帮到大家,记得帮我点赞哦。 敲黑板!!!读了这篇文章,你将知道如何才能进大厂,如何实现财务自由,如何在工作中游刃有余,这篇文章很长,但绝对是精品,记得帮我点赞哦!!!! 一腔肺腑之言,能看进去多少,就看你自己了!!! 目录: 在校生篇: 为什么要尽量进大厂? 如何选择语言及方...
大学四年自学走来,这些私藏的实用工具/学习网站我贡献出来了
大学四年,看课本是不可能一直看课本的了,对于学习,特别是自学,善于搜索网上的一些资源来辅助,还是非常有必要的,下面我就把这几年私藏的各种资源,网站贡献出来给你们。主要有:电子书搜索、实用工具、在线视频学习网站、非视频学习网站、软件下载、面试/求职必备网站。 注意:文中提到的所有资源,文末我都给你整理好了,你们只管拿去,如果觉得不错,转发、分享就是最大的支持了。 一、电子书搜索 对于大部分程序员...
linux系列之常用运维命令整理笔录
本博客记录工作中需要的linux运维命令,大学时候开始接触linux,会一些基本操作,可是都没有整理起来,加上是做开发,不做运维,有些命令忘记了,所以现在整理成博客,当然vi,文件操作等就不介绍了,慢慢积累一些其它拓展的命令,博客不定时更新 free -m 其中:m表示兆,也可以用g,注意都要小写 Men:表示物理内存统计 total:表示物理内存总数(total=used+free) use...
比特币原理详解
一、什么是比特币 比特币是一种电子货币,是一种基于密码学的货币,在2008年11月1日由中本聪发表比特币白皮书,文中提出了一种去中心化的电子记账系统,我们平时的电子现金是银行来记账,因为银行的背后是国家信用。去中心化电子记账系统是参与者共同记账。比特币可以防止主权危机、信用风险。其好处不多做赘述,这一层面介绍的文章很多,本文主要从更深层的技术原理角度进行介绍。 二、问题引入 假设现有4个人...
GitHub开源史上最大规模中文知识图谱
近日,一直致力于知识图谱研究的 OwnThink 平台在 Github 上开源了史上最大规模 1.4 亿中文知识图谱,其中数据是以(实体、属性、值),(实体、关系、实体)混合的形式组织,数据格式采用 csv 格式。 到目前为止,OwnThink 项目开放了对话机器人、知识图谱、语义理解、自然语言处理工具。知识图谱融合了两千五百多万的实体,拥有亿级别的实体属性关系,机器人采用了基于知识图谱的语义感...
程序员接私活怎样防止做完了不给钱?
首先跟大家说明一点,我们做 IT 类的外包开发,是非标品开发,所以很有可能在开发过程中会有这样那样的需求修改,而这种需求修改很容易造成扯皮,进而影响到费用支付,甚至出现做完了项目收不到钱的情况。 那么,怎么保证自己的薪酬安全呢? 我们在开工前,一定要做好一些证据方面的准备(也就是“讨薪”的理论依据),这其中最重要的就是需求文档和验收标准。一定要让需求方提供这两个文档资料作为开发的基础。之后开发...
网页实现一个简单的音乐播放器(大佬别看。(⊙﹏⊙))
今天闲着无事,就想写点东西。然后听了下歌,就打算写个播放器。 于是乎用h5 audio的加上js简单的播放器完工了。 演示地点演示 html代码如下` music 这个年纪 七月的风 音乐 ` 然后就是css`*{ margin: 0; padding: 0; text-decoration: none; list-...
微信支付崩溃了,但是更让马化腾和张小龙崩溃的竟然是……
loonggg读完需要3分钟速读仅需1分钟事件还得还原到昨天晚上,10 月 29 日晚上 20:09-21:14 之间,微信支付发生故障,全国微信支付交易无法正常进行。然...
Python十大装B语法
Python 是一种代表简单思想的语言,其语法相对简单,很容易上手。不过,如果就此小视 Python 语法的精妙和深邃,那就大错特错了。本文精心筛选了最能展现 Python 语法之精妙的十个知识点,并附上详细的实例代码。如能在实战中融会贯通、灵活使用,必将使代码更为精炼、高效,同时也会极大提升代码B格,使之看上去更老练,读起来更优雅。
数据库优化 - SQL优化
以实际SQL入手,带你一步一步走上SQL优化之路!
2019年11月中国大陆编程语言排行榜
2019年11月2日,我统计了某招聘网站,获得有效程序员招聘数据9万条。针对招聘信息,提取编程语言关键字,并统计如下: 编程语言比例 rank pl_ percentage 1 java 33.62% 2 cpp 16.42% 3 c_sharp 12.82% 4 javascript 12.31% 5 python 7.93% 6 go 7.25% 7 p...
通俗易懂地给女朋友讲:线程池的内部原理
餐盘在灯光的照耀下格外晶莹洁白,女朋友拿起红酒杯轻轻地抿了一小口,对我说:“经常听你说线程池,到底线程池到底是个什么原理?”
《奇巧淫技》系列-python!!每天早上八点自动发送天气预报邮件到QQ邮箱
将代码部署服务器,每日早上定时获取到天气数据,并发送到邮箱。 也可以说是一个小型人工智障。 知识可以运用在不同地方,不一定非是天气预报。
经典算法(5)杨辉三角
杨辉三角 是经典算法,这篇博客对它的算法思想进行了讲解,并有完整的代码实现。
英特尔不为人知的 B 面
从 PC 时代至今,众人只知在 CPU、GPU、XPU、制程、工艺等战场中,英特尔在与同行硬件芯片制造商们的竞争中杀出重围,且在不断的成长进化中,成为全球知名的半导体公司。殊不知,在「刚硬」的背后,英特尔「柔性」的软件早已经做到了全方位的支持与支撑,并持续发挥独特的生态价值,推动产业合作共赢。 而对于这一不知人知的 B 面,很多人将其称之为英特尔隐形的翅膀,虽低调,但是影响力却不容小觑。 那么,在...
腾讯算法面试题:64匹马8个跑道需要多少轮才能选出最快的四匹?
昨天,有网友私信我,说去阿里面试,彻底的被打击到了。问了为什么网上大量使用ThreadLocal的源码都会加上private static?他被难住了,因为他从来都没有考虑过这个问题。无独有偶,今天笔者又发现有网友吐槽了一道腾讯的面试题,我们一起来看看。 腾讯算法面试题:64匹马8个跑道需要多少轮才能选出最快的四匹? 在互联网职场论坛,一名程序员发帖求助到。二面腾讯,其中一个算法题:64匹...
面试官:你连RESTful都不知道我怎么敢要你?
干货,2019 RESTful最贱实践
刷了几千道算法题,这些我私藏的刷题网站都在这里了!
遥想当年,机缘巧合入了 ACM 的坑,周边巨擘林立,从此过上了"天天被虐似死狗"的生活… 然而我是谁,我可是死狗中的战斗鸡,智力不够那刷题来凑,开始了夜以继日哼哧哼哧刷题的日子,从此"读题与提交齐飞, AC 与 WA 一色 ",我惊喜的发现被题虐既刺激又有快感,那一刻我泪流满面。这么好的事儿作为一个正直的人绝不能自己独享,经过激烈的颅内斗争,我决定把我私藏的十几个 T 的,阿不,十几个刷题网...
为啥国人偏爱Mybatis,而老外喜欢Hibernate/JPA呢?
关于SQL和ORM的争论,永远都不会终止,我也一直在思考这个问题。昨天又跟群里的小伙伴进行了一番讨论,感触还是有一些,于是就有了今天这篇文。 声明:本文不会下关于Mybatis和JPA两个持久层框架哪个更好这样的结论。只是摆事实,讲道理,所以,请各位看官勿喷。 一、事件起因 关于Mybatis和JPA孰优孰劣的问题,争论已经很多年了。一直也没有结论,毕竟每个人的喜好和习惯是大不相同的。我也看...
白话阿里巴巴Java开发手册高级篇
不久前,阿里巴巴发布了《阿里巴巴Java开发手册》,总结了阿里巴巴内部实际项目开发过程中开发人员应该遵守的研发流程规范,这些流程规范在一定程度上能够保证最终的项目交付质量,通过在时间中总结模式,并推广给广大开发人员,来避免研发人员在实践中容易犯的错误,确保最终在大规模协作的项目中达成既定目标。 无独有偶,笔者去年在公司里负责升级和制定研发流程、设计模板、设计标准、代码标准等规范,并在实际工作中进行...
SQL-小白最佳入门sql查询一
不要偷偷的查询我的个人资料,即使你再喜欢我,也不要这样,真的不好;
项目中的if else太多了,该怎么重构?
介绍 最近跟着公司的大佬开发了一款IM系统,类似QQ和微信哈,就是聊天软件。我们有一部分业务逻辑是这样的 if (msgType = "文本") { // dosomething } else if(msgType = "图片") { // doshomething } else if(msgType = "视频") { // doshomething } else { // doshom...
Nginx 原理和架构
Nginx 是一个免费的,开源的,高性能的 HTTP 服务器和反向代理,以及 IMAP / POP3 代理服务器。Nginx 以其高性能,稳定性,丰富的功能,简单的配置和低资源消耗而闻名。 Nginx 的整体架构 Nginx 里有一个 master 进程和多个 worker 进程。master 进程并不处理网络请求,主要负责调度工作进程:加载配置、启动工作进程及非停升级。worker 进程负责处...
YouTube排名第一的励志英文演讲《Dream(梦想)》
Idon’t know what that dream is that you have, I don't care how disappointing it might have been as you've been working toward that dream,but that dream that you’re holding in your mind, that it’s po...
“狗屁不通文章生成器”登顶GitHub热榜,分分钟写出万字形式主义大作
一、垃圾文字生成器介绍 最近在浏览GitHub的时候,发现了这样一个骨骼清奇的雷人项目,而且热度还特别高。 项目中文名:狗屁不通文章生成器 项目英文名:BullshitGenerator 根据作者的介绍,他是偶尔需要一些中文文字用于GUI开发时测试文本渲染,因此开发了这个废话生成器。但由于生成的废话实在是太过富于哲理,所以最近已经被小伙伴们给玩坏了。 他的文风可能是这样的: 你发现,...
程序员:我终于知道post和get的区别
是一个老生常谈的话题,然而随着不断的学习,对于以前的认识有很多误区,所以还是需要不断地总结的,学而时习之,不亦说乎
《程序人生》系列-这个程序员只用了20行代码就拿了冠军
你知道的越多,你不知道的越多 点赞再看,养成习惯GitHub上已经开源https://github.com/JavaFamily,有一线大厂面试点脑图,欢迎Star和完善 前言 这一期不算《吊打面试官》系列的,所有没前言我直接开始。 絮叨 本来应该是没有这期的,看过我上期的小伙伴应该是知道的嘛,双十一比较忙嘛,要值班又要去帮忙拍摄年会的视频素材,还得搞个程序员一天的Vlog,还要写BU...
相关热词 c# 引用mysql c#动态加载非托管dll c# 两个表数据同步 c# 返回浮点json c# imap 链接状态 c# 漂亮字 c# 上取整 除法 c#substring c#中延时关闭 c#线段拖拉
立即提问