系统:centos 7
分布式环境,现有3个节点,主节点node0,hadoop中node0,node1,node2同时作为datanode的节点
启动hbase时主节点没有Hmaster线程
hbase的日志错误如下:
2015-11-13 16:23:59,178 INFO [main] zookeeper.ZooKeeper: Initiating client connection, connectString=node1:2181,node0:2181,node2:2181 sessionTimeout=60000 watcher=master:600000x0, quorum=node1:2181,node0:2181,node2:2181, baseZNode=/hbase
2015-11-13 16:23:59,195 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Opening socket connection to server node1/192.168.0.161:2181. Will not attempt to authenticate using SASL (unknown error)
2015-11-13 16:23:59,200 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Socket connection established to node1/192.168.0.161:2181, initiating session
2015-11-13 16:23:59,211 INFO [main-SendThread(node1:2181)] zookeeper.ClientCnxn: Session establishment complete on server node1/192.168.0.161:2181, sessionid = 0x150feccba590005, negotiated timeout = 40000
2015-11-13 16:23:59,223 INFO [main] zookeeper.RecoverableZooKeeper: Node /hbase already exists and this is not a retry
2015-11-13 16:23:59,245 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: starting
2015-11-13 16:23:59,246 INFO [RpcServer.listener,port=60000] ipc.RpcServer: RpcServer.listener,port=60000: starting
2015-11-13 16:23:59,301 INFO [master:namenode:60000] mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2015-11-13 16:23:59,344 INFO [master:namenode:60000] http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter)
2015-11-13 16:23:59,346 INFO [master:namenode:60000] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context master
2015-11-13 16:23:59,346 INFO [master:namenode:60000] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2015-11-13 16:23:59,355 INFO [master:namenode:60000] http.HttpServer: Jetty bound to port 60010
2015-11-13 16:23:59,355 INFO [master:namenode:60000] mortbay.log: jetty-6.1.26
2015-11-13 16:23:59,656 INFO [master:namenode:60000] mortbay.log: Started HttpServer$SelectChannelConnectorWithSafeStartup@0.0.0.0:60010
2015-11-13 16:23:59,724 DEBUG [main-EventThread] master.ActiveMasterManager: A master is now available
2015-11-13 16:23:59,725 INFO [master:namenode:60000] master.ActiveMasterManager: Registered Active Master=namenode,60000,1447403038605
2015-11-13 16:23:59,731 INFO [master:namenode:60000] Configuration.deprecation: fs.default.name is deprecated. Instead, use fs.defaultFS
2015-11-13 16:23:59,875 FATAL [master:namenode:60000] master.HMaster: Unhandled exception. Starting shutdown.
java.lang.NoSuchMethodError: org.apache.hadoop.fs.FSOutputSummer.<init>(Ljava/util/zip/Checksum;II)V
at org.apache.hadoop.hdfs.DFSOutputStream.<init>(DFSOutputStream.java:1342)
at org.apache.hadoop.hdfs.DFSOutputStream.<init>(DFSOutputStream.java:1371)
at org.apache.hadoop.hdfs.DFSOutputStream.<init>(DFSOutputStream.java:1371)
at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1403)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1382)
at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1307)
at org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:384)
at org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:380)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:380)
at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:324)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:908)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:889)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:786)
at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:775)
at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:664)
at org.apache.hadoop.hbase.util.FSUtils.setVersion(FSUtils.java:642)
at org.apache.hadoop.hbase.util.FSUtils.checkVersion(FSUtils.java:599)
at org.apache.hadoop.hbase.master.MasterFileSystem.checkRootDir(MasterFileSystem.java:481)
at org.apache.hadoop.hbase.master.MasterFileSystem.createInitialFileSystemLayout(MasterFileSystem.java:154)
at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:130)
at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:881)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:684)
at java.lang.Thread.run(Thread.java:745)
2015-11-13 16:23:59,876 INFO [master:namenode:60000] master.HMaster: Aborting
2015-11-13 16:23:59,877 DEBUG [master:namenode:60000] master.HMaster: Stopping service threads
2015-11-13 16:23:59,877 INFO [master:namenode:60000] ipc.RpcServer: Stopping server on 60000
2015-11-13 16:23:59,877 INFO [RpcServer.listener,port=60000] ipc.RpcServer: RpcServer.listener,port=60000: stopping
2015-11-13 16:23:59,877 INFO [master:namenode:60000] master.HMaster: Stopping infoServer
2015-11-13 16:23:59,877 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped
2015-11-13 16:23:59,877 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping
2015-11-13 16:23:59,879 INFO [master:namenode:60000] mortbay.log: Stopped HttpServer$SelectChannelConnectorWithSafeStartup@0.0.0.0:60010
2015-11-13 16:23:59,994 INFO [master:namenode:60000] zookeeper.ZooKeeper: Session: 0x150feccba590005 closed
2015-11-13 16:23:59,994 INFO [main-EventThread] zookeeper.ClientCnxn: EventThread shut down
2015-11-13 16:23:59,994 INFO [master:namenode:60000] master.HMaster: HMaster main thread exiting
2015-11-13 16:23:59,995 ERROR [main] master.HMasterCommandLine: Master exiting
java.lang.RuntimeException: HMaster Aborted
at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:201)
at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:135)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:3062)
每个节点上hadoop和zookeeper的进程都有,hregionserver进程也有,就是主节点没有hmaster,配置文件什么的已经对过很多遍了
另外有一个奇怪的问题是,每次系统重启过后,启动hadoop,可以看到相应的进程,但是在外部无法访问http://node0:50070端口,必须要输入iptables -F才可以访问,zookeeper启动过后,如果没有输入过上述命令,zookeeper的MODE也显示不出来,也就是集群模式没有启动成功