weixin_38484005
2021-09-17 20:37
采纳率: 0%
浏览 197

namenode和datanode无法启动解决方案

本人初学hadoop,启动Hadoop集群时发现错误,请各位帮下忙,谢谢!
在启动Hadoop集群后,发现从节点的datanode 和NodeManager没有启动,并且通过查询从节点,发现从节点Hadoop目录下logs也没有自动创建。
通过查询主节点namenode-node01.log、datanode-node01.log、nodemanager-node01.log和resourcemanager-node01.log,发现是同一个问题。
以下是日志截图:

img

以下为完整日志:

STARTUP_MSG:   build = Unknown -r Unknown; compiled by 'root' on 2015-05-27T13:56Z
STARTUP_MSG:   java = 1.7.0_79
************************************************************/
2021-09-17 19:49:04,446 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT]
2021-09-17 19:49:06,645 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2021-09-17 19:49:06,899 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2021-09-17 19:49:06,899 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started
2021-09-17 19:49:06,912 INFO org.apache.hadoop.hdfs.server.datanode.BlockScanner: Disabled block scanner.
2021-09-17 19:49:06,920 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is node01
2021-09-17 19:49:06,954 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting DataNode with maxLockedMemory = 0
2021-09-17 19:49:07,059 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened streaming server at /0.0.0.0:50010
2021-09-17 19:49:07,070 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is 1048576 bytes/s
2021-09-17 19:49:07,070 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Number threads for balancing is 5
2021-09-17 19:49:07,344 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4j
Log
2021-09-17 19:49:07,370 INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSe
cretProvider, falling back to use random secrets.
2021-09-17 19:49:07,379 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.datanode is not defined
2021-09-17 19:49:07,398 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServe
r2$QuotingInputFilter)
2021-09-17 19:49:07,400 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.St
aticUserWebFilter$StaticUserFilter) to context datanode
2021-09-17 19:49:07,400 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.St
aticUserWebFilter$StaticUserFilter) to context logs
2021-09-17 19:49:07,400 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.St
aticUserWebFilter$StaticUserFilter) to context static
2021-09-17 19:49:07,432 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 51684
2021-09-17 19:49:07,432 INFO org.mortbay.log: jetty-6.1.26
2021-09-17 19:49:08,151 INFO org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51684
2021-09-17 19:49:08,582 INFO org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer: Listening HTTP traffic on /0.0.0.0:50075
2021-09-17 19:49:09,099 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnUserName = root
2021-09-17 19:49:09,099 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: supergroup = supergroup
2021-09-17 19:49:09,267 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2021-09-17 19:49:09,327 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 50020
2021-09-17 19:49:09,433 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at /0.0.0.0:50020
2021-09-17 19:49:09,488 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received for nameservices: null
2021-09-17 19:49:09,558 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservices: <default>
2021-09-17 19:49:09,603 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool <registering> (Datanode Uuid unassigned) ser
vice to node01/192.168.25.100:8020 starting to offer service
2021-09-17 19:49:09,627 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting
2021-09-17 19:49:09,661 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2021-09-17 19:49:10,868 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /opt/software/hadoop-2.7.0/datas/datanode/datanode
datas/in_use.lock acquired by nodename 6143@node01
2021-09-17 19:49:10,871 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory /opt/software/hadoop-2.7.0/datas/datanod
e/datanodedatas is not formatted for BP-1365563250-192.168.25.100-1631879139497
2021-09-17 19:49:10,871 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting ...
2021-09-17 19:49:11,188 INFO org.apache.hadoop.hdfs.server.common.Storage: Analyzing storage directories for bpid BP-1365563250-192.1
68.25.100-1631879139497
2021-09-17 19:49:11,188 INFO org.apache.hadoop.hdfs.server.common.Storage: Locking is disabled for /opt/software/hadoop-2.7.0/datas/d
atanode/datanodedatas/current/BP-1365563250-192.168.25.100-1631879139497
2021-09-17 19:49:11,189 INFO org.apache.hadoop.hdfs.server.common.Storage: Block pool storage directory /opt/software/hadoop-2.7.0/da
tas/datanode/datanodedatas/current/BP-1365563250-192.168.25.100-1631879139497 is not formatted for BP-1365563250-192.168.25.100-16318
79139497
2021-09-17 19:49:11,189 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting ...
2021-09-17 19:49:11,189 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting block pool BP-1365563250-192.168.25.100-1631879
139497 directory /opt/software/hadoop-2.7.0/datas/datanode/datanodedatas/current/BP-1365563250-192.168.25.100-1631879139497/current
2021-09-17 19:49:11,201 INFO org.apache.hadoop.hdfs.server.common.Storage: Restored 0 block files from trash.
2021-09-17 19:49:11,203 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage: nsid=720496585;bpid=BP-1365563250-1
92.168.25.100-1631879139497;lv=-56;nsInfo=lv=-63;cid=CID-eb2a5abd-a25b-47ee-bf5f-7a013cf2787e;nsid=720496585;c=0;bpid=BP-1365563250-1
92.168.25.100-1631879139497;dnuuid=null
2021-09-17 19:49:11,209 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Generated and persisted new Datanode UUID 10e730c1-d6fe
-4689-98a4-0cee10b87f64
2021-09-17 19:49:11,394 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Added new volume: DS-442261f3-022b-
46f8-9e3d-8d63d7605270
2021-09-17 19:49:11,394 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Added volume - /opt/software/hadoop
-2.7.0/datas/datanode/datanodedatas/current, StorageType: DISK
2021-09-17 19:49:11,417 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Registered FSDatasetState MBean
2021-09-17 19:49:11,432 INFO org.apache.hadoop.hdfs.server.datanode.DirectoryScanner: Periodic Directory Tree Verification scan start
ing at 1631892629432 with interval 21600000
2021-09-17 19:49:11,436 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adding block pool BP-1365563250-192
.168.25.100-1631879139497
2021-09-17 19:49:11,442 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Scanning block pool BP-1365563250-1
92.168.25.100-1631879139497 on volume /opt/software/hadoop-2.7.0/datas/datanode/datanodedatas/current...
2021-09-17 19:49:11,612 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Time taken to scan block pool BP-13
65563250-192.168.25.100-1631879139497 on /opt/software/hadoop-2.7.0/datas/datanode/datanodedatas/current: 171ms
2021-09-17 19:49:11,612 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Total time to scan all replicas for
 block pool BP-1365563250-192.168.25.100-1631879139497: 177ms
2021-09-17 19:49:11,617 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adding replicas to map for block po
ol BP-1365563250-192.168.25.100-1631879139497 on volume /opt/software/hadoop-2.7.0/datas/datanode/datanodedatas/current...
2021-09-17 19:49:11,617 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Time to add replicas to map for blo
ck pool BP-1365563250-192.168.25.100-1631879139497 on volume /opt/software/hadoop-2.7.0/datas/datanode/datanodedatas/current: 0ms
2021-09-17 19:49:11,617 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Total time to add all replicas to m
ap: 4ms
2021-09-17 19:49:11,620 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool BP-1365563250-192.168.25.100-1631879139497 (
Datanode Uuid null) service to node01/192.168.25.100:8020 beginning handshake with NN
2021-09-17 19:49:11,803 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool Block pool BP-1365563250-192.168.25.100-1631
879139497 (Datanode Uuid null) service to node01/192.168.25.100:8020 successfully registered with NN
2021-09-17 19:49:11,803 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: For namenode node01/192.168.25.100:8020 using DELETEREP
ORT_INTERVAL of 300000 msec  BLOCKREPORT_INTERVAL of 21600000msec CACHEREPORT_INTERVAL of 10000msec Initial delay: 0msec; heartBeatIn
terval=3000
2021-09-17 19:49:12,331 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Namenode Block pool BP-1365563250-192.168.25.100-163187
9139497 (Datanode Uuid 10e730c1-d6fe-4689-98a4-0cee10b87f64) service to node01/192.168.25.100:8020 trying to claim ACTIVE state with 
txid=1
2021-09-17 19:49:12,331 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Acknowledging ACTIVE Namenode Block pool BP-1365563250-
192.168.25.100-1631879139497 (Datanode Uuid 10e730c1-d6fe-4689-98a4-0cee10b87f64) service to node01/192.168.25.100:8020
2021-09-17 19:49:12,528 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Successfully sent block report 0x579066c2d62b,  contain
ing 1 storage report(s), of which we sent 1. The reports had 0 total blocks and used 1 RPC(s). This took 20 msec to generate and 175 
msecs for RPC and NN processing. Got back one command: FinalizeCommand/5.
2021-09-17 19:49:12,529 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Got finalize command for block pool BP-1365563250-192.1
68.25.100-1631879139497

  • 收藏

4条回答 默认 最新

  • syyyyyyyyyyyyyyh 2021-09-18 09:56

    2版本的话看下你的 slave文件
    3版本的话看下你的workers和whitelist

    1 打赏 评论
    1人已打赏
  • syyyyyyyyyyyyyyh 2021-09-18 10:45

    使用hadoop用户启动,把hadoop的拥有权都改为hadoop chown -R hadoop:hadoop $HADOOP_HOME
    第一个文件里 把datanode节点的主机名加进去
    vim /etc/hosts里所有机器的信息配一下 类似这种
    ip 主机名

    img

    打赏 评论
  • 吃肉的小馒头 2021-09-18 15:05

    把集群关掉,node1 node2 node3上所有logs和data目录删掉 重新格式化 再启动试试

    打赏 评论
  • 问题解决了吗 好奇这100块花落谁家

    打赏 评论

相关推荐 更多相似问题