weixin_38484005 2021-09-17 20:37 采纳率: 0%
浏览 385
已结题

namenode和datanode无法启动解决方案

本人初学hadoop,启动Hadoop集群时发现错误,请各位帮下忙,谢谢!
在启动Hadoop集群后,发现从节点的datanode 和NodeManager没有启动,并且通过查询从节点,发现从节点Hadoop目录下logs也没有自动创建。
通过查询主节点namenode-node01.log、datanode-node01.log、nodemanager-node01.log和resourcemanager-node01.log,发现是同一个问题。
以下是日志截图:

img

以下为完整日志:

STARTUP_MSG:   build = Unknown -r Unknown; compiled by 'root' on 2015-05-27T13:56Z
STARTUP_MSG:   java = 1.7.0_79
************************************************************/
2021-09-17 19:49:04,446 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: registered UNIX signal handlers for [TERM, HUP, INT]
2021-09-17 19:49:06,645 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2021-09-17 19:49:06,899 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2021-09-17 19:49:06,899 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics system started
2021-09-17 19:49:06,912 INFO org.apache.hadoop.hdfs.server.datanode.BlockScanner: Disabled block scanner.
2021-09-17 19:49:06,920 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Configured hostname is node01
2021-09-17 19:49:06,954 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting DataNode with maxLockedMemory = 0
2021-09-17 19:49:07,059 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened streaming server at /0.0.0.0:50010
2021-09-17 19:49:07,070 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is 1048576 bytes/s
2021-09-17 19:49:07,070 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Number threads for balancing is 5
2021-09-17 19:49:07,344 INFO org.mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4j
Log
2021-09-17 19:49:07,370 INFO org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSe
cretProvider, falling back to use random secrets.
2021-09-17 19:49:07,379 INFO org.apache.hadoop.http.HttpRequestLog: Http request log for http.requests.datanode is not defined
2021-09-17 19:49:07,398 INFO org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServe
r2$QuotingInputFilter)
2021-09-17 19:49:07,400 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.St
aticUserWebFilter$StaticUserFilter) to context datanode
2021-09-17 19:49:07,400 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.St
aticUserWebFilter$StaticUserFilter) to context logs
2021-09-17 19:49:07,400 INFO org.apache.hadoop.http.HttpServer2: Added filter static_user_filter (class=org.apache.hadoop.http.lib.St
aticUserWebFilter$StaticUserFilter) to context static
2021-09-17 19:49:07,432 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 51684
2021-09-17 19:49:07,432 INFO org.mortbay.log: jetty-6.1.26
2021-09-17 19:49:08,151 INFO org.mortbay.log: Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51684
2021-09-17 19:49:08,582 INFO org.apache.hadoop.hdfs.server.datanode.web.DatanodeHttpServer: Listening HTTP traffic on /0.0.0.0:50075
2021-09-17 19:49:09,099 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: dnUserName = root
2021-09-17 19:49:09,099 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: supergroup = supergroup
2021-09-17 19:49:09,267 INFO org.apache.hadoop.ipc.CallQueueManager: Using callQueue class java.util.concurrent.LinkedBlockingQueue
2021-09-17 19:49:09,327 INFO org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 50020
2021-09-17 19:49:09,433 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Opened IPC server at /0.0.0.0:50020
2021-09-17 19:49:09,488 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Refresh request received for nameservices: null
2021-09-17 19:49:09,558 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Starting BPOfferServices for nameservices: <default>
2021-09-17 19:49:09,603 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool <registering> (Datanode Uuid unassigned) ser
vice to node01/192.168.25.100:8020 starting to offer service
2021-09-17 19:49:09,627 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 50020: starting
2021-09-17 19:49:09,661 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2021-09-17 19:49:10,868 INFO org.apache.hadoop.hdfs.server.common.Storage: Lock on /opt/software/hadoop-2.7.0/datas/datanode/datanode
datas/in_use.lock acquired by nodename 6143@node01
2021-09-17 19:49:10,871 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory /opt/software/hadoop-2.7.0/datas/datanod
e/datanodedatas is not formatted for BP-1365563250-192.168.25.100-1631879139497
2021-09-17 19:49:10,871 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting ...
2021-09-17 19:49:11,188 INFO org.apache.hadoop.hdfs.server.common.Storage: Analyzing storage directories for bpid BP-1365563250-192.1
68.25.100-1631879139497
2021-09-17 19:49:11,188 INFO org.apache.hadoop.hdfs.server.common.Storage: Locking is disabled for /opt/software/hadoop-2.7.0/datas/d
atanode/datanodedatas/current/BP-1365563250-192.168.25.100-1631879139497
2021-09-17 19:49:11,189 INFO org.apache.hadoop.hdfs.server.common.Storage: Block pool storage directory /opt/software/hadoop-2.7.0/da
tas/datanode/datanodedatas/current/BP-1365563250-192.168.25.100-1631879139497 is not formatted for BP-1365563250-192.168.25.100-16318
79139497
2021-09-17 19:49:11,189 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting ...
2021-09-17 19:49:11,189 INFO org.apache.hadoop.hdfs.server.common.Storage: Formatting block pool BP-1365563250-192.168.25.100-1631879
139497 directory /opt/software/hadoop-2.7.0/datas/datanode/datanodedatas/current/BP-1365563250-192.168.25.100-1631879139497/current
2021-09-17 19:49:11,201 INFO org.apache.hadoop.hdfs.server.common.Storage: Restored 0 block files from trash.
2021-09-17 19:49:11,203 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Setting up storage: nsid=720496585;bpid=BP-1365563250-1
92.168.25.100-1631879139497;lv=-56;nsInfo=lv=-63;cid=CID-eb2a5abd-a25b-47ee-bf5f-7a013cf2787e;nsid=720496585;c=0;bpid=BP-1365563250-1
92.168.25.100-1631879139497;dnuuid=null
2021-09-17 19:49:11,209 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Generated and persisted new Datanode UUID 10e730c1-d6fe
-4689-98a4-0cee10b87f64
2021-09-17 19:49:11,394 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Added new volume: DS-442261f3-022b-
46f8-9e3d-8d63d7605270
2021-09-17 19:49:11,394 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Added volume - /opt/software/hadoop
-2.7.0/datas/datanode/datanodedatas/current, StorageType: DISK
2021-09-17 19:49:11,417 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Registered FSDatasetState MBean
2021-09-17 19:49:11,432 INFO org.apache.hadoop.hdfs.server.datanode.DirectoryScanner: Periodic Directory Tree Verification scan start
ing at 1631892629432 with interval 21600000
2021-09-17 19:49:11,436 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adding block pool BP-1365563250-192
.168.25.100-1631879139497
2021-09-17 19:49:11,442 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Scanning block pool BP-1365563250-1
92.168.25.100-1631879139497 on volume /opt/software/hadoop-2.7.0/datas/datanode/datanodedatas/current...
2021-09-17 19:49:11,612 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Time taken to scan block pool BP-13
65563250-192.168.25.100-1631879139497 on /opt/software/hadoop-2.7.0/datas/datanode/datanodedatas/current: 171ms
2021-09-17 19:49:11,612 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Total time to scan all replicas for
 block pool BP-1365563250-192.168.25.100-1631879139497: 177ms
2021-09-17 19:49:11,617 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Adding replicas to map for block po
ol BP-1365563250-192.168.25.100-1631879139497 on volume /opt/software/hadoop-2.7.0/datas/datanode/datanodedatas/current...
2021-09-17 19:49:11,617 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Time to add replicas to map for blo
ck pool BP-1365563250-192.168.25.100-1631879139497 on volume /opt/software/hadoop-2.7.0/datas/datanode/datanodedatas/current: 0ms
2021-09-17 19:49:11,617 INFO org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl: Total time to add all replicas to m
ap: 4ms
2021-09-17 19:49:11,620 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool BP-1365563250-192.168.25.100-1631879139497 (
Datanode Uuid null) service to node01/192.168.25.100:8020 beginning handshake with NN
2021-09-17 19:49:11,803 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Block pool Block pool BP-1365563250-192.168.25.100-1631
879139497 (Datanode Uuid null) service to node01/192.168.25.100:8020 successfully registered with NN
2021-09-17 19:49:11,803 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: For namenode node01/192.168.25.100:8020 using DELETEREP
ORT_INTERVAL of 300000 msec  BLOCKREPORT_INTERVAL of 21600000msec CACHEREPORT_INTERVAL of 10000msec Initial delay: 0msec; heartBeatIn
terval=3000
2021-09-17 19:49:12,331 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Namenode Block pool BP-1365563250-192.168.25.100-163187
9139497 (Datanode Uuid 10e730c1-d6fe-4689-98a4-0cee10b87f64) service to node01/192.168.25.100:8020 trying to claim ACTIVE state with 
txid=1
2021-09-17 19:49:12,331 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Acknowledging ACTIVE Namenode Block pool BP-1365563250-
192.168.25.100-1631879139497 (Datanode Uuid 10e730c1-d6fe-4689-98a4-0cee10b87f64) service to node01/192.168.25.100:8020
2021-09-17 19:49:12,528 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Successfully sent block report 0x579066c2d62b,  contain
ing 1 storage report(s), of which we sent 1. The reports had 0 total blocks and used 1 RPC(s). This took 20 msec to generate and 175 
msecs for RPC and NN processing. Got back one command: FinalizeCommand/5.
2021-09-17 19:49:12,529 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Got finalize command for block pool BP-1365563250-192.1
68.25.100-1631879139497

  • 写回答

4条回答 默认 最新

  • syyyyyyyyyyyyyyh 2021-09-18 09:56
    关注

    2版本的话看下你的 slave文件
    3版本的话看下你的workers和whitelist

    评论
    1人已打赏

报告相同问题?

问题事件

  • 系统已结题 9月25日
  • 创建了问题 9月17日

悬赏问题

  • ¥15 机器学习教材中的例题询问
  • ¥15 求.net core 几款免费的pdf编辑器
  • ¥20 SQL server表计算问题
  • ¥15 C# P/Invoke的效率问题
  • ¥20 thinkphp适配人大金仓问题
  • ¥20 Oracle替换.dbf文件后无法连接,如何解决?(相关搜索:数据库|死循环)
  • ¥15 数据库数据成问号了,前台查询正常,数据库查询是?号
  • ¥15 算法使用了tf-idf,用手肘图确定k值确定不了,第四轮廓系数又太小才有0.006088746097507285,如何解决?(相关搜索:数据处理)
  • ¥15 彩灯控制电路,会的加我QQ1482956179
  • ¥200 相机拍直接转存到电脑上 立拍立穿无线局域网传