跪求大佬指点:hadoop集群启动jobhistory时,启动完就挂

HIVE操作建表时报错,在查看Hive日志发现是jobhistory没有启动,随后启动了很多次jobhistory,全部都是启动完就挂,查看jobhistory日志说是File /data/mapred/done 不存在,然后我就自己手动建立,权限直接给了777. 操作完后启动还是立马就挂了.跪求各位大佬指点!

下面是日志信息

************************************************************/
2020-09-25 20:26:52,513 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: registered UNIX signal handlers for [TERM, HUP, INT]
2020-09-25 20:26:53,388 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory: JobHistory Init
2020-09-25 20:27:03,937 INFO org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://ns1:8020]
2020-09-25 20:27:04,079 INFO org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Directory: [hdfs://ns1:8020/data/mapred/done] already exists.
2020-09-25 20:27:04,094 INFO org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://ns1:8020]
2020-09-25 20:27:04,100 INFO org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Directory: [hdfs://ns1:8020/data/mapred/tmp] already exists.
2020-09-25 20:27:14,112 INFO org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager: Initializing Existing Jobs...
2020-09-25 20:27:14,118 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.hs.JobHistory failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to intialize existing directories
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to intialize existing directories
at org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:98)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:221)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:231)
Caused by: java.io.FileNotFoundException: File /data/mapred/done does not exist.
at org.apache.hadoop.fs.Hdfs$DirListingIterator.(Hdfs.java:211)
at org.apache.hadoop.fs.Hdfs$DirListingIterator.(Hdfs.java:195)
at org.apache.hadoop.fs.Hdfs$2.(Hdfs.java:177)
at org.apache.hadoop.fs.Hdfs.listStatusIterator(Hdfs.java:177)
at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1494)
at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1489)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.listStatus(FileContext.java:1496)
at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.localGlobber(JobHistoryUtils.java:457)
at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.localGlobber(JobHistoryUtils.java:444)
at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.localGlobber(JobHistoryUtils.java:439)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.findTimestampedDirectories(HistoryFileManager.java:822)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.initExisting(HistoryFileManager.java:716)
at org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:96)
... 6 more
2020-09-25 20:27:14,122 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory: Stopping JobHistory
2020-09-25 20:27:14,122 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to intialize existing directories
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to intialize existing directories
at org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:98)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:221)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:231)
Caused by: java.io.FileNotFoundException: File /data/mapred/done does not exist.
at org.apache.hadoop.fs.Hdfs$DirListingIterator.(Hdfs.java:211)
at org.apache.hadoop.fs.Hdfs$DirListingIterator.(Hdfs.java:195)
at org.apache.hadoop.fs.Hdfs$2.(Hdfs.java:177)
at org.apache.hadoop.fs.Hdfs.listStatusIterator(Hdfs.java:177)
at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1494)
at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1489)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.listStatus(FileContext.java:1496)
at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.localGlobber(JobHistoryUtils.java:457)
at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.localGlobber(JobHistoryUtils.java:444)
at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.localGlobber(JobHistoryUtils.java:439)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.findTimestampedDirectories(HistoryFileManager.java:822)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.initExisting(HistoryFileManager.java:716)
at org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:96)
... 6 more
2020-09-25 20:27:14,123 FATAL org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: Error starting JobHistoryServer
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to intialize existing directories
at org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:98)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.serviceInit(JobHistoryServer.java:143)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.launchJobHistoryServer(JobHistoryServer.java:221)
at org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer.main(JobHistoryServer.java:231)
Caused by: java.io.FileNotFoundException: File /data/mapred/done does not exist.
at org.apache.hadoop.fs.Hdfs$DirListingIterator.(Hdfs.java:211)
at org.apache.hadoop.fs.Hdfs$DirListingIterator.(Hdfs.java:195)
at org.apache.hadoop.fs.Hdfs$2.(Hdfs.java:177)
at org.apache.hadoop.fs.Hdfs.listStatusIterator(Hdfs.java:177)
at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1494)
at org.apache.hadoop.fs.FileContext$22.next(FileContext.java:1489)
at org.apache.hadoop.fs.FSLinkResolver.resolve(FSLinkResolver.java:90)
at org.apache.hadoop.fs.FileContext.listStatus(FileContext.java:1496)
at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.localGlobber(JobHistoryUtils.java:457)
at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.localGlobber(JobHistoryUtils.java:444)
at org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils.localGlobber(JobHistoryUtils.java:439)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.findTimestampedDirectories(HistoryFileManager.java:822)
at org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.initExisting(HistoryFileManager.java:716)
at org.apache.hadoop.mapreduce.v2.hs.JobHistory.serviceInit(JobHistory.java:96)
... 6 more
2020-09-25 20:27:14,124 INFO org.apache.hadoop.util.ExitUtil: Exiting with status -1
2020-09-25 20:27:14,126 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down JobHistoryServer at nn1.hadoop/192.168.44.121
************************************************************/

2个回答

cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to intialize existing directories
无法初始化已经存在的目录,看看是不是权限的问题,给777权限

walyqian
浅梦绿萝裳 我给了777的权限,但是就是不行
大约一个月之前 回复
Caused by: java.io.FileNotFoundException: File /data/mapred/done does not exist.

" /data/mapred/done"这个目录不存在

walyqian
浅梦绿萝裳 回复浅梦绿萝裳: 谢谢二位,问题已经解决了,是我在安装完HDFS后,测试上传文件时,建了个data目录造成的,删除后一切正常
大约一个月之前 回复
walyqian
浅梦绿萝裳 这个我知道,我也自己创建了,权限也给足了,直接777,但就是启动完成立马挂
大约一个月之前 回复
Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
立即提问
相关内容推荐