noiyes38 2020-04-15 17:43 采纳率: 0%
浏览 213

hadoop搭建集群,运行pi实例,map和reduce0%就失败了,我用的是root用户,hadoop文件夹在桌面,8088页面也是可以看到的,报错如下,请帮忙看看什么问题呀?

执行集群Stack trace: ExitCodeException exitCode=1: /home/hadoop _tmp/nm-local-dir/usercache/root/appcach
cd share/hadoop/mapreduce
hadoop jar hadoop-mapreduce-examples-2.7.1.jar pi 5 5
Exception message: /home/hadoop _tmp/nm-local-dir/usercache/root/appcache/application_1586797329216_0001/container_1586797329216_0001_01_000003/default_container_executor_session.sh: line 3: /home/hadoop: Is a directory
/bin/mv: target _tmp/nm-local-dir/nmPrivate/application_1586797329216_0001/container_1586797329216_0001_01_000003/container_1586797329216_0001_01_000003.pid' is not a directory
Stack trace: ExitCodeException exitCode=1: /home/hadoop _tmp/nm-local-dir/usercache/root/appcache/application_1586797329216_0001/container_1586797329216_0001_01_000003/default_container_executor_session.sh: line 3: /home/hadoop: Is a directory
/bin/mv: target
_tmp/nm-local-dir/nmPrivate/application_1586797329216_0001/container_1586797329216_0001_01_000003/container_1586797329216_0001_01_000003.pid' is not a directory
Exception from container-launch.
Container id: container_1586797329216_0001_01_000003
Exit code: 1
Exception message: /home/hadoop _tmp/nm-local-dir/usercache/root/appcache/application_1586797329216_0001/container_1586797329216_0001_01_000003/default_container_executor_session.sh: line 3: /home/hadoop: Is a directory
/bin/mv: target `_tmp/nm-local-dir/nmPrivate/application_1586797329216_0001/container_1586797329216_0001_01_000003/container_1586797329216_0001_01_000003.pid' is not a directory

Stack trace: ExitCodeException exitCode=1: /home/hadoop _tmp/nm-local-dir/usercache/root/appcache/application_1586797329216_0001/container_1586797329216_0001_01_000003/default_container_executor_session.sh: line 3: /home/hadoop: Is a directory
/bin/mv: target `_tmp/nm-local-dir/nmPrivate/application_1586797329216_0001/container_1586797329216_0001_01_000003/container_1586797329216_0001_01_000003.pid' is not a directory

at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Container exited with a non-zero exit code 1

  • 写回答

1条回答 默认 最新

  • 码农阿豪@新空间 Java领域优质创作者 2024-07-25 15:26
    关注
    让阿豪来帮你解答,本回答参考chatgpt3.5编写提供,如果还有疑问可以评论或留言
    根据提供的执行集群Stack trace信息,可以看出是在执行Hadoop MapReduce的`pi`作业时出现了异常,具体问题在于`/home/hadoop`被当做一个目录处理,而不是一个可执行文件。 解决方法如下:检查`hadoop-mapreduce-examples-2.7.1.jar`包是否正确配置,确保`pi`作业可以正确访问`/home/hadoop`。另外,检查文件夹结构和权限,确保所需的文件和目录可被程序正常访问。 案例代码如下:
    import org.apache.hadoop.conf.Configuration;
    import org.apache.hadoop.util.ToolRunner;
    import org.apache.hadoop.util.Tool;
    import org.apache.hadoop.examples.*;
    public class CalculatePi implements Tool {
        private Configuration conf;
        public static void main(String[] args) {
            Configuration conf = new Configuration();
            CalculatePi calculatePi = new CalculatePi();
            calculatePi.setConf(conf);
            int res = ToolRunner.run(calculatePi, args);
            System.exit(res);
        }
        @Override
        public int run(String[] args) throws Exception {
            if (args.length != 2) {
                System.err.println("Usage: CalculatePi <numMaps> <numSamples>");
                return 1;
            }
            int numMaps = Integer.parseInt(args[0]);
            int numSamples = Integer.parseInt(args[1]);
            Job piJob = PiEstimator.createJob(numMaps, numSamples);
            if (piJob.waitForCompletion(true)) {
                return 0;
            } else {
                return 1;
            }
        }
        @Override
        public void setConf(Configuration conf) {
            this.conf = conf;
        }
        @Override
        public Configuration getConf() {
            return conf;
        }
    }
    

    以上是一个简单的示例代码,演示了如何使用Hadoop MapReduce计算Pi的近似值。对于pi作业中涉及的/home/hadoop目录,需要确保文件和目录的权限设置正确,且程序能够正确访问。

    评论

报告相同问题?