hadoop集群测试pi实例出错

Question

2022-04-24 18:42:02,232 DEBUG security.UserGroupInformation: PrivilegedAction [as: ydh (auth:SIMPLE)][action: org.apache.hadoop.mapreduce.Job$1@58faa93b]
java.lang.Exception
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
at org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:330)
at org.apache.hadoop.mapreduce.Job.isComplete(Job.java:614)
at org.apache.hadoop.mapreduce.Job.monitorAndPrintJob(Job.java:1631)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1592)
at org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:307)
at org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:360)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:81)
at org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:368)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
at org.apache.hadoop.util.RunJar.main(RunJar.java:236)

hadoop集群测试pi实例出错一直running没有结果，请问大家是什么问题

Answer 1

你好，我之前测试成功过，如有帮助，请采纳！

然后进入到etc/hadoop 目录下，使用命令vim mapred-site.xml将路径添加到<property></property>标签，注意把路径中的:替换成,

<property>
  <name>yarn.application.classpath</name>
  <value>/opt/hadoop-3.3.2/etc/hadoop,/opt/hadoop-3.3.2/share/hadoop/common/lib/*,/opt/hadoop-3.3.2/share/hadoop/common/*,/opt/hadoop-3.3.2/share/hadoop/hdfs,/opt/hadoop-3.3.2/share/hadoop/hdfs/lib/*,/opt/hadoop-3.3.2/share/hadoop/hdfs/*,/opt/hadoop-3.3.2/share/hadoop/mapreduce/*,/opt/hadoop-3.3.2/share/hadoop/yarn,/opt/hadoop-3.3.2/share/hadoop/yarn/lib/*,/opt/hadoop-3.3.2/share/hadoop/yarn/*
  </value>
</property>

配置修改完成后，保存退出。关闭集群(stop-all.sh)，再打开集群(start-all.sh)。

hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.2.jar pi 5 5

运行成功.

Estimated value of Pi is 3.6800000000000000000