墨白ed 2021-05-11 11:01 采纳率: 57.1%
浏览 163
已采纳

请问在集群上安装HiBench7.1后,编译spark出现错误,怎么解决

设计组件版本:

jdk v15.0.1

maven  v3.5.4

spark  v3.0.1

scala v2.12.10

hadoop  v3.3.0

 

安装HiBench7.1后,编译spark,

执行的是   mvn -Psparkbench -Dspark=3.0 -Dscala=2.12 clean package

报错

[INFO] Reactor Summary:
[INFO]
[INFO] hibench 8.0-SNAPSHOT ............................... SUCCESS [  0.089 s]
[INFO] hibench-common ..................................... FAILURE [  0.851 s]
[INFO] HiBench data generation tools ...................... SKIPPED
[INFO] sparkbench ......................................... SKIPPED
[INFO] sparkbench-common .................................. SKIPPED
[INFO] sparkbench micro benchmark ......................... SKIPPED
[INFO] sparkbench machine learning benchmark .............. SKIPPED
[INFO] sparkbench-websearch ............................... SKIPPED
[INFO] sparkbench-graph ................................... SKIPPED
[INFO] sparkbench-sql ..................................... SKIPPED
[INFO] sparkbench project assembly ........................ SKIPPED
[INFO] flinkbench ......................................... SKIPPED
[INFO] flinkbench-streaming ............................... SKIPPED
[INFO] gearpumpbench ...................................... SKIPPED
[INFO] gearpumpbench-streaming ............................ SKIPPED
[INFO] hadoopbench ........................................ SKIPPED
[INFO] hadoopbench-sql .................................... SKIPPED
[INFO] mahout ............................................. SKIPPED
[INFO] PEGASUS: A Peta-Scale Graph Mining System 2.0-SNAPSHOT SKIPPED
[INFO] nutchindexing ...................................... SKIPPED
[INFO] stormbench ......................................... SKIPPED
[INFO] stormbench-streaming 8.0-SNAPSHOT .................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.130 s
[INFO] Finished at: 2021-05-11T10:53:28+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.0:compile (scala-compile-first) on project hibench-common: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hibench-common
[centos@ombd-research-1 HiBench-remove_spark_2.3]$ clear
[centos@ombd-research-1 HiBench-remove_spark_2.3]$ clea
-bash: clea: command not found
[centos@ombd-research-1 HiBench-remove_spark_2.3]$
[centos@ombd-research-1 HiBench-remove_spark_2.3]$
[centos@ombd-research-1 HiBench-remove_spark_2.3]$
[centos@ombd-research-1 HiBench-remove_spark_2.3]$
[centos@ombd-research-1 HiBench-remove_spark_2.3]$
[centos@ombd-research-1 HiBench-remove_spark_2.3]$
[centos@ombd-research-1 HiBench-remove_spark_2.3]$ mvn -Psparkbench -Dspark=3.0 -Dscala=2.12 clean package
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for com.intel.hibench.stormbench:stormbench-streaming:jar:8.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-shade-plugin is missing. @ com.intel.hibench.stormbench:stormbench-streaming:[unknown-version], /home/centos/soft/HiBench7.1/HiBench-remove_spark_2.3/stormbench/streaming/pom.xml, line 97, column 15
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] hibench                                                            [pom]
[INFO] hibench-common                                                     [jar]
[INFO] HiBench data generation tools                                      [jar]
[INFO] sparkbench                                                         [pom]
[INFO] sparkbench-common                                                  [jar]
[INFO] sparkbench micro benchmark                                         [jar]
[INFO] sparkbench machine learning benchmark                              [jar]
[INFO] sparkbench-websearch                                               [jar]
[INFO] sparkbench-graph                                                   [jar]
[INFO] sparkbench-sql                                                     [jar]
[INFO] sparkbench project assembly                                        [pom]
[INFO] flinkbench                                                         [pom]
[INFO] flinkbench-streaming                                               [jar]
[INFO] gearpumpbench                                                      [pom]
[INFO] gearpumpbench-streaming                                            [jar]
[INFO] hadoopbench                                                        [pom]
[INFO] hadoopbench-sql                                                    [jar]
[INFO] mahout                                                             [jar]
[INFO] PEGASUS: A Peta-Scale Graph Mining System                          [jar]
[INFO] nutchindexing                                                      [jar]
[INFO] stormbench                                                         [pom]
[INFO] stormbench-streaming                                               [jar]
[INFO]
[INFO] ---------------------< com.intel.hibench:hibench >----------------------
[INFO] Building hibench 8.0-SNAPSHOT                                     [1/22]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hibench ---
[INFO]
[INFO] ------------------< com.intel.hibench:hibench-common >------------------
[INFO] Building hibench-common 8.0-SNAPSHOT                              [2/22]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ hibench-common ---
[INFO] Deleting /home/centos/soft/HiBench7.1/HiBench-remove_spark_2.3/common/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ hibench-common ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/centos/soft/HiBench7.1/HiBench-remove_spark_2.3/common/src/main/resources
[INFO]
[INFO] --- scala-maven-plugin:3.2.0:add-source (scala-compile-first) @ hibench-common ---
[INFO] Add Source directory: /home/centos/soft/HiBench7.1/HiBench-remove_spark_2.3/common/src/main/scala
[INFO] Add Test Source directory: /home/centos/soft/HiBench7.1/HiBench-remove_spark_2.3/common/src/test/scala
[INFO]
[INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) @ hibench-common ---
[INFO] /home/centos/soft/HiBench7.1/HiBench-remove_spark_2.3/common/src/main/java:-1: info: compiling
[INFO] /home/centos/soft/HiBench7.1/HiBench-remove_spark_2.3/common/src/main/scala:-1: info: compiling
[INFO] Compiling 17 source files to /home/centos/soft/HiBench7.1/HiBench-remove_spark_2.3/common/target/classes at 1620701650401
[ERROR] error: error while loading package, Missing dependency 'object java.lang.Object in compiler mirror', required by /home/centos/.m2/repository/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar(scala/package.class)
[ERROR] error: error while loading package, Missing dependency 'object java.lang.Object in compiler mirror', required by /home/centos/.m2/repository/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar(scala/runtime/package.class)
[ERROR] error: scala.reflect.internal.MissingRequirementError: object java.lang.Object in compiler mirror not found.
[ERROR]         at scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:16)
[ERROR]         at scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:17)
[ERROR]         at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:48)
[ERROR]         at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:40)
[ERROR]         at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:40)
[ERROR]         at scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:61)
[ERROR]         at scala.reflect.internal.Mirrors$RootsBase.getClassByName(Mirrors.scala:99)
[ERROR]         at scala.reflect.internal.Mirrors$RootsBase.getRequiredClass(Mirrors.scala:102)
[ERROR]         at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass$lzycompute(Definitions.scala:264)
[ERROR]         at scala.reflect.internal.Definitions$DefinitionsClass.ObjectClass(Definitions.scala:264)
[ERROR]         at scala.reflect.internal.Definitions$DefinitionsClass.AnyRefClass$lzycompute(Definitions.scala:263)
[ERROR]         at scala.reflect.internal.Definitions$DefinitionsClass.AnyRefClass(Definitions.scala:263)
[ERROR]         at scala.reflect.internal.Definitions$DefinitionsClass.specialPolyClass(Definitions.scala:1120)
[ERROR]         at scala.reflect.internal.Definitions$DefinitionsClass.RepeatedParamClass$lzycompute(Definitions.scala:407)
[ERROR]         at scala.reflect.internal.Definitions$DefinitionsClass.RepeatedParamClass(Definitions.scala:407)
[ERROR]         at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreClasses$lzycompute(Definitions.scala:1154)
[ERROR]         at scala.reflect.internal.Definitions$DefinitionsClass.syntheticCoreClasses(Definitions.scala:1152)
[ERROR]         at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode$lzycompute(Definitions.scala:1196)
[ERROR]         at scala.reflect.internal.Definitions$DefinitionsClass.symbolsNotPresentInBytecode(Definitions.scala:1196)
[ERROR]         at scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1261)
[ERROR]         at scala.tools.nsc.Global$Run.<init>(Global.scala:1290)
[ERROR]         at scala.tools.nsc.Driver.doCompile(Driver.scala:32)
[ERROR]         at scala.tools.nsc.Main$.doCompile(Main.scala:79)
[ERROR]         at scala.tools.nsc.Driver.process(Driver.scala:54)
[ERROR]         at scala.tools.nsc.Driver.main(Driver.scala:67)
[ERROR]         at scala.tools.nsc.Main.main(Main.scala)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
[ERROR]         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR]         at java.base/java.lang.reflect.Method.invoke(Method.java:564)
[ERROR]         at scala_maven_executions.MainHelper.runMain(MainHelper.java:164)
[ERROR]         at scala_maven_executions.MainWithArgsInFile.main(MainWithArgsInFile.java:26)
[ERROR]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] hibench 8.0-SNAPSHOT ............................... SUCCESS [  0.085 s]
[INFO] hibench-common ..................................... FAILURE [  0.853 s]
[INFO] HiBench data generation tools ...................... SKIPPED
[INFO] sparkbench ......................................... SKIPPED
[INFO] sparkbench-common .................................. SKIPPED
[INFO] sparkbench micro benchmark ......................... SKIPPED
[INFO] sparkbench machine learning benchmark .............. SKIPPED
[INFO] sparkbench-websearch ............................... SKIPPED
[INFO] sparkbench-graph ................................... SKIPPED
[INFO] sparkbench-sql ..................................... SKIPPED
[INFO] sparkbench project assembly ........................ SKIPPED
[INFO] flinkbench ......................................... SKIPPED
[INFO] flinkbench-streaming ............................... SKIPPED
[INFO] gearpumpbench ...................................... SKIPPED
[INFO] gearpumpbench-streaming ............................ SKIPPED
[INFO] hadoopbench ........................................ SKIPPED
[INFO] hadoopbench-sql .................................... SKIPPED
[INFO] mahout ............................................. SKIPPED
[INFO] PEGASUS: A Peta-Scale Graph Mining System 2.0-SNAPSHOT SKIPPED
[INFO] nutchindexing ...................................... SKIPPED
[INFO] stormbench ......................................... SKIPPED
[INFO] stormbench-streaming 8.0-SNAPSHOT .................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.124 s
[INFO] Finished at: 2021-05-11T10:54:10+08:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.0:compile (scala-compile-first) on project hibench-common: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hibench-common

  • 写回答

2条回答 默认 最新

  • CSDN专家-黄老师 2021-05-11 17:18
    关注

    使用旧版本的Maven3.2.5和JDK 8试试

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
查看更多回答(1条)

报告相同问题?

悬赏问题

  • ¥100 set_link_state
  • ¥15 虚幻5 UE美术毛发渲染
  • ¥15 CVRP 图论 物流运输优化
  • ¥15 Tableau online 嵌入ppt失败
  • ¥100 支付宝网页转账系统不识别账号
  • ¥15 基于单片机的靶位控制系统
  • ¥15 真我手机蓝牙传输进度消息被关闭了,怎么打开?(关键词-消息通知)
  • ¥15 装 pytorch 的时候出了好多问题,遇到这种情况怎么处理?
  • ¥20 IOS游览器某宝手机网页版自动立即购买JavaScript脚本
  • ¥15 手机接入宽带网线,如何释放宽带全部速度