weixin_37260217 2019-08-28 20:39 采纳率: 0%
浏览 534

spring boot 集成spark 初始化spark context 报错,"datanucleus" yet this has not been found

@Configuration
public class SparkContextBean{

    private String sparkHome = ".";

    private String appName = "sparkTest";

    private String master = "local";

    @Autowired
    KerborseUtil kerborseUtil;

    @Bean
    @ConditionalOnMissingBean(SparkConf.class)
    public SparkConf sparkConf() throws Exception {
        kerborseUtil.zkAuthentication();
        kerborseUtil.hiveAuthentication();
        SparkConf conf = new SparkConf().setAppName(appName).setMaster(master);
        return conf;
    }

    @Bean
    @ConditionalOnMissingBean(JavaSparkContext.class)
    public JavaSparkContext javaSparkContext() throws Exception {
        return new JavaSparkContext(sparkConf());
    }

    @Bean
    @ConditionalOnMissingBean(HiveContext.class)
    public HiveContext hiveContext() throws Exception {
        return new HiveContext(javaSparkContext().sc());
    }
    }

然后项目启动的时候报错
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
... 75 common frames omitted
Caused by: org.datanucleus.exceptions.NucleusUserException: Persistence process has been specified to use a ClassLoaderResolver of name "datanucleus" yet this has not been found by the DataNucleus plugin mechanism. Please check your CLASSPATH and plugin specification.
at org.datanucleus.NucleusContext.(NucleusContext.java:283)
at org.datanucleus.NucleusContext.(NucleusContext.java:247)
at org.datanucleus.NucleusContext.(NucleusContext.java:225)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.(JDOPersistenceManagerFactory.java:416)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:301)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
... 83 common frames omitted

环境:hadoop2.7.2+hive1.3.0+spark1.5.1+scala2.10.4
网上看了
https://blog.csdn.net/qq_38426934/article/details/81902830
但是没有搞定,不知道怎么解决,哎惭愧
猜想应该是和datanucleus相关的三个jar包有关系,jar包依赖也贴上
求大神们帮忙,看看

c币没了,不好意思,还是想得到大神的帮助,不胜感激

<dependency>
            <groupId>org.datanucleus</groupId>
            <artifactId>datanucleus-core</artifactId>
            <version>3.2.10</version>
        </dependency>
        <dependency>
            <groupId>org.datanucleus</groupId>
            <artifactId>datanucleus-api-jdo</artifactId>
            <version>3.2.6</version>
        </dependency>
        <dependency>
            <groupId>org.datanucleus</groupId>
            <artifactId>datanucleus-rdbms</artifactId>
            <version>3.2.9</version>
        </dependency>
  • 写回答

1条回答

  • zqbnqsdsmd 2019-10-05 23:02
    关注
    评论

报告相同问题?

悬赏问题

  • ¥15 gwas 分析-数据质控之过滤稀有突变中出现的问题
  • ¥15 没有注册类 (异常来自 HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG))
  • ¥15 知识蒸馏实战博客问题
  • ¥15 用PLC设计纸袋糊底机送料系统
  • ¥15 simulink仿真中dtc控制永磁同步电机如何控制开关频率
  • ¥15 用C语言输入方程怎么
  • ¥15 网站显示不安全连接问题
  • ¥15 51单片机显示器问题
  • ¥20 关于#qt#的问题:Qt代码的移植问题
  • ¥50 求图像处理的matlab方案