- hive可以进去但是操作不了 一直报错
-
Retrying HMSHandler after 2000 ms (attempt 1 of 10) with error: javax.jdo.JDODataStoreException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'OPTION SQL_SELECT_LIMIT=DEFAULT' at line 1
- 其他相关推荐
- Hive on spark查询报错。
- 求助！！！在hadoop使用Hive on spark执行Bigbench测试时，一直会有报错，log信息： FAILED: SemanticException Failed to get a spark session: org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create spark client. WARN: The method class org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked. WARN: Please see http://www.slf4j.org/codes.html#release for an explanation. An error occured while running command: ========== runEngineCmd -f /var/lib/hadoop-hdfs/Big-Bench/engines/hive/queries/q04/q04.sql ========== 在网上查了很多资料，有说版本不匹配的，有说是概率性问题，有没有大佬来瞅一眼啊。。哭了
- 运行hive --service metastore命令报错，如何解决
- 运行hive --service metastore命令报错，如何解决
- 启动Hive报错unknownHostException
- **之前改过一次主机名，修改了etc/hosts和 etc/sysconfig/network里的hostname， hadoop的core-site.xml中也做出相应修改， 但用hive命令启动后，报错：** ``` Exception in thread "main" java.lang.IllegalArgumentException: java.net.UnknownHostException: hadoop000 at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:374) at org.apache.hadoop.hdfs.NameNodeProxies.createNonHAProxy(NameNodeProxies.java:310) at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:176) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:708) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:651) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2696) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:94) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2733) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2715) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:382) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:181) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:583) at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:553) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: java.net.UnknownHostException: hadoop000 ... 22 more ``` hadoop000是我之前的主机名，搞不明白hive是在哪个配置文件里读到的，所以无从下手修改，求助各位大佬，感激不尽！！
- hive 设置开启事务功能，运行报错
- 现在想实现hive的删除和修改功能，hive版本1.2。 但是配置了好久总是报错，求大神们指点。 按照网上帖子配置了hive-site.xml ``` <property> <name>hive.support.concurrency</name> <value>true</value> </property> <property> <name>hive.exec.dynamic.partition.mode</name> <value>nonstrict</value> </property> <property> <name>hive.txn.manager</name> <value>org.apache.hadoop.hive.ql.lockmgr.DbTxnManager</value> </property> <property> <name>hive.compactor.initiator.on</name> <value>true</value> </property> <property> <name>hive.compactor.worker.threads</name> <value>1</value> </property> ``` 也在元数据里面加入了以下表和数据 ``` INSERT INTO NEXT_LOCK_ID VALUES(1); INSERT INTO NEXT_COMPACTION_QUEUE_ID VALUES(1); INSERT INTO NEXT_TXN_ID VALUES(1); COMMIT; ``` 参考：https://blog.csdn.net/wzy0623/article/details/51483674 现在在做查询的时候直接报错 FAILED: Error in acquiring locks: Error communicating with the metastore 后台日志报错： ``` 2018-10-15 22:01:53,144 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=compile start=1539655310604 end=1539655313144 duration=2540 from=org.apache.hadoop.hive.ql.Driver> 2018-10-15 22:01:53,144 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=acquireReadWriteLocks from=org.apache.hadoop.hive.ql.Driver> 2018-10-15 22:01:53,410 ERROR [main]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invoke(159)) - MetaException(message:Unable to update transaction database com.mysql.jdbc.NotImplemented: Feature not implemented at com.mysql.jdbc.Connection.setSavepoint(Connection.java:841) at com.jolbox.bonecp.ConnectionHandle.setSavepoint(ConnectionHandle.java:1334) at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:1456) at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:1430) at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:422) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:5527) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107) at com.sun.proxy.$Proxy8.lock(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1863) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:156) at com.sun.proxy.$Proxy9.lock(Unknown Source) at org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:83) at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:226) at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:91) at org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:994) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1182) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) ) at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:427) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:5527) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107) at com.sun.proxy.$Proxy8.lock(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1863) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:156) at com.sun.proxy.$Proxy9.lock(Unknown Source) at org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:83) at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:226) at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:91) at org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:994) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1182) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 2018-10-15 22:01:53,425 ERROR [main]: ql.Driver (SessionState.java:printError(957)) - FAILED: Error in acquiring locks: Error communicating with the metastore org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with the metastore at org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:110) at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:226) at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:91) at org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:994) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1182) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: MetaException(message:Unable to update transaction database com.mysql.jdbc.NotImplemented: Feature not implemented at com.mysql.jdbc.Connection.setSavepoint(Connection.java:841) at com.jolbox.bonecp.ConnectionHandle.setSavepoint(ConnectionHandle.java:1334) at org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(TxnHandler.java:1456) at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:1430) at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:422) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:5527) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107) at com.sun.proxy.$Proxy8.lock(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1863) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:156) at com.sun.proxy.$Proxy9.lock(Unknown Source) at org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:83) at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:226) at org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.acquireLocks(DbTxnManager.java:91) at org.apache.hadoop.hive.ql.Driver.acquireLocksAndOpenTxn(Driver.java:994) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1182) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) ) at org.apache.hadoop.hive.metastore.txn.TxnHandler.lock(TxnHandler.java:427) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.lock(HiveMetaStore.java:5527) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107) at com.sun.proxy.$Proxy8.lock(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:1863) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:156) at com.sun.proxy.$Proxy9.lock(Unknown Source) at org.apache.hadoop.hive.ql.lockmgr.DbLockManager.lock(DbLockManager.java:83) ... 18 more ```
- hive连接mysql 报错 readonly server
- 各位大侠，我搭建了一个hadoop环境，用hive做数据仓库，mysql做hive的元数据仓库，用于定时分析用户数据中的日志文件，但在hive访问mysql的过程中，不定时的报如下错误： java.sql.SQLException: Query returned non-zero code: 1, cause: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: Could not retrieve transation read-only status server at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:451) at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:732) at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752) at org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:784) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:98) at com.sun.proxy.$Proxy0.createTable(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1374) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1407) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:102) at com.sun.proxy.$Proxy10.create_table_with_environment_context(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.create_table_with_environment_context(HiveMetaStoreClient.java:1884) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.create_table_with_environment_context(SessionHiveMetaStoreClient.java:96) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:607) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:595) at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90) at com.sun.proxy.$Proxy11.createTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:670) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3959) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:295) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1604) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1364) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1177) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:197) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:644) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:628) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) NestedThrowablesStackTrace: java.sql.SQLException: Could not retrieve transation read-only status server at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1094) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:997) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:983) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:928) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:959) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:949) at com.mysql.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:3967) at com.mysql.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:3938) at com.jolbox.bonecp.ConnectionHandle.isReadOnly(ConnectionHandle.java:867) at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:422) at org.datanucleus.store.rdbms.valuegenerator.TableGenerator.createRepository(TableGenerator.java:270) at org.datanucleus.store.rdbms.valuegenerator.AbstractRDBMSGenerator.obtainGenerationBlock(AbstractRDBMSGenerator.java:162) at org.datanucleus.store.valuegenerator.AbstractGenerator.obtainGenerationBlock(AbstractGenerator.java:197) at org.datanucleus.store.valuegenerator.AbstractGenerator.next(AbstractGenerator.java:105) at org.datanucleus.store.rdbms.RDBMSStoreManager.getStrategyValueForGenerator(RDBMSStoreManager.java:2005) at org.datanucleus.store.AbstractStoreManager.getStrategyValue(AbstractStoreManager.java:1386) at org.datanucleus.ExecutionContextImpl.newObjectId(ExecutionContextImpl.java:3827) at org.datanucleus.state.JDOStateManager.setIdentity(JDOStateManager.java:2571) at org.datanucleus.state.JDOStateManager.initialiseForPersistentNew(JDOStateManager.java:513) at org.datanucleus.state.ObjectProviderFactoryImpl.newForPersistentNew(ObjectProviderFactoryImpl.java:232) at org.datanucleus.ExecutionContextImpl.newObjectProviderForPersistentNew(ExecutionContextImpl.java:1414) at org.datanucleus.ExecutionContextImpl.persistObjectInternal(ExecutionContextImpl.java:2218) at org.datanucleus.ExecutionContextImpl.persistObjectWork(ExecutionContextImpl.java:2065) at org.datanucleus.ExecutionContextImpl.persistObject(ExecutionContextImpl.java:1913) at org.datanucleus.ExecutionContextThreadedImpl.persistObject(ExecutionContextThreadedImpl.java:217) at org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:727) at org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:752) at org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:784) at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:98) at com.sun.proxy.$Proxy0.createTable(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1374) at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1407) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:102) at com.sun.proxy.$Proxy10.create_table_with_environment_context(Unknown Source) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.create_table_with_environment_context(HiveMetaStoreClient.java:1884) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.create_table_with_environment_context(SessionHiveMetaStoreClient.java:96) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:607) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:595) at sun.reflect.GeneratedMethodAccessor38.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:90) at com.sun.proxy.$Proxy11.createTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:670) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3959) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:295) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1604) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1364) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1177) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1004) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:994) at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:197) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:644) at org.apache.hadoop.hive.service.ThriftHive$Processor$execute.getResult(ThriftHive.java:628) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure 在网上查询说是mysql驱动包的问题，但更新了mysql驱动之后，依然还是会报这个错误， 有时候能执行成功，有时候执行不成功，不知道各位大侠谁遇到过类似的问题！
- Hive执行SQL语句报错hive> show databases; ----已解决
- ``` hive> show databases; FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient hive> ``` 百度了很多，都是下面的解决方法 ``` 修改hive的配置文件hive-site.xml <property> <name>datanucleus.schema.autoCreateAll</name> <value>true</value> </property> 然后删除MySQL中原来的表 drop database hive_metastore; 重新初始化元数据 schematool -dbType mysql -initSchema ``` 按照上面的操作执行后，还是报相同的错误 请问，这个如何解决？ 已解决：在多次挣扎后，我尝试了网上很多的方法，但都没有成功；经过多次修改与测试，最终终于找到了问题所在，原来只是java的版本出现的问题，与hive的版本不匹配，我开始使用的是java的最新版jdk-10.0.2,我开始的想法是高版本的能兼容低版本，后来验证我的想法是错的，我将java换成jdk8后，嘿，一下子就可以了，哎，怪我没有好好的认真看一下官方的文档，浪费了我两天的时间啊。
- hive初始化数据库报错
- 这是我的hive-site.xml配置文件中的代码 ``` <configuration> <property> <name>hive.exec.scratchdir</name> <value>/user/hive/tmp</value> </property> <property> <name>hive.metastore.warehouse.dir</name> <value>/user/hive/warehouse</value> </property> <property> <name>hive.querylog.location</name> <value>/user/hive/log</value> </property> <!-- Configure connection information for Mysql --> <property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://172.18.0.5:3306/metastore?createDatabaseIfNotExist=true&characterEncoding=UTF-8&allowPublicKeyRetrieval=true</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.cj.jdbc.driver</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>root</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>123456</value> </property> <!-- change {system:java.io.tmpdir} to /home/hadoop/hive/tmp/; change {system:user.name} to {user.name} --> <property> <name>hive.exec.local.scratchdir</name> <value>/home/hadoop/hive/tmp/${user.name}</value> <description>Local scratch space for Hive jobs</description> </property> </configuration> ``` 进行初始化数据库时显示报错 ``` root@hadoop1:/usr/local/hive/bin# schematool -dbType mysql -initSchema --verbose SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection URL: jdbc:mysql://172.18.0.5:3306/metastore?createDatabaseIfNotExist=true&characterEncoding=UTF-8&allowPublicKeyRetrieval=true Metastore Connection Driver : com.mysql.cj.jdbc.driver Metastore connection User: root org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver Underlying cause: java.lang.ClassNotFoundException : com.mysql.cj.jdbc.driver org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver at org.apache.hive.beeline.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:79) at org.apache.hive.beeline.HiveSchemaTool.getConnectionToMetastore(HiveSchemaTool.java:144) at org.apache.hive.beeline.HiveSchemaTool.testConnectionToMetastore(HiveSchemaTool.java:473) at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:577) at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:563) at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1145) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:564) at org.apache.hadoop.util.RunJar.run(RunJar.java:244) at org.apache.hadoop.util.RunJar.main(RunJar.java:158) Caused by: java.lang.ClassNotFoundException: com.mysql.cj.jdbc.driver at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:582) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:190) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:291) at org.apache.hive.beeline.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:70) ... 11 more *** schemaTool failed *** ``` 请问这个如何解决，上面说没有加载驱动程序，但是在hive的lib文件夹中我有jar包的 ``` root@hadoop1:/usr/local/hive/bin# ll ../lib/mysql-connector-java-8.0.17.jar -rw-r--r-- 1 root root 2321813 Jun 7 07:49 ../lib/mysql-connector-java-8.0.17.jar root@hadoop1:/usr/local/hive/bin# ``` 我所使用的数据库时msyql8.0.17,数据库的操作如下： ``` mysql> create database metastore; mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | metastore | | mysql | | performance_schema | | sys | +--------------------+ 5 rows in set (0.01 sec) ``` 请问一下，这个如何解决?
- hive on hbase报错，在hive中创建映射表，关联到hbase上
- hive on hbase,在hive中创建映射表，关联到hbase上，在hive中已经创建hbase中的row_key,还是报错FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=xlwang5, access=EXECUTE, inode="/user/hive/warehouse":hive:hive:drwxrwx--T
- hive配置oarcle为metastore报错ORA-01754
- 在hive配置远程模式metastore为Oracle，启动正常，创建表时报错 ``` hive> create table dht_tab(name1 int,name45 varchar(50))row format delimited fields terminated by '\t'; FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: An exception was thrown while adding/validating class(es) : ORA-01754: a table may contain only one column of type LONG java.sql.SQLSyntaxErrorException: ORA-01754: a table may contain only one column of type LONG ``` 根据网上修改"hive/lib/hive-metastore-1.2.1.jar"包中package.jdo文件，将LONGVARCHAR类型修改为CLOB，操作如下： ``` cd $HIVE\_HOME/lib mkdir temp cp hive-metastore-1.2.1.jar temp cd temp jar -xvf hive-metastore-1.2.1.jar sed -i -e 's/LONGVARCHAR/CLOB/g' package.jdo jar cfm hive-metastore-1.2.1.jar META-INF/MANIFEST.MF * cp hive-metastore-1.2.1.jar $HIVE_HOME/lib ``` 再通过 hive --service metastore 初始化hive 再进入hive创建表仍然报相同错误。 求解？？？？ 还有一个问题 就是hive --service hiveserver2 没有反应  感谢！
- hive jdbc执行报错MoveTask
- 详细错误 java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:296) at org.apache.hive.jdbc.HiveStatement.executeUpdate(HiveStatement.java:406) at com.dmp.dmp_hive.service.Media_toolsService.loadDate(Media_toolsService.java:96) at com.dmp.dmp_hive.service.Media_toolsService.run(Media_toolsService.java:38) at com.dmp.dmp_hive.test.Test.main(Test.java:48)
- hive+mongodb报错IllegalAccessError
- 用hive+mongodb, ``` CREATE EXTERNAL TABLE test ( id string, test string ) STORED BY 'com.mongodb.hadoop.hive.MongoStorageHandler' WITH SERDEPROPERTIES('mongo.columns.mapping'='{"id":"_id","test":"test"}') TBLPROPERTIES('mongo.uri'='mongodb://root:root@127.0.0.1:27017/test.test'); ``` 创建成功过后，执行查询会报一下错误，不知道谁能帮忙看看 ``` Exception in thread "main" java.lang.IllegalAccessError: tried to access field org.apache.hadoop.hive.ql.io.HiveInputFormat.LOG from class com.mongodb.hadoop.hive.input.HiveMongoInputFormat at com.mongodb.hadoop.hive.input.HiveMongoInputFormat.getSplits(HiveMongoInputFormat.java:81) at com.mongodb.hadoop.hive.input.HiveMongoInputFormat.getSplits(HiveMongoInputFormat.java:44) at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextSplits(FetchOperator.java:363) at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:295) at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:446) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:415) at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140) at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1693) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) ```
- Hive请正常，在show databases;报错，求大神解答
- 2018-02-01T09:46:28,400 WARN [9a4cc1b4-8396-471b-8df0-b1eb3ca1fd82 main] ql.Driver: Caught exception attempting to write metadata call information org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:236) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:388) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:332) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:312) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:354) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:350) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.Driver.dumpMetaCallTimingWithoutEx(Driver.java:683) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:621) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) ~[hive-cli-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) ~[hive-cli-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) ~[hive-cli-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821) ~[hive-cli-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) ~[hive-cli-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) ~[hive-cli-2.3.2.jar:2.3.2] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_151] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_151] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_151] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_151] at org.apache.hadoop.util.RunJar.run(RunJar.java:239) ~[hadoop-common-2.9.0.jar:?] at org.apache.hadoop.util.RunJar.main(RunJar.java:153) ~[hadoop-common-2.9.0.jar:?] Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1701) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3600) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3652) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3632) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3894) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:248) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:231) ~[hive-exec-2.3.2.jar:2.3.2] ... 23 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_151] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_151] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_151] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_151] at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1699) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3600) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3652) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3632) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3894) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:248) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:231) ~[hive-exec-2.3.2.jar:2.3.2] ... 23 more Caused by: org.apache.hadoop.hive.metastore.api.MetaException: Version information not found in metastore. at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:83) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:92) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6893) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:164) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70) ~[hive-exec-2.3.2.jar:2.3.2] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_151] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_151] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_151] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_151] at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1699) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3600) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3652) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3632) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3894) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:248) ~[hive-exec-2.3.2.jar:2.3.2] at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:231) ~[hive-exec-2.3.2.jar:2.3.2] ... 23 more 2018-02-01T09:46:28,400 INFO [9a4cc1b4-8396-471b-8df0-b1eb3ca1fd82 main] ql.Driver: Completed compiling command(queryId=root_20180201094627_6a378f28-ae24-4c00-8d15-a8df87e7020e); Time taken: 0.474 seconds
- hive执行select报错，求大神指导
- hive> select * from user; Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.mapred.JobConf.unset(Ljava/lang/String;)V at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushFilters(HiveInputFormat.java:432) at org.apache.hadoop.hive.ql.exec.FetchTask.initialize(FetchTask.java:76) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:443) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:303) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1067)
- hive查询某个表时报错，查询其他表正常
- 1.如题 2.执行语句为 ``` select * from ccy_info_dist_tmp limit 5; ``` 3.报错信息如下 ``` Error: java.io.IOException: java.io.IOException: Two readers for {originalWriteId: 44, bucket: 536870912(1.0.0), row: 0, currentWriteId 45}: new [key={originalWriteId: 44, bucket: 536870912(1.0.0), row: 0, currentWriteId 45}, nextRecord={2, 44, 536870912, 0, 45, null}, reader=Hive ORC Reader(hdfs:XXX/managed/hive/dw.db/ccy_info_dist_tmp/delete_delta_0000045_0000050/bucket_00001, 9223372036854775807)], old [key={originalWriteId: 44, bucket: 536870912(1.0.0), row: 0, currentWriteId 45}, nextRecord={2, 44, 536870912, 0, 45, null}, reader=Hive ORC Reader(hdfs:XXX/managed/hive/dw.db/ccy_info_dist_tmp/delete_delta_0000045_0000050/bucket_00000, 9223372036854775807)] (state=,code=0) ```
- hive建hbase关联表报错
- hadoop-2.5.2 ，hbase-1.0.1.1，hive-0.9.0 hive> CREATE TABLE hbase_table_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "xyz"); FAILED: Error in metadata: MetaException(message:java.io.IOException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240) at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:410) at org.apache.hadoop.hbase.client.ConnectionManager.createConnection(ConnectionManager.java:403) at org.apache.hadoop.hbase.client.ConnectionManager.getConnectionInternal(ConnectionManager.java:281) at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:202) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:73) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:147) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:398) at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:538) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3305) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:242) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1326) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1118) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:951) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:557) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:212) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238) ... 25 more Caused by: java.lang.NoClassDefFoundError: org/apache/htrace/Trace at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:218) at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481) at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65) at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:833) at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:623) ... 30 more Caused by: java.lang.ClassNotFoundException: org.apache.htrace.Trace at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 36 more ) FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
- hive udf报错SemanticException [Error 10014]
- **hive的udf怎么引入依赖呢,我通过下面方式添加了第三方依赖** ``` add file /home/chenxy/hive/GeoLite2-City.mmdb; add jar /home/chenxy/jars/spark-hive_2.11-2.1.0.jar; add jar /home/chenxy/jars/geoip2-2.12.0.jar; add jar /home/chenxy/jars/IpCity.jar; create temporary function ip2poi as 'com.tianzhuo.portrait.Ip2PoiUDF'; select ip2poi("183.128.104.19"); ``` select ip2poi("183.128.104.19");操作后错误日志如下: ``` FAILED: SemanticException [Error 10014]: Line 1:7 Wrong arguments '"183.128.104.19"': org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute method public java.lang.String com.tianzhuo.portrait.Ip2PoiUDF.evaluate(java.lang.String) on object com.tianzhuo.portrait.Ip2PoiUDF@23592946 of class com.tianzhuo.portrait.Ip2PoiUDF with arguments {183.128.104.19:java.lang.String} of size 1 ``` 如果evaluate函数直接返回字符串是没有问题的，加上geoip2-2.12.0.jar相关代码就会报错 求助大佬们，困扰我两天了，百度谷歌搜遍了，也没合适解决方案
- hadoop hive 初始化报错
- 环境：centos7 hadoop3.1.2 hive2.3.0 [root@hadoop metastore]# schematool -initSchema -dbType mysql Exception in thread "main" java.lang.ClassNotFoundException: org.apache.hive.beeline.HiveSchemaTool at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.hadoop.util.RunJar.run(RunJar.java:311) at org.apache.hadoop.util.RunJar.main(RunJar.java:232) [root@hadoop metastore]# hive which: no hbase in (/root/mysql/apache-hive-2.3.0-bin/bin:/root/train/hadoop-3.1.2/sbin:/root/train/hadoop-3.1.2/bin:/root/train/jdk1.8.0_211/bin:/root/train/hive/bin:/root/train/hadoop-3.1.2/sbin:/root/train/hadoop-3.1.2/bin:/root/train/jdk1.8.0_211/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin) Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/CommandNeedRetryException at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.hadoop.util.RunJar.run(RunJar.java:311) at org.apache.hadoop.util.RunJar.main(RunJar.java:232) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.CommandNeedRetryException at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 4 more Navicat Premium 都能成功登入mysql建立的hive数据库 hive 配置文件都没问题 mysql连接器也放入了lib
- Hive1.1对接sequoiadb1.12报错
- 【版本信息】 Hive1.1 hadoop2.6 sequoiadb1.12 【问题详细描述】 sdb存在default.sdb_tab表和数据，在hive执行"select id from sdb_tab"对该表做查询时报“Exception in thread "main" java.lang.NoSuchMethodError”，详细操作如下： 1、sdb中存在default.sdb_tab表和数据，可正常执行如下命令查看： > db.default.sdb_tab.find(); { "_id": { "$oid": "57610212324d1a5779000000" }, "name": "chen", "id": 123 } Return 1 row(s). Takes 0.2828s. 复制代码 3、hive中执行如下命令报错 hive> select id from sdb_tab; OK Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hive.serde2.ColumnProjectionUtils.getReadColumnIDs(Lorg/apache/hadoop/conf/Configuration;)Ljava/util/ArrayList; at com.sequoiadb.hive.SdbHiveInputFormat.getRecordReader(SdbHiveInputFormat.java:35) at org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit.getRecordReader(FetchOperator.java:667) at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:323) at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:445) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:414) at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:138) at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1655) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:227) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:754) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) 复制代码 【提问】 1、报错原因？ 2、Hive如何在指定集合空间创建一张表，如:foo.bar ，并关联到sequoiadb？
- hive跟hbase整合用hive导入数据报错，报一个路径不是目录。
- hive>load data local inpath '/home/hadoop/ha1.txt' into table ha1; FAILED: Hive Internal Error: java.lang.RuntimeException(org.apache.hadoop.ipc.RemoteException: java.io.FileNotFoundException: Parent path is not a directory: /usr/local at org.apache.hadoop.hdfs.server.namenode.FSDirectory.mkdirs(FSDirectory.java:956) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:2101) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:2062) at org.apache.hadoop.hdfs.server.namenode.NameNode.mkdirs(NameNode.java:892) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1439) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1435) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1433) ) java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException: java.io.FileNotFoundException: Parent path is not a directory: /usr/local at org.apache.hadoop.hdfs.server.namenode.FSDirectory.mkdirs(FSDirectory.java:956) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:2101) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:2062) at org.apache.hadoop.hdfs.server.namenode.NameNode.mkdirs(NameNode.java:892) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1439) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1435) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1433) at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:170) at org.apache.hadoop.hive.ql.Context.getExternalScratchDir(Context.java:222) at org.apache.hadoop.hive.ql.Context.getExternalTmpFileURI(Context.java:315) at org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.analyzeInternal(LoadSemanticAnalyzer.java:225) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:243) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:431) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:336) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:909) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:689) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:557) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:197) Caused by: org.apache.hadoop.ipc.RemoteException: java.io.FileNotFoundException: Parent path is not a directory: /usr/local at org.apache.hadoop.hdfs.server.namenode.FSDirectory.mkdirs(FSDirectory.java:956) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:2101) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:2062) at org.apache.hadoop.hdfs.server.namenode.NameNode.mkdirs(NameNode.java:892) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1439) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1435) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1433) at org.apache.hadoop.ipc.Client.call(Client.java:1150) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226) at com.sun.proxy.$Proxy4.mkdirs(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) at com.sun.proxy.$Proxy4.mkdirs(Unknown Source) at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:1295) at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:323) at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1298) at org.apache.hadoop.hive.ql.Context.getScratchDir(Context.java:165) ... 17 more
- 爬虫福利二 之 妹子图网MM批量下载
- 爬虫福利一：27报网MM批量下载 点击 看了本文，相信大家对爬虫一定会产生强烈的兴趣，激励自己去学习爬虫，在这里提前祝：大家学有所成！ 目标网站：妹子图网 环境：Python3.x 相关第三方模块：requests、beautifulsoup4 Re：各位在测试时只需要将代码里的变量 path 指定为你当前系统要保存的路径，使用 python xxx.py 或IDE运行即可。
- Java学习的正确打开方式
- 在博主认为，对于入门级学习java的最佳学习方法莫过于视频+博客+书籍+总结，前三者博主将淋漓尽致地挥毫于这篇博客文章中，至于总结在于个人，实际上越到后面你会发现学习的最好方式就是阅读参考官方文档其次就是国内的书籍，博客次之，这又是一个层次了，这里暂时不提后面再谈。博主将为各位入门java保驾护航，各位只管冲鸭！！！上天是公平的，只要不辜负时间，时间自然不会辜负你。 何谓学习？博主所理解的学习，它
- 程序员必须掌握的核心算法有哪些？
- 由于我之前一直强调数据结构以及算法学习的重要性，所以就有一些读者经常问我，数据结构与算法应该要学习到哪个程度呢？，说实话，这个问题我不知道要怎么回答你，主要取决于你想学习到哪些程度，不过针对这个问题，我稍微总结一下我学过的算法知识点，以及我觉得值得学习的算法。这些算法与数据结构的学习大多数是零散的，并没有一本把他们全部覆盖的书籍。下面是我觉得值得学习的一些算法以及数据结构，当然，我也会整理一些看过
- 大学四年自学走来，这些私藏的实用工具/学习网站我贡献出来了
- 大学四年，看课本是不可能一直看课本的了，对于学习，特别是自学，善于搜索网上的一些资源来辅助，还是非常有必要的，下面我就把这几年私藏的各种资源，网站贡献出来给你们。主要有：电子书搜索、实用工具、在线视频学习网站、非视频学习网站、软件下载、面试/求职必备网站。 注意：文中提到的所有资源，文末我都给你整理好了，你们只管拿去，如果觉得不错，转发、分享就是最大的支持了。 一、PDF搜索网站推荐 对于大部
- linux系列之常用运维命令整理笔录
- 本博客记录工作中需要的linux运维命令，大学时候开始接触linux，会一些基本操作，可是都没有整理起来，加上是做开发，不做运维，有些命令忘记了，所以现在整理成博客，当然vi，文件操作等就不介绍了，慢慢积累一些其它拓展的命令，博客不定时更新 顺便拉下票，我在参加csdn博客之星竞选，欢迎投票支持，每个QQ或者微信每天都可以投5票，扫二维码即可，http://m234140.nofollow.ax.
- 比特币原理详解
- 一、什么是比特币 比特币是一种电子货币，是一种基于密码学的货币，在2008年11月1日由中本聪发表比特币白皮书，文中提出了一种去中心化的电子记账系统，我们平时的电子现金是银行来记账，因为银行的背后是国家信用。去中心化电子记账系统是参与者共同记账。比特币可以防止主权危机、信用风险。其好处不多做赘述，这一层面介绍的文章很多，本文主要从更深层的技术原理角度进行介绍。 二、问题引入 假设现有4个人
- 程序员接私活怎样防止做完了不给钱？
- 首先跟大家说明一点，我们做 IT 类的外包开发，是非标品开发，所以很有可能在开发过程中会有这样那样的需求修改，而这种需求修改很容易造成扯皮，进而影响到费用支付，甚至出现做完了项目收不到钱的情况。 那么，怎么保证自己的薪酬安全呢？ 我们在开工前，一定要做好一些证据方面的准备（也就是“讨薪”的理论依据），这其中最重要的就是需求文档和验收标准。一定要让需求方提供这两个文档资料作为开发的基础。之后开发
- 网页实现一个简单的音乐播放器（大佬别看。(⊙﹏⊙)）
- 今天闲着无事，就想写点东西。然后听了下歌，就打算写个播放器。 于是乎用h5 audio的加上js简单的播放器完工了。 欢迎 改进 留言。 演示地点跳到演示地点 html代码如下`<!DOCTYPE html> <html> <head> <title>music</title> <meta charset="utf-8">
- Python十大装B语法
- Python 是一种代表简单思想的语言，其语法相对简单，很容易上手。不过，如果就此小视 Python 语法的精妙和深邃，那就大错特错了。本文精心筛选了最能展现 Python 语法之精妙的十个知识点，并附上详细的实例代码。如能在实战中融会贯通、灵活使用，必将使代码更为精炼、高效，同时也会极大提升代码B格，使之看上去更老练，读起来更优雅。 1. for - else 什么？不是 if 和 else 才
- 数据库优化 - SQL优化
- 前面一篇文章从实例的角度进行数据库优化，通过配置一些参数让数据库性能达到最优。但是一些“不好”的SQL也会导致数据库查询变慢，影响业务流程。本文从SQL角度进行数据库优化，提升SQL运行效率。 判断问题SQL 判断SQL是否有问题时可以通过两个表象进行判断： 系统级别表象 CPU消耗严重 IO等待严重 页面响应时间过长
- 2019年11月中国大陆编程语言排行榜
- 2019年11月2日，我统计了某招聘网站，获得有效程序员招聘数据9万条。针对招聘信息，提取编程语言关键字，并统计如下： 编程语言比例 rank pl_ percentage 1 java 33.62% 2 c/c++ 16.42% 3 c_sharp 12.82% 4 javascript 12.31% 5 python 7.93% 6 go 7.25% 7
- 通俗易懂地给女朋友讲：线程池的内部原理
- 餐厅的约会 餐盘在灯光的照耀下格外晶莹洁白，女朋友拿起红酒杯轻轻地抿了一小口，对我说：“经常听你说线程池，到底线程池到底是个什么原理？”我楞了一下，心里想女朋友今天是怎么了，怎么突然问出这么专业的问题，但做为一个专业人士在女朋友面前也不能露怯啊，想了一下便说：“我先给你讲讲我前同事老王的故事吧！” 大龄程序员老王 老王是一个已经北漂十多年的程序员，岁数大了，加班加不动了，升迁也无望，于是拿着手里
- 经典算法（5）杨辉三角
- 写在前面： 我是 扬帆向海，这个昵称来源于我的名字以及女朋友的名字。我热爱技术、热爱开源、热爱编程。技术是开源的、知识是共享的。 这博客是对自己学习的一点点总结及记录，如果您对 Java、算法 感兴趣，可以关注我的动态，我们一起学习。 用知识改变命运，让我们的家人过上更好的生活。 目录一、杨辉三角的介绍二、杨辉三角的算法思想三、代码实现1.第一种写法2.第二种写法 一、杨辉三角的介绍 百度
- 腾讯算法面试题：64匹马8个跑道需要多少轮才能选出最快的四匹？
- 昨天，有网友私信我，说去阿里面试，彻底的被打击到了。问了为什么网上大量使用ThreadLocal的源码都会加上private static？他被难住了，因为他从来都没有考虑过这个问题。无独有偶，今天笔者又发现有网友吐槽了一道腾讯的面试题，我们一起来看看。 腾讯算法面试题：64匹马8个跑道需要多少轮才能选出最快的四匹？ 在互联网职场论坛，一名程序员发帖求助到。二面腾讯，其中一个算法题：64匹
- 面试官：你连RESTful都不知道我怎么敢要你？
- 面试官：了解RESTful吗？ 我：听说过。 面试官：那什么是RESTful？ 我：就是用起来很规范，挺好的 面试官：是RESTful挺好的，还是自我感觉挺好的 我：都挺好的。 面试官：… 把门关上。 我：… 要干嘛？先关上再说。 面试官：我说出去把门关上。 我：what ？，夺门而去 文章目录01 前言02 RESTful的来源03 RESTful6大原则1. C-S架构2. 无状态3.统一的接
- 为啥国人偏爱Mybatis，而老外喜欢Hibernate/JPA呢？
- 关于SQL和ORM的争论，永远都不会终止，我也一直在思考这个问题。昨天又跟群里的小伙伴进行了一番讨论，感触还是有一些，于是就有了今天这篇文。 声明：本文不会下关于Mybatis和JPA两个持久层框架哪个更好这样的结论。只是摆事实，讲道理，所以，请各位看官勿喷。 一、事件起因 关于Mybatis和JPA孰优孰劣的问题，争论已经很多年了。一直也没有结论，毕竟每个人的喜好和习惯是大不相同的。我也看
- SQL-小白最佳入门sql查询一
- 一 说明 如果是初学者，建议去网上寻找安装Mysql的文章安装，以及使用navicat连接数据库，以后的示例基本是使用mysql数据库管理系统； 二 准备前提 需要建立一张学生表，列分别是id，名称，年龄，学生信息；本示例中文章篇幅原因SQL注释略； 建表语句： CREATE TABLE `student` ( `id` int(11) NOT NULL AUTO_INCREMENT, `
- 项目中的if else太多了，该怎么重构？
- 介绍 最近跟着公司的大佬开发了一款IM系统，类似QQ和微信哈，就是聊天软件。我们有一部分业务逻辑是这样的 if (msgType = "文本") { // dosomething } else if(msgType = "图片") { // doshomething } else if(msgType = "视频") { // doshomething } else { // dosho
- 【图解经典算法题】如何用一行代码解决约瑟夫环问题
- 约瑟夫环问题算是很经典的题了，估计大家都听说过，然后我就在一次笔试中遇到了，下面我就用 3 种方法来详细讲解一下这道题，最后一种方法学了之后保证让你可以让你装逼。 问题描述：编号为 1-N 的 N 个士兵围坐在一起形成一个圆圈，从编号为 1 的士兵开始依次报数（1，2，3…这样依次报），数到 m 的 士兵会被杀死出列，之后的士兵再从 1 开始报数。直到最后剩下一士兵，求这个士兵的编号。 1、方
- 致 Python 初学者
- 文章目录1. 前言2. 明确学习目标，不急于求成，不好高骛远3. 在开始学习 Python 之前，你需要做一些准备2.1 Python 的各种发行版2.2 安装 Python2.3 选择一款趁手的开发工具3. 习惯使用IDLE，这是学习python最好的方式4. 严格遵从编码规范5. 代码的运行、调试5. 模块管理5.1 同时安装了py2/py35.2 使用Anaconda，或者通过IDE来安装模
- “狗屁不通文章生成器”登顶GitHub热榜，分分钟写出万字形式主义大作
- 一、垃圾文字生成器介绍 最近在浏览GitHub的时候，发现了这样一个骨骼清奇的雷人项目，而且热度还特别高。 项目中文名：狗屁不通文章生成器 项目英文名：BullshitGenerator 根据作者的介绍，他是偶尔需要一些中文文字用于GUI开发时测试文本渲染，因此开发了这个废话生成器。但由于生成的废话实在是太过富于哲理，所以最近已经被小伙伴们给玩坏了。 他的文风可能是这样的： 你发现，
- 程序员：我终于知道post和get的区别
- IT界知名的程序员曾说：对于那些月薪三万以下，自称IT工程师的码农们，其实我们从来没有把他们归为我们IT工程师的队伍。他们虽然总是以IT工程师自居，但只是他们一厢情愿罢了。 此话一出，不知激起了多少(码农)程序员的愤怒，却又无可奈何，于是码农问程序员。 码农：你知道get和post请求到底有什么区别？ 程序员：你看这篇就知道了。 码农：你月薪三万了？ 程序员：嗯。 码农：你是怎么做到的? 程序员：
- 《程序人生》系列-这个程序员只用了20行代码就拿了冠军
- 你知道的越多，你不知道的越多 点赞再看，养成习惯GitHub上已经开源https://github.com/JavaFamily，有一线大厂面试点脑图，欢迎Star和完善 前言 这一期不算《吊打面试官》系列的，所有没前言我直接开始。 絮叨 本来应该是没有这期的，看过我上期的小伙伴应该是知道的嘛，双十一比较忙嘛，要值班又要去帮忙拍摄年会的视频素材，还得搞个程序员一天的Vlog，还要写BU
- 加快推动区块链技术和产业创新发展，2019可信区块链峰会在京召开
- 11月8日，由中国信息通信研究院、中国通信标准化协会、中国互联网协会、可信区块链推进计划联合主办，科技行者协办的2019可信区块链峰会将在北京悠唐皇冠假日酒店开幕。 区块链技术被认为是继蒸汽机、电力、互联网之后，下一代颠覆性的核心技术。如果说蒸汽机释放了人类的生产力，电力解决了人类基本的生活需求，互联网彻底改变了信息传递的方式，区块链作为构造信任的技术有重要的价值。 1
- 程序员把地府后台管理系统做出来了，还有3.0版本！12月7号最新消息：已在开发中有github地址
- 第一幕：缘起 听说阎王爷要做个生死簿后台管理系统，我们派去了一个程序员…… 996程序员做的梦： 第一场：团队招募 为了应对地府管理危机，阎王打算找“人”开发一套地府后台管理系统，于是就在地府总经办群中发了项目需求。 话说还是中国电信的信号好，地府都是满格，哈哈！！！ 经常会有外行朋友问：看某网站做的不错，功能也简单，你帮忙做一下？ 而这次，面对这样的需求，这个程序员
- 网易云6亿用户音乐推荐算法
- 网易云音乐是音乐爱好者的集聚地，云音乐推荐系统致力于通过 AI 算法的落地，实现用户千人千面的个性化推荐，为用户带来不一样的听歌体验。 本次分享重点介绍 AI 算法在音乐推荐中的应用实践，以及在算法落地过程中遇到的挑战和解决方案。 将从如下两个部分展开： AI 算法在音乐推荐中的应用 音乐场景下的 AI 思考 从 2013 年 4 月正式上线至今，网易云音乐平台持续提供着：乐屏社区、UGC
- 【技巧总结】位运算装逼指南
- 位算法的效率有多快我就不说，不信你可以去用 10 亿个数据模拟一下，今天给大家讲一讲位运算的一些经典例子。不过，最重要的不是看懂了这些例子就好，而是要在以后多去运用位运算这些技巧，当然，采用位运算，也是可以装逼的，不信，你往下看。我会从最简单的讲起，一道比一道难度递增，不过居然是讲技巧，那么也不会太难，相信你分分钟看懂。 判断奇偶数 判断一个数是基于还是偶数，相信很多人都做过，一般的做法的代码如下
- 日均350000亿接入量，腾讯TubeMQ性能超过Kafka
- 整理 | 夕颜出品 | AI科技大本营（ID:rgznai100） 【导读】近日，腾讯开源动作不断，相继开源了分布式消息中间件TubeMQ，基于最主流的 OpenJDK8开发的
- 8年经验面试官详解 Java 面试秘诀
- 作者 | 胡书敏 责编 | 刘静 出品 | CSDN（ID：CSDNnews） 本人目前在一家知名外企担任架构师，而且最近八年来，在多家外企和互联网公司担任Java技术面试官，前后累计面试了有两三百位候选人。在本文里，就将结合本人的面试经验，针对Java初学者、Java初级开发和Java开发，给出若干准备简历和准备面试的建议。 Java程序员准备和投递简历的实
- 面试官如何考察你的思维方式？
- 1.两种思维方式在求职面试中，经常会考察这种问题：北京有多少量特斯拉汽车？ 某胡同口的煎饼摊一年能卖出多少个煎饼？ 深圳有多少个产品经理？ 一辆公交车里能装下多少个乒乓球？ 一
