2 leoricxu leoricxu 于 2013.08.15 19:11 提问

hive中执行查询报错,请哥哥们解决

org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /tmp/hive-root/hive_2013-08-15_13-46-55_630_3316993537720067107/-mr-10002/1/emptyFile could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1639)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:729)
at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387)
at org.apache.hadoop.ipc.Client.call(Client.java:1107)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at $Proxy5.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at $Proxy5.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3686)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3546)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2749)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2989)
Job Submission failed with exception 'org.apache.hadoop.ipc.RemoteException(java.io.IOException: File /tmp/hive-root/hive_2013-08-15_13-46-55_630_3316993537720067107/-mr-10002/1/emptyFile could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1639)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:729)
at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1387))'
java.lang.IllegalArgumentException: Can not create a Path from an empty string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
at org.apache.hadoop.fs.Path.(Path.java:90)
at org.apache.hadoop.hive.ql.exec.Utilities.getHiveJobID(Utilities.java:379)
at org.apache.hadoop.hive.ql.exec.Utilities.clearMapRedWork(Utilities.java:192)
at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:476)
at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:136)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:133)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1332)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1123)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:931)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MapRedTask

2个回答

zgc_sitech
zgc_sitech   2013.10.11 17:28
已采纳

这是你hadoop的block的冗余数量没设置正确啊,看看你的hadoop集群有几个datanode,然后就设置成几个

zenglan_930603
zenglan_930603   2016.04.07 10:08

楼主有没有解决呀,我也报了这个错,输入命令

  create table my_test as select * from testmongodb;

报错信息:

 WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Query ID = root_20160407093010_f3279c7a-d06e-4976-bf94-66a3136ff1aa
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there's no reduce operator
java.lang.RuntimeException: Error caching map.xml: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /tmp/hive/root/86998678-c305-498d-9c29-6456a79d0de4/hive_2016-04-07_09-30-10_999_1883084428170569741-1/-mr-10006/3bebc352-c112-4f2d-b462-f672a1518581/map.xml could only be replicated to 0 nodes instead of minReplication (=1).  There are 3 datanode(s) running and 3 node(s) are excluded in this operation.
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1550)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3067)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:722)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)

        at org.apache.hadoop.hive.ql.exec.Utilities.setBaseWork(Utilities.java:577)
        at org.apache.hadoop.hive.ql.exec.Utilities.setMapWork(Utilities.java:511)
        at org.apache.hadoop.hive.ql.exec.Utilities.setMapRedWork(Utilities.java:503)
        at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:377)
        at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:138)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:158)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:101)
        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1840)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1584)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1361)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1184)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1172)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:400)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:778)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:717)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:645)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /tmp/hive/root/86998678-c305-498d-9c29-6456a79d0de4/hive_2016-04-07_09-30-10_999_1883084428170569741-1/-mr-10006/3bebc352-c112-4f2d-b462-f672a1518581/map.xml could only be replicated to 0 nodes instead of minReplication (=1).  There are 3 datanode(s) running and 3 node(s) are excluded in this operation.
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1550)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3067)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:722)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)

        at org.apache.hadoop.ipc.Client.call(Client.java:1476)
        at org.apache.hadoop.ipc.Client.call(Client.java:1407)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
        at com.sun.proxy.$Proxy30.addBlock(Unknown Source)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:418)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
        at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
        at com.sun.proxy.$Proxy31.addBlock(Unknown Source)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1430)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1226)
        at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:449)
Job Submission failed with exception 'java.lang.RuntimeException(Error caching map.xml: org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /tmp/hive/root/86998678-c305-498d-9c29-6456a79d0de4/hive_2016-04-07_09-30-10_999_1883084428170569741-1/-mr-10006/3bebc352-c112-4f2d-b462-f672a1518581/map.xml could only be replicated to 0 nodes instead of minReplication (=1).  There are 3 datanode(s) running and 3 node(s) are excluded in this operation.
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1550)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3067)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:722)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2049)
        at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2045)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2043)
)'
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
Csdn user default icon
上传中...
上传图片
插入图片
准确详细的回答,更有利于被提问者采纳,从而获得C币。复制、灌水、广告等回答会被删除,是时候展现真正的技术了!
其他相关推荐
hive count表提交mapreduce任务报错。
今天使用hive时发现用select * 查询单表,查询正常。 使用select 字段名 ,select count(*) ,及关联表查询时发生如下异常错误: Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0:      file:///service/cdh/hive/lib
hive无法执行带where语句的SQL
应用场景 当在伪分布式集群上,搭建部署了hive以后,发现hive无法执行带where语句的sql,那hive将无法使用,下面介绍解决该问题的方案! 操作步骤 hive连接执行sql,可以执行带where语句的sql,但是采用beeline连接,无法执行带where的sql语句,报could not initialize calss org.apache.hadoop.hive.ql.op
java程序调用hive查询的一个异常
最近在java程序中调用hive做查询时,碰到一个异常,被困扰了许久,经过几番调试,逐步把问题定位清楚。 在异常描述前先给出异常信息:java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.
hive中Hql查询时错误
查询时错误  hive> select count(*) from student_info; Query ID = hadoop_20170127022626_4c071a70-f0d0-484a-8e29-dbd875356ef9 Total jobs = 1 Launching Job 1 out of 1 Number of reduce tasks determ
hiveshell不能执行select语句问题
最近碰到一个很玄学的问题,就是我的CDH集群中部署的hive,在hue上可以执行hive查询语句,但是到hiveshell中不能执行!废话不多说直接看报错图吧 猛的一看以为缺啥jar包,但是想想不应该啊,我用cludera CDH部署的,然后带着这个问题去问度娘,度娘告诉我说不hive版本与hadoop版本不一致,我当时是崩溃的,不应该啊,我cdh、 部署不存在这个问题的。然后各种求教
Hive执行sql,报内存溢出
应用场景 搭建了hadoop伪分布式集群,并且在其上搭建了hive环境,但是在执行HQL语句的时候,老是报内存溢出,很麻烦,只需要简单设置,就可以解决该问题。 操作方案 运行程序过程中,报内存溢出:java.sql.SQLException: Error running query: java.lang.OutOfMemoryError: PermGen space 解决方案: 在
关于hive数据库查询操作出现的空指针问题
StringBuilder pSql = new StringBuilder(); pSql.append(" select p.* from bigdata_app_push_seq_").append(dateTime); pSql.append(" p "); pSql.append(" where p.push_time >=").append(dayStartTim
Hive调用MapReduce任务失败
1. Hive简介Hive(蜂巢)Apache Hadoop生态圈的构成之一,其主要功能是基于Hadoop提供MapReduce的类SQL查询。Hive的语法规则和Mysql中SQL的语法规则极为相似,有Mysql使用经验的同学都能轻松上手。 Hive的执行分为本地模式和集群模式。本地模式执行简单的Hive SQL,不需要启动MapReduce程序,如-- 本地模式 select col_name
Hue中Hive editor中文报错
今天一个同事在Hue中使用Hive 进行数据分析时,报错了,是因为在Hive sql中加了一个过滤条件,过滤条件的值中包含中文,错误如下:              一看到这个,自然就想到了字符编码的问题。     既然知道是编码问题,可是在哪里修改呢?     经过查阅了一些资料,找到了一种解决方法(新建sitecustomize.py文件): /usr/lib/python2.7/s
记一次hive的内存溢出(OutOfMemoryError: Java heap space)排查
刚刚从java组转岗找数据组,学习大数据的知识,开发语言也从java转到python新奇之外也遇到了诸多问题,其中最令我头疼的就是在hive上的统计任务总是三天两头地报告OutOfMemory一开始因为有别的同事的任务干扰,我以为是计算资源不够导致的,但后来把任务转移到了另一个集群后,问题仍然存在,让我百思不得其解。 问题的具体描述如下: 统计逻辑使用python pyhs2连接hiv