问题遇到的现象和发生背景
在创建hive UDTF函数报错,Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.FunctionTask
问题相关代码,请勿粘贴截图
create function explode_json_array as 'com.boboli.ExplodeJsonArray' using jar 'hdfs://hadoop102:9870/user/hive/jars/hiveUDTF-1.0-SNAPSHOT.jar';
运行结果及报错内容
Failed on local exception: org.apache.hadoop.ipc.RpcException: RPC response exceeds maximum data length; Host Details : local host is: "hadoop102/192.168.18.102"; destination host is: "hadoop102":9870;
Failed to register default.explode_json_array using class com.boboli.ExplodeJsonArray
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.FunctionTask
我的解答思路和尝试过的方法
按照网上修改yarn-site.xml的设置,如下
<property>
<name>yarn.scheduler.maximum-allocation-mb</name>
<value>10000</value>
</property>
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>2048</value>
</property>
<property>
<name>yarn.nodemanager.vmem-pmem-ratio</name>
<value>2.1</value>
</property>
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx1024m</value>
</property>
我想要达到的结果
能够顺利得到udtf函数表达