qq_40678224 2018-10-11 08:59 采纳率: 0%
浏览 2119

使用sqoop2把hive表导出道pgsql时报错,数据只导出了一部分,如何解决?

18/10/11 16:30:39 INFO mapreduce.Job: Running job: job_1539239753824_0018
18/10/11 16:30:45 INFO mapreduce.Job: Job job_1539239753824_0018 running in uber mode : false
18/10/11 16:30:45 INFO mapreduce.Job: map 0% reduce 0%
18/10/11 16:30:56 INFO mapreduce.Job: map 19% reduce 0%
18/10/11 16:30:59 INFO mapreduce.Job: map 27% reduce 0%
18/10/11 16:31:01 INFO mapreduce.Job: map 100% reduce 0%
18/10/11 16:31:02 INFO mapreduce.Job: Job job_1539239753824_0018 failed with state FAILED due to: Task failed task_1539239753824_0018_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

18/10/11 16:31:02 INFO mapreduce.Job: Counters: 8
Job Counters
Failed map tasks=1
Launched map tasks=1
Rack-local map tasks=1
Total time spent by all maps in occupied slots (ms)=13204
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=13204
Total vcore-milliseconds taken by all map tasks=13204
Total megabyte-milliseconds taken by all map tasks=13520896
18/10/11 16:31:02 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
18/10/11 16:31:02 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 24.5676 seconds (0 bytes/sec)
18/10/11 16:31:02 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
18/10/11 16:31:02 INFO mapreduce.ExportJobBase: Exported 0 records.
18/10/11 16:31:02 ERROR mapreduce.ExportJobBase: Export job failed!
18/10/11 16:31:02 ERROR tool.ExportTool: Error during export:
Export job failed!
at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:445)
at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:931)
at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:80)
at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:99)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

  • 写回答

1条回答 默认 最新

  • qq_40678224 2018-10-11 09:00
    关注

    hive的表字段是4个string,一个int,一个timestamp,对应pg里面是4个text,一个integer,一个timestamp

    sqoop export --connect jdbc:postgresql://192.168.136.151:5432/test --username postgres --password 'postgres' --table t_merchant_infocontent --export-dir /user/hive/warehouse/userdb.db/t_merchant_infocontent_for_mysql/* --input-fields-terminated-by '\t'

    评论

报告相同问题?

悬赏问题

  • ¥15 drone 推送镜像时候 purge: true 推送完毕后没有删除对应的镜像,手动拷贝到服务器执行结果正确在样才能让指令自动执行成功删除对应镜像,如何解决?
  • ¥15 求daily translation(DT)偏差订正方法的代码
  • ¥15 js调用html页面需要隐藏某个按钮
  • ¥15 ads仿真结果在圆图上是怎么读数的
  • ¥20 Cotex M3的调试和程序执行方式是什么样的?
  • ¥20 java项目连接sqlserver时报ssl相关错误
  • ¥15 一道python难题3
  • ¥15 牛顿斯科特系数表表示
  • ¥15 arduino 步进电机
  • ¥20 程序进入HardFault_Handler