xinggu_ 2022-12-17 11:31 采纳率: 100%
浏览 63
已结题

ubuntu虚拟机,在MySQL的数据库dblab中创建一个新表user_action时报错

输入

./bin/sqoop export --connect jdbc:mysql://localhost:3306/dblab --username root --password hadoop --table user_action --export-dir '/user/hive/warehouse/dblab.db/user_action' --fields-terminated-by '\t';

img

1.已启动hadoop,mysql
2.报错了N多bug,实在是不知道哪一个方面出错了,前面HCatalog,Accumulo,Zookeeper是没有配置的,不用配置,后面的不清楚怎么解决
3.使用版本 hbase2.2.2 hadoop3.3.1 mysql 5.1.40 hive2.3.9
4.查到重装hadoop好像可以解决,但要重装hbase,mysql和hive吗?实在是没时间重装了,看看有没有解决办法,辛苦各位了!
输入代码解析:

./bin/sqoop export  ##表示数据从 hive 复制到 mysql 中
--connect jdbc:mysql://localhost:3306/dblab 
--username root  #mysql登陆用户名
--password hadoop  #登录密码
--table user_action  #mysql 中的表,即将被导入的表名称  
--export-dir '/user/hive/warehouse/dblab.db/user_action '  #hive 中被导出的文件 
--fields-terminated-by '\t'   #Hive 中被导出的文件字段的分隔符

以下是输出结果:

Warning: /usr/local/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /usr/local/sqoop/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /usr/local/sqoop/../zookeeper does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
错误: 找不到或无法加载主类 org.apache.hadoop.hbase.util.GetJavaProperty
2022-12-17 10:35:51,205 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
2022-12-17 10:35:51,275 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
2022-12-17 10:35:51,484 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
2022-12-17 10:35:51,484 INFO tool.CodeGenTool: Beginning code generation
Sat Dec 17 10:35:51 ULAT 2022 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2022-12-17 10:35:52,717 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_action` AS t LIMIT 1
2022-12-17 10:35:52,808 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `user_action` AS t LIMIT 1
2022-12-17 10:35:52,813 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/local/hadoop
注: /tmp/sqoop-hadoop/compile/1c6411929697a69709780e8a71806dbe/user_action.java使用或覆盖了已过时的 API。
注: 有关详细信息, 请使用 -Xlint:deprecation 重新编译。
2022-12-17 10:35:55,350 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/1c6411929697a69709780e8a71806dbe/user_action.jar
2022-12-17 10:35:55,358 INFO mapreduce.ExportJobBase: Beginning export of user_action
2022-12-17 10:35:55,359 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
2022-12-17 10:35:55,562 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2022-12-17 10:35:55,644 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
2022-12-17 10:35:57,495 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
2022-12-17 10:35:57,498 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
2022-12-17 10:35:57,503 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
2022-12-17 10:35:57,716 INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
2022-12-17 10:35:57,906 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
2022-12-17 10:35:57,906 INFO impl.MetricsSystemImpl: JobTracker metrics system started
2022-12-17 10:35:58,155 INFO input.FileInputFormat: Total input files to process : 1
2022-12-17 10:35:58,165 INFO input.FileInputFormat: Total input files to process : 1
2022-12-17 10:35:58,274 INFO mapreduce.JobSubmitter: number of splits:4
2022-12-17 10:35:58,561 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1990169669_0001
2022-12-17 10:35:58,561 INFO mapreduce.JobSubmitter: Executing with tokens: []
2022-12-17 10:35:58,952 INFO mapred.LocalDistributedCacheManager: Creating symlink: /usr/local/hadoop/tmp/mapred/local/job_local1990169669_0001_adf29d2b-fc90-4352-81d4-05bd048183d7/libjars <- /usr/local/sqoop/libjars/*
2022-12-17 10:35:58,961 WARN fs.FileUtil: Command 'ln -s /usr/local/hadoop/tmp/mapred/local/job_local1990169669_0001_adf29d2b-fc90-4352-81d4-05bd048183d7/libjars /usr/local/sqoop/libjars/*' failed 1 with: ln: 无法创建符号链接'/usr/local/sqoop/libjars/*': 没有那个文件或目录

2022-12-17 10:35:58,961 WARN mapred.LocalDistributedCacheManager: Failed to create symlink: /usr/local/hadoop/tmp/mapred/local/job_local1990169669_0001_adf29d2b-fc90-4352-81d4-05bd048183d7/libjars <- /usr/local/sqoop/libjars/*
2022-12-17 10:35:58,961 INFO mapred.LocalDistributedCacheManager: Localized file:/tmp/hadoop/mapred/staging/hadoop1990169669/.staging/job_local1990169669_0001/libjars as file:/usr/local/hadoop/tmp/mapred/local/job_local1990169669_0001_adf29d2b-fc90-4352-81d4-05bd048183d7/libjars
2022-12-17 10:35:59,234 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
2022-12-17 10:35:59,235 INFO mapreduce.Job: Running job: job_local1990169669_0001
2022-12-17 10:35:59,237 INFO mapred.LocalJobRunner: OutputCommitter set in config null
2022-12-17 10:35:59,241 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.sqoop.mapreduce.NullOutputCommitter
2022-12-17 10:35:59,321 INFO mapred.LocalJobRunner: Waiting for map tasks
2022-12-17 10:35:59,323 INFO mapred.LocalJobRunner: Starting task: attempt_local1990169669_0001_m_000000_0
2022-12-17 10:35:59,410 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2022-12-17 10:35:59,435 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dblab.db/user_action/000000_0:0+3897554
2022-12-17 10:35:59,460 INFO Configuration.deprecation: map.input.file is deprecated. Instead, use mapreduce.map.input.file
2022-12-17 10:35:59,464 INFO Configuration.deprecation: map.input.start is deprecated. Instead, use mapreduce.map.input.start
2022-12-17 10:35:59,464 INFO Configuration.deprecation: map.input.length is deprecated. Instead, use mapreduce.map.input.length
Sat Dec 17 10:35:59 ULAT 2022 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2022-12-17 10:35:59,632 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
2022-12-17 10:35:59,635 INFO mapred.LocalJobRunner: Starting task: attempt_local1990169669_0001_m_000001_0
2022-12-17 10:35:59,635 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2022-12-17 10:35:59,637 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dblab.db/user_action/000000_0:3897554+3897554
Sat Dec 17 10:35:59 ULAT 2022 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2022-12-17 10:35:59,800 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
2022-12-17 10:35:59,805 INFO mapred.LocalJobRunner: Starting task: attempt_local1990169669_0001_m_000002_0
2022-12-17 10:35:59,807 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2022-12-17 10:35:59,808 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dblab.db/user_action/000000_0:7795108+3897554
Sat Dec 17 10:35:59 ULAT 2022 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2022-12-17 10:35:59,975 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
2022-12-17 10:35:59,979 INFO mapred.LocalJobRunner: Starting task: attempt_local1990169669_0001_m_000003_0
2022-12-17 10:35:59,980 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
2022-12-17 10:35:59,982 INFO mapred.MapTask: Processing split: Paths:/user/hive/warehouse/dblab.db/user_action/000000_0:11692662+3897554
Sat Dec 17 10:36:00 ULAT 2022 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
2022-12-17 10:36:00,148 INFO mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
2022-12-17 10:36:00,160 INFO mapred.LocalJobRunner: map task executor complete.
2022-12-17 10:36:00,164 WARN mapred.LocalJobRunner: job_local1990169669_0001
java.lang.Exception: java.io.IOException: java.lang.ClassNotFoundException: user_action
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:552)
Caused by: java.io.IOException: java.lang.ClassNotFoundException: user_action
    at org.apache.sqoop.mapreduce.TextExportMapper.setup(TextExportMapper.java:70)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
    at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:800)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:348)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:271)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: user_action
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.sqoop.mapreduce.TextExportMapper.setup(TextExportMapper.java:66)
    ... 10 more
2022-12-17 10:36:00,236 INFO mapreduce.Job: Job job_local1990169669_0001 running in uber mode : false
2022-12-17 10:36:00,237 INFO mapreduce.Job:  map 0% reduce 0%
2022-12-17 10:36:00,239 INFO mapreduce.Job: Job job_local1990169669_0001 failed with state FAILED due to: NA
2022-12-17 10:36:00,245 INFO mapreduce.Job: Counters: 0
2022-12-17 10:36:00,292 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
2022-12-17 10:36:00,302 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 2.7183 seconds (0 bytes/sec)
2022-12-17 10:36:00,313 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2022-12-17 10:36:00,322 INFO mapreduce.ExportJobBase: Exported 0 records.
2022-12-17 10:36:00,322 ERROR tool.ExportTool: Error during export: Export job failed!

img

img

img



  • 写回答

3条回答 默认 最新

  • 玥司 2022-12-17 20:23
    关注

    我做平常作业时候也会有奇奇怪怪报错,建议你重启试试看,开启的窗口有的打开了最好别关(除非你确定用不着了),或者多运行几遍(整个部分除了开启Hadoop上面的),有时候做之前启动Hadoop时候开启历史服务器也会减少报错,反正有的时候报错莫名其妙。

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
查看更多回答(2条)

报告相同问题?

问题事件

  • 系统已结题 4月27日
  • 已采纳回答 4月19日
  • 创建了问题 12月17日

悬赏问题

  • ¥15 求指导ADS低噪放设计
  • ¥15 CARSIM前车变道设置
  • ¥50 三种调度算法报错 有实例
  • ¥15 关于#python#的问题,请各位专家解答!
  • ¥200 询问:python实现大地主题正反算的程序设计,有偿
  • ¥15 smptlib使用465端口发送邮件失败
  • ¥200 总是报错,能帮助用python实现程序实现高斯正反算吗?有偿
  • ¥15 对于squad数据集的基于bert模型的微调
  • ¥15 为什么我运行这个网络会出现以下报错?CRNN神经网络
  • ¥20 steam下载游戏占用内存