abigalexy 2021-12-28 11:49 采纳率: 0%
浏览 310
已结题

头疼问题:sqoop import 设置了分隔符,但是导出查看文件分隔符并不生效

目标使用 sqoop 导入mysql 表到 hive,当前这个脚本有些表是没问题的,这个表就有问题,分隔符失效了
mysql 表结构如下:

img

img


但是导出文件格式如下:

img

添加 --hive-drop-import-delims 后:
导出文件没有任何变化:

img

恳请大家指点一下

  • 写回答

11条回答 默认 最新

  • 大数据技术派 2022-01-04 16:27
    关注
    获得3.50元问题酬金

    我试了一下,没有问题,导出有分隔符。

    生成数据

    DROP TABLE IF EXISTS ct_industry;
    CREATE TABLE ct_industry(
    ID bigint(20) NOT NULL COMMENT 'ID',
    IndustryNum int(11) NOT NULL COMMENT '行业编码',
    IndustryName varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT '行业名称',
    IndustryCode varchar(20) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL COMMENT '所属行业代码',
    IndustryPlate varchar(20) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '行业板块',
    XGRQ datetime(0) NOT NULL COMMENT '修改日期',
    JSID bigint(20) NOT NULL COMMENT 'JSID',
    UNIQUE INDEXIX_CT_Industry(IndustryNum) USING BTREE,
    UNIQUE INDEXIX_CT_Industry_JSID(JSID) USING BTREE,
    UNIQUE INDEXPK_CT_Industry(ID) USING BTREE
    ) ENGINE = InnoDB CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;
    
    INSERT INTO ct_industry VALUES (4747621348152, 1, '农、林、牧、渔业', 'A', 'A', '2006-06-14 11:43:44', 203600622544);
    INSERT INTO ct_industry VALUES (4747623710988, 2, '农业', 'A01', 'A01', '2006-06-14 11:43:44', 203600622545);
    INSERT INTO ct_industry VALUES (4747625764139, 3, '种植业', 'A0101', 'A01', '2006-06-14 11:43:44', 203600622546);
    INSERT INTO ct_industry VALUES (4747631517547, 4, '其他农业', 'A0199', 'A01', '2006-06-14 11:43:44', 203600622547);
    INSERT INTO ct_industry VALUES (4747636155935, 5, '林业', 'A03', 'A03', '2006-06-14 11:43:44', 203600622548);
    INSERT INTO ct_industry VALUES (4747638884790, 6, '畜牧业', 'A05', 'A05', '2006-06-14 11:43:44', 203600622549);
    INSERT INTO ct_industry VALUES (4747641912105, 7, '牲畜饲养放牧业', 'A0501', 'A05', '2006-06-14 11:43:44', 203600622550);
    INSERT INTO ct_industry VALUES (4747644131080, 8, '家禽饲养业', 'A0505', 'A05', '2006-06-14 11:43:44', 203600622551);
    INSERT INTO ct_industry VALUES (4747646482277, 9, '其他畜牧业', 'A0599', 'A05', '2006-06-14 11:43:44', 203600622552);
    INSERT INTO ct_industry VALUES (4747653097931, 10, '渔业', 'A07', 'A07', '2006-06-14 11:43:44', 203600622553);
    INSERT INTO ct_industry VALUES (4747654632813, 11, '海洋渔业', 'A0701', 'A07', '2006-06-14 11:43:44', 203600622554);
    INSERT INTO ct_industry VALUES (4747656185728, 12, '淡水渔业', 'A0705', 'A07', '2006-06-14 11:43:44', 203600622555);
    

    导出命令

    sqoop import \
    --connect jdbc:mysql://{mysql}:3306/test \
    --username root \
    --password root \
    --table ct_industry \
    -m 1 \
    --hive-import \
    --create-hive-table \
    --fields-terminated-by ',' \
    --hive-table test.ct_industry
    

    导出结果

    4747621348152,1,农、林、牧、渔业,A,A,2006-06-14 11:43:44.0,203600622544
    4747623710988,2,农业,A01,A01,2006-06-14 11:43:44.0,203600622545
    4747625764139,3,种植业,A0101,A01,2006-06-14 11:43:44.0,203600622546
    4747631517547,4,其他农业,A0199,A01,2006-06-14 11:43:44.0,203600622547
    4747636155935,5,林业,A03,A03,2006-06-14 11:43:44.0,203600622548
    4747638884790,6,畜牧业,A05,A05,2006-06-14 11:43:44.0,203600622549
    4747641912105,7,牲畜饲养放牧业,A0501,A05,2006-06-14 11:43:44.0,203600622550
    4747644131080,8,家禽饲养业,A0505,A05,2006-06-14 11:43:44.0,203600622551
    4747646482277,9,其他畜牧业,A0599,A05,2006-06-14 11:43:44.0,203600622552
    4747653097931,10,渔业,A07,A07,2006-06-14 11:43:44.0,203600622553
    4747654632813,11,海洋渔业,A0701,A07,2006-06-14 11:43:44.0,203600622554
    4747656185728,12,淡水渔业,A0705,A07,2006-06-14 11:43:44.0,203600622555
    
    评论

报告相同问题?

问题事件

  • 系统已结题 1月5日
  • 修改了问题 12月28日
  • 创建了问题 12月28日