WilliamKZhy 2016-10-19 13:26 采纳率: 0%
浏览 4221

【新手】Hadoop MapReduce 执行中Map没有输出

hadoop - hadoop2.6 分布式 - 简单实例学习 - 统计某年的最高温度和按年份将温度从高到底排序 - 原明卓 - 博客频道 - CSDN.NET http://blog.csdn.net/lablenet/article/details/50608197#java

我按照这篇博客做的,运行结果见图。

16/10/19 05:27:51 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
16/10/19 05:27:52 INFO input.FileInputFormat: Total input paths to process : 1
16/10/19 05:27:52 INFO util.NativeCodeLoader: Loaded the native-hadoop library
16/10/19 05:27:52 WARN snappy.LoadSnappy: Snappy native library not loaded
16/10/19 05:27:54 INFO mapred.JobClient: Running job: job_201610190234_0013
16/10/19 05:27:55 INFO mapred.JobClient: map 0% reduce 0%
16/10/19 05:28:24 INFO mapred.JobClient: map 100% reduce 0%
16/10/19 05:28:41 INFO mapred.JobClient: map 100% reduce 20%
16/10/19 05:28:42 INFO mapred.JobClient: map 100% reduce 40%
16/10/19 05:28:50 INFO mapred.JobClient: map 100% reduce 46%
16/10/19 05:28:51 INFO mapred.JobClient: map 100% reduce 60%
16/10/19 05:29:01 INFO mapred.JobClient: map 100% reduce 100%
16/10/19 05:29:01 INFO mapred.JobClient: Job complete: job_201610190234_0013
16/10/19 05:29:01 INFO mapred.JobClient: Counters: 28
16/10/19 05:29:01 INFO mapred.JobClient: Job Counters
16/10/19 05:29:01 INFO mapred.JobClient: Launched reduce tasks=6
16/10/19 05:29:01 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=26528
16/10/19 05:29:01 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
16/10/19 05:29:01 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
16/10/19 05:29:01 INFO mapred.JobClient: Launched map tasks=1
16/10/19 05:29:01 INFO mapred.JobClient: Data-local map tasks=1
16/10/19 05:29:01 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=107381
16/10/19 05:29:01 INFO mapred.JobClient: File Output Format Counters
16/10/19 05:29:01 INFO mapred.JobClient: Bytes Written=0
16/10/19 05:29:01 INFO mapred.JobClient: FileSystemCounters
16/10/19 05:29:01 INFO mapred.JobClient: FILE_BYTES_READ=30
16/10/19 05:29:01 INFO mapred.JobClient: HDFS_BYTES_READ=1393
16/10/19 05:29:01 INFO mapred.JobClient: FILE_BYTES_WRITTEN=354256
16/10/19 05:29:01 INFO mapred.JobClient: File Input Format Counters
16/10/19 05:29:01 INFO mapred.JobClient: Bytes Read=1283
16/10/19 05:29:01 INFO mapred.JobClient: Map-Reduce Framework
16/10/19 05:29:01 INFO mapred.JobClient: Map output materialized bytes=30
16/10/19 05:29:01 INFO mapred.JobClient: Map input records=46
16/10/19 05:29:01 INFO mapred.JobClient: Reduce shuffle bytes=30
16/10/19 05:29:01 INFO mapred.JobClient: Spilled Records=0
16/10/19 05:29:01 INFO mapred.JobClient: Map output bytes=0
16/10/19 05:29:01 INFO mapred.JobClient: CPU time spent (ms)=16910
16/10/19 05:29:01 INFO mapred.JobClient: Total committed heap usage (bytes)=195301376
16/10/19 05:29:01 INFO mapred.JobClient: Combine input records=0
16/10/19 05:29:01 INFO mapred.JobClient: SPLIT_RAW_BYTES=110
16/10/19 05:29:01 INFO mapred.JobClient: Reduce input records=0
16/10/19 05:29:01 INFO mapred.JobClient: Reduce input groups=0
16/10/19 05:29:01 INFO mapred.JobClient: Combine output records=0
16/10/19 05:29:01 INFO mapred.JobClient: Physical memory (bytes) snapshot=331567104
16/10/19 05:29:01 INFO mapred.JobClient: Reduce output records=0
16/10/19 05:29:01 INFO mapred.JobClient: Virtual memory (bytes) snapshot=2264113152

16/10/19 05:29:01 INFO mapred.JobClient: Map output records=0

yyyy-MM-dd HH:mm:ss\t温度
example:1995-10-10 10:10:10 6.54
这是数据源格式,我把
RunJob中的
int year=c.get(1);

String hot=ss[1].substring(0,ss[1].lastIndexOf("°C"));

KeyPari keyPari=new KeyPari();

keyPari.setYear(year);

中的°C改成了\n。


代码和博文的一样,只删掉了MAP里面的IF判断和修改了输入输出路径。求前辈们指教一下为什么会这样,深表感激。

  • 写回答

2条回答 默认 最新

  • WilliamKZhy 2016-10-19 13:33
    关注

    yyyy-MM-dd HH:mm:ss\t温度
    example:1995-10-10 10:10:10 6.54
    这是数据源格式,我把
    RunJob中的
    int year=c.get(1);

    String hot=ss[1].substring(0,ss[1].lastIndexOf("°C"));

    KeyPari keyPari=new KeyPari();

    keyPari.setYear(year);

    中的°C改成了\n。

    评论

报告相同问题?

悬赏问题

  • ¥15 #MATLAB仿真#车辆换道路径规划
  • ¥15 java 操作 elasticsearch 8.1 实现 索引的重建
  • ¥15 数据可视化Python
  • ¥15 要给毕业设计添加扫码登录的功能!!有偿
  • ¥15 kafka 分区副本增加会导致消息丢失或者不可用吗?
  • ¥15 微信公众号自制会员卡没有收款渠道啊
  • ¥100 Jenkins自动化部署—悬赏100元
  • ¥15 关于#python#的问题:求帮写python代码
  • ¥20 MATLAB画图图形出现上下震荡的线条
  • ¥15 关于#windows#的问题:怎么用WIN 11系统的电脑 克隆WIN NT3.51-4.0系统的硬盘