hadoop - hadoop2.6 分布式 - 简单实例学习 - 统计某年的最高温度和按年份将温度从高到底排序 - 原明卓 - 博客频道 - CSDN.NET http://blog.csdn.net/lablenet/article/details/50608197#java
我按照这篇博客做的,运行结果见图。
16/10/19 05:27:51 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
16/10/19 05:27:52 INFO input.FileInputFormat: Total input paths to process : 1
16/10/19 05:27:52 INFO util.NativeCodeLoader: Loaded the native-hadoop library
16/10/19 05:27:52 WARN snappy.LoadSnappy: Snappy native library not loaded
16/10/19 05:27:54 INFO mapred.JobClient: Running job: job_201610190234_0013
16/10/19 05:27:55 INFO mapred.JobClient: map 0% reduce 0%
16/10/19 05:28:24 INFO mapred.JobClient: map 100% reduce 0%
16/10/19 05:28:41 INFO mapred.JobClient: map 100% reduce 20%
16/10/19 05:28:42 INFO mapred.JobClient: map 100% reduce 40%
16/10/19 05:28:50 INFO mapred.JobClient: map 100% reduce 46%
16/10/19 05:28:51 INFO mapred.JobClient: map 100% reduce 60%
16/10/19 05:29:01 INFO mapred.JobClient: map 100% reduce 100%
16/10/19 05:29:01 INFO mapred.JobClient: Job complete: job_201610190234_0013
16/10/19 05:29:01 INFO mapred.JobClient: Counters: 28
16/10/19 05:29:01 INFO mapred.JobClient: Job Counters
16/10/19 05:29:01 INFO mapred.JobClient: Launched reduce tasks=6
16/10/19 05:29:01 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=26528
16/10/19 05:29:01 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
16/10/19 05:29:01 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
16/10/19 05:29:01 INFO mapred.JobClient: Launched map tasks=1
16/10/19 05:29:01 INFO mapred.JobClient: Data-local map tasks=1
16/10/19 05:29:01 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=107381
16/10/19 05:29:01 INFO mapred.JobClient: File Output Format Counters
16/10/19 05:29:01 INFO mapred.JobClient: Bytes Written=0
16/10/19 05:29:01 INFO mapred.JobClient: FileSystemCounters
16/10/19 05:29:01 INFO mapred.JobClient: FILE_BYTES_READ=30
16/10/19 05:29:01 INFO mapred.JobClient: HDFS_BYTES_READ=1393
16/10/19 05:29:01 INFO mapred.JobClient: FILE_BYTES_WRITTEN=354256
16/10/19 05:29:01 INFO mapred.JobClient: File Input Format Counters
16/10/19 05:29:01 INFO mapred.JobClient: Bytes Read=1283
16/10/19 05:29:01 INFO mapred.JobClient: Map-Reduce Framework
16/10/19 05:29:01 INFO mapred.JobClient: Map output materialized bytes=30
16/10/19 05:29:01 INFO mapred.JobClient: Map input records=46
16/10/19 05:29:01 INFO mapred.JobClient: Reduce shuffle bytes=30
16/10/19 05:29:01 INFO mapred.JobClient: Spilled Records=0
16/10/19 05:29:01 INFO mapred.JobClient: Map output bytes=0
16/10/19 05:29:01 INFO mapred.JobClient: CPU time spent (ms)=16910
16/10/19 05:29:01 INFO mapred.JobClient: Total committed heap usage (bytes)=195301376
16/10/19 05:29:01 INFO mapred.JobClient: Combine input records=0
16/10/19 05:29:01 INFO mapred.JobClient: SPLIT_RAW_BYTES=110
16/10/19 05:29:01 INFO mapred.JobClient: Reduce input records=0
16/10/19 05:29:01 INFO mapred.JobClient: Reduce input groups=0
16/10/19 05:29:01 INFO mapred.JobClient: Combine output records=0
16/10/19 05:29:01 INFO mapred.JobClient: Physical memory (bytes) snapshot=331567104
16/10/19 05:29:01 INFO mapred.JobClient: Reduce output records=0
16/10/19 05:29:01 INFO mapred.JobClient: Virtual memory (bytes) snapshot=2264113152
16/10/19 05:29:01 INFO mapred.JobClient: Map output records=0
yyyy-MM-dd HH:mm:ss\t温度
example:1995-10-10 10:10:10 6.54
这是数据源格式,我把
RunJob中的
int year=c.get(1);
String hot=ss[1].substring(0,ss[1].lastIndexOf("°C"));
KeyPari keyPari=new KeyPari();
keyPari.setYear(year);
中的°C改成了\n。
代码和博文的一样,只删掉了MAP里面的IF判断和修改了输入输出路径。求前辈们指教一下为什么会这样,深表感激。