Prayer X 2019-09-09 21:25 采纳率: 0%
浏览 646

MapReducer中不同key输入一个Reducer进行计算时reducer方法运行了几次?

写了一个普通的WordCount在运行时发现有多个不同key输入后,在reducer中加入一个控制台显示语句,该语句只会输出一次,但是如果只执行了一次reduce方法key是如何实现遍历的?

map和reduce方法


     private static class MyMapper extends Mapper<LongWritable, Text, MedcineWritable, NullWritable>{
         static MedcineWritable outkey = new MedcineWritable();;
         static NullWritable outval = NullWritable.get();
         static String[] tmp ;
         @Override
         protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException {
             tmp = value.toString().split(" ");
            context.getCounter("Mapper_line","Total_line").increment(1);
            try {
                outkey.setDate(tmp[0]);
                outkey.setCid((tmp[1]));
                outkey.setSid(Integer.parseInt(tmp[2]));
                outkey.setName(tmp[3]);
                outkey.setNumber(Double.parseDouble(tmp[4]));
                outkey.setExceptprice(Double.parseDouble(tmp[5]));
                outkey.setRealprice(Double.parseDouble(tmp[6]));
                context.getCounter("Mapper_line","good_line").increment(1);
                System.out.println(outkey);
                context.write(outkey, outval);
            }
            catch (ArrayIndexOutOfBoundsException e){
                context.getCounter("Mapper_line","bad_line").increment(1);
            }catch (ClassFormatError e){
                context.getCounter("Mapper_line","bad_line").increment(1);
            }
         }
     }
     private static class MyReducer extends Reducer<MedcineWritable,NullWritable,MedcineWritable,NullWritable>{
         @Override
         protected void reduce(MedcineWritable key, Iterable<NullWritable> values, Context context) throws IOException, InterruptedException {
             System.out.println("上面是遍历map的输出,下面是reduce的输出");
             System.out.println(key);
         }
     }

图片说明

  • 写回答

1条回答 默认 最新

  • 青衣沽酒醉风尘 2019-09-11 10:34
    关注

    如果是在reduce方法里面加的输出,理论上应该是有几个key就输出几次的。如果是加在reduce的类的初始化方法里面,应该是有几个reduce就输出几次

    评论

报告相同问题?

悬赏问题

  • ¥15 jupyterthemes 设置完毕后没有效果
  • ¥15 matlab图像高斯低通滤波
  • ¥15 针对曲面部件的制孔路径规划,大家有什么思路吗
  • ¥15 钢筋实图交点识别,机器视觉代码
  • ¥15 如何在Linux系统中,但是在window系统上idea里面可以正常运行?(相关搜索:jar包)
  • ¥50 400g qsfp 光模块iphy方案
  • ¥15 两块ADC0804用proteus仿真时,出现异常
  • ¥15 关于风控系统,如何去选择
  • ¥15 这款软件是什么?需要能满足我的需求
  • ¥15 SpringSecurityOauth2登陆前后request不一致