Hadoop序列化问题,实现WritableComparable,readFields报错EOFException
 public class MyKey implements WritableComparable<MyKey> {

    //flag == 1 : user
    //flag == 0 : shopping
    private Integer flag;
    private Integer u_id;
    private Integer s_id;
    private Integer s_u_id;
    private String u_info;
    private String s_info;

    @Override
    public int compareTo(MyKey o) {
        if (flag.equals(1)){
            //user
            return u_id - o.u_id;
        }else {
            //shopping
            return s_id - o.s_id;
        }
    }

    @Override
    public void write(DataOutput out) throws IOException {
        out.writeInt(flag);
        out.writeInt(u_id);
        out.writeInt(s_id);
        out.writeInt(s_u_id);
        out.writeUTF(u_info);
        out.writeUTF(s_info);
    }

    @Override
    public void readFields(DataInput in) throws IOException {
        flag = in.readInt();
        u_id = in.readInt();
        s_id = in.readInt();
        s_u_id = in.readInt();
        u_info = in.readUTF();
        s_info = in.readUTF();
    }
}

报错异常

2018-10-08 19:55:15,246 INFO Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2018-10-08 19:55:15,250 INFO mapred.LocalJobRunner: reduce task executor complete.
2018-10-08 19:55:15,253 WARN mapred.LocalJobRunner: job_local85671337_0001
java.lang.Exception: java.lang.RuntimeException: java.io.EOFException
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:559)
Caused by: java.lang.RuntimeException: java.io.EOFException
at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:165)
at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKeyValue(ReduceContextImpl.java:158)
at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKey(ReduceContextImpl.java:121)
at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.nextKey(WrappedReducer.java:302)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:170)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:628)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:347)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:392)
at sortjoin.MyKey.readFields(MyKey.java:43)
at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:158)
... 12 more
2018-10-08 19:55:15,962 INFO mapreduce.Job: Job job_local85671337_0001 running in uber mode : false
2018-10-08 19:55:15,964 INFO mapreduce.Job: map 100% reduce 0%

1个回答

这个异常是自带的异常,可以忽略的;该异常是输入流已经到结尾了的标志,我们可以将其捕获,然后不做任何操作,即结束了该次反序列化操作。

Csdn user default icon
上传中...
上传图片
插入图片
抄袭、复制答案,以达到刷声望分或其他目的的行为,在CSDN问答是严格禁止的,一经发现立刻封号。是时候展现真正的技术了!
立即提问