期待97 2018-10-08 04:14 采纳率: 0%
浏览 1008

Hadoop序列化问题,实现WritableComparable,readFields报错EOFException

 public class MyKey implements WritableComparable<MyKey> {

    //flag == 1 : user
    //flag == 0 : shopping
    private Integer flag;
    private Integer u_id;
    private Integer s_id;
    private Integer s_u_id;
    private String u_info;
    private String s_info;

    @Override
    public int compareTo(MyKey o) {
        if (flag.equals(1)){
            //user
            return u_id - o.u_id;
        }else {
            //shopping
            return s_id - o.s_id;
        }
    }

    @Override
    public void write(DataOutput out) throws IOException {
        out.writeInt(flag);
        out.writeInt(u_id);
        out.writeInt(s_id);
        out.writeInt(s_u_id);
        out.writeUTF(u_info);
        out.writeUTF(s_info);
    }

    @Override
    public void readFields(DataInput in) throws IOException {
        flag = in.readInt();
        u_id = in.readInt();
        s_id = in.readInt();
        s_u_id = in.readInt();
        u_info = in.readUTF();
        s_info = in.readUTF();
    }
}

报错异常

2018-10-08 19:55:15,246 INFO Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords
2018-10-08 19:55:15,250 INFO mapred.LocalJobRunner: reduce task executor complete.
2018-10-08 19:55:15,253 WARN mapred.LocalJobRunner: job_local85671337_0001
java.lang.Exception: java.lang.RuntimeException: java.io.EOFException
at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:559)
Caused by: java.lang.RuntimeException: java.io.EOFException
at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:165)
at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKeyValue(ReduceContextImpl.java:158)
at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKey(ReduceContextImpl.java:121)
at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.nextKey(WrappedReducer.java:302)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:170)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:628)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:390)
at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:347)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.EOFException
at java.io.DataInputStream.readInt(DataInputStream.java:392)
at sortjoin.MyKey.readFields(MyKey.java:43)
at org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:158)
... 12 more
2018-10-08 19:55:15,962 INFO mapreduce.Job: Job job_local85671337_0001 running in uber mode : false
2018-10-08 19:55:15,964 INFO mapreduce.Job: map 100% reduce 0%

展开全部

  • 写回答

1条回答 默认 最新

  • huwao 2018-10-08 16:53
    关注

    这个异常是自带的异常,可以忽略的;该异常是输入流已经到结尾了的标志,我们可以将其捕获,然后不做任何操作,即结束了该次反序列化操作。

    评论
    编辑
    预览

    报告相同问题?

    悬赏问题

    • ¥15 宝塔面板一键迁移使用不了
    • ¥15 求一个按键录像存储到内存卡的ESP32CAM代码
    • ¥15 如何单独修改下列canvas推箱子代码target参数?,插入图片代替其形状,就是哪个绿色的圆圈每关用插入的图片替代
    • ¥20 四叉树的创建和输出问题
    • ¥15 javaweb连接数据库,jsp文件加载不出来
    • ¥15 matlab关于高斯赛德尔迭代的应用编撰。(相关搜索:matlab代码|迭代法)
    • ¥15 损失匹配问题,求解答
    • ¥15 3500常用汉字书法体检测数据集下载
    • ¥15 odoo17在制造模块或采购模块良品与次品如何分流和在质检模块下如何开发
    • ¥15 Qt音乐播放器的音乐文件相对路径怎么写
    手机看
    程序员都在用的中文IT技术交流社区

    程序员都在用的中文IT技术交流社区

    专业的中文 IT 技术社区,与千万技术人共成长

    专业的中文 IT 技术社区,与千万技术人共成长

    关注【CSDN】视频号,行业资讯、技术分享精彩不断,直播好礼送不停!

    关注【CSDN】视频号,行业资讯、技术分享精彩不断,直播好礼送不停!

    客服 返回
    顶部