小鱼-简简单单 2016-05-05 03:49
浏览 4084

strom+kafka,strom会一直重复解析同条数据,有时解析不到数据

protected StormTopology buildTopology() {

// ConsumerConfig.

    ZkHosts zkHosts = new ZkHosts(brokerZkStr, brokerZkPath);

// storm.kafka.KafkaSpout

    TridentTopology tridentTopology = new TridentTopology();
    TridentKafkaConfig spoutConf = new TridentKafkaConfig(zkHosts, topic);
    spoutConf.forceFromStart = true;// 和startOffsetTime,一起用,默认情况下,为false,一旦startOffsetTime被设置,就要置为true
    spoutConf.startOffsetTime = -1L;// -2 从kafka头开始 -1 是从最新的开始 0 =无 从ZK开始
    spoutConf.scheme = new SchemeAsMultiScheme(new StringScheme());

// OpaqueTridentKafkaSpout opaqueTridentKafkaSpout = new OpaqueTridentKafkaSpout(spoutConf);

    TransactionalTridentKafkaSpout opaqueTridentKafkaSpout = new TransactionalTridentKafkaSpout(spoutConf);

    Stream opTrdtSptStream = tridentTopology.newStream("opTrdtSptStream", opaqueTridentKafkaSpout);
    opTrdtSptStream.shuffle().each(new Fields("str"), new Utils.FilterLogInfo()).each(new Fields("str"), new
            LogFilterUtil(), new Fields("analyzeResult")).each(new Fields("analyzeResult"),new PropertyAnalyzeUtils
            (),new Fields("autoItem")).each(new Fields("autoItem"), new AnsjIllegalWordsUtil
                    (), new Fields("ansjIllegalWordsResult")).each(new Fields("ansjIllegalWordsResult"), new
            AnsjKeyWordsUtil(), new Fields("ansjKeyWordsResult")).each(new Fields("ansjKeyWordsResult"), new
            AnsjPicturesUtil(), new Fields("ansjPicturesResult"));
    return tridentTopology.build();
}
  • 写回答

0条回答

    报告相同问题?

    悬赏问题

    • ¥15 #MATLAB仿真#车辆换道路径规划
    • ¥15 java 操作 elasticsearch 8.1 实现 索引的重建
    • ¥15 数据可视化Python
    • ¥15 要给毕业设计添加扫码登录的功能!!有偿
    • ¥15 kafka 分区副本增加会导致消息丢失或者不可用吗?
    • ¥15 微信公众号自制会员卡没有收款渠道啊
    • ¥100 Jenkins自动化部署—悬赏100元
    • ¥15 关于#python#的问题:求帮写python代码
    • ¥20 MATLAB画图图形出现上下震荡的线条
    • ¥15 关于#windows#的问题:怎么用WIN 11系统的电脑 克隆WIN NT3.51-4.0系统的硬盘