java解析orc格式文件谓词下推
使用Java解析orc文件,设置谓词下推,并没有生效
SearchArgument sarg = SearchArgumentFactory
.newBuilder(conf)
.startAnd()
.startNot()
.lessThan("id", PredicateLeaf.Type.LONG, 100L)
.end()
.lessThan("id", PredicateLeaf.Type.LONG, 400L)
.end()
.build();
RecordReader rowIterator = reader
.rows(reader.options()
.range(0L, Long.MAX_VALUE)
.include(new boolean[]{true, true, true, true, true})
.searchArgument(sarg, new String[]{null, "id", "name", "age", "sex"}));
VectorizedRowBatch batch = reader.getSchema().createRowBatch();
while (rowIterator.nextBatch(batch)) {
System.out.println(batch.toString())
}
但是从打印结果来看,不符合条件的数据也打印出来了,为什么?
谓词下推如何生效,百度google都试了,没有解决。求解谢谢!
0 stone_0 53 Y
.....
19 stone_19 50 X
20 stone_20 58 Y
21 stone_21 50 X
22 stone_22 55 Y
23 stone_23 58 X
24 stone_24 53 Y
25 stone_25 52 X
26 stone_26 57 Y
27 stone_27 58 X
28 stone_28 57 Y
29 stone_29 56 X
30 stone_30 50 Y
31 stone_31 55 X
32 stone_32 55 Y
33 stone_33 50 X
34 stone_34 57 Y
35 stone_35 52 X
36 stone_36 55 Y
....
999 stone_999 50 X