如何对一个txt文档就行庖丁分词

Question

就是实现对一个txt文档就行分词，用庖丁中文分词器。给出代码。最好给出解释

Answer 1

知道paoding就应该知道Lucene,可以用Lucene的分词器Analyzer,代码如下，测试加入lucene包和paoding配置即可
Analyzer analyzer = new PaodingAnalyzer();
FileInputStream in = null;
in = new FileInputStream(new File("d://1.txt"));
InputStreamReader inReader = new InputStreamReader(in);
BufferedReader br = new BufferedReader(inReader);
TokenStream ts = analyzer.tokenStream(content, br);
Token t = ts.next();
while (t != null) {
contentAnalyzer += t.termText() + " ";
System.out.println(t.termText());
t = ts.next();
}