lihe55966 2015-10-21 12:14 采纳率: 0%
浏览 1924

java 遍历hdfs文件夹报错!!急!!急

FileStatus[] srcFileStatus = hdfs.listStatus(new Path(srcPath));执行到这句话的时候会报如下错:

java.lang.NullPointerException
at org.apache.hadoop.fs.Path.(Path.java:104)
at org.apache.hadoop.fs.Path.makeQualified(Path.java:440)
at org.apache.hadoop.hdfs.protocol.HdfsFileStatus.makeQualified(HdfsFileStatus.java:263)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:802)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:849)
at test.HdfsOperate.downloadFile(HdfsOperate.java:142)
at test.HdfsTest.main(HdfsTest.java:45)

代码:

public void downloadFile(String dstPath, String srcPath) throws Exception
{
Path path = new Path(srcPath);

    Configuration conf = new Configuration();   
    FileSystem hdfs = path.getFileSystem(conf);
    //hdfs = FileSystem.get(URI.create(srcPath), conf);
    File rootfile = new File(dstPath);
    if(!rootfile.exists()) rootfile.mkdirs();
    hdfs.getFileStatus(path);
    if (hdfs.isFile(path))
    {
        FSDataInputStream in = null;
        FileOutputStream out = null;
        try
        {   
            in = hdfs.open(path);

            File srcfile = new File(rootfile ,path.getName());
            if(!srcfile.exists())srcfile.createNewFile();
            out = new FileOutputStream(srcfile);
            IOUtils.copyBytes(in, out, 4096, false);
        }
        finally
        {
            IOUtils.closeStream(in);
            IOUtils.closeStream(out);
        }
    }
    else
    {
        File dstDir = new File(dstPath);
        if (!dstDir.exists())
        {
            dstDir.mkdirs();
        }


        FileStatus[] srcFileStatus = hdfs.listStatus(path);
        Path[] srcFilePath = FileUtil.stat2Paths(srcFileStatus);
        for (int i = 0; i < srcFilePath.length; i++)
        {
            String srcFile = srcFilePath[i].toString();
            int fileNamePosi = srcFile.lastIndexOf("/");
            String fileName = srcFile.substring(fileNamePosi + 1);
            //String fileName = srcFilePath[i].getName();
            downloadFile(dstPath  + fileName, srcPath  + fileName +"/");
        }
    }
}
  • 写回答

3条回答 默认 最新

  • 运维如诗 2015-10-21 13:14
    关注

    就给这行代码,也是6。。。。

    评论

报告相同问题?

悬赏问题

  • ¥15 DIFY API Endpoint 问题。
  • ¥20 sub地址DHCP问题
  • ¥15 delta降尺度计算的一些细节,有偿
  • ¥15 Arduino红外遥控代码有问题
  • ¥15 数值计算离散正交多项式
  • ¥30 数值计算均差系数编程
  • ¥15 redis-full-check比较 两个集群的数据出错
  • ¥15 Matlab编程问题
  • ¥15 训练的多模态特征融合模型准确度很低怎么办
  • ¥15 kylin启动报错log4j类冲突