douyanpeng0748 2016-08-20 04:44
浏览 899
已采纳

从Golang中的io.Reader到io.Writer读取/复制一定数量的字节,或者如果超过一定字节数限制,则返回错误?

I have an io.Reader in Golang and I want to double-check that the size of its data is below a predetermined maximum before or while running io.Copy() to save it to disk using io.Writer.

Since the file data in io.Reader could theoretically be quite large, I want to minimize memory usage and processing here if avoidable.
I don't think there's a function that's like io.CopyLessThanOrEqualToThisManyBytesOrReturnError(), but I did notice that io.ReadFull() can do the opposite to return an error if not enough bytes are there to fill the provided buffer.

Does anyone have a solution to this?


EDIT:
To clarify, copying a fraction of the data is not OK. It either needs to fail if it's over the threshold, or work if it's under.

  • 写回答

2条回答 默认 最新

  • dongtong0796 2016-08-20 05:28
    关注

    Since io.Reader interface not knows anything about size or length of underlying data, there is only one solution for this problem:

    You may use one buffer with maximum size nMax (predetermined maximum)+1 and in every call to Your CopyLessThanOrEqualToThisManyBytesOrReturnError function, inside this function read input and buffer it, and check for this buffer length, if it is less than or equal to nMax then do io.Write, otherwise return error:

    const nMax = 5 // your predetermined maximum
    
    func CopyLessThanOrEqualToThisManyBytesOrReturnError(r io.Reader, w io.Writer) error {
        var buf = make([]byte, nMax+1)
        nRead, e := io.ReadFull(r, buf)
        if nRead > 0 && nRead <= nMax {
            w.Write(buf[:nRead])
            return nil
        }
        if nRead > nMax {
            return fmt.Errorf("there is more data")
        }
        return e
    }
    

    Like this working sample code unsing string:

    package main
    
    import (
        "fmt"
        "io"
        "os"
        "strings"
    )
    
    const nMax = 5 // your predetermined maximum
    
    func CopyLessThanOrEqualToThisManyBytesOrReturnError(r io.Reader, w io.Writer) error {
        var buf = make([]byte, nMax+1)
        nRead, e := io.ReadFull(r, buf)
        if nRead > 0 && nRead <= nMax {
            w.Write(buf[:nRead])
            return nil
        }
        if nRead > nMax {
            return fmt.Errorf("there is more data")
        }
        return e
    }
    
    func main() {
        r := strings.NewReader("123456789")
        err := CopyLessThanOrEqualToThisManyBytesOrReturnError(r, os.Stdout)
        if err != nil {
            fmt.Println(err) // there is more data
        }
    
        r = strings.NewReader("123
    ")
        err = CopyLessThanOrEqualToThisManyBytesOrReturnError(r, os.Stdout) // 123
        if err != nil {
            fmt.Println(err)
        }
    
        r = strings.NewReader("")
        err = CopyLessThanOrEqualToThisManyBytesOrReturnError(r, os.Stdout)
        if err != nil {
            fmt.Println(err) // EOF
        }
    }
    

    output:

    there is more data
    123
    EOF
    

    Working sample code, using files:

    package main
    
    import (
        "fmt"
        "io"
        "os"
    )
    
    const nMax = 5 // your predetermined maximum
    
    func CopyLessThanOrEqualToThisManyBytesOrReturnError(r io.Reader, w io.Writer) error {
        var buf = make([]byte, nMax+1)
        nRead, e := io.ReadFull(r, buf)
        if nRead > 0 && nRead <= nMax {
            w.Write(buf[:nRead])
            return nil
        }
        if nRead > nMax {
            return fmt.Errorf("there is more data")
        }
        return e
    }
    
    func main() {
        r, err := os.Open("input.bin")
        if err != nil {
            panic(err)
        }
        defer r.Close()
    
        w, err := os.Create("output.bin")
        if err != nil {
            panic(err)
        }
        defer w.Close()
    
        err = CopyLessThanOrEqualToThisManyBytesOrReturnError(r, w)
        if err != nil {
            fmt.Println(err)
        }
        fmt.Println("Done.")
    }
    

    Working sample code, using []byte:

    package main
    
    import (
        "bytes"
        "fmt"
        "io"
    )
    
    const nMax = 5 // your predetermined maximum
    
    func CopyLessThanOrEqualToThisManyBytesOrReturnError(r io.Reader, w io.Writer) error {
        var buf = make([]byte, nMax+1)
        nRead, e := io.ReadFull(r, buf)
        if nRead > 0 && nRead <= nMax {
            w.Write(buf[:nRead])
            return nil
        }
        if nRead > nMax {
            return fmt.Errorf("there is more data")
        }
        return e
    }
    
    func main() {
        bs := []byte{1, 2, 3, 4, 5}
        r := bytes.NewReader(bs)
    
        w := &bytes.Buffer{}
    
        err := CopyLessThanOrEqualToThisManyBytesOrReturnError(r, w)
        if err != nil {
            fmt.Println(err)
        }
    
        fmt.Println("Done.")
        fmt.Println(w.Bytes())
    }
    

    output:

    Done.
    [1 2 3 4 5]
    
    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论
查看更多回答(1条)

报告相同问题?

悬赏问题

  • ¥15 delta降尺度计算的一些细节,有偿
  • ¥15 Arduino红外遥控代码有问题
  • ¥15 数值计算离散正交多项式
  • ¥30 数值计算均差系数编程
  • ¥15 redis-full-check比较 两个集群的数据出错
  • ¥15 Matlab编程问题
  • ¥15 训练的多模态特征融合模型准确度很低怎么办
  • ¥15 kylin启动报错log4j类冲突
  • ¥15 超声波模块测距控制点灯,灯的闪烁很不稳定,经过调试发现测的距离偏大
  • ¥15 import arcpy出现importing _arcgisscripting 找不到相关程序