douwei1921 2016-10-19 11:45
浏览 41
已采纳

如何在Go中用gzip压缩版本替换文件变量?

I have the following Go code:

file, err := os.Open(fileName)
if err != nil {
    fatalf(service, "Error opening %q: %v", fileName, err)
}

// Check if gzip should be applied
if *metaGzip == true {
    var b bytes.Buffer
    w := gzip.NewWriter(&b)
    w.Write(file)
    w.Close()
    file = w
}

I want to replace the file content of file with a gzipped version if metaGzip = true.

PS:
I followed this advice: Getting "bytes.Buffer does not implement io.Writer" error message but I still get the error: cannot use file (type *os.File) as type []byte in argument to w.Write

  • 写回答

1条回答 默认 最新

  • douxianglu4370 2016-10-19 12:59
    关注

    There are quite a few errors in your code.

    As a "pre-first", always check returned errors!

    First, os.Open() opens the file in read-only mode. To be able to replace the file content on the disk, you must open it in read-write mode instead:

    file, err := os.OpenFile(fileName, os.O_RDWR, 0)
    

    Next, when you open something that is an io.Closer (*os.File is an io.Closer), make sure you close it with the Close() method, best done as a deferred statement.

    Next, *os.File is an io.Reader, but that is not the same thing as a byte slice []byte. An io.Reader may be used to read bytes into a byte slice. Use io.Copy() to copy the content from the file to the gzip stream (which will end up in the buffer).

    In certain situation (where you don't close the gzip.Writer), you must call gzip.Writer.Flush() to ensure everything is flushed into its writer (which is the buffer in this case). Note that gzip.Writer.Close() also flushes, so this may seem like an unnecessary step, but must be done for example if the Close() of the gzip.Writer is also called as a deferred statemement, because then it may not be executed before we use the content of the buffer. Since in our examle we close the gzip writer after io.Copy(), that will take care of necessary flushes.

    Next, to replace the content of the original file, you must seek back to the beginning of the file to replace. For that, you may use File.Seek().

    Next, you may again use io.Copy() to copy the contents of the buffer (the gzipped data) to the file.

    And last, since the gzipped content will most likely be shorter than the original file size, you must truncate the file at the size of the gzipped content (else uncompressed content of the original file may be left there).

    Here's the complete code:

    file, err := os.OpenFile(fileName, os.O_RDWR, 0)
    if err != nil {
        log.Fatalf("Error opening %q: %v", fileName, err)
    }
    defer file.Close()
    
    // Check if gzip should be applied
    if *metaGzip {
        var b = &bytes.Buffer{}
        w := gzip.NewWriter(b)
        if _, err := io.Copy(w, file); err != nil {
            panic(err)
        }
        if err := w.Close(); err != nil { // This also flushes
            panic(err)
        }
        if _, err := file.Seek(0, 0); err != nil {
            panic(err)
        }
        if _, err := io.Copy(file, b); err != nil {
            panic(err)
        }
        if err := file.Truncate(int64(b.Len())); err != nil {
            panic(err)
        }
    }
    

    Note: The above code will replace the file content on your disk. If you don't want this and you just need the compressed data, you may do it like this. Note that I used a new input variable of type io.Reader, as a value of bytes.Buffer (or *bytes.Buffer) cannot be assigned to a variable of type *os.File, and we will most likely only need the result as a value of io.Reader (and this is implemented by both):

    var input io.Reader
    
    file, err := os.Open(fileName)
    if err != nil {
        log.Fatalf("Error opening %q: %v", fileName, err)
    }
    defer file.Close()
    
    // Check if gzip should be applied
    if *metaGzip {
        var b = &bytes.Buffer{}
        w := gzip.NewWriter(b)
        if _, err := io.Copy(w, file); err != nil {
            panic(err)
        }
        if err := w.Close(); err != nil { // This also flushes
            panic(err)
        }
        input = b
    } else {
        input = file
    }
    
    // Use input here
    

    Note #2: If you don't want to "work" with the compressed data but you just want to send it e.g. as the web response, you don't even need the bytes.Buffer, you can just "stream" the compressed data to the http.ResponseWriter.

    It could look like this:

    func myHandler(w http.ResponseWriter, r *http.Request) {
        file, err := os.Open(fileName)
        if err != nil {
            http.NotFound(w, r)
        }
        defer file.Close()
    
        gz := gzip.NewWriter(w)
        defer gz.Close()
    
        if _, err := io.Copy(gz, file); err != nil {
            // handle error
        }
    }
    

    Proper content type will be detected and set automatically.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥15 flink cdc无法实时同步mysql数据
  • ¥100 有人会搭建GPT-J-6B框架吗?有偿
  • ¥15 求差集那个函数有问题,有无佬可以解决
  • ¥15 【提问】基于Invest的水源涵养
  • ¥20 微信网友居然可以通过vx号找到我绑的手机号
  • ¥15 寻一个支付宝扫码远程授权登录的软件助手app
  • ¥15 解riccati方程组
  • ¥15 使用rabbitMQ 消息队列作为url源进行多线程爬取时,总有几个url没有处理的问题。
  • ¥15 Ubuntu在安装序列比对软件STAR时出现报错如何解决
  • ¥50 树莓派安卓APK系统签名