dtrz17917
2016-05-23 04:19 阅读 124
已采纳

一旦被字节占用就无法释放内存。

I receive bytes of compressed ASCII text in compressedbytes of type []byte. The problem I face is that the following procedure occupies a lot of memory that does not get freed after the function reaches its end and remains occupied during the whole runtime of the program.

    b := bytes.NewReader(compressedbytes)
    r, err := zlib.NewReader(b)
    if err != nil {
        panic(err)
    }
    cleartext, err = ioutil.ReadAll(r)
    if err != nil {
        panic(err)
    }

I noticed that the type in use is bytes.Buffer and this type has the Reset() and Truncate() functions but none of them allows to free the memory that is once occupied.

The documentation of Reset() states the following:

Reset resets the buffer to be empty, but it retains the underlying storage for use by future writes. Reset is the same as Truncate(0).

How can I unset the buffer and free the memory again? My program needs about 50MB of memory during the run that takes 2h. When I import strings that are zlib compressed the program needs 200 MB of memory.

Thanks for your help.

=== Update

I even created a separate function for the decompression and call the garbage collector manually with runtime.GC() after the program returns from that function without success.

// unpack decompresses zlib compressed bytes
func unpack(packedData []byte) []byte {
    b := bytes.NewReader(packedData)
    r, err := zlib.NewReader(b)
    if err != nil {
        panic(err)
    }
    cleartext, err := ioutil.ReadAll(r)
    if err != nil {
        panic(err)
    }
    r.Close()
    return cleartext
}
  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享

2条回答 默认 最新

  • 已采纳
    doudang1052 doudang1052 2016-05-23 06:05

    Some things to clear. Go is a garbage collected language, which means that memory allocated and used by variables is automatically freed by the garbage collector when those variables become unreachable (if you have another pointer to the variable, that still counts as "reachable").

    Freed memory does not mean it is returned to the OS. Freed memory means the memory can be reclaimed, reused for another variable if there is a need. So from the operating system you won't see memory decreasing right away just because some variable became unreachable and the garbage collector detected this and freed memory used by it.

    The Go runtime will however return memory to the OS if it is not used for some time (which is usually around 5 minutes). If the memory usage increases during this period (and optionally shrinks again), the memory will most likely not be returned to the OS.

    If you wait some time and not allocate memory again, freed memory will be returned to the OS eventually (obviously not all, but unused "big chunks" will be). If you can't wait for this to happen, you may call debug.FreeOSMemory() to force this behavior:

    FreeOSMemory forces a garbage collection followed by an attempt to return as much memory to the operating system as possible. (Even if this is not called, the runtime gradually returns memory to the operating system in a background task.)

    Check out this kind of old but really informative question+answers:

    Go 1.3 Garbage collector not releasing server memory back to system

    点赞 评论 复制链接分享
  • douduan1953 douduan1953 2016-05-23 04:23

    It will eventually get released when nothing references it anymore, Go has a rather decent GC.

    点赞 评论 复制链接分享

相关推荐