I receive bytes of compressed ASCII text in compressedbytes
of type []byte. The problem I face is that the following procedure occupies a lot of memory that does not get freed after the function reaches its end and remains occupied during the whole runtime of the program.
b := bytes.NewReader(compressedbytes)
r, err := zlib.NewReader(b)
if err != nil {
panic(err)
}
cleartext, err = ioutil.ReadAll(r)
if err != nil {
panic(err)
}
I noticed that the type in use is bytes.Buffer
and this type has the Reset()
and Truncate()
functions but none of them allows to free the memory that is once occupied.
The documentation of Reset()
states the following:
Reset resets the buffer to be empty, but it retains the underlying storage for use by future writes. Reset is the same as Truncate(0).
How can I unset the buffer and free the memory again? My program needs about 50MB of memory during the run that takes 2h. When I import strings that are zlib compressed the program needs 200 MB of memory.
Thanks for your help.
=== Update
I even created a separate function for the decompression and call the garbage collector manually with runtime.GC()
after the program returns from that function without success.
// unpack decompresses zlib compressed bytes
func unpack(packedData []byte) []byte {
b := bytes.NewReader(packedData)
r, err := zlib.NewReader(b)
if err != nil {
panic(err)
}
cleartext, err := ioutil.ReadAll(r)
if err != nil {
panic(err)
}
r.Close()
return cleartext
}