In a pretty memory intense program, I'm trying to find memory leaks (or GC issues) because it eats too much memory over hours, and so decided to start pprof (by importing "net/http/pprof", starting server on 6060, and finally connecting to it with "http://localhost:6060/debug/pprof/heap?debug=1"). There are some points I don't understand, maybe somebody can help?
First, the memory summary at the very end says (I added some ' to make i more readable):
# Alloc = 4'702'361'800
# TotalAlloc = 22'597'867'947'496
# Sys = 32'198'957'880
# Lookups = 0
# Mallocs = 43'685'679'881
# Frees = 43'674'183'514
# HeapAlloc = 4'702'361'800
# HeapSys = 31'045'058'560
# HeapIdle = 25'753'124'864
# HeapInuse = 5'291'933'696
# HeapReleased = 24'779'595'776
# HeapObjects = 11'496'367
suggesting there is 4.7GB allocated on heap (HeapUse 5.2GB, probably due to fragmentation?)... but at the very first line, I see
heap profile: 4117: 118'349'192 [33286087: 7'831'672'653'232] @ heap/1048576
which seems to say that 118MB was seen in 4117 objects - both values are much smaller than the sizes above, how can that be, and how so I find about the other ~4.5GB? The profile seems to be about only these 120MB.
Then I also checked the whole situation with htop (it is on a Ubuntu 18 box with 64GB of RAM). This reports for that process 37.9GB of virtual memory and 29.9GB of reserved memory. While the 29.9GB match more or less HeapSys, I don't understand the virtual memory (which I think is underlied by physical memory)... Go seems to tell me that 24GB (HeapReleased) of the 30GB was returned to the OS, but htop does not seem to see this? I also read at some place Go does never return heap back to the OS, but the Go doc says HeapReleased is bytes of physical memory returned to the OS..
A final problem: when I write a memroy dump using debug.WriteHeapDump, I can't read the result data back with "go tool pprof binaryfile heapdumpfile", it tells me hheapdump: parsing profile: unrecognized profile format and failed to fetch any source profiles
