donglan6777
2018-12-12 16:36
浏览 59

如何减少gccgo编译的可执行文件所需的虚拟内存?

When I compile this simple hello world example using gccgo, the resulting executable uses over 800 MiB of VmData. I would like to know why, and if there is anything I can do to lower that. The sleep is just to give me time to observe the memory usage.

The source:

package main

import (
  "fmt"
  "time"
)

func main() {
  fmt.Println("hello world")
  time.Sleep(1000000000 * 5)
}

The script I use to compile:

#!/bin/bash

TOOLCHAIN_PREFIX=i686-linux-gnu
OPTIMIZATION_FLAG="-O3"

CGO_ENABLED=1 \
CC=${TOOLCHAIN_PREFIX}-gcc-8 \
CXX=${TOOLCHAIN_PREFIX}-g++-8 \
AR=${TOOLCHAIN_PREFIX}-ar \
GCCGO=${TOOLCHAIN_PREFIX}-gccgo-8 \
CGO_CFLAGS="-g ${OPTIMIZATION_FLAG}" \
CGO_CPPFLAGS="" \
CGO_CXXFLAGS="-g ${OPTIMIZATION_FLAG}" \
CGO_FFLAGS="-g ${OPTIMIZATION_FLAG}" \
CGO_LDFLAGS="-g ${OPTIMIZATION_FLAG}" \
GOOS=linux \
GOARCH=386 \
go build -x \
   -compiler=gccgo \
   -gccgoflags=all="-static -g ${OPTIMIZATION_FLAG}" \
   $1

The version of gccgo:

$ i686-linux-gnu-gccgo-8 --version
i686-linux-gnu-gccgo-8 (Ubuntu 8.2.0-1ubuntu2~18.04) 8.2.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The output from /proc/<pid>/status:

VmPeak:  811692 kB
VmSize:  811692 kB
VmLck:        0 kB
VmPin:        0 kB
VmHWM:     5796 kB
VmRSS:     5796 kB
VmData:  807196 kB
VmStk:      132 kB
VmExe:     2936 kB
VmLib:        0 kB
VmPTE:       52 kB
VmPMD:        0 kB
VmSwap:       0 kB

I ask because my device only has 512 MiB of RAM. I know that this is virtual memory but I would like to reduce or remove the overcommit if possible. It does not seem reasonable to me for a simple executable to require that much allocation.

  • 写回答
  • 关注问题
  • 收藏
  • 邀请回答

2条回答 默认 最新

  • dongyong8071 2018-12-14 21:56
    已采纳

    I was able to locate where gccgo is asking for so much memory. It's in the libgo/go/runtime/malloc.go file in the mallocinit function:

    // If we fail to allocate, try again with a smaller arena.
    // This is necessary on Android L where we share a process
    // with ART, which reserves virtual memory aggressively.
    // In the worst case, fall back to a 0-sized initial arena,
    // in the hope that subsequent reservations will succeed.
    arenaSizes := [...]uintptr{
      512 << 20,
      256 << 20,
      128 << 20,
      0,
    }
    
    for _, arenaSize := range &arenaSizes {
      // SysReserve treats the address we ask for, end, as a hint,
      // not as an absolute requirement. If we ask for the end
      // of the data segment but the operating system requires
      // a little more space before we can start allocating, it will
      // give out a slightly higher pointer. Except QEMU, which
      // is buggy, as usual: it won't adjust the pointer upward.
      // So adjust it upward a little bit ourselves: 1/4 MB to get
      // away from the running binary image and then round up
      // to a MB boundary.
      p = round(getEnd()+(1<<18), 1<<20)
      pSize = bitmapSize + spansSize + arenaSize + _PageSize
      if p <= procBrk && procBrk < p+pSize {
        // Move the start above the brk,
        // leaving some room for future brk
        // expansion.
        p = round(procBrk+(1<<20), 1<<20)
      }
      p = uintptr(sysReserve(unsafe.Pointer(p), pSize, &reserved))
      if p != 0 {
        break
      }
    }
    if p == 0 {
      throw("runtime: cannot reserve arena virtual address space")
    }
    

    The interesting part is that it falls back to smaller arena sizes if larger ones fail. So limiting the virtual memory available to a go executable will actually limit how much it will successfully allocate.

    I was able to use ulimit -v 327680 to limit the virtual memory to smaller numbers:

    VmPeak:   300772 kB
    VmSize:   300772 kB
    VmLck:         0 kB
    VmPin:         0 kB
    VmHWM:      5712 kB
    VmRSS:      5712 kB
    VmData:   296276 kB
    VmStk:       132 kB
    VmExe:      2936 kB
    VmLib:         0 kB
    VmPTE:        56 kB
    VmPMD:         0 kB
    VmSwap:        0 kB
    

    These are still big numbers, but the best that a gccgo executable can achieve. So the answer to the question is, yes you can reduce the VmData of a gccgo compiled executable, but you really shouldn't worry about it. (On a 64 bit machine gccgo tries to allocate 512 GB.)

    已采纳该答案
    打赏 评论
  • dongpi9494 2018-12-12 19:25

    The likely cause is that you are linking libraries into the code. My guess is that you'd be able to get a smaller logical address space if you were to explicitly link to static libraries so that you get the minimum added to your executable. In any event, there is minimum harm in having a large logical address space.

    打赏 评论

相关推荐 更多相似问题