Golang和大内存块分配

Question

I have the rather stupid benchmark to test the efficiency of memory allocation:

package main

import (
    "time"
    "fmt"
)

func main() {

    size := 1024 * 1024 * 1024

    t := time.Now()
    for j := 0; j < 100; j += 1 {
        a := make([]int, size)
        for i := 0; i < size; i += 1 {
            a[i] = i
        }
        a = nil
    }
    t1 := time.Now()
    fmt.Printf("Duration: %1d", t1.Sub(t).Seconds())
}

It takes roughly 2-3 minutes on Mac Pro with 16GB of RAM, the memory usage is consistent at 5-8 GB for the process.

The very similar code in Java takes 3 GB of memory and completes in 30 seconds.

What am I missing here?

Answer 1

What am I missing here?

In Java, size of the int type is fixed 4 bytes. In Go int is an architecture dependent type, on 32-bit architectures it's 32 bits (4 bytes), and on 64-bit architectures it's 64 bits (8 bytes).

Most likely you're running it on a 64-bit arch. Which means the size of the Go slice / array you allocate is 8 * 1 GB = 8 GB, while in Java it's only 4 * 1 GB = 4 GB.

Moreover, since you're using int in your loop, Java only has to increment and set 4-byte values, while in Go you're incrementing and setting 8-byte values (type of i and j will be int).

Change your Go code to use int32, and give it a go again.

Also note that your memory usage measurement is flawed, as just the array size in Java is 4 GB and 8 GB in Go, so that 3 GB for Java and 5-8 GB in Go is not the total memory usage!

Also note that []int in Go is a slice and not an array, they are not the same. Slices in Go are struct-like headers containing a pointer to a backing array (see reflect.SliceHeader for details), so there is an implicit indirection step involved using them. For details see Why have arrays in Go? Also related: Array vs Slice: accessing speed

One last note: your code does not measure memory allocation, as that is just the tiny part of the app's execution time. The majority (like 99.99999%) of the execution time is to increment the loop variable a billion times and to fill the arrays with a billion elements.