意外的同步池分配

Question

all, been learning Golang for a month now and came across this problem . Basically I'm trying to reuse a []byte from sync.Pool to do some hashing.

I've added a minimum viable sample for reproducing it below:

type mWriter struct {
    pool *sync.Pool
}

func (m *mWriter) writeSpan(span interface{}) {
    haha := m.pool.Get().([]byte)
    // in real code some actions here instead of simply setting haha[0] = 1
    haha[0] = 1
    m.pool.Put(haha)
}

func NewWriter() *mWriter {
    bytepool := &sync.Pool{
        New: func() interface{} {
            return make([]byte, 16)
        },
    }
    return &mWriter{
        pool: bytepool,
    }
}

func Benchmark_WriteSpan(b *testing.B) {
    c := NewWriter()

    b.ResetTimer()
    b.ReportAllocs()
    for it := 0; it < b.N; it++ {
        c.writeSpan(nil)
    }
}

Under my impression sync.Pool doesn't allocate new memory for []byte, but I'm seeing extra allocation with the m.pool.get() here.

Benchmark_WriteSpan-12 30000000 47.6 ns/op 32 B/op 1 allocs/op PASS

What are the explanations behind this? I also tried some more simple benchmark here:

func Benchmark_2(b *testing.B) {
    // Initializing pool
    pool := &sync.Pool{
        New: func() interface{} {
            return make([]byte, 4)
        },
    }
    b.ResetTimer()
    b.ReportAllocs()
    // Get hold of instance one
    one := pool.Get().([]byte)
    one[1] = 1
    one[2] = 2
    // Submit back the instance after using
    pool.Put(one)
}

But this shows no allocation:

Benchmark_2-12 2000000000 0.00 ns/op 0 B/op 0 allocs/op

Appreciate any help here! (If this is not how sync.Pool is used, any suggestions?)

Edited:

OK, I added a simple loop inside the benchmark to writeSpan and now it gives following benchmark:

func Benchmark_WriteSpan(b *testing.B) {
    c := NewWriter()

    b.ResetTimer()
    b.ReportAllocs()
    for it := 0; it < b.N; it++ {
        for i := 0; i < 5; i++ {
            c.writeSpan(nil)
        }
    }
}

Benchmark_WriteSpan-12 5000000 226 ns/op 160 B/op 5 allocs/op

Looks like the pool allocates 32B in every write, shouldn't it be reusing the same byte[] after first get?

Update @JimB I do have some logic in real code which does updating the byte slice and hash on it. like the following:

    byteSlice := ds.poolInstance.bytepool.Get().([]byte)
    copy(byteSlice[:len(ds.hashSalt)], ds.hashSalt)
    span.TraceID.MarshalTo(byteSlice[len(ds.hashSalt):])
    hashVal := ds.hashBytes(byteSlice) 
    ds.poolInstance.bytepool.Put(byteSlice)

I'm not sure if this counts as free list maintained as part of a short-lived object, could you be more specific on this?

Answer 1

I think I found out the reason behind it. In my real code, I used to have:

bytepool: &sync.Pool{
    New: func() interface{} {
        return make([]byte, length)
    },
},
...
...
bytepool.Get().([]byte)

somehow when I changed it to

bytepool: &sync.Pool{
    New: func() interface{} {
        byteP := make([]byte, length)
        return &byteP
    },
},
...
...
bytepool.Get().(*[]byte)

the allocation drops to 0. So sorry folks looks like not an issue with sync.Pool but the underlying data structure. Thanks for your comments though!