内置追加与字节。缓冲区写入

Question

In the situation where I need to add an unknown amount of data to a slice of byte, let say in a loop, I could either way use the builtin function append() or create a new Buffer and use the Write() function.

Which method is fastest ?

Answer 1

It depends to use case.
Here in both cases bytes.Buffer is faster than append (Samples: 1, 2, 3, 4).

Using buf.Write(make([]byte, 16)) takes 4.6482659s,
Using buf = append(buf, make([]byte, 16)...) takes 6.6623811s.

For samples 5, 6:
Using buf = append(buf, byte(i)) takes 445.0255ms,
Using buf.WriteByte(byte(i)) takes 1.4410824s

---

And bytes.Buffer uses builtin function copy and it is fast:

// Write appends the contents of p to the buffer, growing the buffer as
// needed. The return value n is the length of p; err is always nil. If the
// buffer becomes too large, Write will panic with ErrTooLarge.

func (b *Buffer) Write(p []byte) (n int, err error) {
  b.lastRead = opInvalid
  m := b.grow(len(p))
  return copy(b.buf[m:], p), nil
}

---

bytes.Buffer takes 4.8892797s and append takes 7.7514434s

See these benchmarks:

1- Using append:

package main

import (
    "fmt"
    "time"
)

func main() {
    buf := []byte{}
    data := []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16}
    t := time.Now()
    for i := 0; i < 100000000; i++ {
        buf = append(buf, data...)
    }
    fmt.Println(time.Since(t))
    fmt.Println(len(buf))
}

output:

7.7514434s
1600000000

---

2- Using bytes.Buffer

package main

import (
    "bytes"
    "fmt"
    "time"
)

func main() {
    buf := &bytes.Buffer{}
    data := []byte{1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16}
    t := time.Now()
    for i := 0; i < 100000000; i++ {
        buf.Write(data)
    }
    fmt.Println(time.Since(t))
    fmt.Println(buf.Len())
}

output:

4.8892797s
1600000000

---

3- Using bytes.Buffer with make([]byte, 16):

package main

import (
    "bytes"
    "fmt"
    "time"
)

func main() {
    buf := &bytes.Buffer{}
    t := time.Now()
    for i := 0; i < 100000000; i++ {
        buf.Write(make([]byte, 16))
    }
    fmt.Println(time.Since(t)) // 4.6482659s
    fmt.Println(buf.Len())     //1600000000
}

---

4- Using append with make([]byte, 16):

package main

import (
    "fmt"
    "time"
)

func main() {
    buf := []byte{}
    t := time.Now()
    for i := 0; i < 100000000; i++ {
        buf = append(buf, make([]byte, 16)...)
    }
    fmt.Println(time.Since(t)) // 6.6623811s
    fmt.Println(len(buf))      // 1600000000
}

---

5- Using buf = append(buf, byte(i)) takes 445.0255ms:

package main

import (
    "fmt"
    "time"
)

func main() {
    buf := []byte{}
    t := time.Now()
    for i := 0; i < 100000000; i++ {
        buf = append(buf, byte(i))
    }
    fmt.Println(time.Since(t)) // 445.0255ms
    fmt.Println(len(buf))      // 100000000
}

---

6- Using buf.WriteByte(byte(i)) takes 1.4410824s:

package main

import (
    "bytes"
    "fmt"
    "time"
)

func main() {
    buf := &bytes.Buffer{}

    t := time.Now()
    for i := 0; i < 100000000; i++ {
        buf.WriteByte(byte(i))
    }
    fmt.Println(time.Since(t)) // 1.4410824s
    fmt.Println(buf.Len())     // 100000000
}

---

And See:

Appending to slice bad performance.. why?
Where is append() implementation?
Efficient appending to a variable-length container of strings (Golang)