Your solution uses way too many allocations due to string concatenations.
We'll create some alternative, faster and/or more elegant solutions. Note that the below solutions do not check if node values contain the quotation mark " character. If they would, those would have to be escaped somehow (else the result would be an invalid query string).
The complete, runnable code can be found on the Go Playground. The complete testing / benchmarking code can also be found on the Go Playground, but it is not runnable, save both to your Go workspace (e.g. $GOPATH/src/query/query.go and $GOPATH/src/query/query_test.go) and run it with go test -bench ..
Also be sure to check out this related question: How to efficiently concatenate strings in Go?
Alternatives
Genesis
Your logic can be captured by the following function:
func buildOriginal(nodes []string) string {
var query string
for _, n := range nodes {
query += fmt.Sprintf("\"node_name\":\"%s\",", n)
}
query = strings.TrimRight(query, ",")
return fmt.Sprintf("where={%s}", query)
}
Using bytes.Buffer
Much better would be to use a single buffer, e.g. bytes.Buffer, build the query in that, and convert it to string at the end:
func buildBuffer(nodes []string) string {
buf := &bytes.Buffer{}
buf.WriteString("where={")
for i, v := range nodes {
if i > 0 {
buf.WriteByte(',')
}
buf.WriteString(`"node_name":"`)
buf.WriteString(v)
buf.WriteByte('"')
}
buf.WriteByte('}')
return buf.String()
}
Using it:
nodes := []string{"node1", "node2"}
fmt.Println(buildBuffer(nodes))
Output:
where={"node_name":"node1","node_name":"node2"}
bytes.Buffer improved
bytes.Buffer will still do some reallocations, although much less than your original solution.
However, we can still reduce the allocations to 1, if we pass a big-enough byte slice when creating the bytes.Buffer using bytes.NewBuffer(). We can calculate the required size prior:
func buildBuffer2(nodes []string) string {
size := 8 + len(nodes)*15
for _, v := range nodes {
size += len(v)
}
buf := bytes.NewBuffer(make([]byte, 0, size))
buf.WriteString("where={")
for i, v := range nodes {
if i > 0 {
buf.WriteByte(',')
}
buf.WriteString(`"node_name":"`)
buf.WriteString(v)
buf.WriteByte('"')
}
buf.WriteByte('}')
return buf.String()
}
Note that in size calculation 8 is the size of the string where={} and 15 is the size of the string "node_name":"",.
Using text/template
We can also create a text template, and use the text/template package to execute it, efficiently generating the result:
var t = template.Must(template.New("").Parse(templ))
func buildTemplate(nodes []string) string {
size := 8 + len(nodes)*15
for _, v := range nodes {
size += len(v)
}
buf := bytes.NewBuffer(make([]byte, 0, size))
if err := t.Execute(buf, nodes); err != nil {
log.Fatal(err) // Handle error
}
return buf.String()
}
const templ = `where={
{{- range $idx, $n := . -}}
{{if ne $idx 0}},{{end}}"node_name":"{{$n}}"
{{- end -}}
}`
Using strings.Join()
This solution is interesting due to its simplicity. We can use strings.Join() to join the nodes with the static text ","node_name":" in between, proper prefix and postfix applied.
An important thing to note: strings.Join() uses the builtin copy() function with a single preallocated []byte buffer, so it's very fast! "As a special case, it (the copy() function) also will copy bytes from a string to a slice of bytes."
func buildJoin(nodes []string) string {
if len(nodes) == 0 {
return "where={}"
}
return `where={"node_name":"` + strings.Join(nodes, `","node_name":"`) + `"}`
}
Benchmark results
We'll benchmark with the following nodes value:
var nodes = []string{"n1", "node2", "nodethree", "fourthNode",
"n1", "node2", "nodethree", "fourthNode",
"n1", "node2", "nodethree", "fourthNode",
"n1", "node2", "nodethree", "fourthNode",
"n1", "node2", "nodethree", "fourthNode",
}
And the benchmarking code looks like this:
func BenchmarkOriginal(b *testing.B) {
for i := 0; i < b.N; i++ {
buildOriginal(nodes)
}
}
func BenchmarkBuffer(b *testing.B) {
for i := 0; i < b.N; i++ {
buildBuffer(nodes)
}
}
// ... All the other benchmarking functions look the same
And now the results:
BenchmarkOriginal-4 200000 10572 ns/op
BenchmarkBuffer-4 500000 2914 ns/op
BenchmarkBuffer2-4 1000000 2024 ns/op
BenchmarkBufferTemplate-4 30000 77634 ns/op
BenchmarkJoin-4 2000000 830 ns/op
Some unsurprising facts: buildBuffer() is 3.6 times faster than buildOriginal(), and buildBuffer2() (with pre-calculated size) is about 30% faster than buildBuffer() because it does not need to reallocate (and copy over) the internal buffer.
Some surprising facts: buildJoin() is extremely fast, even beats buildBuffer2() by 2.4 times (due to only using a []byte and copy()). buildTemplate() on the other hand proved quite slow: 7 times slower than buildOriginal(). The main reason for this is because it uses (has to use) reflection under the hood.
