根据golang中的字节长度分割字符串

Question

The http request header has a 4k length limit. I want to split the string which I want to include in the header based on this limit. Should I use []byte(str) to split first then convert back to string using string([]byte) for each split part? Is there any simpler way to do it?

Answer 1

In Go, a string is really just a sequence of bytes, and indexing a string produces bytes. So you could simply split your string into substrings by slicing it into 4kB substrings.

However, since UTF-8 characters can span multiple bytes, there is the chance that you will split in the middle of a character sequence. This isn't a problem if the split strings will always be joined together again in the same order at the other end before decoding, but if you try to decode each individually, you might end up with invalid leading or trailing byte sequences. If you want to guard against this, you could use the unicode/utf8 package to check that you are splitting on a valid leading byte, like this:

package httputil

import "unicode/utf8"

const maxLen = 4096

func SplitHeader(longString string) []string {
    splits := []string{}

    var l, r int
    for l, r = 0, maxLen; r < len(longString); l, r = r, r+maxLen {
        for !utf8.RuneStart(longString[r]) {
            r--
        }
        splits = append(splits, longString[l:r])
    }
    splits = append(splits, longString[l:])
    return splits
}

Slicing the string directly is more efficient than converting to []byte and back because, since a string is immutable and a []byte isn't, the data must be copied to new memory upon conversion, taking O(n) time (both ways!), whereas slicing a string simply returns a new string header backed by the same array as the original (taking constant time).