I always seem to be converting strings to []byte to string again over and over. Is there a lot of overhead with this? Is there a better way?
For example, here is a function that accepts a UTF8 string, normalizes it, remove accents, then converts special characters to ASCII equivalent:
var transliterations = map[rune]string{'Æ':"AE",'Ð':"D",'Ł':"L",'Ø':"OE",'Þ':"Th",'ß':"ss",'æ':"ae",'ð':"d",'ł':"l",'ø':"oe",'þ':"th",'Œ':"OE",'œ':"oe"}
func RemoveAccents(s string) string {
b := make([]byte, len(s))
t := transform.Chain(norm.NFD, transform.RemoveFunc(isMn), norm.NFC)
_, _, e := t.Transform(b, []byte(s), true)
if e != nil { panic(e) }
r := string(b)
var f bytes.Buffer
for _, c := range r {
temp := rune(c)
if val, ok := transliterations[temp]; ok {
f.WriteString(val)
} else {
f.WriteRune(temp)
}
}
return f.String()
}
So I'm starting with a string because that's what I get, then I'm converting it to a byte array, then back to a string, then to a byte array again, then back to a string again. Surely this is unnecessary but I can't figure out how to not do this..? And does it really have a lot of overhead or do I not have to worry about slowing things down with excessive conversions?
(Also if anyone has the time I've not yet figured out how bytes.Buffer
actually works, would it not be better to initialize a buffer of 2x the size of the string, which is the maximum output size of the return value?)