Golang：TCP客户端/服务器数据定界符

Question

Not sure how to formulate the question and if it really relates only to go language, but what i am trying to do is to have a tcp server and client that will exchange data in between, basically the client will stream big amounts of data into smaller chunks to the server, the server will wait to read every chunk of data and then reply with a status code which will be read by the client and based on that it will do other work.

I use the function below as a test to read the data from client and server (please note, i am aware that is not perfect, but it's just testing) :

func createBufferFromConn(conn net.Conn) *bytes.Buffer {
    buffer := &bytes.Buffer{}
    doBreak := false
    for {
        incoming := make([]byte, BUFFER_SIZE)

        conn.SetReadDeadline(time.Now().Add(time.Second * 2))
        bytesRead, err := conn.Read(incoming)
        conn.SetReadDeadline(time.Time{})

        if err != nil {
            if err == io.EOF {
                fmt.Println(err)
            } else if neterr, ok := err.(net.Error); ok && neterr.Timeout() {
                fmt.Println(err)
            }
            doBreak = true
        }

        if doBreak == false && bytesRead == 0 {
            continue
        }

        if bytesRead > 0 {
            buffer.Write(incoming[:bytesRead])
            if bytes.HasSuffix(buffer.Bytes(), []byte("|")) {
                bb := bytes.Trim(buffer.Bytes(), "|")
                buffer.Reset()
                buffer.Write(bb)
                doBreak = true
            }
        }

        if doBreak {
            break
        }
    }
    return buffer
}

Now in my case if i connect via telnet(the go code also includes a client() to connect to the server()) and i type something like test 12345| fair enough everything works just fine and the buffer contains all the bytes written from telnet(except the pipe which is removed by the Trim() call).

If i remove the if bytes.HasSuffix(buffer.Bytes(), []byte("|")) { block from the code then i will get a timeout after 2 seconds, again, as expected because no data is received in that amount of time and the server closes the connection, and if i don't set a read deadline from the connection, it will wait forever to read data and will never know when to stop.

I guess my question is, if i send multiple chunks of data, do i have to specify a delimiter of my own so that i know when to stop reading from the connection and avoid waiting forever or waiting for the server to timeout the connection ?

Answer 1

I guess my question is, if i send multiple chunks of data, do i have to specify a delimiter of my own so that i know when to stop reading from the connection and avoid waiting forever or waiting for the server to timeout the connection

Yes. TCP is a stream protocol, and there's no way to determine where messages within the protocol start and stop without framing them in some way.

A more common framing method used is to send a size prefix, so that the receiver knows how much to read without having to buffer the results and scan for a delimiter. This can be as simple as message_length:data.... (see also netstring, and type-length-value encoding).