dsg24156
2011-10-16 13:53
浏览 60
已采纳

如何在Go中通过套接字检索文件数据?

I've got two small programs communicating nicely over a socket where the receiving side is in Go. Everything works peachy when my messages are tiny enough to fit in the 1024 byte buffer and can be received in a single Read from the connection but now I want to transfer data from an image that is 100k+ or more. I'm assuming the correct solution is not to increase the buffer until any image can fit inside.

Pseudo-go:

var buf = make([]byte,1024)
conn, err := net.Dial("tcp", ":1234")

for {
    r, err := conn.Read(buf[0:])
    go readHandler(string(buf[0:r]),conn)
}

How can I improve my socket read routine to accept both simple messages of a few bytes and also larger data? Bonus points if you can turn the total image data into an io.Reader for use in image.Decode.

图片转代码服务由CSDN问答提供 功能建议

我有两个小程序可以通过接收端在Go中的套接字很好地通信。 当我的消息很小到可以容纳1024字节缓冲区并且可以从连接中读取一次时,一切都会变好,但是现在我想从100k以上的图像中传输数据。 我假设正确的解决方案是在任何图像都适合内部之前不增加缓冲区。

Pseudo-go:

  var  buf = make([] byte,1024)
conn,err:= net.Dial(“ tcp”,“:1234”)
 
for {
r,err:= conn.Read(buf [0:])  
 go readHandler(string(buf [0:r]),conn)
} 
   
 
 

我如何改善套接字读取例程以接受这两个简单消息 几个字节,还有更大的数据? 奖励积分,如果您可以将全部图像数据转换为io.Reader以用于image.Decode。

  • 写回答
  • 好问题 提建议
  • 关注问题
  • 收藏
  • 邀请回答

2条回答 默认 最新

  • douchi8503 2011-10-18 15:40
    已采纳

    You can use io.ReadFull to read a []byte of a specific length. This assumes that you know beforehand how many bytes you need to read.

    As for image.Decode, it should be possible to pass the conn directly to the image.Decode function. This assumes that you do not perform any reads from the connection until the image is decoded.

    Your code

    for {
        r, err := conn.Read(buf[0:])
        go readHandler(string(buf[0:r]),conn)
    }
    

    seems to be suggesting that the goroutine you are starting is reading from conn This doesn't seem like a good idea, because you will end up having multiple concurrent reads from the connection (without having control over the order in which the reads will happen): one in the for-loop, another one in readHandler.

    已采纳该答案
    评论
    解决 无用
    打赏 举报
  • dshtze500055 2011-10-17 07:43

    I have no direct experience with TCP in Go but to me it seems that you fell victim of a quite typical misunderstanding of what guarntees TCP offers.

    The thing is, in contrast with, say, UDP and SCTP, TCP does not have the concept of message boundaries because it's stream-oriented. It means, TCP transports opaque streams of bytes and you have very little control of "chunking" that stream with regard to the receiving side.

    I suspect what you observe as "sending a 100k+ message" is the runtime/network library on the sender side typically "deceiving" you by consuming your "message" into its internal buffers and then streaming it in whatever chunks OS's TCP stack allows it to (on ubiquitous hardware/software it's usually about 8k). The size of pieces the receiver gets that stream is completely undefined; the only thing defined is ordering of the bytes in the stream, which is preserved.

    Hence it might turn out you have to resonsider your approach to receiving data. The exact approach varies depending on the nature of the data being streamed:

    • The easiest way (if you have the control over the application-level protocol) is to pass the length of the following "message payload" in a special length field of fixed format. Then destreaming the whole message is a two-step process: 1) receive that many bytes to get the length field, read it, check the value for sanity, then 2) read that many following bytes and be done with it.
    • If you have no control over the app-level protocol, parsing messages becomes more involved and usually requires some sort of complicated state machine.

    For more info, look at this and this.

    评论
    解决 无用
    打赏 举报

相关推荐 更多相似问题