douben7260 2017-02-18 13:42
浏览 26
已采纳

在io.Reader中添加前缀

I've written a little server which receives a blob of data in the form of an io.Reader, adds a header and streams the result back to the caller.

My implementation isn't particularly efficient as I'm buffering the blob's data in-memory so that I can calculate the blob's length, which needs to form part of the header.

I've seen some examples of io.Pipe() with io.TeeReader but they're more for splitting an io.Reader into two, and writing them away in parallel.

The blobs I'm dealing with are around 100KB, so not huge but if my server gets busy, memory's going to quickly become an issue...

Any ideas?

func addHeader(in io.Reader) (out io.Reader, err error) {
    buf := new(bytes.Buffer)
    if _, err = io.Copy(buf, in); err != nil {
        return
    }

    header := bytes.NewReader([]byte(fmt.Sprintf("header:%d", buf.Len())))

    return io.MultiReader(header, buf), nil
}

I appreciate it's not a good idea to return interfaces from functions but this code isn't destined to become an API, so I'm not too concerned with that bit.

  • 写回答

1条回答 默认 最新

  • douhanshu5517 2017-02-18 15:41
    关注

    In general, the only way to determine the length of data in an io.Reader is to read until EOF. There are ways to determine the length of the data for specific types.

    func addHeader(in io.Reader) (out io.Reader, err error) {
      n := 0
      switch v := in.(type) {
      case *bytes.Buffer:
        n = v.Len()
      case *bytes.Reader:
        n = v.Len()
      case *strings.Reader:
        n = v.Len()
      case io.Seeker:
        cur, err := v.Seek(0, 1)
        if err != nil {
            return nil, err
        }
        end, err := v.Seek(0, 2)
        if err != nil {
            return nil, err
        }
        _, err = v.Seek(cur, 0)
        if err != nil {
            return nil, err
        }
        n = int(end - cur)
      default:
        var buf bytes.Buffer
        if _, err := buf.ReadFrom(in); err != nil {
            return nil, err
        }
        n = buf.Len()
        in = &buf
      }
      header := strings.NewReader(fmt.Sprintf("header:%d", n))
      return io.MultiReader(header, in), nil
    }
    

    This is similar to how the net/http package determines the content length of the request body.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥15 jupyterthemes 设置完毕后没有效果
  • ¥15 matlab图像高斯低通滤波
  • ¥15 针对曲面部件的制孔路径规划,大家有什么思路吗
  • ¥15 钢筋实图交点识别,机器视觉代码
  • ¥15 如何在Linux系统中,但是在window系统上idea里面可以正常运行?(相关搜索:jar包)
  • ¥50 400g qsfp 光模块iphy方案
  • ¥15 两块ADC0804用proteus仿真时,出现异常
  • ¥15 关于风控系统,如何去选择
  • ¥15 这款软件是什么?需要能满足我的需求
  • ¥15 SpringSecurityOauth2登陆前后request不一致