douben7260 2017-02-18 13:42
浏览 26
已采纳

在io.Reader中添加前缀

I've written a little server which receives a blob of data in the form of an io.Reader, adds a header and streams the result back to the caller.

My implementation isn't particularly efficient as I'm buffering the blob's data in-memory so that I can calculate the blob's length, which needs to form part of the header.

I've seen some examples of io.Pipe() with io.TeeReader but they're more for splitting an io.Reader into two, and writing them away in parallel.

The blobs I'm dealing with are around 100KB, so not huge but if my server gets busy, memory's going to quickly become an issue...

Any ideas?

func addHeader(in io.Reader) (out io.Reader, err error) {
    buf := new(bytes.Buffer)
    if _, err = io.Copy(buf, in); err != nil {
        return
    }

    header := bytes.NewReader([]byte(fmt.Sprintf("header:%d", buf.Len())))

    return io.MultiReader(header, buf), nil
}

I appreciate it's not a good idea to return interfaces from functions but this code isn't destined to become an API, so I'm not too concerned with that bit.

  • 写回答

1条回答 默认 最新

  • douhanshu5517 2017-02-18 15:41
    关注

    In general, the only way to determine the length of data in an io.Reader is to read until EOF. There are ways to determine the length of the data for specific types.

    func addHeader(in io.Reader) (out io.Reader, err error) {
      n := 0
      switch v := in.(type) {
      case *bytes.Buffer:
        n = v.Len()
      case *bytes.Reader:
        n = v.Len()
      case *strings.Reader:
        n = v.Len()
      case io.Seeker:
        cur, err := v.Seek(0, 1)
        if err != nil {
            return nil, err
        }
        end, err := v.Seek(0, 2)
        if err != nil {
            return nil, err
        }
        _, err = v.Seek(cur, 0)
        if err != nil {
            return nil, err
        }
        n = int(end - cur)
      default:
        var buf bytes.Buffer
        if _, err := buf.ReadFrom(in); err != nil {
            return nil, err
        }
        n = buf.Len()
        in = &buf
      }
      header := strings.NewReader(fmt.Sprintf("header:%d", n))
      return io.MultiReader(header, in), nil
    }
    

    This is similar to how the net/http package determines the content length of the request body.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥15 office打开卡退(新电脑重装office系统后)
  • ¥300 FLUENT 火箭发动机燃烧EDC仿真
  • ¥15 【Hadoop 问题】Hadoop编译所遇问题hadoop-common: make failed with error code 2
  • ¥15 vb6.0+webbrowser无法加载某个网页求解
  • ¥15 RPA财务机器人采购付款流程
  • ¥15 计算机图形多边形及三次样条曲线绘制
  • ¥15 根据protues画的图用keil写程序
  • ¥200 如何使用postGis实现最短领规划?
  • ¥15 pyinstaller打包错误
  • ¥20 cesm的气溶胶排放文件