douben7260 2017-02-18 13:42
浏览 26
已采纳

在io.Reader中添加前缀

I've written a little server which receives a blob of data in the form of an io.Reader, adds a header and streams the result back to the caller.

My implementation isn't particularly efficient as I'm buffering the blob's data in-memory so that I can calculate the blob's length, which needs to form part of the header.

I've seen some examples of io.Pipe() with io.TeeReader but they're more for splitting an io.Reader into two, and writing them away in parallel.

The blobs I'm dealing with are around 100KB, so not huge but if my server gets busy, memory's going to quickly become an issue...

Any ideas?

func addHeader(in io.Reader) (out io.Reader, err error) {
    buf := new(bytes.Buffer)
    if _, err = io.Copy(buf, in); err != nil {
        return
    }

    header := bytes.NewReader([]byte(fmt.Sprintf("header:%d", buf.Len())))

    return io.MultiReader(header, buf), nil
}

I appreciate it's not a good idea to return interfaces from functions but this code isn't destined to become an API, so I'm not too concerned with that bit.

  • 写回答

1条回答 默认 最新

  • douhanshu5517 2017-02-18 15:41
    关注

    In general, the only way to determine the length of data in an io.Reader is to read until EOF. There are ways to determine the length of the data for specific types.

    func addHeader(in io.Reader) (out io.Reader, err error) {
      n := 0
      switch v := in.(type) {
      case *bytes.Buffer:
        n = v.Len()
      case *bytes.Reader:
        n = v.Len()
      case *strings.Reader:
        n = v.Len()
      case io.Seeker:
        cur, err := v.Seek(0, 1)
        if err != nil {
            return nil, err
        }
        end, err := v.Seek(0, 2)
        if err != nil {
            return nil, err
        }
        _, err = v.Seek(cur, 0)
        if err != nil {
            return nil, err
        }
        n = int(end - cur)
      default:
        var buf bytes.Buffer
        if _, err := buf.ReadFrom(in); err != nil {
            return nil, err
        }
        n = buf.Len()
        in = &buf
      }
      header := strings.NewReader(fmt.Sprintf("header:%d", n))
      return io.MultiReader(header, in), nil
    }
    

    This is similar to how the net/http package determines the content length of the request body.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥15 envi深度学习模块错误原因
  • ¥15 孟德尔随机化混杂因素
  • ¥15 关于react-hook组件用函数控制是否渲染的及时性问题。
  • ¥50 Linux下的软件,要做模块化拆分。进程间通信是否有开源框架可以借用?
  • ¥100 修改原有的MYSQL存储代码,在最右边添加多列数据
  • ¥20 Open Interpreter 使用时报错: still has pending operation at deallocation, the process may crash
  • ¥15 qt中链接动态链接库,调用其中的函数,该函数的参数需要传入回调函数,自己创建的回调函数无法作为参数传递进去
  • ¥15 matlab svm二分类代码问题
  • ¥40 求一款能支持ios15以上的屏蔽越狱插件。比较好用的
  • ¥15 C++ QT对比内存字符(形式不定)