dsfd3546
dsfd3546
2016-11-02 20:08

如何从外部对golang http上传进行速率限制?

已采纳

I have a golang utility that uploads a lot of data via https to OpenStack Object Storage. I'm running it on Ubuntu Linux, and I would like to ensure that its maximum upload bandwidth does not exceed 2,500KB/s, preferably without affecting other users on the same system (i.e. slowing down the ethernet interface).

How can I do this, preferably without altering my source code? I've tried a few approaches so far:

  1. Traffic Shaping: I've been unable to figure out the appropriate network shaping techniques to affect a single process, though I know that should be possible with iptables and tc. Any guidance on how to do this would be appreciated. My networking background isn't very extensive.
  2. Flowrate Limiting: I tried using the flowrate package in go to rate-limit my writes to the upload. This didn't have any discernable effect. I think this doesn't work because the github.com/ncw/swift.ObjectCreateFile.Write() method doesn't upload the data as it is recieved, but rather when it is closed. I could be mistaken about this though.
  3. trickle: I tried the trickle command, but it isn't compatible with golang executables (see this question)
  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享
  • 邀请回答

2条回答

  • ds000001 ds000001 5年前

    It turns out that you can wrap the Golang process in a Docker container and then rate-limit the Docker container's network interface (from inside of the container) with the tc utility. See this answer for an example Dockerfile to do just that.

    点赞 评论 复制链接分享
  • dsfsda121545 dsfsda121545 5年前

    If you can control how you actually run the utility, you can use trickle

    sudo apt-get install trickle #since you're on Ubuntu
    trickle -u (upload limit in KB/s) -d (download limit in KB/s) executable
    

    Here is some additional documentation about it https://wiki.archlinux.org/index.php/Trickle

    点赞 评论 复制链接分享