I have a list of URLs that I need to use goroutine to fire off HTTP requests concurrently. Is there anyway to check and limit how many of those HTTP requests are sent per second?
2条回答 默认 最新
- douou9786 2017-05-04 20:27关注
A very simple version of this in Go would be an adaptation of a Leaky Bucket algorithm, using a channel and a goroutine. Receiving a token from the
rate
channel before making a request will check the rate and block if the rate limiter is empty.// create a buffered channel. // The capacity of the channel is maximum burst that can be made. rate := make(chan struct{}, 10) go func() { ticker := time.NewTicker(100 * time.Millisecond) for range ticker.C { rate <- struct{}{} } }()
Since a series of requests that take longer than the average rate will end up being concurrent, you may need to limit concurrency too. You can add a second channel as a semaphore, adding a token to the semaphore before making a request, and removing it when it's complete.
// limit concurrency to 5 semaphore := make(chan struct{}, 5) // in request function semaphore <- struct{}{} defer func() { <-semaphore }()
A slightly more complete example is here:
本回答被题主选为最佳回答 , 对您是否有帮助呢?解决 无用评论 打赏 举报
悬赏问题
- ¥15 drone 推送镜像时候 purge: true 推送完毕后没有删除对应的镜像,手动拷贝到服务器执行结果正确在样才能让指令自动执行成功删除对应镜像,如何解决?
- ¥15 求daily translation(DT)偏差订正方法的代码
- ¥15 js调用html页面需要隐藏某个按钮
- ¥15 ads仿真结果在圆图上是怎么读数的
- ¥20 Cotex M3的调试和程序执行方式是什么样的?
- ¥20 java项目连接sqlserver时报ssl相关错误
- ¥15 一道python难题3
- ¥15 牛顿斯科特系数表表示
- ¥15 arduino 步进电机
- ¥20 程序进入HardFault_Handler