暂停for循环的执行

Question

I have an API that I am POSTing a small JSON object too.

This runs as an infinite loop, looping over 7 colours (the rainbow) and send these inside the aforementioned JSON object.

The API I am connecting to has a rate limit on it of 40 requests per minute.

I don't want to hit the rate limit and so I have devised a method to avoid this;

• I have a global variable that stores the number of requests I am allowed to make
• I have a function that holds a ticker set to run every 60 seconds and top up the global variable that stores my requests
• I then have a never ending for loop that runs and checks the number of requests that are still allowed to be made, if it is greater than 0 then we make our next request, if is not then we just sleep for a second and try again

It looks a little something like this:

var rateLimit int

func main() {

    request := gorequest.New().SetDebug(false)

    // Set the initial request bucket to 40
    rateLimit = 40

    go topUpLimiter()

    for {
        makeTheLightsBlinkTheRainbow(request)
    }
}

func topUpLimiter() {
    for range time.Tick(60 * time.Second) {
        rateLimit += 40
    }
}

func makeTheLightsBlinkTheRainbow(request *gorequest.SuperAgent) {
    colours := [7]string{"red", "orange", "yellow", "green", "blue", "purple", "pink"}

    for _, colour := range colours {

        if rateLimit > 0 {

            response, _, _ := request.Post("http://example.com/blink").
                Send(fmt.Sprintf(`{"color":"%v"}`, colour)).
                End()

            rateLimit--

        } else {
            time.Sleep(1 * time.Second)
        }
    }
}

This works and I don't ever hit the rate limit, but once I run out of requests, the loop continues to run and will only starts making requests again when the rateLimit variable is topped up.

I am making an IoT light blink the colours of the rainbow and the result is the colours get out of order once the rateLimit variable runs out and is then later topped up due to the fact that the for loop just keeps running.

I'd like to pause/block the for loop while I wait for the rateLimit variable to replenish so that the colours don't get out of order.

How would I implement something like that? From my searches it seems like it may be possible with channels, but I'm not too sure how to do it like that.

This is an example of what ends up happening: https://play.golang.org/p/r6OG4kK9vCP Once it has finished running, you will notice the colours printed out get out of order about halfway through.

Answer 1

My suggestion: Take an entirely different approach to rate limiting.

I would put it in the HTTP transport, since that's logically where such a limit ought to exist, and then you needn't mangle your application code at all. Something like:

import (
    "net/http"

    "golang.org/x/time/rate"
)

type rateLimitTransport struct {
    limiter *rate.Limiter
    xport   http.RoundTripper
}

var _ http.RoundTripper = &rateLimitTransport{}

func newRateLimitTransport(r float64, xport http.RoundTripper) http.RoundTripper {
    return &rateLimitTransport{
        limiter: rate.NewLimiter(rate.Limit(r), 1),
        xport:   xport,
    }
}

func (t *rateLimitTransport) RoundTrip(r *http.Request) (*http.Response, error) {
    t.limiter.Wait(r.Context())
    return t.xport.RoundTrip(r)
}

This uses the golang.org/x/time/rate package to implement the rate limiting. newRateLimitTransport() creates a new rate-limiting transport, where r is the maximum number of requests to allow per second.

To take advantage of this, use an instance of the rate-limited transport in the HTTP client for your backend API:

// set this in `init()` for example
myClient := http.&Client{
    // Use a rate-limiting transport which falls back to the default
    Transport: newRateLimitTransport(60, http.DefaultTransport)
}

// Then later use `myClient` instead of the default, when making API
// requests:
req, err := http.NewRequest(http.MethodPost, url, body)
if err != nil {
    return err
}
myClient.Do(req)