一次处理多个GET请求

Question

I'm writing this API which is a backend for slack bot application which will be used by many users. In one of the API endpoints in my api I'm calling an external API to get some data so I can massage it in my API to send it to the BOT/User. But that external API call I'm making is very expensive (~5 / ~10 secs). I've noticed that when I hit my API endpoint two or more times at the same time sometimes I don't get any data. Only one of the calls becomes successful. Is there any way that I can fix this? I've been looking into worker queue but not sure if its overkill for this.

I'm using mux to do my routing.

Answer 1

If you want to handle requests more quickly than the external API does, you'll need to separate your outbound calls from the incoming requests and do some sort of caching. For example, you can:

• Run a worker that calls your external API and stores the results in a cache
• Have you request handler pull from the cache
• If the data is recent, return it
• If the data is too old, wait for the cache to be refreshed, and use the new data

If this is a 24/7 busy service, the worker can just refresh the cache on a repeating schedule with whatever frequency is necessary; or, you can have the request handler queue a message to the cache worker signalling that the cache needs to be refreshed so that when you're not getting requests, it won't keep hitting the external API.