I'm running a service where users upload a duration and a function must be repeatedly executed every time the timer runs out.
For example, the user says "Run every 5 minutes" and then this function must be run every 5 minutes. This is done through an API.
For a small numbers of timers, this is trivial:
func doEvery(d time.Duration, f func(time.Time)) {
for x := range time.Tick(d) {
f(x) // Run the function every d duration
}
}
I can run each timer in a goroutine, and that works fine. I can start and stop everything with some basic WaitGroups
and sync functionality.
But what if I have thousands or millions of timers? I can create a goroutine for each one, but that feels very inefficient. This isn't Erlang.
Should I have multiple work queues, sorted by the "delay", and simply allocate more workers for the more frequent functions? If the timer isn't ready, then it's put back on the queue.
This also isn't ideal, because the workers are busy-waiting (popping off, checking times, pushing on the queue) rather than blocking until the next timer finishes.
Maybe I could have some sort of map, indexed by the remaining duration? I'm not sure what the best approach is here.