Let's say for arguments sake, I have a single SMTP server where I can send 60 emails per minute (1 per second).
If I have 60 users wanting to send emails through this SMTP server and they all send me 60 emails.
If I maintain 1 queue in Laravel, all 60 emails from user #1 will get sent first, then all 60 emails from user #2 will get sent next and so on. User #60 will have to wait for 3,540 emails to send before his starts.
I would like use Beanstalkd on a round-robin basis, and the only way I can think of doing this, is maintaining 60 tubes, one for each user but this would require me to run:
php artisan queue:listen --queue=user1,user2,user3,etc,user60
The problem I am facing here is that if user #61 signs up, his tube will not be picked up by the listener until I create a new command like so:
php artisan queue:listen --queue=user1,user2,user3,etc,user60,user61
If I grow to 1 million users, is it advisable to have 1 million tubes?
Is there any easy way to ask Beanstalkd to fetch 1 job per tube because these tubes can be created on the fly, I will never know how many tubes I have?
The problem I am facing is that I have a bottleneck on the SMTP server so I need to send these emails in a round robin fashion. I can achieve this easily using MySQL and Cron Jobs but this has led to duplicate emails in the past when using MySQL as a queueing system.