Let me given you a background of what i am trying to get done.
I have to develop a system in php/mysql running on Linux. I am planning to use RabbitMQ for achieving something that i would say requires parallel processing.
I have a job per client that may take from 5 mins to 70 mins to complete depending on the amount of data to be processed for the given client. Client can post the job request randomly.
Let's assume that 5 clients post their job with the time required to complete respective job is as given below
- client1 job - 65 mins
- client2 job - 10 mins
- client3 job - 5 mins
- client4 job - 10 mins
- client5 job - 20 mins
A normal queue has a problem as job of client1 is going to take 65 mins to complete and so client2 has to wait 75 mins (65 + 10) to complete a job that takes 10 mins.
I know that in RabbitMQ i can create many workers to handle this job, but depending on the time it takes for the job and the position of the job in the queue, it may take alot of time for a client to get their job completed.
So i am looking to create dynamic consumers per client. And each consumer does the job for every client individually.
Is this possible in Rabbitmq? If yes, how can this be achieved?
