func main() {
jobs := []Job{job1, job2, job3}
numOfJobs := len(jobs)
resultsChan := make(chan *Result, numOfJobs)
jobChan := make(chan *job, numOfJobs)
go consume(numOfJobs, jobChan, resultsChan)
for i := 0; i < numOfJobs; i++ {
jobChan <- jobs[i]
}
close(jobChan)
for i := 0; i < numOfJobs; i++ {
<-resultsChan
}
close(resultsChan)
}
func (b *Blockchain) consume(num int, jobChan chan *Job, resultsChan chan *Result) {
for i := 0; i < num; i++ {
go func() {
job := <-jobChan
resultsChan <- doJob(job)
}()
}
}
In the above example, jobs are pushed into the jobChan and goroutines will pull it off the jobChan and execute the jobs concurrently and push results into resultsChan. We will then pull results out of resultsChan.
Question 1:
In my code, there is no serialized/linearilized results. Although jobs go in the order of job1, job2, job3. The results might come out as job3, job1, job2, depending which one takes the longest.
I would still like to execute the jobs concurrently, however, I need to make sure that results come out of the resultsChan in the same order that it went in as jobs.
Question2:
I have approximately 300k jobs, this means the code will generate up to 300k goroutines. Is this efficient to have so many goroutines or would I be better off group the jobs together in a slice of 100 or so and have each goroutine go through 100 rather than 1.