双通道死锁

Question

I'm trying to create a program that send strings to a pool of goroutines (through a channel). Once the goroutine have finish their job, they send some results (through an other channel).

The code is:

package main

import "fmt"
import "os"
import "sync"
import "bufio"

func worker(linkChan <-chan string, outChan chan<- string, wg *sync.WaitGroup, jobId int) {
   defer wg.Done()

   for url := range linkChan {
    // ...
     outChan <- url
   }
}

func main() {
    lCh := make(chan string)
    wg := new(sync.WaitGroup)
    outCh := make(chan string)

    urls := []string{}
    if len(os.Args) > 1 {
        for _, link := range os.Args[1:] {
            urls = append(urls, link)
        }
    } else {
        s := bufio.NewScanner(os.Stdin)
        for s.Scan() {
            urls = append(urls, s.Text())
        }
    }

    num_worker := 10

    for i := 0; i < num_worker; i++ {
        wg.Add(1)
        go worker(lCh, outCh, wg, i)
    }
    for _, link := range urls {
        lCh <- link
    }
    close(lCh)

    for res := range outCh {
        fmt.Printf("%s
", res)
    }
    close(outCh)
    wg.Wait()

}

Running echo "something" | ./main cause a deadlock.

From what I've understood, close(lCh) should stop the for url := range linkChan loop. Am I wrong (it seems so since the code deadlock) ?

How can I resolve this deadlock ?

Thank you for your answers.

Answer 1

You need to pump the urls in a goroutine, otherwise the outCh will fill up which as you aren't emptying it. This will stall all the workers and it will deadlock.

So re-arrange the code to look like this

go func() {
    for _, link := range urls {
        lCh <- link
    }
    close(lCh)
    wg.Wait()
    close(outCh)
}()

for res := range outCh {
    fmt.Printf("%s
", res)
}

And it will work fine

Complete code