golang异步缓冲的通道挂起

Question

As a first go project I decided to write a simple async web-scaper. My idea is to have a queue of tasks and a pool of workers "solving" the tasks. While writing the program I encountered the a problem.

the following code hangs:

package main

import (
    "fmt"
    "net/http"
    "time"
)

type Scraper struct {
    client http.Client
    timeout int
    tasks chan string
    results chan int
    ntasks int
}

func (s Scraper) Init(timeout int, workers int) {
    s.client = http.Client{
        Timeout: time.Second * time.Duration(timeout),
    }
    s.timeout = timeout
    s.ntasks = 0
    s.Dispatch(workers)
}

func (s Scraper) Wait() {
    for i := 0; i < s.ntasks; i++ {
        <-s.results
    }
}

func (s Scraper) Task(task string) {
    s.tasks <- task // hangs on this line
    s.ntasks++;
}

func (s Scraper) Dispatch(workers int) {
    s.tasks   = make(chan string, 100)
    s.results = make(chan int,    100)
    for i := 0; i < workers; i++ {
        go s.worker(i)
    }
}

func (s Scraper) worker(id int) {
    for task := range <-s.tasks {
        fmt.Println(task)
        s.results <- 0
    }
}

func main() {

    s := Scraper{}
    s.Init(10, 5)
    s.Task("Hello World")
    s.Wait()

}

while this doesn't:

func worker(id int, jobs <-chan int, results chan<- int) {
    for j := range jobs {
        fmt.Println("worker", id, "started  job", j)
        time.Sleep(time.Second)
        fmt.Println("worker", id, "finished job", j)
        results <- j * 2
    }
}

func main() {
    jobs    := make(chan int, 100)
    results := make(chan int, 100)

    for w := 1; w <= 3; w++ {
        go worker(w, jobs, results)
    }

    for j := 1; j <= 5; j++ {
        jobs <- j
    }

    close(jobs)

    for a := 1; a <= 5; a++ {
        <-results
    }

}

Looking on stack overflow I saw that unbuffered channels hang but make(chan string, 100) should create a buffered one.

Answer 1

Change all your receivers to pointers like so:

func (s *Scraper) Init(timeout int, workers int) // *Scraper not 'Scraper'

For further details on pointer receivers: https://tour.golang.org/methods/4

As @JimB noted - the range has a bug too, it should be like so:

func (s *Scraper) worker(id int) {
    // `range s.tasks` not `range <-s.tasks`
    for task := range s.tasks {
        fmt.Println(task)
        s.results <- 0
    }
}

Playground with receiver & range fixes: https://play.golang.org/p/RulKHHfnvJo