Goroutine不使用最大CPU和内核

Question

I'm realizing my very first Golang application and I'm having some issues on using MAX CPU & Cores when using GoRoutines and I don't really know why.

When using a tool such as htop, CPU isn't used at its max power and only 1..4 threads are active at time. Also, all cores are active but they are around 25%-40% utilization.

I used:

func MaxParallelism() int {
    maxProcs := runtime.GOMAXPROCS(0)
    numCPU := runtime.NumCPU()
    if maxProcs < numCPU {
        return maxProcs
    }
    return numCPU
}

In order to get the number of goroutines to be instantiated.

Here's How I set up the application:

//Common Channel for the goroutines
tasks := make(chan *exec.Cmd, 64)

    //Spawning Max Cores Number goroutines (8)
    var wg sync.WaitGroup
    cores := MaxParallelism()
    for i := 0; i < cores; i++ {
        wg.Add(1)
        go func(num int, w *sync.WaitGroup) {
            defer w.Done()
            var (
                out []byte
                err error
            )
            for cmd := range tasks {
                out, err = cmd.Output()
                if err != nil {
                    fmt.Printf("Can't get stdout:", err)
                }
                . . .
            }
        }(i, &wg)
    }

    //Generate Tasks
    for i := 0; i < 100000; i++ {
      tasks <- exec.Command(cmd1, args...)
      tasks <- exec.Command(cmd2, args...)
    }

close(tasks)
// wait for the workers to finish
wg.Wait()

I share two screenshots of htop while executing the application

I don't know If it May help but I'm launching it through Intellij Idea.

How do I use Max CPU properly and Cores?

Thanks in advance.

Answer 1

Goroutines and threads are not the same. Ergo you should not expect any CPU affinity. See more for details here http://tleyden.github.io/blog/2014/10/30/goroutines-vs-threads/.

Here are some of the advantages of Goroutines over threads:

• You can run more goroutines on a typical system than you can threads.
• Goroutines have growable segmented stacks.
• Goroutines have a faster startup time than threads.
• Goroutines come with built-in primitives to communicate safely between themselves (channels).
• Goroutines allow you to avoid having to resort to mutex locking when sharing data structures.
• Goroutines are multiplexed onto a small number of OS threads, rather than a 1:1 mapping.
• You can write massively concurrent servers without having to resort to evented programming.

EDIT: Answer to your question in the comments. There is no definitive answer. As others mentioned, it depends on what your code does. You may never end up using 100% CPU if you, for example, do I/O, which is slow. So no matter how many routines you start, I/O is slow. Contrarily, if your goroutine has a very tight loop doing just some computation then it's likely that 8 goroutines will consume your 8 CPUs completely.