在Go中以异步方式执行命令和ZIP文件夹

Question

I’ve a program which executes some command via cmd := exec.Command(name, args…) Now the program is doing this for 10(or more) different directories, I mean I run npm install for 10 different directories for given path. The idea is to wait that the execute process will end and than zip the entire folder (where the command npm install was run) The Problem here is that that when the wait block is executing the program stops and wait (like it should do of course…) but I want in to proceed to the next directory and don't wait to the wait (of each directory execution ) to finish

How it’s suggested to handle it in sufficient way ? I mean to wait the minimum between executing the modules and in addition when specific command is finished to run on specified dir no matter when the program is now in the process, zip it automatically

This function is called on loop 10 times for different directories

func (n Node) Build(path string) error {
    //e.g. Run npm install which build's nodejs project
    command := exec.Command("npm", "install")
    command.Dir = n.path

 //start the execution 
  if err := command.Start(); err != nil {
    log.Printf("Failed to start cmd: %v", err)
  }



// Here I waiting to command to finish to zip the folder
if err := command.Wait(); err != nil {
   // Zip folder
}



  }

the main function is calling to it like following

func main() {

//Here I have loop which 

for _, dir := range dirs {

    switch dir.name {
    case "Java":
        Build(&Java{path})
    case "Go":
        Build(&Golang{path,cgoOptions},) 
    case "Node":
        Build(&Node{path})
    }
}

similar to this post Using inheritance of builders in GO

Answer 1

You just need to start as many goroutines as you have folders to parallelize the execution of your program and then wait for all goroutines to finish using a wait group.

Example:

import "sync"

func (n Node) Build(wg sync.WaitGroup, path string) error {
    wg.Add(1)
    defer wg.Done()

    //e.g. Run npm install which build's nodejs project
    command := exec.Command("npm", "install")
    command.Dir = n.path

    //start the execution 
    if err := command.Start(); err != nil {
        log.Printf("Failed to start cmd: %v", err)
    }

    // Here I waiting to command to finish to zip the folder
    if err := command.Wait(); err != nil {
       // Zip folder
    } 
}

func main() {
    var wg sync.WaitGroup
    node := ...
    for _, path := range dirs {
        go node.Build(wg, path)
    }

    wg.Wait()
}

UPD: some explanations. We need a shared wait group object between the main process and all the goroutines. One can treat our goroutines as logical threads of execution. And the wait group as a thread-safe shared counter variable. Each goroutine in our case is a single logical job. That's why we Add() (i.e. increment) the wait group internal counter by 1 in the beginning of the thread and decrease it by 1 via Done() call before the exiting from the Build() function. And the Wait() method of the wait group just blocks the execution of the main process until the internal counter becomes 0 again.