并发尝试失败

Question

I'm having difficulty getting go concurrency to work correctly. I'm working with data loaded from an XML Data Source. Once I load the data into memory, i loop through the XML elements and perform an operation. The code prior to the concurrency addition has been tested and functional, and I don't believe it has any influence on the concurrency addition. I have 2 failed attempts at concurrency implementations, both with different outputs. I used locking because i dont want to enter a race condition.

For this implementation, it never enters the goroutine.

    var mu sync.Mutex

    // length is 197K
    for i:=0;i<len(listings.Listings);i++{
      go func(){
         mu.Lock()

         // code execution (tested prior to adding concurrency and locking)

         mu.Unlock()     
      }()
    }

For this implementation using waitGroups, a runtime out of memory occurs

    var mu sync.Mutex
    var wg sync.WaitGroup

    // length is 197K
    for i:=0;i<len(listings.Listings);i++{
       wg.Add(1)
       go func(){
           mu.Lock()

           // code execution (tested prior to adding concurrency and locking and wait group)      

           wg.Done()
           mu.Unlock()   
       }()
    }
    wg.Wait()

I'm not really sure what's going on and could use some assistance.

Answer 1

1. You don't need Mutex here if you want to make it concurrent
2. 197K goroitines are a lot, try lower amount of goroutines. You can accomplish it by creating N goroutines, when each of them is listening to the same channel.

https://play.golang.org/p/s4e0YyHdyPq

package main

import (
    "fmt"
    "sync"
)

type Listing struct{}

func main() {
    var (
        wg          sync.WaitGroup
        concurrency = 100
    )

    c := make(chan Listing)

    wg.Add(concurrency)
    for i := 0; i < concurrency; i++ {
        go func(ci <-chan Listing) {
            for l := range ci {
                // code, l is a single Listing
                fmt.Printf("%v", l)
            }
            wg.Done()
        }(c)
    }

    // replace with your var
    listings := []Listing{Listing{}}
    for _, l := range listings {
        c <- l
    }

    close(c)
    wg.Wait()
}