Golang-Go例程和频道有一些麻烦

Question

I'm kinda new to Golang and trying to develop a program that uploads images async to imgur. However I'm having some difficulties with my code.

So this is my task;

func uploadT(url string,c chan string, d chan string)  {

    var subtask string
    subtask=upload(url)

    var status string
    var url string

    if subtask!=""{
        status = "Success!"
        url =subtask

    } else {
        status = "Failed!"
        url =subtask
    }

    c<-url
    d<-status
}

And here is my POST request loop for async uploading;

c:=make(chan string, len(js.Urls))
d:=make(chan string, len(js.Urls))

wg:=sync.WaitGroup{}
for i := range js.Urls{
    wg.Add(1)
    go uploadTask(js.Urls[i],c,d)
    //Below commented out code is slowing down the routine therefore, I commented out.
    //Needs to be working as well, however, it can work if I put this on task as well. I think I'm kinda confused with this one as well
    //pol=append(pol,retro{Url:<-c,Status:<-d})
}
<-c
<-d
wg.Done()
FinishedTime := time.Now().UTC().Format(time.RFC3339)
qwe=append(qwe,outputURLs{
               jobID:jobID,
               retro:pol,
               CreateTime: CreateTime,
               FinishedTime: FinishedTime,
           })
fmt.Println(jobID)

So I think my channels and routine does not work. It does print out jobID before the upload tasks. And also uploads seems too slow for async uploading.

I know the code is kinda mess, sorry for that. Any help is highly appreciated! Thanks in advance!

Answer 1

Your code is kinda confusing. But if I understand correctly what you are trying to do, you are processing a list of requests and want to return the url and status of each request and time each request completed. And you want to process these in parallel.

You don't need to use WaitGroups at all. WaitGroups are good when you just want to run a bunch of tasks without bothering with the results, just want to know when everything is done. But if you are returning results, channels are sufficient.

Here is an example code that does what I think you are trying to do

package main

import (
    "time"
    "fmt"
)

type Result struct {
    URL      string
    Status   string
    Finished string
}

func task(url string, c chan string, d chan string) {
    time.Sleep(time.Second)
    c <- url
    d <- "Success"
}

func main() {
    var results []Result
    urls := []string{"url1", "url2", "url3", "url4", "url5"}
    c := make(chan string, len(urls))
    d := make(chan string, len(urls))
    for _, u := range urls {
        go task(u, c, d)
    }
    for i := 0; i < len(urls); i++ {
        res := Result{}
        res.URL = <-c
        res.Status = <-d
        res.Finished = time.Now().UTC().Format(time.RFC3339)
        results = append(results, res)
    }
    fmt.Println(results)
}

You can try it in the playground https://play.golang.org/p/N3oeA7MyZ8L

That said, this is a bit fragile. You are making channels the same size as your url list. This would work fine for a few urls, but if you have a list of a million urls you will make a rather large channel. You might want to fix the channel buffer size to some reasonable value and check whether or not channel is ready for processing before sending your request. This way you would avoid making a million requests all at once.