如何在Golang中处理HTTP超时错误和访问状态代码

Question

I have some code (see below) written in Go which is supposed to "fan-out" HTTP requests, and collate/aggregate the details back.

I'm new to golang and so expect me to be a nOOb and my knowledge to be limited

The output of the program is currently something like:

{
    "Status":"success",
    "Components":[
        {"Id":"foo","Status":200,"Body":"..."},
        {"Id":"bar","Status":200,"Body":"..."}, 
        {"Id":"baz","Status":404,"Body":"..."}, 
        ...
    ]
}

There is a local server running that is purposely slow (sleeps for 5 seconds and then returns a response). But I have other sites listed (see code below) that sometime trigger an error as well (if they error, then that's fine).

The problem I have at the moment is how best to handle these errors, and specifically the "timeout" related errors; in that I'm not sure how to recognise if a failure is a timeout or some other error?

At the moment I get a blanket error back all the time:

Get http://localhost:8080/pugs: read tcp 127.0.0.1:8080: use of closed network connection

Where http://localhost:8080/pugs will generally be the url that failed (hopefully by timeout!). But as you can see from the code (below), I'm not sure how to determine the error code is related to a timeout nor how to access the status code of the response (I'm currently just blanket setting it to 404 but obviously that's not right - if the server was to error I'd expect something like a 500 status code and obviously I'd like to reflect that in the aggregated response I send back).

The full code can be seen below. Any help appreciated.

    package main

    import (
            "encoding/json"
            "fmt"
            "io/ioutil"
            "net/http"
            "sync"
            "time"
    )

    type Component struct {
            Id  string `json:"id"`
            Url string `json:"url"`
    }

    type ComponentsList struct {
            Components []Component `json:"components"`
    }

    type ComponentResponse struct {
            Id     string
            Status int
            Body   string
    }

    type Result struct {
            Status     string
            Components []ComponentResponse
    }

    var overallStatus string = "success"

    func main() {
            var cr []ComponentResponse
            var c ComponentsList

            b := []byte(`{"components":[{"id":"local","url":"http://localhost:8080/pugs"},{"id":"google","url":"http://google.com/"},{"id":"integralist","url":"http://integralist.co.uk/"},{"id":"sloooow","url":"http://stevesouders.com/cuzillion/?c0=hj1hfff30_5_f&t=1439194716962"}]}`)

            json.Unmarshal(b, &c)

            var wg sync.WaitGroup

            timeout := time.Duration(1 * time.Second)
            client := http.Client{
                    Timeout: timeout,
            }

            for i, v := range c.Components {
                    wg.Add(1)

                    go func(i int, v Component) {
                            defer wg.Done()

                            resp, err := client.Get(v.Url)

                            if err != nil {
                                fmt.Printf("Problem getting the response: %s
", err)

                                cr = append(cr, ComponentResponse{
                                    v.Id,
                                    404,
                                    err.Error(),
                                })
                            } else {
                                    defer resp.Body.Close()
                                    contents, err := ioutil.ReadAll(resp.Body)
                                    if err != nil {
                                            fmt.Printf("Problem reading the body: %s
", err)
                                    }

                                    cr = append(cr, ComponentResponse{
                                            v.Id,
                                            resp.StatusCode,
                                            string(contents),
                                    })
                            }
                    }(i, v)
            }
            wg.Wait()

            j, err := json.Marshal(Result{overallStatus, cr})
            if err != nil {
                    fmt.Printf("Problem converting to JSON: %s
", err)
                    return
            }

            fmt.Println(string(j))
    }

Answer 1

The Go 1.5 release solved this issue by being more specific about the type of error it has handled.

So if you see this example https://github.com/Integralist/Go-Requester/blob/master/requester.go#L38 you'll see that I'm able to apply a regex pattern to the error message to decipher if the error was indeed a timeout or not

status := checkError(err.Error())

func checkError(msg string) int {
    timeout, _ := regexp.MatchString("Timeout", msg)

    if timeout {
        return 408
    }

  return 500
}