douzuanze0486
douzuanze0486
2013-06-03 11:06
浏览 304
已采纳

如何在Golang中为http.Get()请求设置超时?

I'm making a URL fetcher in Go and have a list of URLs to fetch. I send http.Get() requests to each URL and obtain their response.

resp,fetch_err := http.Get(url)

How can I set a custom timeout for each Get request? (The default time is very long and that makes my fetcher really slow.) I want my fetcher to have a timeout of around 40-45 seconds after which it should return "request timed out" and move on to the next URL.

How can I achieve this?

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 邀请回答

7条回答 默认 最新

  • duanniu4106
    duanniu4106 2014-08-16 22:10
    已采纳

    Apparently in Go 1.3 http.Client has Timeout field

    timeout := time.Duration(5 * time.Second)
    client := http.Client{
        Timeout: timeout,
    }
    client.Get(url)
    

    That's done the trick for me.

    点赞 评论
  • duandaishi9268
    duandaishi9268 2013-06-03 11:36

    A quick and dirty way:

    http.DefaultTransport.(*http.Transport).ResponseHeaderTimeout = time.Second * 45
    

    This is mutating global state w/o any coordination. Yet it might be possibly okay for your url fetcher. Otherwise create a private instance of http.RoundTripper:

    var myTransport http.RoundTripper = &http.Transport{
            Proxy:                 http.ProxyFromEnvironment,
            ResponseHeaderTimeout: time.Second * 45,
    }
    
    var myClient = &http.Client{Transport: myTransport}
    
    resp, err := myClient.Get(url)
    ...
    

    Nothing above was tested.

    点赞 评论
  • douhan4243
    douhan4243 2013-06-03 11:40

    You need to set up your own Client with your own Transport which uses a custom Dial function which wraps around DialTimeout.

    Something like (completely untested) this:

    var timeout = time.Duration(2 * time.Second)
    
    func dialTimeout(network, addr string) (net.Conn, error) {
        return net.DialTimeout(network, addr, timeout)
    }
    
    func main() {
        transport := http.Transport{
            Dial: dialTimeout,
        }
    
        client := http.Client{
            Transport: &transport,
        }
    
        resp, err := client.Get("http://some.url")
    }
    
    点赞 评论
  • dongwo6477
    dongwo6477 2013-06-05 02:33

    To add to Volker's answer, if you would also like to set the read/write timeout in addition to the connect timeout you can do something like the following

    package httpclient
    
    import (
        "net"
        "net/http"
        "time"
    )
    
    func TimeoutDialer(cTimeout time.Duration, rwTimeout time.Duration) func(net, addr string) (c net.Conn, err error) {
        return func(netw, addr string) (net.Conn, error) {
            conn, err := net.DialTimeout(netw, addr, cTimeout)
            if err != nil {
                return nil, err
            }
            conn.SetDeadline(time.Now().Add(rwTimeout))
            return conn, nil
        }
    }
    
    func NewTimeoutClient(connectTimeout time.Duration, readWriteTimeout time.Duration) *http.Client {
    
        return &http.Client{
            Transport: &http.Transport{
                Dial: TimeoutDialer(connectTimeout, readWriteTimeout),
            },
        }
    }
    

    This code is tested and is working in production. The full gist with tests is available here https://gist.github.com/dmichael/5710968

    Be aware that you will need to create a new client for each request because of the conn.SetDeadline which references a point in the future from time.Now()

    点赞 评论
  • duanqianmou4661
    duanqianmou4661 2014-01-21 19:42

    You may use https://github.com/franela/goreq which handles timeouts in a fashion and simple way.

    点赞 评论
  • doupi7619
    doupi7619 2018-08-27 21:41

    If you want to do it per request, err handling ignored for brevity:

    ctx, cncl := context.WithTimeout(context.Background(), time.Second*3)
    defer cncl()
    
    req, _ := http.NewRequest(http.MethodGet, "https://google.com", nil)
    
    resp, _ := http.DefaultClient.Do(req.WithContext(ctx))
    
    点赞 评论
  • douyan1921
    douyan1921 2019-06-28 21:50
    timeout := time.Duration(5 * time.Second)
    transport := &http.Transport{Proxy: http.ProxyURL(proxyUrl), ResponseHeaderTimeout:timeout}
    

    This may help, but notice that ResponseHeaderTimeout starts only after the connection is established.

    点赞 评论

相关推荐