dongpalou5352
2017-09-26 10:01
浏览 259
已采纳

当使用go http client Do方法时,httpResponse和错误可以不同时为nil吗?

As visible in the official documentation as well as almost everywhere else online, the common pattern for handling http client errors is the following:

req, err := http.NewRequest("GET", "http://example.com", nil)
req.Header.Add("If-None-Match", `W/"wyzzy"`)
resp, err := client.Do(req)
if err != nil {
    // handle error
}
defer resp.Body.Close()

Reading the docs about the methods on the http client, I'm not able to understand if it is possible to receive both resp and err not nil, it seems like it's possible if we consider what is written in the Do method documentation:

The request Body, if non-nil, will be closed by the underlying Transport, even on errors.

On error, any Response can be ignored. A non-nil Response with a non-nil error only occurs when CheckRedirect fails, and even then the returned Response.Body is already closed.

Therefore my questions are the following:

  1. Is it possible to receive non-nil resp and err at the same time?
  2. If it is possible, and in my program I end up receiving a lot of errors, and I return or panic in the //handle error piece of the code shown above, thus never reaching the defer right after, is it possible that on the long run I end up having too many open bodies that the transport hasn't been able to close?

Context Addendum :

The problem I encounter is that, after a while, the program crashes with the following big print of goroutines on the stdErr, and following the trace I end up to a piece of code that seems normal, unless something bad happens if an error is encountered and there's a responseBody upon which no defer close will be made, but since the official docs say not to worry about it, the problem must be elsewhere...

Stderr logs
  Display timestamps
 net/http.(*persistConn).readLoop(0xc4244e0240)
    /usr/local/go/src/net/http/transport.go:1474 +0x196
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1117 +0xa35

goroutine 194055 [select, 2 minutes]:
net/http.(*persistConn).readLoop(0xc42421ec60)
    /usr/local/go/src/net/http/transport.go:1599 +0x9ec
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1117 +0xa35

goroutine 215944 [IO wait]:
net.runtime_pollWait(0x7f63736f0628, 0x72, 0x21b)
    /usr/local/go/src/runtime/netpoll.go:164 +0x59
net.(*pollDesc).wait(0xc420963568, 0x72, 0xa85c40, 0xa81350)
    /usr/local/go/src/net/fd_poll_runtime.go:75 +0x38
net.(*pollDesc).waitRead(0xc420963568, 0xc422289000, 0x1000)
    /usr/local/go/src/net/fd_poll_runtime.go:80 +0x34
net.(*netFD).Read(0xc420963500, 0xc422289000, 0x1000, 0x1000, 0x0, 0xa85c40, 0xa81350)
    /usr/local/go/src/net/fd_unix.go:250 +0x1b7
net.(*conn).Read(0xc42025bbb8, 0xc422289000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:181 +0x70
net/http.(*persistConn).Read(0xc4217af8c0, 0xc422289000, 0x1000, 0x1000, 0xc420b33400, 0x8b61a3, 0xc423dd6b80)
    /usr/local/go/src/net/http/transport.go:1316 +0x14b
bufio.(*Reader).fill(0xc421af0060)
    /usr/local/go/src/bufio/bufio.go:97 +0x117
bufio.(*Reader).Peek(0xc421af0060, 0x1, 0xc421af01e0, 0xc420536c80, 0xc420b17e00, 0x60, 0xc420536c60)
    /usr/local/go/src/bufio/bufio.go:129 +0x67
net/http.(*persistConn).readLoop(0xc4217af8c0)
    /usr/local/go/src/net/http/transport.go:1474 +0x196
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1117 +0xa35

goroutine 199855 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc421db1320)
    /usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1118 +0xa5a

goroutine 198707 [select, 1 minutes]:
net/http.(*persistConn).readLoop(0xc4236ea000)
    /usr/local/go/src/net/http/transport.go:1599 +0x9ec
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1117 +0xa35

goroutine 192577 [select, 2 minutes]:
net/http.(*persistConn).readLoop(0xc424a8fc20)
    /usr/local/go/src/net/http/transport.go:1599 +0x9ec
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1117 +0xa35

goroutine 213681 [select]:
net/http.(*persistConn).writeLoop(0xc42455e6c0)
    /usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1118 +0xa5a

goroutine 190222 [select, 2 minutes]:
net/http.(*persistConn).writeLoop(0xc4238617a0)
    /usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1118 +0xa5a

goroutine 208970 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc4234530e0)
    /usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1118 +0xa5a

goroutine 219699 [select]:
net/http.(*persistConn).writeLoop(0xc42155cb40)
    /usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1118 +0xa5a

goroutine 221664 [select]:
net/http.setRequestCancel.func3(0x0, 0xc4231af680, 0xc423657000, 0xc4214aeecc, 0xc422f926c0)
    /usr/local/go/src/net/http/client.go:320 +0x17c
created by net/http.setRequestCancel
    /usr/local/go/src/net/http/client.go:330 +0x287

goroutine 220550 [select]:
net/http.(*persistConn).readLoop(0xc4218f0c60)
    /usr/local/go/src/net/http/transport.go:1599 +0x9ec
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1117 +0xa35

goroutine 216169 [IO wait]:
net.runtime_pollWait(0x7f6373700fb0, 0x72, 0x221)
    /usr/local/go/src/runtime/netpoll.go:164 +0x59
net.(*pollDesc).wait(0xc4212fdb18, 0x72, 0xa85c40, 0xa81350)
    /usr/local/go/src/net/fd_poll_runtime.go:75 +0x38
net.(*pollDesc).waitRead(0xc4212fdb18, 0xc42393c000, 0x1000)
    /usr/local/go/src/net/fd_poll_runtime.go:80 +0x34
net.(*netFD).Read(0xc4212fdab0, 0xc42393c000, 0x1000, 0x1000, 0x0, 0xa85c40, 0xa81350)
    /usr/local/go/src/net/fd_unix.go:250 +0x1b7
net.(*conn).Read(0xc42300aa38, 0xc42393c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:181 +0x70
net/http.(*persistConn).Read(0xc422102c60, 0xc42393c000, 0x1000, 0x1000, 0xc423d30820, 0xc42002a800, 0xc424422b80)
    /usr/local/go/src/net/http/transport.go:1316 +0x14b
bufio.(*Reader).fill(0xc4220598c0)
    /usr/local/go/src/bufio/bufio.go:97 +0x117
bufio.(*Reader).Peek(0xc4220598c0, 0x1, 0xc422059a40, 0xc420535c80, 0xc422059700, 0x60, 0xc420535c60)
    /usr/local/go/src/bufio/bufio.go:129 +0x67
net/http.(*persistConn).readLoop(0xc422102c60)
    /usr/local/go/src/net/http/transport.go:1474 +0x196
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1117 +0xa35

goroutine 213706 [select]:
net/http.(*persistConn).writeLoop(0xc42321d200)
    /usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1118 +0xa5a

goroutine 214335 [select]:
net/http.(*persistConn).writeLoop(0xc4230d7560)
    /usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1118 +0xa5a

goroutine 221651 [IO wait]:
net.runtime_pollWait(0x7f63736eff38, 0x72, 0x254)
    /usr/local/go/src/runtime/netpoll.go:164 +0x59
net.(*pollDesc).wait(0xc422449b18, 0x72, 0xa85c40, 0xa81350)
    /usr/local/go/src/net/fd_poll_runtime.go:75 +0x38
net.(*pollDesc).waitRead(0xc422449b18, 0xc423d93000, 0x1000)
    /usr/local/go/src/net/fd_poll_runtime.go:80 +0x34
net.(*netFD).Read(0xc422449ab0, 0xc423d93000, 0x1000, 0x1000, 0x0, 0xa85c40, 0xa81350)
    /usr/local/go/src/net/fd_unix.go:250 +0x1b7
net.(*conn).Read(0xc421eb21b8, 0xc423d93000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:181 +0x70
net/http.(*persistConn).Read(0xc422e5b440, 0xc423d93000, 0x1000, 0x1000, 0x8cc888, 0x0, 0xc42347bb80)
    /usr/local/go/src/net/http/transport.go:1316 +0x14b
bufio.(*Reader).fill(0xc422f2c960)
    /usr/local/go/src/bufio/bufio.go:97 +0x117
bufio.(*Reader).Peek(0xc422f2c960, 0x1, 0xc422f2d6e0, 0xc420b8ec80, 0xc42180ff00, 0x60, 0xc420b8ec60)
    /usr/local/go/src/bufio/bufio.go:129 +0x67
net/http.(*persistConn).readLoop(0xc422e5b440)
    /usr/local/go/src/net/http/transport.go:1474 +0x196
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1117 +0xa35

goroutine 221670 [IO wait]:
net.runtime_pollWait(0x7f63736ff7b0, 0x72, 0x259)
    /usr/local/go/src/runtime/netpoll.go:164 +0x59
net.(*pollDesc).wait(0xc422449b88, 0x72, 0xa85c40, 0xa81350)
    /usr/local/go/src/net/fd_poll_runtime.go:75 +0x38
net.(*pollDesc).waitRead(0xc422449b88, 0xc423f28000, 0x1000)
    /usr/local/go/src/net/fd_poll_runtime.go:80 +0x34
net.(*netFD).Read(0xc422449b20, 0xc423f28000, 0x1000, 0x1000, 0x0, 0xa85c40, 0xa81350)
    /usr/local/go/src/net/fd_unix.go:250 +0x1b7
net.(*conn).Read(0xc423eac3c8, 0xc423f28000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/net.go:181 +0x70
net/http.(*persistConn).Read(0xc4235930e0, 0xc423f28000, 0x1000, 0x1000, 0xa82dc0, 0xc42000e0b0, 0xc423be7b80)
    /usr/local/go/src/net/http/transport.go:1316 +0x14b
bufio.(*Reader).fill(0xc422bb70e0)
    /usr/local/go/src/bufio/bufio.go:97 +0x117
bufio.(*Reader).Peek(0xc422bb70e0, 0x1, 0xc422bb7320, 0xc424191480, 0xc4237a4820, 0x0, 0xc422ac9138)
    /usr/local/go/src/bufio/bufio.go:129 +0x67
net/http.(*persistConn).readLoop(0xc4235930e0)
    /usr/local/go/src/net/http/transport.go:1474 +0x196
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1117 +0xa35

goroutine 213543 [select]:
net/http.(*persistConn).readLoop(0xc424795c20)
    /usr/local/go/src/net/http/transport.go:1599 +0x9ec
created by net/http.(*Transport).dialConn
    /usr/local/go/src/net/http/transport.go:1117 +0xa35

goroutine 220529 [select]:
net/http.(*persistConn).roundTrip(0xc4215fd680, 0xc4212c5c80, 0x0, 0x0, 0x0)
    /usr/local/go/src/net/http/transport.go:1898 +0x974
net/http.(*Transport).RoundTrip(0xc420847b30, 0xc421248800, 0xc420847b30, 0xed15b2cf4, 0x16cd81f7)
    /usr/local/go/src/net/http/transport.go:391 +0x74c
net/http.send(0xc421248700, 0xa84080, 0xc420847b30, 0xed15b2cf4, 0x16cd81f7, 0xab7060, 0x0, 0x8, 0xc4213a2a80, 0x40f258)
    /usr/local/go/src/net/http/client.go:249 +0x162
net/http.(*Client).send(0xc42075ed20, 0xc421248700, 0xed15b2cf4, 0x16cd81f7, 0xab7060, 0xc4213a2a80, 0x0, 0x1, 0x4084fe)
    /usr/local/go/src/net/http/client.go:173 +0x108
net/http.(*Client).Do(0xc42075ed20, 0xc421248700, 0xa, 0x8b359b, 0x5)
    /usr/local/go/src/net/http/client.go:595 +0x254
github.com/myCompanyName/gojsonrpc.(*Client).sendJsonRequest(0xc42078c6e0, 0xc4215fd320, 0xbb, 0x11a, 0x0, 0x0, 0x0, 0x0, 0x0)
    /home/ubuntu/go/src/github.com/myCompanyName/gojsonrpc/client.go:45 +0x2f2
github.com/myCompanyName/gojsonrpc.(*Client).Run(0xc42078c6e0, 0x8be6b2, 0x1d, 0x838780, 0xc420789ce0, 0x7fb2e0, 0xc4212c5b40, 0x1, 0xe)
    /home/ubuntu/go/src/github.com/myCompanyName/gojsonrpc/client.go:72 +0x219
github.com/myCompanyName/app/connectors/vendor.(*Connector).ReadStatus(0xc42078aa30, 0xc4212c5a60, 0xd, 0xc4212c5a6e, 0x7, 0xc4212c59c0, 0x1d, 0x0, 0x0, 0x0, ...)
    /home/ubuntu/go/src/github.com/myCompanyName/app/connectors/vendor/tickets.go:52 +0x386
github.com/myCompanyName/app/obscuredPkgName/connabslayer.(*ConnAbsLayer).ReadMetrics(0xc42075ac40, 0xc4212c5a60, 0x15, 0xc4212c59c0, 0x1d, 0xc42121eb30, 0xf)
    /home/ubuntu/go/src/github.com/myCompanyName/app/obscuredPkgName/connabslayer/connector_abstraction_layer.go:145 +0xdc
github.com/myCompanyName/app/obscuredPkgName/updater/plugins.(*UpdaterPlugin).SyncPluggedData(0xc42078c820, 0xc4208b9ef0, 0x28, 0xf)
    /home/ubuntu/go/src/github.com/myCompanyName/app/obscuredPkgName/updater/plugins/vendor_plugin.go:69 +0x1e1
github.com/myCompanyName/app/obscuredPkgName/updater.(*Updater).syncData.func1(0xc420763a00, 0xc4208b9ef0, 0x28, 0xc4201824f0, 0xf)
    /home/ubuntu/go/src/github.com/myCompanyName/app/obscuredPkgName/updater/updater.go:127 +0xf6
created by github.com/myCompanyName/app/obscuredPkgName/updater.(*Updater).syncData
    /home/ubuntu/go/src/github.com/myCompanyName/app/obscuredPkgName/updater/updater.go:131 +0x17b

// AND SO ON

Solution:

The problem was elsewhere, as visible in the comments below, but the question might be useful for somebody else asking the same.

  • 写回答
  • 好问题 提建议
  • 关注问题
  • 收藏
  • 邀请回答

1条回答 默认 最新

  • dongnan1989 2017-09-26 10:59
    已采纳

    Yes, they can both be non-nil in one situation, apparently. From the source, we see:

            if err != nil {
                // Special case for Go 1 compatibility: return both the response
                // and an error if the CheckRedirect function failed.
                // See https://golang.org/issue/3795
                // The resp.Body has already been closed.
                ue := uerr(err)
                ue.(*url.Error).URL = loc
                return resp, ue
            }
    

    Every other return is returning either a response and nil err, or a nil response and non-nil error.
    So for the questions above:

    1. Yes
    2. Can't happen, for the only case in which both a body and an error are non-nil, the body has already been closed
    已采纳该答案
    评论
    解决 无用
    打赏 举报