As visible in the official documentation as well as almost everywhere else online, the common pattern for handling http client errors is the following:
req, err := http.NewRequest("GET", "http://example.com", nil)
req.Header.Add("If-None-Match", `W/"wyzzy"`)
resp, err := client.Do(req)
if err != nil {
// handle error
}
defer resp.Body.Close()
Reading the docs about the methods on the http client, I'm not able to understand if it is possible to receive both resp
and err
not nil, it seems like it's possible if we consider what is written in the Do method documentation:
The request Body, if non-nil, will be closed by the underlying Transport, even on errors.
On error, any Response can be ignored. A non-nil Response with a non-nil error only occurs when CheckRedirect fails, and even then the returned Response.Body is already closed.
Therefore my questions are the following:
- Is it possible to receive non-nil
resp
anderr
at the same time? - If it is possible, and in my program I end up receiving a lot of errors, and I
return
orpanic
in the//handle error
piece of the code shown above, thus never reaching the defer right after, is it possible that on the long run I end up having too many open bodies that the transport hasn't been able to close?
Context Addendum :
The problem I encounter is that, after a while, the program crashes with the following big print of goroutines on the stdErr, and following the trace I end up to a piece of code that seems normal, unless something bad happens if an error is encountered and there's a responseBody upon which no defer close will be made, but since the official docs say not to worry about it, the problem must be elsewhere...
Stderr logs
Display timestamps
net/http.(*persistConn).readLoop(0xc4244e0240)
/usr/local/go/src/net/http/transport.go:1474 +0x196
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1117 +0xa35
goroutine 194055 [select, 2 minutes]:
net/http.(*persistConn).readLoop(0xc42421ec60)
/usr/local/go/src/net/http/transport.go:1599 +0x9ec
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1117 +0xa35
goroutine 215944 [IO wait]:
net.runtime_pollWait(0x7f63736f0628, 0x72, 0x21b)
/usr/local/go/src/runtime/netpoll.go:164 +0x59
net.(*pollDesc).wait(0xc420963568, 0x72, 0xa85c40, 0xa81350)
/usr/local/go/src/net/fd_poll_runtime.go:75 +0x38
net.(*pollDesc).waitRead(0xc420963568, 0xc422289000, 0x1000)
/usr/local/go/src/net/fd_poll_runtime.go:80 +0x34
net.(*netFD).Read(0xc420963500, 0xc422289000, 0x1000, 0x1000, 0x0, 0xa85c40, 0xa81350)
/usr/local/go/src/net/fd_unix.go:250 +0x1b7
net.(*conn).Read(0xc42025bbb8, 0xc422289000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:181 +0x70
net/http.(*persistConn).Read(0xc4217af8c0, 0xc422289000, 0x1000, 0x1000, 0xc420b33400, 0x8b61a3, 0xc423dd6b80)
/usr/local/go/src/net/http/transport.go:1316 +0x14b
bufio.(*Reader).fill(0xc421af0060)
/usr/local/go/src/bufio/bufio.go:97 +0x117
bufio.(*Reader).Peek(0xc421af0060, 0x1, 0xc421af01e0, 0xc420536c80, 0xc420b17e00, 0x60, 0xc420536c60)
/usr/local/go/src/bufio/bufio.go:129 +0x67
net/http.(*persistConn).readLoop(0xc4217af8c0)
/usr/local/go/src/net/http/transport.go:1474 +0x196
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1117 +0xa35
goroutine 199855 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc421db1320)
/usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1118 +0xa5a
goroutine 198707 [select, 1 minutes]:
net/http.(*persistConn).readLoop(0xc4236ea000)
/usr/local/go/src/net/http/transport.go:1599 +0x9ec
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1117 +0xa35
goroutine 192577 [select, 2 minutes]:
net/http.(*persistConn).readLoop(0xc424a8fc20)
/usr/local/go/src/net/http/transport.go:1599 +0x9ec
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1117 +0xa35
goroutine 213681 [select]:
net/http.(*persistConn).writeLoop(0xc42455e6c0)
/usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1118 +0xa5a
goroutine 190222 [select, 2 minutes]:
net/http.(*persistConn).writeLoop(0xc4238617a0)
/usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1118 +0xa5a
goroutine 208970 [select, 1 minutes]:
net/http.(*persistConn).writeLoop(0xc4234530e0)
/usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1118 +0xa5a
goroutine 219699 [select]:
net/http.(*persistConn).writeLoop(0xc42155cb40)
/usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1118 +0xa5a
goroutine 221664 [select]:
net/http.setRequestCancel.func3(0x0, 0xc4231af680, 0xc423657000, 0xc4214aeecc, 0xc422f926c0)
/usr/local/go/src/net/http/client.go:320 +0x17c
created by net/http.setRequestCancel
/usr/local/go/src/net/http/client.go:330 +0x287
goroutine 220550 [select]:
net/http.(*persistConn).readLoop(0xc4218f0c60)
/usr/local/go/src/net/http/transport.go:1599 +0x9ec
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1117 +0xa35
goroutine 216169 [IO wait]:
net.runtime_pollWait(0x7f6373700fb0, 0x72, 0x221)
/usr/local/go/src/runtime/netpoll.go:164 +0x59
net.(*pollDesc).wait(0xc4212fdb18, 0x72, 0xa85c40, 0xa81350)
/usr/local/go/src/net/fd_poll_runtime.go:75 +0x38
net.(*pollDesc).waitRead(0xc4212fdb18, 0xc42393c000, 0x1000)
/usr/local/go/src/net/fd_poll_runtime.go:80 +0x34
net.(*netFD).Read(0xc4212fdab0, 0xc42393c000, 0x1000, 0x1000, 0x0, 0xa85c40, 0xa81350)
/usr/local/go/src/net/fd_unix.go:250 +0x1b7
net.(*conn).Read(0xc42300aa38, 0xc42393c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:181 +0x70
net/http.(*persistConn).Read(0xc422102c60, 0xc42393c000, 0x1000, 0x1000, 0xc423d30820, 0xc42002a800, 0xc424422b80)
/usr/local/go/src/net/http/transport.go:1316 +0x14b
bufio.(*Reader).fill(0xc4220598c0)
/usr/local/go/src/bufio/bufio.go:97 +0x117
bufio.(*Reader).Peek(0xc4220598c0, 0x1, 0xc422059a40, 0xc420535c80, 0xc422059700, 0x60, 0xc420535c60)
/usr/local/go/src/bufio/bufio.go:129 +0x67
net/http.(*persistConn).readLoop(0xc422102c60)
/usr/local/go/src/net/http/transport.go:1474 +0x196
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1117 +0xa35
goroutine 213706 [select]:
net/http.(*persistConn).writeLoop(0xc42321d200)
/usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1118 +0xa5a
goroutine 214335 [select]:
net/http.(*persistConn).writeLoop(0xc4230d7560)
/usr/local/go/src/net/http/transport.go:1704 +0x43a
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1118 +0xa5a
goroutine 221651 [IO wait]:
net.runtime_pollWait(0x7f63736eff38, 0x72, 0x254)
/usr/local/go/src/runtime/netpoll.go:164 +0x59
net.(*pollDesc).wait(0xc422449b18, 0x72, 0xa85c40, 0xa81350)
/usr/local/go/src/net/fd_poll_runtime.go:75 +0x38
net.(*pollDesc).waitRead(0xc422449b18, 0xc423d93000, 0x1000)
/usr/local/go/src/net/fd_poll_runtime.go:80 +0x34
net.(*netFD).Read(0xc422449ab0, 0xc423d93000, 0x1000, 0x1000, 0x0, 0xa85c40, 0xa81350)
/usr/local/go/src/net/fd_unix.go:250 +0x1b7
net.(*conn).Read(0xc421eb21b8, 0xc423d93000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:181 +0x70
net/http.(*persistConn).Read(0xc422e5b440, 0xc423d93000, 0x1000, 0x1000, 0x8cc888, 0x0, 0xc42347bb80)
/usr/local/go/src/net/http/transport.go:1316 +0x14b
bufio.(*Reader).fill(0xc422f2c960)
/usr/local/go/src/bufio/bufio.go:97 +0x117
bufio.(*Reader).Peek(0xc422f2c960, 0x1, 0xc422f2d6e0, 0xc420b8ec80, 0xc42180ff00, 0x60, 0xc420b8ec60)
/usr/local/go/src/bufio/bufio.go:129 +0x67
net/http.(*persistConn).readLoop(0xc422e5b440)
/usr/local/go/src/net/http/transport.go:1474 +0x196
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1117 +0xa35
goroutine 221670 [IO wait]:
net.runtime_pollWait(0x7f63736ff7b0, 0x72, 0x259)
/usr/local/go/src/runtime/netpoll.go:164 +0x59
net.(*pollDesc).wait(0xc422449b88, 0x72, 0xa85c40, 0xa81350)
/usr/local/go/src/net/fd_poll_runtime.go:75 +0x38
net.(*pollDesc).waitRead(0xc422449b88, 0xc423f28000, 0x1000)
/usr/local/go/src/net/fd_poll_runtime.go:80 +0x34
net.(*netFD).Read(0xc422449b20, 0xc423f28000, 0x1000, 0x1000, 0x0, 0xa85c40, 0xa81350)
/usr/local/go/src/net/fd_unix.go:250 +0x1b7
net.(*conn).Read(0xc423eac3c8, 0xc423f28000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:181 +0x70
net/http.(*persistConn).Read(0xc4235930e0, 0xc423f28000, 0x1000, 0x1000, 0xa82dc0, 0xc42000e0b0, 0xc423be7b80)
/usr/local/go/src/net/http/transport.go:1316 +0x14b
bufio.(*Reader).fill(0xc422bb70e0)
/usr/local/go/src/bufio/bufio.go:97 +0x117
bufio.(*Reader).Peek(0xc422bb70e0, 0x1, 0xc422bb7320, 0xc424191480, 0xc4237a4820, 0x0, 0xc422ac9138)
/usr/local/go/src/bufio/bufio.go:129 +0x67
net/http.(*persistConn).readLoop(0xc4235930e0)
/usr/local/go/src/net/http/transport.go:1474 +0x196
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1117 +0xa35
goroutine 213543 [select]:
net/http.(*persistConn).readLoop(0xc424795c20)
/usr/local/go/src/net/http/transport.go:1599 +0x9ec
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1117 +0xa35
goroutine 220529 [select]:
net/http.(*persistConn).roundTrip(0xc4215fd680, 0xc4212c5c80, 0x0, 0x0, 0x0)
/usr/local/go/src/net/http/transport.go:1898 +0x974
net/http.(*Transport).RoundTrip(0xc420847b30, 0xc421248800, 0xc420847b30, 0xed15b2cf4, 0x16cd81f7)
/usr/local/go/src/net/http/transport.go:391 +0x74c
net/http.send(0xc421248700, 0xa84080, 0xc420847b30, 0xed15b2cf4, 0x16cd81f7, 0xab7060, 0x0, 0x8, 0xc4213a2a80, 0x40f258)
/usr/local/go/src/net/http/client.go:249 +0x162
net/http.(*Client).send(0xc42075ed20, 0xc421248700, 0xed15b2cf4, 0x16cd81f7, 0xab7060, 0xc4213a2a80, 0x0, 0x1, 0x4084fe)
/usr/local/go/src/net/http/client.go:173 +0x108
net/http.(*Client).Do(0xc42075ed20, 0xc421248700, 0xa, 0x8b359b, 0x5)
/usr/local/go/src/net/http/client.go:595 +0x254
github.com/myCompanyName/gojsonrpc.(*Client).sendJsonRequest(0xc42078c6e0, 0xc4215fd320, 0xbb, 0x11a, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/ubuntu/go/src/github.com/myCompanyName/gojsonrpc/client.go:45 +0x2f2
github.com/myCompanyName/gojsonrpc.(*Client).Run(0xc42078c6e0, 0x8be6b2, 0x1d, 0x838780, 0xc420789ce0, 0x7fb2e0, 0xc4212c5b40, 0x1, 0xe)
/home/ubuntu/go/src/github.com/myCompanyName/gojsonrpc/client.go:72 +0x219
github.com/myCompanyName/app/connectors/vendor.(*Connector).ReadStatus(0xc42078aa30, 0xc4212c5a60, 0xd, 0xc4212c5a6e, 0x7, 0xc4212c59c0, 0x1d, 0x0, 0x0, 0x0, ...)
/home/ubuntu/go/src/github.com/myCompanyName/app/connectors/vendor/tickets.go:52 +0x386
github.com/myCompanyName/app/obscuredPkgName/connabslayer.(*ConnAbsLayer).ReadMetrics(0xc42075ac40, 0xc4212c5a60, 0x15, 0xc4212c59c0, 0x1d, 0xc42121eb30, 0xf)
/home/ubuntu/go/src/github.com/myCompanyName/app/obscuredPkgName/connabslayer/connector_abstraction_layer.go:145 +0xdc
github.com/myCompanyName/app/obscuredPkgName/updater/plugins.(*UpdaterPlugin).SyncPluggedData(0xc42078c820, 0xc4208b9ef0, 0x28, 0xf)
/home/ubuntu/go/src/github.com/myCompanyName/app/obscuredPkgName/updater/plugins/vendor_plugin.go:69 +0x1e1
github.com/myCompanyName/app/obscuredPkgName/updater.(*Updater).syncData.func1(0xc420763a00, 0xc4208b9ef0, 0x28, 0xc4201824f0, 0xf)
/home/ubuntu/go/src/github.com/myCompanyName/app/obscuredPkgName/updater/updater.go:127 +0xf6
created by github.com/myCompanyName/app/obscuredPkgName/updater.(*Updater).syncData
/home/ubuntu/go/src/github.com/myCompanyName/app/obscuredPkgName/updater/updater.go:131 +0x17b
// AND SO ON
Solution:
The problem was elsewhere, as visible in the comments below, but the question might be useful for somebody else asking the same.