tr := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: tr}
response, err := client.Get(link)
if err != nil {
fmt.Println(err)
}
defer response.Body.Close()
//block forever at the next line
content, _ = ioutil.ReadAll(response.Body)
The above is my code to read content from a webpage which resides in a loop. I found sometimes the line ioutil.ReadAll(response.Body)
will block forever. This happens randomly, however, it almost always happens on this webpage: http://xkcd.com/55
. It's very interesting that when I do curl http://xkcd.com/55
, it returns nothing, however, wget http://xkcd.com/55
returns the whole webpage.