使用Go http客户端发生意外的EOF

Question

I am learning Go and came across this problem.

I am just downloading web page content using HTTP client:

package main

import (
    "fmt"
    "io/ioutil"
    "log"
    "net/http"
)

func main() {
    client := &http.Client{}

    req, err := http.NewRequest("GET", "https://mail.ru/", nil)
    req.Close = true

    response, err := client.Do(req)
    if err != nil {
        log.Fatal(err)
    }

    defer response.Body.Close()

    content, err := ioutil.ReadAll(response.Body)
    if err != nil {
        fmt.Println(err)
    }

    fmt.Println(string(content)[:100])
}

I get an unexpected EOF error when reading response body. At the same time content variable has full page content.

This error appear only when I downloading https://mail.ru/ content. With other URLs everything works fine - without any errors.

I used curl for downloading this page content - everything works as expected.

I am confused a bit - what's happening here?

Go v1.2, tried on Ubuntu and MacOS X

Answer 1

It looks like the that server (Apache 1.3, wow!) is serving up a truncated gzip response. If you explicitly request the identity encoding (preventing the Go transport from adding gzip itself), you won't get the ErrUnexpectedEOF:

req.Header.Add("Accept-Encoding", "identity")