大量内容的go io.copy函数中的一个错误始终一致

Question

This one works consistently.

    _, err = io.Copy(out, resp.Body)
    if err != nil {
        ErrLog.Fatal(err)
    }

This one gives pretty consistent off by one errors (the last byte of the downloaded content is left off, in my case a closing ] in a json response) for large-ish responses (MBs).

    if _, err := io.Copy(out, resp.Body); err != nil {
        ErrLog.Fatal(err)
    }

From the examples on the official golang blog, it looks like this should be valid syntax.

Edit: Some more details and context

This is the error I get with the 2nd version of the code (more compact error handling)

ERROR: 2015/08/05 08:09:31 pull.go:257: unexpected end of JSON input

From this code in another function

err = json.Unmarshal(dat, &all_data)
if err != nil {
    return err
}

I found the off by one issue by looking at the first 10 and last 10 characters of the file in each case. Here are the before and afters:

# Before (with error)
START:  [{"tags":[              END:    ersion":1}
START:  [{"_create              END:    "tags":[]}

# After
START:  [{"tags":[              END:    rsion":1}]
START:  [{"_create              END:    tags":[]}]

The files are 15-20 Mb json strings.

Answer 1

It turned out the issue was at least partially the result of a race condition.

I was not calling .Close() on the file out before exiting the function. After adding that I have not been any more issues.

Why exactly this was causing just the last byte of the file to get dropped sometimes is a mystery to me.