Golang IO读取文件无效的内存地址

Question

I'm having some troubles trying to read a GZIP file from AWS S3. I have only a simple code and I'm getting the error panic: runtime error: invalid memory address or nil pointer dereference. But it still hard to find the problem.

The error message is saying that is the gzip.NewReader.... Why golang is reporting it? How could i solve it?

goroutine 1 [running]:
github.com/hensg/pushego/aws.GetGZIPProductsDump(0x7ffead44098f, 0xc, 0x7ffead44099f, 0x2f, 0x0, 0xc420190580, 0x0, 0x0)
        /home/henrique/go/src/github.com/hensg/pushego/aws/s3.go:47 +0x2e7
main.main()
        /home/henrique/go/src/github.com/hensg/pushego/pushego.go:28 +0x249

Main code (pushego.go)

package main

import (
    "bufio"
    "flag"
    "log"
    "os"
    "time"

    "github.com/hensg/pushego/aws"
)

func init() {
    log.SetOutput(os.Stdout)
}

func main() {
    log.Println("Starting...")

    var bucket, key string
    var timeout time.Duration

    flag.StringVar(&bucket, "b", "", "Bucket name")
    flag.StringVar(&key, "k", "", "Object key name")
    flag.DurationVar(&timeout, "d", 0, "Download timeout")
    flag.Parse()

    gzipReader, err := aws.GetGZIPDump(bucket, key, timeout) // line 28
    if err != nil {
        log.Fatal("Failed to create GZIP reader")
    }
   defer gzipReader.Close()

    scanner := bufio.NewScanner(gzipReader)
    for scanner.Scan() {
        log.Printf("Read: %s
", scanner.Text())
    }

    log.Printf("Successfully download file from %s/%s
", bucket, key)
}

Aws code (aws/s3.go)

package aws

import (
    "compress/gzip"
    "context"
    "log"
    "time"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/awserr"
    "github.com/aws/aws-sdk-go/aws/request"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/s3"
)

func GetGZIPProductsDump(bucket, key string, timeout time.Duration) (*gzip.Reader, error) {
    sess := session.Must(session.NewSession())
    svc := s3.New(sess)

    // Create a context with a timeout that will abort the download if it takes
    // more than the passed in timeout.
    ctx := context.Background()
    var cancelFn func()
    if timeout > 0 {
        ctx, cancelFn = context.WithTimeout(ctx, timeout)
    }
    // Ensure the context is canceled to prevent leaking.
    // See context package for more information, https://golang.org/pkg/context/
    defer cancelFn()

    // Uploads the object to S3. The Context will interrupt the request if the
    // timeout expires.
    resp, err := svc.GetObjectWithContext(ctx, &s3.GetObjectInput{
        Bucket: aws.String(bucket),
        Key:    aws.String(key),
    })
    if err != nil {
        if aerr, ok := err.(awserr.Error); ok && aerr.Code() == request.CanceledErrorCode {
            // If the SDK can determine the request or retry delay was canceled
            // by a context the CanceledErrorCode error code will be returned.
            log.Fatal("Download canceled due to timeout, %v
", err)
        } else {
            log.Fatal("Failed to download object, %v
", err)
        }
    }

    return gzip.NewReader(resp.Body) // line 47
}

Answer 1

The problem is related with the defer cancelFn(), currently that func is not set if the timeout is 0, this provoke a panic with a null pointer, so, you should move the defer inside of the previous if statement because it's the only point when is required to be used. The code of aws/s3.go have to be as the following.

...
// Create a context with a timeout that will abort the download if it takes
// more than the passed in timeout.
ctx := context.Background()
var cancelFn func()
if timeout > 0 {
    ctx, cancelFn = context.WithTimeout(ctx, timeout)
    // Ensure the context is canceled to prevent leaking.
    // See context package for more information, https://golang.org/pkg/context/
    defer cancelFn()
}
...