用于Google云存储的Golang SDK：分段下载

Question

Using GoLang SDK for google cloud storage.

Cannot find how to download files in chunks.

Answer 1

The Google Cloud documentation says to download an object from Cloud Storage, you should use the following:

rc, err := client.Bucket(bucket).Object(object).NewReader(ctx)
if err != nil {
        return nil, err
}
defer rc.Close()

data, err := ioutil.ReadAll(rc)
if err != nil {
        return nil, err
}
return data, nil

Source: https://cloud.google.com/storage/docs/downloading-objects#storage-download-object-code_sample

Given their SDK returns an io.Reader, you don't need to worry about the underlying method being used to be able to reference the download in chunks (although, quickly looking through their source, it just implements http.NewRequest, which does what you want, using the same logic).

The reason it doesn't seem to "chunked" from their example is because of the usage of ioutil.ReadAll, which although great for simple use cases, extracts all of the Readers data into memory (meaning it also has to wait for the data to become available).

For a better understanding of how to deal with a Reader in steps, I recommend taking a look at https://tour.golang.org/methods/21 for a tour of io.Reader and how you can use it more efficiently.