如何将S3数据加载到本地内存中，而不是保存到文件中

Question

This question already has an answer here:

• How to keep files in memory - transferring files between servers without storing them locally 1 answer

This is my current code:

import (
  "time"
  "reflect"
  "io/ioutil"
  "fmt"
  "os"
  "github.com/golang/protobuf/proto"
  "github.com/aws/aws-sdk-go/service/s3"
  "github.com/aws/aws-sdk-go/service/s3/s3manager"
  "github.com/aws/aws-sdk-go/aws"
  "github.com/aws/aws-sdk-go/aws/session"
  "github.com/aws/aws-sdk-go/aws/credentials"
)

f, err := os.Create(filename)
if err != nil {
  panic(fmt.Errorf("failed to create file %q, %v", filename, err))
}
_, err = db.Downloader.Download(f, &s3.GetObjectInput{
    Bucket: aws.String("bucketName"),
    Key:    aws.String("myKeyThatMightExist"),
})
if err != nil { // key doesn't exist
  os.Remove(filename)
  return nil
} else {
  file, err = ioutil.ReadFile(filename)
}

I need file to be a string that is what was pulled down from S3. The local file that I am storing is a cache for what is on S3.

This means that when I try to pull down data, I need to create a file for that data. Due to the database I'm working with, I don't actually know if the key I'm accessing even exists. I was wondering if either of these solutions would be possible: 1) Create some type of buffer object, to load the whole file into RAM, without actually saving it. 2) Check if the file exists on S3, without creating a local file. This would let me then pull down the file if it existed, while not creating an empty file if I didnt exist on S3.

Sorry, but I can't create a playground link, as go playground does not have the libraries I need for this example.

Thanks, Neil

</div>

Answer 1

buf := aws.NewWriteAtBuffer([]byte{})
downloader.Download(buf, &requestInput)

Now you can use buf.Bytes()

https://docs.aws.amazon.com/sdk-for-go/api/aws/#NewWriteAtBuffer