dtnat7146
dtnat7146
2019-01-29 11:34
浏览 172
已采纳

使用Golang的AWS S3并行下载

I am writing a function to download a large file (9GB) from AWS S3 bucket using aws-sdk for go. I need to optimize this and download the file quickly.

func DownloadFromS3Bucket(bucket, item, path string) {
    os.Setenv("AWS_ACCESS_KEY_ID", constants.AWS_ACCESS_KEY_ID)
    os.Setenv("AWS_SECRET_ACCESS_KEY", constants.AWS_SECRET_ACCESS_KEY)

    file, err := os.Create(filepath.Join(path, item))
    if err != nil {
        fmt.Printf("Error in downloading from file: %v 
", err)
        os.Exit(1)
    }

    defer file.Close()

    sess, _ := session.NewSession(&aws.Config{
        Region: aws.String(constants.AWS_REGION)},
    )

    downloader := s3manager.NewDownloader(sess)

    numBytes, err := downloader.Download(file,
        &s3.GetObjectInput{
            Bucket: aws.String(bucket),
            Key:    aws.String(item),
        })
    if err != nil {
        fmt.Printf("Error in downloading from file: %v 
", err)
        os.Exit(1)
    }

    fmt.Println("Download completed", file.Name(), numBytes, "bytes")
}

Can someone suggest a solution to extend this function.

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 邀请回答

1条回答 默认 最新

相关推荐