duanji1056
2016-02-06 19:48
浏览 103
已采纳

使用golang的http.ResponseWriter的AWS S3大文件反向代理

I have a request handler named Download which I want to access a large file from Amazon S3 and push it to the user's browser. My goals are:

  • To record some request information before granting the user access to the file
  • To not buffer the file into memory too much. Files may become too large.

Here is what I've explored so far:

func Download(w http.ResponseWriter, r *http.Request) {

    sess := session.New(&aws.Config{
        Region:             aws.String("eu-west-1"),
        Endpoint:           aws.String("s3-eu-west-1.amazonaws.com"),
        S3ForcePathStyle:   aws.Bool(true),
        Credentials:        cred,
    })

    downloader := s3manager.NewDownloader(sess)
    // I can't write directly into the ResponseWriter. It doesn't implement WriteAt. 
    // Besides, it doesn't seem like the right thing to do.
    _, err := downloader.Download(w, &s3.GetObjectInput{
        Bucket: aws.String(BUCKET),
        Key: aws.String(filename),
    })
    if err != nil {
        log.Error(4, err.Error())
        return
    }

}

I'm wondering if there isn't a better approach (given the goals I'm trying to achieve).

Any suggestions are welcome. Thank you in advance :-)

  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 邀请回答

2条回答 默认 最新

  • doumi1884 2016-02-07 00:54
    已采纳

    If the file is potentially large, you don't want it to go trough your own server. The best approach (in my opinion) is to have the user download it directly from S3.

    You can do this by generating a presigned url:

    func Download(w http.ResponseWriter, r *http.Request) {
    
        ...
    
        sess := session.New(&aws.Config{
            Region:             aws.String("eu-west-1"),
            Endpoint:           aws.String("s3-eu-west-1.amazonaws.com"),
            S3ForcePathStyle:   aws.Bool(true),
            Credentials:        cred,
        })
    
        s3svc := s3.New(sess)
        req, _ := s3svc.GetObjectRequest(&s3.GetObjectInput{
            Bucket: aws.String(BUCKET),
            Key: aws.String(filename),
        })
    
        url, err := req.Presign(5 * time.Minute)
        if err != nil {
            //handle error
        }
    
        http.Redirect(w, r, url, http.StatusTemporaryRedirect)
    }
    

    The presigned url is only valid for a limited time (5 minutes in this example, adjust to your needs) and takes the user directly to S3. No need to worry about downloads anymore!

    点赞 评论
  • douji1877 2019-04-24 13:30

    If you do want to stream the file through your service (rather than download directly as recommended in the accepted answer) -

    import (
        ...
    
        "github.com/aws/aws-sdk-go/aws"
        "github.com/aws/aws-sdk-go/service/s3"
    )
    
    func StreamDownloadHandler(w http.ResponseWriter, r *http.Request) {
    
        sess, awsSessErr := session.NewSession(&aws.Config{
            Region:      aws.String("eu-west-1"),
            Credentials: credentials.NewStaticCredentials("my-aws-id", "my-aws-secret", ""),
        })
        if awsSessErr != nil {
            http.Error(w, fmt.Sprintf("Error creating aws session %s", awsSessErr.Error()), http.StatusInternalServerError)
            return
        }
    
        result, err := s3.New(sess).GetObject(&s3.GetObjectInput{
            Bucket: aws.String("my-bucket"),
            Key:    aws.String("my-file-id"),
        })
        if err != nil {
            http.Error(w, fmt.Sprintf("Error getting file from s3 %s", err.Error()), http.StatusInternalServerError)
            return
        }
    
        w.Header().Set("Content-Disposition", fmt.Sprintf("attachment; filename=\"%s\"", "my-file.csv"))
        w.Header().Set("Cache-Control", "no-store")
    
        bytesWritten, copyErr := io.Copy(w, result.Body)
        if copyErr != nil {
            http.Error(w, fmt.Sprintf("Error copying file to the http response %s", copyErr.Error()), http.StatusInternalServerError)
            return
        }
        log.Printf("Download of \"%s\" complete. Wrote %s bytes", "my-file.csv", strconv.FormatInt(bytesWritten, 10))
    }
    
    点赞 评论

相关推荐 更多相似问题