如何在Go中将预签名的POST上传到AWS S3？

Question

I would like to do a pre-signed POST to upload files to an AWS S3 bucket - how would this be done in Go?

Please note that this is not the same as Pre-signed upload with PUT.

Answer 1

So in order to help others I will answer the question myself and provide some code to help others who might have the same problem.

Example web app for Google App Engine rendering a pre-signed POST form can be found here.

And a small library I created doing the pre-signed POST in Go.

In short, doing a presigned POST to a public-read Amazon S3 bucket you need to:

1. Configure the S3 bucket to only allow public download.

Example bucket policy that allow only public read.

{
    "Version": "2012-10-17",
    "Id": "akjsdhakshfjlashdf",
    "Statement": [
        {
            "Sid": "kjahsdkajhsdkjasda",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::BUCKETNAMEHERE/*"
        }
    ]
}

2. Create a policy for the HTTP POST that allows the upload.

AWS S3 docs

Example POST policy template with expiration to upload a specific key, into a specific bucket and allow public-read access.

{ "expiration": "%s",
    "conditions": [
        {"bucket": "%s"},
        ["starts-with", "$key", "%s"],
        {"acl": "public-read"},

        {"x-amz-credential": "%s"},
        {"x-amz-algorithm": "AWS4-HMAC-SHA256"},
        {"x-amz-date": "%s" }
    ]
}

3. Generate and sign the policy using the S3 bucket owner's credentials.

AWS docs

• Fill in the correct values for expiration, bucket, key, credentials and date.
• base64 encode the policy.
• HMAC-SHA256 the policy to get a signature.
• hex encode the signature.

4. Construct and POST the multipart form data

AWS S3 docs

Now either you would generate an HTML form and automatically get the correct multipart form data request like described in the above link.

I wanted to do this by hand in Go so here's how to do that.

Either way you need to provide all the parts that are specified in the POST policy you created in steps 2 and 3. You can also not have additional fields in the request except for the mandatory ones (not in the policy).

The order of the fields is also specified and all of them are multipart fields in the HTTP POST request.

func Upload(url string, fields Fields) error {
    var b bytes.Buffer
    w := multipart.NewWriter(&b)
    for _, f := range fields {
            fw, err := w.CreateFormField(f.Key)
            if err != nil {
                    return err
            }
            if _, err := fw.Write([]byte(f.Value)); err != nil {
                    return err
            }
    }
    w.Close()

    req, err := http.NewRequest("POST", url, &b)
    if err != nil {
            return err
    }
    req.Header.Set("Content-Type", w.FormDataContentType())

    client := &http.Client{}
    res, err := client.Do(req)
    if err != nil {
            return err
    }
    if res.StatusCode != http.StatusOK {
            err = fmt.Errorf("bad status: %s", res.Status)
    }
    return nil
}