Golang-尝试GET请求时来自S3的SignatureDoesNotMatch错误

Question

I am attempting to download a track from http://freemusicarchive.org. Generally speaking, you can download a file by appending /download to the track URL, which responds with a redirect to the asset on S3.

For example, try this link: http://freemusicarchive.org//music//Zola_Jesus//Live_at_WFMU_on_Scott_McDowells_Show_1709//Odessa/download

To see the redirect, put that link here: http://www.wheregoes.com/retracer.php

I am able to get the redirect location with code that looks like this:

req, err := http.NewRequest("GET", url, nil)
errHndlr(err)

transport := http.Transport{}
resp, err := transport.RoundTrip(req)
defer resp.Body.Close()
errHndlr(err)

redirect := resp.Header.Get("Location")

I have verified the redirect link works by printing it to the console and copy/pasting it into my browser, but when I call http.Get on the same url, I get a "SignatureDoesNotMatch" error from AWS.

If anyone can offer insight as to what is going wrong here, I would greatly appreciate it.

Answer 1

Figured it out. Go parses unicode back to plain-text when processing a url. I needed to use request.URL.Opaque.

More info here: https://stackoverflow.com/a/17322831/733860

The issue was unicode-related. There was a %2F in my command (not displayed in my original question) that Go was converting to / that should have been left as %2F (cURL was properly leaving it as %2F). Changing the %2F to %252F fixed the issue.

It also appears that when creating a new HTTP request Go will parse your unicode back to plain text, so if you have %3D in the URL you submit to the HTTP request initializer it will convert it to =. I thought an obvious solution would be to put %253D into the URL but apparently there is a bug in Go that will convert %3D to = but NOT %25 to %. I had to use the Opaque URL request (request.Url.Opaque) to get around this.