如何使用Go向urlscanio发出发布请求？

Question

What I'm doing is trying to submit a URL to scan to urlscan.io. I can do a search but have issues with submissions, particularly correctly sending the right headers/encoded data.

from their site on how to submit a url:

curl -X POST "https://urlscan.io/api/v1/scan/" \ -H "Content-Type: application/json" \ -H "API-Key: $apikey" \ -d "{\"url\": \"$url\", \"public\": \"on\"}"

This works to satisfy the Api key header requirement but

req.Header.Add("API-Key", authtoken)

This is my attempt that fails

data := make(url.Values)
data.Add("url", myurltoscan)

The URL property I have struggled with tremendously.

This is my error: "message": "Missing URL properties", "description": "The URL supplied was not OK, please specify it including the protocol, host and path (e.g. http://example.com/bar)", "status": 400

Answer 1

url.Value is a map[string][]string containing values used in query parameters or POST form. You would need it if you were trying to do something like:

curl -X GET https://urlscan.io/api/v1/scan?url=<urltoscan>

or

curl -X POST -F 'url=<urltoscan>' https://urlscan.io/api/v1/scan

See documentation https://golang.org/pkg/net/url/#Values.

To send a regular POST request with JSON data, you can encode the JSON as bytes and send with http.Post:

var payload = []byte(`{"url":"<your-url>","public":"on"}`)
req, err := http.NewRequest("POST", url, 
bytes.NewBuffer(payload))
req.Header.Set("API-Key", authtoken)
req.Header.Set("Content-Type", "application/json")

client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
        panic(err)
}
defer resp.Body.Close()