使用UrlFetchApp的POST请求返回404，其中等效curl成功

Question

I have a lengthy curl request that I'd like to emulate using UrlFetchApp:

curl 'https://example.com/index.php' -H 'Cookie: "XXXXXXXXXX"' -H 'Origin: https://example.com' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36' -H 'Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryK0ckVcd9Rae277Ae' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Cache-Control: max-age=0' -H 'Referer: https://example.com/index.php' -H 'Connection: keep-alive' --data-binary $'------WebKitFormBoundaryK0ckVcd9Rae277Ae
Content-Disposition: form-data; name="user"

staff
------WebKitFormBoundaryK0ckVcd9Rae277Ae
Content-Disposition: form-data; name="name"

John Smith
------WebKitFormBoundaryK0ckVcd9Rae277Ae
Content-Disposition: form-data; name="staffid"

e00000
------WebKitFormBoundaryK0ckVcd9Rae277Ae
Content-Disposition: form-data; name="preferred"

Email
------WebKitFormBoundaryK0ckVcd9Rae277Ae
Content-Disposition: form-data; name="phone"

9999 9999
------WebKitFormBoundaryK0ckVcd9Rae277Ae
Content-Disposition: form-data; name="altemail"

[e-mail]
------WebKitFormBoundaryK0ckVcd9Rae277Ae
Content-Disposition: form-data; name="subject"

other
------WebKitFormBoundaryK0ckVcd9Rae277Ae
Content-Disposition: form-data; name="desc"

TEST ONLY.
------WebKitFormBoundaryK0ckVcd9Rae277Ae
Content-Disposition: form-data; name="userfield"


------WebKitFormBoundaryK0ckVcd9Rae277Ae
Content-Disposition: form-data; name="timezone"

Mon Mar 23 2015 10:28:33 GMT+1100 (AEDT)
------WebKitFormBoundaryK0ckVcd9Rae277Ae
Content-Disposition: form-data; name="browserstring"

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36
------WebKitFormBoundaryK0ckVcd9Rae277Ae
Content-Disposition: form-data; name="files[]"; filename=""
Content-Type: application/octet-stream


------WebKitFormBoundaryK0ckVcd9Rae277Ae--
' --compressed

I'm using the following UrlFetchApp code, which simply creates a payload and a header object, and encodes the payload as a binary blob:

function sendHttpPost(mailBody) {

   var reqPayload = '------WebKitFormBoundaryTIE5gBodnUrDhzC9
Content-Disposition: form-data; name="rmituser"

staff
' + 
     '------WebKitFormBoundaryTIE5gBodnUrDhzC9
Content-Disposition: form-data; name="name"

John Smith
' + 
     '------WebKitFormBoundaryTIE5gBodnUrDhzC9
Content-Disposition: form-data; name="staffid"

E00000
' + 
     '------WebKitFormBoundaryTIE5gBodnUrDhzC9
Content-Disposition: form-data; name="phone"

9999 9999
' + 
     '------WebKitFormBoundaryTIE5gBodnUrDhzC9
Content-Disposition: form-data; name="altemail"


' + 
     '------WebKitFormBoundaryTIE5gBodnUrDhzC9
Content-Disposition: form-data; name="subject"

other
' + 
     '------WebKitFormBoundaryTIE5gBodnUrDhzC9
Content-Disposition: form-data; name="desc"

TEST ONLY.
' + 
     '------WebKitFormBoundaryTIE5gBodnUrDhzC9
Content-Disposition: form-data; name="userfield"


' + 
     '------WebKitFormBoundaryTIE5gBodnUrDhzC9
Content-Disposition: form-data; name="timezone"

Thu Mar 19 2015 16:25:47 GMT+1100 (AEDT)
' + 
     '------WebKitFormBoundaryTIE5gBodnUrDhzC9
Content-Disposition: form-data; name="browserstring"

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36
' + 
     '------WebKitFormBoundaryTIE5gBodnUrDhzC9
Content-Disposition: form-data; name="files[]"; filename=""
Content-Type: application/octet-stream


' + 
     '------WebKitFormBoundaryTIE5gBodnUrDhzC9--
';


  var reqHeaders = {

    'Cookie' : 'XXXXXXXXXXXXXXX',
    'Origin' : 'https://example.com',
    'Accept-Encoding' : 'gzip, deflate',
    'Accept-Language' : 'en-US,en;q=0.8',
    'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36',
    'Content-Type' : 'multipart/form-data; boundary=----WebKitFormBoundaryK0ckVcd9Rae277Ae',
    'Accept' : 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
    'Cache-Control' : 'max-age=0',
    'Referer' : 'https://example.com/index.php',
    'Connection' : 'keep-alive' 

  };


  var blob = Utilities.newBlob(reqPayload);

  var options =
   {
     'method' : 'post',
     'payload' : blob.getBytes(),
     'headers' : reqHeaders,
     'muteHttpExceptions' : true

   };

   var response = UrlFetchApp.fetch("https://www.example.com/index.php", options); 

     Logger.log("Response Full: " + response);

 }

While I've altered some of the details in order to anonymise the example (including removing the cookie details), the two requests should be identical. The curl request works, and the UrlFetchApp request is constructed based on copying from the curl request. But the UrlFetchApp request returns 404.

I'd be keen to know if I've missed anything obvious.

Answer 1

Turns out the problem here relates to firewalls.

In the instance where this didn't work, the UrlFetchApp target was behind a firewall. The curl command worked where this was sent from a terminal on a machine within that firewall. However, Google's Apps Script servers are outside that firewall, and hence the UrlFetchApp request fails.