I have been working on a golang script that uses the chrome devtools protocol to:
1) Intercept a request
2) Grab the response body for the intercepted request
3) Make some modifications to the html
document
4) Continue the intercepted request
The script works for HTML documents except when Content-Encoding
is set to gzip
. The step-by-step process looks like this"
1) Intercept Request
s.Debugger.CallbackEvent("Network.requestIntercepted", func(params godet.Params) {
iid := params.String("interceptionId")
rtype := params.String("resourceType")
reason := responses[rtype]
headers := getHeadersString(params["responseHeaders"])
log.Println("[+] Request intercepted for", iid, rtype, params.Map("request")["url"])
if reason != "" {
log.Println(" abort with reason", reason)
}
// Alter HTML in request response
if s.Options.AlterDocument && rtype == "Document" && iid != "" {
res, err := s.Debugger.GetResponseBodyForInterception(iid)
if err != nil {
log.Println("[-] Unable to get intercepted response body!")
}
rawAlteredResponse, err := AlterDocument(res, headers)
if err != nil{
log.Println("[-] Unable to alter HTML")
}
if rawAlteredResponse != "" {
log.Println("[+] Sending modified body")
err := s.Debugger.ContinueInterceptedRequest(iid, godet.ErrorReason(reason), rawAlteredResponse, "", "", "", nil)
if err != nil {
fmt.Println("OH NOES AN ERROR!")
log.Println(err)
}
}
} else {
s.Debugger.ContinueInterceptedRequest(iid, godet.ErrorReason(reason), "", "", "", "", nil)
}
})
2) Alter the response body
Here I am making small changes to the HTML markup in procesHtml()
(but the code for that function is not relevant to this issue, so will not post it here). I also grab headers from the request and when necessary update the content-length
and date
before continue the reponse. Then, I gzip compress the body when calling r := gZipCompress([]byte(alteredBody)
, which returns a string. The string is then concatenated to the headers so I can craft the rawResponse
.
func AlterDocument(debuggerResponse []byte, headers map[string]string) (string, error) {
alteredBody, err := processHtml(debuggerResponse)
if err != nil {
return "", err
}
alteredHeader := ""
for k, v := range headers{
switch strings.ToLower(k) {
case "content-length":
v = strconv.Itoa(len(alteredBody))
fmt.Println("Updating content-length to: " + strconv.Itoa(len(alteredBody)))
break
case "date":
v = fmt.Sprintf("%s", time.Now().Format(time.RFC3339))
break
}
alteredHeader += k + ": " + v + "
"
}
r := gZipCompress([]byte(alteredBody))
rawAlteredResponse :=
base64.StdEncoding.EncodeToString([]byte("HTTP/1.1 200 OK" + "
" + alteredHeader + "
" + r))
return rawAlteredResponse, nil
}
Note: I am now gzip compressing the body for all responses. The above is temporary while I figure out how to solve this issue.
The gzip compress function looks like this:
func gZipCompress(dataToWorkWith []byte) string{
var b bytes.Buffer
gz, err := gzip.NewWriterLevel(&b, 5)
if err != nil{
panic(err)
}
if _, err := gz.Write(dataToWorkWith); err != nil {
panic(err)
}
if err := gz.Flush(); err != nil {
panic(err)
}
if err := gz.Close(); err != nil {
panic(err)
}
return b.String()
}
As seen in the first code snippet, the response body and headers are set here:
err := s.Debugger.ContinueInterceptedRequest(iid, godet.ErrorReason(reason), rawAlteredResponse, "", "", "", nil)
The result is a bunch of garbled characters in the browser. This works without the gzip functions for non gzipped requests. I have changed the compression level as well (without success). Am I processing the body in the wrong order (string > []byte > gzip > string > base64)? Should this be done in a different order to work? Any help would be immensely appreciated.
The response looks like this, which Chrome puts inside a <body></body>
tag
����rܸ� ��_A��Q%GH��Kʔ��vU�˷c�v�}
or in the response:
I can also tell that it is compressing correctly as, when I remove headers, the request results in a .gz
file download with all the correct .html
when uncompressed. Additionally, the first few bytes in the object returned in gZipCompress
tell me that it is gzipped correctly:
31 139 8
or
0x1f 0x8B 0x08