doucuo9126
2017-11-13 23:35 阅读 89
已采纳

Go HTTP客户端添加了服务不支持的分块编码

The go HTTP client is adding a "chunked" transfer encoding field to my client request. Unfortunately, this is not supported by the service I'm connecting to and it comes back with an error.

Is there a way to disable this?

This is my Request code:

// DoHTTPRequest Do a full HTTP Client Request, with timeout
func DoHTTPRequest(method, url string, body io.Reader, headers map[string]string, timeout time.Duration) (*http.Response, error) {

    // Create the request
    req, err := http.NewRequest(method, url, body)
    if err != nil {
        return nil, err
    }

    // Add headers
    for k, v := range headers {
        req.Header.Set(k, v)
    }

    client := &http.Client{
        Timeout: timeout,
    }

    return client.Do(req)
}

Basically I would like this header dropped. This client is talking to S3 which is rather sensitive as to what headers it is sent.

I get this error:

A header you provided implies functionality that is not implemented
  • 点赞
  • 写回答
  • 关注问题
  • 收藏
  • 复制链接分享

2条回答 默认 最新

  • 已采纳
    drqja5919276 drqja5919276 2017-11-14 00:13

    TransferEncoding is a field directly on the Request struct. If you set it explicitly it will not be overridden.

    req.TransferEncoding = []string{"identity"}

    https://golang.org/pkg/net/http/#Request

    点赞 评论 复制链接分享
  • duandi2853 duandi2853 2017-11-14 00:54

    Setting Transfer Encoding header like this makes it not use chunked:

    req.TransferEncoding = []string{"identity"}
    

    However the http client sources give the reason for it choosing chunked in my case. Specifically, I was using "PUT" as the method and had no content-length specified. So all I needed was to set the req.ContentLength.

    However, you can see my DoHTTPRequest wrapper function doesn't know how big to set it. And I had assumed that setting the header will make it work originally. Well, it doesn't work by setting the header. And you can see why in the sources that determine whether to make it use chunked encoding.

    // shouldSendChunkedRequestBody reports whether we should try to send a
    // chunked request body to the server. In particular, the case we really
    // want to prevent is sending a GET or other typically-bodyless request to a
    // server with a chunked body when the body has zero bytes, since GETs with
    // bodies (while acceptable according to specs), even zero-byte chunked
    // bodies, are approximately never seen in the wild and confuse most
    // servers. See Issue 18257, as one example.
    //
    // The only reason we'd send such a request is if the user set the Body to a
    // non-nil value (say, ioutil.NopCloser(bytes.NewReader(nil))) and didn't
    // set ContentLength, or NewRequest set it to -1 (unknown), so then we assume
    // there's bytes to send.
    //
    // This code tries to read a byte from the Request.Body in such cases to see
    // whether the body actually has content (super rare) or is actually just
    // a non-nil content-less ReadCloser (the more common case). In that more
    // common case, we act as if their Body were nil instead, and don't send
    // a body.
    func (t *transferWriter) shouldSendChunkedRequestBody() bool {
        // Note that t.ContentLength is the corrected content length
        // from rr.outgoingLength, so 0 actually means zero, not unknown.
        if t.ContentLength >= 0 || t.Body == nil { // redundant checks; caller did them
            return false
        }
        if requestMethodUsuallyLacksBody(t.Method) {
            // Only probe the Request.Body for GET/HEAD/DELETE/etc
            // requests, because it's only those types of requests
            // that confuse servers.
            t.probeRequestBody() // adjusts t.Body, t.ContentLength
            return t.Body != nil
        }
        // For all other request types (PUT, POST, PATCH, or anything
        // made-up we've never heard of), assume it's normal and the server
        // can deal with a chunked request body. Maybe we'll adjust this
        // later.
        return true
    }
    

    So my solution is simply:

    // DoHTTPRequest Do a full HTTP Client Request, with timeout
    func DoHTTPRequest(method, url string, body io.Reader, headers map[string]string, timeout time.Duration) (*http.Response, error) {
    
        // Create the request
        req, err := http.NewRequest(method, url, body)
        if err != nil {
            return nil, err
        }
    
        // Add headers
        for k, v := range headers {
            req.Header.Set(k, v)
            // Set the Content Length correctly if specified.
            if strings.EqualFold(k, "Content-Length") {
                length, err := strconv.Atoi(v)
                if err != nil {
                   return nil, fmt.Errorf("Bad Content-Length header")
                }
                req.ContentLength = int64(length)
            }
        }
    
        client := &http.Client{
            Timeout:   timeout,
            Transport: &loghttp.Transport{},
        }
    
        return client.Do(req)
    }
    

    Which satisfies S3 as far as having a correct content length. I didn't need to set the TransferEncoding to identity.

    点赞 评论 复制链接分享

相关推荐