dongzhong8691 2015-05-30 02:26
浏览 138
已采纳

从Go到CouchDB的大型PUT请求

I've been having a vexing problem with CouchDB and Golang. When sending POST/PUT requests to CouchDB with relatively large Body sizes (the threshold seems to be ~8000 bytes or so), the connection times out and I get a "tcp: use of closed network connection" error from Go.

Eventually (a second or two later), CouchDB sends a 500 Response along with a:

{"error":"unknown_error", "reason": "noproc"}

in the body. The couchdb log also has a stack trace written to it:

Stacktrace: [{couch_db,collect_results,3,
                [{file,"couch_db.erl"},{line,833}]},
             {couch_db,write_and_commit,4,
                [{file,"couch_db.erl"},{line,845}]},
             {couch_db,update_docs,4,
                [{file,"couch_db.erl"},{line,782}]},
             {couch_db,update_doc,4,
                [{file,"couch_db.erl"},{line,426}]},
             {couch_httpd_db,update_doc,6,
                [{file,"couch_httpd_db.erl"},{line,753}]},
             {couch_httpd_db,do_db_req,2,
                [{file,"couch_httpd_db.erl"},{line,234}]},
             {couch_httpd,handle_request_int,5,
                [{file,"couch_httpd.erl"},{line,318}]},
             {mochiweb_http,headers,5,
                [{file,"mochiweb_http.erl"},{line,94}]}]

So I wrote up a quick unit test to replicate the issue (and to make sure it's not my wrapper causing the problem). I'm making the request like so:

client := &http.Client{}
req, err := http.NewRequest(
    "PUT",
    "http://localhost:5984/unittestdb/testdocid1",
    bytes.NewReader(testBody1), //testBody1 is 10000 bytes of json object
)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Accept", "application/json")
resp, err := client.Do(req)

This replicated the issue... so I tried curling a large json document to couchdb. That worked.

So, I fired up wireshark and examined the request made by my code to CouchDB and compared it to a request sent from curl. I noticed one additional header in the Curl Request:

Expect: 100-continue

I had to look that one up, as I couldn't recall having seen (or maybe I just hadn't had to deal with) that header before. So then I did a google search to see if the Golang http client supported the Expect/Continue functionality, and found this: https://github.com/golang/go/issues/3665

So Go doesn't support it and won't at least until 1.6. So I assumed it must be an obscure thing and wasn't the source of my problem. I spent a couple of hours randomly trying other stuff with the http.Client, Transport, etc.

Eventually, I manually set the "Expect: 100-continue" header in my http request in Go (if the body size is over a certain size), and ... it worked. No timeout, no error, no couchdb barfing stack traces into the logs.

Now I am confused, if Go doesn't support this, how is it working? Am I just masking the problem by doing this? Or can I just shrug my shoulders and move on?

I suspect there is an issue on the CouchDB side, perhaps there's a configuration setting I'm missing?

  • 写回答

1条回答 默认 最新

  • douhao2548 2015-05-30 03:10
    关注

    I don't know the answer, but I think that the critical spec for you is RFC 2616 section 8.2.3. The whole section is interesting, but particularly this:

    "There is an exception to this rule: for compatibility with RFC 2068, a server MAY send a 100 (Continue) status in response to an HTTP/1.1 PUT or POST request that does not include an Expect request-header field with the "100-continue" expectation. This exception, the purpose of which is to minimize any client processing delays associated with an undeclared wait for 100 (Continue) status, applies only to HTTP/1.1 requests, and not to requests with any other HTTP-version value."

    It sounds like you're getting caught between Go's missing 100-continue implementation, and some kind of corner case on the CouchDB side. (Whatever the behavior of the client, I don't think 500 Internal error is an appropriate response from the server.)

    I'd try HTTP/1.0 as a workaround, if the Go client will let you set that.

    本回答被题主选为最佳回答 , 对您是否有帮助呢?
    评论

报告相同问题?

悬赏问题

  • ¥20 求快手直播间榜单匿名采集ID用户名简单能学会的
  • ¥15 DS18B20内部ADC模数转换器
  • ¥15 做个有关计算的小程序
  • ¥15 MPI读取tif文件无法正常给各进程分配路径
  • ¥15 如何用MATLAB实现以下三个公式(有相互嵌套)
  • ¥30 关于#算法#的问题:运用EViews第九版本进行一系列计量经济学的时间数列数据回归分析预测问题 求各位帮我解答一下
  • ¥15 setInterval 页面闪烁,怎么解决
  • ¥15 如何让企业微信机器人实现消息汇总整合
  • ¥50 关于#ui#的问题:做yolov8的ui界面出现的问题
  • ¥15 如何用Python爬取各高校教师公开的教育和工作经历