duanlinghe8417 2015-07-13 14:01
浏览 138

GOLANG,HTTP具有“使用封闭的网络连接”错误

I am getting a lot of error like below mentioned,

read tcp xx.xx.xx.xx:80: use of closed network connection

read tcp xx.xx.xx.xx:80: connection reset by peer

//function for HTTP connection

func GetResponseBytesByURL_raw(restUrl, connectionTimeOutStr, readTimeOutStr string) ([]byte, error) {
    connectionTimeOut, _ /*err*/ := time.ParseDuration(connectionTimeOutStr)
    readTimeOut, _ /*err*/ := time.ParseDuration(readTimeOutStr)
    timeout := connectionTimeOut + readTimeOut // time.Duration((strconv.Atoi(connectionTimeOutStr) + strconv.Atoi(readTimeOutStr)))
    //timeout = 200 * time.Millisecond
    client := http.Client{
        Timeout: timeout,
    }
    resp, err := client.Get(restUrl)
    if nil != err {
        logger.SetLog("Error GetResponseBytesByURL_raw |err: ", logs.LevelError, err)
        return make([]byte, 0), err
    }
    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)
    return body, err
}

Update (July 14):

Server : NumCPU=8, RAM=24GB, GO=go1.4.2.linux-amd64

I am getting such error during some high traffic. 20000-30000 request per minutes, and I have a time frame of 500ms to fetch response from third party api.

netstat status from my server (using : netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n) to get frequency

      1 established)
      1 Foreign
      9 LISTEN
     33 FIN_WAIT1
    338 ESTABLISHED
   5530 SYN_SENT
  32202 TIME_WAIT

sysctl -p

**sysctl -p**
fs.file-max = 2097152
vm.swappiness = 10
vm.dirty_ratio = 60
vm.dirty_background_ratio = 2
net.ipv4.tcp_synack_retries = 2
net.ipv4.ip_local_port_range = 2000 65535
net.ipv4.tcp_rfc1337 = 1
net.ipv4.tcp_fin_timeout = 5
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_keepalive_probes = 5
net.ipv4.tcp_keepalive_intvl = 15
net.core.rmem_default = 31457280
net.core.rmem_max = 12582912
net.core.wmem_default = 31457280
net.core.wmem_max = 12582912
net.core.somaxconn = 65536
net.core.netdev_max_backlog = 65536
net.core.optmem_max = 25165824
net.ipv4.tcp_mem = 65536 131072 262144
net.ipv4.udp_mem = 65536 131072 262144
net.ipv4.tcp_rmem = 8192 87380 16777216
net.ipv4.udp_rmem_min = 16384
net.ipv4.tcp_wmem = 8192 65536 16777216
net.ipv4.udp_wmem_min = 16384
net.ipv4.tcp_max_tw_buckets = 1440000
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_tw_reuse = 1
net.ipv6.bindv6only = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.icmp_echo_ignore_broadcasts = 1
error: "net.ipv4.icmp_ignore_bogus_error_messages" is an unknown key
kernel.exec-shield = 1
kernel.randomize_va_space = 1
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1
net.ipv4.icmp_ignore_bogus_error_responses = 1
net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
  • 写回答

2条回答 默认 最新

  • dongtuhe0506 2015-07-14 14:17
    关注

    When making connections at a high rate over the internet, it's very likely you're going to encounter some connection problems. You can't mitigate them completely, so you may want to add retry logic around the request. The actual error type at this point probably doesn't matter, but matching the error string for use of closed network connection or connection reset by peer is about the best you can do if you want to be specific. Make sure to limit the retries with a backoff, as some systems will drop or reset connections as a way to limit request rates, and you may get more errors the faster you reconnect.

    Depending on the number of remote hosts you're communicating with, you will want to increase Transport.MaxIdleConnsPerHost (the default is only 2). The fewer hosts you talk to, the higher you can set this. This will decrease the number of new connections made, and speed up the requests overall.

    If you can, try the go1.5 beta. There have been a couple changes around keep-alive connections that may help reduce the number of errors you see.

    评论

报告相同问题?

悬赏问题

  • ¥15 python天天向上类似问题,但没有清零
  • ¥30 3天&7天&&15天&销量如何统计同一行
  • ¥30 帮我写一段可以读取LD2450数据并计算距离的Arduino代码
  • ¥15 C#调用python代码(python带有库)
  • ¥15 矩阵加法的规则是两个矩阵中对应位置的数的绝对值进行加和
  • ¥15 活动选择题。最多可以参加几个项目?
  • ¥15 飞机曲面部件如机翼,壁板等具体的孔位模型
  • ¥15 vs2019中数据导出问题
  • ¥20 云服务Linux系统TCP-MSS值修改?
  • ¥20 关于#单片机#的问题:项目:使用模拟iic与ov2640通讯环境:F407问题:读取的ID号总是0xff,自己调了调发现在读从机数据时,SDA线上并未有信号变化(语言-c语言)