如何在客户端无错误地进行大量连接

Question

I have written a program in golang to make request about 2000qps to different remote ip with local port randomly selected by linux, and close request immediately after connection established, but still encounter bind: address already in use error periodically

what I have done:

1. net.ipv4.ip_local_port_range is 15000-65535
2. net.ipv4.tcp_tw_recycle=1 net.ipv4.tcp_tw_reuse=1 net.ipv4.tcp_fin_timeout=30

above is sockstat:

sockets: used 1200 TCP: inuse 2302 orphan 1603 tw 40940 alloc 2325 mem 201

I don't figure it out why this error still there with kernel selecting available local port,will kernel return a port in use ?

Answer 1

This is a good answer from 2012:

https://serverfault.com/questions/342741/what-are-the-ramifications-of-setting-tcp-tw-recycle-reuse-to-1#434669

As of 2018, tcp_tw_recycle exists only in the sysctl binary, is otherwise gone from the kernel:

https://github.com/torvalds/linux/search?utf8=%E2%9C%93&q=tcp_tw_recycle&type=

tcp_tw_reuse is still in use as described in the above answer:

https://github.com/torvalds/linux/blob/master/net/ipv4/tcp_ipv4.c#L128

However, while a TCP_TIMEWAIT_LEN is in use:

https://github.com/torvalds/linux/search?utf8=%E2%9C%93&q=TCP_TIMEWAIT_LEN&type=

the value is hardcoded:

https://github.com/torvalds/linux/blob/master/include/net/tcp.h#L120

and tcp_fin_timeout refers to a different state:

https://github.com/torvalds/linux/blob/master/Documentation/networking/ip-sysctl.txt#L294

One can relatively safely change the local port range to 1025-65535.

For kicks, if there were a situation where this client was talking to servers and network under my control, I would build a new kernel with a not-to-spec TCP_TIMEWAIT_LEN, and perhaps also fiddle with tcp_max_tw_buckets:

https://github.com/torvalds/linux/blob/master/Documentation/networking/ip-sysctl.txt#L379

But doing so in other circumstances- if this client is behind a NAT and talking to common public servers- will likely be disruptive.