同时请求拨打TCP I / O超时

Question

I am building a tool in Go that needs to make a very large number of simultaneous HTTP requests to many different servers. My initial prototype in Python had no problem doing a few hundred simultaneous requests.

However, I have found that in Go this almost always results in a Get http://www.google.com: dial tcp 216.58.205.228:80: i/o timeout for some if the number of simultaneous requests exceeds ~30-40.

I've tested on macOS, openSUSE, different hardware, in different networks and with different domain lists, and changing the DNS server as described in other Stackoverflow answers does not work either.

The interesting thing is that the failed requests do not even produce a packet, as can be seen when checking with Wireshark.

Is there anything that I am doing wrong or is that a bug in Go?

Minimum reproducible program below:

package main

import (
    "fmt"
    "net/http"
    "sync"
)

func main() {
    domains := []string{/* large domain list here, eg from https://moz.com/top500 */}

    limiter := make(chan string, 50) // Limits simultaneous requests

    wg := sync.WaitGroup{} // Needed to not prematurely exit before all requests have been finished

    for i, domain := range domains {
        wg.Add(1)
        limiter <- domain

        go func(i int, domain string) {
            defer func() { <-limiter }()
            defer wg.Done()

            resp, err := http.Get("http://"+domain)
            if err != nil {
                fmt.Printf("%d %s failed: %s
", i, domain, err)
                return
            }

            fmt.Printf("%d %s: %s
", i, domain, resp.Status)
        }(i, domain)
    }

    wg.Wait()
}

Two particular error messages are happening, a net.DNSError that does not make any sense and a non-descript poll.TimeoutError:

&url.Error{Op:"Get", URL:"http://harvard.edu", Err:(*net.OpError)(0xc00022a460)}
&net.OpError{Op:"dial", Net:"tcp", Source:net.Addr(nil), Addr:net.Addr(nil), Err:(*net.DNSError)(0xc000aca200)}
&net.DNSError{Err:"no such host", Name:"harvard.edu", Server:"", IsTimeout:false, IsTemporary:false}

&url.Error{Op:"Get", URL:"http://latimes.com", Err:(*net.OpError)(0xc000d92730)}
&net.OpError{Op:"dial", Net:"tcp", Source:net.Addr(nil), Addr:net.Addr(nil), Err:(*poll.TimeoutError)(0x14779a0)}
&poll.TimeoutError{}

Update:

Running the requests with a seperate http.Client as well as http.Transport and net.Dialer does not make any difference as can be seen when running code from this playground.