Go-尝试创建超过一定数量的goroutine时出现细分违规问题

Question

I'm trying to stress test an http client in Go.

At first I simply tried to run 10 concurrent requests for 10 iterations.

Here's my client code:

// stress.go
package main

import (
    "fmt"
    "io/ioutil"
    "net/http"
    "time"
)

func MakeRequest(url string, ch chan<- string) {
    start := time.Now()
    resp, _ := http.Get(url)

    secs := time.Since(start).Seconds()
    body, _ := ioutil.ReadAll(resp.Body)
    ch <- fmt.Sprintf("%.2f elapsed with response length: %d %s", secs, len(body), url)
}

func main() {
    start := time.Now()
    goroutines := 10
    iterations := 10
    ch := make(chan string)
    url := "http://localhost:8000"
    for i := 0; i < iterations; i++ {
        for j := 0; j < goroutines; j++ {
            go MakeRequest(url, ch)
        }
    }
    for i := 0; i < goroutines*iterations; i++ {
        fmt.Println(<-ch)
    }

    fmt.Printf("%.2fs elapsed
", time.Since(start).Seconds())
}

It works fine for this combination of iterations and goroutines. But when number of goroutines and iterations go beyond a certain level, in my case, for a single iteration, when number of goroutines go above 634, I receive this error:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0x11eb9f0]

goroutine 7099 [running]:
main.MakeRequest(0x1271897, 0x15, 0xc420074120)
        stress.go:16 +0xa0
                created by main.main
                        stress.go:28 +0xb2

According to this, one can easily create 10000 goroutines. What am I doing wrong in my case? How do I test max number of GET or POST requests my client can make concurrently for given number of iterations?

Answer 1

You're discarding errors from http.Get and ioutil.ReadAll, so I'm not surprised you're getting a nil pointer dereference; you're assuming return values are good when that is not a valid assumption. The server may be returning errors because you've exceeded the maximum throughput of the server.

You're also not testing 10 goroutines for 10 iterations, you're testing 100 goroutines; and you're not closing the request bodies after you finish reading them. Lastly, how much throughput you can get will depend on the server, how much resources the server has available to it, and how much resources the client has available to it; and the competition between the two, since you're hitting localhost the client and server are contending for the same system resources.

There is no static performance characteristic for any language; it depends on the context in which it is running.