当连接保持活动状态时，我获得了太多打开的连接

Question

I have a simple Go file with a purpose of returning a json response of a client's information. When I am benchmarking this script with apache benchmark, and the requests are kept alive

ab -t 10s -kc 1000 http://127.0.0.1:8080/clients/show/1

But when the requests are not kep alive, I do not suffer from this panic

ab -t 10s -c 1000 http://127.0.0.1:8080/clients/show/1

error:

2018/10/26 03:26:42 http: panic serving 127.0.0.1:44800: Error 1040: Too many connections goroutine 220522 [running]: net/http.(*conn).serve.func1(0xc001779e00)

My code:

package main

import (
    "database/sql"
    "encoding/json"
    "fmt"
    _ "github.com/go-sql-driver/mysql"
    "github.com/gorilla/mux"
    "net/http"
    "runtime"
)

type Client struct {
    ID         int    `json:"id"`
    UserID     int    `json:"user_id"`
    Name       string `json:"name"`
    Telephone  string `json:"telephone"`
    Email      string    `json:"email"`
    Category   sql.NullString `json:"string"`
    Notes      string `json:"notes"`
    Additional sql.NullString `json:"additional"`
    CreatedAt  sql.NullString `json:"created_at"`
    UpdatedAt  sql.NullString `json:"updated_at"`
    DeletedAt  sql.NullString `json:"deleted_at"`
}

func main() {
    runtime.GOMAXPROCS(runtime.NumCPU())
    r := mux.NewRouter()

    r.HandleFunc("/clients/show/{id}", showClient).Methods("GET")

    http.ListenAndServe(":8080", r)
}

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hello World!")
}

func showClient(w http.ResponseWriter, r *http.Request) {

    db, err := sql.Open("mysql", "root@tcp(127.0.0.1:3306)/crm")

    if err != nil {
        panic(err.Error())
    }

    var client Client

    // Execute the query
    err = db.QueryRow("SELECT * FROM clients where id = ?", 1).Scan(
        &client.ID,
        &client.UserID,
        &client.Name,
        &client.Telephone,
        &client.Email,
        &client.Category,
        &client.Notes,
        &client.Additional,
        &client.CreatedAt,
        &client.UpdatedAt,
        &client.DeletedAt,
    )

    if err != nil {
        panic(err.Error())
    }

    db.Close()

    json.NewEncoder(w).Encode(client)
}

Can someone please explain why this happens at such a low concurrent requests rate and what the correct way to fix this is.

Answer 1

There are two problems, the program use of sockets and the server configuration

Each connection probably requires 2 sockets, one to the database and one to the http client. Moving the sql.Open() to a different scope so it is not repeatedly called would help with the database sockets. For the http server, the defaults are actually quite low - close connections if there are more than 2 unused keep alives

Additionally, your server probably needs modification

As the top answer to this question discusses Increasing the maximum number of tcp/ip connections in linux the default connections per second for Linux are quite conservative and a small simple Go server could well overwealm it

Tune the server in line with the above answer and that should help