如何测试/重构测试调用http.ListenAndServe的函数

Question

I'm learning go and am working on a simple service that ingests some data from a queue and sticks it in the database. It also runs a web server to allow scraping of data. Right now I have two go files (omitted some text for brevity):

func main() {
    parseConfig()

    s := &Service{ServiceConfig: config}
    err := s.Run()
    if err != nil {
        panic(err)
    }
}

And then the definition of the service (again left out some pieces for brevity):

func (s *Service) Run() error {

    if err := s.validate(); err != nil {
        return err
    }

    if err := s.initDB(); err != nil {
        return err
    }
    defer s.db.Close()

    // Same pattern with health check library (init, start, close)
    // Same pattern starting queue consumer (init, start, close)

    s.mux = http.NewServeMux()
    s.registerHandlers(s.mux)

    http.ListenAndServe(":8080", s.mux)

    return nil
}

And the struct

type Service struct {
    Config // Hold db connection info
    db  *sql.DB
    hc  *health
}

I'm able to test the individual pieces fine (like initDB or validate) but I'm not unclear how one would test the Run function because http.ListenAndServe blocks. I eventually time out. Previously, I would use httpTest and make a test server but that was when main would start the server (the application was more basic at first).

Some things I would test:

That I can hit the metrics endpoint once started. That I can hit the health endpoint once started. That I can push a message on the queue and it is received once started. That Run actually starts w/o a panic.

Some notes: I am using docker to spin up a queue and database. The point of testing the Run function is to ensure that the bootstrapping works and the application can run successfully. Eventually I will want to push data through the queue and assert that its been processed correctly.

Question: How should I test this or refactor it so that it is more easily testable end to end?

Answer 1

You can build a test harness using a goroutine to execute Run in your unit test:

func TestRun(t *testing.T) {
    service := Service{}
    serviceRunning := make(chan struct{})
    serviceDone := make(chan struct{})
    go func() {
        close(serviceRunning)
        err := service.Run()            
        defer close(serviceDone)
    }()

    // wait until the goroutine started to run (1)
    <-serviceRunning

    //
    // interact with your service to test whatever you want
    //

    // stop the service (2)
    service.Shutdown()

    // wait until the service is shutdown (3)
    <-serviceDone
}

This is just a basic example to show how it could be done in principle. There are several points that should be improved for use in production:

(0) The most important detail: Do not use http.ListenAndServe in production! Create your own instance of http.Server instead. This saves you a lot of trouble.

s.httpServer := http.Server {
    Addr: ":8080",
    Handler: s.mux,
}

(1) The indication that the service is running should be moved into your Service type. The initialization part in Run might take a while and the indication channel should be closed right before ListenAndServe is called:

close(s.Running)
s.httpServer.ListenAndServe()

Of course, you need to add the indication channel to your Service type.

(2) Add a Shutdown method to Service that calls s.httpServer.Shutdown(). This will cause the call to s.httpServer.ListenAndServe to return with the error http.ErrServerClosed.

if err := s.httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
    return err
}
return nil

It is important to shutdown your service at the end of the test. Otherwise you will not be able to have more than one unit test for your service. And it is good citizenship anyway to clean up the resources.

(3) You need to wait until service.Run returned to make sure that the service is actually shutdown.