Overview
If I have a parent test with setup and teardown logic, how do I run subtests within it in parallel without running into a race condition with the teardown logic?
func TestFoo(t *testing.T) {
// setup logic
t.Run("a", func(t *testing.T) {
t.Parallel()
// test code
})
// teardown logic
}
Example
As a contrived example: let's say that the test needs to create a tmp file that will be used by all subtests and delete it when the test is over.
For the example, the parent test also calls t.Parallel(), as that's what I ultimately want. But my problem and the output below is the same even if the parent does not call t.Parallel().
Sequential Subtest
If I run the subtests sequentially, they pass no problem:
package main
import (
"fmt"
"io/ioutil"
"os"
"testing"
)
func setup(t *testing.T) (tmpFile string) {
f, err := ioutil.TempFile("/tmp", "subtests")
if err != nil {
t.Fatalf("could not setup tmp file: %+v", err)
}
f.Close()
return f.Name()
}
var ncase = 2
func TestSeqSubtest(t *testing.T) {
t.Parallel()
// setup test variables
fname := setup(t)
// cleanup test variables
defer func() {
os.Remove(fname)
}()
for i := 0; i < ncase; i++ {
t.Run(fmt.Sprintf("test_%d", i), func(t *testing.T) {
if _, err := os.Stat(fname); os.IsNotExist(err) {
t.Fatalf("file was removed before subtest finished")
}
})
}
}
Output:
$ go test subtests
ok subtests 0.001s
Parallel Subtest
If, however, I run the subtests in parallel, then the parent test's teardown logic ends up getting called before the subtest has a chance to run, making it impossible to get the subtest to run correctly.
This behavior, while unfortunate, fits with what the "Using Subtests and Sub-benchmarks" go blog says:
A test is called a parallel test if its test function calls the Parallel method on its instance of testing.T. A parallel test never runs concurrently with a sequential test and its execution is suspended until its calling test function, that of the parent test, has returned.
func TestParallelSubtest(t *testing.T) {
t.Parallel()
// setup test variables
fname := setup(t)
// cleanup test variables
defer func() {
os.Remove(fname)
}()
for i := 0; i < ncase; i++ {
t.Run(fmt.Sprintf("test_%d", i), func(t *testing.T) {
t.Parallel() // the change that breaks things
if _, err := os.Stat(fname); os.IsNotExist(err) {
t.Fatalf("file was removed before subtest finished")
}
})
}
}
Output:
$ go test subtests
--- FAIL: TestParallelSubtest (0.00s)
--- FAIL: TestParallelSubtest/test_0 (0.00s)
main_test.go:58: file was removed before subtest finished
--- FAIL: TestParallelSubtest/test_1 (0.00s)
main_test.go:58: file was removed before subtest finished
FAIL
FAIL subtests 0.001s
Parallel Subtest with WaitGroup
As the above quote states, parallel subtests won't execute until their parent has finished, which means that trying to solve this with a sync.WaitGroup results in a deadlock:
func TestWaitGroupParallelSubtest(t *testing.T) {
t.Parallel()
var wg sync.WaitGroup
// setup test variables
fname := setup(t)
// cleanup test variables
defer func() {
os.Remove(fname)
}()
for i := 0; i < ncase; i++ {
t.Run(fmt.Sprintf("test_%d", i), func(t *testing.T) {
wg.Add(1)
defer wg.Done()
t.Parallel()
if _, err := os.Stat(fname); os.IsNotExist(err) {
t.Fatalf("file was removed before subtest finished")
}
})
}
wg.Wait() // causes deadlock
}
output:
$ go test subtests
--- FAIL: TestParallelSubtest (0.00s)
--- FAIL: TestParallelSubtest/test_0 (0.00s)
main_test.go:58: file was removed before subtest finished
--- FAIL: TestParallelSubtest/test_1 (0.00s)
main_test.go:58: file was removed before subtest finished
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
testing.tRunner.func1(0xc00009a000)
/path/to/golang1.1.11/src/testing/testing.go:803 +0x1f3
testing.tRunner(0xc00009a000, 0xc00005fe08)
/path/to/golang1.1.11/src/testing/testing.go:831 +0xc9
testing.runTests(0xc00000a0a0, 0x6211c0, 0x3, 0x3, 0x40b36f)
/path/to/golang1.1.11/src/testing/testing.go:1117 +0x2aa
testing.(*M).Run(0xc000096000, 0x0)
/path/to/golang1.1.11/src/testing/testing.go:1034 +0x165
main.main()
_testmain.go:46 +0x13d
goroutine 7 [semacquire]:
sync.runtime_Semacquire(0xc0000a2008)
/path/to/golang1.1.11/src/runtime/sema.go:56 +0x39
sync.(*WaitGroup).Wait(0xc0000a2000)
/path/to/golang1.1.11/src/sync/waitgroup.go:130 +0x64
subtests.TestWaitGroupParallelSubtest(0xc00009a300)
/path/to/go_code/src/subtests/main_test.go:91 +0x2b5
testing.tRunner(0xc00009a300, 0x540f38)
/path/to/golang1.1.11/src/testing/testing.go:827 +0xbf
created by testing.(*T).Run
/path/to/golang1.1.11/src/testing/testing.go:878 +0x353
goroutine 8 [chan receive]:
testing.runTests.func1.1(0xc00009a000)
/path/to/golang1.1.11/src/testing/testing.go:1124 +0x3b
created by testing.runTests.func1
/path/to/golang1.1.11/src/testing/testing.go:1124 +0xac
goroutine 17 [chan receive]:
testing.(*T).Parallel(0xc0000f6000)
/path/to/golang1.1.11/src/testing/testing.go:732 +0x1fa
subtests.TestWaitGroupParallelSubtest.func2(0xc0000f6000)
/path/to/go_code/src/subtests/main_test.go:85 +0x86
testing.tRunner(0xc0000f6000, 0xc0000d6000)
/path/to/golang1.1.11/src/testing/testing.go:827 +0xbf
created by testing.(*T).Run
/path/to/golang1.1.11/src/testing/testing.go:878 +0x353
goroutine 18 [chan receive]:
testing.(*T).Parallel(0xc0000f6100)
/path/to/golang1.1.11/src/testing/testing.go:732 +0x1fa
subtests.TestWaitGroupParallelSubtest.func2(0xc0000f6100)
/path/to/go_code/src/subtests/main_test.go:85 +0x86
testing.tRunner(0xc0000f6100, 0xc0000d6040)
/path/to/golang1.1.11/src/testing/testing.go:827 +0xbf
created by testing.(*T).Run
/path/to/golang1.1.11/src/testing/testing.go:878 +0x353
FAIL subtests 0.003s
Summary
So how can I go about having a teardown method in the parent test that gets called after the parallel subtests run?
