mdawar.dev

A blog about programming, Web development, Open Source, Linux and DevOps.

Go - Testing

Test Files

  • Test files have file names ending in _test.go
  • Test files may contain: tests, benchmarks, examples and fuzz tests
  • The *_test.go files are excluded from regular package builds

Test Package

A test file can be in the same package as the one being tested.

go
// A test file in the same package may refer to unexported identifiers.
package calculator

It’s a good practice for the test file to be in a corresponding package with the suffix _test.

go
// A test file in a separate `_test` package, may only use the exported identifiers.
package calculator_test
// The package being tested must be imported
import (
"calculator"
)

Test Functions

Test function signature:

go
// Starts with the word `Test`.
// Where `Xxx` does not start with a lowercase letter.
func TestXxx(t *testing.T)

Example:

calculator_test.go
go
package calculator_test
import (
"calculator" // The package being tested
"testing" // Standard library testing package
)
func TestAdd(t *testing.T) {
// By convention the expected result is defined as a `want` variable
// and the actual result as a `got` variable.
want := 17
got := calculator.Add(10, 7)
// Compare the expected result with the actual result.
if want != got {
// Make the test fail with a formatted error message showing the
// expected and actual values.
t.Errorf("want %d, got %d", want, got)
}
}

Test Failure

Tests pass by default unless we explicitly make them fail with the Error or Fatal methods.

go
func TestSomething(t *testing.T) {
// Report an error and continue execution of the test.
t.Error("expected an error, got nil")
t.Errorf("want %d, got %d", want, got) // Formatted error message
// Report an error and abort the test.
// Execution will continue at the next test or benchmark.
t.Fatal(err)
t.Fatalf("did not expect an error, got %v", err) // Formatted error message
}

Logging Text

Text can be logged with the Log and Logf methods.

go
func TestSomething(t *testing.T) {
// Record the text in the error log (newline is added if not provided).
// The text will be printed if the test fails or the verbose flag `-v` is set.
// For benchmarks, the text is always printed.
t.Log("report a useful message")
t.Logf("another useful message with data: %v", data) // Formatted error message
}

Run Tests

The tests are run in 2 different modes:

1. Local Directory Mode

When go test is invoked with no package arguments, it runs the tests in the current directory.

bash
$go test

2. Package List Mode

When go test is invoked with explicit package arguments, it runs tests for the listed packages.

bash
$# Run the tests for a specific package.
$go test packageName
$go test moduleName/subPackageName
$# Run the tests for multiple packages.
$go test package1 package2 package3
$# Run the tests for the package in the current directory.
$go test .
$# Run the tests for the current package and all the sub-packages.
$go test ./...

In the package list mode, successful test results are cached.

To disable test caching:

bash
$# Disable test caching with `-count 1`.
$go test -count 1 ./...

Run Specific Tests

bash
$# The -run flag is followed by a regexp to match the functions to run.
$go test -run NameToMatch
$# Match the functions to run in the current package and all sub-packages.
$go test -run NameToMatch ./...

Verbose Test Output

bash
$# Run the tests with verbose output.
$go test -v
$# Log verbose output and test results in JSON.
$go test -json

Test Timeout

Run the tests with a timeout (Default 10m):

bash
$go test -timeout 10s

Run Tests in Parallel

The Parallel method can be used to run the test in parallel with other parallel tests.

go
func TestFoo(t *testing.T) {
t.Parallel()
}

By default the number of tests run in parallel is equal to GOMAXPROCS (number of CPUs), the -parallel flag can be used to define this number explicitly.

bash
$go test -parallel 16

Comparisons

The equality operators == and != can be used to compare basic types like integers, floats, strings and booleans.

go
func TestSomething(t *testing.T) {
if want != got {
t.Errorf("want %v, got %v", want, got)
}
}

Errors can be comapred against nil.

go
func TestSomething(t *testing.T) {
if err != nil {
t.Errorf("expected no error, got %v", err)
}
if err == nil {
t.Error("expected an error, got nil")
}
}

Specific errors can be checked using errors.Is and errors.As.

go
import (
"testing"
"errors"
)
func TestSomething(t *testing.T) {
// Check the error type.
if errors.Is(err, ErrSomeError) {
t.Error("got error of type ErrSomeError")
}
// If the error value is needed for the test.
var errVal ErrSomeError
if errors.As(err, &ErrSomeError) {
t.Errorf("got error code %d", errVal.Code)
}
}

The reflect.DeepEqual function can be used to compare slices, maps, and structs.

go
import (
"testing"
"reflect"
)
func TestSomething(t *testing.T) {
if !reflect.DeepEqual(want, got) {
t.Errorf("want %v, got %v", want, got)
}
}

The cmp package is a better alternative to reflect.DeepEqual for comparing values and displaying the difference.

go
import (
"testing"
"github.com/google/go-cmp/cmp"
)
func TestSomething(t *testing.T) {
if cmp.Equal(want, got) {
t.Error(cmp.Diff(want, got)) // Display the difference
}
}

Unlike reflect.DeepEqual, unexported struct fields are not compared by default and result in a panic unless they are ignored or explicitly compared.

go
import (
"testing"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
)
// Struct with unexported fields.
type Example struct {
a, b, c int
Foo, Bar string
}
func TestSomething(t *testing.T) {
// Ignore all unexported struct fields.
if !cmp.Equal(want, got, cmpopts.IgnoreUnexported(Example{})) {
t.Error(cmp.Diff(want, got))
}
// Forcibly introspect unexported fields of the specified structs.
if !cmp.Equal(want, got, cmp.AllowUnexported(Example{})) {
t.Error(cmp.Diff(want, got))
}
}

Subtests

The Run method allows defining subtests without having to define separate functions.

go
func TestSomething(t *testing.T) {
// The first argument is the name of the subtest.
// `Run` runs the function in a separate goroutine and blocks until it returns.
t.Run("Subtest 1", func(t *testing.T) {
// ...
})
t.Run("Subtest 2", func(t *testing.T) {})
}

Each subtest has a unique name, a combination of the top level test name and the name passed to Run separated by a slash.

The name can be used to run individual subtests using the -run flag:

bash
$go test -run TestSomething/Subtest_1
$go test -run TestSomething/Subtest_2

Subtests provide a way to share common setup and tear down code.

go
func TestSomething(t *testing.T) {
// Setup code
v := Setup()
t.Run("Subtest 1", func(t *testing.T) {})
t.Run("Subtest 2", func(t *testing.T) {})
t.Run("Subtest 3", func(t *testing.T) {})
// Teardown code
TearDown(v)
}

Parallel Subtests

Subtests can be used to run a group of tests in parallel with each other but not with other parallel tests.

go
// The outer test will not complete until all the parallel tests have completed.
func TestParallelSubtests(t *testing.T) {
t.Run("Subtest 1", func(t *testing.T) {
t.Prallel()
})
t.Run("Subtest 2", func(t *testing.T) {
t.Prallel()
})
t.Run("Subtest 3", func(t *testing.T) {
t.Prallel()
})
}

Parallel tests that share common resources can be grouped together to wait for them to complete before cleaning up the shared resources.

go
func TestParallelGroup(t *testing.T) {
// Setup code
v := Setup()
// This subtest will not return until its parallel subtests complete.
t.Run("Group", func(t *testing.T) {
t.Run("Test 1", func(t *testing.T) { t.Prallel() })
t.Run("Test 2", func(t *testing.T) { t.Prallel() })
t.Run("Test 3", func(t *testing.T) { t.Prallel() })
})
// Teardown code
TearDown(v)
}

Table Driven Tests

A series of related checks can be implemented by looping over a slice of test cases:

go
func TestAdd(t *testing.T) {
// A struct that defines the test inputs and expected output.
type testCase struct {
a, b int // The test inputs
want int // The expected output
}
// The test cases to run (slice of testCase).
testCases := []testCase{
{a: 10, b: 7, want: 17},
{a: -10, b: 20, want: 10},
{a: -1, b: 0, want: -1},
{a: -1, b: 100, want: 99},
// Adding new tests later is easier.
}
for _, tc := range testCases {
got := calculator.Add(tc.a, tc.b)
if tc.want != got {
// Display the function name and inputs in the failure message.
t.Errorf("Add(%d, %d): want %d, got %d", tc.a, tc.b, tc.want, got)
}
}
}

Using an anonymous struct literal:

go
func TestAdd(t *testing.T) {
// An anonymous struct literal can be used instead of creating a named type.
testCases := []struct {
a, b int
want int
}{
{a: 10, b: 7, want: 17},
{a: -10, b: 20, want: 10},
{a: -1, b: 0, want: -1},
{a: -1, b: 100, want: 99},
}
// ...
}

Naming test cases:

go
func TestSplit(t *testing.T) {
// Using a map to name the test cases and display the name on failure.
// Maps are unordered, the tests are going to run in a different order each time.
testCases := map[string]struct {
input string
sep string
want []string
}{
"space": {input: "a b c", sep: " ", want: []string{"a", "b", "c"}},
"comma": {input: "a,b,c", sep: ",", want: []string{"a", "b", "c"}},
"wrong sep": {input: "a,b,c", sep: "/", want: []string{"a,b,c"}},
}
// Loop over the test cases and run each one as a subtest.
for name, tc := range testCases {
// Use the name as the subtest name.
t.Run(name, func(t *testing.T) {
got := strings.Split(tc.input, tc.sep)
if !cmp.Equal(tc.want, got) {
t.Error(cmp.Diff(tc.want, got)) // Display the diff on failure
}
})
}
}

Running individual subtests:

bash
$go test -run TestSplit/space
$go test -run TestSplit/comma
$go test -run TestSplit/wrong_sep

Test Helpers

The Helper method marks the calling function as a test helper function.

Failures will be reported at the calling test function and not in the helper function which will be skipped.

go
// Example helper function.
func assertStringEqual(t testing.TB, want, got string) {
t.Helper() // Mark this function as a test helper
if want != got {
t.Errorf("want %q, got %q", want, got)
}
}
// Test function using the helper function.
func TestSomething(t *testing.T) {
// ...
assertStringEqual(t, want, got) // Failure will be reported at this line
}

The testing.TB is the interface common to testing.T, testing.B, and testing.F, it makes the helper function usable from test, benchmark and fuzz test functions.

Cleanup

The Cleanup method registers a function to be called when the test and all its subtests complete.

Cleanup functions are called in LIFO order (Last in first out).

go
func TestSomething(t *testing.T) {
resource := createResource()
t.Cleanup(func() {
// Cleanup after the test.
resource.Close()
})
}

Cleanup functions can be used in helper functions.

go
func createTestResource(t *testing.T) *Resource {
resource := createResource()
// The cleanup will happen after the test has completed.
// If the `defer` statement was used, the function will be called when
// the helper function returns and not when the test completes.
t.Cleanup(func() {
resource.Close()
})
return resource
}

Temporary Directory

The TempDir method can be used to create a temporary directory that will be automatically removed on cleanup when the test and all its subtests complete.

go
func TestSomething(t *testing.T) {
temp1 := t.TempDir()
temp2 := t.TempDir() // Each call returns a unique directory
}

Test Data

A directory named testdata can be used to hold data needed by the tests.

The go tool will ignore directories named testdata.

go
func TestSomething(t *testing.T) {
// Read a file used by the test.
f, err := os.ReadFile("testdata/data.json")
// ...
}

Skipping

bash
$# The -run flag is followed by a regexp to match the functions to skip.
$go test -skip NameToMatch

Tests or benchmarks may be skipped at run time by calling the Skip() method:

go
func TestTimeConsuming(t *testing.T) {
// `testing.Short()` is `true` when the tests are run in short mode.
if testing.Short() {
t.Skip("skipping test in short mode.")
}
}

Run the tests in short mode:

bash
$go test -short

Race Detector

Run the tests with the race detector to detect race conditions:

bash
$$ go test -race

List Functions

List all the tests, benchmarks, fuzz tests and examples without running them:

bash
$# The -list flag is followed by a regexp to match the functions to list.
$go test -list .

Fuzzing

Fuzz test function signature:

go
// Starts with the word `Fuzz`.
// Where `Xxx` does not start with a lowercase letter.
func FuzzXxx(f *testing.F)

Example:

go
func FuzzFoo(f *testing.F) {
// Add the seed inputs to the seed corpus for the fuzz test.
// Seed inputs are optional but they can be used to guide the fuzzing engine.
// The arguments must match the arguments for the fuzz target.
// The seed inputs are run by default even when not fuzzing.
f.Add(5, "hello")
f.Add(100, "world") // Can be called as many times as we want.
// The function passed to `f.Fuzz` is the fuzz target.
// It takes a `*testing.T` parameter followed by one or more
// parameters for random inputs.
f.Fuzz(func(t *testing.T, i int, s string) {
out, err := Foo(i, s)
// When fuzzing, we can't predict the expected output, since we don't have
// control over the inputs, instead we can verify the properties of the output.
if err != nil && out != "" {
t.Errorf("%q, %v", out, err)
}
})
}

Run the fuzz test:

bash
$# Without the -fuzz flag, the fuzz test will run with the seed inputs only.
$go test
$# Run the fuzz test with randomly generated inputs.
$# The -fuzz flag is followed by a regexp to match the function to run.
$# Fuzzing will not run if the regexp matches more than one fuzz test.
$go test -fuzz .
$# Match a single fuzz test function to run.
$# The fuzz test will never terminate if no failures were found.
$# The test can be terminated with Ctrl-C.
$go test -fuzz FuzzSomething
$# Run the fuzz target for a specified duration (Default: forever).
$go test -fuzz . -fuzztime 1h30m
$# Run the fuzz target N times.
$go test -fuzz . -fuzztime 1000x

When fuzzing is enabled, the fuzz target is called with arguments generated by repeatedly making random changes to the seed inputs.

When the fuzz target fails for a given input, the inputs that caused the failure are written to a seed corpus file in the testdata/fuzz/<Name> directory within the package.

When fuzzing is disabled, the fuzz target is called with the seed inputs added with the Add method and seed inputs from the testdata/fuzz/<Name> directory.

bash
$# The entries saved to the `testdata/fuzz` directory are run by default
$# whether fuzzing or not.
$# In this mode the fuzz test acts like a regular test.
$go test

Benchmarks

Benchmark function signature:

go
// Starts with the word `Benchmark`.
// Where `Xxx` does not start with a lowercase letter.
func BenchmarkXxx(b *testing.B)

The benchmark function must run the target code b.N times:

go
func BenchmarkRepeat(b *testing.B) {
// The benchmark code is executed `b.N` times.
// The time taken to run the code is also measured.
for i := 0; i < b.N; i++ {
SomeFunction() // Code to benchmark
}
}

The timer can be reset if the benchmark needs some expensive setup before running:

go
func BenchmarkExpensiveSetup(b *testing.B) {
// Expensive setup code that might take time
// that should not be included in the benchmark time.
a := ExpensiveSetup()
b.ResetTimer() // Reset the timer after an expensive setup
for i := 0; i < b.N; i++ {
a.DoSomething()
}
}

By default, no benchmark functions are run with go test.

Run the benchmarks with the -bench flag:

bash
$# The -bench flag is followed by a regexp to match the functions to run.
$# In this case run all the benchmarks.
$go test -bench .
$# Run a specific benchmark function.
$go test -bench BenchmarkFunctionName
$# Run iterations of each benchmark to take a specific time.
$go test -bench . -benchtime 10m30s # Default 1s
$# Run the benchmark N times.
$go test -bench . -benchtime 1000x

Examples

Example functions are used to demonstrate the usage of a package’s components.

Instead of reporting success or failure, example functions print the output to os.Stdout.

Godoc displays the examples in the documentation of the package.

go
// Example function that is compiled but not executed.
func ExampleXxx() {
// ...
}
// If the last comment in the function starts with `Output:` then
// the function is executed and the output is checked against the comment.
// The output comparison ignores leading and trailing spaces.
func ExampleXxx() {
// Output: expected output
}
// An example with no text after `Output:` is executed and expected
// to produce no output.
func ExampleXxx() {
// Output:
}
// To comment `Unordered output:` can be used to ignore the order of the lines.
func ExampleXxx() {
// Unordered output:
// 3
// 1
// 2
}

Examples:

go
// This example function is compiled but not executed (No output comment).
func ExamplePrintln() {
fmt.Println("Hello")
}
// This example function is executed and its output will be checked.
func ExampleAdd() {
sum := calculator.Add(1, 5)
fmt.Println(sum)
// Output: 6
}
// This example function is executed and its output will be checked but the
// lines order is ignored.
func ExampleMap() {
// Maps are not ordered
m := map[int]string{1: "a", 2: "b", 3: "c"}
for k, v := range m {
fmt.Println(k, v)
}
// Unordered output:
// 3 c
// 2 b
// 1 a
}

Naming conventions:

go
// Example for the package.
func Example() { ... }
// Example for a function F.
func ExampleF() { ... }
// Example for a type T.
func ExampleT() { ... }
// Example for a method M on type T.
func ExampleT_M() { ... }
// Multiple example functions may be provided by appending a suffix (lowercase).
func Example_suffix() { ... }
func ExampleF_suffix() { ... }
func ExampleT_suffix() { ... }
func ExampleT_M_suffix() { ... }

Test Coverage

Coverage analysis can be enabled with the -cover flag:

bash
$# Run the tests and print the coverage results.
$go test -cover

Detailed coverage information:

bash
$# Save the coverage results to a file.
$go test -coverprofile=coverage.out
$# Print detailed results.
$go tool cover -func=coverage.out
$# Or view the coverage results as an HTML page.
$go tool cover -html=coverage.out