Go Testing - mdawar.dev

Test Files

Test files have file names ending in _test.go
Test files may contain: tests, benchmarks, examples and fuzz tests
The *_test.go files are excluded from regular package builds

Test Package

A test file can be in the same package as the one being tested.

// A test file in the same package may refer to unexported identifiers.
package calculator

It’s a good practice for the test file to be in a corresponding package with the suffix _test.

// A test file in a separate `_test` package, may only use the exported identifiers.
package calculator_test

// The package being tested must be imported
import (
  "calculator"
)

Test Functions

Test function signature:

// Starts with the word `Test`.
// Where `Xxx` does not start with a lowercase letter.
func TestXxx(t *testing.T)

Example:

calculator_test.go

package calculator_test

import (
  "calculator" // The package being tested
  "testing"    // Standard library testing package
)

func TestAdd(t *testing.T) {
  // By convention the expected result is defined as a `want` variable
  // and the actual result as a `got` variable.
  want := 17
  got := calculator.Add(10, 7)

  // Compare the expected result with the actual result.
  if want != got {
    // Make the test fail with a formatted error message showing the
    // expected and actual values.
    t.Errorf("want %d, got %d", want, got)
  }
}

Test Failure

Tests pass by default unless we explicitly make them fail with the Error or Fatal methods.

func TestSomething(t *testing.T) {
  // Report an error and continue execution of the test.
  t.Error("expected an error, got nil")
  t.Errorf("want %d, got %d", want, got) // Formatted error message

  // Report an error and abort the test.
  // Execution will continue at the next test or benchmark.
  t.Fatal(err)
  t.Fatalf("did not expect an error, got %v", err) // Formatted error message
}

Logging Text

Text can be logged with the Log and Logf methods.

func TestSomething(t *testing.T) {
  // Record the text in the error log (newline is added if not provided).
  // The text will be printed if the test fails or the verbose flag `-v` is set.
  // For benchmarks, the text is always printed.
  t.Log("report a useful message")
  t.Logf("another useful message with data: %v", data) // Formatted error message
}

Run Tests

The tests are run in 2 different modes:

1. Local Directory Mode

When go test is invoked with no package arguments, it runs the tests in the current directory.

bash

$go test

2. Package List Mode

When go test is invoked with explicit package arguments, it runs tests for the listed packages.

bash

$# Run the tests for a specific package.
$go test packageName
$go test moduleName/subPackageName
$# Run the tests for multiple packages.
$go test package1 package2 package3
$# Run the tests for the package in the current directory.
$go test .
$# Run the tests for the current package and all the sub-packages.
$go test ./...

In the package list mode, successful test results are cached.

To disable test caching:

bash

$# Disable test caching with `-count 1`.
$go test -count 1 ./...

Run Specific Tests

bash

$# The -run flag is followed by a regexp to match the functions to run.
$go test -run NameToMatch
$# Match the functions to run in the current package and all sub-packages.
$go test -run NameToMatch ./...

Verbose Test Output

bash

$# Run the tests with verbose output.
$go test -v
$# Log verbose output and test results in JSON.
$go test -json

Test Timeout

Run the tests with a timeout (Default 10m):

bash

$go test -timeout 10s

Run Tests in Parallel

The Parallel method can be used to run the test in parallel with other parallel tests.

func TestFoo(t *testing.T) {
  t.Parallel()
}

By default the number of tests run in parallel is equal to GOMAXPROCS (number of CPUs), the -parallel flag can be used to define this number explicitly.

bash

$go test -parallel 16

Comparisons

The equality operators == and != can be used to compare basic types like integers, floats, strings and booleans.

func TestSomething(t *testing.T) {
  if want != got {
    t.Errorf("want %v, got %v", want, got)
  }
}

Errors can be comapred against nil.

func TestSomething(t *testing.T) {
  if err != nil {
    t.Errorf("expected no error, got %v", err)
  }

  if err == nil {
    t.Error("expected an error, got nil")
  }
}

Specific errors can be checked using errors.Is and errors.As.

import (
  "testing"
  "errors"
)

func TestSomething(t *testing.T) {
  // Check the error type.
  if errors.Is(err, ErrSomeError) {
    t.Error("got error of type ErrSomeError")
  }

  // If the error value is needed for the test.
  var errVal ErrSomeError

  if errors.As(err, &ErrSomeError) {
    t.Errorf("got error code %d", errVal.Code)
  }
}

The reflect.DeepEqual function can be used to compare slices, maps, and structs.

import (
  "testing"
  "reflect"
)

func TestSomething(t *testing.T) {
  if !reflect.DeepEqual(want, got) {
    t.Errorf("want %v, got %v", want, got)
  }
}

The cmp package is a better alternative to reflect.DeepEqual for comparing values and displaying the difference.

import (
  "testing"

  "github.com/google/go-cmp/cmp"
)

func TestSomething(t *testing.T) {
  if cmp.Equal(want, got) {
    t.Error(cmp.Diff(want, got)) // Display the difference
  }
}

Unlike reflect.DeepEqual, unexported struct fields are not compared by default and result in a panic unless they are ignored or explicitly compared.

import (
  "testing"

  "github.com/google/go-cmp/cmp"
  "github.com/google/go-cmp/cmp/cmpopts"

)

// Struct with unexported fields.
type Example struct {
  a, b, c  int
  Foo, Bar string
}

func TestSomething(t *testing.T) {
  // Ignore all unexported struct fields.
  if !cmp.Equal(want, got, cmpopts.IgnoreUnexported(Example{})) {
    t.Error(cmp.Diff(want, got))
  }

  // Forcibly introspect unexported fields of the specified structs.
  if !cmp.Equal(want, got, cmp.AllowUnexported(Example{})) {
    t.Error(cmp.Diff(want, got))
  }
}

Subtests

The Run method allows defining subtests without having to define separate functions.

func TestSomething(t *testing.T) {
  // The first argument is the name of the subtest.
  // `Run` runs the function in a separate goroutine and blocks until it returns.
  t.Run("Subtest 1", func(t *testing.T) {
    // ...
  })

  t.Run("Subtest 2", func(t *testing.T) {})
}

Each subtest has a unique name, a combination of the top level test name and the name passed to Run separated by a slash.

The name can be used to run individual subtests using the -run flag:

bash

$go test -run TestSomething/Subtest_1
$go test -run TestSomething/Subtest_2

Subtests provide a way to share common setup and tear down code.

func TestSomething(t *testing.T) {
  // Setup code
  v := Setup()

  t.Run("Subtest 1", func(t *testing.T) {})
  t.Run("Subtest 2", func(t *testing.T) {})
  t.Run("Subtest 3", func(t *testing.T) {})

  // Teardown code
  TearDown(v)
}

Parallel Subtests

Subtests can be used to run a group of tests in parallel with each other but not with other parallel tests.

// The outer test will not complete until all the parallel tests have completed.
func TestParallelSubtests(t *testing.T) {
  t.Run("Subtest 1", func(t *testing.T) {
    t.Prallel()
  })

  t.Run("Subtest 2", func(t *testing.T) {
    t.Prallel()
  })

  t.Run("Subtest 3", func(t *testing.T) {
    t.Prallel()
  })
}

Parallel tests that share common resources can be grouped together to wait for them to complete before cleaning up the shared resources.

func TestParallelGroup(t *testing.T) {
  // Setup code
  v := Setup()

  // This subtest will not return until its parallel subtests complete.
  t.Run("Group", func(t *testing.T) {
    t.Run("Test 1", func(t *testing.T) { t.Prallel() })
    t.Run("Test 2", func(t *testing.T) { t.Prallel() })
    t.Run("Test 3", func(t *testing.T) { t.Prallel() })
  })

  // Teardown code
  TearDown(v)
}

Table Driven Tests

A series of related checks can be implemented by looping over a slice of test cases:

func TestAdd(t *testing.T) {
  // A struct that defines the test inputs and expected output.
  type testCase struct {
    a, b int // The test inputs
    want int // The expected output
  }

  // The test cases to run (slice of testCase).
  testCases := []testCase{
    {a: 10, b: 7, want: 17},
    {a: -10, b: 20, want: 10},
    {a: -1, b: 0, want: -1},
    {a: -1, b: 100, want: 99},
    // Adding new tests later is easier.
  }

  for _, tc := range testCases {
    got := calculator.Add(tc.a, tc.b)

    if tc.want != got {
      // Display the function name and inputs in the failure message.
      t.Errorf("Add(%d, %d): want %d, got %d", tc.a, tc.b, tc.want, got)
    }
  }
}

Using an anonymous struct literal:

func TestAdd(t *testing.T) {
  // An anonymous struct literal can be used instead of creating a named type.
  testCases := []struct {
    a, b int
    want int
  }{
    {a: 10, b: 7, want: 17},
    {a: -10, b: 20, want: 10},
    {a: -1, b: 0, want: -1},
    {a: -1, b: 100, want: 99},
  }

  // ...
}

Naming test cases:

func TestSplit(t *testing.T) {
  // Using a map to name the test cases and display the name on failure.
  // Maps are unordered, the tests are going to run in a different order each time.
  testCases := map[string]struct {
    input string
    sep   string
    want  []string
  }{
    "space":     {input: "a b c", sep: " ", want: []string{"a", "b", "c"}},
    "comma":     {input: "a,b,c", sep: ",", want: []string{"a", "b", "c"}},
    "wrong sep": {input: "a,b,c", sep: "/", want: []string{"a,b,c"}},
  }

  // Loop over the test cases and run each one as a subtest.
  for name, tc := range testCases {
    // Use the name as the subtest name.
    t.Run(name, func(t *testing.T) {
      got := strings.Split(tc.input, tc.sep)

      if !cmp.Equal(tc.want, got) {
        t.Error(cmp.Diff(tc.want, got)) // Display the diff on failure
      }
    })
  }
}

Running individual subtests:

bash

$go test -run TestSplit/space
$go test -run TestSplit/comma
$go test -run TestSplit/wrong_sep

Test Helpers

The Helper method marks the calling function as a test helper function.

Failures will be reported at the calling test function and not in the helper function which will be skipped.

// Example helper function.
func assertStringEqual(t testing.TB, want, got string) {
  t.Helper() // Mark this function as a test helper

  if want != got {
    t.Errorf("want %q, got %q", want, got)
  }
}

// Test function using the helper function.
func TestSomething(t *testing.T) {
  // ...
  assertStringEqual(t, want, got) // Failure will be reported at this line
}

The testing.TB is the interface common to testing.T, testing.B, and testing.F, it makes the helper function usable from test, benchmark and fuzz test functions.

Cleanup

The Cleanup method registers a function to be called when the test and all its subtests complete.

Cleanup functions are called in LIFO order (Last in first out).

func TestSomething(t *testing.T) {
  resource := createResource()

  t.Cleanup(func() {
    // Cleanup after the test.
    resource.Close()
  })
}

Cleanup functions can be used in helper functions.

func createTestResource(t *testing.T) *Resource {
  resource := createResource()

  // The cleanup will happen after the test has completed.
  // If the `defer` statement was used, the function will be called when
  // the helper function returns and not when the test completes.
  t.Cleanup(func() {
    resource.Close()
  })

  return resource
}

Temporary Directory

The TempDir method can be used to create a temporary directory that will be automatically removed on cleanup when the test and all its subtests complete.

func TestSomething(t *testing.T) {
  temp1 := t.TempDir()
  temp2 := t.TempDir() // Each call returns a unique directory
}

Test Data

A directory named testdata can be used to hold data needed by the tests.

The go tool will ignore directories named testdata.

func TestSomething(t *testing.T) {
  // Read a file used by the test.
  f, err := os.ReadFile("testdata/data.json")
  // ...
}

Skipping

bash

$# The -run flag is followed by a regexp to match the functions to skip.
$go test -skip NameToMatch

Tests or benchmarks may be skipped at run time by calling the Skip() method:

func TestTimeConsuming(t *testing.T) {
  // `testing.Short()` is `true` when the tests are run in short mode.
  if testing.Short() {
    t.Skip("skipping test in short mode.")
  }
}

Run the tests in short mode:

bash

$go test -short

Race Detector

Run the tests with the race detector to detect race conditions:

bash

$$ go test -race

List Functions

List all the tests, benchmarks, fuzz tests and examples without running them:

bash

$# The -list flag is followed by a regexp to match the functions to list.
$go test -list .

Fuzzing

Fuzz test function signature:

// Starts with the word `Fuzz`.
// Where `Xxx` does not start with a lowercase letter.
func FuzzXxx(f *testing.F)

Example:

func FuzzFoo(f *testing.F) {
  // Add the seed inputs to the seed corpus for the fuzz test.
  // Seed inputs are optional but they can be used to guide the fuzzing engine.
  // The arguments must match the arguments for the fuzz target.
  // The seed inputs are run by default even when not fuzzing.
  f.Add(5, "hello")
  f.Add(100, "world") // Can be called as many times as we want.

  // The function passed to `f.Fuzz` is the fuzz target.
  // It takes a `*testing.T` parameter followed by one or more
  // parameters for random inputs.
  f.Fuzz(func(t *testing.T, i int, s string) {
    out, err := Foo(i, s)

    // When fuzzing, we can't predict the expected output, since we don't have
    // control over the inputs, instead we can verify the properties of the output.
    if err != nil && out != "" {
      t.Errorf("%q, %v", out, err)
    }
  })
}

Run the fuzz test:

bash

$# Without the -fuzz flag, the fuzz test will run with the seed inputs only.
$go test
$# Run the fuzz test with randomly generated inputs.
$# The -fuzz flag is followed by a regexp to match the function to run.
$# Fuzzing will not run if the regexp matches more than one fuzz test.
$go test -fuzz .
$# Match a single fuzz test function to run.
$# The fuzz test will never terminate if no failures were found.
$# The test can be terminated with Ctrl-C.
$go test -fuzz FuzzSomething
$# Run the fuzz target for a specified duration (Default: forever).
$go test -fuzz . -fuzztime 1h30m
$# Run the fuzz target N times.
$go test -fuzz . -fuzztime 1000x

When fuzzing is enabled, the fuzz target is called with arguments generated by repeatedly making random changes to the seed inputs.

When the fuzz target fails for a given input, the inputs that caused the failure are written to a seed corpus file in the testdata/fuzz/<Name> directory within the package.

When fuzzing is disabled, the fuzz target is called with the seed inputs added with the Add method and seed inputs from the testdata/fuzz/<Name> directory.

bash

$# The entries saved to the `testdata/fuzz` directory are run by default
$# whether fuzzing or not.
$# In this mode the fuzz test acts like a regular test.
$go test

Benchmarks

Benchmark function signature:

// Starts with the word `Benchmark`.
// Where `Xxx` does not start with a lowercase letter.
func BenchmarkXxx(b *testing.B)

The benchmark function must run the target code b.N times:

func BenchmarkRepeat(b *testing.B) {
	// The benchmark code is executed `b.N` times.
	// The time taken to run the code is also measured.
	for i := 0; i < b.N; i++ {
		SomeFunction() // Code to benchmark
	}
}

The timer can be reset if the benchmark needs some expensive setup before running:

func BenchmarkExpensiveSetup(b *testing.B) {
  // Expensive setup code that might take time
  // that should not be included in the benchmark time.
  a := ExpensiveSetup()

  b.ResetTimer() // Reset the timer after an expensive setup

  for i := 0; i < b.N; i++ {
    a.DoSomething()
  }
}

By default, no benchmark functions are run with go test.

Run the benchmarks with the -bench flag:

bash

$# The -bench flag is followed by a regexp to match the functions to run.
$# In this case run all the benchmarks.
$go test -bench .
$# Run a specific benchmark function.
$go test -bench BenchmarkFunctionName
$# Run iterations of each benchmark to take a specific time.
$go test -bench . -benchtime 10m30s # Default 1s
$# Run the benchmark N times.
$go test -bench . -benchtime 1000x

Examples

Example functions are used to demonstrate the usage of a package’s components.

Instead of reporting success or failure, example functions print the output to os.Stdout.

Godoc displays the examples in the documentation of the package.

// Example function that is compiled but not executed.
func ExampleXxx() {
  // ...
}

// If the last comment in the function starts with `Output:` then
// the function is executed and the output is checked against the comment.
// The output comparison ignores leading and trailing spaces.
func ExampleXxx() {
  // Output: expected output
}

// An example with no text after `Output:` is executed and expected
// to produce no output.
func ExampleXxx() {
  // Output:
}

// To comment `Unordered output:` can be used to ignore the order of the lines.
func ExampleXxx() {
  // Unordered output:
  // 3
  // 1
  // 2
}

Examples:

// This example function is compiled but not executed (No output comment).
func ExamplePrintln() {
  fmt.Println("Hello")
}

// This example function is executed and its output will be checked.
func ExampleAdd() {
  sum := calculator.Add(1, 5)
  fmt.Println(sum)
  // Output: 6
}

// This example function is executed and its output will be checked but the
// lines order is ignored.
func ExampleMap() {
  // Maps are not ordered
  m := map[int]string{1: "a", 2: "b", 3: "c"}

  for k, v := range m {
    fmt.Println(k, v)
  }

  // Unordered output:
  // 3 c
  // 2 b
  // 1 a
}

Naming conventions:

// Example for the package.
func Example() { ... }

// Example for a function F.
func ExampleF() { ... }

// Example for a type T.
func ExampleT() { ... }

// Example for a method M on type T.
func ExampleT_M() { ... }

// Multiple example functions may be provided by appending a suffix (lowercase).
func Example_suffix() { ... }
func ExampleF_suffix() { ... }
func ExampleT_suffix() { ... }
func ExampleT_M_suffix() { ... }

Test Coverage

Coverage analysis can be enabled with the -cover flag:

bash

$# Run the tests and print the coverage results.
$go test -cover

Detailed coverage information:

bash

$# Save the coverage results to a file.
$go test -coverprofile=coverage.out
$# Print detailed results.
$go tool cover -func=coverage.out
$# Or view the coverage results as an HTML page.
$go tool cover -html=coverage.out

Go - Testing

Test Files

Test Package

Test Functions

Test Failure

Logging Text

Run Tests

1. Local Directory Mode

2. Package List Mode

Run Specific Tests

Verbose Test Output

Test Timeout

Run Tests in Parallel

Comparisons

Subtests

Parallel Subtests

Table Driven Tests

Test Helpers

Cleanup

Temporary Directory

Test Data

Skipping

Race Detector

List Functions

Fuzzing

Benchmarks

Examples

Test Coverage