Bazel Tests

Scope

This document describes how to define, build and run tests that exclusively exist in the Bazel graph.

Fuchsia only supports Bazel host tests that meet the following requirements:

These tests must be buildable exclusively with Bazel; i.e. they do not depend on artifacts generated by a prior Ninja invocation.
These tests only include binaries and runtime files that can run on a compatible host system that is NOT connected to any Fuchsia device or emulator.

fxbug.dev/546173288 tracks supporting target tests in Bazel.

Defining Fuchsia host tests in Bazel

Regular Bazel test rules (e.g. rustc_test, go_test) are not compatible with Fuchsia test runners. They will not be visible to fx test and will not be run by infra builders.

Hence, to define a Fuchsia-compatible host test in Bazel:

Use language-specific host test wrapper from //build/bazel/rules/host_tests:host_<xxx>_test.bzl, such as host_rustc_test(), host_go_test(), etc.

These are designed to be drop-in replacements for regular Bazel test rules, with extra arguments like test_data, test_args, so Fuchsia test runners can invoke them with the right runtime environment and arguments.
Only if no language-specific wrappers are available for your test, use the generic host_test() wrapper from //build/bazel/rules/host_tests:host_test.bzl. For example:
```
load("//build/bazel/rules/host_tests:host_test.bzl", "host_test")
load("@rules_sh//sh:sh_binary.bzl", "sh_binary")

sh_binary(
    name = "test_bin",
    ...
)

host_test(
    name = "shell_test",
    binary = ":test_bin",
)
```
Note: ALWAYS prefer using language-specific wrappers if available, as they also implement extra Fuchsia-specific features, which are critical for parsing test results in infra.
Do NOT assume the execution location of a test binary (e.g. from out/default), as it is launched from a directory whose exact path is not predictable.

Bazel host tests MUST be hermetic and list all their runtime dependencies explicitly, just like GN host tests. Runtime dependencies can be provided through one of the mechanisms:
- As test_data dependencies to a host_test_data_files() or host_test_data_map() target (see //build/bazel/rules/host_tests:host_test_data.bzl). These make the files available at runtime at fixed locations, relative to the test's execution directory, similar to the GN host_test_data() template.
- As data attribute on a host_xxx_test() target. In this case, the files will be added to the test's runfiles, and can be accessed using a runfiles library.
Use test_suite() instead of filegroup() to define groups of tests. Test suites can depend on tests and other test suites.

Warning: Bazel skips tests from test_suites if they are incompatible with the current configuration. See Bazel test compatibility for more details.

Exporting Bazel host tests to `fx test` and infra builders

Defining the host_xxx_test() target alone will NOT make it visible to fx test and, more importantly, to infra builders.

To make a Bazel host test visible to fx test and infra builders, write a bazel_test_suite target in GN that lists the Bazel labels of one or more Bazel host tests or suites, and add that target to a "tests" group in GN. For example:

//src/foo/bar/BUILD.bazel:

load("//build/bazel/platforms:constraints.bzl", "HOST_OS_CONSTRAINTS")

test_suite(
  name = "host_tests",
  host_tests = [
    "//src/foo/bar/lib:lib_tests",
    "//src/foo/bar/util:util_tests",
  ]
  target_compatible_with = HOST_OS_CONSTRAINTS,
)

//src/foo/bar/BUILD.gn:

import("//build/bazel/bazel_test_suite.gni")

bazel_test_suite("bazel_tests") {
  host_tests = [
    # This references the target in BUILD.bazel.
    "//src/foo/bar:host_tests",
  ]
}

group("tests") {
  deps = [
    ":bazel_tests",
    ...
  ]
}

This will cause entries for the Bazel tests to appear in out/default/tests.json. When in doubt, examine the content of this file after fx set or fx gen. The label field for Bazel tests begins with @.

During development, it is possible to build and run test targets once they are defined, before they are exported as described above. See the Running tests directly with Bazel section below.

Best practices

It's recommended to avoid deep nesting of Bazel test_suite targets. Instead, use just one test_suite Bazel target and one corresponding bazel_test_suite GN target per source code sub-area (e.g. host tool or Fuchsia component).

Running Fuchsia Bazel host tests

Local testing

Running tests with `fx test`

To run Bazel host tests with fx test, they MUST be exported by following the instructions in the Exporting Bazel host tests to fx test and infra builders section above.

After the tests are exported, use fx test --host <name_or_label> to build and run a Bazel host test locally, same as with GN-defined host tests.

The only difference is that the test will be built on demand by invoking Bazel directly, skipping Ninja entirely.

Note that when using fx test:

<name_or_label> will be subject to fuzzy matching, and fx test will report multiple candidates in case of ambiguity.
<name_or_label> cannot reference a Bazel test_suite() label, just like it cannot reference a GN group() one.

See the User guide for fx test for more details about how to use fx test.

Running tests directly with `fx bazel test`

You can use fx bazel test --config=host <target_pattern> to build and run tests locally directly with Bazel. This does not require the extra plumbing to make tests visible to fx test as described in the previous section, and has the following advantages:

<target_pattern> is a set of Bazel target patterns that can match "all test targets in a given package", in which case Bazel will skip non-test targets automatically.
<target_pattern> can match test_suite() labels which can be convenient to run a set of host tests repeatedly during development.
bazel test caches test results, and will only re-run the tests impacted by your latest changes since the last invocation.

(Use --nocache_test_results flag to disable this).
<target_pattern> can reference labels that are not visible in tests.json. This can be handy during development when adding new tests.

Infra testing

Only host tests that are listed in tests.json can be launched on infra test bots. This applies equally to GN and Bazel host tests. So you MUST ensure they are visible in tests.json by following the Exporting Bazel host tests to fx test and infra builders section above, otherwise they will NOT be continuously tested on infra.

Infra builds Bazel tests using fint, which directly calls bazel build after running ninja, to build all Bazel tests that are listed in bazel_test_suite GN targets.

There is no explicit distinction between GN and Bazel-defined host tests when running them on infra test runners, or when collecting results. In particular, infra does not run tests using bazel test. Instead, it takes the binaries produced by bazel build and runs them directly, just like it does with host tests from ninja.

Debug symbol propagation

Host test binaries compiled with debug symbols must have their unstripped ELF binaries registered so that tooling can symbolize backtraces and correlate LLVM code coverage profiles (-profile-correlate=binary).

For general background on how Fuchsia manages Bazel debug symbols, see Technical Note on Debug Symbol Generation.

Authoring host test rules with debug symbols

The generic host_test() rule exposes the unstripped_binary field in its FuchsiaHostTestInfo provider:

If unstripped_binary is not explicitly set, host_test() inspects the underlying binary attribute:
- If binary provides DebugPackageInfo (C++ targets), it uses DebugPackageInfo.unstripped_file.
- If binary provides CrateInfo (Rust targets), it uses CrateInfo.output.
If the test uses a wrapper script launcher (such as rust_test_parser in host_rustc_test()), the macro MUST explicitly forward the original unstripped binary target via unstripped_binary:
```
host_test(
    name = name,
    binary = wrapper_script,
    unstripped_binary = ":" + binary_name,
    ...
)
```

Propagation to infra and debuginfod

When Bazel host tests are exported to the GN build:

Test Query: bazel_tests_utils.py runs a bazel cquery using //build/bazel/starlark/FuchsiaHostTestInfo.cquery, extracting the execroot path of unstripped_binary for all registered host tests.
Host Test Manifest: The paths are normalized relative to the Ninja build directory and written to ${root_build_dir}/bazel_host_tests.debug_symbols.json.
GN Build API Integration: //:bazel_test_suites in BUILD.gn attaches debug_symbol_manifests metadata pointing to bazel_host_tests.debug_symbols.json, which is consumed by the build_api_module("debug_symbols") target.
Artifactory Upload: In CI/CQ builds, Artifactory processes debug_symbols.json and uploads the unstripped ELF binaries to cloud storage / debuginfod servers, allowing tools like covargs to resolve Build IDs.

Local development and `fx coverage`

When running coverage locally at desk, uncommitted binaries have newly generated Build IDs that are not present on debuginfod servers.

To support local symbol correlation, fx coverage runs:

python3 ${FUCHSIA_DIR}/build/bazel/scripts/copy_bazel_debug_symbols.py ${FUCHSIA_BUILD_DIR}

This script reads bazel_host_tests.debug_symbols.json, computes the GNU Build ID for each ELF binary, and copies or symlinks the unstripped binary into ${root_build_dir}/.build-id/xx/yyyyyyyy.debug. Tooling such as ffx coverage and llvm-profdata then resolves these binaries via .symbol-index.json.

FAQ

Bazel test compatibility

If any test in a test_suite() (or any of its dependencies) is incompatible with the current configuration (e.g., using target_compatible_with), that test will be skipped during evaluation of the suite instead of failing the build.

The same is true when using patterns such as //... and :all on the command line.

For more information, see the handling of incompatible targets.

When running a test_suite() or using a target pattern, developers must verify that the test(s) of interest are reported as PASSED (and not SKIPPED).