Bazel Tests

Scope

This document describes how to define, build and run tests that exclusively exist in the Bazel graph.

Fuchsia supports Bazel host tests that meet the following requirements:

  • These tests must be buildable exclusively with Bazel; i.e. they do not depend on artifacts generated by a prior Ninja invocation.

  • These tests only include binaries and runtime files that can run on a compatible host system that is NOT connected to any Fuchsia device or emulator.

Fuchsia also supports Bazel device tests, declared with fx_test() and run on a Fuchsia device or emulator. These have the same requirement of being buildable exclusively with Bazel, and additionally:

  • Aside from environments (which defaults to the build's default test environments), build_only, max_log_severity (which defaults to WARN), and test_type (for running in a non-hermetic test realm such as "starnix" or "system"), test spec fields (timeouts, etc.) cannot be customized per test yet.

fxbug.dev/564574581 tracks the remaining work for target tests in Bazel.

Defining Fuchsia host tests in Bazel

Regular Bazel test rules (e.g. rustc_test, go_test) are not compatible with Fuchsia test runners. They will not be visible to fx test and will not be run by infra builders.

Hence, to define a Fuchsia-compatible host test in Bazel:

  • Use language-specific host test wrapper from //build/bazel/rules/host_tests:host_<xxx>_test.bzl, such as host_rustc_test(), host_go_test(), etc.

    These are designed to be drop-in replacements for regular Bazel test rules, with extra arguments like test_data, test_args, so Fuchsia test runners can invoke them with the right runtime environment and arguments.

  • Only if no language-specific wrappers are available for your test, use the generic host_test() wrapper from //build/bazel/rules/host_tests:host_test.bzl. For example:

    load("//build/bazel/rules/host_tests:host_test.bzl", "host_test")
load("@rules_sh//sh:sh_binary.bzl", "sh_binary")

sh_binary(
    name = "test_bin",
    ...
)

host_test(
    name = "shell_test",
    binary = ":test_bin",
)

  • For C/C++ tests, host_cc_test() builds its binary with fx_cc_test_binary() and adds nothing else to host_test(). To also run an fx_cc_test_binary() that is already packaged as a device test on host, wrap it with host_test() instead of defining a second binary from the same sources, like a GN test() that is built in both the Fuchsia and host toolchains:

    load("//build/bazel/platforms:constraints.bzl", "HOST_OS_CONSTRAINTS")
load("//build/bazel/rules/host_tests:host_test.bzl", "host_test")

host_test(
    name = "foo_unittests_host_test",
    binary = ":foo_unittests",
    target_compatible_with = HOST_OS_CONSTRAINTS,
)

  • Do NOT assume the execution location of a test binary (e.g. from out/default), as it is launched from a directory whose exact path is not predictable.

    Bazel host tests MUST be hermetic and list all their runtime dependencies explicitly, just like GN host tests. Runtime dependencies can be provided through one of the mechanisms:

    • As test_data dependencies to a host_test_data_files() or host_test_data_map() target (see //build/bazel/rules/host_tests:host_test_data.bzl). These make the files available at runtime at fixed locations, relative to the test's execution directory, similar to the GN host_test_data() template.

    • As data attribute on a host_xxx_test() target. In this case, the files will be added to the test's runfiles, and can be accessed using a runfiles library.

  • Use test_suite() instead of filegroup() to define groups of tests. Test suites can depend on tests and other test suites.

Exporting Bazel host tests to fx test and infra builders

Defining the host_xxx_test() target alone will NOT make it visible to fx test and, more importantly, to infra builders.

To make a Bazel host test visible to fx test and infra builders, expose a "host_tests" test_suite() in its package, include that package suite in an area aggregation test_suite() target, reference the aggregation suite from a bazel_test_suite() in the same directory's BUILD.gn, and add that target to group("tests") in GN. For example:

//src/foo/lib/parser/BUILD.bazel:

load("//build/bazel/platforms:constraints.bzl", "HOST_OS_CONSTRAINTS")
load("//build/bazel/rules/host_tests:host_rustc_test.bzl", "host_rustc_test")

test_suite(
    name = "host_tests",
    # Include all tests in the package. Only use `tests = []` when every test
    # in the package belongs to this suite; if the package has both host and
    # device tests, define both `"host_tests"` and `"tests"` and list each
    # suite's tests explicitly.
    tests = [],
    visibility = ["//src/foo:__pkg__"],
)

host_rustc_test(
    name = "parser_test",
    srcs = ["src/lib.rs"],
    edition = "2024",
    target_compatible_with = HOST_OS_CONSTRAINTS,
)

//src/foo/BUILD.bazel:

test_suite(
    name = "foo_host_tests",
    tests = [
        "//src/foo/lib/parser:host_tests",
        "//src/foo/util:host_tests",
    ],
)

//src/foo/BUILD.gn:

import("//build/bazel/bazel_test_suite.gni")

bazel_test_suite("bazel_tests") {
  host_tests = [
    # This references the target in BUILD.bazel.
    "//src/foo:foo_host_tests",
  ]
}

group("tests") {
  testonly = true
  deps = [
    ":bazel_tests",
    ...
  ]
}

This will cause entries for the Bazel tests to appear in out/default/tests.json. When in doubt, examine the content of this file after fx set or fx gen. The label field for Bazel tests begins with @.

During development, it is possible to build and run test targets once they are defined, before they are exported as described above. See the Running tests directly with Bazel section below.

Best practices

For style and best practices on structuring Bazel tests and test_suite() targets, see Tests and test suites. For how bazel_test_suite() connects into the GN test graph and what to do when migrating tests from GN, see Hooking Bazel tests into the GN test graph.

Defining Fuchsia device tests in Bazel

Device tests are components in a Fuchsia package, exactly like in the GN build. Declare the test component with fx_test_component(), package it with fx_package(), and expose it as a test with fx_test():

//src/foo/bar/BUILD.bazel:

load("//build/bazel/rules/packages:fx_packaged_binary.bzl", "fx_packaged_binary")
load(
    "//build/bazel/rules/components:fx_component.bzl",
    "fx_component_manifest",
    "fx_test_component",
)
load("//build/bazel/rules/packages:fx_package.bzl", "fx_package")
load("//build/bazel/rules/testing:fx_test.bzl", "fx_test")

fx_packaged_binary(
    name = "bar_test_packaged_bin",
    testonly = True,
    binary = ":bar_test_bin",
    binary_name = "bar_test",
)

fx_component_manifest(
    name = "bar_test_manifest",
    testonly = True,
    component_name = "bar_test",
    includes = ["//src/sys/test_runners/gtest:default.shard.cml"],
    manifest = "meta/bar_test.cml",
)

fx_test_component(
    name = "bar_test_component",
    compiled_manifest = ":bar_test_manifest",
    deps = [":bar_test_packaged_bin"],
)

fx_package(
    name = "bar_tests_package",
    package_name = "bar_tests",
    test_components = [":bar_test_component"],
)

fx_test(
    name = "bar_tests",
    package = ":bar_tests_package",
    # Optional: defaults to the build's default test environments when omitted.
    environments = [
        "//build/testing/environments:aemu_env",
        "//build/testing/environments:nuc11_env",
    ],
)

Every component in test_components becomes a separate test, whose name is its package URL, e.g. fuchsia-pkg://fuchsia.com/bar_tests#meta/bar_test.cm.

To customize which target environments a test runs in, pass environments to fx_test() as a list of environment targets. //build/testing/environments has one target for each individual environment in //build/testing/environments.gni, with the same name (e.g. :emu_env or :nuc11_env). When environments is omitted, the test runs in the build's default test environments (GN's basic_envs), exactly like a GN test that doesn't set environments. To build a test without running it, set build_only = True instead.

To add infra settings, such as tags, to one of these environments, use fx_test_environment_variant():

load(
    "//build/bazel/rules/testing:fx_test_environment.bzl",
    "fx_test_environment_variant",
)

fx_test_environment_variant(
    name = "nuc11_isolated_env",
    base = "//build/testing/environments:nuc11_env",
    env_tags = ["e2e-isolated"],
)

New environments should be rare; most tests should use one of the existing environments in //build/testing/environments. When a new environment is needed, add it to both //build/testing/environments.gni and //build/testing/environments/BUILD.bazel, rather than defining it with fx_test_environment() next to a test. Environments' Swarming dimensions are an interface with infra, so they're kept in one place.

To run a test in a non-hermetic test realm (such as "starnix", "system", or "vulkan"), pass test_type to fx_test_component() (or to fx_test() to apply to all test components in the package). Valid test_type values match type_moniker_map in //build/components/fuchsia_test_component.gni.

Exporting Bazel device tests to fx test and infra builders

Like host tests, expose device tests from their package via a "tests" test_suite(), group those into an area aggregation test_suite() target, list the aggregation suite in target_tests on a GN bazel_test_suite() in the same directory, and add that target to group("tests") in GN (see Hooking Bazel tests into the GN test graph):

//src/foo/BUILD.gn:

import("//build/bazel/bazel_test_suite.gni")

bazel_test_suite("bazel_tests") {
  target_tests = [
    # This references the aggregation test_suite() in //src/foo/BUILD.bazel.
    "//src/foo:foo_tests",
  ]
}

group("tests") {
  testonly = true
  deps = [
    ":bazel_tests",
    ...
  ]
}

Device tests are tracked separately from host tests because they are queried and built in the fuchsia_platform Bazel configuration, rather than the host one.

In addition to the tests.json entries, this writes ${root_build_dir}/bazel_test_packages.list, which enumerates the package manifests of all Bazel test packages in the build graph. Full fx build invocations merge this list into all_package_manifests.list and publish its packages into amber-files, while fx test also merges it directly when building specific test targets on demand.

The manifests in this list, and in the package_manifests of the tests.json entries, aren't the ones in the Bazel output base. Right after Bazel builds the test packages, the build hardlinks their blobs into the Ninja build directory and writes copies of their manifests that point at those blobs (see //build/bazel/scripts/bazel_test_packages.py). Bazel only guarantees that an output is on disk at the end of an invocation that requested it, so a later Bazel invocation could otherwise delete files that these consumers need.

Hooking Bazel tests into the GN test graph

GN and Ninja are still the entrypoint to the build (especially in infra; in some local workflows, such as running a single Bazel test with fx test, the build invokes Bazel directly), so any Bazel test target must be visible to GN for the build and tests.json to include it. Every Bazel test must be reachable from a GN target that builders already reach:

fx_test() / host_xxx_test()          (package BUILD.bazel)
  <- package test_suite()            (package BUILD.bazel: "tests" / "host_tests")
  <- aggregation test_suite()        (area BUILD.bazel, e.g. "foo_tests")
  <- bazel_test_suite()              (area BUILD.gn)
  <- group("tests")                  (area BUILD.gn)
  <- area groups such as //src:tests, then builder groups in //bundles/buildbot

No presubmit check verifies that a Bazel test has this path. If any link in this path is missing, fx gen and fx build succeed without any warning, and the test is silently neither built nor run in infra. After adding a test, include its GN test group in your build (for example, with fx add-test //src/foo:tests, or by passing --with-test //src/foo:tests or --with //bundles/buildbot/... to fx set and running fx gen), and check that the test's label (which starts with @) appears in out/default/tests.json. Do this with a build configuration the test should run in, because Bazel also silently skips tests that are incompatible with the current configuration.

Builders can also include device tests through a product_bundle_test_group() in //bundles/buildbot, which runs the tests it reaches against a specific product bundle instead of the builder's default one. This is an additional mechanism; tests reached this way still need a bazel_test_suite().

Tests and test suites defines the full style rules for structuring Bazel tests and test_suite() targets. At a high level, test suites connect to GN in three layers while GN still drives the build:

  • Expose a package test_suite() in each package that has tests. Name it "tests" for device tests or "host_tests" for host tests, keep individual *_test targets private (default) to the package, and set the suite's visibility to the area package that aggregates it (using :__pkg__, never :__subpackages__). Don't include tests or test_suite() targets from subpackages in a package's test_suite().

  • Group package suites in an area aggregation test_suite(), paired with one bazel_test_suite() in the same directory's BUILD.gn. Keep aggregation packages as few as is reasonable, and don't recreate GN's pattern of a tests group in every intermediate directory. Pick the level at which developers add packages and at which OWNERS approve them. For example, use //src/connectivity/bluetooth rather than //src/connectivity, so adding a new Bluetooth package doesn't need approval from connectivity-wide owners. Name the aggregation test_suite() after its scope (ending in "tests" or "host_tests"), list only package test_suite() targets in it, and reference it with a full // label from bazel_test_suite() in the matching BUILD.gn:

    //src/foo/bar/BUILD.bazel:

    test_suite(
    name = "tests",
    # Include all tests in the package. Only use `tests = []` when every test
    # in the package belongs to this suite; if the package has both device and
    # host tests, define both `"tests"` and `"host_tests"` and list each
    # suite's tests explicitly.
    tests = [],
    visibility = ["//src/foo:__pkg__"],
)

# Private *_test target(s) in this package.
...

    //src/foo/BUILD.bazel:

    test_suite(
    name = "foo_tests",
    tests = [
        "//src/foo/bar:tests",
        "//src/foo/baz:tests",
    ],
)

    //src/foo/BUILD.gn:

    import("//build/bazel/bazel_test_suite.gni")

bazel_test_suite("bazel_tests") {
  target_tests = [ "//src/foo:foo_tests" ]
}

group("tests") {
  testonly = true
  deps = [
    ":bazel_tests",
    ...
  ]
}

  • Add new packages to an existing area aggregation suite when one exists; otherwise agree on an area directory with its owners and add the aggregation test_suite() and bazel_test_suite() there. Older area suites such as //tools:host_tests still list some individual test targets directly; leave existing entries alone, and expose a package test_suite() for newly added or migrated packages.

  • Keep host and device tests apart. Put device test suites only in target_tests, and host test suites only in host_tests, and don't mix host and device tests inside any Bazel test_suite(). Doing so will fail the build and/or silently drop tests.

  • Hook each aggregation test_suite() into the GN test graph once. Reaching the same Bazel test suite from two bazel_test_suite() targets doesn't run its tests twice (identical tests.json entries are merged), but it makes the layout harder to follow.

When migrating tests from GN

  • Don't keep a BUILD.gn just to hook Bazel tests into GN. If every target in a directory moved to Bazel and no GN target depends on it, delete its BUILD.gn, remove the deleted directory's :tests group from the parent BUILD.gn, and add the package's test_suite() to the area's aggregation test_suite(). Even when BUILD.gn stays behind for targets that still have GN dependents (see Testing libraries in both build systems), hook the Bazel test suites through the area's aggregation test_suite(), not a per-directory bazel_test_suite(). In a BUILD.bazel file synced by bazel2gn, put # @bazel2gn:skip on the package test_suite() so bazel2gn doesn't generate a group("tests") or group("host_tests") target in BUILD.gn.

  • Clean up references to deleted GN test targets. Once a GN test target is removed (see Testing libraries in both build systems), delete it from the GN test groups that listed it. Don't replace a dependency that actually builds or packages the test with a bazel_test_suite(): bazel_test_suite() only emits metadata for tests.json and builds nothing on its own.

  • Keep E2E tests in GN. E2E tests (host tests that need a device and a product bundle) can't be defined in Bazel yet (fxbug.dev/576736818).

Testing libraries in both build systems

While a library is used by targets in both build systems, run its tests in both build systems. Once the library has users in only Bazel, run its tests only in Bazel. Tests for targets that are only built in Bazel—such as migrated host tools, libraries with no remaining GN users, and leaf integration test suites that don't export a library to GN—run only in Bazel.

A library migrated to Bazel stays defined in GN (usually generated by bazel2gn) until no GN targets depend on it. During that time, the GN and Bazel builds of the library can differ in ways that only tests catch. Compiler flags, build variants such as sanitizers, and GN-only bazel2gn overrides all differ between the two build systems. Some attributes are also dropped during translation, and the GN target is what unmigrated dependents actually link against. Running each test in both build systems costs extra build time and test capacity, but only while a library has users in both build systems.

In practice:

  • Define tests in Bazel and let bazel2gn generate the GN copy. When you migrate a library whose BUILD.gn is generated by bazel2gn, migrate its tests too, and don't put # @bazel2gn:skip on them. bazel2gn translates fx_cc_test_binary(), rustc_test(), go_test(), host_rustc_test(), host_go_test(), fx_unittest_package(), fx_package() targets with test_components (with their fx_test_component()s and the fx_test() for them), and the with_unit_tests, with_host_unit_tests, and test_deps attributes of Rust rules. Hook the Bazel test into the GN test graph as described above, and list the generated GN copy in the GN test groups that listed the original GN test.

  • Device test packages get different names in GN. A GN test package and its Bazel copy can't share a package name, because both are published to the same package repository. bazel2gn appends -gn to the package name of the GN copy of an fx_unittest_package() or a test fx_package(), and the Bazel test keeps the original name, so its test URL doesn't change when you migrate it.

  • If bazel2gn can't translate a test's packaging yet and you want to keep running it in GN, translate the test binary with bazel2gn and write the GN test package by hand above the ## BAZEL2GN SENTINEL in BUILD.gn with package_name = "<name>-gn". This works for device tests that use fx_unittest_package(resources = [...]) or a test fx_package() with attributes that bazel2gn doesn't support (put # @bazel2gn:skip on the package and on fuchsia_package_resource_group(), not on the test binary).

  • C++ host tests need a hand-written GN host_test(). bazel2gn translates fx_cc_test_binary() to GN cc_test_executable(), which only builds the binary, and doesn't translate host_cc_test() or host_test() yet. To keep running a C++ host test in GN, write a GN host_test() that runs the generated binary above the ## BAZEL2GN SENTINEL in BUILD.gn, inside if (is_host), and list it in the GN test groups with ($host_toolchain).

  • When to opt out with # @bazel2gn:skip. By default, keep running a library's tests in both build systems when bazel2gn can translate them automatically. It's fine to put # @bazel2gn:skip on a test target or test attribute (with a brief comment explaining why) and run it only in Bazel when:

    • The target under test is only built in Bazel (for example, a binary or library in the same BUILD.bazel file that itself has # @bazel2gn:skip).
    • bazel2gn doesn't support the test's rules or attributes (such as cc_test(), host_cc_test(), host_py_test(), host_shac_test(), host_test(), host tests with data / test_data / test_label, fx_unittest_package() with resources, or rustc_test() with cross-package srcs/crate or conflicting rustc_env), and maintaining a hand-written GN test target above the sentinel isn't worth the extra complexity.
    • The cost of running or maintaining the test in both build systems outweighs the risk of a GN-only regression. Use these criteria to judge the tradeoff:
    • Target parity: Does the library's BUILD.bazel use # @bazel2gn:raw_overwrite, # @bazel2gn:path_overwrite, or # @bazel2gn:skip on dependencies, configs, or compiler flags, or is it an IDK/SDK library compiled across multiple configurations in GN? If the generated GN target is a 1:1 translation of the Bazel target, the Bazel test already exercises the same code.
    • Unmigrated GN dependencies: Does the library depend on GN-only targets that change frequently? Presubmit affected-test analysis doesn't trace from GN targets into Bazel tests, so changes to a GN-only dependency won't trigger a Bazel-only test in presubmit.
    • Test weight: Fast, self-contained unit tests are cheap to mirror; slow or resource-heavy integration tests (or libraries whose remaining GN users are actively being migrated) are good candidates to run only in Bazel.

  • If Bazel can't express a test yet, keep it in GN above the ## BAZEL2GN SENTINEL and say why in a comment. Examples are e2e tests, tests that need Ninja-generated inputs, bootfs_test()s, and sdk_cpp_unittest_package()s that compile across multiple C++ standards.

  • Remove the GN copy with the library. When the last GN user of a library migrates to Bazel, delete the GN definition of the library and its tests, and remove the GN tests from the GN test groups that listed them.

Running Fuchsia Bazel device tests

Once exported, run a Bazel device test the same way as a GN one, by package URL or by package name:

fx test fuchsia-pkg://fuchsia.com/bar_tests#meta/bar_test.cm

fx test builds the test by invoking Bazel directly (as fx build --fuchsia_platform bazel_label), then publishes the package manifests listed in bazel_test_packages.list before running the test. A running package server and a connected device or emulator are required, exactly as for GN device tests.

Running Fuchsia Bazel host tests

Local testing

Running tests with fx test

To run Bazel host tests with fx test, they MUST be exported by following the instructions in the Exporting Bazel host tests to fx test and infra builders section above.

After the tests are exported, use fx test --host <name_or_label> to build and run a Bazel host test locally, same as with GN-defined host tests.

The only difference is that the test will be built on demand by invoking Bazel directly, skipping Ninja entirely.

Note that when using fx test:

  • <name_or_label> will be subject to fuzzy matching, and fx test will report multiple candidates in case of ambiguity.

  • <name_or_label> cannot reference a Bazel test_suite() label, just like it cannot reference a GN group() one.

See the User guide for fx test for more details about how to use fx test.

Running tests directly with fx bazel test

You can use fx bazel test --config=host <target_pattern> to build and run tests locally directly with Bazel. This does not require the extra plumbing to make tests visible to fx test as described in the previous section, and has the following advantages:

  • <target_pattern> is a set of Bazel target patterns that can match "all test targets in a given package", in which case Bazel will skip non-test targets automatically.

  • <target_pattern> can match test_suite() labels which can be convenient to run a set of host tests repeatedly during development.

  • bazel test caches test results, and will only re-run the tests impacted by your latest changes since the last invocation.

    (Use --nocache_test_results flag to disable this).

  • <target_pattern> can reference labels that are not visible in tests.json. This can be handy during development when adding new tests.

Infra testing

Only host tests that are listed in tests.json can be launched on infra test bots. This applies equally to GN and Bazel host tests. So you MUST ensure they are visible in tests.json by following the Exporting Bazel host tests to fx test and infra builders section above, otherwise they will NOT be continuously tested on infra.

Infra builds Bazel tests using fint, which directly calls bazel build after running ninja, to build all Bazel tests that are listed in bazel_test_suite GN targets.

There is no explicit distinction between GN and Bazel-defined host tests when running them on infra test runners, or when collecting results. In particular, infra does not run tests using bazel test. Instead, it takes the binaries produced by bazel build and runs them directly, just like it does with host tests from ninja.

Debug symbol propagation

Host test binaries compiled with debug symbols must have their unstripped ELF binaries registered so that tooling can symbolize backtraces and correlate LLVM code coverage profiles (-profile-correlate=binary).

For general background on how Fuchsia manages Bazel debug symbols, see Technical Note on Debug Symbol Generation.

Authoring host test rules with debug symbols

The generic host_test() rule exposes the unstripped_binary field in its FuchsiaHostTestInfo provider:

  • If unstripped_binary is not explicitly set, host_test() inspects the underlying binary attribute:

    • If binary provides DebugPackageInfo (C++ targets), it uses DebugPackageInfo.unstripped_file.
    • If binary provides CrateInfo (Rust targets), it uses CrateInfo.output.

  • If the test uses a wrapper script launcher (such as rust_test_parser in host_rustc_test()), the macro MUST explicitly forward the original unstripped binary target via unstripped_binary:

    host_test(
    name = name,
    binary = wrapper_script,
    unstripped_binary = ":" + binary_name,
    ...
)

Propagation to infra and debuginfod

When Bazel host tests are exported to the GN build:

  1. Test Query: bazel_tests_utils.py runs a bazel cquery using //build/bazel/starlark/FuchsiaHostTestInfo.cquery, extracting the execroot path of unstripped_binary for all registered host tests.

  2. Host Test Manifest: The paths are normalized relative to the Ninja build directory and written to ${root_build_dir}/bazel_host_tests.debug_symbols.json.

  3. GN Build API Integration: //:bazel_host_test_suites in BUILD.gn attaches debug_symbol_manifests metadata pointing to bazel_host_tests.debug_symbols.json, which is consumed by the build_api_module("debug_symbols") target.

  4. Artifactory Upload: In CI/CQ builds, Artifactory processes debug_symbols.json and uploads the unstripped ELF binaries to cloud storage / debuginfod servers, allowing tools like covargs to resolve Build IDs.

Local development and fx coverage

When running coverage locally at desk, uncommitted binaries have newly generated Build IDs that are not present on debuginfod servers.

To support local symbol correlation, fx coverage runs:

python3 ${FUCHSIA_DIR}/build/bazel/scripts/copy_bazel_debug_symbols.py ${FUCHSIA_BUILD_DIR}

This script reads bazel_host_tests.debug_symbols.json, computes the GNU Build ID for each ELF binary, and copies or symlinks the unstripped binary into ${root_build_dir}/.build-id/xx/yyyyyyyy.debug. Tooling such as ffx coverage and llvm-profdata then resolves these binaries via .symbol-index.json.

FAQ

Bazel test compatibility

If any test in a test_suite() (or any of its dependencies) is incompatible with the current configuration (e.g., using target_compatible_with), that test will be skipped during evaluation of the suite instead of failing the build.

The same is true when using patterns such as //... and :all on the command line.

For more information, see the handling of incompatible targets.

When running a test_suite() or using a target pattern, developers must verify that the test(s) of interest are reported as PASSED (and not SKIPPED).