Run Fuchsia tests

This guide provides instructions on how to run Fuchsia tests using the fx test command.

In Fuchsia, a test can be a component that runs on a Fuchsia device (see Tests as components) or a standalone executable that runs on the host machine.

To run a Fuchsia test, use the fx test command with the name of the test:

fx test <TEST_NAME>

If TEST_NAME is a test component, fx test connects to your Fuchsia device to load and run the test component. That is, the command finds the component's corresponding component URI and calls ffx test run. However, if TEST_NAME is a host test, fx test directly invokes that test binary to run on the host machine.

Similar to a host test, an end-to-end test also runs on a host machine. The test then may interact with various services on a Fuchsia device for testing purposes (see Scripting Layer for Fuchsia). To run an end-to-end test, provide an additional flag (--e2e) to fx test:

fx test --e2e <END_TO_END_TEST_NAME>

Customize invocations

fx test can run multiple tests or test suites at once. The command can also filter those tests to be only device, host, or end-to-end tests.

To customize fx test, you can add flags and provide a number of tests:

fx test <FLAGS> <TEST_NAME_01> <TEST_NAME_02> ...

Common ways to customize fx test are listed in the sections below.

Run multiple tests

If you want to run multiple sets of Fuchsia tests, configure your Fuchsia build to include several of the primary testing bundles, build Fuchsia, and then run all tests in the build. For example:

fx set core.x64 --with //bundles/tools,//bundles/tests
fx build
fx test

You can also provide multiple targets in a single invocation:

fx test <PACKAGE_01> <PACKAGE_02> <COMPONENT_01> <COMPONENT_02>

See Specify a test in multiple ways for various ways to specify tests.

Pass arguments to tests

Use the -- flag to pass additional arguments to test components.

The following example passes an arg flag to a test:

fx test <TEST_NAME> -- --arg=5

For example, the command above internally calls the following command:

fx ffx test run <TEST_COMPONENT_URI> -- --arg=5

Specify a test in multiple ways

fx test supports multiple ways to reference a specific test:

Host test path

For a host test, provide a relative path to the test binary from the root of the Fuchsia build output directory. If the path points to a subdirectory, not a file, fx test runs all matching test binaries in that directory.

For example, you can provide a relative path to specify a test binary:

fx test host_x64/pm_cmd_pm_genkey_test

Or you can provide a relative path to a test directory:

fx test host_x64/gen/sdk

Package URL

Provide a full Fuchsia component URL to specify a test component. For example:

fx test fuchsia-pkg://fuchsia.com/my_example_test_pkg#meta/my_example_test.cm

Provide a partial package URL to match and run all test components in the package with the provided Fuchsia package URL. For example:

fx test fuchsia-pkg://fuchsia.com/my_example_test_pkg

Package name

Provide a package name to run all test components in that package. For example:

fx test my_example_test_pkg

To explicitly specify the input to be a package name, use the flag -p. For example:

fx test -p my_example_test_pkg

Component name

Provide a component name (or a resource path) to test a single component in a package. For example:

fx test my_example_test

To explicitly specify the input to be a component name, use the flag -c. For example:

fx test -c my_example_test

To run a component on a specific package, use both -p <PACKAGE_NAME> and -c <COMPONENT_NAME>. For example:

fx test -p my_example_test_pkg -c my_example_test

Set the minimum log severity

fx test (and the underlying ffx test) accept a flag `--min-severity-logs which allows you to set the minimum severity of the logs that are emitted by the test and components under the test.

If the test or components under it are using logging libraries which support setting dynamic log severity (Fuchsia Rust and C++ log libraries support this). For test components that don't support this, test_manager manually filters their logs if the tests emit logs of a severity that is lower than the minimum you set.

This flag accepts two ways of defining the minimum severity:

  • <severity>: one of FATAL, ERROR, WARN, INFO, DEBUG, TRACE. This applies to logs emitted by the test itself and all components under the test.

  • <component selector>#<severity> in which <component selector> specifies a set of components under the test (for example foo/bar) and severity is one of the accepted severities mentioned earlier.

A few examples:

  • --min-severity-logs DEBUG: the test and all components under the test are instructed to emit logs of severity DEBUG or higher. This is equivalent to using a component selector: --min-severity-logs **#DEBUG

  • --min-severity-logs a#DEBUG --min-severity-logs b/c#ERROR: the component under the test a emits logs of severity DEBUG or higher and the component under the test b/c emits logs of severity ERROR or higher. Logs emitted by the test itself uses their default minimum severity.

  • --min-severity-logs '<root>#DEBUG': the test is instructed to emit logs of severity DEBUG or higher, but components under it emits logs using their default minimum severity.

  • --min-severity-logs foo/*/bar#ERROR: all components named bar under a child of foo with any name emits logs of severity ERROR or higher. For example, foo/a/bar and foo/baz/bar are affected, but the test component, foo/bar and a/b aren't.

Convert from run-host-tests

The fx run-host-tests and fx run-e2e-tests commands are being deprecated in favor of fx test. See the following instructions on how to use fx test in place of these commands:

fx test <NESTED_COMPONENT_NAME>

run-host-tests

Substitute fx run-host-tests with fx test.

From:

fx run-host-tests <PATH_TO_HOST_TEST>

To:

fx test <PATH_TO_HOST_TEST>

run-e2e-tests

Substitute fx run-e2e-tests with fx test and an additional --e2e flag.

From:

fx run-e2e-tests <END_TO_END_TEST>

To:

fx test --e2e <END_TO_END_TEST>

Test-driven development

The fx smoke-test command automatically detects all tests that are known to the build system as affected by changes in your checkout. Try the following:

fx -i smoke-test --verbose

In the command above, --verbose also prints which tests fx smoke-test thinks are affected by your change. -i automatically repeats this command every time you save your changes. For test-driven development, try launching this command in a separate shell and watching your code rebuild and retest as you're working on it.

fx smoke-test works best with hermetic test packages. A test package is hermetic if the package contains all the dependencies of any tests in it. That is to say, any code changes that affect the outcome of this test should require rebuilding that test's package as well.

Inspect artifacts from test components

Component tests may produce additional artifacts that cannot be displayed to stdout, such as custom artifacts and coverage profile. By default, fx test silently discards these artifacts. To see these artifacts, specify an output directory to fx test using --ffx-output-directory. The artifacts are pulled out of the test and saved to the specified directory.