Create Test Realm

If you or your team owns non-hermetic tests or tests which require custom runners, you need to create a test realm to execute those tests.

Below you can see a sample test realm

// Copyright 2023 The Fuchsia Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

// This realm installs various testing collections.
{
    include: [
        // Required: Include various test collections/realms.
        "//sdk/ctf/test_realm/meta/test-collection.shard.cml",
        "//src/devices/testing/devices_test_realm.shard.cml",
        "//src/lib/vulkan/vulkan_test_realm.shard.cml",
        "//src/media/testing/drm_test_realm.shard.cml",
        "//src/sys/component_manager/tests/capability_provider_vfs_compliance/vfs_compliance_test_realm.shard.cml",
        "//src/sys/testing/meta/chromium_test_realm.shard.cml",

        // Optional: Include optional runners.
        "//src/sys/testing/meta/runners/fuzz-test-runner.shard.cml",
        "//src/sys/testing/meta/runners/inspect-test-runner.shard.cml",
        "//src/sys/testing/meta/runners/netemul-test-runner.shard.cml",
        "//src/sys/testing/meta/runners/standard-test-runners.shard.cml",
        "//src/sys/testing/meta/runners/starnix-test-runner.shard.cml",
        "//src/sys/testing/meta/runners/stress-test-runner.shard.cml",

        // Required: Include various test collections/realms.
        "//src/sys/testing/meta/starnix-tests.shard.cml",
        "//src/sys/testing/meta/system-tests.shard.cml",
        "//src/sys/testing/meta/test-arch-tests.shard.cml",
        "//src/testing/system-validation/meta/system_validation_test_realm.shard.cml",

        // Required: Include other required shards.
        "sys/component/realm_builder.shard.cml",
        "syslog/client.shard.cml",
    ],
    expose: [
        // Required: Expose Realm protocol so that test manager can launch tests
        // in the realm.
        {
            protocol: "fuchsia.component.Realm",
            from: "framework",
        },
    ],

    // TODO(https://fxbug.dev/42063673): Move to corresponding runner and collection
    // files.
    environments: [
        {
            name: "test-env",
            extends: "realm",
            runners: [
                {
                    runner: "elf_test_runner",
                    from: "#elf_test_runner",
                },
                {
                    runner: "elf_test_ambient_exec_runner",
                    from: "#elf_test_ambient_exec_runner",
                },
                {
                    runner: "elf_test_create_raw_processes_runner",
                    from: "#elf_test_create_raw_processes_runner",
                },
                {
                    runner: "elf_test_create_raw_processes_ambient_exec_runner",
                    from: "#elf_test_create_raw_processes_ambient_exec_runner",
                },
                {
                    runner: "gtest_runner",
                    from: "#gtest_runner",
                },
                {
                    runner: "gunit_runner",
                    from: "#gunit_runner",
                },
                {
                    runner: "netemul_test_runner",
                    from: "#netemul_test_runner",
                },
                {
                    runner: "rust_test_runner",
                    from: "#rust_test_runner",
                },
                {
                    runner: "rust_test_create_raw_processes_runner",
                    from: "#rust_test_create_raw_processes_runner",
                },
                {
                    runner: "rust_test_next_vdso_runner",
                    from: "#rust_test_next_vdso_runner",
                },
                {
                    runner: "starnix_test_runner",
                    from: "#starnix_test_runners",
                },
                {
                    runner: "starnix_unit_test_runner",
                    from: "#starnix_test_runners",
                },
                {
                    runner: "inspect_test_runner",
                    from: "#inspect_test_runner",
                },
                {
                    runner: "go_test_runner",
                    from: "#go_test_runner",
                },
                {
                    runner: "stress_test_runner",
                    from: "#stress_test_runner",
                },
                {
                    runner: "realm_builder",
                    from: "#realm_builder_server",
                },
            ],
            resolvers: [
                {
                    resolver: "realm_builder_resolver",
                    from: "#realm_builder_server",
                    scheme: "realm-builder",
                },
            ],
            __stop_timeout_ms: 30000,
        },
    ],
}

Link to the file.

Required sections

The test realm manifest must include specific sections to facilitate the execution of test components by the Test Manager.

Test Collections

Every test realm must have one or more test collections where the Test Manager can execute tests. In the provided sample test realm, we utilize test collections using shards. You can refer to the System tests collection as an example, which can run various non-hermetic tests.

Expose Section

Each test realm must expose protocol fuchsia.component.Realm. This protocol provides Test Manager the capability to launch components inside the realm.

Realm Builder

Each test realm must include the realm builder shard which helps the Test Manager to execute tests dynamically inside the collection.

Optional Sections

Test Runners

While optional, it is recommended for most test realms to include standard test runners for executing basic tests within the realm. In cases where custom test runners are needed, they can be included and added to the environment of the test collection.

Integration with in-tree build rules

Register your moniker in our build rules:

_type_moniker_map = {
  system = "/core/testing:system-tests"
  test_arch = "/core/testing:test-arch-tests"
  ...
  <new_type> = "/path/to/test/realm/moniker"
}

Assign the new test_type to your test:

// BUILD.gn
fuchsia_test_component("my_test") {
  component_name = "my_test"
  manifest = "meta/my_test.cml"
  deps = [ ":my_test_bin" ]
  test_type = "<new_type>"
}

Now you can run test can be executed using fx test.

Out of tree tests

Integrate your testing scripts to call ffx test with the realm moniker:

ffx test run --realm "/path/to/test/realm/moniker" <test_url>