DriverTestRealm

A driver integration testing framework

DriverTestRealm is an integration testing framework that runs drivers in a hermetic environment. It is useful for driver authors to test their drivers, and for system developers to run integration tests that use specific driver stacks. DriverTestRealm provides a hermetic version of all of the Driver Frameworks APIs, and provides an environment that is nearly identical to a running system.

DriverTestRealm is for integration testing. For a light weight unit testing framework, use mock DDK instead.

Overview of DriverTestRealm

DriverTestRealm is a component that your test can access. This component contains all of the DriverFramework's components, like DriverManager and DriverIndex. It mocks out all of the capabilities these components need.

Figure: The non-hermetic setup for DriverTestRealm

DriverTestRealm exposes the fuchsia.driver.test/Realm protocol which is used to start the DriverTestRealm. The Start function takes arguments which can be used to configure things like which drivers are loaded, the root driver, and how DriverManager shuts down. Start can only be called once per component; if a new DriverTestRealm is needed for each test, then RealmBuilder must be used.

Interacting with drivers

The DriverTestRealm component exposes the /dev/ directory from DriverManager, which is how test code will interact with drivers. The /dev/ directory works identically to the devfs on a running system. For example, if the test adds a mock input device, it will show up at /dev/class/input-report/XXX.

Including drivers

By default, the DriverTestRealm component loads drivers from its own package. The test author must be sure to include all the drivers in the test package that they expect to load.

Binding drivers

By default, the root driver for DriverTest realm is the test-parent driver. That means that in order to bind a driver, you should create a mock driver that binds to the test parent. This can be accomplished with the following bind rules:

fuchsia.BIND_PROTOCOL == fuchsia.test.BIND_PROTOCOL.PARENT;

Your mock driver can then add a device with the right properties so that your driver-under-test will bind to the mock driver.

Hermetic vs Non-Hermetic

There are two different ways of using DriverTestRealm. It can be used hermetically or non-hermetically.

Hermetic

In the hermetic version of Driver Test Realm every test gets its own version of the Driver Test Realm component. This means every test is hermetic, or isolated, from the other tests. Tests will not share any state, as each Driver Test Realm is unique to that test.

Figure: The hermetic setup for DriverTestRealm

Using a hermetic Driver Test Realm may be slower as each test has to spawn and setup new components.

Non Hermetic

The non-hermetic way of using Driver Test Realm is to have a single Driver Test Realm child component that is shared between every test instance.

Figure: The non-hermetic setup for DriverTestRealm

The test author needs to be extra careful to make sure that their driver's state is cleared between each tests so that the individual tests do not interact with each other.

Using a non-hermetic Driver Test Realm may be much faster since each test does not need to spawn and setup new components. The test code may also be simpler.

DriverTestRealm examples

Here are some examples of using DriverTestRealm hermetically and non-hermetically, in both C++ and Rust.

The examples can be seen at //examples/drivers/driver_test_realm/.

Hermetic

Test authors can use RealmBuilder to create a new DriverTestRealm for each test. The DriverFramework has provided a helpful library using DriverTestRealm in RealmBuilder.

Here's an example BUILD.gn file for using DriverTestRealm with RealmBuilder. Notice that there's a specific DriverTestRealm GN target to depend on so the test's generated CML has the correct RealmBuilder permissions.

C++

test("driver_test_realm_example_hermetic_cpp") {
  sources = [ "test.cc" ]
  deps = [
    "//examples/drivers/driver_test_realm/sample-driver:fuchsia.hardware.sample_cpp",
    "//sdk/fidl/fuchsia.driver.test:fuchsia.driver.test_hlcpp",
    "//sdk/fidl/fuchsia.io:fuchsia.io_hlcpp",
    "//sdk/lib/async-loop",
    "//sdk/lib/async-loop:async-loop-cpp",
    "//sdk/lib/async-loop:async-loop-default",
    "//sdk/lib/component/outgoing/cpp",
    "//sdk/lib/device-watcher/cpp",
    "//sdk/lib/driver_test_realm/realm_builder/cpp",
    "//src/lib/fxl/test:gtest_main",
    "//src/lib/testing/loop_fixture",
    "//zircon/system/ulib/fbl",
  ]
}

fuchsia_unittest_package("package") {
  package_name = "driver_test_realm_example_hermetic_cpp"
  deps = [
    # Include your test component.
    ":driver_test_realm_example_hermetic_cpp",

    # Include the driver(s) you will be testing.
    "//examples/drivers/driver_test_realm/sample-driver",

    # Include the test parent (if your driver binds to it).
    "//src/devices/misc/drivers/test-parent",
  ]
}

Rust

rustc_test("driver_test_realm_example_realm_builder_rust") {
  edition = "2021"
  testonly = true
  source_root = "test.rs"
  sources = [ "test.rs" ]
  deps = [
    "//examples/drivers/driver_test_realm/sample-driver:fuchsia.hardware.sample_rust",
    "//sdk/fidl/fuchsia.driver.test:fuchsia.driver.test_rust",
    "//sdk/lib/device-watcher/rust",
    "//sdk/lib/driver_test_realm/realm_builder/rust",
    "//src/lib/fuchsia-async",
    "//src/lib/fuchsia-component-test",
    "//third_party/rust_crates:anyhow",
  ]
}

fuchsia_unittest_package("package") {
  package_name = "driver_test_realm_example_realm_builder_rust"
  deps = [
    # Include your test component.
    ":driver_test_realm_example_realm_builder_rust",

    # Include the driver(s) you will be testing.
    "//examples/drivers/driver_test_realm/sample-driver",

    # Include the platform bus (if your driver binds to it).
    "//src/devices/bus/drivers/platform:platform-bus",

    # Include the test parent (if your driver binds to it).
    "//src/devices/misc/drivers/test-parent",
  ]

  # There's expected error logs that happen due to races in driver enumeration.
  test_specs = {
    log_settings = {
      max_severity = "ERROR"
    }
  }
}

Here's test code that spawns a new DriverTestRealm per test.

C++

class DriverTestRealmTest : public gtest::TestLoopFixture {};

TEST_F(DriverTestRealmTest, DriversExist) {
  // Create and build the realm.
  auto realm_builder = component_testing::RealmBuilder::Create();
  driver_test_realm::Setup(realm_builder);
  auto realm = realm_builder.Build(dispatcher());

  // Start DriverTestRealm.
  fidl::SynchronousInterfacePtr<fuchsia::driver::test::Realm> driver_test_realm;
  ASSERT_EQ(ZX_OK, realm.component().Connect(driver_test_realm.NewRequest()));
  fuchsia::driver::test::Realm_Start_Result realm_result;
  ASSERT_EQ(ZX_OK, driver_test_realm->Start(fuchsia::driver::test::RealmArgs(), &realm_result));
  ASSERT_FALSE(realm_result.is_err()) << zx_status_get_string(realm_result.err());

  // Connect to dev.
  fidl::InterfaceHandle<fuchsia::io::Node> dev;
  ASSERT_EQ(ZX_OK, realm.component().Connect("dev-topological", dev.NewRequest().TakeChannel()));

  fbl::unique_fd root_fd;
  ASSERT_EQ(ZX_OK, fdio_fd_create(dev.TakeChannel().release(), root_fd.reset_and_get_address()));

  // Wait for driver.
  zx::result channel =
      device_watcher::RecursiveWaitForFile(root_fd.get(), "sys/test/sample_driver");
  ASSERT_EQ(channel.status_value(), ZX_OK);

  // Turn the connection into FIDL.
  fidl::WireSyncClient client(
      fidl::ClientEnd<fuchsia_hardware_sample::Echo>(std::move(channel.value())));

  // Send a FIDL request.
  constexpr std::string_view sent_string = "hello";
  fidl::WireResult result = client->EchoString(fidl::StringView::FromExternal(sent_string));
  ASSERT_EQ(ZX_OK, result.status());
  ASSERT_EQ(sent_string, result.value().response.get());
}

Rust

#[fasync::run_singlethreaded(test)]
async fn test_sample_driver() -> Result<()> {
    // Create the RealmBuilder.
    let builder = RealmBuilder::new().await?;
    builder.driver_test_realm_setup().await?;
    // Build the Realm.
    let instance = builder.build().await?;
    // Start DriverTestRealm
    instance.driver_test_realm_start(fdt::RealmArgs::default()).await?;

    // Connect to our driver.
    let dev = instance.driver_test_realm_connect_to_dev()?;
    let driver =
        device_watcher::recursive_wait_and_open::<fidl_fuchsia_hardware_sample::EchoMarker>(
            &dev,
            "sys/test/sample_driver",
        )
        .await?;

    // Call a FIDL method on the driver.
    let response = driver.echo_string("Hello world!").await.unwrap();

    // Verify the response.
    assert_eq!(response, "Hello world!");
    Ok(())
}

#[fasync::run_singlethreaded(test)]
async fn test_platform_bus() -> Result<()> {
    // Create the RealmBuilder.
    let builder = RealmBuilder::new().await?;
    builder.driver_test_realm_setup().await?;
    // Build the Realm.
    let instance = builder.build().await?;
    // Start DriverTestRealm.
    let args = fdt::RealmArgs {
        root_driver: Some("fuchsia-boot:///platform-bus#meta/platform-bus.cm".to_string()),
        ..Default::default()
    };
    instance.driver_test_realm_start(args).await?;
    // Connect to our driver.
    let dev = instance.driver_test_realm_connect_to_dev()?;
    device_watcher::recursive_wait(&dev, "sys/platform").await?;
    Ok(())
}

Non Hermetic

Here's a basic example of a test that starts a DriverTestRealm component and then connects to /dev to see a driver that has been loaded.

First, it's important that the build rules are set up correctly. The test package needs to contain the DriverTestRealm component, as well as any drivers that are going to be loaded. Adding drivers to the package will automatically make those drivers visible to DriverTestRealm.

C++

test("driver_test_realm_example_non_hermetic_cpp") {
  sources = [ "test.cc" ]
  deps = [
    "//sdk/fidl/fuchsia.driver.test:fuchsia.driver.test_cpp",
    "//sdk/lib/component/incoming/cpp",
    "//sdk/lib/device-watcher/cpp",
    "//sdk/lib/driver_test_realm:static",
    "//sdk/lib/syslog/cpp",
    "//third_party/googletest:gtest",
  ]
}

fuchsia_unittest_package("package") {
  package_name = "driver_test_realm_example_non_hermetic_cpp"
  deps = [
    ":driver_test_realm_example_non_hermetic_cpp",

    # Add drivers to the package here.
    # The test-parent driver is the default root driver for DriverTestRealm.
    "//src/devices/misc/drivers/test-parent",
  ]
}

Here's what the test setup would look like. Notice that you have to call fuchsia.driver.test/Realm:Start before your test framework is run. The arguments to Start can be setup to configure the DriverManager implementation.

C++

TEST(DdkFirmwaretest, DriverWasLoaded) {
  zx::result channel = device_watcher::RecursiveWaitForFile("/dev/sys/test");
  ASSERT_EQ(channel.status_value(), ZX_OK);
}

int main(int argc, char **argv) {
  fuchsia_logging::LogSettingsBuilder builder;
  builder.WithTags({"driver_test_realm_test"}).BuildAndInitialize();

  // Connect to DriverTestRealm.
  auto client_end = component::Connect<fuchsia_driver_test::Realm>();
  if (!client_end.is_ok()) {
    FX_LOG_KV(ERROR, "Failed to connect to Realm FIDL", FX_KV("error", client_end.error_value()));
    return 1;
  }
  fidl::WireSyncClient client{std::move(*client_end)};

  // Start the DriverTestRealm with correct arguments.
  auto wire_result = client->Start(fuchsia_driver_test::wire::RealmArgs());
  if (wire_result.status() != ZX_OK) {
    FX_LOG_KV(ERROR, "Failed to call to Realm:Start", FX_KV("status", wire_result.status()));
    return 1;
  }
  if (wire_result.value().is_error()) {
    FX_LOG_KV(ERROR, "Realm:Start failed", FX_KV("status", wire_result.value().error_value()));
    return 1;
  }

  // Run the tests.
  ::testing::InitGoogleTest(&argc, argv);
  return RUN_ALL_TESTS();
}

Notice that once DriverTestRealm has been started the test environment has all of DriverFramework's APIs available to it in its component namespace. The /dev/ directory can be watched and opened to connect to drivers.

Simple example

The Simple library lets a test use DriverTestRealm with the default arguments.

Most integration tests are fine with DriverTestRealm's default configuration; they don't need to pass arguments to Start. If this is the case, then SimpleDriverTestRealm starts automatically.

C++

test("driver_test_realm_example_simple_cpp") {
  sources = [ "test.cc" ]
  deps = [
    "//sdk/lib/device-watcher/cpp",
    "//sdk/lib/driver_test_realm/simple",
    "//src/lib/fxl/test:gtest_main",
  ]
}

fuchsia_unittest_package("package") {
  package_name = "driver_test_realm_example_simple_cpp"
  deps = [ ":driver_test_realm_example_simple_cpp" ]
}

Rust

rustc_test("driver_test_realm_example_simple_rust") {
  edition = "2021"
  source_root = "test.rs"
  sources = [ "test.rs" ]
  deps = [
    "//sdk/lib/device-watcher/rust",
    "//sdk/lib/driver_test_realm/simple",
    "//src/lib/fuchsia-async",
    "//src/lib/fuchsia-fs",
    "//third_party/rust_crates:anyhow",
  ]
}

fuchsia_unittest_package("package") {
  package_name = "driver_test_realm_example_simple_rust"
  deps = [ ":driver_test_realm_example_simple_rust" ]
}

The test looks identical except that it doesn't need to set up a main function to call fuchsia.driver.test/Realm:Start.

C++

TEST(SimpleDriverTestRealmTest, DriversExist) {
  zx::result channel = device_watcher::RecursiveWaitForFile("/dev/sys/test");
  ASSERT_EQ(channel.status_value(), ZX_OK);
}

Rust

#[fasync::run_singlethreaded(test)]
async fn test_driver() -> Result<()> {
    let dev = fuchsia_fs::directory::open_in_namespace("/dev", fuchsia_fs::Flags::empty())?;
    device_watcher::recursive_wait(&dev, "sys/test").await?;
    Ok(())
}

Common Issues:

A driver isn't binding
- Make sure the driver is being included in the package
- Make sure src/devices/misc/drivers/test-parent is included if the default root driver is being used.
Calls to /dev/ are hanging
- Make sure fuchsia.driver.test/Realm:Start is called.
fuchsia.driver.test/Realm:Start returns ZX_ERR_ALREADY_BOUND
- Start can only be called once per component. If you want a new DriverTestRealm per test, please see the RealmBuilder section.