Run an end-to-end test

This guide provides instructions on how to run an end-to-end test for testing a Fuchsia product.

This guide uses the Fuchsia emulator (FEMU) to emulate a device that runs Fuchsia. As for the end-to-end test, the guide uses the screen_is_not_black end-to-end test.

The screen_is_not_black end-to-end test reboots a device under test (DUT), waits 100 seconds, and takes a snapshot of the device’s screen. If the snapshot image is not a black screen, the test concludes that Fuchsia is successfully up and running on the device after reboot.

To run this end-to-end test, the steps are:

Prerequisites.
Build a Fuchsia image to include the end-to-end test.
Start the emulator with the Fuchsia image.
Run the end-to-end test.

Also, to run any end-to-end test, see the Appendices section.

1. Prerequisites

This guide requires that you've completed the following guides:

2. Build a Fuchsia image to include the end-to-end test

Before you can run the screen_is_not_black end-to-end test, you first need to build your Fuchsia image to include the test in the build artifacts:

To add the end-to-end test, run the fx set command with the following --with option:
```
fx set workstation.qemu-x64 --with //src/tests/end_to_end/screen_is_not_black
```
//src/tests/end_to_end/screen_is_not_black is a test directory in the Fuchsia source tree. The BUILD.gn file in this directory defines the screen_is_not_black target to include the screen_is_not_black end-to-end test in the build artifacts.
Build your Fuchsia image:
```
fx build
```
When the fx build command completes, the build artifacts now include the screen_is_not_black end-to-end test, which you can run from your host machine.

3. Start the emulator with the Fuchsia image

Start the emulator with your Fuchsia image and run a package repository server:

Configure an IPv6 network for the emulator (you only need to do this once):

sudo ip tuntap add dev qemu mode tap user $USER && sudo ip link set qemu up

In a new terminal, start the emulator:
```
fx vdl start -N -u <PATH_TO_UPSCRIPT>
```
Replace the following:
- PATH_TO_UPSCRIPT: The path to a FEMU network setup script; for example, ~/fuchsia/scripts/start-unsecure-internet.sh.

Set the emulator to be your device:

fx set-device

If you have multiple devices, select fuchsia-5254-0063-5e7a (the emulator’s default device name), for example:

$ fx set-device

Multiple devices found, please pick one from the list:
1) fuchsia-4407-0bb4-d0eb
2) fuchsia-5254-0063-5e7a
#? 2
New default device: fuchsia-5254-0063-5e7a

4. Run the end-to-end test

Run the screen_is_not_black end-to-end test:

fx test --e2e screen_is_not_black

When the test passes, this command prints output similar to the following:

Saw a screen that is not black; waiting for 0:00:59.924543 now.

...

[FINE]: Running over ssh: killall sl4f.cmx
01:46 +1: All tests passed!

Appendices

Run any end-to-end test

Use the fx test --e2e command to run an end-to-end test from your host machine:

fx test --e2e <TEST_NAME>

Some product configurations may include a set of end-to-end tests by default (see Examine product configuration files). However, if you want to run an end-to-end test that is not part of your product configuration, configure your Fuchsia image to include the specific test:

fx set <PRODUCT>.<BOARD> --with <TEST_DIRECTORY>:<TARGET>

For example, the following commands configure and build your Fuchsia image with all the end-to-end tests in the //src/tests/end_to_end/perf test directory:

$ fx set workstation.qemu-x64 --with //src/tests/end_to_end/perf:test
$ fx build

For the list of all available end-to-end tests in the Fuchsia repository, see the //src/tests/end_to_end directory.

Examine product configuration files

To find out which end-to-end tests are included in a specific product configuration, examine product configuration files (.gni) in the Fuchsia repository's //products directory.

The following example shows the product configurations files in the //products directory:

~/fuchsia/products$ ls *.gni
bringup.gni  core.gni  terminal.gni  workstation.gni

To see the list of all available product configurations, you can run the following command:

fx list-products

Among these product configurations, terminal and workstation include end-to-end tests by default. The following example shows the end-to-end tests included in terminal.gni:

cache_package_labels += [
  ...
  "//src/tests/end_to_end/bundles:end_to_end_deps",
  "//src/tests/end_to_end/bundles:terminal_end_to_end_deps",
]

...

universe_package_labels += [
  "//src/tests/end_to_end/screen_is_not_black",
  "//src/tests/end_to_end/sl4f:test",
  "//src/tests/end_to_end/perf:test",
  ...
]