Testing is central to the continuous integration processes that Fuchsia contributors practice to sustain quality and velocity. Good tests become an asset to the team, while bad tests can turn into a liability.
This document reviews topics related to testing on Fuchsia, and provides references to additional resources. This document assumes that you are familiarized with general software testing concepts.
Fuchsia platform testing should serve the goals of the project and align with Fuchsia's architectural principles:
- Simple: Simple tests are better than complex tests. Unit tests are preferred over integration, system, or manual tests. Testing solutions on Fuchsia should exercise the same mechanisms that exist in production.
- Secure: Tests are subject to the same security, isolation, and hermeticity mechanisms that apply to production software. Tests leverage those same mechanisms for their benefit. The security properties of the system are testable.
- Updatable: It's important to test the stable interfaces between components and other parts of the stable Fuchsia System Interface. If components under tests change or are entirely replaced, tests that only exercise interfaces between components should continue to work. Tests that are outside of Fuchsia tree should not assume implementation details of platform components.
- Performant: Tests on Fuchsia should be fast, reliable, and flexible. When tests run quickly, it is easier to iterate and expend fewer resources. Tests should not be flaky, or picky about how they run on Fuchsia. It’s easier to run tests on an emulator than on real hardware.
How is testing different on Fuchsia?
Operating systems are complex programs
Every domain of software development and testing brings unique challenges. There are special problems and solutions to testing an operating system, as there are for testing a mobile application or server software or a spacecraft.
For instance, tests for the Zircon kernel run in a special runtime environment that makes as few assumptions as possible that the kernel is functional, and can detect mistakes in low-level kernel code. Contrast this with typical application testing, where the tests run on some operating system that is assumed to be working.
Isolation and hermeticity
The Component Framework promotes Fuchsia’s security goals by running each component in a sandbox environment that is strictly-defined in a component manifest. It then promotes Fuchsia’s updatability goals by only allowing components to interconnect using those component capabilities that are typed as updatable contracts.
These same mechanisms of isolation and hermeticity can also be used by tests as a form of dependency injection. For instance a component under test can be provided a test double for a capability that it depends on by a test harness, making contract testing easier.
Multiple repositories
Fuchsia is a large project with many external dependencies, and builds against hundreds of other source code repositories. Multi-repository development introduces unique challenges to testing. A contributor working on the WebRTC project published a blog post detailing many of the problems and solutions that are also encountered when developing Fuchsia.