|RFC-0072: Standalone Image Assembly Tool|
Create a standalone image assembly tool that can also be used by the Fuchsia build.
|Date submitted (year-month-day)||2021-01-29|
|Date reviewed (year-month-day)||2021-03-05|
This document proposes a standalone tool, suitable for inclusion in a Fuchsia SDK, for assembling Fuchsia "system images" from packages and other compiled artifacts.
Product assembly, the process of creating a set of artifacts for delivering
Fuchsia to a device out of the components and packages that are in the Fuchsia
platform fuchsia.git and the product's own, and other repo(s), should be
possible without using a complete
fuchsia.git checkout and build.
This tool performs the final step of the product assembly process, the creation of the system artifacts, from previously built inputs.
This tool is also used as part of the
fuchsia.git build itself, replacing
many of the scripts and GN templates that are in
The creation of the final system output files that can be used to deliver Fuchsia to a device.
The package in BlobFS, identified to
pkgfs by its content identity, which
contains the base system, (e.g.
/system). This is called the
package in the current build. See this for more information.
Board Support Data
These are the inputs that describe the low-level hardware details needed to perform the assembly process (e.g partition tables, flash-block sizes, device bootloader images for flashing onto the device or including in the update package, etc.).
Content-addressing is a way of identifying a thing based on a cryptographically-secure hash of its contents. On Fuchsia, this is used by BlobFS, pkgfs, and other parts of the system to identify files in a secure manner.
The FVM image is the block-device image for the Fuchsia Volume Manager.
System Image Artifacts
The set of final output artifacts that are created by the build which contain Fuchsia. This is the larger set of artifacts which are used to deliver Fuchsia to devices via different means (OTA, flashing, paving, etc.)
The package that contains files and rules for updating the system. See this for more information.
The verified-boot metadata is used by the bootloader(s) running on the device to validate that the zbi is trusted for execution by the bootloader.
This tool is primarily a replication of steps and processes that are currently
//build/resources/BUILD.gn and its associated
The tool is an
ffx plug-in that allows it to be used both as part of the
fuchsia build, as well as outside of it.
Operationally, the tool takes as input:
A set of options specifying which image files it should create:
- The ZBI
- The vbmeta image
- The flash images:
- BlobFS block device image
- FVM block device image
- Final image manifest
The following diagrams show which inputs (and outputs) are used to create the various final outputs:
┌─────────┐┌────────┐┌────────┐┌───────────┐┌─────────────────┐ │ Board ││ Kernel ││ Kernel ││ List of ││ List of │ │ Support ││ Image ││ Args ││ BootFS ││ packages │ └┬──┬─────┘└┬───────┘└┬───────┘│ Files │└┬───────────────┬┘ │ │ │ │ └┬──────────┘ │ │ │ │ │ │ ┌V───────┐ ┌V─────────────┐ │ │ │ │ │ │ BootFS │ │ base package │ │ │ │ │ │ └┬───────┘ └┬──┬──────────┘ │ │ │ ┌V─────────V─────────V────────────V┐ │ │ │ │ │ZBI │ │ │ │ │ └┬───────┬─────────────────────────┘ │ │ │ │ ┌V─────┐ │ │ │ │ │ │VBMeta│ │ │ │ │ │ └┬─────┘ │ │ │ │ ┌V───────V───────V───────────────────────────V────────────V┐ │ │ update package │ │ └┬──┬──────────────────────────────────────────────────────┘ │ │ ┌V───────┐ │ │ │ BlobFS │ │ │ └┬───────┘ ┌V──V──V┐ │ FVM │ └───────┘
To create the
base package, the tool needs:
- The list of package files to incorporate into the system (packages in the the "base" and "cache" package sets)
To create the ZBI, the tool needs:
- A list of files (with access to them) to incorporate into bootfs
basepackage's "content identity"
- A Zircon kernel image to place in the ZBI
- The command line arguments to pass to the Zircon kernel
To create the vbmeta image, the tool needs:
- The ZBI
To create the update package, the tool needs:
- The list of packages incorporated into
- The ZBI
- The vbmeta image
- The ZBI for the recovery slot (optional)
- The vmbeta image for the recovery slot (if recovery zbi is given)
- The bootloader firmware images
To create the flashable block device images, the tool needs:
- Board Support Data:
- partition table
- and either:
- or directly provide:
- packages for BlobFS
The assembly tool generates the following outputs, depending on which it is instructed to create:
- blobfs block device image
- fvm flash image
The following manifests are produced, when the tool is instructed to create the output files that are described within the manifests.
- Manifest of all packages (including
- Manifest of all image files produced, containing:
- The content-identity hash of the image
- The architecture the images are for
- For all files incorporated into the image:
- Their own content-identity
- The origination (file-path) of that file
Inputs and Schemas
For build-tool compatibility, inputs will initially be what GN produces. For example, the results of the metadata walks that are used to describe all packages that are being built.
The final tool will be constructed from:
- An ffx plugin allowing it to be used via
- A Rust library containing the majority of the implementation and unit-tests.
- GN template for correctly using the tool in the fuchsia.git build.
- The existing tools, packaged for its use:
To facilitate the transition for the fuchsia.git in-tree build, there will also be:
- A CLI tool that exposes specific functionality as-needed for having a smooth transition from the existing GN templates and scripts to the new tool.
- Updated GN templates to wrap that functionality.
The above transitional tools will not be part of the permanent interface for the tool, but used to provide a transition path that minimizes risk to the fuchsia.git build.
Soft Launch Plan
To mitigate risks, the tool will be integrated into fuchsia.git carefully:
- As an integration test that logs if its output does not match that of the existing scripts and tools
- Then that test becomes a failing test
- After rolling and a few days of baking against numerous CQ builds, the new tools replace the existing
The addition of this tool to the SDK doesn't change any existing backwards compatibility concerns, as it is a different way of using existing tools that currently exist in the SDK. The restriction where the kernel and drivers used with the tool(s) should match the SDK tools (or be newer than the tooling) remains.
Use of the CLI tool has a neglible impact to the speed of the Fuchsia build. While it moves a number of operations from Python to Rust, it also adds itself as a compilation step that must be performed.
The existence of the tool allows for the re-assembly of a different set of components without performing a full build of Fuchsia itself.
The output manifests allow for auditing of the contents and provenance of the image artifacts produced.
No privacy concerns.
The core library will have unit tests which covers:
- input validation
- schema parsing
- schema generation
- each step of the assembly process's operation:
- generated command lines for external tools that are run
- parsing the output of external tools
- correct generation and parsing of intermediate files
ffx interface for this tool will need to be documented.