| RFC-0144: Size Check Tool ||
|---|---|
| Status | Accepted |
| Areas | - Developer |
| Description | Proposes a rewrite of size_checker.go to create a more cohesive in-tree and out-of-tree developer experience. |
| Issues | - [85371](https://fxbug.dev/85371) |
| Gerrit change | - [609963](https://fuchsia-review.googlesource.com/c/fuchsia/+/609963) |
| Authors | - [awolter@google.com](https://fuchsia-review.googlesource.com/q/owner:awolter@google.com) |
| Reviewers | - [sebmarchand@google.com](https://fuchsia-review.googlesource.com/q/owner:sebmarchand@google.com) - [amituttam@google.com](https://fuchsia-review.googlesource.com/q/owner:amituttam@google.com) - [samans@google.com](https://fuchsia-review.googlesource.com/q/owner:samans@google.com) - [atyfto@google.com](https://fuchsia-review.googlesource.com/q/owner:atyfto@google.com) - [rohpavone@google.com](https://fuchsia-review.googlesource.com/q/owner:rohpavone@google.com) - [hjfreyer@google.com](https://fuchsia-review.googlesource.com/q/owner:hjfreyer@google.com) |
| Date submitted (year-month-day) | 2021-11-22 |
| Date reviewed (year-month-day) | 2021-12-10 |

[Edit this RFC](https://ci.android.com/edit?repo=fuchsia/fuchsia/main&file=docs/contribute/governance/rfcs/0144_size_check_tool.md)
[Edit RFC metadata](https://ci.android.com/edit?repo=fuchsia/fuchsia/main&file=docs/contribute/governance/rfcs/_rfcs.yaml&searchAndJump=-+name:+'RFC-0144')

## Summary

This document proposes the development of a new size check tool that will be
added to the SDK, which allows developers and product owners to verify that
their packages and products do not exceed specified size budgets. While the
implementation details are intentionally excluded, because out-of-tree Assembly
is continually shifting the foundation, design requirements and goals are
mentioned.

## Motivation

During software development, it is helpful to know whether your software fits
within the size constraints of your target. Fuchsia has a size check tool that
analyzes the space consumed by different groups of software, and enforces a
budget for each group. This tool is tightly bound to the implementation details
of `fuchsia.git`, therefore it cannot be used without a `fuchsia.git` checkout.
As a result, consumers of Fuchsia, such as Chromium, have implemented their own
ad-hoc method for checking the size of their software. By rewriting the size
check tool as a ffx plugin and adding it to the Fuchsia SDK, the method of
enforcing size budgets on Fuchsia software will be unified, making it easier for
future developers to get started with Fuchsia.

## Stakeholders

Who has a stake in whether this RFC is accepted.

*Facilitator:*

- Hunter Freyer (hjfreyer@google.com)

*Reviewers:*

- Saman Sami (samans@google.com) - Functionality
- Anthony Fadrianto (atyfto@google.com) - Integration with Infrastructure
- Sébastien Marchand (sebmarchand@google.com) - Integration with Infrastructure
- Rohan Pavone (rohpavone@google.com) - Integration with Chromium
- Amit Uttamchandani (amituttam@google.com) - Integration into ffx

*Consulted:*

- Aaron Wood (aaronwood@google.com) - Assembly Integration

*Socialization:*

This project was socialized as an internal document and discussed in meetings
with the stakeholders ahead of time. The requirements were already gathered
before this RFC was drafted.

## Background

This RFC assumes an understanding of the following topics.

- [Filesystems](https://fuchsia.dev/docs/concepts/filesystems/filesystems)
- [Packages](https://fuchsia.dev/docs/development/idk/documentation/packages)
- [BlobFS](https://fuchsia.dev/docs/concepts/filesystems/blobfs)
- [OTAs](https://fuchsia.dev/docs/concepts/packages/ota)

### Two flavors of size checking

Size checking can be separated into two categories:

1. Images do not exceed their target partition size when flashing.
2. Blobs fit within the FVM budget during updates.

**Flashing**: A developer can flash new images onto a Fuchsia target. When an
image is flashed, the contents of the partition is completely replaced by the
bytes of the new image. To ensure a successful flash, the new images must not be
larger than the size of the partitions.

**Updating** : A Fuchsia target can take a update [Over-The-Air](https://fuchsia.dev/docs/concepts/packages/ota). To ensure
a successful update, the total space consumed by blobs must not exceed the
budget set for the FVM. Often times, space in the FVM is reserved for an
[UpdatePackage](https://fuchsia.dev/docs/concepts/packages/update_pkg) and additional blobs, which means that the tool
cannot simply fill the FVM then check that the final image fits on the
partition.

### Current tools

Fuchsia supplies two tools that assist in size checking:

- size_checker.go
- blobfs-compression

#### `size_checker.go`

[Link to source](https://fuchsia.googlesource.com/fuchsia/+/refs/heads/main/tools/size_checker)

This tool can be used in-tree (inside `fuchsia.git`) to ensure blobs fit within
their budget. `size_checker.go` does this by determining the compressed and
aligned size of a directory of blobs in the Fuchsia repository. The budgets can
be defined per-product, but are currently not used for any products in
`fuchsia.git`. Because it specifically checks directories in `fuchsia.git`, the
tool is incapable of being used without a `fuchsia.git` checkout.

First, the Fuchsia build produces a `blobs.json` file that lists the compressed
and aligned size of every blob in blobfs, and a `blob.manifest` file that lists
the merkle and source path of those blobs.

Next, `size_checker.go` is run and performs the following:

1. Reads `blobs.json` to collect the compressed size of every blob: `compressed_size`.
2. Reads `blob.manifest` to collect all packages.
3. Calculates the number of times a blob is included in a package: `share_count`.
4. Calculates the shared size of each blob: `shared_size = compressed_size /
   share_count`. Because multiple budgets could include the same blob, we use the `shared_size` of a blob to distribute the size consumption evenly across the budgets.
5. Constructs an N-ary sum tree representing the space consumed by every directory in the build output by adding each blob's `shared_size` to the nodes in the blob's path.
6. Iterates over each budget, and asserts that the size budgeted for a directory is larger than the space consumed.
7. Iterates over each non-blobfs package in the budgets, uses `blobfs-compression` to calculate the sum of the blob sizes, and asserts that it falls within the budget.
8. Prints the results.

![Sum Tree](https://fuchsia.dev/static/fuchsia-src/contribute/governance/rfcs/resources/0144_size_check_tool/sum_tree.png)

#### `blobfs-compression`

[Link to source](https://fuchsia.googlesource.com/fuchsia/+/refs/heads/main/src/storage/tools/blobfs-compression)

This tool can estimate the compressed and aligned size of a set of blobs, but is
not 100% accurate, and may occasionally over-estimate the size of a blob. The
tool will never under-estimate the size of a blob.

`blobfs-compression` is delivered to clients via the Fuchsia SDK, and is
currently the only method of estimating the size of code built for Fuchsia that
exists outside of `fuchsia.git`. Chromium is a client of this tool.

### Problems with current tools

- In-Tree and Out-Of-Tree methods for checking package sizes are different tools.
- Neither tool can directly check the size of images to ensure flashing succeeds.
- `size_checker.go`
  - Is not documented.
  - Depends on the implementation details of the Fuchsia build, which is not a stable contract.
  - Does not work out-of-tree.
- `blobfs-compression`
  - Is not guaranteed to return the accurate compressed size of a blob.
  - Operates at a blob-level, which is not as useful as operating at a package-level. Developers add software to a product on a per-package basis.
  - Does not have a standard output format, therefore it is not easily scriptable, and automated usage is fragile.

## Requirements

1. Provide a tool or tools in the SDK that work both In-Tree and Out-Of-Tree.
2. The tool(s) support all desired existing use cases of the current tools.
3. The tool(s) can perform two types of size checks:
   - (a) Images
   - (b) Blobs in a set of packages
4. In case (2), blob deduplication across the set of all packages in all sets will be performed.
5. The tool(s) will return a failure when a user-specified budget is exceeded.
6. The tool(s) work together nicely with the other SDK tools, such as the Image Assembler ([RFC-0072](https://fuchsia.dev/docs/contribute/governance/rfcs/0072_standalone_image_assembly_tool)).
7. The tool(s) can be easily used in scripts, with parsable output, and produces the output used by the [Gerrit Size Plugin](https://chromium.googlesource.com/infra/gerrit-plugins/binary-size/+/HEAD/README.md).
8. The tool(s) usage and architecture will be documented in fuchsia.dev.

## Design

### Cohesive Developer Experience

A new `ffx` plugin will be written in Rust that performs the two types of size
checks. Incorporating it into `ffx` is preferable, as per the
[CLI Rubric](https://fuchsia.dev/docs/development/api/cli), to encourage a shared workflow, and increase the
discoverability. Also, preference will be given to expanding existing file
formats, rather than inventing new ones.

### Scriptable

This tool will primarily be used in build systems, therefore the output should
be parsable. The output format will likely use json5 as the output format, but
as the Assembly project progresses, alternate formats may be considered.

### Image Sizes

The image size checker should be run at the same time as the construction of the
flash manifest, where the assembled images are mapped to partitions. Currently,
flash manifest creation is done in the Fuchsia build system, so the `ffx` plugin
will be invoked via a GN action. In the future, a SDK tool may be provided for
constructing the flash manifest, and as a `ffx` plugin, our size checker tool
should work seamlessly in that scenario.

Measuring the size of an image is a simple as opening the file and reading the
length in the metadata. This works for any images that are not compressed or
sparse. An example of where this method does not work is sparse FVMs, and
because there currently exists no library for calculating the expanded size of a
sparse FVM, the tool will ignore size checks for sparse FVMs.

### Package Sizes

Blob and package size checks should be closely integrated with Assembly, because
it is during Assembly that engineers group packages into sets and combine these
sets to specify products.

The size check tool uses `blobs.json` files that list the compressed and aligned
sizes of each blob. The developer may provide multiple `blobs.json` files as an
input to the tool. If a package's blob is not found in any provided `blobs.json`
files, then the size check tool will generate a single blobfs image for all the
missing blobs, using the `blobfs` tool, which generates an additional
`blobs.json`. The size check tool will read the generated `blobs.json` to get
the sizes of the missing blobs.

Similar to the existing `size_checker.go` tool, the new size check tool will
calculate the shared size of each blob by dividing the size by the number of
packages that use the blob. This shared size is what is used during budgeting.

## Implementation

The implementation and integration of this tool will be completed in several
stages.

1. Declare the budgets for the package sets. See the [explanation](https://fuchsia.dev/fuchsia-src/contribute/governance/rfcs/0144_size_check_tool.md.txt#budget-generation) below.
2. Write a ffx plugin that ensures the package sets fall within their budget
3. Ensure that the output of the ffx plugin matches that of the previous tool as part of the build system. If the outputs are different, then the build should fail. See the [explanation](https://fuchsia.dev/fuchsia-src/contribute/governance/rfcs/0144_size_check_tool.md.txt#output-comparison) below.
4. Declare the budgets for the images
5. Add functionality to ensure the image sizes fall within their budgets
6. Disable `size_checker.go` and make the new ffx plugin load bearing
7. Delete the `size_checker.go` code.

### Budget file generation

In order to avoid the toil of maintaining two identical sets of budgets for a
given product, the build system will be updated to read the existing
`size_limits.json` and assembly [product config](https://fuchsia.googlesource.com/fuchsia/+/refs/heads/main/src/developer/ffx/plugins/assembly/#product-config) and produce the
new budgets file. The product config will be parsed to collect every package
manifest, and those manifest will be categoried according to their directory
budgets set in `size_limits.json`.

A potential budget file format is:

```json5
[
    {
        name: "name-of-package-set",
        packages: [
            "path/to/manifest1.json",
            "path/to/manifest2.json",
        ],
        budget_bytes: 12000,
    },
]
```

Once the new size check tool is load-bearing for the build, the generated
budgets file will be checked into the repository and the generated code will be
deleted.

### Output comparison

During the transition to the new size checker, the build system will assert that
the output of the new tool matches that of the new tool. A script will be
written that parses the outputs of both tools, and ensures that the budgets and
calculated consumption for each package group (or directory) is equivalent. In
order to correlate the two output formats, the script will assume that the names
identifying the sets of packages are identical. Because both the old tool and
the new tool are using the same underlying mechanism for calculating the size of
a blob (using `blobfs`), the calculated consumption will also be identical.

## Performance

The size check tool is intended for use in a build system, therefore is has a
higher tolerance for performance instability than, for example, a network stack.
On the other hand, lowering build times is important for developer productivity.
Additionally, this tool is on the critical path for a build, because it must
wait for all the packages to be build, and the images to be assembled before it
can start running, therefore it is not parallelizable.

The current `size_checker.go` tool takes approximately 1-2 seconds to run, and
our new tool should aim to be close to the same duration.

## Ergonomics

See the [Design](https://fuchsia.dev/fuchsia-src/contribute/governance/rfcs/0144_size_check_tool.md.txt#design) section.

## Backwards Compatibility

This design does not need to be backwards compatible.

## Security considerations

This design has no security implications.

## Privacy considerations

This design has no privacy implications.

## Testing

In the Fuchsia build, the new size check tool will be ran alongside
`size_checker.go`, and the output will be compared to ensure that they have the
same budgets and calculated space consumption.

Unit tests will also be written as features are added to the tool.

## Documentation

Documentation will be added to [fuchsia.dev](https://fuchsia.dev).

## Drawbacks, alternatives, and unknowns

There exist a few unfavorable alternatives.

**Do nothing**: This would require clients to create their own ad-hoc method for
completing size checks, which does not scale well as our client base increases.

**Refactor existing tools** : `size_checker.go` could be refactored to be
supported out-of-tree. According to the [CLI Rubric](https://fuchsia.dev/docs/development/api/cli), the preference
is that public tools are added to `ffx` to encourage a shared workflow and
better discovery. Also, the changes required to refactor the current code are so
drastic that it is actually less work to rewrite the tool entirely.

**Use blobfs-compression** : In order to calculate the compressed and aligned
size of each blob, the `blobfs-compression` tool could be used instead of
generating a new blobfs image. This alternative has the major drawback, that the
`blobfs-compression` tool is not guaranteed to have an accurate compression or
alignment. To be specific the `blobfs-compression` tool assumes non-compact
merkle trees, which often causes it to result in a larger blob size than
reality. Additionally, the `blobfs-compression` tool can only measure one blob
at a time, and is therefore more cumbersome to use, and likely slower than
running the `blobfs` tool once. Generating a blobfs image is more inline with
the current `size_checker.go` and more accurate.

## Prior art and references

See the [Background](https://fuchsia.dev/fuchsia-src/contribute/governance/rfcs/0144_size_check_tool.md.txt#background) section.
[](https://issues.fuchsia.dev/issues/new?component=1439230&template=1881640&description=Issue+on:+https://fuchsia.dev/docs/contribute/governance/rfcs/0144_size_check_tool.md "Report a bug or suggest improvements on this page") [](https://cs.opensource.google/fuchsia/fuchsia/+/main:docs/contribute/governance/rfcs/0144_size_check_tool.md "View the source code of this page") [](https://ci.android.com/edit?repo=fuchsia/fuchsia/main&file=docs/contribute/governance/rfcs/0144_size_check_tool.md&cloneRepo=fuchsia/fuchsia:refs/heads/main&fullPath=fuchsia/docs/contribute/governance/rfcs/0144_size_check_tool.md "Edit the source code of this page")