# Build Clang toolchain

Fuchsia is using Clang as the official compiler.

## Prerequisites

You need [CMake](https://cmake.org/download/) version 3.13.4 or newer to
execute these commands. This is the [minimum required version](https://reviews.llvm.org/rGafa1afd4108)
to build LLVM.

While CMake supports different build systems, it is recommended to use
[Ninja](https://github.com/ninja-build/ninja/releases).

Both should be present in your Fuchsia checkout as prebuilts. The commands below
assume that `cmake` and `ninja` are in your `PATH`:

    export PATH=${FUCHSIA_DIR}/prebuilt/third_party/cmake/${platform}/bin:${PATH}
    export PATH=${FUCHSIA_DIR}/prebuilt/third_party/ninja/${platform}/bin:${PATH}

Where `${FUCHSIA_DIR}` refers to the root directory of your Fuchsia source tree.

### Getting Source

The example commands below use `${LLVM_SRCDIR}` to refer to the root of
your LLVM source tree checkout. You can use the official monorepo
<https://github.com/llvm/llvm-project>
maintained by the LLVM community:

    LLVM_SRCDIR=${HOME}/llvm/llvm-project
    git clone https://github.com/llvm/llvm-project ${LLVM_SRCDIR}

> [!NOTE]
> **Note:** It is recommended checking out to the revision that's currently used for Fuchsia. The latest upstream revision may be broken or fail to build Fuchsia, whereas it is guaranteed that the prebuilt revision can always build Fuchsia. This revision can be found in `[//integration/toolchain]`. Search for the package `fuchsia/third_party/clang/${platform}`, and checkout the `git_revision` associated with it.

    cd ${LLVM_SRCDIR}
    git checkout ${REVISON_NUMBER}

### Fuchsia IDK

Before building the runtime libraries that are built along with the
toolchain, you need a Fuchsia [IDK](https://fuchsia.dev/docs/development/idk)
(formerly known as the SDK).
The IDK must be located in the directory pointed to by the `${IDK_DIR}`
variable:

    IDK_DIR=${HOME}/fuchsia-idk

To download the latest IDK, you can use the following:

    # For Linux
    cipd install fuchsia/sdk/core/linux-amd64 latest -root ${IDK_DIR}

### Sysroot for Linux

To include compiler runtimes and C++ library for Linux, download the sysroot.
It must be located in the directory pointed by the `${SYSROOT_DIR}` variable.

    SYSROOT_DIR=${FUCHSIA_DIR}/prebuilt/third_party/sysroot

## Building a Clang Toolchain for Fuchsia

The Clang CMake build system supports bootstrap (aka multi-stage)
builds. Fuchsia uses [two-stage bootstrap build](https://fuchsia.dev/fuchsia-src/development/build/toolchain#two-stage-build) for the
Clang compiler.
However, for toolchain related development it is recommended to use
the [single-stage build](https://fuchsia.dev/fuchsia-src/development/build/toolchain#single-stage-build).

If your goal is to experiment with clang, the single-stage build is likely what you are looking for.
The first stage compiler is a host-only compiler with some options set
needed for the second stage. The second stage compiler is the fully
optimized compiler intended to ship to users.

Setting up these compilers requires a lot of options. To simplify the
configuration the Fuchsia Clang build settings are contained in CMake
cache files, which are part of the Clang codebase (`Fuchsia.cmake` and
`Fuchsia-stage2.cmake`).

> [!NOTE]
> **Note:** Clang must be built in a separate build directory. The directory itself can be a subdirectory or in a whole other path.

    mkdir llvm-build
    mkdir llvm-install  # For placing stripped binaries here
    INSTALL_DIR=${pwd}/llvm-install
    cd llvm-build

### Single Stage Build Fuchsia Configuration

When developing Clang for Fuchsia, you can use the cache file to
test the Fuchsia configuration, but run only the second stage, with LTO
disabled, which gives you a faster build time suitable even for
incremental development, without having to manually specify all options:

    cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug \
      -DCMAKE_TOOLCHAIN_FILE=${FUCHSIA_DIR}/scripts/clang/ToolChain.cmake \
      -DLLVM_ENABLE_LTO=OFF \
      -DLINUX_x86_64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux \
      -DLINUX_aarch64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux \
      -DLINUX_riscv64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/ubuntu20.04 \
      -DLINUX_armv7-unknown-linux-gnueabihf_SYSROOT=${SYSROOT_DIR}/linux \
      -DLINUX_i386-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux \
      -DFUCHSIA_SDK=${IDK_DIR} \
      -DCMAKE_INSTALL_PREFIX= \
      -C ${LLVM_SRCDIR}/clang/cmake/caches/Fuchsia-stage2.cmake \
      ${LLVM_SRCDIR}/llvm
    ninja toolchain-distribution  -j1000  # Build the distribution

If the above fails with an error related to Ninja, then you may need to add
`ninja` to your PATH. You can find the prebuilt executable at
`${FUCHSIA_DIR}//prebuilt/third_party/ninja/${platform}/bin`.

`ninja toolchain-distribution` should be enough for building all binaries, but
the Fuchsia build assumes some libraries are stripped so `ninja
install-toolchain-distribution-stripped` is necessary.

### Two-Stage Build Fuchsia Configuration

This is roughly equivalent to what is run on the prod builders and used to build
a toolchain that Fuchsia ships to users.

    cmake -GNinja \
      -DCMAKE_TOOLCHAIN_FILE=${FUCHSIA_DIR}/scripts/clang/ToolChain.cmake \
      -DCMAKE_INSTALL_PREFIX= \
      -DSTAGE2_LINUX_x86_64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux \
      -DSTAGE2_LINUX_aarch64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux \
      -DSTAGE2_LINUX_riscv64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/ubuntu20.04 \
      -DSTAGE2_LINUX_armv7-unknown-linux-gnueabihf_SYSROOT=${SYSROOT_DIR}/linux \
      -DSTAGE2_LINUX_i386-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux \
      -DSTAGE2_FUCHSIA_SDK=${IDK_DIR} \
      -C ${LLVM_SRCDIR}/clang/cmake/caches/Fuchsia.cmake \
      ${LLVM_SRCDIR}/llvm
    ninja stage2-toolchain-distribution -j1000
    DESTDIR=${INSTALL_DIR} ninja stage2-install-toolchain-distribution-stripped -j1000

> [!NOTE]
> **Note:** The second stage build uses LTO (Link Time Optimization) to achieve better runtime performance of the final compiler. LTO often requires a large amount of memory and is very slow. Therefore it may not be very practical for day-to-day development.

### runtime.json

If the Fuchsia build fails due to a missing `runtime.json` file, you must generate a new `runtime.json` file by running the following command:

    python3 ${FUCHSIA_DIR}/scripts/clang/generate_runtimes.py  \
      --clang-prefix ${INSTALL_DIR} --sdk-dir ${IDK_DIR}          \
      --build-id-dir ${INSTALL_DIR}/lib/debug/.build-id > ${INSTALL_DIR}/lib/runtime.json

The generated file contains relative paths used by the Fuchsia build to know where
various libraries from the toolchain are located.

### Putting it All Together

Copy-paste code for building a single-stage toolchain. This code can be run
from inside your LLVM build directory and assumes a linux environment.

    cd ${LLVM_BUILD_DIR}  # The directory your toolchain will be installed in

    # Environment setup
    FUCHSIA_DIR=${HOME}/fuchsia/  # Replace with wherever Fuchsia lives
    LLVM_SRCDIR=${HOME}/llvm/llvm-project  # Replace with wherever llvm-project lives
    IDK_DIR=${HOME}/fuchsia-idk/
    SYSROOT_DIR=${FUCHSIA_DIR}/prebuilt/third_party/sysroot
    CLANG_TOOLCHAIN_PREFIX=${FUCHSIA_DIR}/prebuilt/third_party/clang/linux-x64/bin/

    # Download necessary dependencies
    cipd install fuchsia/sdk/core/linux-amd64 latest -root ${IDK_DIR}

    # CMake invocation
    cmake -G Ninja -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_TOOLCHAIN_FILE=${FUCHSIA_DIR}/scripts/clang/ToolChain.cmake \
      -DLLVM_ENABLE_LTO=OFF \
      -DLINUX_x86_64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux \
      -DLINUX_aarch64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux \
      -DLINUX_riscv64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/ubuntu20.04 \
      -DLINUX_armv7-unknown-linux-gnueabihf_SYSROOT=${SYSROOT_DIR}/linux \
      -DLINUX_i386-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux \
      -DFUCHSIA_SDK=${IDK_DIR} \
      -DCMAKE_INSTALL_PREFIX= \
      -C ${LLVM_SRCDIR}/clang/cmake/caches/Fuchsia-stage2.cmake \
      ${LLVM_SRCDIR}/llvm

    # Build and strip binaries and place them in the install directory
    ninja toolchain-distribution -j1000
    DESTDIR=${INSTALL_DIR} ninja install-toolchain-distribution-stripped -j1000

    # Generate runtime.json

    python3 ${FUCHSIA_DIR}/scripts/clang/generate_runtimes.py    \
      --clang-prefix ${INSTALL_DIR} --sdk-dir ${IDK_DIR}            \
      --build-id-dir ${INSTALL_DIR}/lib/debug/.build-id > ${INSTALL_DIR}/lib/runtime.json

### Building Fuchsia with a Custom Clang

To specify a custom clang toolchain for building Fuchsia, pass
`--args clang_prefix=\"${INSTALL_DIR}/bin\"`
to `fx set` command and run `fx build`.

    fx set core.x64 --args=clang_prefix=\"${INSTALL_DIR}/bin\"
    fx build

This file contains relative paths used by the Fuchsia build to know where
various libraries from the toolchain are located.

> [!NOTE]
> **Note:** If you make another change to Clang after building Fuchsia with a previous version of Clang, re-running `fx build` may not always guarantee that all necessary targets will be built with the new Clang. For this case, should instead run `fx
> clean-build`, which will rebuild everything but definitely use the new Clang.

## Developing Clang

When developing Clang, you may want to use a setup that is more suitable for
incremental development and fast turnaround time.

The simplest way to build LLVM is to use the following commands:

    cmake -GNinja \
      -DCMAKE_BUILD_TYPE=Debug \
      -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld" \
      ${LLVM_SRCDIR}/llvm
    ninja

You can enable additional projects using the `LLVM_ENABLE_PROJECTS`
variable. To enable all common projects, you would use:

      -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;compiler-rt;libcxx;libcxxabi;libunwind"

Similarly, you can also enable some projects to be built as runtimes
which means these projects will be built using the just-built rather
than the host compiler:

      -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld" \
      -DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libunwind" \

Both `LLVM_ENABLE_PROJECTS` and `LLVM_ENABLE_RUNTIMES` are already set in the
CMake cache files, so you normally don't need to set these unless you would
like to explicitly add more projects or runtimes.

Clang is a large project and compiler performance is absolutely critical. To
reduce the build time, it is recommended to use Clang as a host compiler, and if
possible, LLD as a host linker. These should be ideally built using LTO and
for best possible performance also using Profile-Guided Optimizations (PGO).

To set the host compiler to Clang and the host linker to LLD, you can
use the following extra flags:

      -DCMAKE_C_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang \
      -DCMAKE_CXX_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang++ \
      -DLLVM_ENABLE_LLD=ON

This assumes that `${CLANG_TOOLCHAIN_PREFIX}` points to the `bin` directory
of a Clang installation, with a trailing slash (as this Make variable is used
in the Zircon build). For example, to use the compiler from your Fuchsia
checkout (on Linux):

    CLANG_TOOLCHAIN_PREFIX=${FUCHSIA_DIR}/prebuilt/third_party/clang/linux-x64/bin/

> [!NOTE]
> **Note:** To build Fuchsia, you need a stripped version of the toolchain runtime binaries. Use `DESTDIR=${INSTALL_DIR} ninja install-toolchain-distribution-stripped` to get a stripped install and then point your build configuration to `${INSTALL_DIR}/bin` as your toolchain.

### Sanitizers

Most sanitizers can be used on LLVM tools by adding
`LLVM_USE_SANITIZER=<sanitizer name>` to your cmake invocation. MSan is
special however because some LLVM tools trigger false positives. To
build with MSan support you first need to build libc++ with MSan
support. You can do this in the same build. To set up a build with MSan
support first run CMake with `LLVM_USE_SANITIZER=Memory` and
`LLVM_ENABLE_LIBCXX=ON`.

    cmake -GNinja \
      -DCMAKE_BUILD_TYPE=Debug \
      -DCMAKE_C_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang \
      -DCMAKE_CXX_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang++ \
      -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;libcxx;libcxxabi;libunwind" \
      -DLLVM_USE_SANITIZER=Memory \
      -DLLVM_ENABLE_LIBCXX=ON \
      -DLLVM_ENABLE_LLD=ON \
      ${LLVM_SRCDIR}/llvm

Normally you would run Ninja at this point but we want to build
everything using a sanitized version of libc++ but if we build now it
will use libc++ from `${CLANG_TOOLCHAIN_PREFIX}`, which isn't sanitized.
So first we build just the cxx and cxxabi targets. These will be used in
place of the ones from `${CLANG_TOOLCHAIN_PREFIX}` when tools
dynamically link against libcxx

    ninja cxx cxxabi

Now that you have a sanitized version of libc++ you can set your build to use
it instead of the one from `${CLANG_TOOLCHAIN_PREFIX}` and then build
everything.

    ninja

Putting that all together:

    cmake -GNinja \
      -DCMAKE_BUILD_TYPE=Debug \
      -DCMAKE_C_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang \
      -DCMAKE_CXX_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang++ \
      -DLLVM_USE_SANITIZER=Address \
      -DLLVM_ENABLE_LIBCXX=ON \
      -DLLVM_ENABLE_LLD=ON \
      ${LLVM_SRCDIR}/llvm
    ninja libcxx libcxxabi
    ninja

## Testing Clang

To run Clang tests, you can use the `check-<component>` target:

    ninja check-llvm check-clang

You can all use `check-all` to run all tests, but keep in mind that this
can take significant amount of time depending on the number of projects
you have enabled in your build.

To test only one specific test, you can use the environment variable
`LIT_FILTER`. If the path to the test is `clang/test/subpath/testname.cpp`, you
can use:

    LIT_FILTER=testname.cpp ninja check-clang

The same trick can be applied for running tests in other sub-projects by
specifying different a different `check-<component>`.

## Downloading Toolchains from CAS

Our Clang Toolchain CI builders upload all build artifacts to Content Addressed Storage (CAS).
It provides a convenient way to quickly download a specific toolchain without having to build from scratch.
This can greatly speedup investigations into toolchain issues, since you can avoid the long LLVM build times, on top of building Fuchsia.

Below is an example of how to install the `cas` tool from scratch and download a specific toolchain into a corpus directory:

    $ cipd install infra/tools/luci/cas/linux-amd64 latest -root luci
    $ ./luci/cas download -cas-instance chromium-swarm -digest \
        ad53e1f315a849955190594fde6b07e11e76b40563db5779fcc69d6a6e04dc71/267 -dir corpus

In the example above the `-digest` field is passed a unique id, which is used by the `cas` tool to fetch the correct artifacts.
The `digest` can be obtained from Fuchsia's CI builder by selecting the builder you want the toolchain from, then expanding the `clang`-\>`cas`-\>`archive` fields and then clicking on the `CAS_UI` link.
The resulting page will give show you some information about the CAS upload, including the digest.

## Useful CMake Flags

There are many other [CMake flags](https://llvm.org/docs/CMake.html#id11) that
are useful for building, but these are some that may be useful for toolchain
building.

### `-DLLVM_PARALLEL_LINK_JOBS`

Increase the number of link jobs that can be run in parallel (locally). The number of
link jobs is dependent on RAM size. For LTO build you will
need at least 10GB for each job.

## Additional Resources

Documentation:

- [Getting Started with the LLVM System](http://llvm.org/docs/GettingStarted.html)
- [Building LLVM with CMake](http://llvm.org/docs/CMake.html)
- [Advanced Build Configurations](http://llvm.org/docs/AdvancedBuilds.html)

Talks:

- [2016 LLVM Developers' Meeting: C. Bieneman "Developing and Shipping LLVM and Clang with CMake"](https://www.youtube.com/watch?v=StF77Cx7pz8)
- [2017 LLVM Developers' Meeting: Petr Hosek "Compiling cross-toolchains with CMake and runtimes build"](https://www.youtube.com/watch?v=OCQGpUzXDsY)

[](https://issues.fuchsia.dev/issues/new?component=1439230&template=1881640&description=Issue+on:+https://fuchsia.dev/docs/development/build/toolchain.md "Report a bug or suggest improvements on this page") [](https://cs.opensource.google/fuchsia/fuchsia/+/main:docs/development/build/toolchain.md "View the source code of this page") [](https://ci.android.com/edit?repo=fuchsia/fuchsia/main&file=docs/development/build/toolchain.md&cloneRepo=fuchsia/fuchsia:refs/heads/main&fullPath=fuchsia/docs/development/build/toolchain.md "Edit the source code of this page")