Google is committed to advancing racial equity for Black communities. See how.

Build Clang toolchain

Fuchsia is using Clang as the official compiler.

Prerequisites

You need CMake version 3.13.4 or newer to execute these commands. This is the minimum required version to build LLVM.

While CMake supports different build systems, it is recommended to use Ninja.

Both should be present in your Fuchsia checkout as prebuilts. The commands below assume that cmake and ninja are in your PATH:

export PATH=${FUCHSIA}/prebuilt/third_party/cmake/${platform}/bin:${PATH}
export PATH=${FUCHSIA}/prebuilt/third_party/ninja/${platform}/bin:${PATH}

Getting Source

The example commands below use ${LLVM_SRCDIR} to refer to the root of your LLVM source tree checkout. You can use the official monorepo https://github.com/llvm/llvm-project maintained by the LLVM community:

LLVM_SRCDIR=${HOME}/llvm/llvm-project
git clone https://github.com/llvm/llvm-project ${LLVM_SRCDIR}
cd ${LLVM_SRCDIR}
git checkout ${REVISON_NUMBER}

Fuchsia IDK/SDK

Before building the runtime libraries that are built along with the toolchain, you need a Fuchsia IDK (formerly known as the SDK). The IDK must be located in the directory pointed to by the ${IDK_DIR} variable:

IDK_DIR=${HOME}/fuchsia-idk

To download the latest IDK, you can use the following:

# For Linux
cipd install fuchsia/sdk/core/linux-amd64 latest -root ${IDK_DIR}

# For macOS
cipd install fuchsia/sdk/core/mac-amd64 latest -root ${IDK_DIR}

Sysroot for Linux

To include compiler runtimes and C++ library for Linux, download the sysroot for both arm64 and x64. Both sysroots must be located in the directory pointed by the ${SYSROOT_DIR} variable.

SYSROOT_DIR=${HOME}/fuchsia-sysroot/

To download the latest sysroots, you can use the following:

cipd install fuchsia/sysroot/linux-arm64 latest -root ${SYSROOT_DIR}/linux-arm64
cipd install fuchsia/sysroot/linux-amd64 latest -root ${SYSROOT_DIR}/linux-x64

Building a Clang Toolchain for Fuchsia

The Clang CMake build system supports bootstrap (aka multi-stage) builds. Fuchsia uses two-stage bootstrap build for the Clang compiler. However, for toolchain related development it is recommended to use the single-stage build.

If your goal is to experiment with clang, the single-stage build is likely what you are looking for. The first stage compiler is a host-only compiler with some options set needed for the second stage. The second stage compiler is the fully optimized compiler intended to ship to users.

Setting up these compilers requires a lot of options. To simplify the configuration the Fuchsia Clang build settings are contained in CMake cache files which are part of the Clang codebase (Fuchsia.cmake and Fuchsia-stage2.cmake).

In the following CMake invocations, ${CLANG_TOOLCHAIN_PREFIX} refers to the directory of binaries from a previous Clang toolchain. Normally, this refers to the current toolchain shipped with Fuchsia, but any references to binaries from this directory could theoretically be replaced with one's own binaries.

# FUCHSIA_SRCDIR refers to the root directory of your Fuchsia source tree
CLANG_TOOLCHAIN_PREFIX=${FUCHSIA_SRCDIR}/prebuilt/third_party/clang/linux-x64/bin/
mkdir llvm-build
mkdir llvm-install  # For placing stripped binaries here
INSTALL_DIR=${pwd}/llvm-install
cd llvm-build

Single Stage Build Fuchsia Configuration

When developing Clang for Fuchsia, you can use the cache file to test the Fuchsia configuration, but run only the second stage, with LTO disabled, which gives you a faster build time suitable even for incremental development, without having to manually specify all options:

cmake -G Ninja -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_C_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang \
  -DCMAKE_CXX_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang++ \
  -DCMAKE_ASM_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang \
  -DCMAKE_C_COMPILER_LAUNCHER=${GOMA_DIR}/gomacc \
  -DCMAKE_CXX_COMPILER_LAUNCHER=${GOMA_DIR}/gomacc \
  -DCMAKE_ASM_COMPILER_LAUNCHER=${GOMA_DIR}/gomacc \
  -DLLVM_ENABLE_LTO=OFF \
  -DLINUX_x86_64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux-x64 \
  -DLINUX_aarch64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux-arm64 \
  -DFUCHSIA_SDK=${IDK_DIR} \
  -DCMAKE_INSTALL_PREFIX= \
  -C ${LLVM_SRCDIR}/clang/cmake/caches/Fuchsia-stage2.cmake \
  ${LLVM_SRCDIR}/llvm
ninja distribution  -j1000  # Build the distribution

If the above fails with an error related to Ninja, then you may need to add ninja to your PATH. You can find the prebuilt executable at //prebuilt/third_party/ninja/${platform}/bin.

ninja distribution should be enough for building all binaries, but the Fuchsia build assumes some libraries are stripped so ninja install-distribution-stripped is necessary.

Two-Stage Build Fuchsia Configuration

This is roughly equivalent to what is run on the prod builders and used to build a toolchain that Fuchsia ships to users.

cmake -GNinja \
  -DCMAKE_C_COMPILER=${CLANG_TOOLCHAIN_PREFIX}/clang \
  -DCMAKE_CXX_COMPILER=${CLANG_TOOLCHAIN_PREFIX}/clang++ \
  -DCMAKE_ASM_COMPILER=${CLANG_TOOLCHAIN_PREFIX}/clang \
  -DCMAKE_C_COMPILER_LAUNCHER=${GOMA_DIR}/gomacc \
  -DCMAKE_CXX_COMPILER_LAUNCHER=${GOMA_DIR}/gomacc \
  -DCMAKE_ASM_COMPILER_LAUNCHER=${GOMA_DIR}/gomacc \
  -DCMAKE_INSTALL_PREFIX= \
  -DSTAGE2_LINUX_aarch64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux-arm64 \
  -DSTAGE2_LINUX_x86_64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux-amd64 \
  -DSTAGE2_FUCHSIA_SDK=${IDK_DIR} \
  -C ${LLVM_SRCDIR}/clang/cmake/caches/Fuchsia.cmake \
  ${LLVM_SRCDIR}/llvm
ninja stage2-distribution -j1000
DESTDIR=${INSTALL_DIR} ninja stage2-install-distribution-stripped -j1000

runtime.json

If the Fuchsia build fails due to a missing runtime.json file, you can copy them over from the prebuilt toolchain.

cp ${FUCHSIA_SRCDIR}/prebuilt/third_party/clang/linux-x64/lib/runtime.json ${INSTALL_DIR}/lib/

This file contains relative paths used by the Fuchsia build to know where various libraries from the toolchain are located.

Putting it All Together

Copy-paste code for building a single-stage toolchain. This code can be run from inside your LLVM build directory and assumes a linux environment.

cd ${LLVM_BUILD_DIR}  # The directory your toolchain will be installed in

# Environment setup
FUCHSIA_SRCDIR=${HOME}/fuchsia/  # Replace with wherever Fuchsia lives
LLVM_SRCDIR=${HOME}/llvm/llvm-project  # Replace with wherever llvm-project lives
IDK_DIR=${HOME}/fuchsia-idk/
SYSROOT_DIR=${HOME}/fuchsia-sysroot/
CLANG_TOOLCHAIN_PREFIX=${FUCHSIA_SRCDIR}/prebuilt/third_party/clang/linux-x64/bin/
GOMA_DIR=${FUCHSIA_SRCDIR}/prebuilt/third_party/goma/linux-x64/

# Download necessary dependencies
cipd install fuchsia/sdk/core/linux-amd64 latest -root ${IDK_DIR}
cipd install fuchsia/sysroot/linux-arm64 latest -root ${SYSROOT_DIR}/linux-arm64
cipd install fuchsia/sysroot/linux-amd64 latest -root ${SYSROOT_DIR}/linux-x64

# CMake invocation
cmake -G Ninja -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_C_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang \
  -DCMAKE_CXX_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang++ \
  -DCMAKE_C_COMPILER_LAUNCHER=${GOMA_DIR}/gomacc \
  -DCMAKE_CXX_COMPILER_LAUNCHER=${GOMA_DIR}/gomacc \
  -DLLVM_ENABLE_LTO=OFF \
  -DLINUX_x86_64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux-x64 \
  -DLINUX_aarch64-unknown-linux-gnu_SYSROOT=${SYSROOT_DIR}/linux-arm64 \
  -DFUCHSIA_SDK=${IDK_DIR} \
  -DCMAKE_INSTALL_PREFIX= \
  -C ${LLVM_SRCDIR}/clang/cmake/caches/Fuchsia-stage2.cmake \
  ${LLVM_SRCDIR}/llvm

# Build and strip binaries and place them in the install directory
ninja distribution -j1000
DESTDIR=${INSTALL_DIR} ninja install-distribution-stripped -j1000

# Get runtimes.json
cp ${FUCHSIA_SRCDIR}/prebuilt/third_party/clang/linux-x64/lib/runtime.json ${INSTALL_DIR}/lib/

Building Fuchsia with a Custom Clang

To specify a custom clang toolchain for building Fuchsia, pass --args clang_prefix=\"${LLVM_BUILD_DIR}/bin\" --no-goma to fx set command and run fx build.

fx set core.x64 --args=clang_prefix=\"${LLVM_BUILD_DIR}/bin\" --no-goma
fx build

This file contains relative paths used by the Fuchsia build to know where various libraries from the toolchain are located.

Developing Clang

When developing Clang, you may want to use a setup that is more suitable for incremental development and fast turnaround time.

The simplest way to build LLVM is to use the following commands:

cmake -GNinja \
  -DCMAKE_BUILD_TYPE=Debug \
  -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld" \
  ${LLVM_SRCDIR}/llvm
ninja

You can enable additional projects using the LLVM_ENABLE_PROJECTS variable. To enable all common projects, you would use:

  -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;compiler-rt;libcxx;libcxxabi;libunwind"

Similarly, you can also enable some projects to be built as runtimes which means these projects will be built using the just-built rather than the host compiler:

  -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld" \
  -DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libunwind" \

Both LLVM_ENABLE_PROJECTS and LLVM_ENABLE_RUNTIMES are already set in the CMake cache files, so you normally don't need to set these unless you would like to explicitly add more projects or runtimes.

Clang is a large project and compiler performance is absolutely critical. To reduce the build time, it is recommended to use Clang as a host compiler, and if possible, LLD as a host linker. These should be ideally built using LTO and for best possible performance also using Profile-Guided Optimizations (PGO).

To set the host compiler to Clang and the host linker to LLD, you can use the following extra flags:

  -DCMAKE_C_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang \
  -DCMAKE_CXX_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang++ \
  -DLLVM_ENABLE_LLD=ON

This assumes that ${CLANG_TOOLCHAIN_PREFIX} points to the bin directory of a Clang installation, with a trailing slash (as this Make variable is used in the Zircon build). For example, to use the compiler from your Fuchsia checkout (on Linux):

CLANG_TOOLCHAIN_PREFIX=${FUCHSIA}/prebuilt/third_party/clang/linux-x64/bin/

Sanitizers

Most sanitizers can be used on LLVM tools by adding LLVM_USE_SANITIZER=<sanitizer name> to your cmake invocation. MSan is special however because some LLVM tools trigger false positives. To build with MSan support you first need to build libc++ with MSan support. You can do this in the same build. To set up a build with MSan support first run CMake with LLVM_USE_SANITIZER=Memory and LLVM_ENABLE_LIBCXX=ON.

cmake -GNinja \
  -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_C_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang \
  -DCMAKE_CXX_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang++ \
  -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;libcxx;libcxxabi;libunwind" \
  -DLLVM_USE_SANITIZER=Memory \
  -DLLVM_ENABLE_LIBCXX=ON \
  -DLLVM_ENABLE_LLD=ON \
  ${LLVM_SRCDIR}/llvm

Normally you would run Ninja at this point but we want to build everything using a sanitized version of libc++ but if we build now it will use libc++ from ${CLANG_TOOLCHAIN_PREFIX} which isn't sanitized. So first we build just the cxx and cxxabi targets. These will be used in place of the ones from ${CLANG_TOOLCHAIN_PREFIX} when tools dynamically link against libcxx

ninja cxx cxxabi

Now that you have a sanitized version of libc++ you can set your build to use it instead of the one from ${CLANG_TOOLCHAIN_PREFIX} and then build everything.

ninja

Putting that all together:

cmake -GNinja \
  -DCMAKE_BUILD_TYPE=Debug \
  -DCMAKE_C_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang \
  -DCMAKE_CXX_COMPILER=${CLANG_TOOLCHAIN_PREFIX}clang++ \
  -DLLVM_USE_SANITIZER=Address \
  -DLLVM_ENABLE_LIBCXX=ON \
  -DLLVM_ENABLE_LLD=ON \
  ${LLVM_SRCDIR}/llvm
ninja libcxx libcxxabi
ninja

Testing Clang

To run Clang tests, you can use the check-<component> target:

ninja check-llvm check-clang

You can all use check-all to run all tests, but keep in mind that this can take significant amount of time depending on the number of projects you have enabled in your build.

To test only one specific test, you can use the environment variable LIT_FILTER. If the path to the test is clang/test/subpath/testname.cpp, you can use:

LIT_FILTER=testname.cpp ninja check-clang

The same trick can be applied for running tests in other sub-projects by specifying different a different check-<component>.

Useful CMake Flags

There are many other CMake flags that are useful for building, but these are some that may be useful for toolchain building.

Increase the number of link jobs that can be run in parallel (locally). The number of link jobs is dependent on RAM size. For LTO build you will need at least 10GB for each job.

Additional Resources

Documentation:

Talks: