Using natural and wire domain objects

Prerequisites

This tutorial builds on the Compiling FIDL tutorial. For more information on other FIDL tutorials, see the overview.

Overview

This tutorial details how to use the natural and wire domain objects by creating a unit test exercising those data types.

This document covers how to complete the following tasks:

Using the domain objects example code

The example code accompanying this tutorial is located in your Fuchsia checkout at //examples/fidl/cpp/domain_objects. It consists of a unit test component and its containing package. For more information about building unit test components, see Build components.

You may build and run the example on a running instance of Fuchsia emulator via the following:

# Add the domain objects unit test to the build.
# This only needs to be done once.
fx set core.x64 --with //examples/fidl/cpp/domain_objects
# Run the domain objects unit test.
fx test -vo fidl-examples-domain-objects-cpp-test

Add the C++ bindings of a FIDL library as a build dependency

GN build

For each FIDL library declaration, such as the one in Compiling FIDL, the C++ bindings code for that library is generated under the original target name suffixed with _cpp:

"//examples/fidl/fuchsia.examples:fuchsia.examples_cpp",

The test target looks like:

test("test") {
  testonly = true
  output_name = "fidl_examples_domain_objects_cpp_test"
  sources = [
    "advanced.cc",
    "main.cc",
  ]
  deps = [
    "//examples/fidl/fuchsia.examples:fuchsia.examples_cpp",

    "//src/lib/fxl/test:gtest_main",
  ]
}

Note the line which adds the dependency on the C++ bindings by referencing that _cpp target.

(Optional) To view the generated bindings:

  1. Build using fx build.
  2. Change to the generated files directory: out/default/fidling/gen/examples/fidl/fuchsia.examples/fuchsia.examples/cpp/fidl/fuchsia.examples/cpp, where the generated files are located. You may need to change out/default if you have set a different build output directory. You can check your build output directory with cat .fx-build-dir.

For more information on how to find generated bindings code, see Viewing generated bindings code.

Bazel build

When depending on the FIDL library from the Bazel build, an extra build rule is required if the FIDL library is not from the SDK:

# Given a FIDL library declaration like the following
fuchsia_fidl_library(
    name = "fuchsia.examples",
    srcs = [
        "echo.test.fidl",
        "types.test.fidl",
    ],
    library = "fuchsia.examples",
    visibility = ["//visibility:public"],
)

# This rule describes the generated C++ bindings code for that library
fuchsia_fidl_llcpp_library(
    name = "fuchsia.examples_llcpp_cc",
    library = ":fuchsia.examples",
    visibility = ["//visibility:public"],
    deps = ["@fuchsia_sdk//pkg/fidl_cpp_v2"],
)

If the FIDL library is from the Bazel SDK, the above step is not needed.

The C++ bindings code for a FIDL library is generated under the original target name suffixed with _llcpp_cc:

deps = [
  # Example when depending on an SDK library, `fuchsia.io`.
  "@fuchsia_sdk//fidl/fuchsia.io:fuchsia.io_llcpp_cc",

  # Example when depending on a local FIDL library, `fuchsia.examples`
  # defined above.
  # Suppose the library lives in the `//path/to/fidl/library` folder.
  "//path/to/fidl/library:fuchsia.examples_llcpp_cc",

  # ... other dependencies ...
]

Include the bindings header into your code

After adding the build dependency, you may include the bindings header. The include pattern is #include <fidl/my.library.name/cpp/fidl.h>.

The following include statement at the top of domain_objects/main.cc includes the bindings and makes the generated APIs available to the source code:

#include <fidl/fuchsia.examples/cpp/fidl.h>

Using natural domain objects

Natural types are the ergonomics and safety focused flavor of C++ domain objects. A tree of FIDL values is represented as a tree of C++ objects with hierarchical ownership. That means if a function receives some object of natural type, it can assume unique ownership of all child objects in the entire tree. The tree is torn down when the root object goes out of scope.

At a high level the natural types embrace std:: containers and concepts. For example, a table is represented as a collection of std::optional<Field>s. A vector is std::vector<T>, etc. They also implement idiomatic C++ moves, copies, and equality. For example, a resource type is move-only, while a value type will implement both copy and moves, where moves are designed to optimize the transfer of objects. Moving a table doesn't make it empty (it just recursively moves the fields), similar to std::optional.

Natural bits

Using the strict fuchsia.examples/FileMode FIDL type and the flexible fuchsia.examples/FlexibleFileMode FIDL type as examples:

// Bits implement bitwise operators such as |, ~, &, ^.
auto flags = ~fuchsia_examples::FileMode::kRead & fuchsia_examples::FileMode::kExecute;
flags = fuchsia_examples::FileMode::kRead | fuchsia_examples::FileMode::kWrite;

// Bits implement the set difference operation (clearing bits) under -.
ASSERT_EQ(flags - fuchsia_examples::FileMode::kRead, fuchsia_examples::FileMode::kWrite);
flags -= fuchsia_examples::FileMode::kRead;
ASSERT_EQ(flags, fuchsia_examples::FileMode::kWrite);

// Bits may be explicitly casted to their underlying integer type.
flags = fuchsia_examples::FileMode::kRead | fuchsia_examples::FileMode::kWrite;
ASSERT_EQ(static_cast<uint16_t>(flags), 0b11);

// They may also be explicitly constructed from an underlying type, but
// this may result in invalid values for strict bits.
flags = fuchsia_examples::FileMode(0b11);

// A safer alternative is |TryFrom|, which constructs an instance of
// |FileMode| only if underlying primitive does not contain any unknown
// members that is not defined in the FIDL schema. Otherwise, returns
// |std::nullopt|.
std::optional<fuchsia_examples::FileMode> maybe_flags =
    fuchsia_examples::FileMode::TryFrom(0b1111);
ASSERT_FALSE(maybe_flags.has_value());

// Another alternative is |TruncatingUnknown| which clears any bits not
// defined in the FIDL schema.
fuchsia_examples::FileMode truncated_flags =
    fuchsia_examples::FileMode::TruncatingUnknown(0b1111);
ASSERT_EQ(truncated_flags, fuchsia_examples::FileMode(0b111));

// Bits implement bitwise-assignment.
flags |= fuchsia_examples::FileMode::kExecute;

// They also support equality and expose a |kMask| that is the
// bitwise OR of all defined bit members.
ASSERT_EQ(flags, fuchsia_examples::FileMode::kMask);

// A flexible bits type additionally supports querying the unknown bits.
fuchsia_examples::FlexibleFileMode flexible_flags = fuchsia_examples::FlexibleFileMode(0b1111);
ASSERT_TRUE(flexible_flags.has_unknown_bits());
ASSERT_EQ(static_cast<uint16_t>(flexible_flags.unknown_bits()), 0b1000);

Natural enums

Using the strict fuchsia.examples/LocationType FIDL type and the flexible fuchsia.examples/FlexibleLocationType FIDL type as examples:

// Enums members are scoped constants under the enum type.
fuchsia_examples::LocationType location = fuchsia_examples::LocationType::kAirport;

// They may be explicitly casted to their underlying type.
ASSERT_EQ(static_cast<uint32_t>(fuchsia_examples::LocationType::kMuseum), 1u);

// They may also be casted to their underlying type without specifying the precise type.
uint32_t strict_underlying = fidl::ToUnderlying(fuchsia_examples::LocationType::kMuseum);
ASSERT_EQ(strict_underlying, 1u);

// Enums support switch case statements.
// A strict enum can be switched exhaustively.
(void)[=] {
  switch (location) {
    case fuchsia_examples::LocationType::kAirport:
      return 1;
    case fuchsia_examples::LocationType::kMuseum:
      return 2;
    case fuchsia_examples::LocationType::kRestaurant:
      return 3;
  }
};

// A flexible enum requires a `default:` case.
fuchsia_examples::FlexibleLocationType flexible_location =
    fuchsia_examples::FlexibleLocationType::kAirport;
(void)[=] {
  switch (flexible_location) {
    case fuchsia_examples::FlexibleLocationType::kAirport:
      return 1;
    case fuchsia_examples::FlexibleLocationType::kMuseum:
      return 2;
    case fuchsia_examples::FlexibleLocationType::kRestaurant:
      return 3;
    default:  // Removing this branch will fail to compile.
      return 4;
  }
};

// A flexible enum also supports asking if the current enum value was
// not known in the FIDL schema, or marked with `@unknown`.
ASSERT_FALSE(flexible_location.IsUnknown());

// Strict enums may be uninitialized. Their value will be undefined.
fuchsia_examples::LocationType strict_location;
(void)strict_location;

// Flexible enums may be default initialized. They will either contain
// the member marked with `@unknown` in the FIDL schema if exists,
// or a compiler-reserved unknown value otherwise.
fuchsia_examples::FlexibleLocationType default_flexible_location;
ASSERT_TRUE(default_flexible_location.IsUnknown());

Natural structs

Natural structs are straightforward record objects that expose const and mutable accessors. Using the fuchsia.examples/Color FIDL type as an example:

// Structs may be default constructed with fields set to default values,
// provided that all fields are also default constructible.
fuchsia_examples::Color default_color;
ASSERT_EQ(default_color.id(), 0u);
ASSERT_EQ(default_color.name(), "red");

// They support constructing by supplying fields in a sequence.
fuchsia_examples::Color blue = {1, "blue"};
ASSERT_EQ(blue.id(), 1u);

// They also support a more readable syntax that names individual fields,
// similar to C++ designated initialization. The double brace (`{{`) syntax
// is necessary to workaround C++ limitations on aggregate initialization.
fuchsia_examples::Color red{{.id = 2, .name = "red"}};
ASSERT_EQ(red.id(), 2u);
fuchsia_examples::Color designated_1 = {{.id = 1, .name = "designated"}};
ASSERT_EQ(designated_1.id(), 1u);
fuchsia_examples::Color designated_2{{.id = 2, .name = "designated"}};
ASSERT_EQ(designated_2.id(), 2u);

// Setters take the value to be set as argument.
fuchsia_examples::Color color;
color.id(100);
color.name("green");
ASSERT_EQ(color.id(), 100u);
ASSERT_EQ(color.name(), "green");

// Setters may also be chained.
color.id(42).name("yellow");
ASSERT_EQ(color.id(), 42u);
ASSERT_EQ(color.name(), "yellow");

// Equality is implemented for value types.
ASSERT_EQ(color, fuchsia_examples::Color(42, "yellow"));

// Copies and moves.
fuchsia_examples::Color color_copy{color};
ASSERT_EQ(color_copy.name(), "yellow");
fuchsia_examples::Color color_moved{std::move(color)};
ASSERT_EQ(color_moved.name(), "yellow");
// The state of |color| is now unspecified.

Natural unions

Natural unions are sum types similar to std::variant. Using the strict fuchsia.examples/JsonValue FIDL type and the flexible fuchsia.examples/FlexibleJsonValue FIDL type as examples:

// Factory functions are used to construct natural union objects.
// To construct a union whose active member is |int_value|, use |WithIntValue|.
auto int_val = fuchsia_examples::JsonValue::WithIntValue(1);

// |Which| obtains an enum corresponding to the active member, which may be
// used in switch cases.
ASSERT_EQ(int_val.Which(), fuchsia_examples::JsonValue::Tag::kIntValue);

// When directly accessing a field, one must first check if the field is
// active before dereferencing it.
ASSERT_TRUE(int_val.int_value().has_value());
ASSERT_TRUE(static_cast<bool>(int_val.int_value()));
ASSERT_EQ(int_val.int_value().value(), 1);

// Another example, this time activating the |string_value| member.
auto str_val = fuchsia_examples::JsonValue::WithStringValue("1");
ASSERT_EQ(str_val.Which(), fuchsia_examples::JsonValue::Tag::kStringValue);
ASSERT_TRUE(str_val.string_value().has_value());

// Unions are not default constructible, to avoid invalid states.
static_assert(!std::is_default_constructible_v<fuchsia_examples::JsonValue>,
              "Unions cannot be default constructed");

fuchsia_examples::JsonValue value = fuchsia_examples::JsonValue::WithStringValue("hello");
ASSERT_FALSE(value.int_value());
ASSERT_TRUE(value.string_value());

// |value_or| returns a fallback if the corresponding member is not active.
ASSERT_EQ(value.int_value().value_or(42), 42);

// Setters take the value to be set as argument.
// Setting a field causes that field to become the active member.
value.int_value(2);
ASSERT_TRUE(value.int_value());
ASSERT_FALSE(value.string_value());

// |take| invokes the move operation on the member if it is active.
value.string_value("foo");
std::optional<std::string> str = value.string_value().take();
ASSERT_TRUE(str.has_value());
ASSERT_EQ(str.value(), "foo");

// Equality is implemented for value types.
value.string_value("bar");
ASSERT_EQ(value, fuchsia_examples::JsonValue::WithStringValue("bar"));

// Copies and moves.
fuchsia_examples::JsonValue value_copy{value};
ASSERT_EQ(value.string_value().value(), "bar");
fuchsia_examples::JsonValue value_moved{std::move(value)};
ASSERT_EQ(value_moved.string_value().value(), "bar");

// When switching over the tag from a flexible union, one must add a `default:`
// case, to handle members not understood by the FIDL schema or to handle
// newly added members in a source compatible way.
fuchsia_examples::FlexibleJsonValue flexible_value =
    fuchsia_examples::FlexibleJsonValue::WithIntValue(1);
switch (flexible_value.Which()) {
  case fuchsia_examples::FlexibleJsonValue::Tag::kIntValue:
    ASSERT_EQ(flexible_value.int_value().value(), 1);
    break;
  case fuchsia_examples::FlexibleJsonValue::Tag::kStringValue:
    FAIL() << "Unexpected tag. |flexible_value| was set to int";
    break;
  default:  // Removing this branch will fail to compile.
    break;
}

Natural tables

Natural tables are record types where every field is optional. Using the fuchsia.examples/User FIDL type as an example:

// A default constructed table is empty. That is, every field is absent.
fuchsia_examples::User user;
ASSERT_TRUE(user.IsEmpty());

// Each accessor returns a |std::optional<T>|, where |T| is the field type.
ASSERT_FALSE(user.age().has_value());

// Setters take the value to be set as argument.
user.age(100);
user.age(*user.age() + 100);
ASSERT_EQ(user.age().value(), 200);

// Setters may also be chained.
user.name("foo").age(30);
ASSERT_EQ(user.name().value(), "foo");
ASSERT_EQ(user.age().value(), 30);

// Since each field is an |std::optional<T>|, they may also be cleared.
user.name().reset();
ASSERT_FALSE(user.name().has_value());

// Assigning an |std::nullopt| also clears the field.
user.name("bar");
ASSERT_TRUE(user.name().has_value());
user.name() = std::nullopt;
ASSERT_FALSE(user.name().has_value());

// |value_or| returns a fallback if the corresponding field is absent.
ASSERT_EQ(user.name().value_or("anonymous"), "anonymous");
user.age().reset();
ASSERT_TRUE(user.IsEmpty());

// Similar to structs, tables support constructing by naming individual fields.
// Fields that are omitted from the designated initialization syntax will be
// absent from the table.
user = {{.age = 100, .name = "foo"}};
ASSERT_TRUE(user.age());
ASSERT_TRUE(user.name());

user = {{.age = 100}};
ASSERT_TRUE(user.age());
ASSERT_FALSE(user.name());

// Equality is implemented for value types.
ASSERT_EQ(user, fuchsia_examples::User{{.age = 100}});

// Copies and moves.
fuchsia_examples::User user_copy{user};
ASSERT_EQ(*user.age(), 100);
fuchsia_examples::User user_moved{std::move(user)};
ASSERT_EQ(*user_moved.age(), 100);

Using wire domain objects

Wire types are the performance oriented flavor of C++ domain objects. Differing from natural types which maintain hierarchical object ownership, wire objects never own their out-of-line children. Whether a child object is stored inline or out-of-line is determined by the FIDL wire format.

Natural types may implicitly heap allocate the necessary storage. Conversely, the user has complete control over memory allocation of wire types. For example, you may allocate the elements of a FIDL vector on the stack, from a memory pool, or as part of a larger object. The wire vector type, fidl::VectorView<T>, is an unowned view type consisting of a raw pointer and a length. One may send the vector as part of a FIDL request without extra heap allocations by borrowing the elements via this type.

To distinguish from the natural types, wire types from a FIDL library are defined in the ...::wire nested namespace, e.g. fuchsia_my_library::wire.

The prevalence of unowned pointers in wire types makes them flexible but very unsafe. This tutorial will focus on the safer side of using wire types based on memory arenas. For more advanced usages involving unsafe memory borrows, refer to Memory ownership of wire domain objects.

Wire bits and enums

Because bits and enums have a very simple memory layout and do not have any out-of-line children, the wire types for FIDL bits and enums are the same as their natural type counterparts. To stay coherent with the overall namespace naming profiles, bits and enums are aliased into the fuchsia_my_library::wire nested namespace, appearing alongside wire structs, unions, and tables.

Using the fuchsia.examples/FileMode FIDL bits as an example, fuchsia_examples::wire::FileMode is a type alias of fuchsia_examples::FileMode.

static_assert(std::is_same<fuchsia_examples::FileMode, fuchsia_examples::wire::FileMode>::value,
              "natural bits should be equivalent to wire bits");
static_assert(fuchsia_examples::FileMode::kMask == fuchsia_examples::wire::FileMode::kMask,
              "natural bits should be equivalent to wire bits");

using fuchsia_examples::wire::FileMode;
auto flags = FileMode::kRead | FileMode::kWrite | FileMode::kExecute;
ASSERT_EQ(flags, FileMode::kMask);

Similarly, using the fuchsia.examples/LocationType FIDL enum as an example, fuchsia_examples::wire::LocationType is a type alias of fuchsia_examples::LocationType.

static_assert(
    std::is_same<fuchsia_examples::LocationType, fuchsia_examples::wire::LocationType>::value,
    "natural enums should be equivalent to wire enums");

ASSERT_EQ(static_cast<uint32_t>(fuchsia_examples::wire::LocationType::kMuseum), 1u);

Wire structs

Wire structs are simple C++ structs that hold public member variables. Using the fuchsia.examples/Color FIDL type as an example:

// Wire structs are simple C++ structs with all their member fields declared
// public. One may invoke aggregate initialization:
fuchsia_examples::wire::Color blue = {1, "blue"};
ASSERT_EQ(blue.id, 1u);
ASSERT_EQ(blue.name.get(), "blue");

// ..or designated initialization.
fuchsia_examples::wire::Color blue_designated = {.id = 1, .name = "blue"};
ASSERT_EQ(blue_designated.id, 1u);
ASSERT_EQ(blue_designated.name.get(), "blue");

// A wire struct may be default constructed, but user-defined default values
// are not supported.
// Default-initializing a struct means all fields are zero-initialized.
fuchsia_examples::wire::Color default_color;
ASSERT_EQ(default_color.id, 0u);
ASSERT_TRUE(default_color.name.is_null());
ASSERT_TRUE(default_color.name.empty());

// There are no getters/setters. One simply reads or mutates the member field.
blue.id = 2;
ASSERT_EQ(blue.id, 2u);

// Here we demonstrate that wire structs do not own their out-of-line children.
// Copying a struct will not copy their out-of-line children. Pointers are
// simply aliased.
{
  fuchsia_examples::wire::Color blue2 = blue;
  ASSERT_EQ(blue2.name.data(), blue.name.data());
}
// Similarly, destroying a wire struct object does not destroy out-of-line
// children. Destroying |blue2| does not invalidate the string contents in |name|.
ASSERT_EQ(blue.name.get(), "blue");

Wire unions

Wire unions are sum types with a memory layout akin to a discriminator tag followed by a reference to the active member. Using the strict fuchsia.examples/JsonValue FIDL type and the flexible fuchsia.examples/FlexibleJsonValue FIDL type as examples:

// When the active member is larger than 4 bytes, it is stored out-of-line,
// and the union will borrow the out-of-line content. The lifetimes can be
// tricky to reason about, hence the FIDL runtime provides a |fidl::AnyArena|
// interface for arena-based allocation of members. The built-in
// implementation is |fidl::Arena|.
//
// Pass the arena as the first argument to |With...| factory functions, to
// construct the member content on the arena, and have the union reference it.
fidl::Arena arena;
fuchsia_examples::wire::JsonValue str_union =
    fuchsia_examples::wire::JsonValue::WithStringValue(arena, "1");

// |Which| obtains an enum corresponding to the active member, which may be
// used in switch cases.
ASSERT_EQ(str_union.Which(), fuchsia_examples::wire::JsonValue::Tag::kStringValue);

// Before accessing the |string_value| member, one should check if the union
// indeed currently holds this member, by querying |is_string_value|.
// Accessing the wrong member will cause a panic.
ASSERT_TRUE(str_union.is_string_value());
ASSERT_EQ("1", str_union.string_value().get());

// When the active member is smaller or equal to 4 bytes, such as an
// |int32_t| here, the entire member is inlined into the union object.
// In these cases, arena allocation is not necessary, and the union
// object wholly owns the member.
fuchsia_examples::wire::JsonValue int_union = fuchsia_examples::wire::JsonValue::WithIntValue(1);
ASSERT_TRUE(int_union.is_int_value());
ASSERT_EQ(1, int_union.int_value());

// A default constructed wire union is invalid.
// It must be initialized with a valid member before use.
// One is not allowed to send invalid unions through FIDL client/server APIs.
fuchsia_examples::wire::JsonValue default_union;
ASSERT_TRUE(default_union.has_invalid_tag());
default_union = fuchsia_examples::wire::JsonValue::WithStringValue(arena, "hello");
ASSERT_FALSE(default_union.has_invalid_tag());
ASSERT_TRUE(default_union.is_string_value());
ASSERT_EQ(default_union.string_value().get(), "hello");

// Optional unions are represented with |fidl::WireOptional|.
fidl::WireOptional<fuchsia_examples::wire::JsonValue> optional_json;
ASSERT_FALSE(optional_json.has_value());
optional_json = fuchsia_examples::wire::JsonValue::WithIntValue(42);
ASSERT_TRUE(optional_json.has_value());
// |fidl::WireOptional| has a |std::optional|-like API.
fuchsia_examples::wire::JsonValue& value = optional_json.value();
ASSERT_TRUE(value.is_int_value());

// When switching over the tag from a flexible union, one must add a `default:`
// case, to handle members not understood by the FIDL schema or to handle
// newly added members in a source compatible way.
fuchsia_examples::wire::FlexibleJsonValue flexible_value =
    fuchsia_examples::wire::FlexibleJsonValue::WithIntValue(1);
switch (flexible_value.Which()) {
  case fuchsia_examples::wire::FlexibleJsonValue::Tag::kIntValue:
    ASSERT_EQ(flexible_value.int_value(), 1);
    break;
  case fuchsia_examples::wire::FlexibleJsonValue::Tag::kStringValue:
    FAIL() << "Unexpected tag. |flexible_value| was set to int";
    break;
  default:  // Removing this branch will fail to compile.
    break;
}

Wire tables

Wire tables are record types where every field is optional. Differing from natural tables, wire tables do not own any member field. Copying a wire table is akin to aliasing (copying) a pointer. Similar to pointers, moving a wire table is an anti-pattern because that equates to a copy.

Because of the memory layout constraints of wire tables, one always use an associated Builder type to create new instances. Once a table is built, one may not add new members or clear existing members.

Using the fuchsia.examples/User FIDL type as an example:

fidl::Arena arena;
// To construct a wire table, you need to first create a corresponding
// |Builder| object, which borrows an arena. The |arena| will be used to
// allocate the table frame, a bookkeeping structure for field presence.
auto builder = fuchsia_examples::wire::User::Builder(arena);

// To set a table field, call the member function with the same name on the
// builder. The arguments will be forwarded to the field constructor, and the
// field is allocated on the initial |arena|.
builder.age(10);

// Note that only the inline portion of the field is automatically placed in
// the arena. The field itself may reference its own out-of-line content,
// such as in the case of |name| whose type is |fidl::StringView|. |name|
// will reference the "jdoe" literal, which lives in static program storage.
builder.name("jdoe");

// Call |Build| to finalize the table builder into a |User| table.
// The builder is no longer needed after this point. |user| will continue to
// reference objects allocated in the |arena|.
fuchsia_examples::wire::User user = builder.Build();
ASSERT_FALSE(user.IsEmpty());

// Before accessing a field, one should check if it is present, by querying
// |has_...|. Accessing an absent field will panic.
ASSERT_TRUE(user.has_name());
ASSERT_EQ(user.name().get(), "jdoe");

// Setters may be chained, leading to a fluent syntax.
user = fuchsia_examples::wire::User::Builder(arena).age(30).name("bob").Build();
ASSERT_FALSE(user.IsEmpty());
ASSERT_TRUE(user.has_age());
ASSERT_EQ(user.age(), 30);
ASSERT_TRUE(user.has_name());
ASSERT_EQ(user.name().get(), "bob");

// A default constructed wire table is empty.
// This is mostly useful to make requests or replies with empty tables.
fuchsia_examples::wire::User defaulted_user;
ASSERT_TRUE(defaulted_user.IsEmpty());

// In some situations it could be difficult to provide an arena when
// constructing tables. For example, here it is hard to provide constructor
// arguments to 10 tables at once. Because a default constructed wire table is
// empty, a new table instance should be built and assigned in its place.
fidl::Array<fuchsia_examples::wire::User, 10> users;
for (auto& user : users) {
  ASSERT_TRUE(user.IsEmpty());
  user = fuchsia_examples::wire::User::Builder(arena).age(30).Build();
  ASSERT_FALSE(user.IsEmpty());
  ASSERT_EQ(user.age(), 30);
}
ASSERT_EQ(users[0].age(), 30);

// Finally, tables support checking if it was received with unknown fields.
// A table created by ourselves will never have unknown fields.
ASSERT_FALSE(user.HasUnknownData());

For more information on the bindings, see the bindings reference.

Convert between natural and wire domain objects

To streamline interoperability, you may call fidl::ToWire and fidl::ToNatural functions to convert between wire and natural domain objects. Using the fuchsia.examples/User FIDL type as an example:

Convert from natural to wire: fidl::ToWire

// Let's start with a natural table.
fuchsia_examples::User user{{.age = 100, .name = "foo"}};

// To convert it to its corresponding wire domain object, we need a
// |fidl::AnyArena| implementation to allocate the storage, here an |arena|.
fidl::Arena arena;

// Call |fidl::ToWire| with the arena and the natural domain object.
// All out-of-line fields will live on the |arena|.
fuchsia_examples::wire::User wire_user = fidl::ToWire(arena, user);
ASSERT_TRUE(wire_user.has_age());
ASSERT_EQ(wire_user.age(), 100);
ASSERT_TRUE(wire_user.has_name());
ASSERT_EQ(wire_user.name().get(), "foo");

Convert from wire to natural: fidl::ToNatural

fidl::Arena arena;

// Let's start with a wire table.
fuchsia_examples::wire::User wire_user =
    fuchsia_examples::wire::User::Builder(arena).age(30).name("bob").Build();

// Call |fidl::ToNatural| with the wire domain object.
// All child fields will be owned by |user|.
fuchsia_examples::User user = fidl::ToNatural(wire_user);
ASSERT_TRUE(user.age().has_value());
ASSERT_EQ(user.age().value(), 30);
ASSERT_TRUE(user.name().has_value());
ASSERT_EQ(user.name().value(), "bob");

Persist natural and wire domain objects

You may use fidl::Persist to serialize a natural or wire domain object into a byte vector, the primary use case being long term data persistence.

fidl::Unpersist deserializes and copies a sequence of bytes into some instance of natural domain object.

fidl::InplaceUnpersist deserializes a sequence of bytes into some instance of wire domain object, mutating the bytes in the process.

FIDL recipe: Persistence

Persistent FIDL refers to wire-encoded binary FIDL data that is stored with no underlying transport. Instead, the data is stored for some arbitrarily long period of time using a persistent, byte-oriented interface like a file or database entry.

A simple way to extend the key-value store to support exporting backups would be to simply add a new method that stops the world, serializes the state of the store, and sends it back as a FIDL vector<Item>. There are two downsides to this approach, however. The first is that it puts all of the burden of the backup on the server - a client pays nothing to ask for a backup operation that is very expensive to the server. The second is that it involves a great deal of copying: the client is almost certainly just going to write the resulting backup to some backing datastore, like a file or a database, as soon as it receives it. Having it decode this (potentially very large) FIDL object, just so that it can immediately re-encode it as it forwards it to whatever protocol will do the actual storage, is very wasteful.

Reasoning

A better solution is to use zircon's virtual memory objects. Instead of constantly copying bytes back and forth in a bucket brigade, we can mint a VMO to hold the backup data on the client, send it to the server, then forward it back to our target data store without deserializing in between. As long as the target data store's protocol has allowances for accepting data transported using a VMO, this is the preferred way to accomplish expensive operations like this. In fact, Fuchsia's file system, for instance, implements this exact pattern. A benefit of this approach is that it forces the client to do some work when asking the server for an expensive operation, minimizing the work imbalance between the two parties.

FIDL value types can be persisted to any byte-oriented storage medium, using the FIDL data persistence binary format. We will persist the newly introduced FIDL type Exportable into the VMO. The object will be encoded and written to the storage (in this case, a VMO that could later be saved as a file), and decoded from it when the data needs to be accessed again, in much the same way that a message is encoded, transported, and decoded again later when using FIDL over IPC.

To do this securely and adhere to the principle of least privilege, we should constrain the privileges the handle representing our VMO may carry. Enter handle rights, FIDL's first-class method of describing the privileges available to a particular handle type. In this case, we allow the empty VMO passed to the server in the Export request to be read from, queried for size, resized, and written to. When the VMO is returned, we remove right to resize and write, ensuring that no process, not even malicious actors in some far away component, can modify this data as it moves through the system.

Implementation

The FIDL, CML, and realm interface definitions are as follows:

FIDL

// Copyright 2022 The Fuchsia Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
library examples.keyvaluestore.supportexports;

using zx;

/// An item in the store. The key must match the regex `^[A-z][A-z0-9_\.\/]{2,62}[A-z0-9]$`. That
/// is, it must start with a letter, end with a letter or number, contain only letters, numbers,
/// periods, and slashes, and be between 4 and 64 characters long.
type Item = struct {
    key string:128;
    value vector<byte>:64000;
};

/// An enumeration of things that may go wrong when trying to write a value to our store.
type WriteError = flexible enum {
    UNKNOWN = 0;
    INVALID_KEY = 1;
    INVALID_VALUE = 2;
    ALREADY_EXISTS = 3;
};

/// An enumeration of things that may go wrong when trying to mint an export.
type ExportError = flexible enum {
    UNKNOWN = 0;
    EMPTY = 1;
    STORAGE_TOO_SMALL = 2;
};

// A data type describing the structure of a single export. We never actually send this data type
// over the wire (we use the file's VMO instead), but whenever data needs to be written to/read from
// its backing storage as persistent FIDL, it will have this schema.
///
/// The items should be sorted in ascending order, following lexicographic ordering of their keys.
type Exportable = table {
    1: items vector<Item>;
};

/// A very basic key-value store - so basic, in fact, that one may only write to it, never read!
@discoverable
open protocol Store {
    /// Writes an item to the store.
    flexible WriteItem(struct {
        attempt Item;
    }) -> () error WriteError;

    /// Exports the entire store as a persistent [`Exportable`] FIDL object into a VMO provided by
    /// the client.
    ///
    /// By having the client provide (and speculatively size) the VMO, we force the party requesting
    /// the relatively heavy load of generating a backup to acknowledge and bear some of the costs.
    ///
    /// This method operates by having the client supply an empty VMO, which the server then
    /// attempts to fill. Notice that the server removes the `zx.Rights.WRITE` and
    /// `zx.Rights.SET_PROPERTY` rights from the returned VMO - not even the requesting client may
    /// alter the backup once it has been minted by the server.
    flexible Export(resource struct {
        /// Note that the empty VMO has more rights than the filled one being returned: it has
        /// `zx.Rights.WRITE` (via `zx.RIGHTS_IO`) so that the VMO may be filled with exported data,
        /// and `zx.Rights.SET_PROPERTY` (via `zx.RIGHTS_PROPERTY`) so that it may be resized to
        /// truncate any remaining empty buffer.
        empty zx.Handle:<VMO, zx.RIGHTS_BASIC | zx.RIGHTS_PROPERTY | zx.RIGHTS_IO>;
    }) -> (resource struct {
        /// The `zx.Rights.WRITE` and `zx.Rights.SET_PROPERTY` rights have been removed from the now
        /// filled VMO. No one, not even the client that requested the export, is able to modify
        /// this VMO going forward.
        filled zx.Handle:<VMO, zx.RIGHTS_BASIC | zx.Rights.GET_PROPERTY | zx.Rights.READ>;
    }) error ExportError;
};

CML

Client

// Copyright 2022 The Fuchsia Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
{
    include: [ "syslog/client.shard.cml" ],
    program: {
        runner: "elf",
        binary: "bin/client_bin",
    },
    use: [
        { protocol: "examples.keyvaluestore.supportexports.Store" },
    ],
    config: {
        write_items: {
            type: "vector",
            max_count: 16,
            element: {
                type: "string",
                max_size: 64,
            },
        },

        // The size, in bytes, allotted to the export VMO
        max_export_size: { type: "uint64" },

    },
}

Server

// Copyright 2022 The Fuchsia Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
{
    include: [ "syslog/client.shard.cml" ],
    program: {
        runner: "elf",
        binary: "bin/server_bin",
    },
    capabilities: [
        { protocol: "examples.keyvaluestore.supportexports.Store" },
    ],
    expose: [
        {
            protocol: "examples.keyvaluestore.supportexports.Store",
            from: "self",
        },
    ],
}

Realm

// Copyright 2022 The Fuchsia Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
{
    children: [
        {
            name: "client",
            url: "#meta/client.cm",
        },
        {
            name: "server",
            url: "#meta/server.cm",
        },
    ],
    offer: [
        // Route the protocol under test from the server to the client.
        {
            protocol: "examples.keyvaluestore.supportexports.Store",
            from: "#server",
            to: "#client",
        },

        // Route diagnostics support to all children.
        {
            protocol: [
                "fuchsia.inspect.InspectSink",
                "fuchsia.logger.LogSink",
            ],
            from: "parent",
            to: [
                "#client",
                "#server",
            ],
        },
    ],
}

Client and server implementations can then be written in any supported language:

Rust

Client

// Copyright 2022 The Fuchsia Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

use {
    anyhow::{Context as _, Error},
    config::Config,
    fuchsia_component::client::connect_to_protocol,
    std::{thread, time},
};

use {
    fidl::unpersist,
    fidl_examples_keyvaluestore_supportexports::{Exportable, Item, StoreMarker},
    fuchsia_zircon::Vmo,
};

#[fuchsia::main]
async fn main() -> Result<(), Error> {
    println!("Started");

    // Load the structured config values passed to this component at startup.
    let config = Config::take_from_startup_handle();

    // Use the Component Framework runtime to connect to the newly spun up server component. We wrap
    // our retained client end in a proxy object that lets us asynchronously send `Store` requests
    // across the channel.
    let store = connect_to_protocol::<StoreMarker>()?;
    println!("Outgoing connection enabled");

    // This client's structured config has one parameter, a vector of strings. Each string is the
    // path to a resource file whose filename is a key and whose contents are a value. We iterate
    // over them and try to write each key-value pair to the remote store.
    for key in config.write_items.into_iter() {
        let path = format!("/pkg/data/{}.txt", key);
        let value = std::fs::read_to_string(path.clone())
            .with_context(|| format!("Failed to load {path}"))?;
        match store.write_item(&Item { key: key, value: value.into_bytes() }).await? {
            Ok(_) => println!("WriteItem Success"),
            Err(err) => println!("WriteItem Error: {}", err.into_primitive()),
        }
    }

    // If the `max_export_size` is 0, no export is possible, so just ignore this block. This check
    // isn't strictly necessary, but does avoid extra work down the line.
    if config.max_export_size > 0 {
        // Create a 100Kb VMO to store the resulting export. In a real implementation, we would
        // likely receive the VMO representing the to-be-written file from file system like vfs of
        // fxfs.
        let vmo = Vmo::create(config.max_export_size)?;

        // Send the VMO to the server, to be populated with the current state of the key-value
        // store.
        match store.export(vmo).await? {
            Err(err) => {
                println!("Export Error: {}", err.into_primitive());
            }
            Ok(output) => {
                println!("Export Success");

                // Read the exported data (encoded in byte form as persistent FIDL) from the
                // returned VMO. In a real implementation, instead of reading the VMO, we would
                // merely forward it to some other storage-handling process. Doing this using a VMO,
                // rather than FIDL IPC, would save us frivolous reads and writes at each hop.
                let content_size = output.get_content_size().unwrap();
                let mut encoded_bytes = vec![0; content_size as usize];
                output.read(&mut encoded_bytes, 0)?;

                // Decode the persistent FIDL that was just read from the file.
                let exportable = unpersist::<Exportable>(&encoded_bytes).unwrap();
                let items = exportable.items.expect("must always be set");

                // Log some information about the exported data.
                println!("Printing {} exported entries, which are:", items.len());
                for item in items.iter() {
                    println!("  * {}", item.key);
                }
            }
        };
    }

    // TODO(https://fxbug.dev/42156498): We need to sleep here to make sure all logs get drained. Once the
    // referenced bug has been resolved, we can remove the sleep.
    thread::sleep(time::Duration::from_secs(2));
    Ok(())
}

Server

// Copyright 2022 The Fuchsia Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

use {
    anyhow::{Context as _, Error},
    fuchsia_component::server::ServiceFs,
    futures::prelude::*,
    lazy_static::lazy_static,
    regex::Regex,
    std::cell::RefCell,
    std::collections::hash_map::Entry,
    std::collections::HashMap,
};

use {
    fidl::{persist, Vmo},
    fidl_examples_keyvaluestore_supportexports::{
        ExportError, Exportable, Item, StoreRequest, StoreRequestStream, WriteError,
    },
};

lazy_static! {
    static ref KEY_VALIDATION_REGEX: Regex =
        Regex::new(r"^[A-Za-z]\w+[A-Za-z0-9]$").expect("Key validation regex failed to compile");
}

/// Handler for the `WriteItem` method.
fn write_item(store: &mut HashMap<String, Vec<u8>>, attempt: Item) -> Result<(), WriteError> {
    // Validate the key.
    if !KEY_VALIDATION_REGEX.is_match(attempt.key.as_str()) {
        println!("Write error: INVALID_KEY, For key: {}", attempt.key);
        return Err(WriteError::InvalidKey);
    }

    // Validate the value.
    if attempt.value.is_empty() {
        println!("Write error: INVALID_VALUE, For key: {}", attempt.key);
        return Err(WriteError::InvalidValue);
    }

    // Write to the store, validating that the key did not already exist.
    match store.entry(attempt.key) {
        Entry::Occupied(entry) => {
            println!("Write error: ALREADY_EXISTS, For key: {}", entry.key());
            Err(WriteError::AlreadyExists)
        }
        Entry::Vacant(entry) => {
            println!("Wrote value at key: {}", entry.key());
            entry.insert(attempt.value);
            Ok(())
        }
    }
}

/// Handler for the `Export` method.
fn export(store: &mut HashMap<String, Vec<u8>>, vmo: Vmo) -> Result<Vmo, ExportError> {
    // Empty stores cannot be exported.
    if store.is_empty() {
        return Err(ExportError::Empty);
    }

    // Build the `Exportable` vector locally. That means iterating over the map, and turning it into
    // a vector of items instead.
    let mut exportable = Exportable::default();
    let mut items = store
        .iter()
        .map(|entry| return Item { key: entry.0.clone(), value: entry.1.clone() })
        .collect::<Vec<Item>>();
    items.sort_by(|a, b| a.key.cmp(&b.key));
    exportable.items = Some(items);

    // Encode the bytes - there is a bug in persistent FIDL if this operation fails. Even if it
    // succeeds, make sure to check that the VMO has enough space to handle the encoded export data.
    let encoded_bytes = persist(&exportable).map_err(|_| ExportError::Unknown)?;
    if encoded_bytes.len() as u64 > vmo.get_content_size().map_err(|_| ExportError::Unknown)? {
        return Err(ExportError::StorageTooSmall);
    }

    // Write the (now encoded) persistent FIDL data to the VMO.
    vmo.set_content_size(&(encoded_bytes.len() as u64)).map_err(|_| ExportError::Unknown)?;
    vmo.write(&encoded_bytes, 0).map_err(|_| ExportError::Unknown)?;
    Ok(vmo)
}

/// Creates a new instance of the server. Each server has its own bespoke, per-connection instance
/// of the key-value store.
async fn run_server(stream: StoreRequestStream) -> Result<(), Error> {
    // Create a new in-memory key-value store. The store will live for the lifetime of the
    // connection between the server and this particular client.
    let store = RefCell::new(HashMap::<String, Vec<u8>>::new());

    // Serve all requests on the protocol sequentially - a new request is not handled until its
    // predecessor has been processed.
    stream
        .map(|result| result.context("failed request"))
        .try_for_each(|request| async {
            // Match based on the method being invoked.
            match request {
                StoreRequest::WriteItem { attempt, responder } => {
                    println!("WriteItem request received");

                    // The `responder` parameter is a special struct that manages the outgoing reply
                    // to this method call. Calling `send` on the responder exactly once will send
                    // the reply.
                    responder
                        .send(write_item(&mut store.borrow_mut(), attempt))
                        .context("error sending reply")?;
                    println!("WriteItem response sent");
                }
                StoreRequest::Export { empty, responder } => {
                    println!("Export request received");

                    responder
                        .send(export(&mut store.borrow_mut(), empty))
                        .context("error sending reply")?;
                    println!("Export response sent");
                } //
                StoreRequest::_UnknownMethod { ordinal, .. } => {
                    println!("Received an unknown method with ordinal {ordinal}");
                }
            }
            Ok(())
        })
        .await
}

// A helper enum that allows us to treat a `Store` service instance as a value.
enum IncomingService {
    Store(StoreRequestStream),
}

#[fuchsia::main]
async fn main() -> Result<(), Error> {
    println!("Started");

    // Add a discoverable instance of our `Store` protocol - this will allow the client to see the
    // server and connect to it.
    let mut fs = ServiceFs::new_local();
    fs.dir("svc").add_fidl_service(IncomingService::Store);
    fs.take_and_serve_directory_handle()?;
    println!("Listening for incoming connections");

    // The maximum number of concurrent clients that may be served by this process.
    const MAX_CONCURRENT: usize = 10;

    // Serve each connection simultaneously, up to the `MAX_CONCURRENT` limit.
    fs.for_each_concurrent(MAX_CONCURRENT, |IncomingService::Store(stream)| {
        run_server(stream).unwrap_or_else(|e| println!("{:?}", e))
    })
    .await;

    Ok(())
}

C++ (Natural)

Client

// Copyright 2022 The Fuchsia Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

#include <fidl/examples.keyvaluestore.supportexports/cpp/fidl.h>
#include <lib/async-loop/cpp/loop.h>
#include <lib/component/incoming/cpp/protocol.h>
#include <lib/syslog/cpp/macros.h>
#include <unistd.h>

#include <examples/fidl/new/key_value_store/support_exports/cpp_natural/client/config.h>
#include <src/lib/files/file.h>
#include <src/lib/fxl/strings/string_printf.h>

int main(int argc, const char** argv) {
  FX_LOGS(INFO) << "Started";

  // Retrieve component configuration.
  auto conf = config::Config::TakeFromStartupHandle();

  // Start up an async loop and dispatcher.
  async::Loop loop(&kAsyncLoopConfigNeverAttachToThread);
  async_dispatcher_t* dispatcher = loop.dispatcher();

  // Connect to the protocol inside the component's namespace. This can fail so it's wrapped in a
  // |zx::result| and it must be checked for errors.
  zx::result client_end = component::Connect<examples_keyvaluestore_supportexports::Store>();
  if (!client_end.is_ok()) {
    FX_LOGS(ERROR) << "Synchronous error when connecting to the |Store| protocol: "
                   << client_end.status_string();
    return -1;
  }

  // Create an asynchronous client using the newly-established connection.
  fidl::Client client(std::move(*client_end), dispatcher);
  FX_LOGS(INFO) << "Outgoing connection enabled";

  for (const auto& action : conf.write_items()) {
    std::string text;
    if (!files::ReadFileToString(fxl::StringPrintf("/pkg/data/%s.txt", action.c_str()), &text)) {
      FX_LOGS(ERROR) << "It looks like the correct `resource` dependency has not been packaged";
      break;
    }

    auto value = std::vector<uint8_t>(text.begin(), text.end());
    client->WriteItem(examples_keyvaluestore_supportexports::Item(action, value))
        .ThenExactlyOnce(
            [&](fidl::Result<examples_keyvaluestore_supportexports::Store::WriteItem> result) {
              // Check if the FIDL call succeeded or not.
              if (!result.is_ok()) {
                if (result.error_value().is_framework_error()) {
                  FX_LOGS(ERROR) << "Unexpected FIDL framework error: " << result.error_value();
                } else {
                  FX_LOGS(INFO) << "WriteItem Error: "
                                << fidl::ToUnderlying(result.error_value().domain_error());
                }
              } else {
                FX_LOGS(INFO) << "WriteItem Success";
              }

              // Quit the loop, thereby handing control back to the outer loop of actions being
              // iterated over.
              loop.Quit();
            });

    // Run the loop until the callback is resolved, at which point we can continue from here.
    loop.Run();
    loop.ResetQuit();
  }

  // If the `max_export_size` is 0, no export is possible, so just ignore this block. This check
  // isn't strictly necessary, but does avoid extra work down the line.
  if (conf.max_export_size() > 0) {
    // Create a 100Kb VMO to store the resulting export. In a real implementation, we would
    // likely receive the VMO representing the to-be-written file from file system like vfs of
    // fxfs.
    zx::vmo vmo;
    if (zx_status_t status = zx::vmo::create(conf.max_export_size(), 0, &vmo); status != ZX_OK) {
      FX_PLOGS(ERROR, status) << "Failed to create VMO";
      return -1;
    }

    client->Export({std::move(vmo)})
        .ThenExactlyOnce(
            [&](fidl::Result<examples_keyvaluestore_supportexports::Store::Export>& result) {
              // Quit the loop, thereby handing control back to the outer loop of actions being
              // iterated over, when we return from this callback.
              loop.Quit();

              if (!result.is_ok()) {
                if (result.error_value().is_framework_error()) {
                  FX_LOGS(ERROR) << "Unexpected FIDL framework error: " << result.error_value();
                } else {
                  FX_LOGS(INFO) << "Export Error: "
                                << fidl::ToUnderlying(result.error_value().domain_error());
                }
                return;
              }

              FX_LOGS(INFO) << "Export Success";
              // Read the exported data (encoded in byte form as persistent FIDL) from the
              // returned VMO. In a real implementation, instead of reading the VMO, we would
              // merely forward it to some other storage-handling process. Doing this using a VMO,
              // rather than FIDL IPC, would save us frivolous reads and writes at each hop.
              size_t content_size = 0;
              zx::vmo vmo = std::move(result->filled());
              if (vmo.get_prop_content_size(&content_size) != ZX_OK) {
                return;
              }
              std::vector<uint8_t> encoded_bytes;
              encoded_bytes.resize(content_size);
              if (vmo.read(encoded_bytes.data(), 0, content_size) != ZX_OK) {
                return;
              }
              // Decode the persistent FIDL that was just read from the file.
              fit::result exportable =
                  fidl::Unpersist<examples_keyvaluestore_supportexports::Exportable>(
                      cpp20::span(encoded_bytes));
              if (exportable.is_error()) {
                FX_LOGS(ERROR) << "Failed to unpersist: " << exportable.error_value();
                return;
              }
              if (!exportable->items().has_value()) {
                FX_LOGS(INFO) << "Expected items to be set";
                return;
              }
              auto& items = exportable->items().value();

              // Log some information about the exported data.
              FX_LOGS(INFO) << "Printing " << items.size() << " exported entries, which are:";
              for (const auto& item : items) {
                FX_LOGS(INFO) << "  * " << item.key();
              }
            });

    // Run the loop until the callback is resolved, at which point we can continue from here.
    loop.Run();
    loop.ResetQuit();
  }

  // TODO(https://fxbug.dev/42156498): We need to sleep here to make sure all logs get drained. Once the
  // referenced bug has been resolved, we can remove the sleep.
  sleep(2);
  return 0;
}

Server

// Copyright 2022 The Fuchsia Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

#include <fidl/examples.keyvaluestore.supportexports/cpp/fidl.h>
#include <lib/async-loop/cpp/loop.h>
#include <lib/async/cpp/task.h>
#include <lib/component/outgoing/cpp/outgoing_directory.h>
#include <lib/fidl/cpp/wire/channel.h>
#include <lib/syslog/cpp/macros.h>
#include <unistd.h>

#include <algorithm>

#include <re2/re2.h>

// An implementation of the |Store| protocol.
class StoreImpl final : public fidl::Server<examples_keyvaluestore_supportexports::Store> {
 public:
  // Bind this implementation to a channel.
  StoreImpl(async_dispatcher_t* dispatcher,
            fidl::ServerEnd<examples_keyvaluestore_supportexports::Store> server_end)
      : binding_(fidl::BindServer(
            dispatcher, std::move(server_end), this,
            [this](StoreImpl* impl, fidl::UnbindInfo info,
                   fidl::ServerEnd<examples_keyvaluestore_supportexports::Store> server_end) {
              if (info.reason() != ::fidl::Reason::kPeerClosedWhileReading) {
                FX_LOGS(ERROR) << "Shutdown unexpectedly";
              }
              delete this;
            })) {}

  void WriteItem(WriteItemRequest& request, WriteItemCompleter::Sync& completer) override {
    FX_LOGS(INFO) << "WriteItem request received";
    auto key = request.attempt().key();
    auto value = request.attempt().value();

    // Validate the key.
    if (!RE2::FullMatch(key, "^[A-Za-z]\\w+[A-Za-z0-9]$")) {
      FX_LOGS(INFO) << "Write error: INVALID_KEY, For key: " << key;
      FX_LOGS(INFO) << "WriteItem response sent";
      return completer.Reply(
          fit::error(examples_keyvaluestore_supportexports::WriteError::kInvalidKey));
    }

    // Validate the value.
    if (value.empty()) {
      FX_LOGS(INFO) << "Write error: INVALID_VALUE, For key: " << key;
      FX_LOGS(INFO) << "WriteItem response sent";
      return completer.Reply(
          fit::error(examples_keyvaluestore_supportexports::WriteError::kInvalidValue));
    }

    if (key_value_store_.find(key) != key_value_store_.end()) {
      FX_LOGS(INFO) << "Write error: ALREADY_EXISTS, For key: " << key;
      FX_LOGS(INFO) << "WriteItem response sent";
      return completer.Reply(
          fit::error(examples_keyvaluestore_supportexports::WriteError::kAlreadyExists));
    }

    // Ensure that the value does not already exist in the store.
    key_value_store_.insert({key, value});
    FX_LOGS(INFO) << "Wrote value at key: " << key;
    FX_LOGS(INFO) << "WriteItem response sent";
    return completer.Reply(fit::ok());
  }

  void Export(ExportRequest& request, ExportCompleter::Sync& completer) override {
    FX_LOGS(INFO) << "Export request received";
    completer.Reply(Export(std::move(request.empty())));
    FX_LOGS(INFO) << "Export response sent";
  }

  void handle_unknown_method(
      fidl::UnknownMethodMetadata<examples_keyvaluestore_supportexports::Store> metadata,
      fidl::UnknownMethodCompleter::Sync& completer) override {
    FX_LOGS(WARNING) << "Received an unknown method with ordinal " << metadata.method_ordinal;
  }

 private:
  using ExportError = ::examples_keyvaluestore_supportexports::ExportError;
  using Exportable = ::examples_keyvaluestore_supportexports::Exportable;
  using Item = ::examples_keyvaluestore_supportexports::Item;

  fit::result<ExportError, zx::vmo> Export(zx::vmo vmo) {
    if (key_value_store_.empty()) {
      return fit::error(ExportError::kEmpty);
    }
    Exportable exportable;
    std::vector<Item> items;
    items.reserve(key_value_store_.size());
    for (const auto& [k, v] : key_value_store_) {
      items.push_back(Item{{.key = k, .value = v}});
    }
    std::sort(items.begin(), items.end(),
              [](const Item& a, const Item& b) { return a.key() < b.key(); });
    exportable.items(std::move(items));
    fit::result encoded = fidl::Persist(exportable);
    if (encoded.is_error()) {
      FX_LOGS(ERROR) << "Failed to encode in persistence convention: " << encoded.error_value();
      return fit::error(ExportError::kUnknown);
    }
    size_t content_size = 0;
    if (vmo.get_prop_content_size(&content_size) != ZX_OK) {
      return fit::error(ExportError::kUnknown);
    }
    if (encoded->size() > content_size) {
      return fit::error(ExportError::kStorageTooSmall);
    }
    if (vmo.set_prop_content_size(encoded->size()) != ZX_OK) {
      return fit::error(ExportError::kUnknown);
    }
    if (vmo.write(encoded->data(), 0, encoded->size()) != ZX_OK) {
      return fit::error(ExportError::kUnknown);
    }
    return fit::ok(std::move(vmo));
  }

  fidl::ServerBindingRef<examples_keyvaluestore_supportexports::Store> binding_;

  // The map that serves as the per-connection instance of the key-value store.
  std::unordered_map<std::string, std::vector<uint8_t>> key_value_store_ = {};
};

int main(int argc, char** argv) {
  FX_LOGS(INFO) << "Started";

  // The event loop is used to asynchronously listen for incoming connections and requests from the
  // client. The following initializes the loop, and obtains the dispatcher, which will be used when
  // binding the server implementation to a channel.
  async::Loop loop(&kAsyncLoopConfigNeverAttachToThread);
  async_dispatcher_t* dispatcher = loop.dispatcher();

  // Create an |OutgoingDirectory| instance.
  //
  // The |component::OutgoingDirectory| class serves the outgoing directory for our component. This
  // directory is where the outgoing FIDL protocols are installed so that they can be provided to
  // other components.
  component::OutgoingDirectory outgoing = component::OutgoingDirectory(dispatcher);

  // The `ServeFromStartupInfo()` function sets up the outgoing directory with the startup handle.
  // The startup handle is a handle provided to every component by the system, so that they can
  // serve capabilities (e.g. FIDL protocols) to other components.
  zx::result result = outgoing.ServeFromStartupInfo();
  if (result.is_error()) {
    FX_LOGS(ERROR) << "Failed to serve outgoing directory: " << result.status_string();
    return -1;
  }

  // Register a handler for components trying to connect to |Store|.
  result = outgoing.AddUnmanagedProtocol<examples_keyvaluestore_supportexports::Store>(
      [dispatcher](fidl::ServerEnd<examples_keyvaluestore_supportexports::Store> server_end) {
        // Create an instance of our StoreImpl that destroys itself when the connection closes.
        new StoreImpl(dispatcher, std::move(server_end));
      });
  if (result.is_error()) {
    FX_LOGS(ERROR) << "Failed to add Store protocol: " << result.status_string();
    return -1;
  }

  // Everything is wired up. Sit back and run the loop until an incoming connection wakes us up.
  FX_LOGS(INFO) << "Listening for incoming connections";
  loop.Run();
  return 0;
}

C++ (Wire)

Client

// Copyright 2022 The Fuchsia Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

#include <fidl/examples.keyvaluestore.supportexports/cpp/wire.h>
#include <lib/async-loop/cpp/loop.h>
#include <lib/component/incoming/cpp/protocol.h>
#include <lib/syslog/cpp/macros.h>
#include <unistd.h>

#include <examples/fidl/new/key_value_store/support_exports/cpp_wire/client/config.h>
#include <src/lib/files/file.h>
#include <src/lib/fxl/strings/string_printf.h>

int main(int argc, const char** argv) {
  FX_LOGS(INFO) << "Started";

  // Retrieve component configuration.
  auto conf = config::Config::TakeFromStartupHandle();

  // Start up an async loop and dispatcher.
  async::Loop loop(&kAsyncLoopConfigNeverAttachToThread);
  async_dispatcher_t* dispatcher = loop.dispatcher();

  // Connect to the protocol inside the component's namespace. This can fail so it's wrapped in a
  // |zx::result| and it must be checked for errors.
  zx::result client_end = component::Connect<examples_keyvaluestore_supportexports::Store>();
  if (!client_end.is_ok()) {
    FX_LOGS(ERROR) << "Synchronous error when connecting to the |Store| protocol: "
                   << client_end.status_string();
    return -1;
  }

  // Create an asynchronous client using the newly-established connection.
  fidl::WireClient client(std::move(*client_end), dispatcher);
  FX_LOGS(INFO) << "Outgoing connection enabled";

  for (const auto& key : conf.write_items()) {
    std::string text;
    if (!files::ReadFileToString(fxl::StringPrintf("/pkg/data/%s.txt", key.c_str()), &text)) {
      FX_LOGS(ERROR) << "It looks like the correct `resource` dependency has not been packaged";
      break;
    }

    auto value = std::vector<uint8_t>(text.begin(), text.end());
    client
        ->WriteItem(
            {fidl::StringView::FromExternal(key), fidl::VectorView<uint8_t>::FromExternal(value)})
        .ThenExactlyOnce(
            [&](fidl::WireUnownedResult<examples_keyvaluestore_supportexports::Store::WriteItem>&
                    result) {
              if (!result.ok()) {
                FX_LOGS(ERROR) << "Unexpected framework error";
              } else if (result->is_error()) {
                FX_LOGS(INFO) << "WriteItem Error: " << fidl::ToUnderlying(result->error_value());
              } else {
                FX_LOGS(INFO) << "WriteItem Success";
              }

              // Quit the loop, thereby handing control back to the outer loop of actions being
              // iterated over.
              loop.Quit();
            });

    // Run the loop until the callback is resolved, at which point we can continue from here.
    loop.Run();
    loop.ResetQuit();
  }

  // If the `max_export_size` is 0, no export is possible, so just ignore this block. This check
  // isn't strictly necessary, but does avoid extra work down the line.
  if (conf.max_export_size() > 0) {
    // Create a 100Kb VMO to store the resulting export. In a real implementation, we would
    // likely receive the VMO representing the to-be-written file from file system like vfs of
    // fxfs.
    zx::vmo vmo;
    if (zx_status_t status = zx::vmo::create(conf.max_export_size(), 0, &vmo); status != ZX_OK) {
      FX_PLOGS(ERROR, status) << "Failed to create VMO";
      return -1;
    }

    client->Export(std::move(vmo))
        .ThenExactlyOnce(
            [&](fidl::WireUnownedResult<examples_keyvaluestore_supportexports::Store::Export>&
                    result) {
              // Quit the loop, thereby handing control back to the outer loop of actions being
              // iterated over, when we return from this callback.
              loop.Quit();

              if (!result.ok()) {
                FX_LOGS(ERROR) << "Unexpected FIDL framework error: " << result.error();
                return;
              }

              if (!result->is_ok()) {
                FX_LOGS(INFO) << "Export Error: " << fidl::ToUnderlying(result->error_value());
                return;
              }

              FX_LOGS(INFO) << "Export Success";
              // Read the exported data (encoded in byte form as persistent FIDL) from the
              // returned VMO. In a real implementation, instead of reading the VMO, we would
              // merely forward it to some other storage-handling process. Doing this using a VMO,
              // rather than FIDL IPC, would save us frivolous reads and writes at each hop.
              size_t content_size = 0;
              zx::vmo vmo = std::move(result->value()->filled);
              if (vmo.get_prop_content_size(&content_size) != ZX_OK) {
                return;
              }
              std::vector<uint8_t> encoded_bytes;
              encoded_bytes.resize(content_size);
              if (vmo.read(encoded_bytes.data(), 0, content_size) != ZX_OK) {
                return;
              }
              // Decode the persistent FIDL that was just read from the file.
              fit::result exportable =
                  fidl::InplaceUnpersist<examples_keyvaluestore_supportexports::wire::Exportable>(
                      cpp20::span(encoded_bytes));
              if (exportable.is_error()) {
                FX_LOGS(ERROR) << "Failed to unpersist: " << exportable.error_value();
                return;
              }
              if (!exportable->has_items()) {
                FX_LOGS(INFO) << "Expected items to be set";
                return;
              }
              auto& items = exportable->items();

              // Log some information about the exported data.
              FX_LOGS(INFO) << "Printing " << items.count() << " exported entries, which are:";
              for (const auto& item : items) {
                FX_LOGS(INFO) << "  * " << item.key.get();
              }
            });

    // Run the loop until the callback is resolved, at which point we can continue from here.
    loop.Run();
    loop.ResetQuit();
  }

  // TODO(https://fxbug.dev/42156498): We need to sleep here to make sure all logs get drained. Once the
  // referenced bug has been resolved, we can remove the sleep.
  sleep(2);
  return 0;
}

Server

// Copyright 2022 The Fuchsia Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.

#include <fidl/examples.keyvaluestore.supportexports/cpp/wire.h>
#include <lib/async-loop/cpp/loop.h>
#include <lib/async/cpp/task.h>
#include <lib/component/outgoing/cpp/outgoing_directory.h>
#include <lib/fidl/cpp/wire/channel.h>
#include <lib/syslog/cpp/macros.h>
#include <unistd.h>

#include <algorithm>

#include <re2/re2.h>

// An implementation of the |Store| protocol.
class StoreImpl final : public fidl::WireServer<examples_keyvaluestore_supportexports::Store> {
 public:
  // Bind this implementation to a channel.
  StoreImpl(async_dispatcher_t* dispatcher,
            fidl::ServerEnd<examples_keyvaluestore_supportexports::Store> server_end)
      : binding_(fidl::BindServer(
            dispatcher, std::move(server_end), this,
            [this](StoreImpl* impl, fidl::UnbindInfo info,
                   fidl::ServerEnd<examples_keyvaluestore_supportexports::Store> server_end) {
              if (info.reason() != ::fidl::Reason::kPeerClosedWhileReading) {
                FX_LOGS(ERROR) << "Shutdown unexpectedly";
              }
              delete this;
            })) {}

  void WriteItem(WriteItemRequestView request, WriteItemCompleter::Sync& completer) override {
    FX_LOGS(INFO) << "WriteItem request received";
    std::string key{request->attempt.key.get()};
    std::vector<uint8_t> value{request->attempt.value.begin(), request->attempt.value.end()};

    // Validate the key.
    if (!RE2::FullMatch(key, "^[A-Za-z]\\w+[A-Za-z0-9]$")) {
      FX_LOGS(INFO) << "Write error: INVALID_KEY, For key: " << key;
      FX_LOGS(INFO) << "WriteItem response sent";
      return completer.Reply(
          fit::error(examples_keyvaluestore_supportexports::WriteError::kInvalidKey));
    }

    // Validate the value.
    if (value.empty()) {
      FX_LOGS(INFO) << "Write error: INVALID_VALUE, For key: " << key;
      FX_LOGS(INFO) << "WriteItem response sent";
      return completer.Reply(
          fit::error(examples_keyvaluestore_supportexports::WriteError::kInvalidValue));
    }

    if (key_value_store_.find(key) != key_value_store_.end()) {
      FX_LOGS(INFO) << "Write error: ALREADY_EXISTS, For key: " << key;
      FX_LOGS(INFO) << "WriteItem response sent";
      return completer.Reply(
          fit::error(examples_keyvaluestore_supportexports::WriteError::kAlreadyExists));
    }

    // Ensure that the value does not already exist in the store.
    key_value_store_.insert({key, value});
    FX_LOGS(INFO) << "Wrote value at key: " << key;
    FX_LOGS(INFO) << "WriteItem response sent";
    return completer.Reply(fit::success());
  }

  void Export(ExportRequestView request, ExportCompleter::Sync& completer) override {
    FX_LOGS(INFO) << "Export request received";
    fit::result result = Export(std::move(request->empty));
    if (result.is_ok()) {
      completer.ReplySuccess(std::move(result.value()));
    } else {
      completer.ReplyError(result.error_value());
    }
    FX_LOGS(INFO) << "Export response sent";
  }

  using ExportError = ::examples_keyvaluestore_supportexports::wire::ExportError;
  using Exportable = ::examples_keyvaluestore_supportexports::wire::Exportable;
  using Item = ::examples_keyvaluestore_supportexports::wire::Item;

  fit::result<ExportError, zx::vmo> Export(zx::vmo vmo) {
    if (key_value_store_.empty()) {
      return fit::error(ExportError::kEmpty);
    }
    fidl::Arena arena;
    fidl::VectorView<Item> items;
    items.Allocate(arena, key_value_store_.size());
    size_t count = 0;
    for (auto& [k, v] : key_value_store_) {
      // Create a wire |Item| object that borrows from |k| and |v|.
      // Since |k| and |v| are references into the long living |key_value_store_|,
      // while |items| only live within the current function scope,
      // this operation is safe.
      items[count] = Item{
          .key = fidl::StringView::FromExternal(k),
          .value = fidl::VectorView<uint8_t>::FromExternal(v),
      };
      count++;
    }
    std::sort(items.begin(), items.end(),
              [](const Item& a, const Item& b) { return a.key.get() < b.key.get(); });
    Exportable exportable = Exportable::Builder(arena).items(items).Build();
    fit::result encoded = fidl::Persist(exportable);
    if (encoded.is_error()) {
      FX_LOGS(ERROR) << "Failed to encode in persistence convention: " << encoded.error_value();
      return fit::error(ExportError::kUnknown);
    }
    size_t content_size = 0;
    if (vmo.get_prop_content_size(&content_size) != ZX_OK) {
      return fit::error(ExportError::kUnknown);
    }
    if (encoded->size() > content_size) {
      return fit::error(ExportError::kStorageTooSmall);
    }
    if (vmo.set_prop_content_size(encoded->size()) != ZX_OK) {
      return fit::error(ExportError::kUnknown);
    }
    if (vmo.write(encoded->data(), 0, encoded->size()) != ZX_OK) {
      return fit::error(ExportError::kUnknown);
    }
    return fit::ok(std::move(vmo));
  }

  void handle_unknown_method(
      fidl::UnknownMethodMetadata<examples_keyvaluestore_supportexports::Store> metadata,
      fidl::UnknownMethodCompleter::Sync& completer) override {
    FX_LOGS(WARNING) << "Received an unknown method with ordinal " << metadata.method_ordinal;
  }

 private:
  fidl::ServerBindingRef<examples_keyvaluestore_supportexports::Store> binding_;

  // The map that serves as the per-connection instance of the key-value store.
  //
  // Out-of-line references in wire types are always mutable. Thus the
  // |const std::vector<uint8_t>| from the baseline needs to be changed to
  // non-const as we're making a vector view pointing to it during |Export|,
  // even though in practice the value is never mutated.
  std::unordered_map<std::string, std::vector<uint8_t>> key_value_store_ = {};
};

int main(int argc, char** argv) {
  FX_LOGS(INFO) << "Started";

  // The event loop is used to asynchronously listen for incoming connections and requests from the
  // client. The following initializes the loop, and obtains the dispatcher, which will be used when
  // binding the server implementation to a channel.
  async::Loop loop(&kAsyncLoopConfigNeverAttachToThread);
  async_dispatcher_t* dispatcher = loop.dispatcher();

  // Create an |OutgoingDirectory| instance.
  //
  // The |component::OutgoingDirectory| class serves the outgoing directory for our component. This
  // directory is where the outgoing FIDL protocols are installed so that they can be provided to
  // other components.
  component::OutgoingDirectory outgoing = component::OutgoingDirectory(dispatcher);

  // The `ServeFromStartupInfo()` function sets up the outgoing directory with the startup handle.
  // The startup handle is a handle provided to every component by the system, so that they can
  // serve capabilities (e.g. FIDL protocols) to other components.
  zx::result result = outgoing.ServeFromStartupInfo();
  if (result.is_error()) {
    FX_LOGS(ERROR) << "Failed to serve outgoing directory: " << result.status_string();
    return -1;
  }

  // Register a handler for components trying to connect to |Store|.
  result = outgoing.AddUnmanagedProtocol<examples_keyvaluestore_supportexports::Store>(
      [dispatcher](fidl::ServerEnd<examples_keyvaluestore_supportexports::Store> server_end) {
        // Create an instance of our StoreImpl that destroys itself when the connection closes.
        new StoreImpl(dispatcher, std::move(server_end));
      });
  if (result.is_error()) {
    FX_LOGS(ERROR) << "Failed to add Store protocol: " << result.status_string();
    return -1;
  }

  // Everything is wired up. Sit back and run the loop until an incoming connection wakes us up.
  FX_LOGS(INFO) << "Listening for incoming connections";
  loop.Run();
  return 0;
}

HLCPP

Client

// TODO(https://fxbug.dev/42060656): HLCPP implementation.

Server

// TODO(https://fxbug.dev/42060656): HLCPP implementation.