Diagnostics and monitoring

Connecting to FIDL protocols within components is a combination of capability routing and directory serving. This means that diagnosing connection issues can cover a few different layers:

• Client requests the protocol capability in its manifest.
• Provider exposes the protocol capability in its manifest.
• Component topology routes the capability from the provider to the client.
• Provider is serving the protocol on the correct handle.
• Client is attempting to connect to the correct protocol handle.

In this section, you'll explore some APIs and tools to help you find and fix problems with component connections, and monitor the long-term health of your components.

Monitoring FIDL connections

The fidlcat tool allows you to monitor and debug FIDL connections to trace individual FIDL messages sent and received by your component. Similar to the Fuchsia debugger (zxdb), fidlcat connects to a running debug_agent component on the target device and monitors running processes.

Setting up the monitoring session requires the following high-level steps:

1. Run the debug_agent component on the target device.
2. Run the fidlcat client and connect to the target device.

The simplest method to start a FIDL debug session is to use the ffx debug fidl command, which does all of these in the context of your local Fuchsia build. However, these steps can also be performed manually if you need to configure them separately.

Below is an example fidlcat message for a FIDL protocol request. The trace output contains helpful information for each translation, including:

• The component or process name
• The system call invoked
• The FIDL library, protocol, and method name
• Message payload containing parameters or return values

echo-client.cm 256109:256122 zx_channel_read(handle:handle: e4c7c57f, options:uint32: 0, num_bytes:uint32: 48, num_handles:uint32: 0)
  -> ZX_OK
    received response fidl.examples.echo/Echo.EchoString = {
      response: string = "hello world!"
    }

Using Inspect

Component Inspection enables Fuchsia components to expose structured diagnostic information about themselves using the Inspect API. Fuchsia provides this information through the developer tools and bug reports to assist in diagnosing issues or monitoring performance.

Components expose inspection metrics as a tree of named Nodes, each containing a set of Properties as key/value pairs. Properties support a variety of numeric, string, and array data types. The component inspector libraries provide an interface to your component's root node where you can attach additional properties of interest to your application.

You can retrieve the current set of metrics published to Inspect using the developer tools:

• ffx inspect: Lets you interactively query the Inspect state using component selectors. This is helpful for debugging components during development.
• ffx target snapshot: Captures a debug snapshot archive of the entire system, which contains the Inspect data in JSON format.

ffx inspect show core/foo-example

core/foo-example:
  metadata:
    filename = fuchsia.inspect.Tree
    component_url = fuchsia-pkg://fuchsia.com/foo-example#meta/foo-example.cm
    timestamp = 55457379176
  payload:
    root:
      version = 1.0
      request_metrics:
        request_count = 3
        error = timeout

Exercise: Monitoring provider components

In this section, you'll use the diagnostics tools to monitor the health and behavior of the echo server component.

Start the emulator

If you do not already have an instance running, start FEMU with networking support:

ffx emu start workstation_eng.x64 --headless

Monitor FIDL traffic

Create a new instance of the echo-realm containing the server and client components:

ffx component run /core/ffx-laboratory:echo-realm fuchsia-pkg://fuchsiasamples.com/echo-realm#meta/echo_realm.cm

You can use fidlcat to monitor and debug the FIDL connections in your components. Launch ffx debug fidl and configure it to monitor the echo server component:

ffx debug fidl --remote-name echo_server.cm --fidl-ir-path bazel-out/

Checking for debug agent on [fe80::d6c5:4526:c282:fb6%qemu]:2345.
Debug agent not found. Starting one.
INFO: [main.cc(238)] Connected to symbol server gs://fuchsia-artifacts-release/debug
INFO: [main.cc(122)] Connecting to port 2345 on fe80::d6c5:4526:c282:fb6%qemu...
INFO: [main.cc(92)] Connected!

Initiate a FIDL connection to the server by starting an echo client instance:

ffx component start /core/ffx-laboratory:echo-realm/echo_client

The client binds to the server component and communicates using the Echo FIDL protocol. Review the ffx debug fidl output to see a list of the FIDL transactions handled by echo server:

Monitoring echo_server.cm

echo_server.cm 58694:58696 zx_channel_read_etc(handle: handle = fb9b5273, options: uint32 = 0, num_bytes: uint32 = 512, num_handles: uint32 = 4)
  -> ZX_OK
    received request fuchsia.io/Directory.Open = { flags: uint32 = 3, mode: uint32 = 493, path: string = "svc/examples.routing.echo.Echo", object: handle = Channel:f93b597b(ZX_RIGHT_TRANSFER | ZX_RIGHT_READ | ZX_RIGHT_WRITE | ZX_RIGHT_SIGNAL | ZX_RIGHT_SIGNAL_PEER | ZX_RIGHT_WAIT | ZX_RIGHT_INSPECT)(channel:0:svc/examples.routing.echo.Echo) }

echo_server.cm 58694:58696 zx_channel_read_etc(handle: handle = Channel:f93b597b(channel:0:svc/examples.routing.echo.Echo), options: uint32 = 0, num_bytes: uint32 = 512, num_handles: uint32 = 4)
  -> ZX_OK
    received request examples.routing.echo/Echo.EchoString = { value: string = "Hello, Fuchsia" }

echo_server.cm 58694:58696 zx_channel_write_etc(handle: handle = Channel:f93b597b(channel:0:svc/examples.routing.echo.Echo), options: uint32 = 0)
  sent response examples.routing.echo/Echo.EchoString = { response: string = "Hello, Fuchsia" }
  -> ZX_OK

echo_server.cm 58694:58696 zx_channel_read_etc(handle: handle = Channel:f93b597b(channel:0:svc/examples.routing.echo.Echo), options: uint32 = 0, num_bytes: uint32 = 512, num_handles: uint32 = 4)
  -> ZX_ERR_PEER_CLOSED

echo_server.cm 58694:58696 zx_handle_close(handle: handle = Channel:f93b597b(channel:0:svc/examples.routing.echo.Echo))
  -> ZX_OK

Notice the sequence of events:

1. A channel to the protocol implementation opens at svc/examples.routing.echo.Echo.
2. The server receives an Echo.EchoString request over the open channel, containing the string payload sent by the client.
3. The server sends a corresponding response with the same string payload.
4. The channel closes.

By tracing the FIDL connections between your components, fidlcat enables you to find and diagnose potential issues such as failed connections or invalid data payloads.

Add request tracking

Component inspection allows you to publish diagnostic information from your components to assist in debugging. You'll use the Inspect API to track some usage statistics for the echo server component.

Update the echo_server request handler to accept a new struct containing numeric Inspect properties for request count and bytes processed. The handler increments these properties on each incoming request:

echo-server/main.cc:

struct EchoConnectionStats {
  inspect::UintProperty bytes_processed;
  inspect::UintProperty total_requests;
};

// Handler for incoming FIDL protocol requests
class EchoImplementation : public fidl::Server<examples_routing_echo::Echo> {
public:
  // The handler for `examples.routing.echo/Echo.EchoString` requests.
  //
  // Replies back to the caller with the original request value.
  void EchoString(EchoStringRequest &request,
                  EchoStringCompleter::Sync &completer) override {
    // Increment connection stats on each request
    stats_->total_requests.Add(1);
    stats_->bytes_processed.Add(request.value()->size());
    completer.Reply({{request.value()}});
  }

  // Called when the FIDL connection is torn down.
  void OnUnbound(fidl::UnbindInfo info,
                 fidl::ServerEnd<examples_routing_echo::Echo> server_end) {
    if (info.is_user_initiated()) {
      return;
    }
    if (info.is_peer_closed()) {
      // The peer (the client) closed their endpoint.
      FX_LOG(DEBUG, "echo_server", "Client disconnected.");
    } else {
      // Treat other unbind causes as errors.
      FX_LOGF(ERROR, "echo_server", "Server error: %s", info.status_string());
    }
  }

  std::unique_ptr<EchoConnectionStats> stats_;
};

Add the following code to main() to initialize the Inspect propertes and pass them to the updated handler:

echo-server/main.cc:

int main(int argc, const char** argv) {
  // ...

  // Serve the Echo protocol
  std::unique_ptr<EchoImplementation> echo_instance =
      std::make_unique<EchoImplementation>();

  // Create request tracking properties
  inspect::Node &root_node = inspector.root();
  auto total_requests = root_node.CreateUint("total_requests", 0);
  auto bytes_processed = root_node.CreateUint("bytes_processed", 0);
  echo_instance->stats_ =
      std::make_unique<EchoConnectionStats>(EchoConnectionStats{
          std::move(bytes_processed),
          std::move(total_requests),
      }); 

  // ...
}

Finally, update the imports to include the new Inspect libraries:

echo-server/main.cc:

#include <fidl/examples.routing.echo/cpp/fidl.h>
#include <lib/async-loop/cpp/loop.h>
#include <lib/async-loop/default.h>
#include <lib/component/outgoing/cpp/outgoing_directory.h>
#include <lib/inspect/component/cpp/component.h>
#include <lib/syslog/global.h>

Build and publish the updated package to the fuchsiasamples.com repository:

bazel run //fuchsia-codelab/echo-realm:pkg.publish -- \
    --repo_name fuchsiasamples.com

Verify the Inspect data

Create a new echo-realm component containing the updated echo-server:

ffx component run /core/ffx-laboratory:echo-realm fuchsia-pkg://fuchsiasamples.com/echo-realm#meta/echo_realm.cm \
    --recreate

Run the echo client component multiple times. This causes the request count in echo-server to increment with each connection:

ffx component start /core/ffx-laboratory:echo-realm/echo_client
ffx component start /core/ffx-laboratory:echo-realm/echo_client
ffx component start /core/ffx-laboratory:echo-realm/echo_client

View the available Inspect data for the echo server component with ffx inspect. You'll see the values for request count and bytes processed in the tree under the root node alongside the component health status:

ffx inspect show 'core/ffx-laboratory\:echo-realm/echo_server'

core/ffx-laboratory\:echo-realm/echo_server:
  metadata:
    filename = fuchsia.inspect.Tree
    component_url = #meta/echo_server.cm
    timestamp = 1476246046122
  payload:
    root:
      bytes_processed = 42
      total_requests = 3
      fuchsia.inspect.Health:
        start_timestamp_nanos = 1467828507317
        status = OK

Publishing health and behavior information using Inspect enables you to observe the current state of your components and diagnose issues on production devices.

Destroy the instance

Clean up the echo-realm instance using the following command:

ffx component destroy /core/ffx-laboratory:echo-realm

What's next?

Congratulations! You've successfully built a Fuchsia IPC interface using FIDL, and connected two components together using that interface.

You have completed all the modules in this course! Take your newfound understanding to the next level and dive deeper into the:

Fuchsia concepts