# Configuring Triage

Triage analyzes Diagnostics data according to config files.

## Overview

Triage allows anyone to easily add new ways to analyze `fx snapshot` data for
off-nominal conditions.

By default, the config files are read from
`//src/diagnostics/config/triage/*.triage`. Just add a new config file there.

Config file syntax is JSON5.

Each config file specifies four kinds of configuration: Selectors and Evals
(collectively called "Metrics"), Actions, and Tests.

- Selectors load values for use by Evals and Actions.
- Evals calculate values for use by Evals and Actions.
- Actions determine how to act upon certain specified values.
- Tests include sample data to verify that specified Actions trigger correctly.

Each Select, Eval, Test, and Action has a name. Thus, the structure of a config
file is:

    {
        "select": {
            "select1": "type:component:node/path:property",
            "select2": "type:component:node/path:property"
        },
        "eval": {
            "name1": "select1+select2",
            "name2": "select1 - select2"
        },
        "act": {
            "action1": { .... },
            "action2": { .... }
        },
        "test": {
            "test1": { .... },
            "test2": { .... }
        }
    }

## Names and namespaces

Select, Eval, Action, Test, and config file names consist of one
lowercase-alphabetic-or-underscore character followed by zero or more
alphanumeric-or-underscore characters. Thus, "abc123" and "_abc_123" are valid
names, but "123abc", "a.b.c", and "abc-123" are not. In particular, file names
are not allowed to contain periods except for the `.triage` extension.

Evals, Tests, and Actions in one file can refer to Selectors, Evals, and Actions
in another file. The file basename is used as a namespace. `::` is used as the
separator. For example, if file `foo.triage` is loaded and contains a Metric
named `bar` then any config file may refer to `foo::bar`.

Names may be reused between Metrics, Tests, and Actions, but not between Select
and Eval.

User-defined names must start with a lowercase letter. Language-supplied
functions start with uppercase letters.

NOTE: The current version of the program is not guaranteed to enforce these
restrictions.

### Selectors

Selectors use the [Selector](https://fuchsia.dev/docs/reference/diagnostics/selectors) format. The text before the first `:` selects the
component name from the `inspect.json` file. The `.`-separated middle section
specifies Inspect Node names forming a path to the Property named after the
second `:`.

### Calculation

Eval strings are infix math expressions with normal operator precedence.

() may be used.

Arithmetic operators are `+` `-` `*` `/` `//` `/?` `//?`. `/` is float division; `//` is int
division. Division operators with a `?` return Problem::Ignore on division by 0.

Functions are a function name, '(', comma-separated expression list, ')'.
Provided functions include:

- Boolean
  - `And(1+ args)`
  - `Or(1+ args)`
  - `Not(1 arg)`
  - `Missing(value)` returns true if the value is a `Missing` type error indication.
  - `Problem(value)` returns true if the value is any kind of error.
  - `True()` returns true
  - `False()` returns false
- Numeric
  - `Min(1+ args)`
  - `Max(1+ args)`
  - `Abs(1 arg)`
- Functional
  - `Fn([name1, name2, ...], expression)`
  - `Map(function, vector1, vector2, ...)`
  - `Fold(function, vector, optional_start_value)`
  - `Filter(function, vector)`
  - `Apply(function, [arg1, arg2, ...])`
  - `CountProperties(vector)` counts the number of properties in a vector.
  - `CountChildren(vector)` counts the number of children (nodes) in a vector.
- Time
  - `Days()`, `Hours()`, `Minutes()`, `Seconds()`, `Millis()`, `Micros()`, and `Nanos()` calculate values for comparison with monotonic timestamps.
  - `Now()` returns the approximate timestamp when the Diagnostic data was created.
- Other
  - `Option(value1, value2, value3...)` returns the first useful value, to support selector migrations and defaults: the first non-empty-list, non-Missing value if any; or empty list if one was given; or Missing.
  - `Annotation(string)` fetches the corresponding key from the annotations.json file, if present.
  - `SyslogHas(regex)`, `KlogHas(regex)`, `BootlogHas(regex)` return `true` if the syslog, kernel log, or previous-boot log contain a line matching the regex.
  - `StringMatches(value, regex)` applies the given regex to the given value and returns true if there is a match. The regex syntax is that supported by the Rust [regex crate](https://docs.rs/regex/latest/regex/).

Metric type follows the type read from the Inspect file. Currently, UInt is
converted to Int upon reading. Operating on mixed Int and Float promotes the
result to Float.

Boolean operations are `>` `<` `>=` `<=` `==` `!=`. The equality tests `==` and
`!=` compare
numbers, Booleans, strings, and vectors. `>` `<` `>=` `<=` only compare numbers.

Whitespace is optional everywhere, but recommended around infix operators.

Metric names, including namespaced names, do not need to be specially delimited.

#### Functional programming and vectors

Every selector actually returns a vector, but one-item vectors are
automatically unwrapped for the purposes of arithmetic and boolean
calculations. Inspect selectors without wildcards return a one-item
vector unless the same moniker occurs multiple times in inspect.json.

Selectors with wildcards, selectors for `bootstrap/driver_manager` and
`core/network/netstack`, and (eventually) selectors on logs, may return
multiple items in a vector. To process such values, Triage provides the
following functions:

- Fn(parameters, expression) - for example, `Fn([a, b], a+b)`
- Map(function, vector1, vector2...)
- Fold(function, vector) or Fold(function, vector, start_value)
- Filter(function, vector)
- Count(vector)

Vectors of values are written `[ expr, expr, expr ]`.

If a `values` argument to Map is not a vector, its value is applied to each
iteration. If all `values` are not vectors, or no `values` are supplied, an
empty vector is returned. If the vector `values` are different lengths, the
shortest one determines the result length and remaining values are not used.

Count() does not check the type of items in a vector `values`. Count() of a
non-vector `values` returns Missing.

If a Fn expression is the entirety of an 'eval' expression, the name of that
expression can be used as the first argument to Map, Fold, or Filter.

If a function has the wrong arity for its arguments, the function it was passed
to returns Missing. If the function's evaluation fails, for example due to
inappropriate types, the function it was passed to may return a partial value:

- Map returns a vector, some elements of which may be Missing.
- Fold returns Missing.
- Filter expects its filter function to return Boolean true or false. If that function returns anything else, including Missing, Filter adds a Missing value at that point in its result list.

## Actions

Each Action determines how to surface information for a given selector.
Currently, there are two types of actions, "Warning" and "Gauge". Actions are
specified by providing the appropriate value for the `type` field.

### Warning Type

A `Warning` is an action that is used to raise an alert when a boolean condition
is met.

`Warning` supports the following fields:

- `trigger`, a required field, specifies the name of a Metric that supplies a Boolean value.
- `print`, a required field, specifies a string to output when the warning is raised.
- `tag`, an optional field, associates a tag with this Action.
- `file_bug`, an optional string field, specifies that a bug should be filed and where. Triage does not file bugs directly; this field simply informs the consumer (a human user or an automated pipeline). It can be an Issue Tracker component (e.g. "Fuchsia \> Internationalization (i18n)") or references to other issue trackers understood by the consumer.

        "actions": {
            "disk_usage_high": {
                "type": "Warning", "trigger": "disk_used / disk_total > 0.95", "print": "Disk usage is high!"
            }
        }

### Gauge Type

A `Gauge` is a snapshot of a particular value at the time Triage is invoked.
`Gauge` supports the following fields:

- `value`, a required field, specifies a value to display.
- `format`, an optional field, specifies formatting rules for the gauge's value.

#### Format

The `format` field allows users to control how the gauge value is displayed. If
this field isn't provided, or if an invalid value is given, then value will be
displayed as is. `format` supports the following values:

- `percentage`: prints a float as a percentage value.

        "actions": {
            "disk_usage": {
                "type": "Gauge", "value": "disk_used / disk_total", "format": "percentage"
            }
        }

## Tests

Each Test specifies:

- Sample data, keyed by `inspect`
- A list of actions that should trigger given that data, keyed by `yes`
- A list of actions that should not trigger given that data, keyed by `no`

The sample data is in the same format as an inspect.json file: an array of maps
where each map contains `path` and `contents` fields.

        "tests": {
            "test1": {
                "yes": ["action1", "action2"],
                "no": ["action3"],
                "inspect": [
                    {
                        "path": "global_data",
                        "contents": {"root": {"stats":
                            {"total_bytes": 10, "used_bytes": 9}}}
                    }
                ]
            }
        }

[](https://issues.fuchsia.dev/issues/new?component=1439230&template=1881640&description=Issue+on:+https://fuchsia.dev/docs/development/diagnostics/triage/config.md "Report a bug or suggest improvements on this page") [](https://cs.opensource.google/fuchsia/fuchsia/+/main:docs/development/diagnostics/triage/config.md "View the source code of this page") [](https://ci.android.com/edit?repo=fuchsia/fuchsia/main&file=docs/development/diagnostics/triage/config.md&cloneRepo=fuchsia/fuchsia:refs/heads/main&fullPath=fuchsia/docs/development/diagnostics/triage/config.md "Edit the source code of this page")