fuchsia.test.manager

Added: 16

PROTOCOLS

DebugDataIterator

Defined in fuchsia.test.manager/test_manager.fidl

An iterator protocol over which a client may retrieve debug data information.

GetNext

Retrieve the next batch of debug data. This is a hanging get; if no data is immediately available, the call hangs until data is available. After all data has been returned, the call returns an empty vector.

Removed: 31

Request

<EMPTY>

Response

NameType
data vector<DebugData>

GetNextCompressed

Retrieve the next batch of debug data (compressed using zstd). This is a hanging get; if no data is immediately available, the call hangs until data is available. After all data has been returned, the call returns an empty vector.

Added: 31

Request

<EMPTY>

Response

NameType
data vector<DebugData>

EarlyBootProfile

Defined in fuchsia.test.manager/test_manager.fidl

Protocol to manage Early boot profiles. This should be called by our clients after running all the tests.

RegisterWatcher

Register iterator for watching early boot profiles.

Request

NameType
iterator server_end:DebugDataIterator

SuiteController

Defined in fuchsia.test.manager/test_manager.fidl

Provides for control and monitoring of a running test suite started with SuiteRunner.RunSuite.

The server closes its end of the channel after the suite run has finished and all events have been delivered via WatchEvents. If the client disconnects, the suite is terminated immediately and all results discarded.

Kill

Immediately terminate the run. SuiteController will disconnect after all resources are released. This method will terminate tests even if they are in progress.

Request

<EMPTY>

Stop

Stop the suite run gracefully. SuiteController will disconnect after all resources are released and all the events in this controller are drained.

Request

<EMPTY>

WatchEvents

Returns events when they're available using a hanging get pattern. Returns an empty vector to indicate there will be no further events.

Added: 20

Request

<EMPTY>

Response

NameType
payload SuiteController_WatchEvents_Result

SuiteRunner

Defined in fuchsia.test.manager/test_manager.fidl

Runs test suites.

Added: 20

Run

Run a test suite. A suite is a test component that implements fuchsia.test.Suite.

Request

NameType
test_suite_url fuchsia.url/Url
options RunSuiteOptions
controller server_end:SuiteController

TestCaseEnumerator

Defined in fuchsia.test.manager/test_manager.fidl

Enumerates cases in test suites, which implement the fuchsia.test.Suite protocol.

Added: 20

Enumerate

Creates an iterator for test cases.

Request

NameType
test_suite_url fuchsia.url/Url
options EnumerateTestCasesOptions
iterator server_end:TestCaseIterator

Response

NameType
payload TestCaseEnumerator_Enumerate_Result

TestCaseIterator

Defined in fuchsia.test.manager/test_manager.fidl

Iterator for listing available test cases.

Added: 20

GetNext

Returns the next batch of test cases. Returns the empty vector to indicate that the iteration is complete.

Request

<EMPTY>

Response

NameType
test_cases vector<TestCase>:1024

STRUCTS

DirectoryAndToken resource

Defined in fuchsia.test.manager/test_manager.fidl

A handle to a directory and a token used to indicate when the client has completed inspecting the directory. The server end will retain all resources, such as subdirectories and files, within |directory| while |release_fence| remains open. |release_fence| is used instead of observing the |directory| channel directly as it is possible to clone and open new channels to the same directory.

FieldTypeDescriptionDefault
directory client_end:fuchsia.io/Directory

Directory channel providing access to the directory. This channel should not be used after release_fence is closed.

 No default
token handle<eventpair>

An eventpair handle used to control the retention of the directory. When this handle is closed, the directory is no longer retained.

 No default

SuiteController_WatchEvents_Response resource

Defined in fuchsia.test.manager/test_manager.fidl

FieldTypeDescriptionDefault
events vector<Event>:1024 No default

TestCaseEnumerator_Enumerate_Response

Defined in fuchsia.test.manager/test_manager.fidl

<EMPTY>

ENUMS

LaunchError flexible

Type: uint32

Defined in fuchsia.test.manager/test_manager.fidl

Error for LaunchSuite call.

NameValueDescription
1

There were insufficient resources to perform the operation.
2

Cannot resolve test_suite_url.
3

Invalid argument(s) passed.
4

Failed to connect to the fuchsia.test.TestSuite that the test should expose.
5

Failed to enumerate tests.
6

Some internal error occurred. Something wrong with test manager setup. Check logs and report bug.
7

No test cases matched the specified test filters. This error is only returned when a test filter is specified. In the case of a test suite with no test cases, the suite will pass.
8

Test manifest is invalid.

LogsIteratorType flexible

Type: uint32

Defined in fuchsia.test.manager/test_manager.fidl

Enumeration of alternative log iteration mechanisms.

Added: 20

NameValueDescription
1

Indicates use of fuchsia.diagnostics.BatchIterator. This iterator type employs VMOs, which yields the best performance locally but cannot be used remotely (e.g. from the host). When this type is selected, artifacts of type log will have values that use the batch variant of the Syslog union.
2

Indicates the use of a socket as described in fuchsia.diagnostics.host.ArchiveAccessor. can be used remote (e.g. from the host). When this type is selected. artifacts of type log will have values that use the stream variant of the Syslog union.

SuiteResult flexible

Type: uint32

Defined in fuchsia.test.manager/test_manager.fidl

Represents the result of a suite run.

Added: 20

NameValueDescription
1

The suite finished normally, with all test case results being either SKIPPED or PASSED.
2

The suite finished normally, with some test case results being neither SKIPPED nor PASSED.
3

Suite implementation crashed, did not send Finish event, or did not report test case result for one or more test cases.
4

The overall suite run timed out .
5

The suite run was stopped.
6

TestCaseResult flexible

Type: uint32

Defined in fuchsia.test.manager/test_manager.fidl

Represent the result of a test case run.

Added: 20

NameValueDescription
1

Test case was skipped.
2

The test case passed.
3

Test case failed.
4

Test case timed out.
5

Suite implementation did not return a result for the test case.

TABLES

CustomArtifact resource

Defined in fuchsia.test.manager/test_manager.fidl

Describes a directory containing 'custom' (unclassified) artifacts produced by a test.

OrdinalFieldTypeDescription
component_moniker string:4096

The moniker of the component that produced the directory, relative to the root of the test realm.
directory_and_token DirectoryAndToken

A directory containing the artifacts.

DebugData resource

Defined in fuchsia.test.manager/test_manager.fidl

Describes available debug data.

OrdinalFieldTypeDescription
name string:512

Name of the file. Must be unique per DebugDataIterator.
socket handle<socket>

Socket over which the file may be accessed.

EnumerateTestCasesOptions resource

Defined in fuchsia.test.manager/test_manager.fidl

Options specifying how test cases should be enumerated.

Added: 20

OrdinalFieldTypeDescription
realm_options RealmOptions

Specifies the realm in which to enumerate test cases. If this field is not supplied, the test cases will be enumerated in a hermetic realm inside the test manager. This option is used by clients that require non-hermetic realms or test realms with custom runners.

Event resource

Defined in fuchsia.test.manager/test_manager.fidl

An event delivered via SuiteController.WatchEvents.

Added: 20

OrdinalFieldTypeDescription
timestamp zx/Time

The time at which the event occurred.
details EventDetails

The details of the event.

RealmOptions resource

Defined in fuchsia.test.manager/test_manager.fidl

Options specifying the realm in which a test suite should be run. These options are used by clients that require non-hermetic realms or test realms with custom runners. See [https://fuchsia.dev/fuchsia-src/development/testing/components/create_test_realm] and [https://fuchsia.dev/fuchsia-src/development/testing/components/test_runner_framework#non-hermetic_tests] for details.

Added: 20

OrdinalFieldTypeDescription
realm client_end:fuchsia.component/Realm

The realm which contains the collection in which to launch the test. This field is required.
offers vector<fuchsia.component.decl/Offer>:1024

All offers from the realm to the test collection. This field is required.
test_collection string:255

The test collection in which to launch the test. This field is required.

RunSuiteOptions resource

Defined in fuchsia.test.manager/test_manager.fidl

Options specifying how a test suite should be run.

Added: 20

OrdinalFieldTypeDescription
realm_options RealmOptions

Specifies the realm in which to run the test suite. If this field is not supplied, the test will run in a hermetic realm inside the test manager. This option is used by clients that require non-hermetic realms or test realms with custom runners.
run_disabled_tests bool

If set to true, test cases that have been disabled by the test author will nonetheless be executed. This value is false by default.
max_concurrent_test_case_runs uint16

Defines the maximum number of test cases to run concurrently. If unspecified, the test suite component decides this value.
arguments vector<string:16384>:4095

Command-line arguments to pass to the test. Test runners decide how to pass these arguments to tests. This value is an empty vector (no arguments) by default.
timeout zx/Duration

Timeout for the entire suite run. If unspecified, there is no timeout, and the suite run may hang indefinitely.
test_case_filters vector<string:4096>:1024

Test case filters as glob patterns [https://en.wikipedia.org/wiki/Glob_(programming)]. Negative filters may be specified by prepending '-'. This value is an empty vector (no filters) by default.

A given test case is run if both of the following are true:

  • No positive filters are specfied, or the test case matches one of the positive filters.
  • The test case does not match any specified negative filter.

For example, given that a suite has the test cases Foo.Test1, Foo.Test2, Bar.Test1, and Bar.Test2:

  • The filters ["Foo.*"] will execute Foo.Test1 and Foo.Test2.
  • The filters ["-Foo.*"] will execute Bar.Test1 and Bar.Test2.
  • The filters ["Foo.*", "-*.Test1"] will execute Foo.Test2.
logs_iterator_type LogsIteratorType

Specifies what kind of iterator the client will use for retrieving logs. This value is BATCH by default.
log_interest vector<fuchsia.diagnostics/LogInterestSelector>:64

Configures the minimum severity to apply when filtering logs from the test suite component.
accumulate_debug_data bool

If set to true, debug data collected for this run will be accumulated in test manager's tmp folder with debug data collected in previous runs with this flag set true. Defaults to false.

This option is used when many tests are run in a batch, and delivering the accumulated data is more performant than delivering the debug data one test at a time.
no_exception_channel bool

If true, indicates the test creates an exception channel. In this case, the test manager will refrain from creating its own exception channels to avoid conflicts. Default value is false.

Added: 26
break_on_failure bool

If true, the test runner should halt (if supported) the test suite when a failure is encountered such that a debugger may attach to the process in the future. Test runners may safely ignore this if they do not support stopping running test suites.

Added: 28

SuiteArtifactGeneratedEventDetails resource

Defined in fuchsia.test.manager/test_manager.fidl

Details for suite_artifact_generated events.

Added: 20

OrdinalFieldTypeDescription
artifact Artifact

Describes the artifact. This field will always be provided.

SuiteStartedEventDetails

Defined in fuchsia.test.manager/test_manager.fidl

Details for suite_started events.

Added: 20

OrdinalFieldTypeDescription

SuiteStoppedEventDetails

Defined in fuchsia.test.manager/test_manager.fidl

Details for suite_stopped events.

Added: 20

OrdinalFieldTypeDescription
result SuiteResult

The suite result. This field will always be provided.

TestCase

Defined in fuchsia.test.manager/test_manager.fidl

Description of an enumerated test case.

OrdinalFieldTypeDescription
name TestCaseName

Name of the test case.

TestCaseArtifactGeneratedEventDetails resource

Defined in fuchsia.test.manager/test_manager.fidl

Details for test_case_artifact_generated events.

Added: 20

OrdinalFieldTypeDescription
test_case_id TestCaseId

The test case to which this event pertains. This event will be preceeded by a 'test_case found' event with a matching id. This field will always be provided.
artifact Artifact

Describes the artifact. This field will always be provided.

TestCaseFinishedEventDetails

Defined in fuchsia.test.manager/test_manager.fidl

Details for test_case_finished events.

Added: 20

OrdinalFieldTypeDescription
test_case_id TestCaseId

The test case to which this event pertains. This event will be preceeded by a 'test_case found' event with a matching id. This field will always be provided.

TestCaseFoundEventDetails

Defined in fuchsia.test.manager/test_manager.fidl

Details for test_case_found events.

Added: 20

OrdinalFieldTypeDescription
test_case_name TestCaseName

Name of the test case that was found. This field will always be provided.
test_case_id TestCaseId

Identifies this test case in subsequent events. This field will always be provided and is unique for a given SuiteController.

TestCaseStartedEventDetails

Defined in fuchsia.test.manager/test_manager.fidl

Details for test_case_started events.

Added: 20

OrdinalFieldTypeDescription
test_case_id TestCaseId

The test case to which this event pertains. This event will be preceeded by a 'test_case found' event with a matching id. This field will always be provided.

TestCaseStoppedEventDetails

Defined in fuchsia.test.manager/test_manager.fidl

Details for test_case_stopped events.

Added: 20

OrdinalFieldTypeDescription
test_case_id TestCaseId

The test case to which this event pertains. This event will be preceeded by a 'test_case found' event with a matching id. This field will always be provided.
result TestCaseResult

The test case result. This field will always be provided.

UNIONS

Artifact flexible resource

Defined in fuchsia.test.manager/test_manager.fidl

Describes one or more artifacts.

OrdinalVariantTypeDescription
stdout handle<socket>

The artifact is the 'stdout' stream of the suite or test case. The artifact is delivered via a socket, the consumer end of which is provided here.
stderr handle<socket>

The artifact is the 'stderr' stream of the suite or test case. The artifact is delivered via a socket, the consumer end of which is provided here.
log Syslog

The artifact is the syslog of the suite or test case. The artifact is delivered using a batch iterator or socket.
custom CustomArtifact

The artifacts are one or more files in a directory and may be read using fuchsia.io.
debug_data client_end:DebugDataIterator

The artifacts are debug data delivered using a DebugDataIterator channel.

EventDetails flexible resource

Defined in fuchsia.test.manager/test_manager.fidl

Details of an event delivered via SuiteController.WatchEvents.

A suite_started event always precedes any events relating to test cases, and a suite_stopped event always follows any test case events. suite_artifact_generated may occur at any point, including before suite_started and after suite_stopped.

A test_case_found event is produced for all test cases found in the suite. If a particular test case run is enabled (based on RunSuiteOptions.test_case_filters and RunSuiteOptions.run_disabled_tests), the following sequence is produced, regardless of whether the test case is actually run, the run completes or whether or not it succeeds:

  • test_case_found
  • test_case_started
  • test_case_stopped
  • test_case_finished

test_case_artifact_generated events for the test case may occur at any point after the test_case_found event and before test_case_finished event for that test case. Note that test case events for multiple events may be interleaved.

If a test case run is not enabled, only the test_case_found event will be produced for that test case.

Added: 20
OrdinalVariantTypeDescription
suite_started SuiteStartedEventDetails

Suite started execution. suite_artifact_generated events may occur before this event.
test_case_found TestCaseFoundEventDetails

A test_case was found. This is always the first event for a given test case.
test_case_started TestCaseStartedEventDetails

A test case started execution. Only one test_case_started event is produced for a given test case, and it always precedes the test_case_stopped event for that test case. test_case_artifact_generated events for the test case may occur before this event.
test_case_artifact_generated TestCaseArtifactGeneratedEventDetails

Artifact from a test case. Note that test_case_artifact_generated events for a given test case may occur before test_case_started and after test_case_stopped.
test_case_stopped TestCaseStoppedEventDetails

A test case stopped executing. This event includes the resulting TestCaseResult of the test case. test_case_artifact_generated events for the case may occur after this event.
test_case_finished TestCaseFinishedEventDetails

A test case has finished and all artifact events have been dispatched to the client. This is always the last event for a given test case.
suite_artifact_generated SuiteArtifactGeneratedEventDetails

Artifact pertaining to the entire suite.
suite_stopped SuiteStoppedEventDetails

Suite run stopped executing. This event includes the resulting SuiteResult of the suite. suite_artifact_generated events may occur after this event.

LogsIterator flexible resource

Defined in fuchsia.test.manager/test_manager.fidl

Holds the server end of an iterator over the isolated logs of a test.

OrdinalVariantTypeDescription
batch server_end:fuchsia.diagnostics/BatchIterator

Server end of the iterator, when this protocol is used by Fuchsia clients.
stream handle<socket>

Server end of the iterator, when this protocol is used by host-side clients. This uses the protocol specified in fuchsia.diagnostics.host.ArchiveReader.

SuiteController_WatchEvents_Result strict resource

Defined in fuchsia.test.manager/test_manager.fidl

OrdinalVariantTypeDescription
response SuiteController_WatchEvents_Response
err LaunchError

Syslog flexible resource

Defined in fuchsia.test.manager/test_manager.fidl

Delivery method for syslog.

OrdinalVariantTypeDescription
batch client_end:fuchsia.diagnostics/BatchIterator

Client end of the iterator used by Fuchsia clients.
stream handle<socket>

Consumer end of the socket used by host-side clients.

TestCaseEnumerator_Enumerate_Result strict

Defined in fuchsia.test.manager/test_manager.fidl

OrdinalVariantTypeDescription
response TestCaseEnumerator_Enumerate_Response
err LaunchError
framework_err internal

CONSTANTS

NameValueTypeDescription
MAX_ARGUMENTS fuchsia.io/MAX_PATH_LENGTH uint64
Added: 20
MAX_ARGUMENT_LENGTH 16384 uint64
Added: 28
MAX_DEBUG_DATAS_PER_GET 1024 uint64
Added: 20
MAX_EVENTS_PER_WATCH 1024 uint64
Added: 20
MAX_FILTERS 1024 uint64
Added: 20
MAX_FILTER_LENGTH fuchsia.component/MAX_MONIKER_LENGTH uint64
Added: 20
MAX_OFFERS 1024 uint64
Added: 20
MAX_TEST_CASES_PER_GET 1024 uint64
Added: 20
MAX_TEST_COLLECTION_NAME_LENGTH fuchsia.io/MAX_NAME_LENGTH uint64
Added: 20

ALIASES

NameValueDescription
TestCaseId uint32

Test case identifier. Unique in a suite run.
TestCaseName string[fuchsia.test/MAX_TEST_NAME]

Human-readable name for a test case.