fuchsia.accessibility.semantics

PROTOCOLS

SemanticsManager

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

An interface to manage connections with views for the purposes of gathering semantic information about their current UI state.

The manager allows clients to register as a semantic provider for their view(s). In return the semantics manager supplies an interface to update, commit and delete information from the semantic tree for that view. If the semantic manager encounters an error, it will close the channel, delete any associated data and rely on the client to re-register.

RegisterViewForSemantics

Request

NameType
view_ref fuchsia.ui.views/ViewRef
listener SemanticListener
semantic_tree_request request<SemanticTree>

SemanticTree

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

Interface to update the semantic tree for a particular view. Nodes can be added, updated or deleted. Because the size of an update may exceed FIDL transfer limits, clients are responsible for breaking up changes into multiple update and delete calls that conform to these limits. The commit function must always be called at the end of a full update push to signal the end of an update.

The client may make several calls to UpdateSemanticNodes(...) or DeleteSemanticNodes(...) before calling CommitUpdates(), and must wait for the semantics manager to reply to the CommitUpdates() method to know whether an update has been processed. This allows the client to break up a set of changes (e.g. a re-computed semantic tree) to the semantic tree into FIDL-compatible chunks, but commit them all at once.

If the semantics manager ever receives inconsistent state from the client, such as an invalid tree or unrecognized parent node id, the server will close the channel. The client is responsible for reconnecting and re-sending its state from scratch.

UpdateSemanticNodes

Sends new/updated nodes to the root to add to the cache on the next commit.

Request

NameType
nodes vector<Node>[2048]

DeleteSemanticNodes

Tells the root to remove nodes with node_ids from the semantic tree on the next commit.

Request

NameType
node_ids vector<uint32>[2048]

CommitUpdates

Commits pending changes to node tree associated with the view using UpdateSemanticNodes and DeleteSemanticNodes. Updates are processed in the order in which they are received. If the committed updates result in an ill-formed tree (for example a missing root node or a cycle) the semantic manager will close the channel.

Request

NameType

Response

NameType

SemanticListener

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

A semantic provider is the client-side interface that the manager can use to enable or disable semantic updates, and to ask clients to perform accessibility actions.

OnAccessibilityActionRequested

Asks the semantics provider to perform an accessibility action on the node with node id in the front-end.

Request

NameType
node_id uint32
action Action

Response

NameType
handled bool

HitTest

Asks the semantics provider to perform hit testing and return the result.

Request

NameType
local_point fuchsia.math/PointF

Response

NameType
result Hit

OnSemanticsModeChanged

Callback telling the client whether or not to send updates to the semantic tree. The semantics manager will clear all state when this is called with updates_enabled = false. When called with updates_enabled = true, the client should sent the full state of the current semantic tree.

Request

NameType
updates_enabled bool

Response

NameType

ENUMS

Action

Type: uint32

Defined in fuchsia.accessibility.semantics/node.fidl

Represents actions that can be applied to Nodes.

NameValueDescription
DEFAULT 1 The default action associated with the role of an element.

Role

Type: uint32

Defined in fuchsia.accessibility.semantics/node.fidl

Represents a role of an element on a UI.

NameValueDescription
UNKNOWN 1 Role used to represent elements which role is not currently supported.

TABLES

Attributes

Defined in fuchsia.accessibility.semantics/node.fidl

An attribute is an essential property to describe an element. Unlike states, attributes do not change over the life of an element. Example: A button with a label attribute 'ok' should never change to 'cancel', as this is not the same element.

OrdinalNameTypeDescription
1 label string[16384] The label for an element. If longer than MAX_LABEL_SIZE the client is responsible for truncating the label.

States

Defined in fuchsia.accessibility.semantics/node.fidl

A state is a dynamic property of an element that may change in response to user action or automated processes. Thus, they are different from attributes in an important point, which is frequency of change. Example: a checkbox can be checked / unchecked, and this state can be altered via user input.

OrdinalNameTypeDescription
1 checked bool Whether the element is checked.

Node

Defined in fuchsia.accessibility.semantics/node.fidl

Node: data structure to represent semantic information about a UI element.

The Node represents a semantic element on an interface. This may be a button, a text field, a checkbox or any element that has a relevant semantic meaning so that assistive technology can understand the current UI.

OrdinalNameTypeDescription
1 node_id uint32 Unique ID that represents a node in a particular UI. Zero is assumed to be the root node and the only entry point to the tree. No forest is allowed.
2 role Role Role of this element, e.g. button, checkbox, etc.
3 states States A table of states of this object, e.g. checked, editable, etc.
4 attributes Attributes A table of attributes of this node.
5 actions vector<Action>[100] A list of actions that can be performed on this node.
6 child_ids vector<uint32>[256] The list of child IDs of this node, in traversal order. Runtimes supplying semantic tree information are responsible for ensuring the tree does not contain cycles. Each node may have only one parent.
7 location fuchsia.ui.gfx/BoundingBox Local bounding box of this element.
8 transform fuchsia.ui.gfx/mat4 Transform from parent coordinate space to local space.

Hit

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

Results of hit testing on a view's semantic tree which is implemented by Runtimes(like Flutter/Chrome) and sent to Accessibility.

OrdinalNameTypeDescription
1 node_id uint32 Unique ID that represents a node in a particular UI. Zero is assumed to be the root node and the only entry point to the tree. node_id will not be filled when there is no hit.
2 path_from_root vector<uint32>[256] The ordered list of node ids which represent path from root node to the hit node.

CONSTANTS

NameValueTypeDescription
MAX_TREE_DEPTH 256 uint64 Maximum depth of the semantic tree.
MAX_FAN_OUT 256 uint64 Maximum number of children for a node in the semantic tree.
MAX_NODES_PER_UPDATE 2048 uint64 Maximum number of semantic nodes that may be sent in a single update.
MAX_LABEL_SIZE 16384 uint64 Maximum size of a label string, in bytes.