Google is committed to advancing racial equity for Black communities. See how.

fuchsia.accessibility.semantics

PROTOCOLS

SemanticListener

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

A semantic provider is the client-side interface that the manager can use to enable or disable semantic updates, and to ask clients to perform accessibility actions.

HitTest

Asks the semantics provider to perform hit testing and return the result.

Request

NameType
local_point fuchsia.math/PointF

Response

NameType
result Hit

OnAccessibilityActionRequested

Asks the semantics provider to perform an accessibility action on the node with node id in the front-end.

Request

NameType
node_id uint32
action Action

Response

NameType
handled bool

OnSemanticsModeChanged

Callback telling the client whether or not to send updates to the semantic tree. The semantics manager will clear all state when this is called with updates_enabled = false. When called with updates_enabled = true, the client should sent the full state of the current semantic tree.

Request

NameType
updates_enabled bool

Response

NameType

SemanticTree

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

Interface to update the semantic tree for a particular view. Nodes can be added, updated or deleted. Because the size of an update may exceed FIDL transfer limits, clients are responsible for breaking up changes into multiple update and delete calls that conform to these limits. The commit function must always be called at the end of a full update push to signal the end of an update.

The client may make several calls to UpdateSemanticNodes(...) or DeleteSemanticNodes(...) before calling CommitUpdates(), and must wait for the semantics manager to reply to the CommitUpdates() method to know whether an update has been processed. This allows the client to break up a set of changes (e.g. a re-computed semantic tree) to the semantic tree into FIDL-compatible chunks, but commit them all at once.

If the semantics manager ever receives inconsistent state from the client, such as an invalid tree or unrecognized parent node id, the server will close the channel. The client is responsible for reconnecting and re-sending its state from scratch.

CommitUpdates

Commits pending changes to node tree associated with the view using UpdateSemanticNodes and DeleteSemanticNodes. Updates are processed in the order in which they are received. If the committed updates result in an ill-formed tree (for example a missing root node or a cycle) the semantic manager will close the channel.

Request

NameType

Response

NameType

DeleteSemanticNodes

Tells the root to remove nodes with node_ids from the semantic tree on the next commit.

Request

NameType
node_ids vector<uint32>[2048]

UpdateSemanticNodes

Sends new/updated nodes to the root to add to the cache on the next commit.

Request

NameType
nodes vector<Node>[2048]

SemanticsManager

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

An interface to manage connections with views for the purposes of gathering semantic information about their current UI state.

The manager allows clients to register as a semantic provider for their view(s). In return the semantics manager supplies an interface to update, commit and delete information from the semantic tree for that view. If the semantic manager encounters an error, it will close the channel, delete any associated data and rely on the client to re-register.

RegisterViewForSemantics

Request

NameType
view_ref fuchsia.ui.views/ViewRef
listener SemanticListener
semantic_tree_request request<SemanticTree>

STRUCTS

ENUMS

Action

Type: uint32

Defined in fuchsia.accessibility.semantics/node.fidl

Represents actions that can be applied to Nodes.

NameValueDescription
DEFAULT 1

The default action associated with the element.

SECONDARY 2

The secondary action associated with the element. This may correspond to a long press (touchscreens) or right click (mouse).

SET_FOCUS 3

Set (input/non-accessibility) focus on this element.

SET_VALUE 4

Set the element's value.

SHOW_ON_SCREEN 5

Scroll node to make it visible.

DECREMENT 6

Decrement a slider by one step value. The step size is defined by the owner of the semantic node that executes this action. For example, a volume slider may want to make steps 1, 2, 3, while a time slider may want to skip 30 seconds each step change.

INCREMENT 7

Increment a slider by one step value. The step size is defined by the owner of the semantic node that executes this action. For example, a volume slider may want to make steps 1, 2, 3, while a time slider may want to skip 30 seconds each step change.

CheckedState

Type: uint32

Defined in fuchsia.accessibility.semantics/node.fidl

Represents the state of a UI checkbox.

NameValueDescription
NONE 1

Used when no data is entered or the element is not a check box.

CHECKED 2

Checked

UNCHECKED 3

Unchecked

MIXED 4

Indeterminate state

Role

Type: uint32

Defined in fuchsia.accessibility.semantics/node.fidl

Represents a role of an element on a UI.

NameValueDescription
UNKNOWN 1

Role used to represent elements which role is not currently supported.

BUTTON 2

Something on screen that can be clicked/activated, that has a single function.

HEADER 3

Header text, e.g. something tagged

in HTML.

IMAGE 4

An image or graphic.

TEXT_FIELD 5

A field containing text that is not a header.

SLIDER 6

A slider, e.g. a volume slider or a time slider of a video.

CHECK_BOX 8

A check box that can be toggled.

RADIO_BUTTON 9

A radio button that selects an option among a group of options.

LIST 10

Role used to represent lists.

LIST_ELEMENT_MARKER 12

Role used to represent a list marker (e.g. a bullet point, number, roman numeral, etc.).

STATIC_TEXT 13

Role used to represent immutable text.

TABLES

Attributes

Defined in fuchsia.accessibility.semantics/node.fidl

An attribute is an essential property to describe an element. Unlike states, attributes do not change over the life of an element. Example: A button with a label attribute 'ok' should never change to 'cancel', as this is not the same element.

OrdinalNameTypeDescription
1 label string[16384]

The primary label for an element. If longer than MAX_LABEL_SIZE the client is responsible for truncating the label.

2 secondary_label string[16384]

The secondary label for an element. If longer than MAX_LABEL_SIZE the client is responsible for truncating the label.

3 secondary_action_description string[16384]

A description of what the secondary action on a node (equivalent to long press or right click) should do.

4 range RangeAttributes

The range attributes are filled if the element is a slider / a range control.

5 set SetAttributes

Set attributes are filled if the element is part of some type of set. For example, radio buttons that are related are part of the same set.

6 list_attributes SetAttributes

The list attributes are filled iff the node is a list. Note that only the size and set_element_ids may be filled for a list node.

7 list_element_attributes SetAttributes

The list element attributes are filled iff the node is a child of a list node.

Hit

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

Results of hit testing on a view's semantic tree which is implemented by Runtimes(like Flutter/Chrome) and sent to Accessibility.

OrdinalNameTypeDescription
1 node_id uint32

Unique ID that represents a node in a particular UI. Zero is assumed to be the root node and the only entry point to the tree. node_id will not be filled when there is no hit.

2 path_from_root vector<uint32>[256]

The ordered list of node ids which represent path from root node to the hit node.

Node

Defined in fuchsia.accessibility.semantics/node.fidl

The Node represents a semantic element on an interface. This may be a button, a text field, a checkbox or any element that has a relevant semantic meaning so that assistive technology can understand the current UI.

OrdinalNameTypeDescription
1 node_id uint32

Unique ID that represents a node in a particular UI. Zero is assumed to be the root node and the only entry point to the tree. No forest is allowed.

2 role Role

Role of this element, e.g. button, checkbox, etc.

3 states States

A table of states of this object, e.g. checked, editable, etc.

4 attributes Attributes

A table of attributes of this node.

5 actions vector<Action>[100]

A list of actions that can be performed on this node.

6 child_ids vector<uint32>[20000]

The list of child IDs of this node, in traversal order. Runtimes supplying semantic tree information are responsible for ensuring the tree does not contain cycles. Each node may have only one parent.

7 location fuchsia.ui.gfx/BoundingBox

Local bounding box of this element.

8 transform fuchsia.ui.gfx/mat4

Transform from this node's coordinate space to its parent's space. 4x4 for compatibility with scenic. This matrix is required to have the form

[ Sx 0 0 Tx ] [ 0 Sy 0 Ty ] [ 0 0 Sz Tz ] [ 0 0 0 1 ]

where Sx, Sy, and Sz are scale factors and Tx, Ty, Tz are the translation factors for the x, y, and z components

RangeAttributes

Defined in fuchsia.accessibility.semantics/node.fidl

Slider / range control attributes.

OrdinalNameTypeDescription
1 min_value float32

The minimum value a range control element can take.

2 max_value float32

The maximum value a range control element can take.

3 step_delta float32

The step delta the element applies when the action DECREMENT or INCREMENT are invoked.

SetAttributes

Defined in fuchsia.accessibility.semantics/node.fidl

Set attributes that control how an element is connected to others in the same set. For example, a group of radio buttons in html containing the same name attribute are part of the same set.

OrdinalNameTypeDescription
1 size uint32

Size of the set.

2 index uint32

Element index in the set, starting from one.

3 set_element_ids vector<uint32>[100]

Node ids of other elements in this set.

States

Defined in fuchsia.accessibility.semantics/node.fidl

A state is a dynamic property of an element that may change in response to user action or automated processes. Thus, they are different from attributes in an important point, which is frequency of change.

OrdinalNameTypeDescription
1 checked bool

DEPRECATED

2 checked_state CheckedState

State of a checkbox.

3 selected bool

Whether the element is currently selected.

4 hidden bool

Whether the element is currently hidden or marked invisible by the framework.

5 value string[16384]

The user-entered value of the element, if applicable. If longer than MAX_VALUE_SIZE the client is responsible for truncating.

6 range_value float32

If the element is a slider or a range control, this field contains the current value. Note that this is not the same as the value field above, as this is generated by the client and just adjusted by the user.

7 viewport_offset fuchsia.ui.gfx/vec2

CONSTANTS

NameValueTypeDescription
MAX_FAN_OUT 20000 uint64

Maximum number of children for a node in the semantic tree.

MAX_LABEL_SIZE 16384 uint64

Maximum size of a label string, in bytes.

MAX_NODES_PER_UPDATE 2048 uint64

Maximum number of semantic nodes that may be sent in a single update.

MAX_SET_ELEMENTS 100 uint32

Max number of elements in a semantic set.

MAX_TREE_DEPTH 256 uint64

Maximum depth of the semantic tree.

MAX_VALUE_SIZE 16384 uint64

Maximum size of a value string, in bytes.