fuchsia.accessibility.semantics

Added: 7

PROTOCOLS

SemanticListener

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

A semantic provider is the client-side interface that the manager can use to enable or disable semantic updates, and to ask clients to perform accessibility actions.

HitTest

Asks the semantics provider to perform hit testing and return the result.

Request

NameType
local_point fuchsia.math/PointF

Response

NameType
result Hit

OnAccessibilityActionRequested

Asks the semantics provider to perform an accessibility action on the node with node id in the front-end.

Request

NameType
node_id uint32
action Action

Response

NameType
handled bool

OnSemanticsModeChanged

Callback telling the client whether or not to send updates to the semantic tree. The semantics manager will clear all state when this is called with updates_enabled = false. When called with updates_enabled = true, the client should sent the full state of the current semantic tree.

Request

NameType
updates_enabled bool

Response

<EMPTY>

SemanticTree

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

Interface to update the semantic tree for a particular view. Nodes can be added, updated or deleted. Because the size of an update may exceed FIDL transfer limits, clients are responsible for breaking up changes into multiple update and delete calls that conform to these limits. The commit function must always be called at the end of a full update push to signal the end of an update.

The client may make several calls to UpdateSemanticNodes(...) or DeleteSemanticNodes(...) before calling CommitUpdates(), and must wait for the semantics manager to reply to the CommitUpdates() method to know whether an update has been processed. This allows the client to break up a set of changes (e.g. a re-computed semantic tree) to the semantic tree into FIDL-compatible chunks, but commit them all at once.

If the semantics manager ever receives inconsistent state from the client, such as an invalid tree or unrecognized parent node id, the server will close the channel. The client is responsible for reconnecting and re-sending its state from scratch.

CommitUpdates

Commits pending changes to node tree associated with the view using UpdateSemanticNodes and DeleteSemanticNodes. Updates are processed in the order in which they are received. If the committed updates result in an ill-formed tree (for example a missing root node or a cycle) the semantic manager will close the channel.

Request

<EMPTY>

Response

<EMPTY>

DeleteSemanticNodes

Tells the root to remove nodes with node_ids from the semantic tree on the next commit.

Request

NameType
node_ids vector<uint32>[2048]

SendSemanticEvent

Sends a Semantic Event related to this tree. This applies immediately and does not wait for CommitUpdates(). It is the caller's responsibility to fire events on nodes that have been already commited. The return callback is invoked to acknowledge that the event was received.

Request

NameType
semantic_event SemanticEvent

Response

<EMPTY>

UpdateSemanticNodes

Sends new/updated nodes to the root to add to the cache on the next commit.

Request

NameType
nodes vector<Node>[2048]

SemanticsManager

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

An interface to manage connections with views for the purposes of gathering semantic information about their current UI state.

The manager allows clients to register as a semantic provider for their view(s). In return the semantics manager supplies an interface to update, commit and delete information from the semantic tree for that view. If the semantic manager encounters an error, it will close the channel, delete any associated data and rely on the client to re-register.

RegisterViewForSemantics

Request

NameType
view_ref fuchsia.ui.views/ViewRef
listener SemanticListener
semantic_tree_request server_end<SemanticTree>

ENUMS

Action strict

Type: uint32

Defined in fuchsia.accessibility.semantics/node.fidl

Represents actions that can be applied to Nodes.

NameValueDescription
1

The default action associated with the element.

2

The secondary action associated with the element. This may correspond to a long press (touchscreens) or right click (mouse).

3

Set (input/non-accessibility) focus on this element.

4

Set the element's value.

5

Scroll node to make it visible.

6

Decrement a slider by one step value. The step size is defined by the owner of the semantic node that executes this action. For example, a volume slider may want to make steps 1, 2, 3, while a time slider may want to skip 30 seconds each step change.

7

Increment a slider by one step value. The step size is defined by the owner of the semantic node that executes this action. For example, a volume slider may want to make steps 1, 2, 3, while a time slider may want to skip 30 seconds each step change.

CheckedState strict

Type: uint32

Defined in fuchsia.accessibility.semantics/node.fidl

Represents the state of a UI checkbox.

NameValueDescription
1

Used when no data is entered or the element is not a check box.

2

Checked

3

Unchecked

4

Indeterminate state

EnabledState strict

Type: uint32

Defined in fuchsia.accessibility.semantics/node.fidl

Represents the enabled/disabled state of an element.

Added: 15

NameValueDescription
1

Element is enabled.

2

Element is disabled.

3

Indeterminate state.

LabelOrigin strict

Type: uint32

Defined in fuchsia.accessibility.semantics/node.fidl

The possible origins of a label.

NameValueDescription
1

The node does not have a label yet.

2

An attribute of this element in the runtime UI explicitly sets its label. For example, an element with aria-label.

3

An attribute of this element in the runtime UI explicitly sets its label to be empty.

4

This label serves as the caption for a table-like element. Because some assistive technology have special modes to deal with tables, a runtime should mark its label with this type of origin so that users know what the table is about when they navigate in the table mode. For example, a <caption> html tag sets the label.

5

This node receives its label from the contents present in the application. For example, text from a web page becomes the label of this node.

6

An element that can receive user entered value and has a suggested input. For example, the placeholder attribute on an html input field.

7

Another element provides the label for this element. For example, via aria-labeledby.

8

This element's label is represented by a visual tooltip. For example, as if from a <title> html tag.

9

The label comes from an user-entered value.

Role strict

Type: uint32

Defined in fuchsia.accessibility.semantics/node.fidl

Represents a role of an element on a UI.

NameValueDescription
1

Role used to represent elements which role is not currently supported.

2

Something on screen that can be clicked/activated, that has a single function.

3

Header text, e.g. something tagged <h1> in HTML.

4

An image or graphic.

5

A field that can receive text input.

6

A slider, e.g. a volume slider or a time slider of a video.

8

A check box that can be toggled.

9

A radio button that selects an option among a group of options.

10

Role used to represent lists.

11

Role used to represent an element in a list, including its marker and its content. For example, a element tagged <li>, or an element with the ARIA listitem role.

12

Role used to represent a list marker (e.g. a bullet point, number, roman numeral, etc.).

13

Role used to represent immutable text.

14

Role used to represent toggle switch.

15

Role used to represent a table of elements.

16

Role used to represent a grid. For example, an element with the aria-grid role.

17

Role used to represent the row of a table.

18

Role used to represent the cell of a table or grid.

19

Role used to represent a column header. For example, an element with the html tag <th>.

20

Role used to represent an aria-rowgroup-like element.

21

Role used to represent a paragraph of text.

22
23
24

ToggledState strict

Type: uint32

Defined in fuchsia.accessibility.semantics/node.fidl

Represents the state of a UI toggle switch.

NameValueDescription
1

Toggle switch is on.

2

Toggle switch is off.

3

Toggle switch is in Indeterminate state.

TABLES

AnnounceEvent

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

An announce event is one where a message is sent to a Semantic Tree, informing a change of some global state in the application. Because this announcement can be disruptive to assistive technology, it should be used sparingly.

OrdinalFieldTypeDescription
message string[16384]

A message describing the change in the application. This message is expected to be localized in the language that the application is in.

Attributes

Defined in fuchsia.accessibility.semantics/node.fidl

An attribute is an essential property to describe an element. Unlike states, attributes do not change over the life of an element. Example: A button with a label attribute 'ok' should never change to 'cancel', as this is not the same element.

OrdinalFieldTypeDescription
label string[16384]

The primary label for an element. If longer than MAX_LABEL_SIZE the client is responsible for truncating the label.

secondary_label string[16384]

The secondary label for an element. If longer than MAX_LABEL_SIZE the client is responsible for truncating the label.

secondary_action_description string[16384]

A description of what the secondary action on a node (equivalent to long press or right click) should do.

range RangeAttributes

The range attributes are filled if the element is a slider / a range control.

set SetAttributes

Set attributes are filled if the element is part of some type of set. For example, radio buttons that are related are part of the same set.

list_attributes SetAttributes

The list attributes should be filled if the node has the LIST role. Note that only size and set_element_ids should be filled.

list_element_attributes SetAttributes

The list element attributes should be filled if the node has a LIST_ELEMENT role and descends from a node with a LIST role. Note that only index should be filled.

hierarchical_level uint32

The hierarchical level of an element. For example, a header can be of level 1 to 6 in html or markdown.

table_attributes TableAttributes

The table attributes are filled when the element is a table.

label_origin LabelOrigin

The origin of the label of this element.

is_keyboard_key bool

Whether the element is part of a virtual keyboard. For example, a key on an onscreen keyboard.

table_row_attributes TableRowAttributes

The table row attributes are filled when the element is a table row.

table_cell_attributes TableCellAttributes

The table cell attributes are filled when the element is a table cell.

Hit

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

Results of hit testing on a view's semantic tree which is implemented by Runtimes(like Flutter/Chrome) and sent to Accessibility.

OrdinalFieldTypeDescription
node_id uint32

Unique ID that represents a node in a particular UI. Zero is assumed to be the root node and the only entry point to the tree. node_id will not be filled when there is no hit.

path_from_root vector<uint32>[256]

The ordered list of node ids which represent path from root node to the hit node.

Node

Defined in fuchsia.accessibility.semantics/node.fidl

The Node represents a semantic element on an interface. This may be a button, a text field, a checkbox or any element that has a relevant semantic meaning so that assistive technology can understand the current UI.

OrdinalFieldTypeDescription
node_id uint32

Unique ID that represents a node in a particular UI. Zero is assumed to be the root node and the only entry point to the tree. No forest is allowed.

role Role

Role of this element, e.g. button, checkbox, etc.

states States

A table of states of this object, e.g. checked, editable, etc.

attributes Attributes

A table of attributes of this node.

actions vector<Action>[100]

A list of actions that can be performed on this node.

child_ids vector<uint32>[20000]

The list of child IDs of this node, in traversal order. Runtimes supplying semantic tree information are responsible for ensuring the tree does not contain cycles. Each node may have only one parent.

location fuchsia.ui.gfx/BoundingBox

Local bounding box of this element.

transform fuchsia.ui.gfx/mat4

Transform from this node's coordinate space to its container's space. 4x4 for compatibility with scenic. This matrix is required to have the form

[ Sx 0 0 Tx ] [ 0 Sy 0 Ty ] [ 0 0 Sz Tz ] [ 0 0 0 1 ]

where Sx, Sy, and Sz are scale factors and Tx, Ty, Tz are the translation factors for the x, y, and z components.

container_id uint32

Node ID of the ancestor of this node that is used as the target of |transform|. If not present, this is interpreted as this node's parent.

node_to_container_transform fuchsia.ui.gfx/mat4

NOTE: We are deprecating transform in favor of node_to_container_transform. Clients must NOT set both fields.

Transform from this node's coordinate space to its container's space. If the container_id field is set, then that id specifies the container node. Otherwise, this node's parent is the container node.

Additionally, if container_id is set, the fuchsia accessibility manager will post-translate this node's coordinate space by its container's location.min after applying node_to_container_transform when converting to the container's coordinate space. If container_id is unset, this post-translation will NOT be applied.

4x4 for compatibility with scenic. This matrix is required to have the form:

[ Sx 0 0 Tx ] [ 0 Sy 0 Ty ] [ 0 0 Sz Tz ] [ 0 0 0 1 ]

where Sx, Sy, and Sz are scale factors and Tx, Ty, Tz are the translation factors for the x, y, and z components.

RangeAttributes

Defined in fuchsia.accessibility.semantics/node.fidl

Slider / range control attributes.

OrdinalFieldTypeDescription
min_value float32

The minimum value a range control element can take.

max_value float32

The maximum value a range control element can take.

step_delta float32

The step delta the element applies when the action DECREMENT or INCREMENT are invoked.

SetAttributes

Defined in fuchsia.accessibility.semantics/node.fidl

Set attributes that control how an element is connected to others in the same set. For example, a group of radio buttons in html containing the same name attribute are part of the same set.

OrdinalFieldTypeDescription
size uint32

Size of the set.

index uint32

Element index in the set, starting from one.

set_element_ids vector<uint32>[100]

Node ids of other elements in this set.

States

Defined in fuchsia.accessibility.semantics/node.fidl

A state is a dynamic property of an element that may change in response to user action or automated processes. Thus, they are different from attributes in an important point, which is frequency of change.

OrdinalFieldTypeDescription
checked bool

DEPRECATED

checked_state CheckedState

State of a checkbox. This field is mutually exclusive with ToggledState.

selected bool

Whether the element is currently selected.

hidden bool

Whether the element is currently hidden or marked invisible by the framework.

value string[16384]

The user-entered value of the element, if applicable. If longer than MAX_VALUE_SIZE the client is responsible for truncating.

range_value float32

If the element is a slider or a range control, this field contains the current value. Note that this is not the same as the value field above, as this is generated by the client and just adjusted by the user.

viewport_offset fuchsia.ui.gfx/vec2

If the element is a scrollable viewport, this field contains the x and y offsets within this node's coordinate space to apply to the children. This offsetting is used to position the children within the viewport to reflect the current scrolling of the element. There are no constraints on these values other than they must be finite.

toggled_state ToggledState

State of a toggle switch. This field must only be set if the element is a toggle switch element and is mutually exclusive with CheckedState.

focusable bool

Whether this element is focusable in the UI. Note that this is not the a11y focus, but the input focus.

has_input_focus bool

Whether this element has the input focus. This corresponds to the system focus. Only one element can have this value set. The behavior is undetermined if more than one node sets this value to 'true'.

enabled_state EnabledState

State of an element with enabled/disabled state.

Added: 15

TableAttributes

Defined in fuchsia.accessibility.semantics/node.fidl

Attributes that control how a table cell node is interpreted.

OrdinalFieldTypeDescription
column_span uint32

The number of columns this table cell spans.

DEPRECATED - Use TableCellAttributes for cell nodes instead

row_span uint32

The number of rows this table cell spans.

DEPRECATED - Use TableCellAttributes for cell nodes instead

number_of_rows uint32

Number of rows in this table.

number_of_columns uint32

Number of columns in this table.

column_header_ids vector<uint32>[100]

Node ids of the table's column headers.

row_header_ids vector<uint32>[100]

Node ids of the table's row headers.

TableCellAttributes

Defined in fuchsia.accessibility.semantics/node.fidl

Attributes that control how a table cell is interpreted.

OrdinalFieldTypeDescription
row_index uint32

Row-index of this cell.

column_index uint32

Column-index of this cell.

row_span uint32

The number of rows this table cell spans.

column_span uint32

The number of columns this table cell spans.

TableRowAttributes

Defined in fuchsia.accessibility.semantics/node.fidl

Attributes that control how a table row node is interpreted.

OrdinalFieldTypeDescription
row_index uint32

Table row index.

UNIONS

SemanticEvent strict

Defined in fuchsia.accessibility.semantics/semantics_manager.fidl

Represents a Semantic Event that is fired on a Semantic Tree.

OrdinalVariantTypeDescription
announce AnnounceEvent

CONSTANTS

NameValueTypeDescription
MAX_FAN_OUT 20000 uint64

Maximum number of children for a node in the semantic tree.

MAX_LABEL_SIZE 16384 uint64

Maximum size of a label string, in bytes.

MAX_MESSAGE_SIZE 16384 uint64

Maximum size of a message string, in bytes.

MAX_NODES_PER_UPDATE 2048 uint64

Maximum number of semantic nodes that may be sent in a single update.

MAX_SET_ELEMENTS 100 uint32

Max number of elements in a semantic set.

MAX_TREE_DEPTH 256 uint64

Maximum depth of the semantic tree.

MAX_VALUE_SIZE 16384 uint64

Maximum size of a value string, in bytes.