PROTOCOLS
SemanticListener
Defined in fuchsia.accessibility.semantics/semantics_manager.fidl
A semantic provider is the client-side interface that the manager can use to enable or disable semantic updates, and to ask clients to perform accessibility actions.
HitTest
Asks the semantics provider to perform hit testing and return the result.
Request
Name | Type |
---|---|
local_point |
fuchsia.math/PointF
|
Response
Name | Type |
---|---|
result |
Hit
|
OnAccessibilityActionRequested
Asks the semantics provider to perform an accessibility action on the node with node id in the front-end.
Request
Name | Type |
---|---|
node_id |
uint32
|
action |
Action
|
Response
Name | Type |
---|---|
handled |
bool
|
OnSemanticsModeChanged
Callback telling the client whether or not to send updates to the semantic tree. The semantics manager will clear all state when this is called with updates_enabled = false. When called with updates_enabled = true, the client should sent the full state of the current semantic tree.
Request
Name | Type |
---|---|
updates_enabled |
bool
|
Response
<EMPTY>
SemanticTree
Defined in fuchsia.accessibility.semantics/semantics_manager.fidl
Interface to update the semantic tree for a particular view. Nodes can be added, updated or deleted. Because the size of an update may exceed FIDL transfer limits, clients are responsible for breaking up changes into multiple update and delete calls that conform to these limits. The commit function must always be called at the end of a full update push to signal the end of an update.
The client may make several calls to UpdateSemanticNodes(...) or DeleteSemanticNodes(...) before calling CommitUpdates(), and must wait for the semantics manager to reply to the CommitUpdates() method to know whether an update has been processed. This allows the client to break up a set of changes (e.g. a re-computed semantic tree) to the semantic tree into FIDL-compatible chunks, but commit them all at once.
If the semantics manager ever receives inconsistent state from the client, such as an invalid tree or unrecognized parent node id, the server will close the channel. The client is responsible for reconnecting and re-sending its state from scratch.
CommitUpdates
Commits pending changes to node tree associated with the view using UpdateSemanticNodes and DeleteSemanticNodes. Updates are processed in the order in which they are received. If the committed updates result in an ill-formed tree (for example a missing root node or a cycle) the semantic manager will close the channel.
Request
<EMPTY>
Response
<EMPTY>
DeleteSemanticNodes
Tells the root to remove nodes with node_ids from the semantic tree on the next commit.
Request
Name | Type |
---|---|
node_ids |
vector<uint32>:2048
|
SendSemanticEvent
Sends a Semantic Event related to this tree. This applies immediately and does not wait for CommitUpdates(). It is the caller's responsibility to fire events on nodes that have been already commited. The return callback is invoked to acknowledge that the event was received.
Request
Name | Type |
---|---|
semantic_event |
SemanticEvent
|
Response
<EMPTY>
UpdateSemanticNodes
Sends new/updated nodes to the root to add to the cache on the next commit.
Request
Name | Type |
---|---|
nodes |
vector<Node>:2048
|
SemanticsManager
Defined in fuchsia.accessibility.semantics/semantics_manager.fidl
An interface to manage connections with views for the purposes of gathering semantic information about their current UI state.
The manager allows clients to register as a semantic provider for their view(s). In return the semantics manager supplies an interface to update, commit and delete information from the semantic tree for that view. If the semantic manager encounters an error, it will close the channel, delete any associated data and rely on the client to re-register.
RegisterViewForSemantics
Request
Name | Type |
---|---|
view_ref |
fuchsia.ui.views/ViewRef
|
listener |
client_end:SemanticListener
|
semantic_tree_request |
server_end:SemanticTree
|
ENUMS
Action strict
Type: uint32
Defined in fuchsia.accessibility.semantics/node.fidl
Represents actions that can be applied to Nodes.
Name | Value | Description |
---|---|---|
DEFAULT |
1 |
The default action associated with the element. |
SECONDARY |
2 |
The secondary action associated with the element. This may correspond to a long press (touchscreens) or right click (mouse). |
SET_FOCUS |
3 |
Set (input/non-accessibility) focus on this element. |
SET_VALUE |
4 |
Set the element's value. |
SHOW_ON_SCREEN |
5 |
Scroll node to make it visible. |
DECREMENT |
6 |
Decrement a slider by one step value. The step size is defined by the owner of the semantic node that executes this action. For example, a volume slider may want to make steps 1, 2, 3, while a time slider may want to skip 30 seconds each step change. |
INCREMENT |
7 |
Increment a slider by one step value. The step size is defined by the owner of the semantic node that executes this action. For example, a volume slider may want to make steps 1, 2, 3, while a time slider may want to skip 30 seconds each step change. |
CheckedState strict
Type: uint32
Defined in fuchsia.accessibility.semantics/node.fidl
Represents the state of a UI checkbox.
Name | Value | Description |
---|---|---|
NONE |
1 |
Used when no data is entered or the element is not a check box. |
CHECKED |
2 |
Checked |
UNCHECKED |
3 |
Unchecked |
MIXED |
4 |
Indeterminate state |
EnabledState strict
Type: uint32
Defined in fuchsia.accessibility.semantics/node.fidl
Represents the enabled/disabled state of an element.
Name | Value | Description |
---|---|---|
ENABLED |
1 |
Element is enabled. |
DISABLED |
2 |
Element is disabled. |
INDETERMINATE |
3 |
Indeterminate state. |
LabelOrigin strict
Type: uint32
Defined in fuchsia.accessibility.semantics/node.fidl
The possible origins of a label.
Name | Value | Description |
---|---|---|
UNITIALIZED |
1 |
The node does not have a label yet. |
ATTRIBUTE |
2 |
An attribute of this element in the runtime UI explicitly sets its label. For example, an element with aria-label. |
ATTRIBUTE_EMPTY |
3 |
An attribute of this element in the runtime UI explicitly sets its label to be empty. |
CAPTION |
4 |
This label serves as the caption for a table-like element. Because some assistive technology have special modes to deal with tables, a runtime should mark its label with this type of origin so that users know what the table is about when they navigate in the table mode. For example, a <caption> html tag sets the label. |
CONTENTS |
5 |
This node receives its label from the contents present in the application. For example, text from a web page becomes the label of this node. |
PLACEHOLDER |
6 |
An element that can receive user entered value and has a suggested input. For example, the placeholder attribute on an html input field. |
RELATED_ELEMENT |
7 |
Another element provides the label for this element. For example, via aria-labeledby. |
TITLE |
8 |
This element's label is represented by a visual tooltip. For example, as if from a <title> html tag. |
VALUE |
9 |
The label comes from an user-entered value. |
Role strict
Type: uint32
Defined in fuchsia.accessibility.semantics/node.fidl
Represents a role of an element on a UI.
Name | Value | Description |
---|---|---|
UNKNOWN |
1 |
Role used to represent elements which role is not currently supported. |
BUTTON |
2 |
Something on screen that can be clicked/activated, that has a single function. |
HEADER |
3 |
Header text, e.g. something tagged <h1> in HTML. |
IMAGE |
4 |
An image or graphic. |
TEXT_FIELD |
5 |
A field that can receive text input. |
SLIDER |
6 |
A slider, e.g. a volume slider or a time slider of a video. |
LINK |
7 |
A link, e.g. a link on a webpage. |
CHECK_BOX |
8 |
A check box that can be toggled. |
RADIO_BUTTON |
9 |
A radio button that selects an option among a group of options. |
LIST |
10 |
Role used to represent lists. |
LIST_ELEMENT |
11 |
Role used to represent an element in a list, including its marker and its content. For example, a element tagged <li>, or an element with the ARIA listitem role. |
LIST_ELEMENT_MARKER |
12 |
Role used to represent a list marker (e.g. a bullet point, number, roman numeral, etc.). |
STATIC_TEXT |
13 |
Role used to represent immutable text. |
TOGGLE_SWITCH |
14 |
Role used to represent toggle switch. |
TABLE |
15 |
Role used to represent a table of elements. |
GRID |
16 |
Role used to represent a grid. For example, an element with the aria-grid role. |
TABLE_ROW |
17 |
Role used to represent the row of a table. |
CELL |
18 |
Role used to represent the cell of a table or grid. |
COLUMN_HEADER |
19 |
Role used to represent a column header. For example, an element with the html tag <th>. |
ROW_GROUP |
20 |
Role used to represent an aria-rowgroup-like element. |
PARAGRAPH |
21 |
Role used to represent a paragraph of text. |
SEARCH_BOX |
22 |
|
TEXT_FIELD_WITH_COMBO_BOX |
23 |
|
ROW_HEADER |
24 |
ToggledState strict
Type: uint32
Defined in fuchsia.accessibility.semantics/node.fidl
Represents the state of a UI toggle switch.
Name | Value | Description |
---|---|---|
ON |
1 |
Toggle switch is on. |
OFF |
2 |
Toggle switch is off. |
INDETERMINATE |
3 |
Toggle switch is in Indeterminate state. |
TABLES
AnnounceEvent
Defined in fuchsia.accessibility.semantics/semantics_manager.fidl
An announce event is one where a message is sent to a Semantic Tree, informing a change of some global state in the application. Because this announcement can be disruptive to assistive technology, it should be used sparingly.
Ordinal | Field | Type | Description |
---|---|---|---|
1 |
message |
string:16384
|
A message describing the change in the application. This message is expected to be localized in the language that the application is in. |
Attributes
Defined in fuchsia.accessibility.semantics/node.fidl
An attribute is an essential property to describe an element. Unlike states, attributes do not change over the life of an element. Example: A button with a label attribute 'ok' should never change to 'cancel', as this is not the same element.
Ordinal | Field | Type | Description |
---|---|---|---|
1 |
label |
string:16384
|
The primary label for an element. If longer than MAX_LABEL_SIZE the client is responsible for truncating the label. |
2 |
secondary_label |
string:16384
|
The secondary label for an element. If longer than MAX_LABEL_SIZE the client is responsible for truncating the label. |
3 |
secondary_action_description |
string:16384
|
A description of what the secondary action on a node (equivalent to long press or right click) should do. |
4 |
range |
RangeAttributes
|
The range attributes are filled if the element is a slider / a range control. |
5 |
set |
SetAttributes
|
Set attributes are filled if the element is part of some type of set. For example, radio buttons that are related are part of the same set. |
6 |
list_attributes |
SetAttributes
|
The list attributes should be filled if the node has the LIST role.
Note that only |
7 |
list_element_attributes |
SetAttributes
|
The list element attributes should be filled if the node has a
LIST_ELEMENT role and descends from a node with a LIST role.
Note that only |
8 |
hierarchical_level |
uint32
|
The hierarchical level of an element. For example, a header can be of level 1 to 6 in html or markdown. |
9 |
table_attributes |
TableAttributes
|
The table attributes are filled when the element is a table. |
10 |
label_origin |
LabelOrigin
|
The origin of the label of this element. |
11 |
is_keyboard_key |
bool
|
Whether the element is part of a virtual keyboard. For example, a key on an onscreen keyboard. |
12 |
table_row_attributes |
TableRowAttributes
|
The table row attributes are filled when the element is a table row. |
13 |
table_cell_attributes |
TableCellAttributes
|
The table cell attributes are filled when the element is a table cell. |
Hit
Defined in fuchsia.accessibility.semantics/semantics_manager.fidl
Results of hit testing on a view's semantic tree which is implemented by Runtimes(like Flutter/Chrome) and sent to Accessibility.
Ordinal | Field | Type | Description |
---|---|---|---|
1 |
node_id |
uint32
|
Unique ID that represents a node in a particular UI. Zero is assumed to be the root node and the only entry point to the tree. node_id will not be filled when there is no hit. |
2 |
path_from_root |
vector<uint32>:256
|
The ordered list of node ids which represent path from root node to the hit node. |
Node
Defined in fuchsia.accessibility.semantics/node.fidl
The Node represents a semantic element on an interface. This may be a button, a text field, a checkbox or any element that has a relevant semantic meaning so that assistive technology can understand the current UI.
Ordinal | Field | Type | Description |
---|---|---|---|
1 |
node_id |
uint32
|
Unique ID that represents a node in a particular UI. Zero is assumed to be the root node and the only entry point to the tree. No forest is allowed. |
2 |
role |
Role
|
Role of this element, e.g. button, checkbox, etc. |
3 |
states |
States
|
A table of states of this object, e.g. checked, editable, etc. |
4 |
attributes |
Attributes
|
A table of attributes of this node. |
5 |
actions |
vector<Action>:100
|
A list of actions that can be performed on this node. |
6 |
child_ids |
vector<uint32>:20000
|
The list of child IDs of this node, in traversal order. Runtimes supplying semantic tree information are responsible for ensuring the tree does not contain cycles. Each node may have only one parent. |
7 |
location |
fuchsia.ui.gfx/BoundingBox
|
Local bounding box of this element. |
8 |
transform |
fuchsia.ui.gfx/mat4
|
Transform from this node's coordinate space to its container's space. 4x4 for compatibility with scenic. This matrix is required to have the form [ Sx 0 0 Tx ] [ 0 Sy 0 Ty ] [ 0 0 Sz Tz ] [ 0 0 0 1 ] where Sx, Sy, and Sz are scale factors and Tx, Ty, Tz are the translation factors for the x, y, and z components. |
9 |
container_id |
uint32
|
Node ID of the ancestor of this node that is used as the target of |transform|. If not present, this is interpreted as this node's parent. |
10 |
node_to_container_transform |
fuchsia.ui.gfx/mat4
|
NOTE: We are deprecating Transform from this node's coordinate space to its container's space. If the Additionally, if 4x4 for compatibility with scenic. This matrix is required to have the form: [ Sx 0 0 Tx ] [ 0 Sy 0 Ty ] [ 0 0 Sz Tz ] [ 0 0 0 1 ] where Sx, Sy, and Sz are scale factors and Tx, Ty, Tz are the translation factors for the x, y, and z components. |
RangeAttributes
Defined in fuchsia.accessibility.semantics/node.fidl
Slider / range control attributes.
Ordinal | Field | Type | Description |
---|---|---|---|
1 |
min_value |
float32
|
The minimum value a range control element can take. |
2 |
max_value |
float32
|
The maximum value a range control element can take. |
3 |
step_delta |
float32
|
The step delta the element applies when the action DECREMENT or INCREMENT are invoked. |
SetAttributes
Defined in fuchsia.accessibility.semantics/node.fidl
Set attributes that control how an element is connected to others in the same set. For example, a group of radio buttons in html containing the same name attribute are part of the same set.
Ordinal | Field | Type | Description |
---|---|---|---|
1 |
size |
uint32
|
Size of the set. |
2 |
index |
uint32
|
Element index in the set, starting from one. |
3 |
set_element_ids |
vector<uint32>:100
|
Node ids of other elements in this set. |
States
Defined in fuchsia.accessibility.semantics/node.fidl
A state is a dynamic property of an element that may change in response to user action or automated processes. Thus, they are different from attributes in an important point, which is frequency of change.
Ordinal | Field | Type | Description |
---|---|---|---|
1 |
checked |
bool
|
DEPRECATED |
2 |
checked_state |
CheckedState
|
State of a checkbox. This field is mutually exclusive with ToggledState. |
3 |
selected |
bool
|
Whether the element is currently selected. |
5 |
value |
string:16384
|
The user-entered value of the element, if applicable. If longer than MAX_VALUE_SIZE the client is responsible for truncating. |
6 |
range_value |
float32
|
If the element is a slider or a range control, this field contains the current value. Note that this is not the same as the value field above, as this is generated by the client and just adjusted by the user. |
7 |
viewport_offset |
fuchsia.ui.gfx/vec2
|
If the element is a scrollable viewport, this field contains the x and y offsets within this node's coordinate space to apply to the children. This offsetting is used to position the children within the viewport to reflect the current scrolling of the element. There are no constraints on these values other than they must be finite. |
8 |
toggled_state |
ToggledState
|
State of a toggle switch. This field must only be set if the element is a toggle switch element and is mutually exclusive with CheckedState. |
9 |
focusable |
bool
|
Whether this element is focusable in the UI. Note that this is not the a11y focus, but the input focus. |
10 |
has_input_focus |
bool
|
Whether this element has the input focus. This corresponds to the system focus. Only one element can have this value set. The behavior is undetermined if more than one node sets this value to 'true'. |
11 |
enabled_state |
EnabledState
|
State of an element with enabled/disabled state. Added: 15
|
TableAttributes
Defined in fuchsia.accessibility.semantics/node.fidl
Attributes that control how a table cell node is interpreted.
Ordinal | Field | Type | Description |
---|---|---|---|
1 |
column_span |
uint32
|
The number of columns this table cell spans. DEPRECATED - Use TableCellAttributes for cell nodes instead |
2 |
row_span |
uint32
|
The number of rows this table cell spans. DEPRECATED - Use TableCellAttributes for cell nodes instead |
3 |
number_of_rows |
uint32
|
Number of rows in this table. |
4 |
number_of_columns |
uint32
|
Number of columns in this table. |
5 |
column_header_ids |
vector<uint32>:100
|
Node ids of the table's column headers. |
6 |
row_header_ids |
vector<uint32>:100
|
Node ids of the table's row headers. |
TableCellAttributes
Defined in fuchsia.accessibility.semantics/node.fidl
Attributes that control how a table cell is interpreted.
Ordinal | Field | Type | Description |
---|---|---|---|
1 |
row_index |
uint32
|
Row-index of this cell. |
2 |
column_index |
uint32
|
Column-index of this cell. |
3 |
row_span |
uint32
|
The number of rows this table cell spans. |
4 |
column_span |
uint32
|
The number of columns this table cell spans. |
TableRowAttributes
Defined in fuchsia.accessibility.semantics/node.fidl
Attributes that control how a table row node is interpreted.
Ordinal | Field | Type | Description |
---|---|---|---|
1 |
row_index |
uint32
|
Table row index. |
UNIONS
SemanticEvent strict
Defined in fuchsia.accessibility.semantics/semantics_manager.fidl
Represents a Semantic Event that is fired on a Semantic Tree.
Ordinal | Variant | Type | Description |
---|---|---|---|
1 |
announce |
AnnounceEvent
|
CONSTANTS
Name | Value | Type | Description |
---|---|---|---|
MAX_FAN_OUT |
20000
|
uint64 |
Maximum number of children for a node in the semantic tree. |
MAX_LABEL_SIZE |
16384
|
uint64 |
Maximum size of a label string, in bytes. |
MAX_MESSAGE_SIZE |
16384
|
uint64 |
Maximum size of a message string, in bytes. |
MAX_NODES_PER_UPDATE |
2048
|
uint64 |
Maximum number of semantic nodes that may be sent in a single update. |
MAX_SET_ELEMENTS |
100
|
uint32 |
Max number of elements in a semantic set. |
MAX_TREE_DEPTH |
256
|
uint64 |
Maximum depth of the semantic tree. |
MAX_VALUE_SIZE |
16384
|
uint64 |
Maximum size of a value string, in bytes. |