PROTOCOLS
MouseSource
Defined in fuchsia.ui.pointer/mouse.fidl
A method for a client to receive mouse pointer events.
The position of a pointer event is defined in the context of a viewport, situated in the view. The dimensions of the view and viewport, and their spatial relationship (defined with a transform matrix), are supplied synchronously in a |ViewParameter| table. A view may retrieve a pointer's position in its local coordinate system by applying the viewport-to-view transform matrix.
The viewport is embedded in an independent and stable coordinate system, suitable for interpreting pointer events in a scale-independent manner; mouse movement will be observed at a constant scale, even under effects such as magnification or panning. However, other effects, such as enlargening the view's clip bounds, may trigger a change in the viewport extents.
Watch
A method for a client to receive mouse pointer events.
This call is formulated as a "hanging get" pattern: the client asks for a set of recent events, and receives them via the callback. This pull-based approach ensures that clients consume events at their own pace; events don't clog up the channel in an unbounded manner.
Flow control. The caller is allowed at most one in-flight |Watch| call at a time; it is a logical error to have concurrent calls to |Watch|. Non-compliance results in channel closure.
Client pacing. The server will dispatch events to the caller on a FIFO, lossless, best-effort basis, but the caller must allocate enough time to keep up with new events.
Event times. The timestamps on each event in the event vector are not guaranteed monotonic; events from different devices may be injected into Scenic at different times. Generally, events from a single device are expected to have monotonically increasing timestamps.
View parameters. Occasionally, changes in view or viewport require notifying the client. If a |MouseEvent| carries |ViewParameters|, these parameters apply to successive |MousePointerSample|s until the next |ViewParameters|.
Request
<EMPTY>
Response
| Name | Type |
|---|---|
events |
vector<MouseEvent>:128
|
TouchSource
Defined in fuchsia.ui.pointer/touch.fidl
A method for a client to receive touch events and respond in a global gesture disambiguation protocol.
The position of a touch event is defined in the context of a viewport, situated in the view. The dimensions of the view and viewport, and their spatial relationship (defined with a transform matrix), are supplied synchronously in a |ViewParameter| table. A view may retrieve a pointer's position in its local coordinate system by applying the viewport-to-view transform matrix.
The viewport is embedded in an independent and stable coordinate system, suitable for interpreting touch events in a scale-independent manner; a swipe will be observed at a constant scale, even under effects such as magnification or panning. However, other effects, such as enlargening the view's clip bounds, may trigger a change in the viewport extents.
UpdateResponse
The gesture protocol allows a client to enact a "hold" on an open interaction of touch events; it prevents resolution of interaction ownership, even after the interaction closes. This method updates the client's previous "hold" by replacing it with a response that allows ownership resolution to proceed.
See |TouchInteractionId| for how a stream is structured into interactions.
Flow control. The caller is allowed at most one |UpdateResponse| call per interaction, and it must be on a closed interaction. It is a logical error to call |UpdateResponse| when a normal response is possible with the |Watch| call.
Validity. This TouchResponse must not be another "hold" response, and the overwritten response is expected to be a "hold" response.
Request
| Name | Type |
|---|---|
interaction |
TouchInteractionId
|
response |
TouchResponse
|
Response
<EMPTY>
Watch
A method for a client to receive touch pointer events.
This call is formulated as a "hanging get" pattern: the client asks for a set of recent events, and receives them via the callback. This pull-based approach ensures that clients consume events at their own pace; events don't clog up the channel in an unbounded manner.
Flow control. The caller is allowed at most one in-flight |Watch| call at a time; it is a logical error to have concurrent calls to |Watch|. Non-compliance results in channel closure.
Client pacing. The server will dispatch events to the caller on a FIFO, lossless, best-effort basis, but the caller must allocate enough time to keep up with new events. An unresponsive client may be categorized as "App Not Responding" and targeted for channel closure.
Responses. The gesture disambiguation scheme relies on the server
receiving a |TouchResponse| for each |TouchEvent|.|TouchPointerSample|;
non-sample events should return an empty |TouchResponse| table to the
server. Responses for previous events are fed to the server on the
next call of |Watch| [1]. Each element in the |responses| vector is
interpreted as the pairwise response to the event in the previous
|events| vector; the vector lengths must match. Note that the client's
contract to respond to events starts as soon as it registers its
endpoint with scenic, NOT when it first calls Watch().
Initial response. The first call to |Watch| must be an empty vector.
Event times. The timestamps on each event in the event vector are not guaranteed monotonic; touch events from different devices may be injected into Scenic at different times. Generally, events from a single device are expected to have monotonically increasing timestamps.
View parameters. Occasionally, changes in view or viewport require notifying the client. If a |TouchEvent| carries |ViewParameters|, these parameters apply to successive |TouchPointerSample|s until the next |ViewParameters|.
[1] The hanging get pattern enables straightforward API evolution, but unfortunately does not admit an idiomatic matching of response to event.
Request
| Name | Type |
|---|---|
responses |
vector<TouchResponse>:128
|
Response
| Name | Type |
|---|---|
events |
vector<TouchEvent>:128
|
STRUCTS
MouseEventStreamInfo
Defined in fuchsia.ui.pointer/mouse.fidl
The status of a mouse event stream, sent from server to client.
Invariant: a client's mouse events are bracketed by MouseViewStatus.ENTERED and MouseViewStatus.EXITED.
| Field | Type | Description | Default |
|---|---|---|---|
device_id |
uint32
|
An identifier for the mouse device that issues a mouse event stream. |
No default |
status |
MouseViewStatus
|
The mouse event stream's enter/exit status, sent from server to client. |
No default |
Rectangle
Defined in fuchsia.ui.pointer/view.fidl
An axis-aligned rectangle. It is defined by its minimal and maximal extents in a coordinate system.
| Field | Type | Description | Default |
|---|---|---|---|
min |
Point2
|
The minimal extent of this rectangle, inclusive.
|
No default |
max |
Point2
|
The maximal extent of this rectangle, inclusive.
|
No default |
TouchInteractionId
Defined in fuchsia.ui.pointer/touch.fidl
A unique identifier for a "interaction" of touch events in an event stream. Touch events are observed as a succession of interactions, as fingers engage and disengage with the display.
A finite sequence of pointer events that follows the EventPhase state
machine, starting from the initial state ADD, is called an interaction.
A closed (or past) interaction is one where it has reached the terminal
state (REMOVE or CANCEL); an open (or current) interaction is one where it
has not.
For a given device pointer, a stream of events is observed as a succession of zero or more closed interactions (the past history of user engagement), followed by at most one open interaction (the current user engagement).
Because we need to group pointer events by their interaction, touch event carries an interaction id that is unique in that pointer stream. This common reference makes it possible to operate on a closed interaction, as well as an open interaction.
Also see EventPhase for a discussion on event streams by mice.
| Field | Type | Description | Default |
|---|---|---|---|
device_id |
uint32
|
An identifier for the pointer device that issues touch event streams. A device may own multiple pointers, each with its own |pointer_id|. |
No default |
pointer_id |
uint32
|
An identifier of the pointer that issued this event. It is unique only to a specific |device_id|. Each (device_id, pointer_id) pair issues at most one open interaction at a time. |
No default |
interaction_id |
uint32
|
An identifier of the interaction. It is unique only to a specific (device_id, pointer_id) pair. |
No default |
TouchInteractionResult
Defined in fuchsia.ui.pointer/touch.fidl
The result of gesture disambiguation for a interaction of touch events, sent from server to client.
| Field | Type | Description | Default |
|---|---|---|---|
interaction |
TouchInteractionId
|
The interaction that this pointer sample belongs to. |
No default |
status |
TouchInteractionStatus
|
The interaction's disposition, sent from server to client. |
No default |
ViewParameters
Defined in fuchsia.ui.pointer/view.fidl
The parameters of the associated view and viewport, sufficient to correctly interpret the position and scale of pointer events dispatched to this view.
Ordering. These parameters arrive over the same channel as pointer events, to provide synchronous context for interpreting the position of pointer events in the view's local coordinate system.
Inter-protocol redundancy. Some of these parameters may also be sent over an independent channel dedicated to view control; the client is responsible for correct use of asynchronously received parameters.
TODO(https://fxbug.dev/42162292): Rename viewport, it is used in Flatland.
| Field | Type | Description | Default |
|---|---|---|---|
view |
Rectangle
|
The view's area of content, placed in the coordinate system of the view. The rectangle is defined by the parent view's setup of clipping on this view. |
No default |
viewport |
Rectangle
|
The viewport's area of interaction, placed in an independent coordinate system. A pointer event's position is defined in the coordinate system of this viewport. A change in viewport extents means the region for pointer interaction has itself moved, or changed in size, or both. |
No default |
viewport_to_view_transform |
Mat3
|
The transform matrix that relates a point's position in the viewport's coordinate system to its position in the view's coordinate system. |
No default |
ENUMS
EventPhase strict
Type: uint32
Defined in fuchsia.ui.pointer/state.fidl
The possible states of a pointer event. These phases of events in a stream
follow a state machine that starts with the ADD phase, followed by zero or
more CHANGE phases, and finally terminates with REMOVE or CANCEL
phase.
ADD ---> CHANGE* -+-> REMOVE
|
+-> CANCEL
A finite sequence of pointer events that follows this state machine, starting from the initial state, is called an interaction. A closed (or past) interaction is one where it has reached the terminal state; an open (or current) interaction is one where it has not.
For a given device pointer, a stream of events is observed as a succession of zero or more closed interactions (the past history of user engagement), followed by at most one open interaction (the current user engagement).
When we need to group pointer events by their interaction, an event carries an interaction id that is unique in that pointer stream. This common reference makes it possible to operate on a closed interaction, as well as an open interaction.
For example, touch events are typically observed as a succession of interactions, as fingers engage and disengage with the display.
| Name | Value | Description |
|---|---|---|
ADD |
1 |
The device has started tracking the pointer. |
CHANGE |
2 |
The device has reported an update to the pointer state. |
REMOVE |
3 |
The device has stopped tracking the pointer. |
CANCEL |
4 |
The pointer is no longer available. |
MouseViewStatus strict
Type: uint32
Defined in fuchsia.ui.pointer/mouse.fidl
A description of mouse event stream's relationship to this view.
| Name | Value | Description |
|---|---|---|
ENTERED |
1 |
The stream is directed towards this view. |
EXITED |
2 |
The stream is directed away from this view. |
TouchInteractionStatus strict
Type: uint32
Defined in fuchsia.ui.pointer/touch.fidl
A description of the interaction's relationship to this client.
| Name | Value | Description |
|---|---|---|
DENIED |
1 |
The client has been denied ownership of the interaction. |
GRANTED |
2 |
The client has been granted ownership of the interaction. |
TouchResponseType strict
Type: uint32
Defined in fuchsia.ui.pointer/touch.fidl
The possible interaction dispositions that a client can respond with to a given |TouchPointerSample|. Used as part of a gesture disambiguation scheme.
The responses are based on the idea of an ownership claim on a interaction. Clients may assert a claim of ownership on an open interaction, but only one client's claim is granted by the server; other clients' claims are denied.
| Name | Value | Description |
|---|---|---|
NO |
1 |
The client has no further interest in this interaction; it declines ownership of the interaction. The client will stop receiving events for this interaction. |
MAYBE |
2 |
The client is interested in this interaction, but needs to see more events to decide; the client has not yet claimed ownership of this interaction. |
MAYBE_PRIORITIZE |
3 |
The client is interested in this interaction, but needs to see more events to decide; the client has not yet claimed ownership of the interaction. During ownership resolution, it exerts its priority over lower-priority "maybe" claims, but always loses to a "yes" claim. |
MAYBE_SUPPRESS |
4 |
The client is interested in this interaction, but needs to see more events to decide; the client has not yet claimed ownership of the interaction. Moreover, it suppresses lower-priority claims that try to resolve interaction ownership. |
MAYBE_PRIORITIZE_SUPPRESS |
5 |
The client is interested in this interaction, but needs to see more events to decide; the client has not yet claimed ownership of the interaction. Moreover, it suppresses lower-priority claims that try to resolve interaction ownership. During ownership resolution, it exerts its priority over lower-priority "maybe" claims, but always loses to a "yes" claim. |
HOLD |
6 |
The client is interested in this interaction, but needs to see a subsequent interaction to decide; the client has not yet claimed ownership of this interaction. It prevents ownership resolution when the interaction closes. |
HOLD_SUPPRESS |
7 |
The client is interested in this interaction, but needs to see a subsequent interaction to decide; the client has not yet claimed ownership of this interaction. It prevents ownership resolution when the interaction closes. Moreover, it suppresses lower-priority claims that try to resolve interaction ownership. |
YES |
8 |
The client wishes exclusive access to the remaining events in this interaction; it claims ownership of this interaction (but that claim may be granted or denied). During ownership resolution, it yields its priority to lower-priority "yes" claims. |
YES_PRIORITIZE |
9 |
The client wishes exclusive access to the remaining events in this interaction; it claims ownership of this interaction (but that claim may be granted or denied). During ownership resolution, it exerts its priority over lower-priority "yes" claims. |
TABLES
MouseDeviceInfo
Defined in fuchsia.ui.pointer/mouse.fidl
Information about a device that issues a mouse event stream.
| Ordinal | Field | Type | Description |
|---|---|---|---|
1 |
id |
uint32
|
An identifier for the mouse device that issues a mouse event stream. Required. |
2 |
scroll_v_range |
fuchsia.input.report/Axis
|
Range of vertical scroll values issued by the device. |
3 |
scroll_h_range |
fuchsia.input.report/Axis
|
Range of horizontal scroll values issued by the device. |
4 |
buttons |
vector<uint8>:32
|
Button identifiers issued by the device, in priority order. |
5 |
relative_motion_range |
RelativeMotionRange
|
Range of relative movement values issued by the device. |
MouseEvent
Defined in fuchsia.ui.pointer/mouse.fidl
The self-sufficient, self-consistent collection of pointer-related data, sent from server to client.
| Ordinal | Field | Type | Description |
|---|---|---|---|
1 |
timestamp |
zx/Time
|
The time this event was observed. Required. |
2 |
view_parameters |
ViewParameters
|
The parameters of the associated view and viewport, sufficient to correctly interpret the position, orientation, magnitude, and inter-event distance of pointer events dispatched to a view.
|
3 |
device_info |
MouseDeviceInfo
|
A description of the mouse device, sufficient to correctly interpret the capabilities and usage intent of the device.
|
4 |
pointer_sample |
MousePointerSample
|
A description of each sampled data point in a mouse event stream. Issuance policy. There are two dispatch modes, "hover" and "latched". Hover mode is default, and the stream is dispatched in fragments to the visible client that each mouse event hovers above. Latched mode directs the stream to a single client (regardless of view boundary) until unlatched. Latched mode is typically toggled when the user presses the primary mouse button, but is ultimately a product-specific policy. |
5 |
stream_info |
MouseEventStreamInfo
|
The signal for view entry/exit in hover mode.
|
6 |
trace_flow_id |
uint64
|
An identifier to correlate this event's send/receive occurrence across component boundaries or abstraction layers. |
MousePointerSample
Defined in fuchsia.ui.pointer/mouse.fidl
A description of each sampled data point in a mouse event stream.
MousePointerSample may bundle multiple state changes into one event.
For example, if user scrolls mouse wheel and presses the left buttton
down at the same time, client may receive scroll and button state changes
together in 1 event, or receive button change and scroll change in
separate events.
| Ordinal | Field | Type | Description |
|---|---|---|---|
1 |
device_id |
uint32
|
An identifier for the mouse device that issues a mouse event stream. Required. |
2 |
position_in_viewport |
Point2
|
The position of this event, in the viewport's coordinate system. Required. |
3 |
scroll_v |
int64
|
Relative vertical scrolling displacement by detent. |
4 |
scroll_h |
int64
|
Relative horizontal scrolling displacement by detent. |
5 |
pressed_buttons |
vector<uint8>:32
|
Identifiers of currently pressed buttons. |
6 |
relative_motion |
RelativeMotion
|
The relative movement performed, independent of the viewport's coordinate system. |
7 |
scroll_v_physical_pixel |
float64
|
Recommended vertical scrolling displacement by physical pixel, it is computed with accelerator, detent / mm to pixel ratio, etc. |
8 |
scroll_h_physical_pixel |
float64
|
Recommended horizontal scrolling displacement by physical pixel, it is computed with accelerator, detent / mm to pixel ratio, etc. |
9 |
is_precision_scroll |
bool
|
Indicated if the scroll event is from a precision scroll device (HI_RES mouse or touchpad). Clients may want to play interpolation animations on non precision scroll device for smooth scrolling. |
TouchDeviceInfo
Defined in fuchsia.ui.pointer/touch.fidl
Information about a device that issues touch event streams.
| Ordinal | Field | Type | Description |
|---|---|---|---|
1 |
id |
uint32
|
An identifier for the touch device that issues touch event streams. A device may own multiple pointers, each with its own pointer id and its own touch event stream. Required. |
TouchEvent
Defined in fuchsia.ui.pointer/touch.fidl
The self-sufficient, self-consistent collection of pointer-related data, sent from server to client.
| Ordinal | Field | Type | Description |
|---|---|---|---|
1 |
timestamp |
zx/Time
|
The time this event was observed. Required. |
2 |
view_parameters |
ViewParameters
|
The parameters of the associated view and viewport, sufficient to correctly interpret the position, orientation, magnitude, and inter-event distance of touch events dispatched to a view.
|
3 |
device_info |
TouchDeviceInfo
|
A description of the pointer device, sufficient to correctly interpret the capabilities and usage intent of the device.
|
4 |
pointer_sample |
TouchPointerSample
|
A description of each sampled data point in an interaction of touch events.
|
5 |
interaction_result |
TouchInteractionResult
|
The result of gesture disambiguation for a interaction of touch events.
|
6 |
trace_flow_id |
uint64
|
An identifier to correlate this event's send/receive occurrence across component boundaries or abstraction layers. |
TouchPointerSample
Defined in fuchsia.ui.pointer/touch.fidl
A description of each sampled data point in a touch event stream. All fields are required.
| Ordinal | Field | Type | Description |
|---|---|---|---|
1 |
interaction |
TouchInteractionId
|
The interaction that this pointer sample belongs to. |
2 |
phase |
EventPhase
|
The state of this event in the interaction's state machine. |
3 |
position_in_viewport |
Point2
|
The position of this event, in the viewport's coordinate system. |
TouchResponse
Defined in fuchsia.ui.pointer/touch.fidl
A feedback event per |Event|, sent from client to server.
Only |TouchPointerSample| requires a |TouchResponseType|; for other events, the server expects an empty |TouchResponse| table.
| Ordinal | Field | Type | Description |
|---|---|---|---|
1 |
response_type |
TouchResponseType
|
The interaction disposition that a client responds with for a given |TouchPointerSample|. |
2 |
trace_flow_id |
uint64
|
An identifier to correlate this response's send/receive occurrence across component boundaries or abstraction layers. |
CONSTANTS
| Name | Value | Type | Description |
|---|---|---|---|
| MOUSE_MAX_EVENT |
128
|
uint32 |
|
| TOUCH_MAX_EVENT |
128
|
uint32 |
ALIASES
| Name | Value | Description |
|---|---|---|
| Mat3 |
array[9] |
A floating-point 3x3 matrix.
|
| Point2 |
array[2] |
A floating-point two-dimensional point.
|
| RelativeMotion |
array[2] |
The relative motion performed by a mouse device.
|
| RelativeMotionRange |
array[2] |
The valid values of relative motion for a mouse device.
|