fuchsia.ui.pointer

Added: 7

PROTOCOLS

MouseSource

Defined in fuchsia.ui.pointer/mouse.fidl

A method for a client to receive mouse pointer events.

The position of a pointer event is defined in the context of a viewport, situated in the view. The dimensions of the view and viewport, and their spatial relationship (defined with a transform matrix), are supplied synchronously in a |ViewParameter| table. A view may retrieve a pointer's position in its local coordinate system by applying the viewport-to-view transform matrix.

The viewport is embedded in an independent and stable coordinate system, suitable for interpreting pointer events in a scale-independent manner; mouse movement will be observed at a constant scale, even under effects such as magnification or panning. However, other effects, such as enlargening the view's clip bounds, may trigger a change in the viewport extents.

Watch

A method for a client to receive mouse pointer events.

This call is formulated as a "hanging get" pattern: the client asks for a set of recent events, and receives them via the callback. This pull-based approach ensures that clients consume events at their own pace; events don't clog up the channel in an unbounded manner.

Flow control. The caller is allowed at most one in-flight |Watch| call at a time; it is a logical error to have concurrent calls to |Watch|. Non-compliance results in channel closure.

Client pacing. The server will dispatch events to the caller on a FIFO, lossless, best-effort basis, but the caller must allocate enough time to keep up with new events.

Event times. The timestamps on each event in the event vector are not guaranteed monotonic; events from different devices may be injected into Scenic at different times. Generally, events from a single device are expected to have monotonically increasing timestamps.

View parameters. Occasionally, changes in view or viewport require notifying the client. If a |MouseEvent| carries |ViewParameters|, these parameters apply to successive |MousePointerSample|s until the next |ViewParameters|.

Request

<EMPTY>

Response

NameType
events vector<MouseEvent>:128

TouchSource

Defined in fuchsia.ui.pointer/touch.fidl

A method for a client to receive touch events and respond in a global gesture disambiguation protocol.

The position of a touch event is defined in the context of a viewport, situated in the view. The dimensions of the view and viewport, and their spatial relationship (defined with a transform matrix), are supplied synchronously in a |ViewParameter| table. A view may retrieve a pointer's position in its local coordinate system by applying the viewport-to-view transform matrix.

The viewport is embedded in an independent and stable coordinate system, suitable for interpreting touch events in a scale-independent manner; a swipe will be observed at a constant scale, even under effects such as magnification or panning. However, other effects, such as enlargening the view's clip bounds, may trigger a change in the viewport extents.

UpdateResponse

The gesture protocol allows a client to enact a "hold" on an open interaction of touch events; it prevents resolution of interaction ownership, even after the interaction closes. This method updates the client's previous "hold" by replacing it with a response that allows ownership resolution to proceed.

See |TouchInteractionId| for how a stream is structured into interactions.

Flow control. The caller is allowed at most one |UpdateResponse| call per interaction, and it must be on a closed interaction. It is a logical error to call |UpdateResponse| when a normal response is possible with the |Watch| call.

Validity. This TouchResponse must not be another "hold" response, and the overwritten response is expected to be a "hold" response.

Request

NameType
interaction TouchInteractionId
response TouchResponse

Response

<EMPTY>

Watch

A method for a client to receive touch pointer events.

This call is formulated as a "hanging get" pattern: the client asks for a set of recent events, and receives them via the callback. This pull-based approach ensures that clients consume events at their own pace; events don't clog up the channel in an unbounded manner.

Flow control. The caller is allowed at most one in-flight |Watch| call at a time; it is a logical error to have concurrent calls to |Watch|. Non-compliance results in channel closure.

Client pacing. The server will dispatch events to the caller on a FIFO, lossless, best-effort basis, but the caller must allocate enough time to keep up with new events. An unresponsive client may be categorized as "App Not Responding" and targeted for channel closure.

Responses. The gesture disambiguation scheme relies on the server receiving a |TouchResponse| for each |TouchEvent|.|TouchPointerSample|; non-sample events should return an empty |TouchResponse| table to the server. Responses for previous events are fed to the server on the next call of |Watch| [1]. Each element in the |responses| vector is interpreted as the pairwise response to the event in the previous |events| vector; the vector lengths must match. Note that the client's contract to respond to events starts as soon as it registers its endpoint with scenic, NOT when it first calls Watch().

Initial response. The first call to |Watch| must be an empty vector.

Event times. The timestamps on each event in the event vector are not guaranteed monotonic; touch events from different devices may be injected into Scenic at different times. Generally, events from a single device are expected to have monotonically increasing timestamps.

View parameters. Occasionally, changes in view or viewport require notifying the client. If a |TouchEvent| carries |ViewParameters|, these parameters apply to successive |TouchPointerSample|s until the next |ViewParameters|.

[1] The hanging get pattern enables straightforward API evolution, but unfortunately does not admit an idiomatic matching of response to event.

Request

NameType
responses vector<TouchResponse>:128

Response

NameType
events vector<TouchEvent>:128

STRUCTS

MouseEventStreamInfo

Defined in fuchsia.ui.pointer/mouse.fidl

The status of a mouse event stream, sent from server to client.

Invariant: a client's mouse events are bracketed by MouseViewStatus.ENTERED and MouseViewStatus.EXITED.

FieldTypeDescriptionDefault
device_id uint32

An identifier for the mouse device that issues a mouse event stream.

No default
status MouseViewStatus

The mouse event stream's enter/exit status, sent from server to client.

No default

Rectangle

Defined in fuchsia.ui.pointer/view.fidl

An axis-aligned rectangle. It is defined by its minimal and maximal extents in a coordinate system.

FieldTypeDescriptionDefault
min Point2

The minimal extent of this rectangle, inclusive.

  • Its coordinates are pairwise less than the maximal extent's coordinates.
No default
max Point2

The maximal extent of this rectangle, inclusive.

  • Its coordinates are pairwise greater than the minimal extent's coordinates.
No default

TouchInteractionId

Defined in fuchsia.ui.pointer/touch.fidl

A unique identifier for a "interaction" of touch events in an event stream. Touch events are observed as a succession of interactions, as fingers engage and disengage with the display.

A finite sequence of pointer events that follows the EventPhase state machine, starting from the initial state ADD, is called an interaction. A closed (or past) interaction is one where it has reached the terminal state (REMOVE or CANCEL); an open (or current) interaction is one where it has not.

For a given device pointer, a stream of events is observed as a succession of zero or more closed interactions (the past history of user engagement), followed by at most one open interaction (the current user engagement).

Because we need to group pointer events by their interaction, touch event carries an interaction id that is unique in that pointer stream. This common reference makes it possible to operate on a closed interaction, as well as an open interaction.

Also see EventPhase for a discussion on event streams by mice.

FieldTypeDescriptionDefault
device_id uint32

An identifier for the pointer device that issues touch event streams. A device may own multiple pointers, each with its own |pointer_id|.

No default
pointer_id uint32

An identifier of the pointer that issued this event. It is unique only to a specific |device_id|. Each (device_id, pointer_id) pair issues at most one open interaction at a time.

No default
interaction_id uint32

An identifier of the interaction. It is unique only to a specific (device_id, pointer_id) pair.

No default

TouchInteractionResult

Defined in fuchsia.ui.pointer/touch.fidl

The result of gesture disambiguation for a interaction of touch events, sent from server to client.

FieldTypeDescriptionDefault
interaction TouchInteractionId

The interaction that this pointer sample belongs to.

No default
status TouchInteractionStatus

The interaction's disposition, sent from server to client.

No default

ViewParameters

Defined in fuchsia.ui.pointer/view.fidl

The parameters of the associated view and viewport, sufficient to correctly interpret the position and scale of pointer events dispatched to this view.

Ordering. These parameters arrive over the same channel as pointer events, to provide synchronous context for interpreting the position of pointer events in the view's local coordinate system.

Inter-protocol redundancy. Some of these parameters may also be sent over an independent channel dedicated to view control; the client is responsible for correct use of asynchronously received parameters.

TODO(https://fxbug.dev/42162292): Rename viewport, it is used in Flatland.

FieldTypeDescriptionDefault
view Rectangle

The view's area of content, placed in the coordinate system of the view.

The rectangle is defined by the parent view's setup of clipping on this view.

No default
viewport Rectangle

The viewport's area of interaction, placed in an independent coordinate system.

A pointer event's position is defined in the coordinate system of this viewport.

A change in viewport extents means the region for pointer interaction has itself moved, or changed in size, or both.

No default
viewport_to_view_transform Mat3

The transform matrix that relates a point's position in the viewport's coordinate system to its position in the view's coordinate system.

No default

ENUMS

EventPhase strict

Type: uint32

Defined in fuchsia.ui.pointer/state.fidl

The possible states of a pointer event. These phases of events in a stream follow a state machine that starts with the ADD phase, followed by zero or more CHANGE phases, and finally terminates with REMOVE or CANCEL phase.

ADD ---> CHANGE* -+-> REMOVE
                  |
                  +-> CANCEL

A finite sequence of pointer events that follows this state machine, starting from the initial state, is called an interaction. A closed (or past) interaction is one where it has reached the terminal state; an open (or current) interaction is one where it has not.

For a given device pointer, a stream of events is observed as a succession of zero or more closed interactions (the past history of user engagement), followed by at most one open interaction (the current user engagement).

When we need to group pointer events by their interaction, an event carries an interaction id that is unique in that pointer stream. This common reference makes it possible to operate on a closed interaction, as well as an open interaction.

For example, touch events are typically observed as a succession of interactions, as fingers engage and disengage with the display.

NameValueDescription
1

The device has started tracking the pointer.

2

The device has reported an update to the pointer state.

3

The device has stopped tracking the pointer.

4

The pointer is no longer available.

MouseViewStatus strict

Type: uint32

Defined in fuchsia.ui.pointer/mouse.fidl

A description of mouse event stream's relationship to this view.

NameValueDescription
1

The stream is directed towards this view.

2

The stream is directed away from this view.

TouchInteractionStatus strict

Type: uint32

Defined in fuchsia.ui.pointer/touch.fidl

A description of the interaction's relationship to this client.

NameValueDescription
1

The client has been denied ownership of the interaction.

2

The client has been granted ownership of the interaction.

TouchResponseType strict

Type: uint32

Defined in fuchsia.ui.pointer/touch.fidl

The possible interaction dispositions that a client can respond with to a given |TouchPointerSample|. Used as part of a gesture disambiguation scheme.

The responses are based on the idea of an ownership claim on a interaction. Clients may assert a claim of ownership on an open interaction, but only one client's claim is granted by the server; other clients' claims are denied.

NameValueDescription
1

The client has no further interest in this interaction; it declines ownership of the interaction. The client will stop receiving events for this interaction.

2

The client is interested in this interaction, but needs to see more events to decide; the client has not yet claimed ownership of this interaction.

3

The client is interested in this interaction, but needs to see more events to decide; the client has not yet claimed ownership of the interaction. During ownership resolution, it exerts its priority over lower-priority "maybe" claims, but always loses to a "yes" claim.

4

The client is interested in this interaction, but needs to see more events to decide; the client has not yet claimed ownership of the interaction. Moreover, it suppresses lower-priority claims that try to resolve interaction ownership.

5

The client is interested in this interaction, but needs to see more events to decide; the client has not yet claimed ownership of the interaction. Moreover, it suppresses lower-priority claims that try to resolve interaction ownership. During ownership resolution, it exerts its priority over lower-priority "maybe" claims, but always loses to a "yes" claim.

6

The client is interested in this interaction, but needs to see a subsequent interaction to decide; the client has not yet claimed ownership of this interaction. It prevents ownership resolution when the interaction closes.

7

The client is interested in this interaction, but needs to see a subsequent interaction to decide; the client has not yet claimed ownership of this interaction. It prevents ownership resolution when the interaction closes. Moreover, it suppresses lower-priority claims that try to resolve interaction ownership.

8

The client wishes exclusive access to the remaining events in this interaction; it claims ownership of this interaction (but that claim may be granted or denied). During ownership resolution, it yields its priority to lower-priority "yes" claims.

9

The client wishes exclusive access to the remaining events in this interaction; it claims ownership of this interaction (but that claim may be granted or denied). During ownership resolution, it exerts its priority over lower-priority "yes" claims.

TABLES

MouseDeviceInfo

Defined in fuchsia.ui.pointer/mouse.fidl

Information about a device that issues a mouse event stream.

OrdinalFieldTypeDescription
id uint32

An identifier for the mouse device that issues a mouse event stream. Required.

scroll_v_range fuchsia.input.report/Axis

Range of vertical scroll values issued by the device.

scroll_h_range fuchsia.input.report/Axis

Range of horizontal scroll values issued by the device.

buttons vector<uint8>:32

Button identifiers issued by the device, in priority order.

relative_motion_range RelativeMotionRange

Range of relative movement values issued by the device.

MouseEvent

Defined in fuchsia.ui.pointer/mouse.fidl

The self-sufficient, self-consistent collection of pointer-related data, sent from server to client.

OrdinalFieldTypeDescription
timestamp zx/Time

The time this event was observed. Required.

view_parameters ViewParameters

The parameters of the associated view and viewport, sufficient to correctly interpret the position, orientation, magnitude, and inter-event distance of pointer events dispatched to a view.

  • It is issued on connection and on change.
device_info MouseDeviceInfo

A description of the mouse device, sufficient to correctly interpret the capabilities and usage intent of the device.

  • It is issued once per device.
pointer_sample MousePointerSample

A description of each sampled data point in a mouse event stream.

Issuance policy. There are two dispatch modes, "hover" and "latched". Hover mode is default, and the stream is dispatched in fragments to the visible client that each mouse event hovers above. Latched mode directs the stream to a single client (regardless of view boundary) until unlatched. Latched mode is typically toggled when the user presses the primary mouse button, but is ultimately a product-specific policy.

stream_info MouseEventStreamInfo

The signal for view entry/exit in hover mode.

  • It is issued on hover entry into a view, and hover exit from a view.
trace_flow_id uint64

An identifier to correlate this event's send/receive occurrence across component boundaries or abstraction layers.

MousePointerSample

Defined in fuchsia.ui.pointer/mouse.fidl

A description of each sampled data point in a mouse event stream.

MousePointerSample may bundle multiple state changes into one event. For example, if user scrolls mouse wheel and presses the left buttton down at the same time, client may receive scroll and button state changes together in 1 event, or receive button change and scroll change in separate events.

OrdinalFieldTypeDescription
device_id uint32

An identifier for the mouse device that issues a mouse event stream. Required.

position_in_viewport Point2

The position of this event, in the viewport's coordinate system. Required.

scroll_v int64

Relative vertical scrolling displacement by detent.

scroll_h int64

Relative horizontal scrolling displacement by detent.

pressed_buttons vector<uint8>:32

Identifiers of currently pressed buttons.

relative_motion RelativeMotion

The relative movement performed, independent of the viewport's coordinate system.

scroll_v_physical_pixel float64

Recommended vertical scrolling displacement by physical pixel, it is computed with accelerator, detent / mm to pixel ratio, etc.

scroll_h_physical_pixel float64

Recommended horizontal scrolling displacement by physical pixel, it is computed with accelerator, detent / mm to pixel ratio, etc.

is_precision_scroll bool

Indicated if the scroll event is from a precision scroll device (HI_RES mouse or touchpad). Clients may want to play interpolation animations on non precision scroll device for smooth scrolling.

TouchDeviceInfo

Defined in fuchsia.ui.pointer/touch.fidl

Information about a device that issues touch event streams.

OrdinalFieldTypeDescription
id uint32

An identifier for the touch device that issues touch event streams. A device may own multiple pointers, each with its own pointer id and its own touch event stream. Required.

TouchEvent

Defined in fuchsia.ui.pointer/touch.fidl

The self-sufficient, self-consistent collection of pointer-related data, sent from server to client.

OrdinalFieldTypeDescription
timestamp zx/Time

The time this event was observed. Required.

view_parameters ViewParameters

The parameters of the associated view and viewport, sufficient to correctly interpret the position, orientation, magnitude, and inter-event distance of touch events dispatched to a view.

  • It is issued on connection and on change.
device_info TouchDeviceInfo

A description of the pointer device, sufficient to correctly interpret the capabilities and usage intent of the device.

  • It is issued once per device.
pointer_sample TouchPointerSample

A description of each sampled data point in an interaction of touch events.

  • It is issued on every sample in the interaction.
interaction_result TouchInteractionResult

The result of gesture disambiguation for a interaction of touch events.

  • It is issued once per interaction.
trace_flow_id uint64

An identifier to correlate this event's send/receive occurrence across component boundaries or abstraction layers.

TouchPointerSample

Defined in fuchsia.ui.pointer/touch.fidl

A description of each sampled data point in a touch event stream. All fields are required.

OrdinalFieldTypeDescription
interaction TouchInteractionId

The interaction that this pointer sample belongs to.

phase EventPhase

The state of this event in the interaction's state machine.

position_in_viewport Point2

The position of this event, in the viewport's coordinate system.

TouchResponse

Defined in fuchsia.ui.pointer/touch.fidl

A feedback event per |Event|, sent from client to server.

Only |TouchPointerSample| requires a |TouchResponseType|; for other events, the server expects an empty |TouchResponse| table.

OrdinalFieldTypeDescription
response_type TouchResponseType

The interaction disposition that a client responds with for a given |TouchPointerSample|.

trace_flow_id uint64

An identifier to correlate this response's send/receive occurrence across component boundaries or abstraction layers.

CONSTANTS

NameValueTypeDescription
MOUSE_MAX_EVENT 128 uint32
TOUCH_MAX_EVENT 128 uint32

ALIASES

NameValueDescription
Mat3 array[9]

A floating-point 3x3 matrix.

  • The values are placed in column-major order.
Point2 array[2]

A floating-point two-dimensional point.

  • The values are placed in (x, y) order.
RelativeMotion array[2]

The relative motion performed by a mouse device.

RelativeMotionRange array[2]

The valid values of relative motion for a mouse device.

  • The ranges are placed in (x, y) order.