fuchsia.ui.pointerinjector

Added: 7

PROTOCOLS

Device

Defined in fuchsia.ui.pointerinjector/device.fidl

The channel for injecting pointer events into a target. The associated |Config| applies to injected pointer events.

A |Device| may inject pointer events into a target in the scene, but the dispatch of pointer events to UI clients within that target's view tree will depend on the dispatch policy and the scene topology.

Inject

Injects events into a target.

Batching: The caller is expected to batch-inject all events that share the same timestamp (e.g., multiple finger samples). There may be multiple such batches per injection call.

Flow control: The caller is allowed at most one in-flight |Inject| call at at time: subsequent |Inject| calls must wait until the acknowledgment returns. Non-compliance results in channel closure.

Request

NameType
events vector<Event>:128

Response

<EMPTY>

Registry

Defined in fuchsia.ui.pointerinjector/device.fidl

A coordination mechanism to clearly define the injecting of pointer events into the scene (injection policy, injection context, etc), for dispatching pointer events to UI clients (referenced by |fuchsia.ui.views.ViewRef|).

NOTE WELL! ONLY COMPONENTS IN THE TRUSTED COMPUTING BASE SHOULD BE GRANTED ACCESS TO THIS PROTOCOL. (1) The injector device acts with the user's authority. UI clients cannot tell the difference between a real user and an injector device. (2) The context is self-specified when registering the injector device. An injector device that gains access to a sensitive context will operate in that scope. (3) The injected events may be arbitrary. An injector device can inject whatever it wants to the clients it can reach.

This protocol defines the context and target for an injector |Device|, places a viewport in spatial relation to the target, and connects the |Device| to the viewport. These parameters are set up in the |Config|. A |Device| may inject pointer events on the viewport, but the dispatch of pointer events to UI clients in the target will depend on the dispatch policy and the scene topology.

A |Device| operates within a context, and it serves two purposes: (1) Scoping. It confirms the |Device|'s authority to inject pointer events on the specified target. (2) Coordinate system. It provides the reference coordinate system to define the viewport's position and size, relative to the target.

Guarantees. A |Device| is directed at a specified target in the scene, and such a target is afforded some guarantees against snooping and interference from UI clients outside the target's view tree ("non-target clients"). Non-target clients never have injected events dispatched to them: a |Device| preserves confidentiality for target clients. Non-target clients never block injected events from reaching the target on injection, or from reaching target clients on dispatch: a |Device| preserves integrity and availability for target clients. However, the |Device| itself is subject to disruption by ancestor views of the target (see connectivity invariant).

Note. This protocol, and its policies, are not a sufficient guard against UI redress attacks! The confidentiality, integrity, and availability guarantees apply only with respect to non-target clients. Within a target, UI redress attacks can still happen, between target clients.

Connectivity invariant. A |Device| operates in a stable view tree that is connected to the root view. When either the target, or both context and target, are disconnected from the view tree by a UI client, the |Device| channel is closed. If an event stream was in flight, the server dispatches a final CANCEL event following channel closure; this CANCEL event is then propagated according to dispatch policy.

Register

Sets up a context for a |Device|, the target for the |Device|, and directs the |Device| at the target via a viewport, positioned in the context. These parameters are specified in a |Config|.

If |config| is invalid (e.g., missing important data), the |Device| request will be denied: the channel will be closed.

The return event tells the caller that (1) the server has processed the call, and (2) the |Device| is connected.

Request

NameType
config Config
injector server_end:Device

Response

<EMPTY>

ENUMS

DeviceType strict

Type: uint32

Defined in fuchsia.ui.pointerinjector/config.fidl

A characterization of a device that issues pointer event streams.

NameValueDescription
1

A device intended for manipulation by direct contact over its surface.
2

A device intended for manipulation by precise movement over a surface.

DispatchPolicy strict

Type: uint32

Defined in fuchsia.ui.pointerinjector/config.fidl

A specification of the UI clients that may have injected events dispatched to them in an |Target|. One is specified in |Config|.

A useful concept is "latching", where one or more clients may start receiving the pointer event stream, prior to assignment of stream ownership. After ownership is assigned (e.g., through a gesture disambiguation protocol), non-owning clients have their latch terminated -- they stop receiving the pointer event stream.

  • A client's latch does not itself confer stream ownership (receiving the entire pointer event stream); gesture disambiguation or device termination may prematurely end the stream dispatched to that client.
  • It's possible for a client to latch while hidden from the user (i.e., manipulate a surface that is invisible to the user), where the occluding surface is owned by a client outside of |Target|. Conversely, these occluding clients cannot latch, because latched clients must be in |Target|'s view tree.

NameValueDescription
1

A single client latches onto a pointer event stream, where:

  • the pointer event stream starts within the viewport rectangle,
  • the latch happens on the ADD phase of the pointer event stream,
  • the client is the |Target| itself. Ownership is assigned immediately to the client.

Note: This policy guarantees confidentiality, integrity, and availability of dispatch to the client, but by itself, does not confer immunity against UI redress attacks.
2

Multiple clients may latch onto a pointer stream, where:

  • the pointer stream starts within the viewport rectangle,
  • a hit test is performed on the ADD phase of the pointer event stream, which returns the top-most surface (in paint order) in the |Target|'s view tree,
  • the top-most surface's client latches onto the pointer stream,
  • the client's ancestors in the |Target|'s view tree also latch onto the pointer stream.

With multiple latches, a pointer stream is dispatched in parallel to each latched client, until ownership is assigned via gesture disambiguation. The owner client will continue to receive the pointer stream, and non-owners will receive a final CANCEL event for the stream.

Note: It's possible for no clients to latch, if the hit test fails to hit any surface in any sub-view of |Target|. Note: Each client will have its own copy of the viewport, placed accordingly in its own coordinate system.
3

The top hit client in the |Target|'s view tree receives hover events when a cursor is positioned over it, unless the mouse is latched to a specific client.

A mouse initiates a latch via button down, and until the release of that latch, mouse events are delivered to that latched client; other clients do not receive hover events in the latch duration.

Note: It's possible for no client to latch, if the hit test fails to hit any surface in any sub-view of |Target|. Note: Each client will have its own copy of the viewport, placed accordingly in its own coordinate system.

EventPhase strict

Type: uint32

Defined in fuchsia.ui.pointerinjector/event.fidl

The possible states of a pointer event stream's state machine.

A typical pointer will move through this state machine: ADD - CHANGE* - REMOVE

NameValueDescription
1

The device has started tracking the pointer.
2

The device has reported an update to the pointer state.
3

The device has stopped tracking the pointer.
4

The event stream is no longer available.

TABLES

Config resource

Defined in fuchsia.ui.pointerinjector/config.fidl

A specification for an injector |Device|.

All fields required.

OrdinalFieldTypeDescription
device_id uint32

An identifier for the pointer device that issues pointer event streams.
device_type DeviceType

A characterization of the pointer device.
context Context

The scope and coordinate system of the injector |Device|.

Note: To avoid circular injection scenarios, |context| must be a strict ancestor of |target|.
target Target

The region where dispatch is attempted for injected events.

Note: To avoid circular injection scenarios, |target| must be a strict descendant of |context|.
viewport Viewport

The initial viewport for injected events. Subsequent modifications to the viewport can also be sent synchronously with injected events.
dispatch_policy DispatchPolicy

A specification of the UI clients in |target| that may have injected events dispatched to them.

For example, if |dispatch_policy| is |EXCLUSIVE|, then only |target| may have injected events dispatched to it, regardless of other factors, such as surface occlusion by clients inside and outside of |target|.
scroll_v_range fuchsia.input.report/Axis

Range of vertical scroll values issued by the device.
scroll_h_range fuchsia.input.report/Axis

Range of horizontal scroll values issued by the device.
buttons vector<uint8>:32

Mouse button identifiers issued by the device, in priority order.

The priority order of mouse buttons is a policy choice enacted by the injector. This priority order is conveyed verbatim to UI clients via fuchsia.ui.pointer.MouseDeviceInfo.buttons, so button priority is shared consistently between multiple UI clients. For example, to swap the left-side (0x1) and right-side (0x2) physical buttons from their traditional roles of primary and secondary mouse buttons, the injector can specify [0x2, 0x1, ... ] in this buttons vector.

Note that preserving the physical button ID is also a policy choice. If preserved, a discerning UI client can distinguish between the ground truth (which physical button was pressed) versus the desired interpretation (what priority does that physical button have, on the current system).
relative_motion_range RelativeMotionRange

Range of X and Y motion by a mouse device.

Event

Defined in fuchsia.ui.pointerinjector/event.fidl

OrdinalFieldTypeDescription
timestamp zx/Time

The time when this event was observed.

Required.
data Data

The event's data.

Required.
trace_flow_id uint64

An identifier to correlate this event's send/receive occurrence across component boundaries or abstraction layers.

PointerSample

Defined in fuchsia.ui.pointerinjector/event.fidl

A description of each sampled data point for a pointer device.

OrdinalFieldTypeDescription
pointer_id uint32

An identifier of the pointer that issued this event. It is unique only to a specific pointer device.
phase EventPhase

The state of this event in the pointer event stream's state machine.
position_in_viewport Point2

The position of this event, in the viewport's coordinate system.
scroll_v int64

Relative vertical scrolling displacement by detent.
scroll_h int64

Relative horizontal scrolling displacement by detent.
pressed_buttons vector<uint8>:32

Identifiers of currently pressed buttons.
relative_motion RelativeMotion

The movement of a mouse, independent of the viewport's coordinate system.
scroll_v_physical_pixel float64

Recommended vertical scrolling displacement by physical pixel, it is computed with accelerator, detent / mm to pixel ratio, etc.
scroll_h_physical_pixel float64

Recommended horizontal scrolling displacement by physical pixel, it is computed with accelerator, detent / mm to pixel ratio, etc.
is_precision_scroll bool

Indicated if the scroll event is from a precision scroll device (HI_RES mouse or touchpad). Clients may want to play interpolation animations on non precision scroll device for smooth scrolling.

Viewport

Defined in fuchsia.ui.pointerinjector/config.fidl

A rectangular region that directs injected events into a target.

The viewport relates a pointer's position across multiple independent coordinate systems: the context, the viewport, and the dispatch clients. Intuitively, the viewport is how a pointer's position is mapped to an interactive part of the scene.

A matrix is used to relate the viewport coordinate system to the context coordinate system. A pair of extents defines the viewport's size in the viewport coordinate system. Together, they define the viewport's placement in the context coordinate system.

The viewport coordinate system is used to convey a pointer's coordinates in a scale-invariant way to dispatch clients, so that pointer movement can be interpreted correctly under effects like magnification. The context defines the viewport's minimal and maximal extents in the viewport coordinate system.

  • The boundary of the viewport, a rectangle, is axis aligned with the viewport coordinate system; however it may otherwise be freely positioned ("float") within it: there is translation and scaling, but no rotation.
  • Floating gives the injector some choice in how to convey coordinates, such as in Vulkan NDC, or in display pixel coordinates.
  • The viewport rectangle defines a latch region used in dispatch (described below).

A dispatch client receives a pointer's coordinates in the viewport coordinate system, along with a matrix to convert coordinates from the viewport coordinate system to the dispatch client's coordinate system.

All fields required.

TODO(https://fxbug.dev/42162296): Rename Viewport, it is used in Flatland.

OrdinalFieldTypeDescription
extents Extents

The viewport's minimal and maximal extents in the viewport coordinate system.
viewport_to_context_transform Mat3

A transformation matrix that describes how to map the viewport coordinate system to the context coordinate system.

This transform, together with |extents|, defines the viewport's placement in the context coordinate system.

This transform must be an invertible matrix (i.e., has a non-zero determinant), which guarantees it describes a bijection between the viewport coordinate system and the context coordinate system. A non-invertible matrix is rejected.

UNIONS

Context strict resource

Defined in fuchsia.ui.pointerinjector/config.fidl

The region from which injection is attempted for injected events. As the context, it serves two purposes: (1) Scoping. It confirms the |Device|'s authority to inject pointer events on the specified target. Specifically, the context must be a view-tree ancestor of the target. (2) Coordinate System. It provides the reference coordinate system to define the viewport's position and size, relative to the target.

OrdinalVariantTypeDescription
view fuchsia.ui.views/ViewRef

A Scenic view from which injection is attempted for injected events.

  • This view must be connected to the scene graph for injection.
  • Injected events are confined to this view and its sub-views.

Data flexible

Defined in fuchsia.ui.pointerinjector/event.fidl

A selection of FIFO data sent over the channel. Each data may have a different issuance policy.

OrdinalVariantTypeDescription
viewport Viewport

The parameters of the viewport, sufficient for a client to correctly interpret the position and scale of pointer events dispatched to it.

  • It is issued on every change to the viewport.
pointer_sample PointerSample

A description of each sampled data point in a pointer event stream.

  • It is issued on every sample in the pointer event stream.

Target strict resource

Defined in fuchsia.ui.pointerinjector/config.fidl

The region in which dispatch is attempted for injected events. The particular dispatch policy is specified in |Config|.

OrdinalVariantTypeDescription
view fuchsia.ui.views/ViewRef

A Scenic view in which dispatch is attempted for injected events.

  • This view must be connected to the scene graph for dispatch.
  • Injected events are confined to this view and its sub-views.

CONSTANTS

NameValueTypeDescription
MAX_INJECT 128 uint32

ALIASES

NameValueDescription
Extents array[2]

A floating-point pair of points, representing minimal and maximal extents.

  • The values are placed in (minimal, maximal) order.
Mat3 array[9]

A floating-point 3x3 matrix.

  • The values are placed in column-major order.
Point2 array[2]

A floating-point two-dimensional point.

  • The values are placed in (x, y) order.
RelativeMotion array[2]

The relative motion performed by a mouse device.
RelativeMotionRange array[2]

The valid values of relative motion for a mouse device.

  • The ranges are placed in (x, y) order.