fuchsia.ui.composition

Added: 7

PROTOCOLS

Allocator

Defined in fuchsia.ui.composition/allocator.fidl

RegisterBufferCollection

A BufferCollection is a set of VMOs created by Sysmem and shared by a number of participants, one of which is the Flatland Renderer. Some content, such as Images, use a BufferCollection as their backing memory.

See RegisterBufferCollectionArgs for information on each argument.

Request

NameType
args RegisterBufferCollectionArgs

Response

NameType
payload Allocator_RegisterBufferCollection_Result

ChildViewWatcher

Defined in fuchsia.ui.composition/flatland.fidl

A protocol that provides information about a particular child View which is attached to the Viewport owned by the parent client; connections to this protocol are estabished in CreateViewport. Since a Flatland instance may contain any number of Viewports, each of which may or may not be attached to a transform, the client can maintain connections to an equal number of ChildViewWatcher instances.

Each ChildViewWatcher instance will remain connected as long as the corresponding child View exists; the connection will also be closed if the child's ViewCreationToken is dropped without using it to create a View.

GetStatus

A hanging get for receiving the status of a View. This provides information to the parent, such as whether or not the child has successfully presented content through this View.

This hanging get will only fire when the ChildViewStatus is different than the previously returned ChildViewStatus. This can happen immediately, and even if the creator of the Viewport hasn't yet called Present() after calling CreateViewport(). This allows the parent to know that the child has content ready to display before the parent modifies their own local scene graph to incorporate the child content.

It is invalid to call GetStatus while a previous call is still pending. Doing so will cause both this channel and the Flatland channel that handed out ChildViewWatcher to be closed.

Request

<EMPTY>

Response

NameType
status ChildViewStatus

GetViewRef

Hanging get to receive the ViewRef of the child View. This will only fire when the View ref is different from the previously-returned View ref. Note: currently the View ref will not change after it is first received, but this will change if/when the API changes to allow relinking of views.

The ViewRef is not returned until the View is included in the View tree, in other words when there is a chain of ancestor transforms all the way back up to the root of the scene graph, i.e. the display.

It is invalid to call GetViewRef while a previous call is still pending. Doing so will cause both this channel and the Flatland channel that handed out ChildViewWatcher to be closed.

A GetViewRef call will hang if the View was created using CreateView, but returns the View if the View was created using CreateView2. This is because CreateView does not mint a ViewRef for that View.

Request

<EMPTY>

Response

NameType
view_ref fuchsia.ui.views/ViewRef

Flatland

Defined in fuchsia.ui.composition/flatland.fidl

Each Flatland instance contains a Graph, which consists of a set of objects, and the relationships between those objects. The client can specify a subset of those objects (specifically, the directed acyclic graph starting at the root transform) to be presented as content to some kind of output -- usually, a display.

Flatland Graphs are both hierarchical, and distributed. Graphs from different Flatland instances may be connected together, allowing multiple processes to be involved in authoring content for a particular output.

All functions in this protocol are feed-forward. The operations they represent are not fully executed until Present is called.

AddChild

Adds a child Transform to a parent Transform. The new child Transform, and any Content attached to it or its children, will be rendered on top of the parent's Content, as well as any previously added children.

Request

NameType
parent_transform_id TransformId
child_transform_id TransformId

Clear

This function will reset all state on this interface. This includes destroying all existing View and Viewports without returning the associated Token to the caller.

Request

<EMPTY>

CreateFilledRect

Creates a solid-color rectangle. By default a filled-rect does not have a defined color or size. It is necessary to call |SetSolidFill| to specify a color and size before a filled rect can be used for rendering. Not doing so will result the

Request

NameType
rect_id ContentId

CreateImage

An Image is a bitmap backed by a specific VMO in a BufferCollection.

Image creation requires an allocated BufferCollection registered with Allocator. This function will fail unless all clients of the specified BufferCollection have set their constraints.

The Image must reference a valid VMO index and must have ImageProperties that fall within the constraints specified by the backing BufferCollection (i.e. width and height within a valid range, etc.)

Zero is not a valid Image id. All other values are valid, assuming they are not already in use for another piece of Content (see ReleaseImage for more details).

Request

NameType
image_id ContentId
import_token BufferCollectionImportToken
vmo_index uint32
properties ImageProperties

CreateTransform

Creates a new Transform node. Transforms are a hierarchical piece of a Flatland graph. They can have children, and can reference Content. A sub-graph represented by a Transform and its descendants can be rendered to a display.

Transforms are kept alive, even when released, as long as they are children of either an unreleased Transform, or the Root Transform.

Each Transform can have a single piece of attached Content. Common types of Content include bitmaps, asynchronous streams of images, and Viewports to Views hosted in other Flatland instances.

Transforms have attributes. Child Transforms inherit the combined attributes of their parents. Content attached to a Transform is also affected by that Transform's attributes.

When a sub-graph of Transforms is rendered, Content will be rendered back-to-front, starting with the Content on the root transform, and continuing recursively through all of its child Transforms in the order the children were added. See AddChild for more information.

Zero is not a valid transform id. All other values are valid, assuming they are not already in use (see ReleaseTransform for more details).

Request

NameType
transform_id TransformId

CreateView

Two Flatland instances may be connected in a parent-child relationship. The parent endpoint is held in a Viewport, and the child endpoint is held in a View. The parent Flatland instance that creates a Viewport has control over how the child's View is integrated into its own View.

The lifecycle of a parent-child connection starts with two endpoints of a channel object: a ViewportCreationToken and a ViewCreationToken. Out-of-band protocols pass the ViewportCreationToken to the parent, which calls CreateViewport, and the ViewCreationToken to the child, which calls CreateView.

Only nodes connected to the Root Transform in this Flatland instance will be rendered into the parent's Viewport.

Calling CreateView a second time will disconnect the Root Transform from the existing parent's Viewport, and attach it to a new parent's Viewport. In other words, each View can only have one parent.

This function is queued, meaning that the Root Transform will not be attached to the parent Viewport until Present is called. However, clients will receive information through their ParentViewportWatcher (e.g., LayoutInfo) immediately after calling this function, even if they have not called Present or SetRootTransform. This allows clients to wait for layout information from their parent before calling Present.

Any illegal operations on ParentViewportWatcher will cause both ParentViewportWatcher channel and this Flatland channel to be torn down.

Lifecycle note. The lifetime of the ParentViewportWatcher channel is bound by the peer ViewportCreationToken. When the ViewportCreationToken dies, this ParentViewportWatcher channel is destroyed.

Views and subgraphs of Views created using CreateView will not be represented in the ViewTree, and hence will not be able to participate in any ViewTree-dependent interactions such as touch, mouse or focus. The Flatland protocol provides no way for Views in the subgraph of a View created with CreateView to know that they are excluded from the ViewTree.

Request

NameType
token fuchsia.ui.views/ViewCreationToken
parent_viewport_watcher server_end:ParentViewportWatcher

CreateView2

Identical to CreateView, except it allows association of View identity (ViewRef) and view-bound protocols.

Request

NameType
token fuchsia.ui.views/ViewCreationToken
view_identity fuchsia.ui.views/ViewIdentityOnCreation
protocols ViewBoundProtocols
parent_viewport_watcher server_end:ParentViewportWatcher

CreateViewport

The Viewport and View pair, together, represent the connection between two Flatland instances. The Viewport is created in the parent, and the View is created in the child. The parent has control over how the child's View is integrated into its own View.

Any illegal operations on ChildViewWatcher will cause both ChildViewWatcher channel and this Flatland channel to be torn down.

ViewportProperties must have logical_size set. This is the initial size that will drive the layout of the child. The logical_size is also used as the default Content size, but subsequent changes to the logical_size will have no effect on the Content size.

ViewportProperties may have inset field not set. In that case, the default value of (0, 0, 0, 0) is used.

The logical_size must have positive X and Y components.

Zero is not a valid ContentId. All other values are valid, assuming they are not already in use for another piece of Content (see ReleaseViewport for more details).

Lifecycle note. The lifetime of the ChildViewWatcher channel is bound by the peer ViewCreationToken. When the ViewCreationToken dies, this ChildViewWatcher channel is destroyed.

Request

NameType
viewport_id ContentId
token fuchsia.ui.views/ViewportCreationToken
properties ViewportProperties
child_view_watcher server_end:ChildViewWatcher

OnError

If an error occurs after a Present, an OnError event will fire with associated information.

FlatlandErrors will close the connection with the client.

  • response error the error a client may receive after performing some invalid operations.

Response

NameType
error FlatlandError

OnFramePresented

This event is fired whenever a set of one or more Presents are presented simultaneously, and are therefore no longer in flight.

This event signifies that the commands enqueued before the Present have taken effect in the scene graph, and are globally visible.

Clients do NOT have to implement a handler for this event for basic frame scheduling, unless they explicitly want feedback on prior frame presentation. All future frame information is given in the OnNextFrameBegin event.

TODO(https://fxbug.dev/42141795): remove num_presents_allowed from this event.

Response

NameType
frame_presented_info fuchsia.scenic.scheduling/FramePresentedInfo

OnNextFrameBegin

This event is fired when clients should expect minimal resource contention. Clients may use the timing of this event to begin their rendering work, using the information returned to inform their scheduling decisions.

Importantly, OnNextFrameBegin is only fired when the client has one or more present credits, including what is returned in this event. It is therefore safe to present once every time this event fires.

  • response values the table of information a client needs to produce its next frame. See OnNextFrameBeginValues for more information.

Response

NameType
values OnNextFrameBeginValues

Present

Complete execution of all feed-forward operations.

If executing an operation produces an error (e.g., CreateTransform(0)), an OnError event is emitted. Operations that produce errors are ignored and the channel is closed.

If the execution is completed successfully, OnNextFrameBegin emits NO_ERROR along with other valid fields.

The client may only call Present when they have a non-zero number of present credits, which are tracked by the server. The server may increment the number of credits when it fires the OnNextFrameBegin event, which informs the client when it receives additional present credits. Each Present call uses one present credit and decrements the server count by one. If the client calls Present with no present credits, the server will return a NO_PRESENTS_REMAINING error.

The client should assume that prior to receiving any OnNextFrameBegin events, they have one present credit.

Every Present call results in one OnNextFrameBegin event, and one OnFramePresented event, typically in that order.

When the commands flushed by Present make it to display, an OnFramePresented event is fired. This event includes information pertaining to all Presents that had content that were part of that frame.

See fuchsia.ui.composition/PresentArgs documentation above for more detailed information on what arguments are passed in and their role.

Request

NameType
args PresentArgs

ReleaseFilledRect

Automatically garbage collects the rectangle when it is no longer needed for rendering. |rect_id| must have been instantiated with a call to |CreateFilledRect|. Once released, the ID immediately goes out of scope and is free to be used again.

Request

NameType
rect_id ContentId

ReleaseImage

Released Images will be garbage collected by the system once they are no longer necessary for rendering. For Images, this means the Image is no longer attached to any Transform and any pending rendering that references the Image is complete.

Use SetContent(transform_id, 0) to clean up references to released Images.

Once released, the id immediately goes out of scope for future function calls and can be reused when creating new Images.

It is an error to call functions with a released id (unless that id has been reused to construct a new Image).

Request

NameType
image_id ContentId

ReleaseTransform

Released Transforms will be garbage collected by the system once they are no longer necessary for rendering. For Transforms, this means there is no path from any unreleased Transform to the newly-released Transform.

Once released, the id immediately goes out of scope for future function calls and can be reused when creating new Transforms.

It is an error to call functions with a released id (unless that id has been reused to construct a new Transform).

Request

NameType
transform_id TransformId

ReleaseView

Request

<EMPTY>

ReleaseViewport

Releases a Viewport from the scene, even if the Viewport is still connected to a Transform. Unlike other resources, Viewports are garbage collected by the system during the next Present because a removed Viewport is guaranteed to provide no renderable content.

Use SetContent(transform_id, 0) to clean up references to released Viewports.

Despite having a return type, this function is still feed-forward like CreateView and requires a call to Present to be executed. The ViewportCreationToken will be returned after the presented operations have been executed.

Request

NameType
viewport_id ContentId

Response

NameType
token fuchsia.ui.views/ViewportCreationToken

RemoveChild

Removes a child Transform from a parent Transform.

Request

NameType
parent_transform_id TransformId
child_transform_id TransformId

SetClipBoundary

Sets the bounds, expressed in the local coordinate space of the transform, that constrains the region that content attached to this transform can be rendered to. If the content's area exceeds the clip bounds, the area outside the bounds will not be rendered. These bounds are valid for all children of this transform node as well, which includes nested Flatland instances and their node hierarchies. If a child transform attempts to set clip bounds larger than that of its parent, it will be clipped to the parent's clip bounds. The default state is for a transform to be unclipped, meaning it will not have any bounds placed on its render region. The clip width/height must be positive. Negative values will result in an error. Passing in an empty box to the |rect| parameter will remove the clip bounds.

Request

NameType
transform_id TransformId
rect fuchsia.math/Rect?

SetContent

Setting a piece of Content on a Transform makes that Content visible in the render tree as long as the Transform is visible from the root Transform. The Content will be rendered before, and therefore "behind", any Content attached to the descendants of the Transform.

Because each Transform can have, at most, a single piece of Content on it, calling this function on a Transform that already has Content will replace that Content.

A Content may be set on more than one Transform.

Calling this function with a Content id of 0 will remove any Content currently on the Transform.

Request

NameType
transform_id TransformId
content_id ContentId

SetDebugName

Set debug name of the current client that can be used by Flatland to print as a prefix to logs to help client distinguish what is theirs. name can be an arbitrary string, but the current process name (see fsl::GetCurrentProcessName()) is a good default.

Request

NameType
name string:64

SetHitRegions

Sets the interactive areas for a Transform. By default, Content is not interactive; hit regions must be placed for a user to interact with the Content in a View. Because hit regions are described in the Flatland protocol, a Flatland instance can synchronize Content and hit regions.

Each hit region is placed in the coordinate space of the owning Transform, and may or may not interact with different types of hit testing, depending on its HitTestInteraction type. When there are multiple hit regions that intersect with a hit test, the precedence rules given below dictate which hit region has interacted with the hit test. Only Transforms that transitively connect to the root Transform have their hit regions interact with a hit test.

Calling this function replaces any previous values set on this Transform. To reset a Transform to "no hit testing": send an empty vector with SetHitRegions.

Note that root transforms get a default hit region installed by Flatland. For more details, see the SetRootTransform documentation.

Precedence rules Within a Transform, if multiple hit regions overlap, the hit test examines each intersecting hit region for a possible interaction. Thus, for a regular hit test R and an accessibility hit test A, where R and A both intersect two hit regions D (DEFAULT) and S (SEMANTICALLY_INVISIBLE) on the same Transform, (1) R interacts with both D and S, and (2) A only interacts with D. Generally, hit regions that overlap in a single Transform can cause confusing behavior.

Within a View, for a given hit test, the front-most Transform's hit regions take precedence over those behind. This follows the expected reverse "render order" of Transforms (described in CreateTransform), where a user expects to interact with Content that is visible, or front-most.

Across Flatland instances, for a given hit test, the front-most instance's front-most Transform's hit regions take precedence over those behind. This follows the expected reverse "render order" of views, where a user expects to interact with the View that is visible, or front-most. For example, if a child View owns Content that is rendered over the parent View, the user expects to interact with the child's Content.

Request

NameType
transform_id TransformId
regions vector<HitRegion>:64

SetImageBlendingFunction

Determines the blend function to use when rendering the content specified by |image_id|. |image_id| must be a valid ContentId associated to a transform through a call to |CreateImage| or |CreateFilledRect|. For details on the different blend functions that are available, please refer to the BlendMode enum. If this function is not called, then the default blendmode is BlendMode::SRC.

Request

NameType
image_id ContentId
blend_mode BlendMode

SetImageDestinationSize

The content size for an Image is the size of the rectangle in the parent's logical coordinate space that the image occupies. This combined with the global translation of the transform it is attached to determines the size and location of where the content is rendered on the display.

If this method is not called, the default image destination size is the width and height set at ImageProperties from CreateImage. The destination size will be affected by scaling if SetScale is used on the attached Transform or its parents.

Request

NameType
image_id ContentId
size fuchsia.math/SizeU

SetImageFlip

Sets the image flip to be applied to a flatland image. This call must be performed after a successful |CreateImage| call. If an invalid |image_id| is supplied, the channel will be closed due to FlatlandError::BAD_OPERATION. This flip will be applied to the Image before parent Transform Orientations. If this function is not called, then the default flip value is ImageFlip::NONE.

Added: 10

Request

NameType
image_id ContentId
flip ImageFlip

SetImageOpacity

Sets an opacity in linear space to be applied to a flatland image. Opacity values must be in the range [0.0, 1.0].

Request

NameType
image_id ContentId
val float32

SetImageSampleRegion

This function is used to determine the region (in texel space) of an image that will be used by Flatland when rendering. The image to be sampled is referenced by image_id and the sample region is specified by rect which itself is comprised of an origin point (x,y) as well as a width and height, in unnormalized coordinates. It is illegal to call this function on non-image content, or to sample a region outside of the texel space of the image. In other words, the region specifed by rect must not exceed the ranges (0, image_width) and (0, image_height). If (rect.x + rect.width > image_width) or (rect.y + rect.height > image_height) or if any of the values are negative, this will result in an error.

If this method is not called, the default sample region is the rectangle with origin at (0, 0) and width and height set at ImageProperties from CreateImage.

Request

NameType
image_id ContentId
rect fuchsia.math/RectF

SetInfiniteHitRegion

Identical to SetHitRegions, except the hit region associated with transform_id covers an infinite region. The hit region is invariant against translation, scaling, and orientation of the Transform.

An infinite hit region is still limited in extent by the View's clip boundary, just like a finite hit region.

Calling this function replaces any previous values set on this Transform. To reset a Transform to "no hit testing": send an empty vector with SetHitRegions.

Added: 10

Request

NameType
transform_id TransformId
hit_test HitTestInteraction

SetOpacity

Sets an opacity in linear space to be applied to a transform and its descendents, which include other transforms and content. Opacity values must be in the range of [0.0, 1.0], where 0.0 is completely transparent and 1.0 is completely opaque. Attempting to call this function with values outside that range will result in an error. A transform's opacity value is multiplied with that of its parent. This effect works differently from group opacity. Using group opacity, child nodes are rendered together first, and then have the parent's opacity applied as a post-effect. Here, opacity is applied to each child individually. This may result in a very different effect.

Request

NameType
transform_id TransformId
value float32

SetOrientation

Sets the orientation on a Transform. The order of geometric attribute application is addressed in the documentation for SetTranslation. In Flatland, the +X axis is to the right and the +Y axis is down. There is no notion of a Z axis. CCW is defined from the POV of the user, as if a skeuomorphoic clock is displayed on the screen.

Request

NameType
transform_id TransformId
orientation Orientation

SetRootTransform

Sets the Root Transform for the graph.

The sub-graph defined by the Root Transform and its children will be rendered as View in the connected parent's Viewport (see CreateView). Any parents of the Root Transform in this Graph will be ignored.

The Root Transform, and all children of the Root Transform, are kept alive if they are released (see ReleaseTransform for more details).

There is only ever one Root. Since 0 is not a valid transform id (see CreateTransform), calling SetRootTransform(0) clears the current Root, destroying any previously released objects that are not referenced by the new root.

Note that every View has a clip boundary equivalent to its logical size. Anything outside that clip boundary will not be rendered. Hence, the Root Transform has a useful coordinate space of (0, 0) to (logical_size.width, logical_size.height), where (0, 0) is the upper left corner.

Setting the root transform installs a full screen hit region on the root transform. Clients may remove this hit region if they don't want users to be able to interact with the root transform's content. For additional details on hit regions, see the SetHitRegions documentation.

Default hit region rules

A default hit region follows these rules:

  • When SetRootTransform(T) is called, T receives a maximal hit region, covering the entire view.
  • If SetHitRegions is called on T, either before or after SetRootTransform(T), then no default hit region is active and the client specified hit regions are used.
  • If a transform is no longer the root transform, i.e., SetRootTransform(U) is called, then the original transform no longer has its default hit region.
  • Clients can remove or modify the root transform's hit regions the same way they would reset any other transform's hit regions, by calling SetHitRegions with the appropriate vector.

Request

NameType
transform_id TransformId

SetScale

Sets the scale on a transform. The order of geometric attribute application is addressed above. The (x,y) values in the VecF |scale| refer to the scale factor in the x-axis (width) and y-axis (height) respectively. Scale values must be normal 32-bit floating point values: https://en.wikipedia.org/wiki/Normal_number_%28computing%29

Request

NameType
transform_id TransformId
scale fuchsia.math/VecF

SetSolidFill

Defines the color and size of a filled rect. |rect_id| must refer to content that was created via a call to CreateFilledRect. The color is not premultiplied. Color values must be within the range [0,1] inclusive, and normal 32-bit floating point values: https://en.wikipedia.org/wiki/Normal_number_%28computing%29. Values that do not conform to these specifications will cause the channel to close. The rectangle's top left corner will be at (0, 0) in its transform's coordinate space. Hence, its bottom right corner will be at (size.width, size.height).

Request

NameType
rect_id ContentId
color ColorRgba
size fuchsia.math/SizeU

SetTranslation

All Transform objects support all attributes.

Geometric attributes are applied in the following order:

  1. Scale (relative to the parent transform's coordinate space)
  2. Orientation (relative to the parent transform's coordinate space)
  3. Translation (relative to the parent transforms's coordinate space, unaffected by scale applied to the current transform).
  4. Clipping (relative to the current transform's coordinate space)

The effects of each of these attributes are cumulative. This means the transform's position in the view space, and its clip boundary, will be calculated based on that chain of geometric attributes going up to the root transform.

For instance, in a nested hierarchy such as the following: [Root-Transform -> Transform1 -> Transform2 -> CurrentTransform] If Transform1 is translated by [2,0] and Transform2 is translated by [0,1] then the view-space position of CurrentTransform will be [2,1].

Sets the translation on a Transform. The order of geometric attribute application is addressed above.

Request

NameType
transform_id TransformId
translation fuchsia.math/Vec

SetViewportProperties

Transforms are usually sufficient to change how Content is presented. Viewports, however, have special properties that are not part of the Transform hierarchy. Those properties can be set using this function.

Request

NameType
viewport_id ContentId
properties ViewportProperties

FlatlandDisplay

Defined in fuchsia.ui.composition/flatland.fidl

This API connects to the singleton "primary display", and allows a tree of Flatland content to be attached underneath. Only one FlatlandDisplay client connection is allowed at one time.

SetContent

Request

NameType
token fuchsia.ui.views/ViewportCreationToken
child_view_watcher server_end:ChildViewWatcher

SetDevicePixelRatio

Sets the ratio of display's physical pixels to device independent pixels that should be used for the tree of Flatland content that are attached to this display. See LayoutInfo.device_pixel_ratio for details.

The default value is (1.0, 1.0). The valid values are 1.0 and above.

Added: 9

Request

NameType
device_pixel_ratio fuchsia.math/VecF

ParentViewportWatcher

Defined in fuchsia.ui.composition/flatland.fidl

A protocol that provides information about the parent Viewport attached to a Flatland instance's sole View. Although Flatland instances can have at most one view, it is possible to have multiple ParentViewportWatchers during the brief transition period when replacing the instance's View with another, e.g. via CreateView. During this period, certain updates may be duplicated and sent to each ParentViewportWatcher connection.

A ParentViewportWatcher will remain connected as long as the corresponding parent Viewport exists; the connection will also be closed if the parents's ViewportCreationToken is dropped without using it to create a Viewport.

GetLayout

A hanging get for receiving layout information. Clients may receive layout information before the ParentViewportWatcher operation has been presented. This allows children to layout their content before their first call to Present. In transition cases where two ParentViewportWatcher channels exist at the same time, both protocol instances will be receiving different layout information.

This hanging get will only fire when the LayoutInfo is different than the previously returned LayoutInfo. Note that, since LayoutInfo is a table, only some fields may have changed.

It is invalid to call GetLayout while a previous call is still pending. Doing so will cause both this channel and the Flatland channel that handed out ParentViewportWatcher to be closed.

Request

<EMPTY>

Response

NameType
info LayoutInfo

GetStatus

A hanging get for receiving the status of the parent Viewport. This provides global connectivity information to the child.

This hanging get will only fire when the ParentViewportStatus is different than the previously returned ParentViewportStatus.

It is invalid to call GetStatus while a previous call is still pending. Doing so will cause both this channel and the Flatland channel that handed out ParentViewportWatcher to be closed.

Request

<EMPTY>

Response

NameType
status ParentViewportStatus

ScreenCapture

Defined in fuchsia.ui.composition/screen_capture.fidl

This protocol provides a low-level ScreenCapture API for clients to use. ScreenCapture clients should familiarize themselves with the fuchsia.sysmem/BufferCollection and fuchsia.ui.composition/Allocator protocols as those are necessary to create the BufferCollections and images ScreenCapture uses.

Configure

Clients should first use the Allocator protocol to register a BufferCollection. This function will fail with BAD_OPERATION unless all clients of the BufferCollection have set their constraints.

Afterwards, clients should create and configure the images that will eventually be rendered to using this method. All the buffers in the collection from 0 to (buffer_count-1) may be used for screen capture.

Clients are responsible for determining the rotation of the display, and applying the corrective rotation. For instance, if the display is mounted 90 degrees clockwise (the "top" is on the right, when looking at the display), then the client should specify a 270 degree rotation to account for it.

Similarly, the clients are responsible for specifying a buffer big enough for the rotated image. If the buffer is too small, a best effort attempt will be made to render the image.

Finally, clients request the server to render the current screen to the shared buffers using GetNextFrame.

Configure can be called again with a new BufferCollectionImportToken if the client wishes to change any of the configuration settings. In this case all the buffers from the previous call to Configure will be released.

Request

NameType
payload ScreenCaptureConfig

Response

NameType
payload ScreenCapture_Configure_Result

GetNextFrame

Following a successful call to Configure, clients can call GetNextFrame. This will populate a buffer with the most recent frame.

Clients should wait on the zx::event they pass for successful completion of the screenshot. It is not guaranteed that the screenshot will be completed by the time this function returns.

The requested image will be in the BufferCollection that the client set up in the VMO at the index specified by buffer_id.

When ScreenCapture is used to provide a stream, the rate that the client calls GetNextFrame will drive the frame rate.

Errors: BAD_OPERATION if Configure was not called, or not called successfully MISSING_ARGS if a required argument is not present BUFFER_FULL if all buffers in the BufferCollection are in use. In this case, ReleaseFrame must be called to make a buffer available before this function can be called successfully.

Request

NameType
payload GetNextFrameArgs

Response

NameType
payload ScreenCapture_GetNextFrame_Result

ReleaseFrame

Once the client no longer needs an image, they can call ReleaseFrame on the VMO index of the buffer so that the server can reuse it in the future.

Request

NameType
buffer_id uint32

Response

NameType
payload ScreenCapture_ReleaseFrame_Result

Screenshot

Defined in fuchsia.ui.composition/screenshot.fidl

Collects the current graphical content of a display.

Added: 12

Take

Collects the current graphical content of a display in a specified buffer format in the sRGB color space. Note that the sRGB color space is non-linear, meaning that unit tests doing pixel equality checks making use of non-fully saturated colors should convert to a linear color space.

See https://fuchsia.dev/fuchsia-src/concepts/ui/scenic/color_spaces for more information.

Screenshot is taken immediately, populated with the display's content from the most recent VSYNC.

If the client calls Take a second time before a first Take call returns, the server will close the Screenshot connection with a ZX_ERR_SHOULD_WAIT epitaph.

If capture fails due to an internal error, the server will close the Screenshot connection with a ZX_ERR_INTERNAL epitaph.

Request

NameType
payload ScreenshotTakeRequest

Response

NameType
payload ScreenshotTakeResponse

TakeFile

Collects the current graphical content of a display in a specified buffer format in the sRGB color space. Note that the sRGB color space is non-linear, meaning that unit tests doing pixel equality checks making use of non-fully saturated colors should convert to a linear color space.

TODO(https://fxbug.dev/42065844): Link to fuchsia.dev documentation when it's up.

Screenshot is taken immediately, populated with the display's content from the most recent VSYNC.

If the client calls TakeFile a second time before a first TakeFile call returns, the server will close the Screenshot connection with a ZX_ERR_SHOULD_WAIT epitaph.

If capture fails due to an internal error, the server will close the Screenshot connection with a ZX_ERR_INTERNAL epitaph.

This call should be used if the client is on the host and does not support VMOs, as is the case for ffx tools.

Request

NameType
payload ScreenshotTakeFileRequest

Response

NameType
payload ScreenshotTakeFileResponse

STRUCTS

Allocator_RegisterBufferCollection_Response

Defined in fuchsia.ui.composition/allocator.fidl

<EMPTY>

BufferCollectionExportToken resource

Defined in fuchsia.ui.composition/allocator.fidl

A typed wrapper for an eventpair, representing the registry endpoint of a buffer collection.

FieldTypeDescriptionDefault
value handle<eventpair> No default

BufferCollectionImportToken resource

Defined in fuchsia.ui.composition/allocator.fidl

A typed wrapper for an eventpair, representing the Image import endpoint of a buffer collection.

FieldTypeDescriptionDefault
value handle<eventpair> No default

ColorRgba

Defined in fuchsia.ui.composition/flatland.fidl

Represents a color with alpha channel. Values are unorm (i.e. the valid range is [0,1]), and are in linear color space. They are not gamma-corrected nor premultiplied.

FieldTypeDescriptionDefault
red float32 No default
green float32 No default
blue float32 No default
alpha float32 No default

ContentId

Defined in fuchsia.ui.composition/flatland.fidl

A user-defined identifier for a particular piece of Content. See Content creation functions (e.g. CreateViewport, CreateImage) for more information.

FieldTypeDescriptionDefault
value uint64 No default

HitRegion

Defined in fuchsia.ui.composition/flatland.fidl

An interactive area of a View, placed in the coordinate space of a specific Transform.

FieldTypeDescriptionDefault
region fuchsia.math/RectF

The position and size of this hit region, in the coordinate space of the Transform that owns this hit region.

No default
hit_test HitTestInteraction

The interaction behavior specified for this hit region. To specify "no interaction at all", remove this hit region from the owning Transform.

No default

ScreenCapture_Configure_Response

Defined in fuchsia.ui.composition/screen_capture.fidl

<EMPTY>

ScreenCapture_ReleaseFrame_Response

Defined in fuchsia.ui.composition/screen_capture.fidl

<EMPTY>

TransformId

Defined in fuchsia.ui.composition/flatland.fidl

A user-defined identifier for a particular transform. See CreateTransform and ReleaseTransform for more information.

FieldTypeDescriptionDefault
value uint64 No default

ENUMS

BlendMode strict

Type: uint32

Defined in fuchsia.ui.composition/flatland.fidl

The set of possible blending functions to choose from when determining how an image should be composited on top of other images.

NameValueDescription
1

Indicates that the source pixels replace the destination pixels. In other words, the source pixels are treated as opaque, regardless of what alpha values are set.

2

The source pixels are drawn over the destination pixels. The final pixel color that is displayed is calculated as: C_src + (1.0 - alpha_src) * C_dst).

ChildViewStatus strict

Type: uint32

Defined in fuchsia.ui.composition/flatland.fidl

NameValueDescription
1

The underlying Flatland instance has connected its View, called Present, and the acquisition fences of the Present call have all be reached, indicating that it has some content ready to be displayed, and NOT that the child content has actually been shown on the screen (a common use case is for the parent to wait for the child content to be ready before attaching the child to the global scene graph).

FlatlandError strict

Type: uint32

Defined in fuchsia.ui.composition/flatland.fidl

The set of error codes returned in OnError. Everything except NO_ERROR, causes Flatland channel to be closed.

NameValueDescription
1

Indicates that the parameters used the function calls are invalid.

2

Indicates that additional_present_credits field was not properly processed and the client queued more Presents than allowed.

3

Indicates that the client has overwritten hanging gets in the protocols returned.

HitTestInteraction flexible

Type: uint8

Defined in fuchsia.ui.composition/flatland.fidl

The kind of hit test interaction expected for a hit region.

NameValueDescription
0

The natural default behavior is for a hit region to interact with both regular hit testing and accessibility hit testing.

1

Some use cases require that a hit region to interact with regular hit testing, but not interact with accessibility hit testing. Here, "semantics" refers to accessibility's semantic tree data, which describes UI elements in a View.

ImageFlip strict

Type: uint32

Defined in fuchsia.ui.composition/flatland.fidl

The set of possible image flip functions to choose from when determining how the image texture should be displayed.

Added: 10

NameValueDescription
0
1

Let V be the vertical axis of reflection, positioned at width/2. Then each pixel's x coordinate is reflected across V. The y coordinates remain constant. For instance: |1234| |4321| |abcd| would become |dcba|

2

Let H be the horizontal axis of reflection, positioned at height/2. Then each pixel's y coordinate is reflected across H. The x coordinates remain constant. For instance: |1234| |abcd| |abcd| would become |1234|

Orientation strict

Type: uint32

Defined in fuchsia.ui.composition/flatland.fidl

In Flatland, the +X axis is to the right and +Y axis is down. There is no notion of a Z axis. Due to the lack of a Z axis, there is no mathematical notion of "handedness" (either right or left) with respect to rotation. Thus, we define a counter-clockwise rotation in the same way as if a skeuomorphic clock were to be displayed on the screen, with the hands of said clock moving in a CCW motion from the POV of the observer.

NameValueDescription
1
2
3
4

ParentViewportStatus strict

Type: uint32

Defined in fuchsia.ui.composition/flatland.fidl

ParentViewportWatchers will be informed when they are actively attached to a output display (either directly, or through a chain of parent Viewports) and when they are not. Until they are connected to a display, some pieces of information (such as pixel scale) may be unavailable.

NameValueDescription
1
2

RegisterBufferCollectionError strict

Type: uint32

Defined in fuchsia.ui.composition/allocator.fidl

The set of error codes returned by Allocator::RegisterBufferCollection().

NameValueDescription
1

RegisterBufferCollectionUsage strict

Type: uint32

Defined in fuchsia.ui.composition/allocator.fidl

The possible usages for registered buffer collection.

NameValueDescription
0

DEFAULT means that the specified buffer collection will be used for Flatland and gfx image creation.

See fuchsia.ui.composition/Flatland.CreateImage for more.

1

SCREENSHOT means that the specified buffer collection will be used for screenshotting purposes.

Rotation strict

Type: uint32

Defined in fuchsia.ui.composition/screen_capture.fidl

The rotation to be applied to the image.

If a given display is rotated, say, 270 degrees according to its display_info config file, then applying the equal and opposite rotation, CW_270_DEGREES, should cancel the display rotation leading to a correctly rendered screenshot.

Clients should allocate an image according to the final dimensions they ultimately want to use, i.e. after rotation. These would be identical to the width and height values found in the display_info config file.

NameValueDescription
0
1
2
3

ScreenCaptureError flexible

Type: uint32

Defined in fuchsia.ui.composition/screen_capture.fidl

The possible errors from the ScreenCapture protocol.

NameValueDescription
1

One or more required arguments are missing in the table argument.

2

One or more of the arguments was not valid.

3

A general error occurred during the method call.

4

Error that is returned if GetNextFrame is called when all of the VMOs in the BufferCollection have been rendered to. ReleaseFrame must be called before a successful call to GetNextFrame can be made.

ScreenshotFormat flexible

Type: uint8

Defined in fuchsia.ui.composition/screenshot.fidl

The different formats of Screenshot that can be requested.

Added: 12

NameValueDescription
0

The default format, requesting a tightly packed pixel data with 32 bit BGRA pixels.

2

Requests tightly packed pixel data with 32 bit RGBA pixels.

Added: 25
1

Requests tightly packed pixel data which is compressed into PNG format.

Added: 23

TABLES

FrameInfo resource

Defined in fuchsia.ui.composition/screen_capture.fidl

Metadata about the frame rendered by GetNextFrame.

OrdinalFieldTypeDescription
buffer_id uint32

The index of the VMO where the requested frame has been rendered. Required.

GetNextFrameArgs resource

Defined in fuchsia.ui.composition/screen_capture.fidl

The arguments passed into the GetNextFrame call. All fields are necessary.

OrdinalFieldTypeDescription
event handle<event>

The event that will signal when the requested frame has been rendered. Required.

ImageProperties

Defined in fuchsia.ui.composition/flatland.fidl

The properties of an Image as defined by the client. These properties determine how an Image uses the backing BufferCollection. See CreateImage for more information.

OrdinalFieldTypeDescription
size fuchsia.math/SizeU

The size of the Image in pixels.

LayoutInfo

Defined in fuchsia.ui.composition/flatland.fidl

The return type of GetLayout. This table contains most of the information necessary for a client to decide how to layout their content in a Flatland instance. This data may be provided to the client before the command that creates the View is presented, so that the client may lay out content properly before their first call to Present.

OrdinalFieldTypeDescription
logical_size fuchsia.math/SizeU

The layout size of a View in logical pixels, defined by the parent's call to SetViewportProperties.

The logical size also serves as the clip boundary of the View. Anything outside the clip boundary will not be rendered. Hence, the View's Root Transform has a useful coordinate space of (0, 0) to (logical_size.width, logical_size.height).

Clients should re-layout their content when this value changes.

device_pixel_ratio fuchsia.math/VecF

The ratio of display's physical pixels to device independent pixels. Each logical pixel of a View is displayed on-screen by one or more physical pixels, as determined by this scale. Clients should not necessarily re-layout their content when this value changes, but accommodate by reallocating their Image buffers to avoid sampling artifacts. The HiDPI-aware client should allocate buffers that are sized (logical_size*device_pixel_ratio).

Added: 9
inset fuchsia.math/Inset

The offsets between the edges and the visible rectangle of the View. The clients can assume that the boundary region between the inset and the View edge is occluded, and should adjust content layout to avoid this region. This inset is described in the view's logical coordinate system. The HiDPI-aware clients should scale this by device_pixel_ratio.

Added: 9

OnNextFrameBeginValues

Defined in fuchsia.ui.composition/flatland.fidl

Fields that a client needs in order to produce its next frame, returned in OnNextFrameBegin. Each field is guaranteed to be set and contain valid information.

OrdinalFieldTypeDescription
additional_present_credits uint32

The number of additional Present calls allowed to the client so that they can call Present further times. This is a delta in the present credit budget, not the absolute number of present credits.

future_presentation_infos FuturePresentationInfos

Information about future presentation and latch times that a client may aim for precise scheduling behavior.

PresentArgs resource

Defined in fuchsia.ui.composition/flatland.fidl

Arguments passed into Present. All arguments are optional, and if an argument is omitted Flatland will use a reasonable default, specified below.

OrdinalFieldTypeDescription
requested_presentation_time zx/Time

requested_presentation_time specifies the time on or after which the client would like the enqueued operations to take visible effect (light up pixels on the screen), expressed in nanoseconds in the CLOCK_MONOTONIC timebase.

The default requested_presentation_time is 0.

Using a requested_presentation_time in the present or past (such as 0) schedules enqueued operations to take visible effect as soon as possible, during the next frame to be prepared.

Using a requested_presentation_time in the future schedules the enqueued operations to take visible effect on or as closely as possible after the stated time, but no earlier.

Each rendered frame has a target presentation time. This is when Flatland aims to have the frame presented to the user. Before rendering a frame, Flatland applies all enqueued operations associated with all squashable calls to Present whose requested_presentation_time is on or before the frame's target presentation time.

acquire_fences vector<handle<event>>:16

Flatland will wait until all of a Flatland instance's acquire_fences are ready before it will execute the presented commands. Not signaling acquire_fences will block the current Present as well as the following ones even if their acquire_fences are signaled.

The default acquire_fences value is the empty vector.

release_fences vector<handle<event>>:16

Flatland will signal all release_fences when it is safe to reuse resources which no longer appear in the local scene graph at the time of the current Present. At the latest, this will happen when the local scene graph (checkpointed at this Present) has been integrated into the global scene graph, and the global scene has been displayed on screen.

(Under some circumstances, the fences may be signaled earlier, but clients do not need to worry about this: the fences will only be signaled when it is safe to reuse the associated resources).

These fences are intended to manage the reuse of shared memory resources such as sysmem buffers. For example, it is undesirable for the client to render into an image which is currently displayed on screen, because this may result in graphical artifacts such as tearing.

It is up to the client to maintain the mapping between each fence and the resources which will become reusable when the fence is signaled. A common strategy is to keep track of resources which were used by the previous Present but are no longer used by the current Present. For example, if an image is removed from the scene by the current Present, the client would insert a fence here. When the fence is later signaled, the client knows that it is safe to render into the image and insert it into the local scene graph in a subsequent Present.

If an error occurs, Flatland may close the channel without signaling these fences. Clients may immediately release shared buffers, but they should not immediately modify such buffers, because they may still be displayed on screen. There is currently no good signal available to the client about when it is safe to reuse shared buffers.

The default release_fences value is the empty vector.

unsquashable bool

If unsquashable is true, then the update is guaranteed to be uniquely shown for at least one vsync interval.

If unsquashable is false, then the update can be combined with those that come after it.

If absent, unsquashable is false.

server_wait_fences vector<handle<event>>:16

Unused. Originally intended to be a renaming of acquire_fences, but was never hooked up.

Removed: 24
server_signal_fences vector<handle<event>>:16

Unused. Originally intended to be a renaming of release_fences, but was never hooked up.

Removed: 24

RegisterBufferCollectionArgs resource

Defined in fuchsia.ui.composition/allocator.fidl

The table of arguments for RegisterBufferCollection. Note that some fields below are REQUIRED.

OrdinalFieldTypeDescription
export_token BufferCollectionExportToken

Clients can send export_token to register buffer collections with Allocator to be used later in fuchsia.ui.composition/Flatland instances or other Scenic APIs, such as Screenshot.

For example, by passing a BufferCollectionImportToken containing the matching peer of BufferCollectionExportToken, they can create image resources via fuchsia.ui.composition/Flatland.CreateImage.

Clients should wait for the response before using import_token.

This field is REQUIRED.

buffer_collection_token client_end:fuchsia.sysmem/BufferCollectionToken
Deprecated: 25 Added: 7
usage RegisterBufferCollectionUsage

The client can register a buffer collection for various uses, each coming with their own unique constraints.

This field is OPTIONAL. If usage is omitted it will be treated as if it has the DEFAULT option.

Deprecation

This arg is deprecated at API version 9 with addition of |usages|.

If both the usages and usage fields are set, usages takes precedence and usage is ignored.

Deprecated: 9
usages RegisterBufferCollectionUsages

The client can register a buffer collection for various uses and has the ability to combine usages if multiple are needed.

This field is OPTIONAL. If usages is omitted it will be treated as if it has only the DEFAULT option.

Added: 9
buffer_collection_token2 client_end:fuchsia.sysmem2/BufferCollectionToken

Flatland participates in the allocation of buffers by setting constraints on the BufferCollection referenced by buffer_collection_token. It will not block on buffers being allocated until the client creates content using the BufferCollection.

The buffer collection registered with export_token is available and kept alive as long as the client holds a valid BufferCollectionImportToken. They will be garbage collected when all BufferCollectionImportTokens are closed and all the associated Image resources are released.

Exactly one of buffer_collection_token2 / buffer_collection_token must be set.

Added: 25

ScreenCaptureConfig resource

Defined in fuchsia.ui.composition/screen_capture.fidl

The arguments passed into the Configure call. Note that not all fields are necessary.

OrdinalFieldTypeDescription
import_token BufferCollectionImportToken

The import token referencing a BufferCollection registered with Allocator. Required.

size fuchsia.math/SizeU

The size of the image in pixels. Required.

buffer_count uint32

The number of buffers in the BufferCollection. Required.

rotation Rotation

The rotation to be applied to the stream of images. Optional; if absent no rotation is applied.

ScreenshotTakeFileRequest resource

Defined in fuchsia.ui.composition/screenshot.fidl

OrdinalFieldTypeDescription
format ScreenshotFormat

Format of the requested screenshot.

ScreenshotTakeFileResponse resource

Defined in fuchsia.ui.composition/screenshot.fidl

OrdinalFieldTypeDescription
file client_end:fuchsia.io/File

|fuchsia.io.File| channel used to read the generated screenshot file data. The server side of the channel is stored on the device until ZX_CHANNEL_PEER_CLOSED is detected.

Basic usage: After the client recieves the client end of the file channel, to avoid memory pile-ups, it should finish reading the data before calling TakeFile again. When finished reading, the client should call Close on the |fuchsia.io.File| channel, this will release the memory allocated on the server side.

size fuchsia.math/SizeU

Size of the screenshot in pixels.

ScreenshotTakeRequest resource

Defined in fuchsia.ui.composition/screenshot.fidl

OrdinalFieldTypeDescription
format ScreenshotFormat

Format of the requested screenshot.

ScreenshotTakeResponse resource

Defined in fuchsia.ui.composition/screenshot.fidl

OrdinalFieldTypeDescription
vmo handle<vmo>

CPU mappable read-only VMO that contains screenshot data. The server owns the VMO and may reuse for the next Take. The VMO is guaranteed to be accessible after mapping. In some allocations, VMO::read() might not be available, i.e. on emulator.

Basic usage: After the client receives a VMO handle, to ensure data stability, it should finish reading the VMO before calling Take again. When finished reading, the client should drop the VMO handle.

Advanced usage: To edit the data, or to persist it beyond the next Take call, the client should copy the data to a private VMO.

size fuchsia.math/SizeU

Size of the screenshot in pixels.

ViewBoundProtocols resource

Defined in fuchsia.ui.composition/flatland.fidl

The protocol endpoints bound to a Flatland ViewCreationToken. These protocols operate on the View that ViewCreationToken created in the Flatland session.

OrdinalFieldTypeDescription
view_ref_focused server_end:fuchsia.ui.views/ViewRefFocused

Learn when a View gains focus.

Server-bound ViewRef. The view_ref_focused client does not specify the ViewRef explicitly; instead, the server implementation uses the ViewRef used in View creation for reporting focus movement on/off this View.

view_focuser server_end:fuchsia.ui.views/Focuser

Enable a View to request focus transfer to a child (target) View.

Server-bound ViewRef. The view_focuser client does not specify the "requestor" ViewRef explicitly, only the "target" ViewRef. Instead, the server implementation uses the ViewRef used in View creation as the "requestor" ViewRef.

touch_source server_end:fuchsia.ui.pointer/TouchSource

Receive touch events that are associated with a View.

mouse_source server_end:fuchsia.ui.pointer/MouseSource

Receive mouse events that are associated with a View.

ViewportProperties

Defined in fuchsia.ui.composition/flatland.fidl

The properties of a Viewport as defined by the parent. This data, along with the set of attached Transforms, will be used to compute the LayoutInfo for the View of the Viewport. ViewportProperties must have logical_size set at least once. This is the initial size that will drive the layout of the child.

OrdinalFieldTypeDescription
logical_size fuchsia.math/SizeU

The size of the Viewport in logical pixels. This maps directly to the logical_size field in LayoutInfo. The valid logical_size must have positive X and Y components.

inset fuchsia.math/Inset

The offsets between the edges and the visible rectangle of the Viewport. This maps directly to the inset field in LayoutInfo. The valid inset must have all components greater than or equal to 0.

Added: 9

UNIONS

Allocator_RegisterBufferCollection_Result strict

Defined in fuchsia.ui.composition/allocator.fidl

OrdinalVariantTypeDescription
response Allocator_RegisterBufferCollection_Response
err RegisterBufferCollectionError

ScreenCapture_Configure_Result strict

Defined in fuchsia.ui.composition/screen_capture.fidl

OrdinalVariantTypeDescription
response ScreenCapture_Configure_Response
err ScreenCaptureError

ScreenCapture_GetNextFrame_Result strict resource

Defined in fuchsia.ui.composition/screen_capture.fidl

OrdinalVariantTypeDescription
response FrameInfo
err ScreenCaptureError

ScreenCapture_ReleaseFrame_Result strict

Defined in fuchsia.ui.composition/screen_capture.fidl

OrdinalVariantTypeDescription
response ScreenCapture_ReleaseFrame_Response
err ScreenCaptureError

BITS

RegisterBufferCollectionUsages flexible

Type: uint16

Defined in fuchsia.ui.composition/allocator.fidl

The possible usages for a registered buffer collection. Can be a combination of options.

Added: 9

NameValueDescription
1

The specified buffer collection can be used for Flatland and GFX image creation.

See fuchsia.ui.composition/Flatland.CreateImage for more.

2

The specified buffer collection can be used for screenshotting purposes.

CONSTANTS

NameValueTypeDescription
MAX_ACQUIRE_RELEASE_FENCE_COUNT 16 int32
MAX_HIT_REGION_COUNT 64 int32

A maximum of 64 hit regions is enough for the expected usage of these APIs.

MAX_PRESENT_ARGS_FENCE_COUNT 16 int32

ALIASES

NameValueDescription
FuturePresentationInfos vector[8]

A user-defined identifier for future presentation info. A maximum of 8 future presentation counts is enough for the current usage of these APIs.