fuchsia.images2

Added: 12

STRUCTS

RectU

Defined in fuchsia.images2/math.fidl

An integral, rectangular, axis-aligned region in a 2D cartesian space, with unsigned location and distance fields.

This type does not specify units. Protocols that use this type should specify the characteristics of the vector space, including orientation and units.

Removed: 18 Added: 12

FieldTypeDescriptionDefault
x uint32

The location of the origin of the rectangle in the x-axis.

No default
y uint32

The location of the origin of the rectangle in the y-axis.

No default
width uint32

The distance along the x-axis.

The region includes x values starting at x and increasing along the x-axis.

No default
height uint32

The distance along the y-axis.

The region includes y values starting at y and increasing along the y-axis.

No default

ENUMS

ColorSpace flexible

Type: uint32

Defined in fuchsia.images2/image_format.fidl

Expresses the color space used to interpret video pixel values.

This list has a separate entry for each variant of a color space standard.

For this reason, should we ever add support for the RGB variant of 709, for example, we'd add a separate entry to this list for that variant. Similarly for the RGB variants of 2020 or 2100. Similarly for the YcCbcCrc variant of 2020. Similarly for the ICtCp variant of 2100.

See ImageFormatIsSupportedColorSpaceForPixelFormat() for whether a combination of PixelFormat and ColorSpace is potentially supported.

Generally, a ColorSpace is not supported for any PixelFormat whose bits-per-sample isn't compatible with the color space's spec, nor for any PixelFormat which is a mismatch in terms of RGB vs. YUV.

The "limited range" in comments below refers to where black and white are defined to be (and simimlar for chroma), but should not be interpreted as guaranteeing that there won't be values outside the nominal "limited range". In other words, "limited range" doesn't necessarily mean there won't be any values below black or above white, or outside the "limited" chroma range. For "full range", black is 0 and white is the max possible/permitted numeric value (and similar for chroma).

NameValueDescription
0

Not a valid color space type.

1

sRGB, gamma transfer function and full range, per spec

2

601 NTSC ("525 line") YCbCr primaries, limited range

3

601 NTSC ("525 line") YCbCr primaries, full range

4

601 PAL ("625 line") YCbCr primaries, limited range

5

601 PAL ("625 line") YCbCr primaries, full range

6

709 YCbCr (not RGB), limited range

7

2020 YCbCr (not RGB, not YcCbcCrc), 10 or 12 bit according to PixelFormat, with primaries, limited range (not full range), transfer function ("gamma"), etc all per spec, wide color gamut SDR

8

2100 YCbCr (not RGB, not ICtCp), 10 or 12 bit according to PixelFormat, BT.2020 primaries (same wide color gamut as REC2020), limited range (not full range), PQ (aka SMPTE ST 2084) HDR transfer function (not HLG, not SDR "gamma" used by REC2020 and REC709), wide color gamut HDR

9

Either the pixel format doesn't represent a color, or it's in an application-specific colorspace that isn't describable by another entry in this enum.

4294967294

A client is explicitly indicating that the client does not care which color space is chosen / used.

PixelFormat flexible

Type: uint32

Defined in fuchsia.images2/image_format.fidl

Expresses the manner in which video pixels are encoded.

The ordering of the channels in the format name reflects the actual layout of the channel.

Each of these values is opinionated re. the color spaces that should be contained within (in contrast with Vulkan).

NameValueDescription
0
1

RGB only, 8 bits per each of R/G/B/A sample

Compatible with VK_FORMAT_R8G8B8A8_UNORM.

119

RGB only, 8 bits per each of R/G/B/X sample

Compatible with VK_FORMAT_R8G8B8A8_UNORM, when treated as opaque.

101

32bpp BGRA, 1 plane. RGB only, 8 bits per each of B/G/R/A sample.

Compatible with VK_FORMAT_B8G8R8A8_UNORM.

In sysmem(1), this is BGRA32.

120

32bpp BGRA, 1 plane. RGB only, 8 bits per each of B/G/R/X sample.

Compatible with VK_FORMAT_B8G8R8A8_UNORM, when treated as opaque.

102

YUV only, 8 bits per Y sample

Compatible with VK_FORMAT_G8_B8_R8_3PLANE_420_UNORM.

103

YUV only, 8 bits per Y sample

Not compatible with any vulkan format.

104

YUV only, 8 bits per Y sample

Compatible with VK_FORMAT_G8_B8R8_2PLANE_420_UNORM.

105

YUV only, 8 bits per Y sample

Compatible with VK_FORMAT_G8B8G8R8_422_UNORM.

106

This value is reserved, and not currently used.

107

YUV only, 8 bits per Y sample

Compatible with VK_FORMAT_G8_B8_R8_3PLANE_420_UNORM with B and R swizzled.

108

24bpp BGR, 1 plane. RGB only, 8 bits per each of B/G/R sample

Compatible with VK_FORMAT_B8G8R8_UNORM.

In sysmem(1), this is BGR24.

109

16bpp RGB, 1 plane. 5 bits R, 6 bits G, 5 bits B

Compatible with VK_FORMAT_R5G6B5_UNORM_PACK16.

In sysmem(1), this is RGB565.

110

8bpp RGB, 1 plane. 3 bits R, 3 bits G, 2 bits B

Not compatible with any vulkan format.

In sysmem(1), this is RGB332.

111

8bpp RGB, 1 plane. 2 bits R, 2 bits G, 2 bits B

Not compatible with any vulkan format.

In sysmem(1), this is RGB2220.

112

8bpp, Luminance-only (red, green and blue have identical values.)

Compatible with VK_FORMAT_R8_UNORM.

Most clients will prefer to use R8 instead.

113

8bpp, Red-only (Green and Blue are to be interpreted as 0).

Compatible with VK_FORMAT_R8_UNORM.

114

16bpp RG, 1 plane. 8 bits R, 8 bits G.

Compatible with VK_FORMAT_R8G8_UNORM.

115

32bpp RGBA, 1 plane. 2 bits A, 10 bits R/G/B.

Compatible with VK_FORMAT_A2R10G10B10_UNORM_PACK32.

116

32bpp BGRA, 1 plane. 2 bits A, 10 bits R/G/B.

Compatible with VK_FORMAT_A2B10G10R10_UNORM_PACK32.

117

YUV only, 16 bits per Y sample

This is like NV12 but with 16 bit samples that have the bottom 6 bits of each sample set to zero and/or ignored. The endianess of each 16 bit sample is host endian-ness (LE on LE system, BE on BE system). The CbCr plane has 16 bit Cb first, then 16 bit Cr, interleaved Cb Cr Cb Cr etc.

Compatible with VK_FORMAT_G10X6_B10X6R10X6_2PLANE_420_UNORM_3PACK16.

118

24bpp RGB, 1 plane. RGB only, 8 bits per each of R/G/B sample

Compatible with VK_FORMAT_R8G8B8_UNORM.

4294967294

A client is explicitly indicating that the client does not care which pixel format is chosen / used. When setting this value, the client must not set pixel_format_modifier.

PixelFormatModifier flexible

Type: uint64

Defined in fuchsia.images2/format_modifier.fidl

The upper 8 bits are a vendor code. The lower 56 bits are vendor-defined.

The defined PixelFormatModifier values are specific, complete, and valid values (except for INVALID and DO_NOT_CARE which have their own meanings).

Some other valid or potentially-valid pixel_format_modifier values are not defined as a PixelFormatModifier value, typically because the value isn't used in practice (or potentially is newly used but not yet defined in PixelFormatModifier). It is permitted to specify such a value as a PixelFormatModifier value in a pixel_format_modifier field, despite the lack of corresponding defined PixelFormatModifier value. If such a value is used outside test code, please consider adding it as a defined value in PixelFormatModifier. All such values must conform to the upper 8 bits vendor code (don't define/use values outside the/an appropriate vendor code).

The separately-defined FORMATMODIFIER* uint64 values are vendor-specific bit field values, not complete valid values on their own. These uint64 values can be used to help create or interpret a PixelFormatModifier value in terms of vendor-specific bitfields.

When the pixel_format_modifier is set to a supported value (excluding DO_NOT_CARE, INVALID, LINEAR), the arrangement of pixel data otherwise specified by the pixel_format field is "modified", typically to allow for some combination of tiling, compression (typically lossless, typically for memory bandwidth reduction not framebuffer size reduction), transaction elimination, dirt tracking, but typically not modifying the bit depth of the pixel_format. In some cases there's a per-image or per-tile header involved, or similar. The pixel_format field often still needs to be set to a valid supported value that works in combination with the pixel_format_modifier, and that pixel_format value can also contribute to the overall meaning of the ImageFormat. In other words, the "modifier" part of the name is more accurate than "override" would be.

NameValueDescription
72057594037927934
72057594037927935
0
72057594037927937
72057594037927938
72057594037927939
72057594054705154
72057594054705155
576460752303423489
576460752303423490
576460752303427584
576460752303427585
576460752303427586
576460752303431697
576460752303423601
576460752303427697
576460752303431793
576460752303435889
7421932185906577409

TABLES

ImageFormat

Defined in fuchsia.images2/image_format.fidl

Describes the format of images.

OrdinalFieldTypeDescription
pixel_format PixelFormat

Describes the manner in which pixels are encoded.

pixel_format_modifier PixelFormatModifier

Vendor-specific pixel format modifier. See format_modifier.fidl.

color_space ColorSpace

Indicates the color space used to interpret pixel values.

size fuchsia.math/SizeU

The size of the image in pixels.

See also bytes_per_row which is also necessary (along with size) to find where each pixel's data is within a buffer.

Not all of the addressable pixel positions in the buffer are necessarily populated with valid pixel data. See valid_size for the potentially-smaller rectangle of valid pixels.

The right and bottom of the image may have some valid pixels which are not to be displayed. See display_rect.

bytes_per_row uint32

Number of bytes per row. For multi-plane YUV formats, this is the number of bytes per row in the Y plane.

When this field is not set, there is no padding at the end of each row of pixels. In other words, when not set, the stride is equal to the "stride bytes per width pixel" times the size.width.

When set, the value in this field must be >= the "stride bytes per width pixel" times the size.width. If equal, there is no padding at the end of each row of pixels. If greater, the difference is how much padding is at the end of each row of pixels, in bytes.

This is also known as the "stride", "line to line offset", "row to row offset", and other names.

As a specific example, it's not uncommon (but also not always required) for BGR24 (3 bytes per pixel) to have some padding at the end of each row so that each row of pixels starts at a 4 byte aligned offset from the start of the image (the upper left pixel). That padding's size is not necessarily divisible by the size in bytes of a pixel ("stride bytes per width pixel"), so we indicate the padding using this field rather than trying to incorporate the padding as a larger "fake" size.width.

display_rect fuchsia.math/RectU

The rect within a frame that's for display. This is the location and size in pixels of the rectangle of pixels that should be displayed, when displaying the "whole image" in a UI display sense.

The x + width must be <= size.width, and the y + height must be <= size.height.

For output from a video decoder, pixels outside the display_rect are never to be displayed (outside of test programs), but must be preserved for correct decoder function. The display_rect will always fall within the rect starting at (0, 0) and having valid_size size, when valid_size is set. In other words, display_rect is a subset (not necessarily a proper subset) of valid_size, and valid_size is a subset (not necessarily a proper subset) of size.

Downstream texture filtering operations should avoid letting any pixel outside of display_rect influence the visual appearance of any displayed pixel, to avoid the potential for the right or bottom edge leaking in arbitrary pixels defined by the decode process but not intended for display.

Behavior when this field is not set is protocol-specific. In some protocols, fallback to valid_size, then to size may be implemented. In others, fallback directly to size may be implemented. In others, this field must be set or the channel will close.

WARNING: fuchsia.sysmem.Images2 (V1) doesn't handle non-zero x, y, so any non-zero x, y here (V2) will prevent conversion to V1. Due to the rarity of non-zero x, y in practice, even components that have moved to V2 may in some cases still assume both x and y are 0, until there's a practical reason to implment and test handling of non-zero x, y. The symptom of sending non-zero x, y to a downstream render and/or display pipeline that assumes 0, 0 will be incorrect display, but not a crash, since assuming 0, 0 for x, y does not cause reading out of buffer bounds.

Added: 18
valid_size fuchsia.math/SizeU

The size of a frame in terms of the number of pixels that have valid pixel data in terms of video decoding, but not in terms of which pixels are intended for display.

To convert valid_size into a rect that's directly comparable to display_rect, one can make a rect with (x: 0, y: 0, width: valid_size.width, height: valid_size.height).

In the case of a video decoder, valid_size can include some pixels outside display_rect. The extra pixels are not meant to be displayed, and may or may not contain any real image data. Typically anything that looks like real image data in these regions is only an artifact of video compression and the existence of the remainder of a macroblock which can be referenced by later frames despite not being within the displayed region, and not really any additional "real" pixels from the source. The pixel values in this region are defined by the codec decode process and must be retained for correct decoder operation. Typically the pixels inside valid_size but outside display_rect will be up to the size of a macroblock minus 1. The valid_size is can be useful for testing video decoders and for certain transcoding scenarios.

pixel_aspect_ratio fuchsia.math/SizeU

Aspect ratio of a single pixel as the video is intended to be displayed.

For YUV formats, this is the pixel aspect ratio (AKA sample aspect ratio aka SAR) for the luma (AKA Y) samples.

Producers should ensure the width and height values are relatively prime by reducing the fraction (dividing both by GCF) if necessary.

A consumer should interpret this field being un-set as an unknown pixel aspect ratio. A default of 1:1 can be appropriate in some cases, but a consumer may determine the actual pixel aspect ratio by OOB means.

CONSTANTS

NameValueTypeDescription
FORMAT_MODIFIER_ARM_BCH_BIT 2048 uint64
FORMAT_MODIFIER_ARM_SPARSE_BIT 64 uint64
FORMAT_MODIFIER_ARM_SPLIT_BLOCK_BIT 32 uint64
FORMAT_MODIFIER_ARM_TE_BIT 4096 uint64
FORMAT_MODIFIER_ARM_TILED_HEADER_BIT 8192 uint64
FORMAT_MODIFIER_ARM_YUV_BIT 16 uint64
FORMAT_MODIFIER_INTEL_CCS_BIT 16777216 uint64

Format has a color control surface after the tile data

FORMAT_MODIFIER_VENDOR_ALLWINNER 648518346341351424 uint64
Added: 19
FORMAT_MODIFIER_VENDOR_AMD 144115188075855872 uint64
Added: 19
FORMAT_MODIFIER_VENDOR_AMLOGIC 720575940379279360 uint64
Added: 19
FORMAT_MODIFIER_VENDOR_ARM 576460752303423488 uint64
Added: 19
FORMAT_MODIFIER_VENDOR_BROADCOM 504403158265495552 uint64
Added: 19
FORMAT_MODIFIER_VENDOR_GOOGLE 7421932185906577408 uint64
Added: 19
FORMAT_MODIFIER_VENDOR_INTEL 72057594037927936 uint64
Added: 19
FORMAT_MODIFIER_VENDOR_NVIDIA 216172782113783808 uint64
Added: 19
FORMAT_MODIFIER_VENDOR_QCOM 360287970189639680 uint64
Added: 19
FORMAT_MODIFIER_VENDOR_SAMSUNG 288230376151711744 uint64
Added: 19
FORMAT_MODIFIER_VENDOR_VIVANTE 432345564227567616 uint64
Added: 19