fuchsia.mediacodec

Decoder or encoder.

The mime type of the compressed format. For decoders this is the mime type of the input. For encoders, this is the mime type of the output.

For each of these fields, the default is the most-capable setting, but if a codec doesn't support the most-capable behavior, then the codec must override the default.

DECODER

ENCODER

OFF

ON

1

1

Input mime type for a decoder.

The recognized mime types for now: video/h264 video/vp9 audio/aac input_details.oob_bytes must be an AudioSpecificConfig() as defined by AAC spec. audio/sbc input_details.oob_bytes must be Codec Specific Information Elements for SBC as defined by the A2DP spec.

2

This must be true in order for the client to be permitted to put a timestamp on an input packet, which is in turn required to get any timestamps on any output packets.

It is always legal to provide separate Access Units (henceforth AUs) to a decoder, but this boolean must be true for a decoder to accept and propagate timestamp values.

This must be true when creating a video encoder, or the CodecFactory channel will close.

If not set, interpreted as false.

3

Require that the selected codec be capable of accepting input where AUs are not separated into separate packets.

This does not imply that the decoder can find the start of the first AU; for that see require_can_find_start. This does not imply that the decoder can re-sync on its own if the stream data is damaged; for that see require_can_re_sync.

If both promise_separate_access_units_on_input and require_can_stream_bytes_input are true, the CodecFactory channel will close.

If this is false, the client must feed separate AUs on the fuchsia.ui.input. This must be false for a video encoder, and if true the CodecFactory channel will close.

Unless a client demands a decoder capable of taking concatenated AUs (require_can_stream_bytes_input true), the client must feed a decoder separate AUs. This means the client cannot have parts of two separate AUs in the same packet, unless require_can_stream_bytes_input is true.

If not set, interpreted as false.

4

A decoder is allowed to be capable of streaming bytes but not capable of searching for the start of the first usable AU. To require both, set both require_can_stream_bytes_input and require_can_find_start. Setting require_can_find_start without require_can_stream_bytes_input is invalid.

With require_can_stream_bytes_input true but require_can_find_start false, the client must start the first packet with the start of an AU, but can send a stream of bytes after that.

If not set, interpreted as false.

5

On problematic input data, all decoders are expected to at least be able to close the channel rather than getting stuck in a failed and/or broken state.

A decoder returned from a request with require_can_re_sync is potentially able to handle damaged input without closing the Codec channel. Such a Codec is encouraged, but not required, to also satisfy requirements of require_report_all_detected_errors.

If not set, interpreted as false.

6

Sometimes a client would rather fail an overall use of a decoder than fail to notice data corruption. For such scenarios, the client can specify require_report_all_detected_errors. For any codec returned from a request with require_report_all_detected_errors set, on detection of any input data corruption the codec will report in one or more of these ways:

• closing the Codec channel
• OnStreamFailed()
• error_detected_before
• error_detected_during

If false, a codec may silently skip past corrupted input data.

No decoder can detect all corruption, because some corruption can look like valid stream data. This requirement is only to request a codec that is written to attempt to detect and report input stream corruption.

This flag is not intended to be 100% bulletproof. If a client needs robust assurance that all detectable stream corruption is always detected, this flag is not enough of a guarantee to achieve that. Since some stream corruption is inherently non-detectable in any case, such a client should consider using stronger techniques upstream to ensure that corruption can be detected with the needed probability very close to 1.

This flag being true doesn't imply anything about whether the codec will discard damaged data vs. producing corresponding damaged output. Only that the codec will set error_detected_* bools to true when appropriate.

Regardless of this setting, not all timestamp_ish values provided on input are guaranteed to show up on output.

If not set, interpreted as false.

7

If true, require that the returned codec is HW-accelerated. See also require_sw.

If not set, interpreted as false.

8

permit_lack_of_split_header_handling

This field is a temporary field that will be going away.

TODO(dustingreen): Remove this field once we're down to zero codecs with problems handling split headers.

By default, a Codec instance is required to handle "split headers", meaning that a client is allowed to deliver parts of an AU one byte at a time, including parts near the beginning of the AU, and the codec is required to tolerate and handle that properly. However, unfortunately not all codecs properly support split headers. If a client is willing to permit such a codec to be used, the client can set this to true. Clients are not encouraged to set this, but setting it may be necessary to find a codec for some formats for now. If a client sets this to true, the client should deliver data of each AU with many contiguous non-split bytes from the start of each AU. The client is not strictly required to deliver one AU at a time, only to ensure that either all the AU bytes are in a single packet or that many bytes at the start of each AU are in a single packet.

The specification for how a client should use this and how a client should behave if setting this to true is intentionally vague, because lack of support for header splitting is not ideal, and is expected to be temporary, and all codecs should handle split headers in the long run. The main intent of this field is to avoid giving an innocent client using default value of false here a codec that can't properly handle split headers. This is not an attempt at a mechanism to fully work around a codec that doesn't handle split headers.

If not set, interpreted as false.

9

If set to ON, the decoder must support secure buffers on output, and must reject non-secure buffers on output.

If set to OFF or not set, the created decoder will reject secure buffers on output by closing the StreamProcessor channel.

If secure_input_mode ON, secure_output_mode must also be ON.

10

If set to ON, the decoder must support secure buffers on input and must reject non-secure buffers on input.

If set to OFF or not set, the created decoder will reject secure buffers on input by closing the StreamProcessor channel.

If secure_input_mode ON, secure_output_mode must also be ON.

11

If true, require that the returned codec is entirely SW-based, not HW-accelerated (other than possibly using vector CPU instructions). This can be useful for testing purposes or other special scenarios, but is not recommended for performance-sensitive scenarios. Also, some builds may lack a SW-based decoder for some formats. See also require_hw.

If not set, interpreted as false.

1

The format of the uncompressed input data.

This field should be a raw mime_type (e.g. 'video/raw') and uncompressed format details for the encoder to use when reading buffers.

To be elibigible an encoder must support the input format.

2

If true, require that the returned codec is HW-accelerated.

If not set, interpreted as false.

1

The codec profile that this decoder supports. If the client wants to use this profile, the requirements specified in this table must be adhered to.

2

The minimum image size this decoder supports for the given |profile|. Decoders must set this field, and this field must specify a size that is >= the min size defined for the profile in the codec spec.

This size refers to the pixel layout in memory, not the display_rect which can be smaller than the fuchsia.Images2.ImageFormat.size / fuchsia.sysmem.ImageFormat2.coded_width/height.

3

The maximum image size this decoder supports for the given |profile|.

Decoders must set this field, and this field must specify a size that is <= the max size defined for the profile in the codec spec.

This size refers to the pixel layout in memory, not the display_rect which can be smaller than the fuchsia.Images2.ImageFormat.size / fuchsia.sysmem.ImageFormat2.coded_width/height.

By setting this field, a decoder is not required to fail to decode a stream that specifies a size that is larger than the profile of the stream would normally allow. The decoder may or may not fail to decode a stream which is not a listed profile or not within the size bounds of a listed profile.

4

This |profile| entry can apply to encrypted input data. If require_encryption is false or unset, this |profile| entry can also apply to unencrypted input data.

This will be un-set (even when allow_input_protection or require_input_protection are true) until Fuchsia supports decryption as part of decode (in contrast to it being a separate step involving protected memory in between). When allow_encryption is false/un-set but allow_input_protection is true, a client setting up DRM decode should set up decryption as a separate step prior to decode with protected memory in between the decrypt and decode.

5

This |profile| entry applies only when input data is encrypted. This |profile| entry does not apply for unencrypted input data. Un-set is interepted as false.

If this is set to true and there is no profile with require_encryption false in the same DetailedCodecDescription, then this decoder only supports encrypted input.

This will be un-set until Fuchsia supports decryption as part of decode (even when allow_input_protection or require_input_protection are true). See also allow_encryption, allow_input_protection, require_input_protection.

6

This |profile| entry can apply when input data delivered via protected memory. Whether to protect input and which protected "heap" to use (if protecting input) is determined during sysmem constraints aggregation and via DRM mechanisms. See also require_input_protection, allow_encryption, require_encryption.

7

This |profile| entry applies only when input data is delivered via protected memory. Whether to protect input and which protected "heap" to use (if protecting input) is determined during sysmem constraints aggregation, and by separate DRM mechanisms.

If this is set to true and there is no profile with require_input_protection false in the same DetailedCodecDescription, then this decoder only supports protected input.

Output protection is negotiated separately during output buffer constraints aggregation in sysmem, and via DRM mechanisms.

See also allow_input_protection, allow_encryption, require_encryption.

8

If set to true, the decoder can handle an input chunk payload (containing compressed data) that's split across a packet boundary (between header and payload, or between payload bytes), and the decoder can also handle more than one input chunk containing compressed input data in a single input packet. See also split_header_handling to determine if the decoder can also handle split headers.

Un-set means false.

While this field always indicates whether it's ok to split an input chunk across a packet boundary or not, and this field being set always implies it's ok to have a single input packet with bytes from more than one input chunk including when the input chunks contain compressed input data, there is some additional somewhat-subtle meaning that differs between video and audio decoders.

Audio decoders are always required to permit more than one input chunk in a single input packet, including mulitple input chunks which each contain compressed input data (not just prepended "context" headers), regardless of this field being set or unset. For audio decoders, this field only indicates whether splitting an input chunk across packets is allowed.

Audio decoders shouldn't set split_header_handling to true unless they also set can_stream_bytes_input to true, since allowing splitting header bytes is meaningless unless it's allowed to split an input chunk.

For video decoders, if this field is un-set or false, the decoder may not be able to handle bytes of more than one input chunk in a single input packet. However, for all decoders (both video and audio) it's always ok for preceding "context" headers such as the h264 SPS and PPS headers to be in the same input packet as the following input chunk containing compressed input data such as an h264 slice (for example). A video decoder is always required to permit continuation of an input chunk in the next packet, with the split between header byte and compressed input data byte or between two compressed input data bytes (but not necessarily between two header bytes; see also split_header_handling).

It's expected to be fairly common for a decoder to set can_stream_bytes_input to true, but leave split_header_handling un-set or set split_header_handling to false, due to parsing limitations in the HW, FW, or driver. Decoders which set can_stream_bytes_input false or leave can_stream_bytes_input un-set may be forcing input data to be re-packed in some scenarios. Profiles which set allow_encryption are encouraged to also set can_stream_bytes_input if feasible, as re-packing input data can be more difficult in some scenarios involving encryption.

9

If set to true, the decoder can scan forward at the start of a stream to find the start of the first fully-present input chunk, even if the input data starts at a byte that's in the middle of a chunk.

For both video and audio decoders, setting this field to true also indicates the ability to handle (skip or only partly use) any input chunks that are not possible to decode (or not possible to fully decode) due to lack of referenced prior data.

If un-set or false, the decoder may not be able to scan forward to sync up with the input stream unless the input stream starts with the beginning of a suitable chunk.

10

If set to true, the decoder can resynchronize with the input stream (eventually) despite a missing input chunk, and can handle partial input chunks without failing the stream or the StreamProcessor control connection.

11

If set to true, the decoder makes efforts to indicate partial or detected-as-missing or detected-as-corrupt input chunks by setting error_detected_before and/or error_detected_during as appropriate. Even when this field is set to true, it is worth noting that in general it is not possible for a decoder to detect all possible data corruptions, as codecs generally aren't inclined to include error detection bits within the stream. This is not a replacment for a real (as in robust with high probability against random bit flips and bit insertion/deletion) data integrity check.

12

If set to true, the decoder can handle the bytes of any header being split across packet boundaries. If false or un-set, the decoder requires all header bytes of a header to be within the same packet.

Starting a new packet at the spec-defined boundary between two headers that are officially in separate input chunks is always permitted.

For purposes of this field, headers like the h264 SPS and PPS headers are considered separate input chunks, and so such headers that have their own input chunk can be delivered in a separate packet regardless of the setting of this field, and a subsequent h264 slice header can be in yet another packet.

It is always permissible regardless of the setting of this field for "context" headers (such as h264 SPS and PPS headers) to be delivered in the same packet as the subsequent chunk which conveys compressed image data (such as an h264 slice).

Splitting a single header between codec_oob_bytes and the stream data is never supported (as in, never to be relied on by clients), not even when this field is set to true.

See also can_stream_bytes_input.

Video decoders with can_stream_bytes_input true but split_header_handling false can handle continuing an chunk in the next packet when split between header and payload or between payload bytes, and can handle bytes of more than one input chunk in a single input packet, but cannot handle splitting a header across packets. Video decoders with can_stream_bytes_input false and split_header_handling true cannot tolerate bytes of more than one input chunk in a single packet, but can tolerate continuation of an input chunk in a following packet regardless of where the split occurs (such as in the middle of an h264 SPS, PPS, or slice header).

Audio decoders shouldn't set split_header_handling to true unless they also set can_stream_bytes_input to true, since for an audio decoder, allowing splitting header bytes is meaningless (for audio decoders) unless also allowing splitting an input chunk.

1

Decoder or encoder.

2

The mime type of the compressed format. For decoders this is the mime type of the input. For encoders, this is the mime type of the output.

3

If this decoder/encoder uses underlying hardware to perform its operations.

4

A list of profile descriptions that describe what codec profiles this encoder/decoder supports along with requirements for using each profile.

1

1

A list of |DecoderProfileDescription| that describe what codec profiles this decoder supports along with requirements that must be adhered to if the client is to use the decoder. The CodecFactory guarantees to the client that each |DecoderProfileDescription| within |decoder_profile_descriptions| will have an unique |profile|.

2

Constrain the length of the vector indicated via OnCodecList, and constrain the max items in any vector within the response from GetDetailedCodecDescriptions. The overall enforced limit for codec caps information is the max channel message size however, since nested vectors each of max size could otherwise exceed the max channel message size.

More than this many calls to AttachLifetimeTracking() without a Create{X}() call will result in CodecFactory channel closing from the server end.

Constrain the mime_type to a size that won't cause problems.