fuchsia.media

Added: 7

PROTOCOLS

ActivityReporter

Defined in fuchsia.media/activity_reporter.fidl

A protocol for monitoring the usage activity of the AudioRenderers and AudioCapturers.

WatchCaptureActivity

Notifies the client whenever there is a change in the set of active AudioCaptureUsages. It returns immediately the first time that it is called.

Request

<EMPTY>

Response

NameType
active_usages vector<AudioCaptureUsage>[4]

WatchRenderActivity

Notifies the client whenever there is a change in the set of active AudioRenderUsages. It returns immediately the first time that it is called.

Request

<EMPTY>

Response

NameType
active_usages vector<AudioRenderUsage>[5]

Audio

Defined in fuchsia.media/audio.fidl

CreateAudioCapturer

Creates an AudioCapturer which either captures from the current default audio input device, or loops-back from the current default audio output device based on value passed for the loopback flag.

Request

NameType
audio_capturer_request server_end<AudioCapturer>
loopback bool

CreateAudioRenderer

Request

NameType
audio_renderer_request server_end<AudioRenderer>

SetSystemGain

DEPRECATED

Request

NameType
gain_db float32

SetSystemMute

DEPRECATED

Request

NameType
muted bool

SystemGainMuteChanged

DEPRECATED

Response

NameType
gain_db float32
muted bool

AudioCapturer

Defined in fuchsia.media/audio_capturer.fidl

AudioCapturer

An AudioCapturer is an interface returned from an fuchsia.media.Audio's CreateAudioCapturer method, which may be used by clients to capture audio from either the current default audio input device, or the current default audio output device depending on the flags passed during creation.

Format support

See (Get|Set)StreamType below. By default, the captured stream type will be initially determined by the currently configured stream type of the source that the AudioCapturer was bound to at creation time. Users may either fetch this type using GetStreamType, or they may choose to have the media resampled or converted to a type of their choosing by calling SetStreamType. Note: the stream type may only be set while the system is not running, meaning that there are no pending capture regions (specified using CaptureAt) and that the system is not currently running in 'async' capture mode.

Buffers and memory management

Audio data is captured into a shared memory buffer (a VMO) supplied by the user to the AudioCapturer during the AddPayloadBuffer call. Please note the following requirements related to the management of the payload buffer.

  • The payload buffer must be supplied before any capture operation may start. Any attempt to start capture (via either CaptureAt or StartAsyncCapture) before a payload buffer has been established is an error.
  • The payload buffer may not be changed while there are any capture operations pending.
  • The stream type may not be changed after the payload buffer has been set.
  • The payload buffer must be an integral number of audio frame sizes (in bytes)
  • When running in 'async' mode (see below), the payload buffer must be at least as large as twice the frames_per_packet size specified during StartAsyncCapture.
  • The handle to the payload buffer supplied by the user must be readable, writable, mappable and transferable.
  • Users should always treat the payload buffer as read-only.

Synchronous vs. Asynchronous capture mode

The AudioCapturer interface can be used in one of two mutually exclusive modes: Synchronous and Asynchronous. A description of each mode and their tradeoffs is given below.

Synchronous mode

By default, AudioCapturer instances are running in 'sync' mode. They will only capture data when a user supplies at least one region to capture into using the CaptureAt method. Regions supplied in this way will be filled in the order that they are received and returned to the client as StreamPackets via the return value of the CaptureAt method. If an AudioCapturer instance has data to capture, but no place to put it (because there are no more pending regions to fill), the next payload generated will indicate that their has been an overflow by setting the Discontinuity flag on the next produced StreamPacket. Synchronous mode may not be used in conjunction with Asynchronous mode. It is an error to attempt to call StartAsyncCapture while the system still regions supplied by CaptureAt waiting to be filled.

If a user has supplied regions to be filled by the AudioCapturer instance in the past, but wishes to reclaim those regions, they may do so using the DiscardAllPackets method. Calling the DiscardAllPackets method will cause all pending regions to be returned, but with NO_TIMESTAMP as their StreamPacket's PTS. See "Timing and Overflows", below, for a discussion of timestamps and discontinuity flags. After a DiscardAllPackets operation, an OnEndOfStream event will be produced. While an AudioCapturer will never overwrite any region of the payload buffer after a completed region is returned, it may overwrite the unfilled portions of a partially filled buffer which has been returned as a result of a DiscardAllPackets operation.

Asynchronous mode

While running in 'async' mode, clients do not need to explicitly supply shared buffer regions to be filled by the AudioCapturer instance. Instead, a client enters into 'async' mode by calling StartAsyncCapture and supplying a callback interface and the number of frames to capture per-callback. Once running in async mode, the AudioCapturer instance will identify which payload buffer regions to capture into, capture the specified number of frames, then deliver those frames as StreamPackets using the OnPacketCapture FIDL event. Users may stop capturing and return the AudioCapturer instance to 'sync' mode using the StopAsyncCapture method.

It is considered an error to attempt any of the following operations.

  • To attempt to enter 'async' capture mode when no payload buffer has been established.
  • To specify a number of frames to capture per payload which does not permit at least two contiguous capture payloads to exist in the established shared payload buffer simultaneously.
  • To send a region to capture into using the CaptureAt method while the AudioCapturer instance is running in 'async' mode.
  • To attempt to call DiscardAllPackets while the AudioCapturer instance is running in 'async' mode.
  • To attempt to re-start 'async' mode capturing without having first stopped.
  • To attempt any operation except for SetGain while in the process of stopping.

Synchronizing with a StopAsyncCapture operation

Stopping asynchronous capture mode and returning to synchronous capture mode is an operation which takes time. Aside from SetGain, users may not call any other methods on the AudioCapturer interface after calling StopAsyncCapture (including calling StopAsyncCapture again) until after the stop operation has completed. Because of this, it is important for users to be able to synchronize with the stop operation. Two mechanisms are provided for doing so.

The first is to use StopAsyncCapture (not the NoReply variant). When the user's callback has been called, they can be certain that stop operation is complete and that the AudioCapturer instance has returned to synchronous operation mode.

The second way to determine that a stop operation has completed is to use the flags on the packets which get delivered via the user-supplied AudioCapturerCallback interface after calling StopAsyncCapture. When asked to stop, any partially filled packet will be returned to the user, and the final packet returned will always have the end-of-stream flag (kFlagsEos) set on it to indicate that this is the final frame in the sequence. If there is no partially filled packet to return, the AudioCapturer will synthesize an empty packet with no timestamp, and offset/length set to zero, in order to deliver a packet with the end-of-stream flag set on it. Once users have seen the end-of-stream flag after calling stop, the AudioCapturer has finished the stop operation and returned to synchronous operating mode.

Timing and Overflows

All media packets produced by an AudioCapturer instance will have their PTS field filled out with the capture time of the audio expressed as a timestamp given by the reference clock timeline. Note: this timestamp is actually a capture timestamp, not a presentation timestamp (it is more of a CTS than a PTS) and is meant to represent the underlying system's best estimate of the capture time of the first frame of audio, including all outboard and hardware introduced buffering delay. As a result, all timestamps produced by an AudioCapturer should be expected to be in the past relative to 'now' on the stream's reference clock timeline.

The one exception to the "everything has an explicit timestamp" rule is when discarding submitted regions while operating in synchronous mode. Discarded packets have no data in them, but FIDL demands that all pending method-return-value callbacks be executed. Because of this, the regions will be returned to the user, but their timestamps will be set to NO_TIMESTAMP, and their payload sizes will be set to zero. Any partially filled payload will have a valid timestamp, but a payload size smaller than originally requested. The final discarded payload (if there were any to discard) will be followed by an OnEndOfStream event.

Two StreamPackets delivered by an AudioCapturer instance are 'continuous' if the first frame of audio contained in the second packet was capture exactly one nominal frame time after the final frame of audio in the first packet. If this relationship does not hold, the second StreamPacket will have the 'kFlagDiscontinuous' flag set in it's flags field.

Even though explicit timestamps are provided on every StreamPacket produced, users who have very precise timing requirements are encouraged to always reason about time by counting frames delivered since the last discontinuity, rather than simply using the raw capture timestamps. This is because the explicit timestamps written on continuous packets may have a small amount of rounding error based on whether or not the units of the capture timeline reference clock are divisible by the chosen audio frame rate.

Users should always expect the first StreamPacket produced by an AudioCapturer to have the discontinuous flag set on it (as there is no previous packet to be continuous with). Similarly, the first StreamPacket after a DiscardAllPackets or a Stop/Start cycle will always be discontinuous. After that, there are only two reasons that a StreamPacket will ever be discontinuous:

  1. The user is operating in synchronous mode and does not supply regions to be filled quickly enough. If the next continuous frame of data has not been captured by the time it needs to be purged from the source buffers, an overflow has occurred and the AudioCapturer will flag the next captured region as discontinuous.
  2. The user is operating in asynchronous mode and some internal error prevents the AudioCapturer instance from capturing the next frame of audio in a continuous fashion. This might be high system load or a hardware error, but in general it is something which should never normally happen. In practice, however, if it does, the next produced packet will be flagged as being discontinuous.

Synchronous vs. Asynchronous Trade-offs

The choice of operating in synchronous vs. asynchronous mode is up to the user, and depending on the user's requirements, there are some advantages and disadvantages to each choice.

Synchronous mode requires only a single Zircon channel under the hood and can achieve some small savings because of this. In addition, the user has complete control over the buffer management. Users specify exactly where audio will be captured to and in what order. Because of this, if users do not need to always be capturing, it is simple to stop and restart the capture later (just by ceasing to supply packets, then resuming later on). Payloads do not need to be uniform in size either, clients may specify payloads of whatever granularity is appropriate.

The primary downside of operating in synchronous mode is that two messages will need to be sent for every packet to be captured. One to inform the AudioCapturer of the instance to capture into, and one to inform the user that the packet has been captured. This may end up increasing overhead and potentially complicating client designs.

Asynchronous mode has the advantage requiring only 1/2 of the messages, however, when operating in 'async' mode, AudioCapturer instances have no way of knowing if a user is processing the StreamPackets being sent in a timely fashion, and no way of automatically detecting an overflow condition. Users of 'async' mode should be careful to use a buffer large enough to ensure that they will be able to process their data before an AudioCapturer will be forced to overwrite it.

AddPayloadBuffer

Adds a payload buffer to the current buffer set associated with the connection. A StreamPacket struct reference a payload buffer in the current set by ID using the StreamPacket.payload_buffer_id field.

A buffer with ID id must not be in the current set when this method is invoked, otherwise the service will close the connection.

Request

NameType
id uint32
payload_buffer handle<vmo>

BindGainControl

Binds to the gain control for this AudioCapturer.

Request

NameType
gain_control_request server_end<fuchsia.media.audio/GainControl>

CaptureAt

Explicitly specifies a region of the shared payload buffer for the audio input to capture into.

Request

NameType
payload_buffer_id uint32
payload_offset uint32
frames uint32

Response

NameType
captured_packet StreamPacket

DiscardAllPackets

Request

<EMPTY>

Response

<EMPTY>

DiscardAllPacketsNoReply

Request

<EMPTY>

GetReferenceClock

Retrieves the stream's reference clock. The returned handle will have READ, DUPLICATE and TRANSFER rights, and will refer to a zx::clock that is MONOTONIC and CONTINUOUS.

Request

<EMPTY>

Response

NameType
reference_clock handle<clock>

GetStreamType

Gets the currently configured stream type. Note: for an AudioCapturer which was just created and has not yet had its stream type explicitly set, this will retrieve the stream type -- at the time the AudioCapturer was created -- of the source (input or looped-back output) to which the AudioCapturer is bound. Even if this matches the client's desired format, SetPcmStreamType must still be called.

Request

<EMPTY>

Response

NameType
stream_type StreamType

OnEndOfStream

Indicates that the stream has ended.

Response

<EMPTY>

OnPacketProduced

Delivers a packet produced by the service. When the client is done with the payload memory, the client must call ReleasePacket to release the payload memory.

Response

NameType
packet StreamPacket

ReleasePacket

Releases payload memory associated with a packet previously delivered via OnPacketProduced.

Request

NameType
packet StreamPacket

RemovePayloadBuffer

Removes a payload buffer from the current buffer set associated with the connection.

A buffer with ID id must exist in the current set when this method is invoked, otherwise the service will will close the connection.

Request

NameType
id uint32

SetPcmStreamType

Sets the stream type of the stream to be delivered. Causes the source material to be reformatted/resampled if needed in order to produce the requested stream type. Must be called before the payload buffer is established.

Request

NameType
stream_type AudioStreamType

SetReferenceClock

Sets the reference clock that controls this capturer's playback rate. If the input parameter is a valid zx::clock, it must have READ, DUPLICATE, TRANSFER rights and refer to a clock that is both MONOTONIC and CONTINUOUS. If instead an invalid clock is passed (such as the uninitialized zx::clock()), this indicates that the stream will use a 'flexible' clock generated by AudioCore that tracks the audio device.

SetReferenceClock cannot be called after the capturer payload buffer has been added. It also cannot be called a second time (even before capture). If the client wants a reference clock that is initially CLOCK_MONOTONIC but may diverge at some later time, they should create a clone of the monotonic clock, set this as the stream's reference clock, then rate-adjust it subsequently as needed.

Request

NameType
reference_clock handle<clock>?

SetUsage

Sets the usage of the capture stream. This may be changed on the fly, but packets in flight may be affected by the new usage. By default the Capturer is created with the FOREGROUND usage.

Request

NameType
usage AudioCaptureUsage

StartAsyncCapture

Places the AudioCapturer into 'async' capture mode and begin to produce packets of exactly 'frames_per_packet' number of frames each. The OnPacketProduced event (of StreamSink) will be used to inform the client of produced packets.

Request

NameType
frames_per_packet uint32

StopAsyncCapture

Stops capturing in 'async' capture mode and (optionally) deliver a callback that may be used by the client if explicit synchronization is needed.

Request

<EMPTY>

Response

<EMPTY>

StopAsyncCaptureNoReply

Request

<EMPTY>

AudioConsumer

Defined in fuchsia.media/audio_consumer.fidl

Interface for playing and controlling audio.

BindVolumeControl

Binds to this AudioConsumer volume control for control and notifications.

Request

NameType
volume_control_request server_end<fuchsia.media.audio/VolumeControl>

CreateStreamSink

Creates a StreamSink for the consumer with the indicated properties.

Multiple stream sinks may be acquired using this method, but they are intended to be used sequentially rather than concurrently. The first stream sink that's created using this method is used as the sole source of packets incoming to the logical consumer until that stream sink is closed or the EndOfStream method is called on that sink. At that point, the second stream sink is used, and so on.

If an unsupported compression type is supplied, the stream_sink_request request will be closed with an epitaph value of ZX_ERR_INVALID_ARGS.

Request

NameType
buffers vector<vmo>[16]
stream_type AudioStreamType
compression Compression?
stream_sink_request server_end<StreamSink>

OnEndOfStream

Indicates that the last packet prior to the end of the stream has been rendered.

Response

<EMPTY>

SetRate

Requests to change the playback rate of the renderer. 1.0 means normal playback. Negative rates are not supported. The new rate will be reflected in the updated status. The default rate of any newly created StreamSink is 1.0.

Request

NameType
rate float32

Start

Starts rendering as indicated by flags.

media_time indicates the packet timestamp that corresponds to reference_time. Typically, this is the timestamp of the first packet that will be rendered. If packets will be supplied with no timestamps, this value should be NO_TIMESTAMP. Passing a media_time value of NO_TIMESTAMP chooses the default media time, established as follows: 1. When starting for the first time, the default media time is the timestamp on the first packet sent to the stream sink. 2. When resuming after stop, the default media time is the media time at which the stream stopped.

reference_time is the monotonic system time at which rendering should be started. For supply-driven sources, this must be the time at which the first packet was (or will be) sent plus a lead time, which must be in the range indicated in the AudioConsumerStatus. For demand-driven sources, the client must ensure that the lead time requirement is met at the start time. Passing the default value of 0 for reference_time causes the consumer to choose a start time based on the availability of packets, the lead time requirements, and whether LOW_LATENCY has been specified.

The actual start time will be reflected in the updated status.

Request

NameType
flags AudioConsumerStartFlags
reference_time zx/time
media_time int64

Stop

Stops rendering as soon as possible after this method is called. The actual stop time will be reflected in the updated status.

Request

<EMPTY>

WatchStatus

Gets the current status of the consumer using the long get pattern. The consumer responds to this method when the status changes - initially with respect to the initial status value and thereafter with respect to the previously-reported status value.

Request

<EMPTY>

Response

NameType
status AudioConsumerStatus

AudioCore

Defined in fuchsia.media/audio_core.fidl

BindUsageVolumeControl

Binds to a volume control protocol for the given usage.

Request

NameType
usage Usage
volume_control server_end<fuchsia.media.audio/VolumeControl>

CreateAudioCapturer

Creates an AudioCapturer which either captures from the current default audio input device, or loops-back from the current default audio output device based on value passed for the loopback flag.

Request

NameType
loopback bool
audio_in_request server_end<AudioCapturer>

CreateAudioCapturerWithConfiguration

Creates an AudioCapturer according to the given requirements.

pcm_stream_type sets the stream type of the stream to be delivered. It causes the source material to be reformatted/resampled if needed in order to produce the requested stream type.

usage is used by Fuchsia to make decisions about user experience. See AudioCaptureUsage for more details.

configuration must be initialized to a variant, or no capturer can be created.

TODO(fxbug.dev/45240): Implement

Request

NameType
stream_type AudioStreamType
configuration AudioCapturerConfiguration
audio_capturer_request server_end<AudioCapturer>

CreateAudioRenderer

Creates an AudioRenderer which outputs audio to the default device.

Request

NameType
audio_out_request server_end<AudioRenderer>

EnableDeviceSettings

DEPRECATED

Request

NameType
enabled bool

GetDbFromVolume

Queries the decibel value that maps to a volume percentage [0, 1] for a particular usage. This is the same mapping as used by the VolumeControl from BindUsageVolumeControl.

Request

NameType
usage Usage
volume float32

Response

NameType
gain_db float32

GetVolumeFromDb

Queries the volume percentage [0, 1] that maps to a gain_db value for a particular usage. This is the same mapping as used by the VolumeControl from BindUsageVolumeControl.

Request

NameType
usage Usage
gain_db float32

Response

NameType
volume float32

LoadDefaults

Re-loads the platform policy configuration. Falls back to a default config if the platform does not provide a config.

Request

<EMPTY>

ResetInteractions

Re-initializes the set of rules that are currently governing the interaction of streams in audio_core. The default behavior is 'NONE'.

Request

<EMPTY>

SetCaptureUsageGain

Sets the Usage gain applied to Capturers. By default, the gain for all capture usages is set to Unity (0 db).

Request

NameType
usage AudioCaptureUsage
gain_db float32

SetInteraction

Sets how audio_core handles interactions of multiple active streams simultaneously. If streams of Usage active are processing audio, and streams of Usage affected are as well, the Behavior specified will be applied to the streams of Usage affected.

Request

NameType
active Usage
affected Usage
behavior Behavior

SetRenderUsageGain

Sets the Usage gain applied to Renderers. By default, the gain for all render usages is set to Unity (0 db).

Request

NameType
usage AudioRenderUsage
gain_db float32

SetSystemGain

System Gain and Mute

Fuchsia clients control the volume of individual audio streams via the fuchsia.media.audio.GainControl protocol. System Gain and Mute affect all audio output, and are controlled with methods that use the same concepts as GainControl, namely: independent gain and mute, with change notifications. Setting System Mute to true leads to the same outcome as setting System Gain to MUTED_GAIN_DB: all audio output across the system is silenced.

Sets the systemwide gain in decibels. gain_db values are clamped to the range -160 db to 0 db, inclusive. This setting is applied to all audio output devices. Audio input devices are unaffected. Does not affect System Mute.

DEPRECATED

Request

NameType
gain_db float32

SetSystemMute

Sets/clears the systemwide 'Mute' state for audio output devices. Audio input devices are unaffected. Changes to the System Mute state do not affect the value of System Gain.

DEPRECATED

Request

NameType
muted bool

SystemGainMuteChanged

Provides current values for systemwide Gain and Mute. When a client connects to AudioCore, the system immediately sends that client a SystemGainMuteChanged event with the current system Gain|Mute settings. Subsequent events will be sent when these Gain|Mute values change.

DEPRECATED

Response

NameType
gain_db float32
muted bool

AudioDeviceEnumerator

Defined in fuchsia.media/audio_device_enumerator.fidl

AddDeviceByChannel

Request

NameType
device_name string[256]
is_input bool
channel fuchsia.hardware.audio/StreamConfig

GetDefaultInputDevice

Default Device

Fetch the device ID of the current default input or output device, or ZX_KOID_INVALID if no such device exists.

DEPRECATED

Request

<EMPTY>

Response

NameType
device_token uint64

GetDefaultOutputDevice

DEPRECATED

Request

<EMPTY>

Response

NameType
device_token uint64

GetDeviceGain

Gain/Mute/AGC control

Note that each of these operations requires a device_token in order to target the proper input/output.

The Get command returns the device_token of the device whose gain is being reported, or ZX_KOID_INVALID in the case that the requested device_token was invalid or the device had been removed from the system before the Get command could be processed.

Set commands which are given an invalid device token are ignored and have no effect on the system. In addition, users do not need to control all of the gain settings for an audio device with each call. Only the settings with a corresponding flag set in the set_flags parameter will be affected. For example, passing SetAudioGainFlag_MuteValid will cause a SetDeviceGain call to care only about the mute setting in the gain_info structure, while passing (SetAudioGainFlag_GainValid | SetAudioGainFlag_MuteValid) will cause both the mute and the gain status to be changed simultaneously.

Request

NameType
device_token uint64

Response

NameType
device_token uint64
gain_info AudioGainInfo

GetDevices

Obtain the list of currently active audio devices.

Request

<EMPTY>

Response

NameType
devices vector<AudioDeviceInfo>

OnDefaultDeviceChanged

DEPRECATED

Response

NameType
old_default_token uint64
new_default_token uint64

OnDeviceAdded

Events sent when devices are added or removed, or when properties of a device change.

Response

NameType
device AudioDeviceInfo

OnDeviceGainChanged

Response

NameType
device_token uint64
gain_info AudioGainInfo

OnDeviceRemoved

Response

NameType
device_token uint64

SetDeviceGain

Request

NameType
device_token uint64
gain_info AudioGainInfo
valid_flags AudioGainValidFlags

AudioRenderer

Defined in fuchsia.media/audio_renderer.fidl

AudioRenderers can be in one of two states at any time: configurable or operational. A renderer is considered operational whenever it has packets queued to be rendered; otherwise it is configurable. Once an AudioRenderer enters the operational state, calls to "configuring" methods are disallowed and will cause the audio service to disconnect the client's connection. The following are considered configuring methods: AddPayloadBuffer, SetPcmStreamType, SetStreamType, SetPtsUnits, SetPtsContinuityThreshold.

If an AudioRenderer must be reconfigured, the client must ensure that no packets are still enqueued when these "configuring" methods are called. Thus it is best practice to call DiscardAllPackets on the AudioRenderer (and ideally Stop before DiscardAllPackets), prior to reconfiguring the renderer.

AddPayloadBuffer

Adds a payload buffer to the current buffer set associated with the connection. A StreamPacket struct reference a payload buffer in the current set by ID using the StreamPacket.payload_buffer_id field.

A buffer with ID id must not be in the current set when this method is invoked, otherwise the service will close the connection.

Request

NameType
id uint32
payload_buffer handle<vmo>

BindGainControl

Binds to the gain control for this AudioRenderer.

Request

NameType
gain_control_request server_end<fuchsia.media.audio/GainControl>

DiscardAllPackets

Discards packets previously sent via SendPacket or SendPacketNoReply and not yet released. The response is sent after all packets have been released.

Request

<EMPTY>

Response

<EMPTY>

DiscardAllPacketsNoReply

Discards packets previously sent via SendPacket or SendPacketNoReply and not yet released.

Request

<EMPTY>

EnableMinLeadTimeEvents

Enables or disables notifications about changes to the minimum clock lead time (in nanoseconds) for this AudioRenderer. Calling this method with 'enabled' set to true will trigger an immediate OnMinLeadTimeChanged event with the current minimum lead time for the AudioRenderer. If the value changes, an OnMinLeadTimeChanged event will be raised with the new value. This behavior will continue until the user calls EnableMinLeadTimeEvents(false).

The minimum clock lead time is the amount of time ahead of the reference clock's understanding of "now" that packets needs to arrive (relative to the playback clock transformation) in order for the mixer to be able to mix packet. For example...

  • Let the PTS of packet X be P(X)
  • Let the function which transforms PTS -> RefClock be R(p) (this function is determined by the call to Play(...)
  • Let the minimum lead time be MLT

If R(P(X)) < RefClock.Now() + MLT Then the packet is late, and some (or all) of the packet's payload will need to be skipped in order to present the packet at the scheduled time.

Request

NameType
enabled bool

EndOfStream

Indicates the stream has ended. The precise semantics of this method are determined by the inheriting interface.

Request

<EMPTY>

GetMinLeadTime

While it is possible to call GetMinLeadTime before SetPcmStreamType, there's little reason to do so. This is because lead time is a function of format/rate, so lead time will be recalculated after SetPcmStreamType. If min lead time events are enabled before SetPcmStreamType (with EnableMinLeadTimeEvents(true)), then an event will be generated in response to SetPcmStreamType.

Request

<EMPTY>

Response

NameType
min_lead_time_nsec int64

GetReferenceClock

Retrieves the stream's reference clock. The returned handle will have READ, DUPLICATE and TRANSFER rights, and will refer to a zx::clock that is MONOTONIC and CONTINUOUS.

Request

<EMPTY>

Response

NameType
reference_clock handle<clock>

OnMinLeadTimeChanged

Response

NameType
min_lead_time_nsec int64

Pause

Immediately puts the AudioRenderer into the paused state and then report the relationship between the media and reference timelines which was established (if requested).

If the AudioRenderer is already in the paused state when this called, the previously-established timeline values are returned (if requested).

Request

<EMPTY>

Response

NameType
reference_time int64
media_time int64

PauseNoReply

Request

<EMPTY>

Play

Immediately puts the AudioRenderer into a playing state. Starts the advance of the media timeline, using specific values provided by the caller (or default values if not specified). In an optional callback, returns the timestamp values ultimately used -- these set the ongoing relationship between the media and reference timelines (i.e., how to translate between the domain of presentation timestamps, and the realm of local system time).

Local system time is specified in units of nanoseconds; media_time is specified in the units defined by the user in the SetPtsUnits function, or nanoseconds if SetPtsUnits is not called.

The act of placing an AudioRenderer into the playback state establishes a relationship between 1) the user-defined media (or presentation) timeline for this particular AudioRenderer, and 2) the real-world system reference timeline. To communicate how to translate between timelines, the Play() callback provides an equivalent timestamp in each time domain. The first value ('reference_time') is given in terms of this renderer's reference clock; the second value ('media_time') is what media instant exactly corresponds to that local time. Restated, the frame at 'media_time' in the audio stream should be presented at 'reference_time' according to the reference clock.

Note: on calling this API, media_time immediately starts advancing. It is possible (if uncommon) for a caller to specify a system time that is far in the past, or far into the future. This, along with the specified media time, is simply used to determine what media time corresponds to 'now', and THAT media time is then intersected with presentation timestamps of packets already submitted, to determine which media frames should be presented next.

With the corresponding reference_time and media_time values, a user can translate arbitrary time values from one timeline into the other. After calling SetPtsUnits(pts_per_sec_numerator, pts_per_sec_denominator) and given the 'ref_start' and 'media_start' values from Play, then for any 'ref_time':

media_time = ( (ref_time - ref_start) / 1e9 * (pts_per_sec_numerator / pts_per_sec_denominator) ) + media_start

Conversely, for any presentation timestamp 'media_time':

ref_time = ( (media_time - media_start) * (pts_per_sec_denominator / pts_per_sec_numerator) * 1e9 ) + ref_start

Users, depending on their use case, may optionally choose not to specify one or both of these timestamps. A timestamp may be omitted by supplying the special value 'NO_TIMESTAMP'. The AudioRenderer automatically deduces any omitted timestamp value using the following rules:

Reference Time If 'reference_time' is omitted, the AudioRenderer will select a "safe" reference time to begin presentation, based on the minimum lead times for the output devices that are currently bound to this AudioRenderer. For example, if an AudioRenderer is bound to an internal audio output requiring at least 3 mSec of lead time, and an HDMI output requiring at least 75 mSec of lead time, the AudioRenderer might (if 'reference_time' is omitted) select a reference time 80 mSec from now.

Media Time If media_time is omitted, the AudioRenderer will select one of two values.

  • If the AudioRenderer is resuming from the paused state, and packets have not been discarded since being paused, then the AudioRenderer will use a media_time corresponding to the instant at which the presentation became paused.
  • If the AudioRenderer is being placed into a playing state for the first time following startup or a 'discard packets' operation, the initial media_time will be set to the PTS of the first payload in the pending packet queue. If the pending queue is empty, initial media_time will be set to zero.

Return Value When requested, the AudioRenderer will return the 'reference_time' and 'media_time' which were selected and used (whether they were explicitly specified or not) in the return value of the play call.

Examples

  1. A user has queued some audio using SendPacket and simply wishes them to start playing as soon as possible. The user may call Play without providing explicit timestamps -- Play(NO_TIMESTAMP, NO_TIMESTAMP).

  2. A user has queued some audio using SendPacket, and wishes to start playback at a specified 'reference_time', in sync with some other media stream, either initially or after discarding packets. The user would call Play(reference_time, NO_TIMESTAMP).

  3. A user has queued some audio using SendPacket. The first of these packets has a PTS of zero, and the user wishes playback to begin as soon as possible, but wishes to skip all of the audio content between PTS 0 and PTS 'media_time'. The user would call Play(NO_TIMESTAMP, media_time).

  4. A user has queued some audio using SendPacket and want to present this media in synch with another player in a different device. The coordinator of the group of distributed players sends an explicit message to each player telling them to begin presentation of audio at PTS 'media_time', at the time (based on the group's shared reference clock) 'reference_time'. Here the user would call Play(reference_time, media_time).

Request

NameType
reference_time int64
media_time int64

Response

NameType
reference_time int64
media_time int64

PlayNoReply

Request

NameType
reference_time int64
media_time int64

RemovePayloadBuffer

Removes a payload buffer from the current buffer set associated with the connection.

A buffer with ID id must exist in the current set when this method is invoked, otherwise the service will will close the connection.

Request

NameType
id uint32

SendPacket

Sends a packet to the service. The response is sent when the service is done with the associated payload memory.

packet must be valid for the current buffer set, otherwise the service will close the connection.

Request

NameType
packet StreamPacket

Response

<EMPTY>

SendPacketNoReply

Sends a packet to the service. This interface doesn't define how the client knows when the sink is done with the associated payload memory. The inheriting interface must define that.

packet must be valid for the current buffer set, otherwise the service will close the connection.

Request

NameType
packet StreamPacket

SetPcmStreamType

Sets the type of the stream to be delivered by the client. Using this method implies that the stream encoding is AUDIO_ENCODING_LPCM.

This must be called before Play or PlayNoReply. After a call to SetPcmStreamType, the client must then send an AddPayloadBuffer request, then the various StreamSink methods such as SendPacket/SendPacketNoReply.

Request

NameType
type AudioStreamType

SetPtsContinuityThreshold

Sets the maximum threshold (in seconds) between explicit user-provided PTS and expected PTS (determined using interpolation). Beyond this threshold, a stream is no longer considered 'continuous' by the renderer.

Defaults to an interval of half a PTS 'tick', using the currently-defined PTS units. Most users should not need to change this value from its default.

Example: A user is playing back 48KHz audio from a container, which also contains video and needs to be synchronized with the audio. The timestamps are provided explicitly per packet by the container, and expressed in mSec units. This means that a single tick of the media timeline (1 mSec) represents exactly 48 frames of audio. The application in this scenario delivers packets of audio to the AudioRenderer, each with exactly 470 frames of audio, and each with an explicit timestamp set to the best possible representation of the presentation time (given this media clock's resolution). So, starting from zero, the timestamps would be..

[ 0, 10, 20, 29, 39, 49, 59, 69, 78, 88, ... ]

In this example, attempting to use the presentation time to compute the starting frame number of the audio in the packet would be wrong the majority of the time. The first timestamp is correct (by definition), but it will be 24 packets before the timestamps and frame numbers come back into alignment (the 24th packet would start with the 11280th audio frame and have a PTS of exactly 235).

One way to fix this situation is to set the PTS continuity threshold (henceforth, CT) for the stream to be equal to 1/2 of the time taken by the number of frames contained within a single tick of the media clock, rounded up. In this scenario, that would be 24.0 frames of audio, or 500 uSec. Any packets whose expected PTS was within +/-CT frames of the explicitly provided PTS would be considered to be a continuation of the previous frame of audio. For this example, calling 'SetPtsContinuityThreshold(0.0005)' would work well.

Other possible uses: Users who are scheduling audio explicitly, relative to a clock which has not been configured as the reference clock, can use this value to control the maximum acceptable synchronization error before a discontinuity is introduced. E.g., if a user is scheduling audio based on a recovered common media clock, and has not published that clock as the reference clock, and they set the CT to 20mSec, then up to 20mSec of drift error can accumulate before the AudioRenderer deliberately inserts a presentation discontinuity to account for the error.

Users whose need to deal with a container where their timestamps may be even less correct than +/- 1/2 of a PTS tick may set this value to something larger. This should be the maximum level of inaccuracy present in the container timestamps, if known. Failing that, it could be set to the maximum tolerable level of drift error before absolute timestamps are explicitly obeyed. Finally, a user could set this number to a very large value (86400.0 seconds, for example) to effectively cause all timestamps to be ignored after the first, thus treating all audio as continuous with previously delivered packets. Conversely, users who wish to always explicitly schedule their audio packets exactly may specify a CT of 0.

Note: explicitly specifying high-frequency PTS units reduces the default continuity threshold accordingly. Internally, this threshold is stored as an integer of 1/8192 subframes. The default threshold is computed as follows: RoundUp((AudioFPS/PTSTicksPerSec) * 4096) / (AudioFPS * 8192) For this reason, specifying PTS units with a frequency greater than 8192x the frame rate (or NOT calling SetPtsUnits, which accepts the default PTS unit of 1 nanosec) will result in a default continuity threshold of zero.

Request

NameType
threshold_seconds float32

SetPtsUnits

Sets the units used by the presentation (media) timeline. By default, PTS units are nanoseconds (as if this were called with numerator of 1e9 and denominator of 1). This ratio must lie between 1/60 (1 tick per minute) and 1e9/1 (1ns per tick).

Request

NameType
tick_per_second_numerator uint32
tick_per_second_denominator uint32

SetReferenceClock

Sets the reference clock that controls this renderer's playback rate. If the input parameter is a valid zx::clock, it must have READ, DUPLICATE, TRANSFER rights and refer to a clock that is both MONOTONIC and CONTINUOUS. If instead an invalid clock is passed (such as the uninitialized zx::clock()), this indicates that the stream will use a 'flexible' clock generated by AudioCore that tracks the audio device.

SetReferenceClock cannot be called once SetPcmStreamType is called. It also cannot be called a second time (even if the renderer format has not yet been set). If a client wants a reference clock that is initially CLOCK_MONOTONIC but may diverge at some later time, they should create a clone of the monotonic clock, set this as the stream's reference clock, then rate-adjust it subsequently as needed.

Request

NameType
reference_clock handle<clock>?

SetUsage

Sets the usage of the render stream. This method may not be called after SetPcmStreamType is called. The default usage is MEDIA.

Request

NameType
usage AudioRenderUsage

ProfileProvider

Defined in fuchsia.media/profile_provider.fidl

RegisterHandlerWithCapacity

Register a thread as a media thread. This notifies the media subsystem that this thread should have an elevated scheduling profile applied to it in order to meet audio or video deadlines.

name is the name of a system scheduling role to apply to the thread given by thread_handle -- different products may customize the underlying scheduling strategy based on the requested role. period is the suggested interval to be scheduled at. period may be zero if the thread has no preferred scheduling interval. capacity is the proportion of the scheduling interval the thread needs to be running to achieve good performance or to meet the scheduling deadline defined by period. capacity may be zero if the workload has no firm runtime requirements. Note that capacity should be a good faith estimate based on the worst case runtime the thread requires each period. Excessive capacity requests may be rejected or result in scaling back the performance of other threads to fit resource limits.

Capacity, max runtime, and period have the following relationship:

capacity = max runtime / period

Where:

0 <= max runtime <= period and 0 <= capacity <= 1

For heterogeneous systems, the capacity should be planned / measured against the highest performance processor(s) in the system. The system will automatically adjust the effective capacity to account for slower processors and operating points and will avoid processors and operating points that are too slow to meet the requested scheduling parameters (provided they are reasonable).

Returns the period and capacity (actually maximum runtime) that was applied, either of which may be zero to indicate not applicable.

Request

NameType
thread_handle handle<thread>
name string[64]
period zx/duration
capacity float32

Response

NameType
period zx/duration
capacity zx/duration

UnregisterHandler

Reset a thread's scheduling profile to the default.

Request

NameType
thread_handle handle<thread>
name string[64]

Response

<EMPTY>

SessionAudioConsumerFactory

Defined in fuchsia.media/audio_consumer.fidl

Interface for creating audio consumers bound to a session.

CreateAudioConsumer

Creates an AudioConsumer, which is an interface for playing audio, bound to a particular session. session_id is the identifier of the media session for which audio is to be rendered.

Request

NameType
session_id uint64
audio_consumer_request server_end<AudioConsumer>

SimpleStreamSink

Defined in fuchsia.media/stream.fidl

A StreamSink that uses StreamBufferSet for buffer management.

AddPayloadBuffer

Adds a payload buffer to the current buffer set associated with the connection. A StreamPacket struct reference a payload buffer in the current set by ID using the StreamPacket.payload_buffer_id field.

A buffer with ID id must not be in the current set when this method is invoked, otherwise the service will close the connection.

Request

NameType
id uint32
payload_buffer handle<vmo>

DiscardAllPackets

Discards packets previously sent via SendPacket or SendPacketNoReply and not yet released. The response is sent after all packets have been released.

Request

<EMPTY>

Response

<EMPTY>

DiscardAllPacketsNoReply

Discards packets previously sent via SendPacket or SendPacketNoReply and not yet released.

Request

<EMPTY>

EndOfStream

Indicates the stream has ended. The precise semantics of this method are determined by the inheriting interface.

Request

<EMPTY>

RemovePayloadBuffer

Removes a payload buffer from the current buffer set associated with the connection.

A buffer with ID id must exist in the current set when this method is invoked, otherwise the service will will close the connection.

Request

NameType
id uint32

SendPacket

Sends a packet to the service. The response is sent when the service is done with the associated payload memory.

packet must be valid for the current buffer set, otherwise the service will close the connection.

Request

NameType
packet StreamPacket

Response

<EMPTY>

SendPacketNoReply

Sends a packet to the service. This interface doesn't define how the client knows when the sink is done with the associated payload memory. The inheriting interface must define that.

packet must be valid for the current buffer set, otherwise the service will close the connection.

Request

NameType
packet StreamPacket

StreamBufferSet

Defined in fuchsia.media/stream.fidl

Manages a set of payload buffers for a stream. This interface is typically inherited along with StreamSink or StreamSource to enable the transport of elementary streams between clients and services.

AddPayloadBuffer

Adds a payload buffer to the current buffer set associated with the connection. A StreamPacket struct reference a payload buffer in the current set by ID using the StreamPacket.payload_buffer_id field.

A buffer with ID id must not be in the current set when this method is invoked, otherwise the service will close the connection.

Request

NameType