DefaultTrackSelector should avoid enabling multiple renderers implementing MediaClock by default

[SteUK]

Version: ExoPlayer release v2
Device: Samsung s7
Android: 6

Hi,

In the project I am working on, we are having to use a custom audio renderer (via an extension) from a 3rd party dev house. I need to use a mixture of their audio format, and a standard 2 channel mp4a aac stream.

All wrapped up into DASH, I get a video renderer and the two audio renderers back in the 'tracks changed' listing. I am having two issues I wonder if someone can comment on what we might be doing wrong...or if there is a bug/feature hole here.

Firstly, I am getting the 'Multiple renderer media clocks enabled' exception fire from ExoPlayerImplInternal.jave : function 'enableRenderers'. This is because we inherit a class from SimpleDecoderAudioRenderer which, naturally, returns itself in the function getMediaClock. What is the best practice here?

Secondly, (diverting around the first issue) when I playback the DASH stream...I am getting the two audio renderers both play their first streams...at the same time...together.

Unfortunately, and totally unhelpfully I know, I don't have a test stream I can share currently - due to license issues with the custom audio format.

Thanks,
Ste

[ojw28]

Firstly, I am getting the 'Multiple renderer media clocks enabled' exception fire from ExoPlayerImplInternal.jave : function 'enableRenderers'. This is because we inherit a class from SimpleDecoderAudioRenderer which, naturally, returns itself in the function getMediaClock. What is the best practice here?

This is a side effect of having both audio renderers enabled at once (i.e. the problem described below). You can only have one renderer that implements MediaClock enabled at a time, since these renderers drive timing for the playback (and two things trying to do this simultaneously doesn't make sense :)). The fix for this is just to fix the issue below.

Secondly, (diverting around the first issue) when I playback the DASH stream...I am getting the two audio renderers both play their first streams...at the same time...together.

This is a track selection problem. It sounds like your DASH manifest contains a mixture of tracks playable with the extension and tracks playable with ExoPlayer's built in audio renderer. What do you actually want to happen?

[SteUK]

Ideally... I would like the first audio renderers track to playback as default. Then when I change audio stream (via setSelectionOverride), could be between renderers, only one audio set is ever playing.

Is my problem that I need to be disabling one of the renderers up front...and enabling/disabling between them as needed when switching audio stream?

Here is the output from onTracksChanged currently:

onTracksChanged: Audio tracks: 4 | Video tracks: 4
EventLogger: Tracks [
EventLogger:   Renderer:0 [
EventLogger:     Group:0, adaptive_supported=YES [
EventLogger:       [X] Track:0, id=C0001/3000, mimeType=video/hevc, bitrate=2997501, res=2880x1920, fps=30.0, supported=YES
EventLogger:       [X] Track:1, id=C0001/4000, mimeType=video/hevc, bitrate=4012110, res=2880x1920, fps=30.0, supported=YES
EventLogger:     ]
EventLogger:     Group:1, adaptive_supported=YES [
EventLogger:       [ ] Track:0, id=C0002/3000, mimeType=video/hevc, bitrate=3008452, res=2880x1920, fps=30.0, supported=YES
EventLogger:       [ ] Track:1, id=C0002/4000, mimeType=video/hevc, bitrate=4010978, res=2880x1920, fps=30.0, supported=YES
EventLogger:     ]
EventLogger:   ]
EventLogger:   Renderer:1 [
EventLogger:     Group:0, adaptive_supported=N/A [
EventLogger:       [X] Track:0, id=A0000/und/mp4a, mimeType=audio/mp4a-latm, bitrate=267000, channels=2, sample_rate=48000, supported=YES
EventLogger:     ]
EventLogger:   ]
EventLogger:   Renderer:2 [
EventLogger:     Group:0, adaptive_supported=N/A [
EventLogger:       [X] Track:0, id=AC0001/und/CUSTOM, mimeType=audio/CUSTOM, bitrate=1195978, channels=1, sample_rate=48000, supported=YES
EventLogger:     ]
EventLogger:     Group:1, adaptive_supported=N/A [
EventLogger:       [ ] Track:0, id=AC0002/und/CUSTOM, mimeType=audio/CUSTOM, bitrate=1162175, channels=1, sample_rate=48000, supported=YES
EventLogger:     ]
EventLogger:   ]
EventLogger: ]

[ojw28]

We should probably make DefaultTrackSelector so that it doesn't enable two renderers implementing MediaClock by default, since that's not a sensible thing to do. In which case behaviour would be that the first one for which a selection can be made will be enabled. Perhaps we only want to enable one renderer of each type by default. We'll use this issue to track coming up with a sensible default behaviour, in any case.

To work around the problem in the meantime, you could extend DefaultTrackSelector in your own codebase and override selectTracks to do something like (untested code):

selectTrack(...args...) {
  // Get the defaults.
  TrackSelection[] selections = super.selectTracks(...args...);
  // Iterate over the defaults, clearing any secondary audio selections.
  int rendererCount = rendererCapabilities.length;
  boolean seenAudioSelection = false;
  for (int i = 0; i < rendererCount; i++) {
    if (C.TRACK_TYPE_AUDIO == rendererCapabilities[i].getTrackType()
        && selections[i] != null) {
      if (seenAudioSelection) {
        // We already have audio. Clear this one.
        selections[i] = null;
      }
      seenAudioSelection = true;
    }
  }
  return selections;
}

[SteUK]

Roger that! Your suggestion to override selectTracks works a treat. I had to couple in disabling of 'none desired' renderers when changing audio stream (trackSelector.setRendererDisabled( iRenderer, true );)

Here is the class for anyone looking to do the same:

public class MultiAudio_DefaultTrackSelector extends DefaultTrackSelector
{
public MultiAudio_DefaultTrackSelector( TrackSelection.Factory adaptiveVideoTrackSelectionFactory )
{
super( adaptiveVideoTrackSelectionFactory );
}

@Override
protected TrackSelection[] selectTracks(RendererCapabilities[] rendererCapabilities, TrackGroupArray[] rendererTrackGroupArrays, int[][][] rendererFormatSupports) throws ExoPlaybackException
{
    // Make a track selection for each renderer.
    int rendererCount = rendererCapabilities.length;
    Parameters params = getParameters();

    // Call parent to fill in video renderer
    TrackSelection[] rendererTrackSelections = super.selectTracks( rendererCapabilities, rendererTrackGroupArrays, rendererFormatSupports );

    boolean bSeenAudio = false;
    for (int i = 0; i < rendererCount; i++)
    {
        if( rendererCapabilities[ i ].getTrackType() == C.TRACK_TYPE_AUDIO && rendererTrackSelections[ i ] != null )
        {
            if( bSeenAudio )
            {
                rendererTrackSelections[ i ] = null;
            }
            bSeenAudio = true;
        }
    }

    return rendererTrackSelections;
}

}