Skip to content

Commit

Permalink
Skip rendering multiple frames on the same vsync
Browse files Browse the repository at this point in the history
When rendering frames at a rate higher than the screen refresh rate,
e.g. playing at 8x, the player is releasing multiple frames at the same
release time (nanos) which are then dropped by the platform. The output
buffers are available later and as a result MediaCodec cannot keep up
decoding fast enough.

This change skips releasing multiple video frames on the same vsync
period and proactivelly drops the frame. The frame is counted as skipped
rather than dropped to differentiate with frames dropped due to slow
decoding.

PiperOrigin-RevId: 510964976
(cherry picked from commit ab7e84f)
  • Loading branch information
christosts authored and tonihei committed Feb 28, 2023
1 parent 0e5dad5 commit 58a977e
Showing 1 changed file with 12 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -153,6 +153,7 @@ public class MediaCodecVideoRenderer extends MediaCodecRenderer {
private long lastRenderRealtimeUs;
private long totalVideoFrameProcessingOffsetUs;
private int videoFrameProcessingOffsetCount;
private long lastFrameReleaseTimeNs;

private int currentWidth;
private int currentHeight;
Expand Down Expand Up @@ -1128,9 +1129,18 @@ && maybeDropBuffersToKeyframe(positionUs, treatDroppedBuffersAsSkipped)) {
if (Util.SDK_INT >= 21) {
// Let the underlying framework time the release.
if (earlyUs < 50000) {
notifyFrameMetadataListener(presentationTimeUs, adjustedReleaseTimeNs, format);
renderOutputBufferV21(codec, bufferIndex, presentationTimeUs, adjustedReleaseTimeNs);
if (adjustedReleaseTimeNs == lastFrameReleaseTimeNs) {
// This frame should be displayed on the same vsync with the previous released frame. We
// are likely rendering frames at a rate higher than the screen refresh rate. Skip
// this buffer so that it's returned to MediaCodec sooner otherwise MediaCodec may not
// be able to keep decoding with this rate [b/263454203].
skipOutputBuffer(codec, bufferIndex, presentationTimeUs);
} else {
notifyFrameMetadataListener(presentationTimeUs, adjustedReleaseTimeNs, format);
renderOutputBufferV21(codec, bufferIndex, presentationTimeUs, adjustedReleaseTimeNs);
}
updateVideoFrameProcessingOffsetCounters(earlyUs);
lastFrameReleaseTimeNs = adjustedReleaseTimeNs;
return true;
}
} else {
Expand Down

0 comments on commit 58a977e

Please sign in to comment.