Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for RTSP H263 #63

Merged
merged 7 commits into from
Jun 16, 2022
Merged

Add support for RTSP H263 #63

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
55bfe71
Add support for RTSP H263
rakeshnitb Mar 15, 2022
ffa04ea
Merge branch 'main' into rtp-h263
rakeshnitb May 5, 2022
ff80a41
Fix review comment in RTP H263 Reader
rakeshnitb May 5, 2022
f8cd3a9
Fix some more review comment in RTP H263 Reader.
rakeshnitb May 5, 2022
aae9f23
Fix some minor review comments in RTPH263Reader
rakeshnitb May 9, 2022
dfc424d
Fix review comment in RTPH263Reader
rakeshnitb May 9, 2022
fc3c57e
Merge branch 'main' into rtp-h263
rakeshnitb Jun 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 17 additions & 15 deletions ...ies/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/reader/RtpH263Reader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@

/** I-frame VOP unit type. */
private static final int I_VOP = 0;
rakeshnitb marked this conversation as resolved.
Show resolved Hide resolved
private static final int PICTURE_START_CODE = 128;

private final RtpPayloadFormat payloadFormat;

Expand Down Expand Up @@ -71,7 +72,7 @@ public RtpH263Reader(RtpPayloadFormat payloadFormat) {
@Override
public void createTracks(ExtractorOutput extractorOutput, int trackId) {
trackOutput = extractorOutput.track(trackId, C.TRACK_TYPE_VIDEO);
castNonNull(trackOutput).format(payloadFormat.format);
trackOutput.format(payloadFormat.format);
}

@Override
Expand All @@ -90,24 +91,25 @@ public void consume(ParsableByteArray data, long timestamp, int sequenceNumber,
// +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
int currPosition = data.getPosition();
rakeshnitb marked this conversation as resolved.
Show resolved Hide resolved
int header = data.readUnsignedShort();
boolean pBit = ((header & 0x400) == 0x400);
boolean pBitIsSet = ((header & 0x400) > 0);

// Check if optional Video Redundancy Coding or PLEN or PEBIT is present, RFC4629 Section 5.1.
// Check if optional V (Video Redundancy Coding), PLEN or PEBIT is present, RFC4629 Section 5.1.
if ((header & 0x200) != 0 || (header & 0x1f8) != 0 || (header & 0x7) != 0) {
Log.w(TAG, "Packet discarded due to (VRC != 0) or (PLEN != 0) or (PEBIT != 0)");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Why not supporting this mode?
  • If you choose to log VRC != 0, you should log the actual value of VRC, PLEN and PEBIT.
  • Mention packet dropped.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are unable to find clip(s) which are having these special headers to add support and verify it.
These special header are used for error resilience.

return;
}
int startCodePayload = data.peekUnsignedByte() & 0xfc;
if (pBit == true) {
if (startCodePayload < 128) {

if (pBitIsSet == true) {
rakeshnitb marked this conversation as resolved.
Show resolved Hide resolved
int startCodePayload = data.peekUnsignedByte() & 0xfc;
// Packets that begin with a Picture Start Code(100000). Refer RFC4629 Section 6.1.1.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i have difficulty understanding here: if picture start code is 0b100000 which is 64, then it'll always fail the check startCodePayload < PICTURE_START_CODE. Maybe you quoted a wrong RFC section?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Picture start code is "100000" from msb, so it is 128. If startCodePayload is less than 128 then packet is follow-On.

if (startCodePayload < PICTURE_START_CODE) {
Log.w(TAG, "Picture start Code (PSC) missing, Dropping packet.");
return;
} else {
// Setting first two bytes of the start code. Refer RFC4629 Section 5.1.
data.getData()[currPosition] = 0;
data.getData()[currPosition + 1] = 0;
data.setPosition(currPosition);
}
// Setting first two bytes of the start code. Refer RFC4629 Section 6.1.1.
data.getData()[currPosition] = 0;
data.getData()[currPosition + 1] = 0;
data.setPosition(currPosition);
} else {
// Check that this packet is in the sequence of the previous packet.
int expectedSequenceNumber = RtpPacket.getNextSequenceNumber(previousSequenceNumber);
Expand All @@ -124,7 +126,7 @@ public void consume(ParsableByteArray data, long timestamp, int sequenceNumber,

if (fragmentedSampleSizeBytes == 0) {
getBufferFlagsAndResolutionFromVop(data, isOutputFormatSet);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's very strange that a method starts with get does not return anything, and for getBufferFlagsAndResolutionFromVop to depend on whether the output format is set (isOutputFormatSet)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getBufferFlagsAndResolutionFromVop() method returns Buffer flags as well as Resolution, so i'm using class variable to get both values. I'm using "isOutputFormatSet" variable to check if outputformat is already set or not, as we need to update actual resolution once throughout decode.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, if the method does not return anything, can you rename it to parseVopHeader and specify it sets width/height/flags in the javadoc.

if (!isOutputFormatSet && isKeyFrame == true) {
if (!isOutputFormatSet && isKeyFrame) {
if (width != payloadFormat.format.width || height != payloadFormat.format.height) {
trackOutput.format(
payloadFormat.format.buildUpon().setWidth(width).setHeight(height).build());
Expand Down Expand Up @@ -168,7 +170,7 @@ public void seek(long nextRtpTimestamp, long timeUs) {
private void getBufferFlagsAndResolutionFromVop(ParsableByteArray data, boolean gotResolution) {
// Picture Segment Packets (RFC4629 Section 6.1).
// Search for SHORT_VIDEO_START_MARKER (0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0).
rakeshnitb marked this conversation as resolved.
Show resolved Hide resolved
int currPosition = data.getPosition();
int currDataOffset = data.getPosition();
long shortHeader = data.readUnsignedInt();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the reason to name this shortHeader? It's not of type short and I can't find reference to any 'short' in the RFC

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its taken from h263 decoder. Added link for the same in javaDoc.

if ((shortHeader & 0xffff) >> 10 == 0x20) {
rakeshnitb marked this conversation as resolved.
Show resolved Hide resolved
int header = data.peekUnsignedByte();
rakeshnitb marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -190,11 +192,11 @@ private void getBufferFlagsAndResolutionFromVop(ParsableByteArray data, boolean
height = (short) (144 << (sourceFormat - 2));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain a bit here? I'm particularly lost why the shift bits are derived from a sourceFormat. Is the related logic in the H263 spec? I can't find it in RFC4629.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://cs.android.com/android/platform/superproject/+/master:frameworks/av/media/codecs/m4v_h263/dec/src/vop.cpp;l=1128. According to this logic :
sourceFormat 2 is 176x144, sourceFormat 3 is 1762x1442, sourceFormat 4 is 17622x14422 and so on.

}
}
data.setPosition(currPosition);
data.setPosition(currDataOffset);
isKeyFrame = (vopType == I_VOP ? true : false);
return;
}
data.setPosition(currPosition);
data.setPosition(currDataOffset);
isKeyFrame = false;
}

Expand Down