The decoder initially read from the socket, decoded the video and sent
the decoded frames to the screen:
+---------+ +----------+
socket ---> | decoder | ---> | screen |
+---------+ +----------+
The design was simple, but the decoder had several responsabilities.
Then we added the recording feature, so we added a recorder, which
reused the packets received from the socket managed by the decoder:
+----------+
---> | screen |
+---------+ / +----------+
socket ---> | decoder | ----
+---------+ \ +----------+
---> | recorder |
+----------+
This lack of separation of concerns now have concrete implications: we
could not (properly) disable the decoder/display to only record the
video.
Therefore, split the decoder to extract the stream:
+----------+ +----------+
---> | decoder | ---> | screen |
+---------+ / +----------+ +----------+
socket ---> | stream | ----
+---------+ \ +----------+
---> | recorder |
+----------+
This will allow to record the stream without decoding the video.
The deprecated avcodec_decode_video2() should always the whole packet,
so there is no need to loop (cf doc/examples/demuxing_decoding.c in
FFmpeg).
This hack changed the packet size and data pointer. This broke recording
which used the same packet.
Configuration packets produced by MediaCodec have no valid PTS, and do
not produce frame. Do not queue their (invalid) PTS not to break the
matching between frames and their PTS.
Several frames may be read by read_packet() before they are consumed
(returned by av_read_frame()), so we need to store the PTS of frames in
order, so that the right PTS is assigned to the right frame.
Since PTS handling has been fixed, the recorder do not associate a PTS
to a wrong frame anymore, so PTS of "configuration packets" (which never
produce a frame), are never read by the recorder. Therefore, there is no
need to ignore them explicitly, so we can remove the MediaCodec flags
completely.
The PTS was read from the socket and set as the current one even before
the frame was consumed, so it could be assigned to the previous frame
"in advance".
Store the PTS for the current frame and the last PTS read from the
packet header of the next frame in separate fields.
As a side-effect, this fixes the warning on quit:
> Application provided invalid, non monotonically increasing dts to
> muxer in stream 0: 17164020 >= 17164020
On H.264 stream EOF, the eof_reached flag is set, but av_read_frame()
still provides a frame, so check the flag only afterwards.
As a side-effect, it also fixes a memory leak (the very last packet was
not unref).
The decoder sometimes returned a non-zero value on error, but not on
every path.
Since we never use the value, always return 0 at the end (like in the
controller).
SDL_net is not very suitable for scrcpy.
For example, SDLNet_TCP_Accept() is non-blocking, so we have to wrap it
by calling many SDL_Net-specific functions to make it blocking.
But above all, SDLNet_TCP_Open() is a server socket only when no IP is
provided; otherwise, it's a client socket. Therefore, it is not possible
to create a server socket bound to localhost, so it accepts connections
from anywhere.
This is a problem for scrcpy, because on start, the application listens
for nearly 1 second until it accepts the first connection, supposedly
from the device. If someone on the local network manages to connect to
the server socket first, then they can stream arbitrary H.264 video.
This may be troublesome, for example during a public presentation ;-)
Provide our own simplified API (net.h) instead, implemented for the
different platforms.
The syntax was correct, but less readable, and it unnecessarily zeroed
the fields other than "type".
Create the event properly, from a separate method.
When the video stream socket is closed and read_packey() returns -1,
av_read_frame() still returns 0.
To detect EOF, check the flag eof_reached in the AVIOContext.
This avoids garbage errors on closing.
Expose frames_offer_decoded_frame() and frames_consume_rendered_frame()
so that callers are not exposed to frame swapping (between the decoding
and rendering frames) details.
The skip_frames flag was a non-configurable runtime flag. Since it is
not exposed to the user, there is no need for a (possible) runtime cost.
For testing purpose, we still want it to be configurable, so make it a
compilation flag.
