Background:
Whenever a caps event is received by appsink, the caps are stored in the
same internal queue as buffers. Only when enough buffers have been
popped from the queue to reach the caps, `priv->sample` gets its caps
updated to match, so that they are correct for the following buffers.
Note that as far as upstream elements are concerned, the caps of appsink
are updated immediately when the CAPS event is sent. Samples pulled from
appsink retain the old caps until a later buffer -- one that was sent by
upstream elements after the new caps -- is pulled.
The race condition:
When a flush is received, appsink clears the entire internal queue. The
caps of `priv->sample` are not updated as part of this process, and
instead remain as those of the sample that was last pulled by the user.
This leaves open a race condition where:
1. Upstream sends a new caps event, and possibly some buffers for the
new caps.
2. Upstream sends a flush (possibly from a different thread).
3. Upstream sends a new buffer for the new caps. Since as far as
upstream is concerned, appsink caps are the new caps already, no new
CAPS event is sent.
4. The appsink user pulls a sample, having not pulled before enough
samples to reach the buffers sent in step 1.
Bug: the pulled sample has the old caps instead of the new caps.
Fixing the race condition:
To avoid this problem, when a buffer is received after a flush,
`priv->sample`'s caps should be updated with the current caps before the
buffer is added to the internal queue.
Interestingly, before this patch, appsink already had code for this, in
gst_app_sink_render_common():
/* queue holding caps event might have been FLUSHed,
* but caps state still present in pad caps */
if (G_UNLIKELY (!priv->last_caps &&
gst_pad_has_current_caps (GST_BASE_SINK_PAD (psink)))) {
priv->last_caps = gst_pad_get_current_caps (GST_BASE_SINK_PAD (psink));
gst_sample_set_caps (priv->sample, priv->last_caps);
GST_DEBUG_OBJECT (appsink, "activating pad caps %" GST_PTR_FORMAT,
priv->last_caps);
}
This code assumes `priv->last_caps` is reset when a flush is received,
which makes sense, but unfortunately, there was no code in the flush
code path resetting it.
This patch adds such code, therefore fixing the race condition. A unit
test demonstrating the bug and testing its behavior with the fix has
also been added.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2442>
The ["level-asymmetry-allowed"] field states that the peer wants the
profile specified in the "profile-level-id" fields but doesn't care
about the level. To express this in GStreamer caps term, we add a
"profile" field in the caps, which reuses the usual "profile" semantics
for H.264 streams and, and remove "profile-level-id" and
"level-asymmetry-allowed" fields.
["level-asymmetry-allowed"]: https://www.iana.org/assignments/media-types/video/H264
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1410>
... in favour of dep.get_variable('foo', ..) which in some
cases allows for further cleanups in future since we can
extract variables from pkg-config dependencies as well as
internal dependencies using this mechanism.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1183>
Currently the extension data length specified in the RTP header would
say it was shorter then the data serialised to a packet. When
combining the resulting buffer, the underlying memory would still
contain the extra (now 0-filled) padding data.
This would mean that parsing the resulting RTP packet would potentially
start with a number of 0-filled bytes which many RTP formats are not
expecting.
Such usage is found by e.g. RTP header extension when allocating the
maximum buffer (which may be larger than the written size) and shrinking
to the required size the data once all the rtp header extension data has
been written.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1146>
Since the base class now does the parsing, there is no need
to reproduce that code in all the subclasses, just pass the attributes
which are the only relevant bit anyway.
Also, only store the direction if the subclass accepted the caps
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/906>
