vtdec: Fix deadlock when attempting to negotiate

This was wrongly calling the base class method, which unnecessairly took the stream lock, already taken by
handle_frame(). The drain() call in negotiate() would then wait for the output loop to pause, while that loop
is stuck waiting to take the stream lock, thus causing a deadlock.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5521>
This commit is contained in:
Piotr Brzeziński 2023-10-20 13:50:16 +02:00
parent 0a29654d7b
commit e79b308f82

View file

@ -812,7 +812,7 @@ gst_vtdec_handle_frame (GstVideoDecoder * decoder, GstVideoCodecFrame * frame)
/* Negotiate now so that we know whether we need to use the GL upload meta or not.
* gst_vtenc_negotiate() will drain before attempting to negotiate. */
if (gst_pad_check_reconfigure (decoder->srcpad)) {
if (!gst_video_decoder_negotiate (decoder)) {
if (!gst_vtdec_negotiate (decoder)) {
gst_pad_mark_reconfigure (decoder->srcpad);
if (GST_PAD_IS_FLUSHING (decoder->srcpad))
ret = GST_FLOW_FLUSHING;