avviddec: fix invalid alignment calculations.

avcodec_align_dimensions2 uses context->pix_fmt to make its
calculations, we thus need to make sure it is adequately set
when calling it.

Fixes:

gst-launch-1.0 videotestsrc ! video/x-raw, width=1920, height=1080 \
! avenc_mpeg4 ! avdec_mpeg4 ! xvimagesink

This showed invalid writes under valgrind, then segfault.

https://bugzilla.gnome.org/show_bug.cgi?id=792900
This commit is contained in:
Mathieu Duponchelle 2018-07-02 04:05:19 +02:00
parent 8562f10969
commit be9b6969cb

View file

@ -678,6 +678,12 @@ gst_ffmpegviddec_ensure_internal_pool (GstFFMpegVidDec * ffmpegdec,
format = gst_ffmpeg_pixfmt_to_videoformat (picture->format);
gst_video_info_set_format (&info, format, picture->width, picture->height);
/* If we have not yet been negotiated, a NONE format here would
* result in invalid initial dimension alignments, and potential
* out of bounds writes.
*/
ffmpegdec->context->pix_fmt = picture->format;
for (i = 0; i < G_N_ELEMENTS (ffmpegdec->stride); i++)
ffmpegdec->stride[i] = -1;