glcolorconvert: Calculate correct swizzles for Y410, Y412 formats

The GST_VIDEO_FORMAT_Y410, GST_VIDEO_FORMAT_Y412_LE and GST_VIDEO_FORMAT_Y412_BE
formats in fact are packed formats, which have just 1 plane. But we have special
setting for them rather than using get_single_planar_format_gl_swizzle_order().

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5314>
This commit is contained in:
He Junyan 2023-09-11 23:10:00 +08:00 committed by GStreamer Marge Bot
parent 30f955f73a
commit 0ace130107

View file

@ -537,7 +537,10 @@ gst_gl_video_format_swizzle (GstVideoFormat video_format, int *swizzle)
{
const GstVideoFormatInfo *finfo = gst_video_format_get_info (video_format);
if (finfo->n_planes == 1) {
if (finfo->n_planes == 1 &&
(finfo->flags & GST_VIDEO_FORMAT_FLAG_RGB ||
video_format == GST_VIDEO_FORMAT_AYUV ||
video_format == GST_VIDEO_FORMAT_VUYA)) {
get_single_planar_format_gl_swizzle_order (video_format, swizzle);
return TRUE;
}