Commit graph

46 commits

Author SHA1 Message Date
Haihao Xiang
38c55de132 msdk: free mfx frames to avoid memory leaks in encoder and vpp 2019-09-13 16:54:21 +00:00
Haihao Xiang
cc6ca4ec61 msdkvpp: free msdk surfaces to avoid memory leak 2019-09-13 16:54:21 +00:00
Haihao Xiang
93f72a5041 msdkvpp: unref buffer pools
Unref the pools first in caps re-negotiation to avoid memory leak
2019-09-13 16:54:21 +00:00
Haihao Xiang
ffa82e07a7 msdkvpp: don't return GST_FLOW_ERROR for MFX_WRN_INCOMPATIBLE_VIDEO_PARAM
Returning MFX_WRN_INCOMPATIBLE_VIDEO_PARAM means MSDK detects some
incompatible parameters but it is resolved, and we may not regard
MFX_WRN_INCOMPATIBLE_VIDEO_PARAM as a fatal error. In this fix,
GST_FLOW_OK is returned but with a warning message so that a pipeline
may run to the end.
2019-08-28 09:37:06 +00:00
Haihao Xiang
d5fd9bc22b msdkvpp: support video-direction property
video-direction property is common property in gstreamer. In addition,
both mirroring & rotation properties are marked as deprecated,
video-direction will override mirroring & rotation properties when they
are set explicitly

Fix https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1058
2019-08-26 16:39:39 +00:00
Haihao Xiang
20f61db2e5 msdkdec: make sure mfx frame width/height meets MSDK's requirement
It is possible that the output region size (e.g. 192x144) is different
from the coded picture size (e.g. 192x256). We may adjust the alignment
parameters so that the padding is respected in GstVideoInfo and use
GstVideoInfo to calculate mfx frame width and height

This fixes the error below when decoding a stream which has different
output region size and coded picture size

0:00:00.057726900 28634 0x55df6c3220a0 ERROR                msdkdec
gstmsdkdec.c:1065:gst_msdkdec_handle_frame:<msdkh265dec0>
DecodeFrameAsync failed (failed to allocate memory)

Sample pipeline:

gst-launch-1.0 filesrc location=output.h265 ! h265parse ! msdkh265dec !
glimagesink
2019-07-24 08:41:13 +08:00
Seungha Yang
9ec62418c3 msdk: Do not expose DMA buffer caps feature on Windows
On Windows, DMA buffer is not supported. PadTemplate with actually
supported feature seems to more make sense.
2019-07-22 16:13:43 +00:00
Haihao Xiang
98e4967337 msdk: don't share context between msdkvpp and msdkenc
msdkenc supports CSC implicitly, so it is possible that two VPP
processes are required when a pipeline contains msdkvpp and msdkenc.
Before this fix, msdkvpp and msdkenc may share the same context, hence
the same mfx session, which results in MFX_ERR_UNDEFINED_BEHAVIOR
in MSDK because a mfx session has at most one VPP process only

This fixes the broken pipelines below:

gst-launch-1.0 videotestsrc ! video/x-raw,format=I420 ! msdkh264enc ! \
msdkh264dec ! msdkvpp ! video/x-raw,format=YUY2 ! fakesink

gst-launch-1.0 videotestsrc ! msdkvpp ! video/x-raw,format=YUY2 ! \
msdkh264enc ! fakesink
2019-06-30 06:08:31 +00:00
Haihao Xiang
5b369e6e97 msdk: fix the build error with libva 2.4.0
This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/949
2019-04-17 13:10:10 +08:00
Haihao Xiang
e81a22d741 msdkvpp: add BGR10A2_LE format in the src pad
example pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=P010_10LE ! msdkvpp ! \
video/x-raw,format=BGR10A2_LE ! fakesink
2019-04-09 14:26:02 +00:00
Haihao Xiang
ac1408874b msdkvpp: add VUYA format in the src and sink pads 2019-03-25 09:50:33 +00:00
Seungha Yang
1d17e3891d mskd: Don't use MFX_FOURCC_RGB565 if it's undefined
../subprojects/gst-plugins-bad/sys/msdk/msdk.c(61): error C2065: 'MFX_FOURCC_RGB565'

The minimum required version for the format seems to MFX_VERSION >= 1028
2019-03-01 07:03:01 +00:00
Haihao Xiang
0806f94e1c msdkvpp: add RGB16 format in the sink pad
Note: MSDK doesn't support RGB16 output, hence don't add RGB16 format in
the src pad
2019-02-27 15:22:43 +00:00
Haihao Xiang
8213d0b858 msdkvpp: allow UYVY in DMABuf mode in the sink pad
Tested with the pipeline below:

gst-launch-1.0 videotestsrc num-buffers=1 ! msdkvpp ! \
video/x-raw\(memory:DMABuf\),format=UYVY ! msdkvpp ! video/x-raw, \
format=YUY2 ! filesink location=a.yuv
2019-02-19 11:04:47 +00:00
Haihao Xiang
56cada6902 msdkvpp: allow UYVY in the src pad
This make the pipeline below works:

gst-launch-1.0 videotestsrc num-buffers=1 ! msdkvpp ! \
video/x-raw,format=UYVY ! filesink location=a.yuv

Once https://github.com/intel/media-driver/pull/526 in the media-driver
is merged, the pipeline below also works:

gst-launch-1.0 videotestsrc num-buffers=1 ! msdkvpp ! \
video/x-raw\(memory:DMABuf\),format=UYVY ! filesink location=a.yuv
2019-02-19 11:04:47 +00:00
Haihao Xiang
ea6789fec8 msdkvpp: allow BGRx in DMABuf mode in the src pad
Tested with the pipeline below:

gst-launch-1.0 videotestsrc ! msdkvpp ! \
video/x-raw\(memory:DMABuf\),format=BGRx ! glimagesink
2019-02-19 11:04:47 +00:00
Haihao Xiang
697d41c923 msdkvpp: add P010_10LE to sink and src caps
example pipelines:

gst-launch-1.0 videotestsrc num-buffers=100 ! \
video/x-raw,format=P010_10LE,width=352,height=288 ! msdkvpp ! \
video/x-raw,format=NV12! fakesink

gst-launch-1.0 videotestsrc num-buffers=100 ! \
video/x-raw,format=NV12,width=352,height=288 ! msdkvpp ! \
video/x-raw,format=P010_10LE ! fakesink
2019-01-21 17:32:17 +00:00
Haihao Xiang
6c887f982e msdk: check whether mfx function call fails
And log the failures for debugging
2019-01-09 12:43:56 +08:00
Haihao Xiang
ba7f4de298 msdk: release resources if failing to create the buffer pool
Needn't check the pointers against NULL because the pointers are
non-NULL
2019-01-09 09:42:56 +08:00
Haihao Xiang
a0943aec69 msdkvpp: don't use NV12 as default output in normal mode
If so, BGRA is the preferred output format hence BGRA will be selected
as input format by default, e.g. in the pipleline below, BGRA instead of
NV12 is selected without renegotiation, so we can avoid the NV12 issue
(see commit 3f2314a) by default.

gst-launch-1.0 videotestsrc ! msdkvpp ! glimagesink
2019-01-08 10:08:43 +08:00
Haihao Xiang
7681566f29 msdkvpp: close the current VPP session if this session has been initialized
Otherwise MFXVideoVPP_Init will fail because it is called twice without
a close.

Example pipeline:

gst-launch-1.0 videotestsrc ! msdkvpp ! glimagesink

Sometimes glimagesink emits GST_EVENT_RECONFIGURE event which results
in that MFXVideoVPP_Init is called twice, then get the negotiation
failure below:

0:00:00.093715518 21218 0x558ef56231e0 ERROR                msdkvpp
gstmsdkvpp.c:995:gst_msdkvpp_initialize:<msdkvpp0> Init failed
(undefined behavior)
WARNING: from element /GstPipeline:pipeline0/GstMsdkVPP:msdkvpp0: not
negotiated

After applying this commit, the pipeline above may run without
negotiation failure, however NV12 layout in dmabuf mode is selected in
renegotiation, the display image is corrupted due to the NV12 issue which
was mentioned in commit 3f2314a. Some other fixes are needed to avoid
renegotiation by default
2019-01-08 10:08:43 +08:00
Haihao Xiang
e8d2f58e23 msdkvpp: don't update pads' info if nothing have changed 2019-01-08 10:08:43 +08:00
Haihao Xiang
88586b4149 msdk: change the wait time for encoder and vpp
In MSDK samples, the wait time for encoder, decoder and vpp is
300000. Let's set the wait time to the same value in msdk plugin
2018-12-12 17:24:00 +00:00
Haihao Xiang
51dc388783 msdk: correct the error message
It is VPP session instead of Encoder session
2018-12-12 17:24:00 +00:00
Haihao Xiang
718b7d0b5d msdk: check the created context against NULL pointer
gst_msdk_context_new_with_parent() may return NULL
2018-12-12 17:24:00 +00:00
Seungha Yang
f106a10994 msdk: Fix building on MSVC
And add missing #ifdef _WIN32 for MSVC build

https://bugzilla.gnome.org/show_bug.cgi?id=797211
2018-10-02 04:04:24 +05:30
Wang,Fei
10f57b73f3 msdk: vpp: remove mfxExtVPPDoUse from vpp filters.
According to msdk spec, there are two ways to enable filters:
1: Filters can be enabled by adding a filter ID
to mfxExtVPPDoUse. In this case, default filter parameters are used
2: Add filter configuration structures directly to mfxVideoParam.

Using 1 with 2 is optional but legal. Unfortunately it won't work
with some specific use cases like Detail/EdgeEnhancement.
Let's stick with option2 which works fine for all VPP operations.

https://bugzilla.gnome.org/show_bug.cgi?id=796468
2018-06-07 15:31:54 -08:00
Sreerenj Balachandran
c4809aa16c msdk: vpp: set passthrough from set_caps method for code clarity
Call passthrough setting method from set_caps so that
msdk initialize subroutine looks more clear.
2018-06-07 15:30:23 -08:00
Sreerenj Balachandran
665f4a140f mskd: vpp: error out gracefully instead of segfaulting if Init failed
Since we do the MSDK initializing in set_caps(), a FALSE
return may still cause the invokation of set_caps() again
and this will leads to buffer allocation and other mess-up.
So make sure the msdk initialized correctly before trying
to do any buffer allocation.

https://bugzilla.gnome.org/show_bug.cgi?id=796465
2018-06-07 15:29:29 -08:00
Sreerenj Balachandran
6cd12cb6a1 msdk: vpp: Add filters to VideoParm before doing the Query
Make sure all the enabled filter structures are added in the
mfxVideoParm before doing the VPPQuery so that msdk
can do the input param validation

https://bugzilla.gnome.org/show_bug.cgi?id=796465
2018-06-07 15:28:44 -08:00
Sreerenj Balachandran
8c7a457669 msdk: vpp: fix the filter count in mfxExtVPPDoUse
Repostion the mfxExtVPPDoUse enabling code
so that it will get the filter algorithm count correctly.
2018-06-05 17:01:13 -08:00
Sreerenj Balachandran
3f2314a1a9 msdk: vpp: don't use NV12 as vpp default output for DMABuf usecase
Using NV12 layout in dmabuf mode giving mis-aligned
VPP output with the media-driver. Keep the NV12 support
(so that we can file the bug agianst msdk or mediadriver),
but lower the ordering so that BGRA picks as default.

NV12 issue can be reproduced with explicit capfilter:
vidoetestsrc ! msdkvpp ! video/x-raw\(memory:DMABuf\),format=NV12 ! glimagesink
2018-05-30 16:29:41 -08:00
Sreerenj Balachandran
a972d76784 msdk: vpp: Add supprot for dmabuf-import
MediaSDK requires all the input and output buffers to be
pre-allocated during init phase and this won't work with
current design of GStreamer or gst-msdk. But this can be
done with https://bugzilla.gnome.org/show_bug.cgi?id=795747

There is a workaround possible as per
https://github.com/Intel-Media-SDK/MediaSDK/issues/155#issuecomment-381790504
by faking the mem-id during MFXInit.
This patch do this in gst-msdk by replacing the MemID of mfxSurface
with dmabuf-backed vasurface dynamically.

Important: v4l2 ! msdkvpp won't work without a copy because
of the GMMLib (https://github.com/intel/gmmlib) memory restrictions.

https://bugzilla.gnome.org/show_bug.cgi?id=794817
2018-05-30 16:24:24 -08:00
Sreerenj Balachandran
e1a90f1ec9 msdkvpp: Disable passthrough if memory capsfeature changes
So far msdk produced dmabuf fds are non-mappable.
If user wants to download the content of underlined surfaces,
dmabufcapsfeature negotiated pipeline will fail. So if the input surface
is dmabuf and downstream doesn't have support for dmabuf capsfeatures,
we do the vpp (no passthrough) and produce the mappable videomemory
buffers.

https://bugzilla.gnome.org/show_bug.cgi?id=794946
2018-04-30 12:40:32 -08:00
Sreerenj Balachandran
ef6e186801 msdk: vpp: Add dmabuf-export support
Currenly, the dmabuf buffer pool can be negotiated
only through DMABuf capsfeatures.
This will not allow to negotiate dmabuf support with
v4l2src (v4l2src ! msdkvpp) where v4l2src always export
the dmabuf based memory with out using the DMABuf capsfeatures.
So it requires fix based on:
https://bugzilla.gnome.org/show_bug.cgi?id=794817

https://bugzilla.gnome.org/show_bug.cgi?id=794946
2018-04-30 12:39:52 -08:00
Sreerenj Balachandran
76bbefe3b0 msdk: vpp: Add YV12, YUY2 and BGRx formats to template 2018-04-25 12:33:16 -08:00
Sreerenj Balachandran
5184f85d77 msdk: vpp: Allocation query fixes
prpose_allocation:
-- always instantiate a pool for for upstream
-- use async_depth + 1 as min buffer count

decide_allocation:
-- always create a new bufferpool for source pad.
Each of the msdk element has to create it's own mfxsurfacepool
which is an msdk contraint. For eg: Each Msdk component (vpp, dec and
enc)
will invoke the external Frame allocator for video-memory usage
So sharing the pool between gst-msdk elements might not be a good idea.

https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-25 12:33:00 -08:00
Sreerenj Balachandran
e4b4f09496 msdk: vpp : Add frame rate control
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 11:10:20 -08:00
Sreerenj Balachandran
c0ea4bdafb msdk: vpp : Add force-aspect-ratio property
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:45 -08:00
Sreerenj Balachandran
fb8c536393 msdk: Add more scaling filter algorithms
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:35 -08:00
Sreerenj Balachandran
51b6345dc4 msdk: vpp: Add support for horizontal and vertical mirroring
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:24 -08:00
Sreerenj Balachandran
108c8fde7f msdk: vpp: Add detail/edge enhancement tuning
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:13 -08:00
Sreerenj Balachandran
93c5dd2478 msdk: vpp: Add ProAmp(colorbalance) support
Added Hue, Saturation, Brightness and Contrast tuning support.

Fixme: Add GstColorBalanceInterface support

https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:02 -08:00
Sreerenj Balachandran
f5a3d3d799 msdk: vpp: Add deinterlacing support
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:38:52 -08:00
Sreerenj Balachandran
36e81744d1 msdk: vpp:Add more filters
-- Add Denoise
-- Add Rotation

https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:38:41 -08:00
Sreerenj Balachandran
d3d89f02b3 msdk: Add VPP element
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:38:19 -08:00