Commit graph

419 commits

Author SHA1 Message Date
Haihao Xiang
28a1b0c418 msdkdec: avoid infinite loop
It is possible MFXVideoDECODE_DecodeFrameAsync returns MFX_ERR_INCOMPATIBLE_VIDEO_PARAM
and this error can't be recovered by retrying MFXVideoDECODE_DecodeFrameAsync
in some cases, so we need to limit the number of retries to avoid infinite loop.

This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/909
2019-03-25 09:10:38 +00:00
Haihao Xiang
2e3022c553 msdk: relicense the plugin to LGPL
Some files licensed under LGPL were included, so relicense the plugin
from BSD to LPGL. In addition, this patch removes the README per the
discussion in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/853

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/853
2019-03-17 18:15:46 +00:00
Seungha Yang
1d17e3891d mskd: Don't use MFX_FOURCC_RGB565 if it's undefined
../subprojects/gst-plugins-bad/sys/msdk/msdk.c(61): error C2065: 'MFX_FOURCC_RGB565'

The minimum required version for the format seems to MFX_VERSION >= 1028
2019-03-01 07:03:01 +00:00
Haihao Xiang
4b8e3b215a msdkdec: fix for small resolution 2019-02-28 11:51:06 +00:00
Haihao Xiang
18d410b81a msdkdec: fix for resolution change
Returning MFX_ERR_INCOMPATIBLE_VIDEO_PARAM from
MFXVideoDECODE_DecodeFrameAsync means the allocated mfx surface is not
suitable for the current frame, we need a new mfx surface and try
MFXVideoDECODE_DecodeFrameAsync again.
2019-02-28 11:08:24 +00:00
Haihao Xiang
a751b33072 msdkdec: force the alignment of width/height to 16 for vp8/vp9
MSDK library requires 16 alignment for vp8/vp9, otherwise a pipeline
for vp8/vp9 decoding might fail.

example pipeline:
gst-launch-1.0 filesrc location=vp8_1280x720.webm ! matroskademux ! \
msdkvp8dec ! fakesink

0:00:00.150565444 10657 0x55c8484036d0 ERROR                msdkdec
gstmsdkdec.c:1056:gst_msdkdec_handle_frame:<msdkvp8dec0>
DecodeFrameAsync failed (invalid video parameters)
2019-02-27 15:56:29 +00:00
Haihao Xiang
0806f94e1c msdkvpp: add RGB16 format in the sink pad
Note: MSDK doesn't support RGB16 output, hence don't add RGB16 format in
the src pad
2019-02-27 15:22:43 +00:00
Haihao Xiang
f39bcd3920 msdk: set some parameters in mfxFrameData for a RGB16 frame 2019-02-27 15:22:43 +00:00
Haihao Xiang
52b5a7e206 msdk: map MFX_FOURCC_RGB565 to VA_FOURCC_RGB565 2019-02-27 15:22:43 +00:00
Haihao Xiang
a09cff84ce msdk: map GST_VIDEO_FORMAT_RGB16 to VA_FOURCC_RGB565 2019-02-27 15:22:43 +00:00
Haihao Xiang
197f7f569b msdk: map GST_VIDEO_FORMAT_RGB16 to MFX_FOURCC_RGB565 2019-02-27 15:22:43 +00:00
Haihao Xiang
6287016a74 msdkdec: release the occupied surface for MFX_WRN_DEVICE_BUSY
When MFXVideoDECODE_DecodeFrameAsync () returns MFX_WRN_DEVICE_BUSY with
an output surface, a new input surface is required when retrying
MFXVideoDECODE_DecodeFrameAsync ().

This fixes the out-of-surface issue mentioned in
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/890
2019-02-24 19:13:57 +00:00
Haihao Xiang
fffb3f628d msdk: don't destroy an image twice
This gets rid of annoying message in the log, e.g. run the pipeline
below:

gst-launch-1.0 videotestsrc num-buffers=100 ! \
video/x-raw,format=NV12,width=352,height=288 ! msdkh264enc ! filesink \
location=test.h264

[LIBVA]:CRITICAL - DdiMedia_DestroyImage:4357: Invalid image
2019-02-24 17:59:43 +00:00
Haihao Xiang
348a4b570d msdk: move the comment to the correct place 2019-02-24 17:59:43 +00:00
Tim-Philipp Müller
16426b3928 msdk: add Hardware tag to element metadata 2019-02-19 23:44:42 +00:00
Haihao Xiang
8213d0b858 msdkvpp: allow UYVY in DMABuf mode in the sink pad
Tested with the pipeline below:

gst-launch-1.0 videotestsrc num-buffers=1 ! msdkvpp ! \
video/x-raw\(memory:DMABuf\),format=UYVY ! msdkvpp ! video/x-raw, \
format=YUY2 ! filesink location=a.yuv
2019-02-19 11:04:47 +00:00
Haihao Xiang
56cada6902 msdkvpp: allow UYVY in the src pad
This make the pipeline below works:

gst-launch-1.0 videotestsrc num-buffers=1 ! msdkvpp ! \
video/x-raw,format=UYVY ! filesink location=a.yuv

Once https://github.com/intel/media-driver/pull/526 in the media-driver
is merged, the pipeline below also works:

gst-launch-1.0 videotestsrc num-buffers=1 ! msdkvpp ! \
video/x-raw\(memory:DMABuf\),format=UYVY ! filesink location=a.yuv
2019-02-19 11:04:47 +00:00
Haihao Xiang
ac9423facb msdk: map GST_VIDEO_FORMAT_UYVY to VA_FOURCC_UYVY 2019-02-19 11:04:47 +00:00
Haihao Xiang
ea6789fec8 msdkvpp: allow BGRx in DMABuf mode in the src pad
Tested with the pipeline below:

gst-launch-1.0 videotestsrc ! msdkvpp ! \
video/x-raw\(memory:DMABuf\),format=BGRx ! glimagesink
2019-02-19 11:04:47 +00:00
Haihao Xiang
028338ff44 msdkdec: set decode_only for output only
MSDK may return MFX_ERR_MORE_DATA but without output surface

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/887
2019-02-12 11:39:04 +00:00
Haihao Xiang
5f2f289b7e msdk: verify the driver name
We need to make sure the Intel graphics card is used in a dual GPU
system.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/895
2019-02-12 12:13:31 +08:00
Haihao Xiang
6c953438f5 msdkdec: Release occupied surface for MFX_ERR_MORE_DATA
An output surface is returned but without sync point when when
MFXVideoDECODE_DecodeFrameAsync () returns MFX_ERR_MORE_DATA, this
surface should be released too, otherwise the surface is occupied
and it is easy to exhaust all pre-allocated mfx surfaces.

Example pipeline (input_vp8.webm contains lots of frame with show_frame
set to 0):

gst-launch-1.0 filesrc location=input_vp8.webm ! matroskademux !
msdkvp8dec ! msdkvpp ! fakesink

0:00:05.995959693 19866 0x563f30f14590 ERROR                default
gstmsdkvideomemory.c:77:gst_msdk_video_allocator_get_surface: failed to
get surface available
ERROR: from element
/GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0: Internal data
stream error.
2019-02-03 13:44:56 +08:00
Víctor Manuel Jáquez Leal
f974246647 msdkdec: explain the the extra ref of input buffer 2019-02-02 17:52:58 +01:00
Haihao Xiang
74f297f31b msdkdec: avoid releasing the input buffer when it is still in use
The input buffer is released in gst_msdkdec_finish_task () when decoding
some special clips however this buffer is still in use, so ref the input
buffer before gst_msdkdec_finish_task () and unref it at the end of
gst_msdkdec_handle_frame ().

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/862
2019-02-02 16:32:02 +00:00
Haihao Xiang
cd55f8635f msdkvp9dec: output P010_10LE buffer if possible 2019-01-31 10:30:22 +08:00
Haihao Xiang
edc0edc960 msdkvp9dec: use separate src caps
We will add more formats in the src caps of msdkvp9dec, so let
msdkvp9dec uses separate src caps. Note it doesn't change any
capability
2019-01-31 10:30:22 +08:00
Haihao Xiang
546797cecf msdkdec: set the mfx picture structure to MFX_PICSTRUCT_PROGRESSIVE if it is unknown
The picture structure in the output parameters from
MFXVideoDECODE_Query is set to MFX_PICSTRUCT_UNKNOWN for some codecs, so
the structure of the corresponding mfx surfaces created for decoding are
unknown. The pipeline will be broken when these surfaces are used as the
input for msdkvpp.

Example pipeline:

gst-launch-1.0 filesrc location=input_vp8.webm ! matroskademux !
msdkvp8dec ! msdkvpp ! fakesink

Error message:

0:00:00.031568911 14259 0x55b79dc684a0 ERROR                msdkvpp
gstmsdkvpp.c:728:gst_msdkvpp_transform:<msdkvpp0> MSDK Failed to do VPP
ERROR: from element
/GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0: Internal data
stream error.

This is a workaround for the above issue
2019-01-25 12:45:43 +08:00
Haihao Xiang
920062cafc msdkh265enc: enable low power mode
Low power mode is disabled by default, set the value of low-power to
true to enable this mode.
2019-01-24 09:59:37 +00:00
Víctor Manuel Jáquez Leal
6b6ed2cad8 msdk: bufferpool: refactor memory type decision
The memory type was used as bitwise enum, but the enum was not
defined in that way.

Nonetheless, most of the usage of the memory type was as mutually
exclusive options, rather than option composition.

This patch refactor how the memory type is defined, so it is kept
the mutual exclusion among options.
2019-01-24 08:49:39 +00:00
Víctor Manuel Jáquez Leal
5d030ed776 msdk: fix some comments typos 2019-01-24 08:49:39 +00:00
Víctor Manuel Jáquez Leal
5f67b4946e msdk: decoder: fatal failure if negotiations fails
Otherwise it crashes when no parser is used placed upstream, v.gr.

gst-launch-1.0 filesrc location=file.264 ! msdkh264dec ! fakesink
2019-01-24 08:49:39 +00:00
Víctor Manuel Jáquez Leal
f238066288 msdk: decoder: comment to explain array reset 2019-01-24 08:49:39 +00:00
Víctor Manuel Jáquez Leal
882befb5db msdk: decoder: fix an usage-after-free pointer bug 2019-01-24 08:49:39 +00:00
Haihao Xiang
697d41c923 msdkvpp: add P010_10LE to sink and src caps
example pipelines:

gst-launch-1.0 videotestsrc num-buffers=100 ! \
video/x-raw,format=P010_10LE,width=352,height=288 ! msdkvpp ! \
video/x-raw,format=NV12! fakesink

gst-launch-1.0 videotestsrc num-buffers=100 ! \
video/x-raw,format=NV12,width=352,height=288 ! msdkvpp ! \
video/x-raw,format=P010_10LE ! fakesink
2019-01-21 17:32:17 +00:00
Haihao Xiang
5e3f94ce28 msdkh265dec: support P010_10LE in DMABuf mode 2019-01-21 17:32:17 +00:00
Nirbheek Chauhan
f710b36ac4 meson: Fix automagic build of msdk plugin
When building the msdk plugin even if libmfx is found, unless the
plugin is explicitly enabled we should not error out if msdk
dependencies are not found.

Also give an error message when we don't build the plugin on Windows
because we're not building with MSVC.
2019-01-19 18:28:42 +00:00
Nirbheek Chauhan
7ffcab3478 build: Don't forget to add windows files to EXTRA_DIST 2019-01-18 13:13:09 +05:30
Haihao Xiang
9aa68dea51 msdk: set the right BRCParamMultiplier
BRCParamMultiplier in mfxInfoMFX is a parameter which specifies a
multiplier for bitrate control parameters [1], it impacts TargetKbps,
MaxKbps, BufferSizeInKB and InitialDelayInKB.

[1]: https://software.intel.com/en-us/node/628473
2019-01-11 09:57:44 +00:00
Haihao Xiang
5fc160fa59 msdk: set the upper bound of max-vbv-bitrate to 2048000 kbps
The upper bound of bitrate is also 2048000 kbps which should be large
enough in practice.

Fix https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/863
2019-01-11 09:57:44 +00:00
Nirbheek Chauhan
0b94ed81a4 meson: Fix building of MSDK plugin on Windows
Variable name was typoed in 604c8d5232
2019-01-10 15:34:51 +05:30
Haihao Xiang
d7ad665d1c msdk: needn't call MFXJoinSession after MFXCloneSession
According to the MSDK documation[1], MFXCloneSession is a light-weight
equivalent of MFXJoinSession after MFXInit, so MFXJoinSession call isn't
needed in the msdk plugin, otherwise the cloned session is joined to the
parent session twice, and we will get a MFX error when closing the
parent session

example pipeline:

gst-launch-1.0 videotestsrc num-buffers=100 ! \
video/x-raw,format=NV12,width=352,height=288 ! msdkh264enc ! msdkh264dec ! \
msdkh264enc ! fakesink

Error message:

0:00:00.211948518 21733 0x5586ee741c60 ERROR                   msdk
msdk.c:148:msdk_close_session: Close failed (undefined behavior)

[1]: https://software.intel.com/en-us/node/628429#MFXCloneSession
2019-01-09 15:52:27 +08:00
Haihao Xiang
6c887f982e msdk: check whether mfx function call fails
And log the failures for debugging
2019-01-09 12:43:56 +08:00
Haihao Xiang
d7606a7667 msdkenc: return NULL instead of FALSE
The returned type is a pointer
2019-01-09 09:50:49 +08:00
Haihao Xiang
ba7f4de298 msdk: release resources if failing to create the buffer pool
Needn't check the pointers against NULL because the pointers are
non-NULL
2019-01-09 09:42:56 +08:00
Víctor Manuel Jáquez Leal
a301150002 msdk: meson: compile vp9dec if available
based on the patches provided by Haihao Xiang <haihao.xiang@intel.com>
2019-01-08 12:59:45 +00:00
Víctor Manuel Jáquez Leal
c22f92c799 msdk: add mfx in include path
Thus removing the preprocessor's directives to included if found.
2019-01-08 12:59:45 +00:00
Víctor Manuel Jáquez Leal
604c8d5232 msdk: meson: use libmfx pkg-config if available
Refactoring to bail out early if MediaSDK is not found.

based on the patches provided by Haihao Xiang <haihao.xiang@intel.com>
2019-01-08 12:59:45 +00:00
Haihao Xiang
3110f3791f msdk: don't reset the external frame allocator
In gst-msdk, a mfx session may be shared between different gst
elements, each element tries to set the frame allocator. However, per
the MSDK documation[1], the behavior is undefined if reset the frame
allocator while the previous allocator is in use. Fortunately all
elements use the same frame allocator, so we can avoid to call
MFXVideoCORE_SetFrameAllocator again.

[1]: https://software.intel.com/en-us/node/628430#MFXVideoCORE3
2019-01-08 09:11:47 +00:00
Haihao Xiang
a0943aec69 msdkvpp: don't use NV12 as default output in normal mode
If so, BGRA is the preferred output format hence BGRA will be selected
as input format by default, e.g. in the pipleline below, BGRA instead of
NV12 is selected without renegotiation, so we can avoid the NV12 issue
(see commit 3f2314a) by default.

gst-launch-1.0 videotestsrc ! msdkvpp ! glimagesink
2019-01-08 10:08:43 +08:00
Haihao Xiang
7681566f29 msdkvpp: close the current VPP session if this session has been initialized
Otherwise MFXVideoVPP_Init will fail because it is called twice without
a close.

Example pipeline:

gst-launch-1.0 videotestsrc ! msdkvpp ! glimagesink

Sometimes glimagesink emits GST_EVENT_RECONFIGURE event which results
in that MFXVideoVPP_Init is called twice, then get the negotiation
failure below:

0:00:00.093715518 21218 0x558ef56231e0 ERROR                msdkvpp
gstmsdkvpp.c:995:gst_msdkvpp_initialize:<msdkvpp0> Init failed
(undefined behavior)
WARNING: from element /GstPipeline:pipeline0/GstMsdkVPP:msdkvpp0: not
negotiated

After applying this commit, the pipeline above may run without
negotiation failure, however NV12 layout in dmabuf mode is selected in
renegotiation, the display image is corrupted due to the NV12 issue which
was mentioned in commit 3f2314a. Some other fixes are needed to avoid
renegotiation by default
2019-01-08 10:08:43 +08:00
Haihao Xiang
e8d2f58e23 msdkvpp: don't update pads' info if nothing have changed 2019-01-08 10:08:43 +08:00
U. Artie Eoff
e91a53f693 msdkenc: break out of flush frames loop on error
In general, we should assume any unhandled error is
non-recoverable.

In the flush frames loop, some error states can cause us
to never increment the task and therefore we get stuck
in an infinite loop and generate GST_ELEMENT_ERROR
over and over again.  This eventually consumes all
system memory and triggers OOM.  Thus, assume the worst
and break out of the loop upon the first "unhandled" error.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/859
2019-01-06 19:07:35 -08:00
Seungha Yang
373580c7be msdk: Fix invalid return type build warning
void function cannot have return value

gst-plugins-bad/sys/msdk/gstmsdkbufferpool.c(332): warning C4098:
'gst_msdk_buffer_pool_release_buffer': ...
2018-12-25 16:14:36 +09:00
Haihao Xiang
88586b4149 msdk: change the wait time for encoder and vpp
In MSDK samples, the wait time for encoder, decoder and vpp is
300000. Let's set the wait time to the same value in msdk plugin
2018-12-12 17:24:00 +00:00
Haihao Xiang
51dc388783 msdk: correct the error message
It is VPP session instead of Encoder session
2018-12-12 17:24:00 +00:00
Haihao Xiang
718b7d0b5d msdk: check the created context against NULL pointer
gst_msdk_context_new_with_parent() may return NULL
2018-12-12 17:24:00 +00:00
Haihao Xiang
801189c02f msdk: decrease the reference count of object
Otherwise there are reference leaks if failed to clone or join a
mfx session
2018-12-12 17:24:00 +00:00
Haihao Xiang
43e5eaeda1 msdk: assign the returned value to status variable
Othervise the subsequent check will use the stale value of status
variable
2018-12-12 17:24:00 +00:00
Xiang, Haihao
9a9e01a1a5 msdkh265enc: output main-10 bitstream if the input is P010_10LE
Tested on KBL using the following command:
gst-launch-1.0 videotestsrc num_buffers=100 ! video/x-raw,format=P010_10LE ! \
msdkh265enc ! filesink location=a.hevc

This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/829
2018-12-12 09:00:13 +00:00
Xiang, Haihao
9711a21008 msdkh265enc: add P010_10LE to the sink pad template 2018-12-12 09:00:13 +00:00
Xiang, Haihao
ba1f162401 msdkh265enc: re-add the sink pad template
We will add more profiles in the sink caps of msdkh265enc, so let
msdkh265enc re-add the sink pad template. Note this change doesn't
impact any capability
2018-12-12 09:00:13 +00:00
Xiang, Haihao
f45d5dc8bc msdkenc: handle P010_10LE input format
Note it is up to each codec to support P010_10LE format
2018-12-12 09:00:13 +00:00
Xiang, Haihao
506835a484 msdkenc: use macro GST_VIDEO_INFO_FORMAT if possible 2018-12-12 09:00:13 +00:00
Xiang, Haihao
0e7e425bb9 msdk: add missing breaks 2018-12-03 07:09:56 +00:00
Xiang, Haihao
a1aae3fd92 msdk: fix the wrong operator
The condition is for video memory only, so the operator should be
& instead of |
2018-12-03 07:09:56 +00:00
Xiang, Haihao
b0d22f8cb4 msdk: remove unnecessary assignment
CodecProfile will be set in MFXVideoDECODE_DecodeHeader() to match
the input stream. Setting the hard-coded profile here will mislead
user that msdkh265dec supports a special profile only.
2018-11-26 16:47:17 +00:00
Xiang, Haihao
b38f8d301c msdk: update the sink and src caps of msdkh265dec
Now hevc 10bit video can be decoded correctly, so update the sink
and src caps accordingly.

This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/720
2018-11-26 16:47:17 +00:00
Xiang, Haihao
2b0923842a msdk: set right BitDepth and Shift for P010 mfx frame
BitDepth is 10 and Shitf must be set to 1 when creating P010 mfx
frame in MSDK
2018-11-26 16:47:17 +00:00
Xiang, Haihao
fda4918cc8 msdk: don't use hard-coded video format
Some codecs may support varied formats, e.g. HEVC may support NV12
and P010_10LE etc
2018-11-26 16:47:17 +00:00
Xiang, Haihao
27ab7fb40f msdk: adjust the RT format for P010 surface
According to VA API, VA_RT_FORMAT_YUV420_10 is expected for P010
surface
2018-11-26 16:47:17 +00:00
Xiang, Haihao
eeb58f64c2 msdk: create VA_FOURCC_P010 surface from dmabuf 2018-11-26 16:47:17 +00:00
Xiang, Haihao
7d58424611 msdk: VA_FOURCC_P010 frame lock
P010 and NV12 have the same layout, so we may reuse the code in
gst_msdk_frame_lock()
2018-11-26 16:47:17 +00:00
Xiang, Haihao
f72c825194 msdk: map MFX_FOURCC_P010 to VA_FOURCC_P010 2018-11-26 16:47:17 +00:00
Xiang, Haihao
dc070a8306 msdk: make sure the surface data is set for GST_VIDEO_FORMAT_P010_10LE
P010_10LE and NV12 have the same layout, so we may reuse the code.
2018-11-26 16:47:17 +00:00
Haihao Xiang
54451b7303 msdk: map GST_VIDEO_FORMAT_P010_10LE to MFX_FOURCC_P010 2018-11-26 16:47:17 +00:00
Xiang, Haihao
1d97a6f00e msdk: add an assert in gst_msdk_frame_lock() for unhandled formats
We will add support for more formats, e.g. P010, Adding an assert
here may catch the error early
2018-11-26 16:47:17 +00:00
Haihao Xiang
1c0160513b msdk: use separate src caps for msdkh265dec
We will add more formats in the src caps of msdkh265dec, so let
msdkh265dec uses separate src caps. Note it doesn't change any
capability
2018-11-26 16:47:17 +00:00
Xiang, Haihao
1bfd1b2bf9 msdk: Make sure a variable is initialized when it is used
Previously alloc_info is initialized when both thiz->initialized
and thiz->allocation_caps are true, but only thiz->initialized is
checked when alloc_info is used.
2018-11-26 16:13:44 +00:00
Xiang, Haihao
38cf42f90f msdk: Close fd handle
Otherwise it will result in resource leak if mem == NULL
2018-11-26 16:13:44 +00:00
Víctor Manuel Jáquez Leal
09dbe83e5d msdk: don't declare headers in meson
This partially reverts commit 0bae648353.

The compiler tells ninja the header dependency, then there is
no need to add them explicity.
2018-11-23 17:10:24 +01:00
Víctor Manuel Jáquez Leal
0bae648353 msdk: declare headers in meson 2018-11-23 13:50:20 +01:00
Wangfei
409d4ff30e msdkdec: move output-order out of decode bass class.
Since output-order is a deprecated attribute, move it out of decode
bass class and configure it in each sub decoder class who need it.

https://bugzilla.gnome.org/show_bug.cgi?id=796853
2018-11-23 12:26:40 +00:00
Víctor Manuel Jáquez Leal
eaa1bd93e8 msdk: add support for open sourced MediaSDK
MediaSDK has been released as open source [1], but the directories
where it installs its files, are different from the binary only
distribution.

This patch adds to the libraries path the directory /lib. Also it
is defined in meson if the include directory has the mfx/ prefix,
something that is already handled in autotools.

1. https://github.com/Intel-Media-SDK/MediaSDK
2018-11-22 17:31:58 +01:00
Jordan Petridis
1e887f9b12
meson: avoid unkown variables in the msdk tests 2018-11-06 23:48:58 +02:00
Florent Thiéry
23b0172ee9 msdk: fix intel sdk libdir path
the 2018.3.1 intel sdk release places libraries into /lib64 instead of
/lib/lin_x64 or /lib/x64, this commit adds /lib64 to the libdir
locations list

Fixes #815
2018-11-06 13:08:45 +00:00
Nirbheek Chauhan
55134df54c meson: Add a feature option for the MSDK plugin 2018-10-28 14:54:52 +00:00
Seungha Yang
f106a10994 msdk: Fix building on MSVC
And add missing #ifdef _WIN32 for MSVC build

https://bugzilla.gnome.org/show_bug.cgi?id=797211
2018-10-02 04:04:24 +05:30
Seungha Yang
44ecf53d1c msdk: Fix meson build
gst-plugins-bad\sys\msdk\meson.build:57:4: ERROR:  File msdk-enum.c does not exist.

https://bugzilla.gnome.org/show_bug.cgi?id=797211
2018-10-02 04:04:24 +05:30
Nirbheek Chauhan
b55dfb5313 Add feature options for almost all plugins
The only plugins remaining are those that haven't been ported to Meson
yet, and msdk. Also, the tests are still automagic.

https://bugzilla.gnome.org/show_bug.cgi?id=795107
2018-07-27 19:04:38 +05:30
Sreerenj Balachandran
dd981d4953 msdkdec: Fix latency calculation
Use async_depth for latency calcuation instead of
the length of Tasks array which could be NULL since we
don't do the msdk decoder init in set_format().
2018-07-03 18:38:04 -08:00
Sreerenj Balachandran
84c33be0c0 msdk: Set 16 bit alignment for width
According to MediaSDK specification,
Width must be a multiple of 16 and Height must be a multiple
of 16 for progressive frame sequence and a multiple of 32 otherwise.

This patch sets a 16 bit alignment for width and 32 bit alignment
for height as default.

https://bugzilla.gnome.org/show_bug.cgi?id=796566
2018-07-02 16:50:46 -08:00
Sreerenj Balachandran
d63a1b4e3f msdkdec: avoid early destruction of frame in dynamic resolution change
In cases where we do hard resest, the current code destroys the frame
which has new resolution bit early and this causes buffer_unmap
warnings. Keep an extra ref to the frame internally to avoid this.
2018-07-02 16:50:02 -08:00
Sreerenj Balachandran
1250af8f09 msdkdec: vc1: Fix handling of advanced profile elementary stream
Advanced profile elementary streams may not have codec_data
always. So make sure we don't do anything with null buffer.
2018-07-02 16:49:23 -08:00
Sreerenj Balachandran
ad6162e99b msdkdec: Fix advanced profile vc1 decode when codec_data presents
The gst-msdk decoders only support packetized formats for
all codecs except VC1. For VC1, it supports codec_data for advanced
profiles and this codec_data wan't submitting to MSDK's DecodeHeader APIs.
Make sure the subclass deocders correctly configured so that
the codec_data buffers are in place in the internal adapter for
MediaSDK's DecoderHeader usage.
2018-07-02 16:48:11 -08:00
Sreerenj Balachandran
9efb4c9179 msdkdec: Fix the PTS of output frames
Currently we use the gst_video_decoder_get_oldest_frame()
to get the old pending frame to output. But this is not correct
if pts re-ordering required. This patch uses a custom made
get_old_frame() which accounts the PTS too similar to the
v4l2decoder.

https://bugzilla.gnome.org/show_bug.cgi?id=796699
2018-07-02 16:42:20 -08:00
Sreerenj Balachandran
5c88da4a4c msdkdec: Remove dead code
We are not using any ExtendedParams for decoding.
2018-07-02 16:41:58 -08:00
Sreerenj Balachandran
1e95c03c7d msdk: dec: Add dynamic-configuration change support
The patch adds a serios of changes to support dynamic resolution
change and efficient utilization of resources.
Major changes:

-- Use MSDK's apis to retrieve the headers instead of only relying
on upsteram notification. For eg: avc decoder requires SEI header
information for dpb count calculation which we don't get from caps.

-- For all codecs other than VP9, we force the reset of decoder
if resoultion changes to fit with gstreamer flow. VP9 enfource
the hard reset only if the new resolution is bigger.

-- delay the src caps setting till msdk api's invokation in
handle_frame to avoid caching multiple configuration values

-- ensure pool negotiation is based on decoder's allocation_caps.

--dynamic resoluttion change use an explicit allocation_query
to reclaim the buffers before closing the decoder (thanks to v4l2dec)

--In case if we don't get upstream notification of res change (for eg,
this can can happen for vp9 frames with ivfheader where ivfparse
is not able to notify the dynamic changes), we handle the the case
based on MFX_ERR_INCOMPATIBLE_VIDEO_PARAM which is the return value
of MFXVideoDECODE_DecodeFrameAsync

-- calculate the minimum surfaces to be preallocated based on
msdk suggestion, downstream requirement, async depth and scratch surface
count for smooth display.

https://bugzilla.gnome.org/show_bug.cgi?id=796566
2018-07-02 16:17:49 -08:00
Tim-Philipp Müller
65c5b9a4f6 msdk: Update for g_type_class_add_private() deprecation in recent GLib
Untested.
2018-06-24 12:22:27 +02:00
Wang,Fei
10f57b73f3 msdk: vpp: remove mfxExtVPPDoUse from vpp filters.
According to msdk spec, there are two ways to enable filters:
1: Filters can be enabled by adding a filter ID
to mfxExtVPPDoUse. In this case, default filter parameters are used
2: Add filter configuration structures directly to mfxVideoParam.

Using 1 with 2 is optional but legal. Unfortunately it won't work
with some specific use cases like Detail/EdgeEnhancement.
Let's stick with option2 which works fine for all VPP operations.

https://bugzilla.gnome.org/show_bug.cgi?id=796468
2018-06-07 15:31:54 -08:00
Sreerenj Balachandran
c4809aa16c msdk: vpp: set passthrough from set_caps method for code clarity
Call passthrough setting method from set_caps so that
msdk initialize subroutine looks more clear.
2018-06-07 15:30:23 -08:00
Sreerenj Balachandran
665f4a140f mskd: vpp: error out gracefully instead of segfaulting if Init failed
Since we do the MSDK initializing in set_caps(), a FALSE
return may still cause the invokation of set_caps() again
and this will leads to buffer allocation and other mess-up.
So make sure the msdk initialized correctly before trying
to do any buffer allocation.

https://bugzilla.gnome.org/show_bug.cgi?id=796465
2018-06-07 15:29:29 -08:00
Sreerenj Balachandran
6cd12cb6a1 msdk: vpp: Add filters to VideoParm before doing the Query
Make sure all the enabled filter structures are added in the
mfxVideoParm before doing the VPPQuery so that msdk
can do the input param validation

https://bugzilla.gnome.org/show_bug.cgi?id=796465
2018-06-07 15:28:44 -08:00
Sreerenj Balachandran
8c7a457669 msdk: vpp: fix the filter count in mfxExtVPPDoUse
Repostion the mfxExtVPPDoUse enabling code
so that it will get the filter algorithm count correctly.
2018-06-05 17:01:13 -08:00
Sreerenj Balachandran
3f2314a1a9 msdk: vpp: don't use NV12 as vpp default output for DMABuf usecase
Using NV12 layout in dmabuf mode giving mis-aligned
VPP output with the media-driver. Keep the NV12 support
(so that we can file the bug agianst msdk or mediadriver),
but lower the ordering so that BGRA picks as default.

NV12 issue can be reproduced with explicit capfilter:
vidoetestsrc ! msdkvpp ! video/x-raw\(memory:DMABuf\),format=NV12 ! glimagesink
2018-05-30 16:29:41 -08:00
Sreerenj Balachandran
57b9875260 msdk: enc: Add supprot for dmabuf-import
MediaSDK requires all the input buffers to be
pre-allocated during init phase and this won't work with
current design of GStreamer or gst-msdk. But this can be
done in future once we have a solution for:
https://bugzilla.gnome.org/show_bug.cgi?id=795747

There is a workaround possible as per
https://github.com/Intel-Media-SDK/MediaSDK/issues/155#issuecomment-381790504
by faking the mem-id during MFXInit.
This patch enabling it in gst-msdk by replacing the MemID of mfxSurface
with dmabuf-backed vasurface dynamically.

Important: v4l2 ! msdkenc won't work without a copy because
of the GMMLib (https://github.com/intel/gmmlib) memory restrictions.

https://bugzilla.gnome.org/show_bug.cgi?id=794817
2018-05-30 16:26:27 -08:00
Sreerenj Balachandran
a972d76784 msdk: vpp: Add supprot for dmabuf-import
MediaSDK requires all the input and output buffers to be
pre-allocated during init phase and this won't work with
current design of GStreamer or gst-msdk. But this can be
done with https://bugzilla.gnome.org/show_bug.cgi?id=795747

There is a workaround possible as per
https://github.com/Intel-Media-SDK/MediaSDK/issues/155#issuecomment-381790504
by faking the mem-id during MFXInit.
This patch do this in gst-msdk by replacing the MemID of mfxSurface
with dmabuf-backed vasurface dynamically.

Important: v4l2 ! msdkvpp won't work without a copy because
of the GMMLib (https://github.com/intel/gmmlib) memory restrictions.

https://bugzilla.gnome.org/show_bug.cgi?id=794817
2018-05-30 16:24:24 -08:00
Sreerenj Balachandran
a7b7939dd7 msdk: Add method to replace internal VASurface of mfxFrameSurface
Added a utility method to replace the MemID (interanl VASurfaceID)
associated with the mfxFrameSurface. This is usefull for dmabuf-import
where we need to replace the memID dynamically

https://bugzilla.gnome.org/show_bug.cgi?id=794817
2018-05-30 16:23:44 -08:00
Sreerenj Balachandran
62d2d8ebf9 msdk: Add method to export dmabuf to VASurface
Exporting DRM_PRIME fd to VASurface requires direct
invocation of VA api VACreateSurface with
VASurfaceAttribExternalBufferDescriptor and other
necessary surface attributes.

https://bugzilla.gnome.org/show_bug.cgi?id=794817
2018-05-30 16:22:49 -08:00
Sreerenj Balachandran
0bdcf51baf msdk: Add conditional build for vp9 decoder
https://bugzilla.gnome.org/show_bug.cgi?id=796119
2018-05-15 16:33:00 -08:00
Sreerenj Balachandran
955c927189 msdk: dec: Add VP9 decoder
https://bugzilla.gnome.org/show_bug.cgi?id=796119
2018-05-15 16:32:22 -08:00
Sreerenj Balachandran
dec0953517 msdk: allow building against open sourced msdk
Building against mfx_dispatcher is used to search for
headers in PREFIX/include/mfx/ only (commit: 62f04e801b),
but it is just PREFIX/include with open source msdk version.

https://bugzilla.gnome.org/show_bug.cgi?id=796118
2018-05-15 16:31:02 -08:00
Sreerenj Balachandran
54482a54d8 msdk:dec: Add new propery to dump frames in decoded order
The new property "output-order" can be set to either "display" order
which is the default where frames will be outputting in display order,
or "decoded-order" which will be outputting the frames in decoded order.

The "decoded order" output is generally useful for debugging. But there
are few
customers who use it for low-latency streaming. For eg if the customer
already knows that the stream doesn't have b-frames (which means no
algorithm requires for display order calculation), then they can use
"decoded-order"
output to skip some of the DPB logic to avoid the frame accumulation at
start-up.

The root cause of the above issue is a bit of unclarity in h264 spec +
lazy implementation of many H264 encoders; This is well handled in
gstreamer-vaapi using "low-latency" property:
https://bugzilla.gnome.org/show_bug.cgi?id=762509

https://bugzilla.gnome.org/show_bug.cgi?id=795783
2018-05-07 14:12:10 -08:00
Sreerenj Balachandran
a372c05f06 msdk: dec: inform msdk if the buffer contains a complete frame
For packetized input, inform the msdk that the buffer has
a complete frame or complementary field pairs. For decoding,
this means that the decoder can proceed with this buffer without
waiting for the start of the next frame, which effectively reduces
decoding latency.

https://bugzilla.gnome.org/show_bug.cgi?id=795783
2018-05-07 14:11:34 -08:00
Sreerenj Balachandran
978bcf8aa6 msdk: dec: reset async depth to one
Currently we use an async depth of 4 as default (based on
recommendations
in msdk apps), which indicates how many asynchronous operations an
application performs
before the application explicitly synchronizes the result. As a result,
we
queue four frames in decoder which might not be good approach for
live streaming.

This patch reset the async-depth to 1 as default so that we do sync for
each frame we decode without queuing. Customer can play with already
exposed "async-depth" property for other use cases

https://bugzilla.gnome.org/show_bug.cgi?id=795783
2018-05-07 14:11:14 -08:00
Sreerenj Balachandran
cb1eb650c6 msdk: enc: Add dmabuf-export support
Current implementation is only supporting dmabuf-export
through DMABufCapsfeatures.
MSDK dmabuf fds are not mappable and dmabuf-import
is not yet supported too (#794817).

https://bugzilla.gnome.org/show_bug.cgi?id=795707
2018-05-02 14:52:24 -08:00
Sreerenj Balachandran
e1a90f1ec9 msdkvpp: Disable passthrough if memory capsfeature changes
So far msdk produced dmabuf fds are non-mappable.
If user wants to download the content of underlined surfaces,
dmabufcapsfeature negotiated pipeline will fail. So if the input surface
is dmabuf and downstream doesn't have support for dmabuf capsfeatures,
we do the vpp (no passthrough) and produce the mappable videomemory
buffers.

https://bugzilla.gnome.org/show_bug.cgi?id=794946
2018-04-30 12:40:32 -08:00
Sreerenj Balachandran
ef6e186801 msdk: vpp: Add dmabuf-export support
Currenly, the dmabuf buffer pool can be negotiated
only through DMABuf capsfeatures.
This will not allow to negotiate dmabuf support with
v4l2src (v4l2src ! msdkvpp) where v4l2src always export
the dmabuf based memory with out using the DMABuf capsfeatures.
So it requires fix based on:
https://bugzilla.gnome.org/show_bug.cgi?id=794817

https://bugzilla.gnome.org/show_bug.cgi?id=794946
2018-04-30 12:39:52 -08:00
Sreerenj Balachandran
76bbefe3b0 msdk: vpp: Add YV12, YUY2 and BGRx formats to template 2018-04-25 12:33:16 -08:00
Sreerenj Balachandran
96c6a04d7a msdk: Add more video format mapping
BGRx format can be supported with Msdk's RGB4
2018-04-25 12:33:08 -08:00
Sreerenj Balachandran
5184f85d77 msdk: vpp: Allocation query fixes
prpose_allocation:
-- always instantiate a pool for for upstream
-- use async_depth + 1 as min buffer count

decide_allocation:
-- always create a new bufferpool for source pad.
Each of the msdk element has to create it's own mfxsurfacepool
which is an msdk contraint. For eg: Each Msdk component (vpp, dec and
enc)
will invoke the external Frame allocator for video-memory usage
So sharing the pool between gst-msdk elements might not be a good idea.

https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-25 12:33:00 -08:00
Xavier Claessens
83d0623293 Meson: Generate pc file for all plugins in bad
https://bugzilla.gnome.org/show_bug.cgi?id=794568
2018-04-25 11:08:09 +01:00
Sreerenj Balachandran
142ad9dbad msdk: jpegdec: Fix non-interleaved sample decode
Using the default value (InterleavedDec == MFX_SCANTYPE_UNKNOWN)
causing issues with non-interleaved sample decode. Ideally the usage
of MFXVideoDECODE_DecodeHeader should fix these type of issue, but
it seems to be not. But hardcoding the InterleaveDec to
MFX_SCANTYPE_NONINTERLEAVED
is fixing the problem and fortunately msdk seems to be taking care of
Interleaved samples
too .So let's hardcode it for now.

https://bugzilla.gnome.org/show_bug.cgi?id=793787
2018-04-16 14:37:21 -08:00
U. Artie Eoff
275d754156 msdk: fix plugin load on implementations with only HW support
We can't assume that MSDK always supports SW implementation
on all platforms.  Thus, msdk_is_available should check for
ANY implementation.

https://bugzilla.gnome.org/show_bug.cgi?id=794991
2018-04-04 17:31:14 -08:00
Tim-Philipp Müller
f7352ecc5c msdk: fix meson syntax 2018-04-03 19:22:01 +01:00
Sreerenj Balachandran
e4b4f09496 msdk: vpp : Add frame rate control
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 11:10:20 -08:00
Sreerenj Balachandran
c0ea4bdafb msdk: vpp : Add force-aspect-ratio property
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:45 -08:00
Sreerenj Balachandran
fb8c536393 msdk: Add more scaling filter algorithms
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:35 -08:00
Sreerenj Balachandran
51b6345dc4 msdk: vpp: Add support for horizontal and vertical mirroring
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:24 -08:00
Sreerenj Balachandran
108c8fde7f msdk: vpp: Add detail/edge enhancement tuning
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:13 -08:00
Sreerenj Balachandran
93c5dd2478 msdk: vpp: Add ProAmp(colorbalance) support
Added Hue, Saturation, Brightness and Contrast tuning support.

Fixme: Add GstColorBalanceInterface support

https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:02 -08:00
Sreerenj Balachandran
f5a3d3d799 msdk: vpp: Add deinterlacing support
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:38:52 -08:00
Sreerenj Balachandran
36e81744d1 msdk: vpp:Add more filters
-- Add Denoise
-- Add Rotation

https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:38:41 -08:00
Sreerenj Balachandran
d3d89f02b3 msdk: Add VPP element
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:38:19 -08:00
Hyunjun Ko
35b6411d4d msdk: dec: rename the function to what it means more exactly.
https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-04-02 15:55:55 -08:00
Hyunjun Ko
90491d889a msdk: allocator: libva: check if it's already using dmabuf when mapping
As long as we negotiate the "DMABuf" capsfeatures for now, map can't be
working. So we need to confirm not to do it if using DMABuf memory.

https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-04-02 15:49:32 -08:00
Hyunjun Ko
cdc591dbc0 msdkdec: use dmabuf if possible
https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:06:40 -08:00
Hyunjun Ko
ea92da6e54 msdk: dmabuf support
This patch includes:
1\ Implements MsdkDmaBufAllocator and allocation of msdk dmabuf memroy.
2\ Each msdk dmabuf memory include its own msdk surface kept by GQuark.
3\ Adds new option GST_BUFFER_POOL_OPTION_MSDK_USE_DMABUF

https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:06:05 -08:00
Hyunjun Ko
5df06545ef msdk: adds new function to get dmabuf information from surface.
https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:05:16 -08:00
Hyunjun Ko
bd8ffcb29d msdk: allocator: get dmabuf handle during allocation if required
https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:04:28 -08:00
Hyunjun Ko
c3438b5a0f msdk: generalize the parameter of msdk video memory functions
There needs to be generalized for the parameter from
GstVideoMsdkVideoMemory to GstMemory.

Thus we can call these functions if using DMABuf memory.

https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:03:00 -08:00
Hyunjun Ko
762eb97092 msdk: specify the way to find a proper cached response by request
The current way to find proper response by just comparing request's
value is wrong.  We need to compare the size of a frame and the
number of suggested frames.

Refer to the sample in https://github.com/Intel-Media-SDK/samples.

https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:02:26 -08:00
Sreerenj Balachandran
e40dc51fcd msdk: dec: remove framerate field from sink caps template
Removes unessential field framerate for decoder so that negotiation
works even if framerate is not provided from upstream.

https://bugzilla.gnome.org/show_bug.cgi?id=789752
2018-03-29 13:06:41 -08:00
Hyunjun Ko
1955ffed3f msdk: dec: set framerate to the driver only if provided
For example, if framerate 0/1 is provided from upstream, the driver
fails to configure and complain about it.

We can let it go and make the driver assuming framerate itself.

https://bugzilla.gnome.org/show_bug.cgi?id=789752
2018-03-29 12:41:48 -08:00
Hyunjun Ko
0eaf3bdcd9 msdk: h265dec: remove framerate field from sink caps template
Removes unessential field framerate for decoder so that negotiation
works even if framerate is not provided from upstream.

https://bugzilla.gnome.org/show_bug.cgi?id=789752
2018-03-29 12:40:34 -08:00
Sreerenj Balachandran
f1f148fe7c msdk: Don't set extended coding options for JPEG encode
MJPEG doesn't have support for extended coding options

https://bugzilla.gnome.org/show_bug.cgi?id=793873
2018-03-29 11:56:19 -08:00
Hyunjun Ko
52f669bf43 msdk: libva: remove unnecessary code and comments
https://bugzilla.gnome.org/show_bug.cgi?id=794276
2018-03-13 14:21:40 -08:00
Hyunjun Ko
6547b638c5 msdk: adds new debug category
https://bugzilla.gnome.org/show_bug.cgi?id=794276
2018-03-13 14:20:50 -08:00
Hyunjun Ko
443b3c98fa msdk: fix typo
https://bugzilla.gnome.org/show_bug.cgi?id=794276
2018-03-13 14:18:23 -08:00
Wang,Fei
0c69867d52 msdk: Fix the I420 video format support
Make sure I420 surface mapping works as expected by using
YV12 format and swap U/V plane's offset and pitches.

https://bugzilla.gnome.org/show_bug.cgi?id=793865
2018-03-13 13:54:17 -08:00
Tim-Philipp Müller
c2bcc2711a meson: fix build when msdk is not found 2018-03-09 23:59:16 +00:00
Sreerenj Balachandran
bdbc1d0bef msdk: Fix the misspelled file name in meson build 2018-03-09 10:33:22 -09:00
Hyunjun Ko
4918430858 msdk: enc: fix missing some frames to be encoded
There was not handling the end of encoding sequence in encoder.
This patch does drain any remaining internal streams while decoder
already does this.

Document says:
"To mark the end of the encoding sequence, call this function with a
NULL surface
pointer. Repeat the call to drain any remaining internally cached
bitstreams—one
frame at a time—until MFX_ERR_MORE_DATA is returned."

https://bugzilla.gnome.org/show_bug.cgi?id=793236
2018-03-08 11:39:25 -09:00
Hyunjun Ko
8a3630ffd7 msdk: dec: fix leaks when flushing
https://bugzilla.gnome.org/show_bug.cgi?id=793708
2018-03-08 11:38:52 -09:00
Hyunjun Ko
c9faf0d612 msdk: manage child sessions on parent GstMsdkContext
Sometimes parent context is released before its children get released.
In this case MFXClose of parent session fails.

To make sure that child sessions are closed before closing a parent
session,
Parent context needs to manage child sessions and close them first when
it's released.

https://bugzilla.gnome.org/show_bug.cgi?id=793412
2018-03-08 11:38:30 -09:00
Hyunjun Ko
37ef61586a msdk: dec: remove code to manage buffers with locked surface
https://bugzilla.gnome.org/show_bug.cgi?id=793413
2018-03-08 11:37:52 -09:00
Hyunjun Ko
b08b8ddae3 msdk: manage MSDK surfaces seperately
Currently a gst buffer has one mfxFrameSurface when it's allocated and
can't be changed.
This is based on that the life of gst buffer and mfxFrameSurface would
be same.
But it's not true. Sometimes even if a gst buffer of a frame is finished
on downstream,
mfxFramesurface coupled with the gst buffer is still locked, which means
it's still being used in the driver.

So this patch does this.
Every time a gst buffer is acquired from the pool, it confirms if the
surface coupled with the buffer is unlocked.
If not, replace it with new unlocked one.
In this way, user(decoder or encoder) doesn't need to manage gst buffers
including locked surface.

To do that, this patch includes the following:
1. GstMsdkContext
- Manages MSDK surfaces available, used, locked respectively as the
following:
  1\ surfaces_avail : surfaces which are free and unused anywhere
  2\ surfaces_used : surfaces coupled with a gst buffer and being used
now.
  3\ surfaces_locked : surfaces still locked even after the gst buffer
is released.

- Provide an api to get MSDK surface available.
- Provide an api to release MSDK surface.

2. GstMsdkVideoMemory
- Gets a surface available when it's allocated.
- Provide an api to get an available surface with new unlocked one.
- Provide an api to release surface in the msdk video memory.

3. GstMsdkBufferPool
- In acquire_buffer, every time a gst buffer is acquired, get new
available surface from the list.
- In release_buffer, it confirms if the buffer's surface is unlocked or
not.
  - If unlocked, it is put to the available list.
  - If still locked, it is put to the locked list.

This also fixes bug #793525.

https://bugzilla.gnome.org/show_bug.cgi?id=793413
https://bugzilla.gnome.org/show_bug.cgi?id=793525
2018-03-08 11:37:12 -09:00
Hyunjun Ko
14f0186741 msdk: remove unused code
There's unused code remaining since MSDK bufferpool patches landed.

https://bugzilla.gnome.org/show_bug.cgi?id=793741
2018-02-23 14:30:56 -09:00
Sreerenj Balachandran
1c81bf4bdc msdkenc: remove unnecessary memset
https://bugzilla.gnome.org/show_bug.cgi?id=791479
2018-02-22 12:32:45 -09:00
Sreerenj Balachandran
3cc61f98b1 msdk: enc: Support force-key-unit events
https://bugzilla.gnome.org/show_bug.cgi?id=791479
2018-02-22 12:32:20 -09:00
Sreerenj Balachandran
a50ecfe46c msdk: enc: Fix typo 2018-02-20 17:22:35 -09:00
Sreerenj Balachandran
e924dec4e1 msdk: h264_enc: Enable B-pyramid prediction support
Since there is already an "adaptive-B" option, just
use boolean property for B-pyramid enabling.

Fixme: Not sure whether this can be supported in vp8 and vp9.
It could be possible through GPB (b without backward ref) but
can't verify currently. We can move this as common property
once verified with vp8 and vp9 without breaking any backward
compatibility.

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:41:18 -09:00
Sreerenj Balachandran
07c05a75a5 msdk: Add more tuning options
Added tuning options for mb level bitrate control,
adaptive I-frame insertion, and adaptive B-frame insertion.

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:41:08 -09:00
Sreerenj Balachandran
f25bcf7cb8 msdk: h264_enc: Add slice size tuning option
According to spec, it is a general property. But based on
testing it only works for h264 encoder.
Let's keep it as h264 specific for now.

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:40:59 -09:00
Sreerenj Balachandran
ddd02be0de msdk: move enum definitions to separte file
Move enum value defintions which are (or in future) supported
by more than one codec into a common file.

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:40:50 -09:00
Sreerenj Balachandran
b7dbcb26b8 msdk: encoder: h264: Enable trellis quantization tuning
Add a new property "trellis" to enable trellis quantization.
Keeping trellis as a flag value (which is boolean for gst x264 enc element)
since it is possible to enable/disable this seperately for
I,P and B frames through MediaSDK ext option headers.

The subclass implementations always need to inform base-encoder
if it requires the inclusion of Extend Header buffers (mfxExtCodingOption2
 and mfxExtCodingOption3).

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:40:42 -09:00
Sreerenj Balachandran
d58c0bd509 msdk: h264_enc: Add LookaheadDownsampling support
This option controls down sampling in look ahead bitrate
control mode. According to spec it is only supported in AVC.

Fixme: Probably HEVC also have support for this in recent
MSDK versions. We could move the enumeration types to common
header usable for multiple codecs.

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:39:53 -09:00
Sreerenj Balachandran
ae8d956c2c msdk: encode: Add more rate control options
MediaSDK has support for a number of rate control algorithms.
Adding all possible options to the property rate-control.

Fixme1: In case of failure, currently we don't have a proper method
to show which rate-control has been failed. It could be better
to add some extensive validation on EncQuery output in case of error.
Unfortunately, not all ratecontrol methods are supported by every codecs
and we don't have the dynamic detection of supported ratecontrol methods yet.

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:39:28 -09:00
Sreerenj Balachandran
f076f1948e msdk: encode: Add property to set slice/partitioning
Adding a new property num-slices to set the number of
slices/partitions per frame. Adding it as a general
property for all codecs (except jpeg).

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:39:13 -09:00
Sreerenj Balachandran
a165a1a1a9 msdk: encoder: h265: generalize the behavior of "i-frames" property
We have the property "i-frames" to set the IDR interval in a
gop. Unfortunately MSDK HEVC encoder behaves bit differently
for IdrInterval field, IdrInteval == 1 indicate every
I-frame should be an IDR (which is IdrInterval == 0 for other codecs),
IdrInteval == 2 means every other I-frame is an IDR
(which is IdrInterval == 1 for other codecs) etc.
So we generalize the behaviour of property "i-frames" by
incrementing the value by one in each case (only for HEVC).

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:39:02 -09:00
Sreerenj Balachandran
fa0911c3f6 msdk: encoder: register only the required properties
The base encoder common properties are not valid for
mjpeg encoder where there is no motion compensation or rate control.
Delaying the property installation on the base gobject
untill the subclass class_init get invoked.

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:38:28 -09:00
Víctor Manuel Jáquez Leal
0156d391ae msdk: add missing files for dist target
https://bugzilla.gnome.org/show_bug.cgi?id=793563
2018-02-19 22:16:07 +01:00
Sreerenj Balachandran
173b07ba11 msdk: vc1_dec: Add Advanced profile (WVC1) support
Only supporting asf header-format having BDUs with startcode.
It might be possible to support other formats too, but haven't tested.

https://bugzilla.gnome.org/show_bug.cgi?id=792589
2018-02-13 14:41:52 -09:00
Sreerenj Balachandran
55c0d7205d msdk: dec: Add non-packetized stream handling support
The gst-msdk decoders prefer packetized streams as input
and in this case we can avoid unnecessary input bitstream copy
to mfxBitstream. This works fine for codecs like h264 where
we only support byte-stream with au alignment. Other format
conversions should be done thorugh parsers. But this won't work
for codecs like vc1 where we don't have an autoplugged parser.
Even the parser is not capable to do format conversions.

Packetizing through base decoders parse() routine will bring a
lot of uncecessary of complexities and codecparser libraray dependency.
So we just use an interal gst_adaper to keep track of bitstream
which is not consumed by msdk durig AsynchronusDecoding.
This adapter will get used only if subclass implementations
set the "is_packetized" to FALSE for msdk base encoder.

https://bugzilla.gnome.org/show_bug.cgi?id=792589
2018-02-13 14:41:20 -09:00
Sreerenj Balachandran
760e6e54a7 msdk: Add VC1 decoder (simple and main profiles)
Adding Simple and Main profiles decode support.

Currently msdkvc1dec is not capable to handle the codec_data,
only instream headers are supported. Also msdk vc1 decoder
expecting instream with Sequence header as per SMPTE 421M Annex L.

Most of the decdoebin/playbin pipeline won't work with the above
constraints
because vc1parse is still not an autoplug element.

Only way to make mskdvc1dec work is by connecting a vc1parse
as an upstream element.

https://bugzilla.gnome.org/show_bug.cgi?id=792589
2018-02-13 14:40:54 -09:00
Sreerenj Balachandran
b622d21d6a msdk : Add RenderNode support
Use drm render node as the first choice of device node file.
Fall backs to use drm primary (/dev/dri/card[0-9])
if there is no render node available

Basic logic is inherited from gstreamer-vaapi, but using
gudev API rather than libudev directly.

Added gudev library as dependency for msdk.

https://bugzilla.gnome.org/show_bug.cgi?id=791599
2018-02-13 14:40:22 -09:00
Hyunjun Ko
76a82feae7 msdk: Avoid build failures on Windows until d3d allocator is implemented
https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:54:03 -09:00
Hyunjun Ko
72c6cd5545 msdkdec: use video memory if there's another MSDK context in a pipeline
1\ If downstream's pool is MSDK bufferpool,
2\ If there's shared GstMsdkContext in the pipeline,

a decoder decides to use video memory.

This policy should be improved to handle more cases.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:53:02 -09:00
Hyunjun Ko
375a50a876 msdk: add async depth from each msdk element to GstMsdkContext to be shared
In case that pipeline is like ".. ! decoder ! encoder ! ..." with using
video memory,
decoder needs to know the async depth of the following msdk element so
that it could
allocate the correct number of video memory.

Otherwise, decoder's memory is exhausted while processing.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:52:14 -09:00
Hyunjun Ko
f2b35abcab msdkdec/enc: query GstContext to share GstMsdkContext
How to share/create GstMsdkcontext is the following:

- Search GstMsdkContext if there's in the pipeline.
  - If found, check if it's decoder, encoder or vpp by job type.
    - If it's same job type, it creates another instance of
GstMsdkContext
     with joined-session.
    - Otherwise just use the shared GstMsdkContext.
  - If not found, just creates new instance of GstMsdkContext.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:51:18 -09:00
Hyunjun Ko
dc82ccb9a2 msdk: context: add job type to figure out if joining session is necessary
According to the driver's instruction, if there are two or more encoders
or decoders in a process, the session should be joined by
MFXJoinSession.

To achieve this successfully by GstContext, this patch adds job type
specified if it's encoder, decoder or vpp.

If a msdk element gets to know if joining session is needed by the
shared context,
it should create another instance of GstContext with joined session,
which
is not shared.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:50:48 -09:00
Hyunjun Ko
dddb84897f msdk: adds util functions to handle GstContext
To share GstMsdkContext with each msdk element,
it will be using GstContext.

Most common code is from gstreamer-vaapi.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:50:08 -09:00
Hyunjun Ko
a66d5620f3 msdkdec: use bufferpool
1\ In decide_allocation, it makes its own msdk bufferpool.
  - If downstream supports video meta, it just replace it with the msdk
bufferpool.
  - If not, it uses the msdk bufferpool as a side pool, which will be
decoded into.
    and will copy it to downstream's bufferpool.

2\ Decide if using video memory or system memory.
  - This is not completed in this patch.
  - It might be decided in update_src_caps.
  - But tested for both system memory and video memory cases.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:49:28 -09:00
Hyunjun Ko
580a52ec49 msdkenc: use bufferpool
1\ Proposes msdk bufferpool to upstream.
  - If upstream has accepted the proposed msdk bufferpool,
    encoder can get msdk surface from the buffer directly.
  - If not, encoder get msdk surface its own msdk bufferpool
    and copy from upstream's frame to the surface.

2\ Replace arrays of surfaces with msdk bufferpool.

3\ In case of using VPP, there should be another msdk bufferpool
   with NV12 info so that it could convert first and encode.

Calls gst_msdk_set_frame_allocator and uses video memory only on linux.
and uses system memory on Windows until d3d allocator is implemented.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:48:32 -09:00
Hyunjun Ko
2542b2d34d msdk: supports bufferpool
Implements 2 memory allocators:
1\ GstMsdkSystemAllocator: This will allocate system memory.
2\ GstMsdkVideoAllocator: This will allocate device memory depending
on the platform. (eg. VASurface)

Currently GstMsdkBufferPool uses video allocator currently by default
only on linux. On Windows, we should use system memory until d3d
allocator
is implemented.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:44:08 -09:00
Hyunjun Ko
4d860ec82b msdk: adds frame allocator using libva
Implements msdk frame allocator which is required from the driver.
Also makes these functions global so that GstMsdkAllocator could use
the allocated video memory later and couple with GstMsdkMemory.

GstMsdkContext keeps allocation information such as mfxFrameAllocRequest
and mfxFrameAllocResponse after allocation.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 12:43:42 -09:00
Hyunjun Ko
3ed9f7c5f1 msdkdec: fix typo
https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 12:43:00 -09:00
Hyunjun Ko
6ce9a66b80 msdk: implements GstMsdkContext.
Makes GstMsdkContext to be a descendant of GstObject so that
we could track the life-cycle of the session of the driver.

Also replaces MsdkContext with this one.
Keeps msdk_d3d.c alive for the future.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 12:41:28 -09:00
Hyunjun Ko
3f9d0fcaa9 msdk: libva: adds utility function between mfx and libva
https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 12:39:44 -09:00
Hyunjun Ko
b45147a6d4 msdk: adds new utility functions for conversion from gstreamer to libmfx
https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 12:37:47 -09:00
Hyunjun Ko
8f0450dad4 msdk: move and rename the function msdk_video_alignment
Move the msdk_video_alignment function from decoder
to msdk.c and rename so that others could call this function
without duplicated declaration.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 12:36:46 -09:00
Víctor Manuel Jáquez Leal
568c552ac0 msdk: change plugin's description
There are not only encoders nowadays, there are decoders too and a
postprocessor in the future.
2018-01-11 11:29:38 +01:00
Sreerenj Balachandran
b6c26919df msdk: init the debug category for mpeg2dec
https://bugzilla.gnome.org/show_bug.cgi?id=792260
2018-01-10 18:24:07 -09:00
Sreerenj Balachandran
7536d12095 msdk: Add mpeg2 decoder
https://bugzilla.gnome.org/show_bug.cgi?id=792260
2018-01-10 10:40:24 -09:00
Hyunjun Ko
3301dd34b5 msdkdec: keep draining even if a finish_task fails
Should continue draining so that it could try to
discard the rest of pending frames even if a finish_task fails.

https://bugzilla.gnome.org/show_bug.cgi?id=790312
2017-11-23 10:08:13 +02:00
Hyunjun Ko
0933b8b45a msdkdec: fix buffer leaks during drain and a leak of videobufferpool
https://bugzilla.gnome.org/show_bug.cgi?id=790312
2017-11-22 17:30:07 +02:00
Hyunjun Ko
ddd9355767 msdkenc: handle the MORE_DATA case
If the driver requires more data, just unref the frame at the moment
then retreive/finish the frame after encoding is finished.

This also fixes a memory leak.

https://bugzilla.gnome.org/show_bug.cgi?id=790312
2017-11-22 17:30:07 +02:00
Sebastian Dröge
538aae2404 msdkh264enc: Implement frame-packing SEI insertion for side-by-side and top-bottom
Ideally this would be supported directly in the Intel MediaSDK but
that's not the case nowadays.

https://bugzilla.gnome.org/show_bug.cgi?id=774920
2017-11-20 14:39:03 +02:00
Hyunjun Ko
3c611da315 msdkenc: Fix handling of YUY2, UYVY, BGRA surfaces
https://bugzilla.gnome.org/show_bug.cgi?id=789847
2017-11-20 14:38:34 +02:00
Sebastian Dröge
d3eeb98f0c msdkenc: Add support for YV12, YUY2, UYVY and BGRA
By doing conversion with VPP to NV12 before the actual encoding.

https://bugzilla.gnome.org/show_bug.cgi?id=789847
2017-11-20 14:38:25 +02:00
Hyunjun Ko
66d2bce8ff msdk: fix to build msdkvp8dec with mfx_dispatch
https://bugzilla.gnome.org/show_bug.cgi?id=789751
2017-11-04 11:14:50 +02:00
Hyunjun Ko
cd2dca6275 msdk: adds msdkvp8dec
https://bugzilla.gnome.org/show_bug.cgi?id=789751
2017-11-01 11:33:50 +02:00
Sebastian Dröge
62f04e801b msdk: Allow building against mfx_dispatcher
E.g. from https://github.com/lu-zero/mfx_dispatch
2017-10-30 18:20:23 +02:00
Tim-Philipp Müller
c29843dc94 meson: msdk: use python3 from python3 module 2017-07-07 12:28:14 +01:00
Scott D Phillips
dcd9032d95 msdk: enc: set pts and dts, fix inverted sync_point flag
Set the pts and dts on the frame that we receive from the msdk.
Also fix the inverted logic in setting sync points, previously we
were marking all frames as sync points except IDRs.

https://bugzilla.gnome.org/show_bug.cgi?id=782801
2017-06-07 19:23:28 +03:00
Tim-Philipp Müller
80d66dfec8 Fix up package name and origin in some plugins 2017-05-18 10:58:20 +01:00
Nicolas Dufresne
4261692187 Remove plugin specific static build option
Static and dynamic plugins now have the same interface. The standard
--enable-static/--enable-shared toggle are sufficient.
2017-05-16 14:05:52 -04:00
Scott D Phillips
b2663e12e4 msdk: Declare libmfx.a as a C++ static library
This lets meson know that the overall plugin needs built with C++
linking.

https://bugzilla.gnome.org/show_bug.cgi?id=781561
2017-05-11 07:17:09 +01:00
Scott D Phillips
732a157cb7 msdk: Propagate GstFlowReturn values
In some places a GST_FLOW_FLUSHING result was return as a FALSE
gboolean and then returned from a parent function as
GST_FLOW_ERROR. This prevented seeking from working.

https://bugzilla.gnome.org/show_bug.cgi?id=776360
2017-01-20 11:11:50 -08:00
Sebastian Dröge
5c99f9cf37 gst: Don't declare variables inside the for loop header
This is a C99 feature.
2016-12-13 22:39:01 +02:00
Scott D Phillips
fbf7e905f0 msdk: add mjpeg decoder
https://bugzilla.gnome.org/show_bug.cgi?id=774793
2016-12-13 00:06:42 +01:00
Scott D Phillips
1a4db8846f msdk: add mjpeg encoder
https://bugzilla.gnome.org/show_bug.cgi?id=774793
2016-12-13 00:06:42 +01:00
Scott D Phillips
1f6903bf3e msdk: Add H.265 decoder
https://bugzilla.gnome.org/show_bug.cgi?id=775726
2016-12-12 23:48:46 +01:00
Scott D Phillips
83774c3eb9 msdk: Add H.264 decoder
The decoder only supports system memory output presently.

https://bugzilla.gnome.org/show_bug.cgi?id=774587
2016-12-12 23:16:11 +01:00
Scott D Phillips
71df82db63 msdk: Initial windows build support
https://bugzilla.gnome.org/show_bug.cgi?id=770990
2016-11-14 15:17:41 +01:00
Scott D Phillips
8949c12aaf meson: add build files for msdk
https://bugzilla.gnome.org/show_bug.cgi?id=770990
2016-11-14 15:17:30 +01:00
Scott D Phillips
35f76fc33a msdk: load plugins in h265 and vp8 init
https://bugzilla.gnome.org/show_bug.cgi?id=770990
2016-11-14 15:17:19 +01:00
Scott D Phillips
2905a7e33e msdk: Width and Height need to be integer multiples of 32
https://bugzilla.gnome.org/show_bug.cgi?id=770990
2016-11-14 15:17:09 +01:00
Scott D Phillips
22551a42cc msdk: add to build
https://bugzilla.gnome.org/show_bug.cgi?id=770990
2016-11-14 15:17:01 +01:00
Josep Torra
7d68d40239 msdk: Create plugin for Intel's Media SDK
[scott.d.phillips@intel.com: gst-indent *.c *.h]
[scott.d.phillips@intel.com: link with libva at build time]
[scott.d.phillips@intel.com: remove unused includes]
[scott.d.phillips@intel.com: Update README]

https://bugzilla.gnome.org/show_bug.cgi?id=770990
2016-11-14 15:16:45 +01:00