Commit graph

444 commits

Author SHA1 Message Date
Haihao Xiang
55f3590317 msdkenc: support tune property in msdkh264enc and msdkh265enc
Introduce a new property for encoding mode selection, the default value
for this new property allows the SDK to decide what to do. In addition,
low-power is marked as deprecated since this fix

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1426>
2020-07-11 13:57:52 +00:00
Haihao Xiang
3ecb2a82be Revert "msdkh265enc: let MSDK select the encoding mode by default"
This reverts commit 9e977832c1.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1426>
2020-07-11 13:57:52 +00:00
Haihao Xiang
9e977832c1 msdkh265enc: let MSDK select the encoding mode by default
MSDK may support lowpower and non-lowpower modes, some features are
available only under one of the two modes, which is hard to know for
user, so let MSDK select the mode by default.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1405>
2020-07-06 14:43:31 +00:00
Xu Guangxin
610e477565 msdkdec: hold a reference for the surfaces locked by msdk
previous code releases GstBuffer too earlier. so we will see

ERROR                default gstmsdkvideomemory.c:77:gst_msdk_video_allocator_get_surface: failed to get surface available
ERROR         msdkbufferpool gstmsdkbufferpool.c:270:gst_msdk_buffer_pool_alloc_buffer:<msdkbufferpool0> failed to create new MSDK memory

We need to hold GstBuffer reference for msdk if the surfaced locked by msdk.

step to reproduce.
1. ffmpeg -f lavfi -i testsrc=duration=10:size=320x240:rate=30 -pix_fmt yuv420p -c:v libx265 test.265
2. GST_GL_PLATFORM=egl  gst-launch-1.0 -v filesrc location=test.265  ! h265parse ! msdkh265dec  ! queue ! glimagesink

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1277>
2020-06-15 02:46:53 +00:00
Haihao Xiang
f685893ed2 msdkh265enc: add support 12-bit 420 encoding
P016 is used for 12-bit encoding in MediaSDK, so the Shift flag is set
in the mfx parameters

Sample pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=P012_LE ! msdkh265enc ! \
fakesink

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1174>
2020-06-01 05:04:30 +00:00
Xu Guangxin
52be289847 msdkvpp: fix "failed to create new MSDK memory"
all msdk output surfaces come from out_alloc_resp, so the buffer count is not resizable.
we need set min_buffers, max_buffers to same size.

steps to reproduce
1. ffmpeg -f lavfi -i testsrc=duration=10:size=320x240:rate=30:decimals=3 -pix_fmt yuv420p -c:v libx265 ~/bits/hevc/test.265
2. GST_GL_PLATFORM=egl gst-launch-1.0 -v filesrc location=~/bits/hevc/test.265  ! h265parse ! msdkh265dec  ! msdkvpp ! queue ! glimagesink
you will see error like this:
ERROR                default gstmsdkvideomemory.c:77:gst_msdk_video_allocator_get_surface: failed to get surface available
ERROR         msdkbufferpool gstmsdkbufferpool.c:270:gst_msdk_buffer_pool_alloc_buffer:<msdkbufferpool2> failed to create new MSDK memory
ERROR                msdkvpp gstmsdkvpp.c:297:create_output_buffer:<msdkvpp0> failed to create output video buffer
ERROR                msdkdec gstmsdkdec.c:699:gst_msdkdec_finish_task:<msdkh265dec0> Failed to finish frame
ERROR                msdkdec gstmsdkdec.c:1085:gst_msdkdec_handle_frame:<msdkh265dec0> Failed to finish a task

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1278>
2020-06-01 02:09:04 +00:00
Xu Guangxin
275518f661 msdkvpp: hold GstBuffer ref count for locked surfaces
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1278>
2020-06-01 02:09:04 +00:00
Haihao Xiang
98600942e4 msdkvpp: clear the parameters after closing the session
Otherwise the stale values are used for the new process.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1159>
2020-05-11 02:16:44 +00:00
Xu Guangxin
cab1ddf48b msdk: bufferpool: set alignment to video meta
else gst_video_meta_validate_alignment will report error like
"videometa gstvideometa.c:416:gst_video_meta_validate_alignment: Stride of plane 0 defined in meta (384) is different from the one computed from the alignment (320)"

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1224>
2020-05-07 05:45:36 +00:00
Jordan Petridis
87fc038f67
gstmsdkdec: fix logical operation that misses parenthesis
in C, & is weaker than the ! operator and clang is giving the following
error about it.

```
../subprojects/gst-plugins-bad/sys/msdk/gstmsdkdec.c:731:7: error: logical not is only applied to the left hand side of this bitwise operator [-Werror,-Wlogical-not-parentheses]
  if (!gst_msdk_context_get_job_type (thiz->context) & GST_MSDK_JOB_DECODER) {
      ^                                              ~
../subprojects/gst-plugins-bad/sys/msdk/gstmsdkdec.c:731:7: note: add parentheses after the '!' to evaluate the bitwise operator first
  if (!gst_msdk_context_get_job_type (thiz->context) & GST_MSDK_JOB_DECODER) {
      ^
       (                                                                   )
../subprojects/gst-plugins-bad/sys/msdk/gstmsdkdec.c:731:7: note: add parentheses around left hand side expression to silence this warning
  if (!gst_msdk_context_get_job_type (thiz->context) & GST_MSDK_JOB_DECODER) {
      ^
      (                                             )
1 error generated.
```
2020-04-10 18:22:21 +03:00
Haihao Xiang
00baed3ebe msdkvp9dec: add support for VP9 12bit
The output formats are P012_LE for 12bit 420 and Y412_LE for 12bit 444.
2020-04-07 00:39:35 +00:00
Haihao Xiang
4cc5321010 msdkh265dec: add support for main-444-12, main-444-12-intra profiles
The video format is Y412_LE
2020-04-03 01:19:45 +00:00
Haihao Xiang
4d756c07b0 msdk: map Y412_LE to VA_FOURCC_Y416
In media driver, VA_FOURCC_Y416 is used for packed 12 bits 4:4:4:4 YUV
format, the corresponding RT format is VA_RT_FORMAT_YUV442_12
2020-04-03 01:19:45 +00:00
Haihao Xiang
55ed9458b5 msdk: map MFX_FOURCC_Y416 to VA_FOURCC_Y416
Y416 is used for packed 12 bits 4:4:4:4 YUV format in media driver, the
RT format is VA_RT_FORMAT_YUV444_12
2020-04-03 01:19:45 +00:00
Haihao Xiang
e769df9689 msdk: map Y412_LE to MFX_FOURCC_Y416
MFX_FOURCC_Y416 is used for packed 12 bits 4:4:4:4 YUV format in
MediaSDK
2020-04-03 01:19:45 +00:00
Haihao Xiang
3de690ceda msdkh265dec: add support for main-422-12, main-422-12-intra profiles
The video format is Y212_LE
2020-04-03 01:19:45 +00:00
Haihao Xiang
01edef09a3 msdk: map Y212_LE to VA_FOURCC_Y216
In media driver, VA_FOURCC_Y216 is used for packed 12 bits 4:2:2 YUV
format, the corresponding RT format is VA_RT_FORMAT_YUV422_12
2020-04-03 01:19:45 +00:00
Haihao Xiang
34177985e2 msdk: map MFX_FOURCC_Y216 to VA_FOURCC_Y216
In media driver, Y216 is used for packed 12 bits 4:2:2 format YUV
format, so the RT format is VA_RT_FORMAT_YUV422_12.
2020-04-03 01:19:45 +00:00
Haihao Xiang
fb200ccfdd msdk: map Y212_LE to MFX_FOURCC_Y216
MFX_FOURCC_Y216 is used for packed 12 bits 422 YUV format in MediaSDK
2020-04-03 01:19:45 +00:00
Seungha Yang
950aa3d1db msdkh265enc: Fix for wrong parser free function 2020-04-03 00:42:23 +09:00
Seungha Yang
71cf93c361 msdkh264enc: Configure parser and SEI array only if it's required 2020-04-02 09:20:11 +00:00
Seungha Yang
206fe1534d msdkh265enc: Add support for CEA708 closed caption insertion
Functionally identical to that of msdkh264enc
2020-04-02 09:20:11 +00:00
Seungha Yang
882531aa54 msdkh264enc: Port to h264parser API for frame packing SEI injection
Create frame packing SEI memory only once per set_format() and
reuse it if possible.
2020-03-31 08:30:50 +00:00
Haihao Xiang
326b755332 msdkh265enc: support 8-bit 422 encoding
The media driver can support HEVC 8-bit 422 encoding for non-lowpower
mode since ICL[1], so VPP is not needed for this case.

Sample pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=YUY2 ! msdkh265enc ! \
filesink location=output.h265

[1] https://github.com/intel/media-driver#decodingencoding-features
2020-03-31 09:27:03 +08:00
Haihao Xiang
7c91c8c980 msdkh265enc: add support for Y210 in sink pad
Sample pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=Y210 !
msdkh265enc ! filesink location=output.h265
2020-03-31 09:27:03 +08:00
Haihao Xiang
fbd1d0cecb msdk: add a helper function to get codename of the platform
The features supported in MSDK vary from platform to platform. We may
support some features based on the codename of the platform in future.
2020-03-31 09:27:03 +08:00
Seungha Yang
afb042cbb9 msdkenc: Try to find corresponding codec frame for encoded output buffer
The input and output buffers should be matched as much as possible
so that various metadata and its ordering to be preserved.
2020-03-30 07:59:10 +00:00
Seungha Yang
c0787ed1cd msdkh264enc: Add support for CEA708 closed caption insertion
Currently supported caption format is CEA708_RAW
2020-03-30 07:59:10 +00:00
Haihao Xiang
d796707dd4 msdkh265enc: slice size control
Add max-slice-size property to set the maximum slice size
2020-03-30 00:20:11 +00:00
Haihao Xiang
49604d8fbe msdkh265dec: add support for main-12, main-12-intra profiles
The video format is P012_LE
2020-03-23 06:31:30 +00:00
Haihao Xiang
94fa88b6da msdk: map P012_LE to VA_FOURCC_P016
In media driver, VA_FOURCC_P016 is used for semi-planar 12 bits YUV
format, the corresponding RT format is VA_RT_FORMAT_YUV420_12
2020-03-23 06:31:29 +00:00
Haihao Xiang
c5763f66c9 msdk: map MFX_FOURCC_P016 to VA_FOURCC_P016
P016 is used for semi-planar 12 bits format in the media driver, so the
RT format is VA_RT_FORMAT_YUV420_12
2020-03-23 06:31:29 +00:00
Haihao Xiang
4d31299c12 msdk: map P012_LE to MFX_FOURCC_P016
MFX_FOURCC_P016 is used for semi planar 12 bits YUV format in MediaSDK
2020-03-23 06:31:29 +00:00
Haihao Xiang
4d70add5b6 msdkvp9enc: add support for 10 bits 444 input
The input format is Y410 in sink pad and the corresponding profile in
src pad is profile 3
2020-03-23 08:20:13 +08:00
Haihao Xiang
e7d4158547 msdkvp9enc: add support for 8 bits 444 input
The input format is VUYA in sink pad and the corresponding profile in
src pad is profile 1
2020-03-23 08:15:21 +08:00
Seungha Yang
2e5fee17a3 msdkbufferpool: Fix build warning on Windows
gstmsdkbufferpool.c(274): warning C4101: 'fd': unreferenced local variable
2020-03-22 22:30:15 +09:00
U. Artie Eoff
09d41b0b7d msdkvpp: add YV12 format to src pad
VPP YV12 output format support added since
MFX_VERSION 1032.

https://github.com/Intel-Media-SDK/MediaSDK/pull/2027
2020-03-19 09:31:59 -07:00
Haihao Xiang
c02f603262 msdk: meson: check whether mfxExtVP9Param is defined
User reported MFX_VERSION is 1026 however the build is broken[1], so add
the check for mfxExtVP9Param to make sure it can be build without
compiler error.

In addtion, it fixes a stupid typo (#endif') introduced by me.

[1] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/988#note_408093
2020-03-13 01:29:04 +00:00
rubenrua
6c3f092afc asfmux: Fix typo in property description
s/milisecs/milliseconds/g
2020-03-12 18:38:11 +01:00
Haihao Xiang
acc14225c9 msdk: libva: Don't set the hint if MFX_VERSION is lower than 1025
MFX_MEMTYPE_VIDEO_MEMORY_ENCODER_TARGET is defined since MFX_VERSION
1025.
2020-02-24 01:26:58 +00:00
Haihao Xiang
50ae5061f9 msdkvp9enc: output raw vp9 stream instead of IVF stream
video/x-vp9 is required in the src pad, however the output includes a
IVF header, which makes the pipeline below doesn't work

  gst-launch-1.0 videotestsrc ! msdkvp9enc ! msdkvp9dec ! fakesink

Since mfx 1.26, the VP9 encoder supports bitstream without IVF header,
so in this patch, the mfx version is checked and msdkvp9enc is enabled
only if mfx 1.26+ is available
2020-02-10 06:46:28 +00:00
Seungha Yang
5d160a8968 msdkdec: Fix GstMsdkContext leak 2020-01-29 15:35:21 +09:00
Seungha Yang
a8fff639c8 msdk: Clear reference counted object in dispose() method
Follow GObject's memory management model
2020-01-29 15:35:21 +09:00
Haihao Xiang
ac9c9d8efc msdk: use cached response for DMABuf when the frame size is same
User is seeing corrupted display when running `videotestsrc !
video/x-raw,format=NV12,width=xxx,height=xxx ! msdkh265enc ! msdkh265dec
! glimagesink` with changed frame size, e.g. from 1920x1080 to 1920x240

The root cause is a same dmabuf fd is used for frames with
different size, which causes some unexpected result. This patch requires
cached response is used for frames with same size only for DMABuf, so a
dmabuf fd can't be used for frames with different size any more.
2020-01-28 03:46:04 +00:00
Haihao Xiang
e3ebebc20b msdk: Fix compiler warning
This patch fixed compiler warning below:

[1/4] Compiling C object 'sys/msdk/dc44ea0@@gstmsdk@sha/gstmsdkvpp.c.o'.
../../gst-plugins-bad/sys/msdk/gstmsdkvpp.c: In function
‘gst_msdkvpp_context_prepare’:
../../gst-plugins-bad/sys/msdk/gstmsdkvpp.c:214:7: warning: suggest
parentheses around operand of ‘!’ or change ‘&’ to ‘&&’ or ‘!’ to ‘~’
[-Wparentheses]
2020-01-21 18:37:10 +00:00
Nirbheek Chauhan
1bcf44bb1f msdk: Fix increasing memory usage in dynamic pipelines
Our context is non-persistent, and we propagate it throughout the
pipeline. This means that if we try to reuse any gstmsdk element by
removing it from the pipeline and then re-adding it, we'll clone the
mfxSession and create a new gstmsdk context as a child of the old one
inside `gst_msdk_context_new_with_parent()`.

Normally this only allocates a few KB inside the driver, but on
Windows it seems to allocate tens of MBs which leads to linearly
increasing memory usage for each PLAYING->NULL->PLAYING state cycle
for the process. The contexts will only be freed when the pipeline
itself goes to `NULL`, which would defeat the purpose of dynamic
pipelines.

Essentially, we need to optimize the case in which the element is
removed from the pipeline and re-added and the same context is re-set
on it. To detect that case, we set the context on `old_context`, and
compare it to the new one when preparing the context. If they're the
same, we don't need to do anything.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/946
2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
6834a12112 msdk: Reorganize context preparation code
Split it out into a separate function with early exits to make the
flow clearer, and document what the function is doing clearly.
No functional changes.
2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
e83d5fd8d6 msdk: Fix warning about unused variable on Windows 2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
c0d778c2c7 msdk: Use gst_clear_object()
`gst_object_replace()` is not supposed to be used for unreffing and
NULLing objects.
2020-01-21 00:38:41 +00:00
Josep Torra
ef37446460 msdkdec: align frame list using decoded timestamp
Before this change decoder used the oldest frame in the list to pair it
with the decoded surface. This only works when there's a perfect stream
like HEADERS,SYNCPOINT,DELTA...

When playing RTSP streams we can get imperfect streams like HEADERS,
DELTA,SYNCPOINT,DELTA... In this case decoder drops the frames
between HEADERS and SYNCPOINT which leads into using wrong PTS on
the output frames.

With this change we inject the input PTS into the bitstream and use it
to align the internal frame list with the actually decoded position.

Fixes playback with:
```
gst-launch-1.0 rtspsrc location=... latency=0 drop-on-latency=1 ! ...
```
2020-01-19 04:26:15 +00:00
Haihao Xiang
52abbeeefb msdkdec: support transform_meta virtual method
Allow the base class to copy GstVideoRegionOfInterestMeta data to the
output buffer when calling gst_video_decoder_finish_frame
2020-01-15 00:47:12 +00:00
Haihao Xiang
8512624a41 msdkenc: set ROI region for msdk{h264, h265}enc
A reconfig is needed when ROI is changed, otherwise the ROI parameters won't
take effect
2020-01-15 00:47:12 +00:00
Haihao Xiang
84e234a8c7 msdkenc: add set_extra_params virtual method
set_extra_params is added to allow sub class to add extra mfx parameters
for changed input frame or meta data
2020-01-15 00:47:12 +00:00
Haihao Xiang
0e2fc39f24 msdkenc: add need_reconfig virtual method
need_reconfig is added to allow sub class requires a reconfig when
the input frame or the MetaData (e.g. GstVideoRegionOfInterestMeta)
attached to the input frame is changed.
2020-01-15 00:47:12 +00:00
Haihao Xiang
bdf9a76226 Revert "msdkdec: make sure to use video memory on Linux"
Commit a1584b6 caused big performance drop if the downstream element
is not a msdk element because it is very slow to read data from video
memory directly.

This reverts commit a1584b6f99.
2020-01-10 01:36:40 +00:00
Haihao Xiang
9064de27f9 msdkvp9enc: fix 10bit encoding
The codec profile should be consistent with the frame fourcc code, this
fixes pipeline below:

gst-launch-1.0 videotestsrc ! \
video/x-raw,width=320,height=240,format=P010_10LE ! msdkvp9enc ! \
fakesink
2020-01-06 03:54:57 +00:00
Haihao Xiang
da22ff3846 msdkvp9enc: fix width and height
The frame width and height is rounded up to 128 and 32 since commit
8daac1c, so the width, height for initialization should be rounded up to
128 and 32 too because the MSDK VP9 encoder will do some check on width
and height.

Sample pipeline:
gst-launch-1.0 videotestsrc ! \
video/x-raw,width=320,height=240,format=NV12 ! msdkvp9enc ! fakesink
2020-01-06 03:54:57 +00:00
Haihao Xiang
d6fbec5a44 msdk: make sure the found response meets the request
Otherwise the re-used msdk response doesn't have enough frames so it is
possible the pipeline will be broken
2020-01-06 02:16:53 +00:00
Julien Isorce
f578eab6fd msdkenc: renegotiate when video info changes
Renegotiation was implemented for bitrate change. We can re-use
the same sequence when video info changes except that this can be
executed right away when receiving the new input format. I.e. no
need to wait for the next call to handle_frame.
2020-01-06 01:13:28 +00:00
Julien Isorce
4e6a1b9634 msdkdec: trigger renegotiation if video info changes
Useful when framerate changes. Previously it was only checking
for resolution change but renego should happen if any video
info changes.
2020-01-03 10:19:57 -08:00
Julien Isorce
a1584b6f99 msdkdec: make sure to use video memory on Linux
The block that sets use_video_memory flag is after the
the condition `if gst_msdk_context_prepare` but it
always returns false when there is no other msdk elements.
So the decoder ends up with use_video_memory as FALSE.
Note that msdkvpp always set use_video_memory as TRUE.

When use_video_memory is FALSE then the msdkdec allocates
the output frames with posix_memalign (see gstmsdksystemmemory.c).
The result is then copied back to the GstVideoPool's buffers
(or to the downstream pool's buffers if any).
When use_video_memory is TRUE then the msdkdec uses vaCreateSurfaces
to create vaapi surfaces for the hw decoder to decode into
(see gstmsdkvideomemory.c). The result is then copied to either
the internal GstVideoPool and to the downstream pool if any.
(vaDeriveImage/vaMapBuffer is used in order to read the surfaces)
2020-01-03 07:08:23 +00:00
Seungha Yang
b88af2e4fe msdkdec: Correct return value of GstVideoDecoder::flush()
Use boolean instead of GstFlowReturn as declared.
Note that since base class does not check return value of GstVideoDecoder::flush(),
this would not cause any change of behavior.
2020-01-02 12:22:07 +09:00
Haihao Xiang
a819f05851 msdkdec: free unlocked msdk surface before output buffer allocation
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/924
is trying to use video memory for decoding on Linux, which reveals a
hidden bug in msdkdec.

For video memory, it is possible that a locked mfx surface is not used
indeed and it will be un-locked later in MSDK, so we have to check the
associated MSDK surface to find out and free un-used surfaces, otherwise
it is easy to exhaust all pre-allocated mfx surfaces and get errors below:

0:00:00.777324879 27290 0x564b65a510a0 ERROR                default
gstmsdkvideomemory.c:77:gst_msdk_video_allocator_get_surface: failed to
get surface available
0:00:00.777429079 27290 0x564b65a510a0 ERROR         msdkbufferpool
gstmsdkbufferpool.c:260:gst_msdk_buffer_pool_alloc_buffer:<msdkbufferpool0>
failed to create new MSDK memory

Note the sample code in MSDK does similar thing in
CBuffering::SyncFrameSurfaces()
2020-01-02 00:43:36 +00:00
Julien Isorce
93bc74284f msdkdec: call finalize on the parent class
Otherwise GstVideoDecoder is not finalized and
resources are leaked.

Somehow GST_TRACERS="leaks" GST_DEBUG="GST_TRACER:7" did not catch it.

Valgrind output:

==31645== 22,480 (1,400 direct, 21,080 indirect) bytes in 5 blocks are definitely lost in loss record 5,042 of 5,049
==31645==    at 0x4C2FB0F: malloc
==31645==    by 0x51D9E88: g_malloc
==31645==    by 0x51FA7B5: g_slice_alloc
==31645==    by 0x51FAC68: g_slice_alloc0
==31645==    by 0x58D9984: g_type_create_instance
==31645==    by 0x58BA344: g_object_new_with_properties
==31645==    by 0x58BADA0: g_object_new
==31645==    by 0x8ECA966: gst_video_decoder_init
==31645==    by 0x58D99E7: g_type_create_instance
==31645==    by 0x58BA344: g_object_new_with_properties
2019-12-13 15:36:26 -08:00
Stéphane Cerveau
6bc0e9527e remove various useless linefeed in logs 2019-12-11 10:51:29 +01:00
Haihao Xiang
b18be3562c msdk: ignore MFX_ERR_UNDEFINED_BEHAVIOR when loading a plugin
An issue can be seen when using msdkh265enc with bitrate change in
playing state. The root cause is the corresponding plugin is loaded
again.

Returning MFX_ERR_UNDEFINED_BEHAVIOR from MSDK just means the plugin has
been loaded, so we may ignore this error when doing configuation again
in the sub class, otherwise the pipeline will be interrupted
2019-12-10 17:23:53 +00:00
Julien Isorce
b681d6f62e msdkvpp: check mfx surface nullity for the input buffer
In theory it should not happen but it happened to me
in some cases where it failed to allocate some video
buffers so this was a consequence of a corner case.
Better to be safe than sorry.
2019-12-10 03:00:13 +00:00
Julien Isorce
3bae199863 msdkdec: fix assertion 'frame->ref_count > 0' failed
Can happen if the oldest frame is the current frame
and if gst_video_decoder_finish_frame failed in which
case the current is unref and then drop instead of
just drop.
This patch also removes some assumptions, it was strange
to call unref and finish_frame in gst_msdkdec_finish_task.
In principle when owning a frame, the code should either
unref, or drop or finish.
2019-12-09 16:10:51 +00:00
Seungha Yang
00cb881431 msdk: Fix mixed declarations warning
msdk.c:194:3: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
2019-11-27 12:25:40 +09:00
Julien Isorce
2af672d00a msdkdec: log an error if leaking surfaces in finalize
The for loop in gst_msdkdec_handle_frame is error prone
about how it manages surfaces. Because sometimes it sets
the surface variable to NULL and sometimes it needs to free
it right away. So better to print an error if surfaces are
leaked to help with any change around the loop.
2019-11-25 17:14:20 +00:00
Aaron Boxer
348494fd34 msdk: query hardware when checking MFX availability
msdk plugin is not used for sofware encode/decode as there are better
solutions available. Also, with MFX_IMPL_AUTO_ANY, if software decode
is not supported, the plugin will still load, but will then fail when trying to
run the (autoplugged) pipeline. With MFX_IMPL_HARDWARE_ANY,
the plugin fails and a better software decoder is auto-plugged.
2019-11-22 05:18:42 +00:00
Aaron Boxer
2f1256ada7 msdk: query platform when opening session 2019-11-22 05:18:42 +00:00
Julien Isorce
a7dc407041 msdkdec: fix surface leak in msdkdec_handle_frame
Can be reproduced with:
  videotestsrc ! x264enc key-int-max=$N ! \
  h264parse ! msdkh264dec ! fakesink sync=1

It happens with any gop size but the smaller is the distance N
between key frames, the quicker it is leaking.

Fixes #1023
2019-11-19 09:14:51 -08:00
Jochen Henneberg
3ecd8666d9 msdk: Fixes for meson include directory setup
In case of pkg-config we need to create the include directories object
from the path using include_directories(). For INTELMEDIASDKROOT or
MFX_HOME we need to add the alternate include path ./include/mfx as
Intel MediaSDK now puts the headers there.
2019-11-17 17:39:44 +00:00
Haihao Xiang
dca3e47bce msdk: remove msdkvp8enc
MediaSDK doesn't support vp8 encode which is not going to be
implemented [1], so remove msdkvp8enc from this plugin

[1]: https://github.com/Intel-Media-SDK/MediaSDK/issues/947
2019-11-08 08:44:46 +00:00
Aaron Boxer
6d3429af34 documentation: fixed a heap o' typos 2019-11-05 09:11:25 -05:00
Haihao Xiang
1dd3ef24b1 msdkmjpegenc: convert UYVY to YUY2 instead of NV12
Before this fix, the chroma subsampling of the output is 4:2:0. It is
4:2:2 with this fix, which is better for UYVY input
2019-11-02 09:52:36 +00:00
Haihao Xiang
e1b8b81e84 msdkmjpegenc: don't need conversion for BGRA and YUY2
jpeg encoder in MSDK can accept BGRA and YUY2 input.
2019-11-02 09:52:36 +00:00
Haihao Xiang
fbeaa37b79 msdkenc: add need_conversion method
In future, a sub class of GstMsdkEncClass may decide a native format by
using this method, e.g. JPEG encoder may accept YUY2 input, however the
current implemation needs a conversion from YUY2 to NV12 before encoding.

In addtion, a sub class may choose a format for encoding if the input
format is not supported by MSDK, e.g. the current implemation does
UYVY->NV12 if the input format is UYVY. We may do UYVY->YUY2 for JPEG
encoder in future
2019-11-02 09:52:36 +00:00
Haihao Xiang
afce02b392 msdk: support for MFX_FOURCC_BGR4 frame allocation
MFX_FOURCC_BGR4 is mapped to VA_FOURCC_ABGR and JPEG encoder needs a
MFX_FOURCC_BGR4 frame for internal usage when the input format is
MFX_FOURCC_RGB4

This is a preparation for supporting native formats of JPEG encoder
2019-11-02 09:52:36 +00:00
Víctor Manuel Jáquez Leal
10d4c0c511 msdkdec: use decoder base class packetized flag
Instead of using a proxy of `is_packetized` flag this patch
replaces it with the accessor to that flag in decoder base class,
avoiding probable mismatches.
2019-10-29 14:58:54 +00:00
Haihao Xiang
cd883427e9 msdkdec: add parse callback for non-packetized input
commit 55c0d720 added the capability to handle non-packetized bitstream,
and there is a loop to handle multiple frames in a non-packetized buffer
in gst_msdkdec_handle_frame. However it is possible that a
non-packetized buffer still contains valid data but there is no long any
pending unfinished frame. Currently gst_video_decoder_decode_frame is
invoked to send a new frame with new input data, the situaltion is
repeated till an EOS is received. An application has to exit when
receiving an EOS, however there is still valid data in a
non-packetezied input buffer, hence some frames are dropped.

This fix adds a parse callback for non-packeteized input, a new frame
will be sent to the subclass as soon as the input buffer has valid data

This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/665
2019-10-29 14:58:54 +00:00
Haihao Xiang
a2cafcfc3e msdkh265enc: Add support for tiled encoding
Add num-tile-rows and num-tile-cols properties to set the number of rows
and columns
2019-10-27 18:40:15 +00:00
Haihao Xiang
9fcd3bdd2b msdk: support P010_10LE in DMABuf mode for VP9/HEVC encoding 2019-10-25 16:54:57 +00:00
Haihao Xiang
351a3934ec msdkvpp: add support Y410 and Y210 formats
And clean up some macros
2019-10-24 13:57:51 +00:00
U. Artie Eoff
b7ee6dc49a msdkvpp: allow cropping via properties
Add crop-left, crop-right, crop-top and crop-bottom
properties to msdkvpp and apply to input surface at
vpp transform run-time.

Implements #1097
2019-10-24 12:44:49 +00:00
Julien Isorce
5de00b9ffb msdkenc: close encoder upon bitrate change
Upon bitrate change, make sure to close the encoder otherwise
the encoder is not re-initialized and the target bitrate is
never reached, and the encoder was flushed at each frame
from this moment.

Regression introduced in f2b35abcab which replaced the call
that was closing the encoder by an early return to avoid
re-initialization.
2019-10-15 15:53:20 -07:00
Tim-Philipp Müller
f218ec2794 Remove autotools build system 2019-10-14 13:54:27 +01:00
Haihao Xiang
e5279cd97a msdk: return the right pointer
The first channel in memory for MFX_FOURCC_RGB4 (VA_FOURCC_ARGB or
GST_VIDEO_FORMAT_BGRA) is B, not A. In MSDK, channle B is used to access
data for RGB4 surface. In addition, the returned pointers for
MFX_FOURCC_AYUV and MFX_FOURCC_Y410 in gst_msdk_video_memory_map_full
were wrong too before this fix.
2019-10-11 14:07:17 +00:00
Josep Torra
f2c75d548d msdkenc: handle bitrate changes in playing state
When the bitrate is changed in playing state the encoder issues a reconfig
that drains and recreates the underlaying hw encoder instance.
With this set of changes we ensure that all this work is only made when
the bitrate did actually change. It also tries to reuse the vpp buffer
pool and fixes the pool leak spotted when testing this feature.
2019-10-08 06:37:48 -07:00
Haihao Xiang
4525f8707f msdkh265enc: add the missing profile string 2019-10-08 09:40:06 +08:00
Víctor Manuel Jáquez Leal
5140bf1a1f msdkenc: guard MFX_FOURCC_Y410 with MFX version 1027 2019-10-03 13:50:15 +00:00
Haihao Xiang
71ff8236b5 msdkdec: set lockable flag
When postpone_free_surface is TRUE, the output buffer is not writable,
however the base decoder needs a writable buffer as output buffer,
otherwise it will make a copy of the output buffer. As the underlying
memory is always lockable, so we may set the LOCKABLE flag for this buffer
to avoid buffer copy in the base class.

The refcount of the output buffer is 1 when postpone_free_surface is
FALSE, so needn't set the LOCKABLE flag for this case.
2019-09-30 18:15:38 +00:00
Haihao Xiang
83100b3504 msdkh265enc: add support for Y410 in sink pad
Sample pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=Y410 !
msdkh265enc low-power=1 ! filesink location=output.h265
2019-09-30 10:44:50 +08:00
Haihao Xiang
0507547646 msdkh265enc: add support for VUYA in sink pad
sample pipelie:
gst-launch-1.0 videotestsrc ! video/x-raw,format=VUYA !
msdkh265enc low-power=1 ! filesink location=output.h265
2019-09-30 10:44:38 +08:00
Haihao Xiang
85914997f5 msdkenc: clean up code by using switch
It is convenient for us to add support for new formats in future, there
is no change in functionality.
2019-09-30 10:42:41 +08:00
Haihao Xiang
0c1c2ce45a msdk: remove some dead code 2019-09-30 09:10:31 +08:00
Haihao Xiang
ce4f9ae531 msdk: fix for mfx frame alloc response
Both MSDK and this plugin use mfxFrameAllocResponse for video and DMABuf
memory, it is possible that some GST buffers are still in use when calling
gst_msdk_frame_free, so add a reference count in the wrapper of
mfxFrameAllocResponse (GstMsdkAllocResponse) to make sure the underlying
mfx resources are still available if the corresponding buffer pool is in
use.

In addtion, currently all allocators for input or output share the same
mfxFrameAllocResponse pointer in an element, so it is possible that
the content of mfxFrameAllocResponse is updated for a new caps then all
GST buffers allocated from an old allocator will use this new content of
mfxFrameAllocResponse, which will result in unexpected behavior. In this
fix, we save the the content of mfxFrameAllocResponse in the corresponding
tructure to avoid such issue

Sample pipeline:

gst-launch-1.0 filesrc location=vp9_multi_resolutions.ivf ! ivfparse ! msdkvp9dec !
msdkvpp ! video/x-raw\(memory:DMABuf\),format=NV12 ! glimagesink
2019-09-23 09:58:28 +08:00
Haihao Xiang
e50011c064 msdk: don't cache mfxFrameAllocResponse pointer
Otherwise it is possible that different wrappers share the same
mfxFrameAllocResponse pointer, so instead of caching the pointer, we may
cache the content of mfxFrameAllocResponse
2019-09-21 19:51:38 +00:00
Haihao Xiang
f840e304f3 msdkdec: postpone surface free for VC1
For a skipped frame in VC1, MSDK returns the mfx surface of the reference
frame, so we have to make sure the corresponding surface for the
reference frame is not freed. In this fix, we postpone surface free because
we don't know whether a surface is referenced

Before this fix, the error is like as below:

New clock: GstSystemClock
0:00:00.181793130 23098 0x55f8a9d622d0 ERROR                msdkdec
gstmsdkdec.c:622:gst_msdkdec_finish_task:<msdkvc1dec0> Couldn't find the
cached MSDK surface

Sample pipeline:
gst-launch-1.0 filesrc location=input_has_skipped_frame.wmv ! asfdemux !
vc1parse ! msdkvc1dec ! glimagesink
2019-09-21 18:05:42 +00:00