Commit graph

3666 commits

Author SHA1 Message Date
Haihao Xiang
e3ebebc20b msdk: Fix compiler warning
This patch fixed compiler warning below:

[1/4] Compiling C object 'sys/msdk/dc44ea0@@gstmsdk@sha/gstmsdkvpp.c.o'.
../../gst-plugins-bad/sys/msdk/gstmsdkvpp.c: In function
‘gst_msdkvpp_context_prepare’:
../../gst-plugins-bad/sys/msdk/gstmsdkvpp.c:214:7: warning: suggest
parentheses around operand of ‘!’ or change ‘&’ to ‘&&’ or ‘!’ to ‘~’
[-Wparentheses]
2020-01-21 18:37:10 +00:00
Seungha Yang
a10f26aa3a nvenc: Do not access to broken encode session
If an encode session failed in initializing, the encode
session would be broken and the next nvenc API will cause crash.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1179
2020-01-21 16:34:41 +09:00
Nirbheek Chauhan
1bcf44bb1f msdk: Fix increasing memory usage in dynamic pipelines
Our context is non-persistent, and we propagate it throughout the
pipeline. This means that if we try to reuse any gstmsdk element by
removing it from the pipeline and then re-adding it, we'll clone the
mfxSession and create a new gstmsdk context as a child of the old one
inside `gst_msdk_context_new_with_parent()`.

Normally this only allocates a few KB inside the driver, but on
Windows it seems to allocate tens of MBs which leads to linearly
increasing memory usage for each PLAYING->NULL->PLAYING state cycle
for the process. The contexts will only be freed when the pipeline
itself goes to `NULL`, which would defeat the purpose of dynamic
pipelines.

Essentially, we need to optimize the case in which the element is
removed from the pipeline and re-added and the same context is re-set
on it. To detect that case, we set the context on `old_context`, and
compare it to the new one when preparing the context. If they're the
same, we don't need to do anything.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/946
2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
6834a12112 msdk: Reorganize context preparation code
Split it out into a separate function with early exits to make the
flow clearer, and document what the function is doing clearly.
No functional changes.
2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
e83d5fd8d6 msdk: Fix warning about unused variable on Windows 2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
c0d778c2c7 msdk: Use gst_clear_object()
`gst_object_replace()` is not supposed to be used for unreffing and
NULLing objects.
2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
bda687344b nvcodec: Print debug info when initializing nvenc
We weren't printing the return value.
2020-01-20 17:15:55 +05:30
Nirbheek Chauhan
1a7ea45ffd nvcodec: Fix crash on 32-bit Windows
We weren't using the correct calling convention when calling CUDA and
CUVID APIs. `CUDAAPI` is `__stdcall` on Windows. This was working fine
on x64 because `__stdcall` is ignored and there's no special calling
convention. However, on x86, we need to use `__stdcall`.
2020-01-20 17:15:55 +05:30
Nirbheek Chauhan
7e93ae0638 nvcodec: cuda.h only needs glib.h, not gst.h
Just a nitpick. Also, force the compiler to use our stub header
instead of searching for it in the include paths.
2020-01-20 15:10:51 +05:30
Josep Torra
ef37446460 msdkdec: align frame list using decoded timestamp
Before this change decoder used the oldest frame in the list to pair it
with the decoded surface. This only works when there's a perfect stream
like HEADERS,SYNCPOINT,DELTA...

When playing RTSP streams we can get imperfect streams like HEADERS,
DELTA,SYNCPOINT,DELTA... In this case decoder drops the frames
between HEADERS and SYNCPOINT which leads into using wrong PTS on
the output frames.

With this change we inject the input PTS into the bitstream and use it
to align the internal frame list with the actually decoded position.

Fixes playback with:
```
gst-launch-1.0 rtspsrc location=... latency=0 drop-on-latency=1 ! ...
```
2020-01-19 04:26:15 +00:00
Seungha Yang
e8d527df93 nvenc: Query supported minimum resolution
Hard-coded 16x16 resolution is likely to differ from the device's support
in most cases. If we can use NV_ENC_CAPS_WIDTH_MIN and NV_ENC_CAPS_HEIGHT_MIN,
update pad template with returned value.
2020-01-16 15:24:03 +00:00
Seungha Yang
58a663c1e5 nvcodec: Bump SDK header to version 9.1
Update header to query minimum resolution of encoder and to control
the number of reference frame if it's supported
2020-01-16 15:24:03 +00:00
Haihao Xiang
52abbeeefb msdkdec: support transform_meta virtual method
Allow the base class to copy GstVideoRegionOfInterestMeta data to the
output buffer when calling gst_video_decoder_finish_frame
2020-01-15 00:47:12 +00:00
Haihao Xiang
8512624a41 msdkenc: set ROI region for msdk{h264, h265}enc
A reconfig is needed when ROI is changed, otherwise the ROI parameters won't
take effect
2020-01-15 00:47:12 +00:00
Haihao Xiang
84e234a8c7 msdkenc: add set_extra_params virtual method
set_extra_params is added to allow sub class to add extra mfx parameters
for changed input frame or meta data
2020-01-15 00:47:12 +00:00
Haihao Xiang
0e2fc39f24 msdkenc: add need_reconfig virtual method
need_reconfig is added to allow sub class requires a reconfig when
the input frame or the MetaData (e.g. GstVideoRegionOfInterestMeta)
attached to the input frame is changed.
2020-01-15 00:47:12 +00:00
Seungha Yang
6e73e762c5 d3d11h265dec: Fix wrong NoRaslOutputFlag setting
... and handle EOS and EOB nals.

Only the first CRA picture should be associated with NoRaslOutputFlag
as the comment in code.
2020-01-14 08:47:30 +00:00
Nirbheek Chauhan
b7d91711f7 ipcpipeline: Minimal fixes that allow building with MSVC 2020-01-14 09:23:02 +05:30
Nirbheek Chauhan
c8db7a9127 ipcpipeline: Rework compiler checks
`pipe()` isn't used since 15927b6511,
and `socketpair()` from `#include <sys/socket.h>` is used only in the
examples. In practice, you can use probably also use anything that
allows you to create fd pairs, such as named pipes or anonymous pipes.

We use the cross-platform GstPollFD API in the plugin.
2020-01-14 09:23:02 +05:30
Seungha Yang
96f0f4b613 d3d11memory: Always use native DXGI format if device support it
Use consistent memory layout between dxva and other shader use case.
For example, use DXGI_FORMAT_NV12 texture format instead of
two textures with DXGI_FORMAT_R8_UNORM and DXGI_FORMAT_R8G8_UNORM.
2020-01-13 01:58:08 +00:00
Seungha Yang
9ee40679b5 Revert "d3d11: Add support for D3D11_USAGE_DYNAMIC"
This reverts commit ddd13fc7c0

Dynamic usage can reduce the number of copy per frame but make
things complicated and the benefit seems to not significant.
Also since we don't provide _map() method for the dynamic usage,
application cannot read buffers which make "last-sample" property
unusable in case of d3d11videosink.
2020-01-13 01:58:08 +00:00
Seungha Yang
e188893963 d3d11: Don't register decoders if unavailable
DXVA requires a hardware interface but may not be available,
such as in the case of VMs or when the GPU vendor does not provide a decoder interface.
2020-01-11 17:42:04 +09:00
Seungha Yang
616082d14a d3d11decoder: Don't return not initialized object from _new() method
... and change some debug levels since initialization failure might not be fatal.
2020-01-11 17:42:04 +09:00
Nicolas Dufresne
6d6e897762 decklink: Silence no driver / no SO messages
This g_once() is called everywhere, even in provider. This cause
spurious error when device monitor is used. Just silence or remove
the spurious logs.
2020-01-10 08:00:05 +00:00
Haihao Xiang
bdf9a76226 Revert "msdkdec: make sure to use video memory on Linux"
Commit a1584b6 caused big performance drop if the downstream element
is not a msdk element because it is very slow to read data from video
memory directly.

This reverts commit a1584b6f99.
2020-01-10 01:36:40 +00:00
Seungha Yang
49a1f022fd d3d11colorconverter: Handle P016_LE format
P016 format is no different than P010. Not much things to add code.
2020-01-09 16:29:47 +00:00
Seungha Yang
bbab229905 d3d11colorconverter: Add support for YUV to YUV conversion 2020-01-09 16:29:47 +00:00
Seungha Yang
a122f305f9 d3d11colorconverter: Add support for RGB to YUV conversion
... and remove code for RGBx since it's not supported format
by our d3d11 implementation for now.
2020-01-09 16:29:47 +00:00
Seungha Yang
da50cc4d92 d3d11format: Remove invalid format from supported format list 2020-01-09 16:29:47 +00:00
Sebastian Dröge
e4389146d9 decklinkvideosink: Always configure 10 bit YUV for VANC frames
If 8 bit are required by the device/mode then it will be converted internally
by the SDK, but the SDK won't automatically convert from 8 to 10 bit. As
such, always use 10 bit VANC.

Some devices require configuring also a 10 bit video format when using
10 bit VANC is required but those would fail regardless and the
application would have to configure the correct video format.

With newer versions of the SDK this information can be retrieved via the
BMDDeckLinkVANCRequires10BitYUVVideoFrames flag but we don't use a new
enough SDK version yet to extract this information.
2020-01-09 12:54:04 +02:00
Seungha Yang
e4daa2ef43 d3d11: Add support for Universal Windows Platform
Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2020-01-06 20:14:51 +09:00
Seungha Yang
43a8eb9e92 d3d11decoder: Fix build on non-desktop target
Although the target platform of D3D11 decoding API are both desktop and UWP app,
DXVA header is blocked by "WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)"
which is meaning that that's only for desktop app.
To workaround this inconsistent annoyingness, we need to define WINAPI_PARTITION_DESKTOP
regardless of target WinAPI partition.
2020-01-06 18:45:56 +09:00
Haihao Xiang
9064de27f9 msdkvp9enc: fix 10bit encoding
The codec profile should be consistent with the frame fourcc code, this
fixes pipeline below:

gst-launch-1.0 videotestsrc ! \
video/x-raw,width=320,height=240,format=P010_10LE ! msdkvp9enc ! \
fakesink
2020-01-06 03:54:57 +00:00
Haihao Xiang
da22ff3846 msdkvp9enc: fix width and height
The frame width and height is rounded up to 128 and 32 since commit
8daac1c, so the width, height for initialization should be rounded up to
128 and 32 too because the MSDK VP9 encoder will do some check on width
and height.

Sample pipeline:
gst-launch-1.0 videotestsrc ! \
video/x-raw,width=320,height=240,format=NV12 ! msdkvp9enc ! fakesink
2020-01-06 03:54:57 +00:00
Haihao Xiang
d6fbec5a44 msdk: make sure the found response meets the request
Otherwise the re-used msdk response doesn't have enough frames so it is
possible the pipeline will be broken
2020-01-06 02:16:53 +00:00
Julien Isorce
f578eab6fd msdkenc: renegotiate when video info changes
Renegotiation was implemented for bitrate change. We can re-use
the same sequence when video info changes except that this can be
executed right away when receiving the new input format. I.e. no
need to wait for the next call to handle_frame.
2020-01-06 01:13:28 +00:00
Philippe Normand
9f1fbd3649 decklink: Fix crash when probing without driver
If there is no decklink hardware/driver, the devices list is empty (NULL), so
this needs to be checked before iterating over the list.
2020-01-05 16:20:18 +00:00
Julien Isorce
4e6a1b9634 msdkdec: trigger renegotiation if video info changes
Useful when framerate changes. Previously it was only checking
for resolution change but renego should happen if any video
info changes.
2020-01-03 10:19:57 -08:00
Julien Isorce
a1584b6f99 msdkdec: make sure to use video memory on Linux
The block that sets use_video_memory flag is after the
the condition `if gst_msdk_context_prepare` but it
always returns false when there is no other msdk elements.
So the decoder ends up with use_video_memory as FALSE.
Note that msdkvpp always set use_video_memory as TRUE.

When use_video_memory is FALSE then the msdkdec allocates
the output frames with posix_memalign (see gstmsdksystemmemory.c).
The result is then copied back to the GstVideoPool's buffers
(or to the downstream pool's buffers if any).
When use_video_memory is TRUE then the msdkdec uses vaCreateSurfaces
to create vaapi surfaces for the hw decoder to decode into
(see gstmsdkvideomemory.c). The result is then copied to either
the internal GstVideoPool and to the downstream pool if any.
(vaDeriveImage/vaMapBuffer is used in order to read the surfaces)
2020-01-03 07:08:23 +00:00
Seungha Yang
b88af2e4fe msdkdec: Correct return value of GstVideoDecoder::flush()
Use boolean instead of GstFlowReturn as declared.
Note that since base class does not check return value of GstVideoDecoder::flush(),
this would not cause any change of behavior.
2020-01-02 12:22:07 +09:00
Haihao Xiang
a819f05851 msdkdec: free unlocked msdk surface before output buffer allocation
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/924
is trying to use video memory for decoding on Linux, which reveals a
hidden bug in msdkdec.

For video memory, it is possible that a locked mfx surface is not used
indeed and it will be un-locked later in MSDK, so we have to check the
associated MSDK surface to find out and free un-used surfaces, otherwise
it is easy to exhaust all pre-allocated mfx surfaces and get errors below:

0:00:00.777324879 27290 0x564b65a510a0 ERROR                default
gstmsdkvideomemory.c:77:gst_msdk_video_allocator_get_surface: failed to
get surface available
0:00:00.777429079 27290 0x564b65a510a0 ERROR         msdkbufferpool
gstmsdkbufferpool.c:260:gst_msdk_buffer_pool_alloc_buffer:<msdkbufferpool0>
failed to create new MSDK memory

Note the sample code in MSDK does similar thing in
CBuffering::SyncFrameSurfaces()
2020-01-02 00:43:36 +00:00
Seungha Yang
a139c8c7e8 d3d11: Add h265 decoder element
Some DPB management implementation is taken from gstreamer-vaapi
2019-12-31 02:13:48 +00:00
Seungha Yang
0e7b6526b8 d3d11: Add vp9 decoder element
Based on gstreamer-vaapi and Chromium implemetation.
2019-12-31 02:13:48 +00:00
Seungha Yang
586390b1ba d3d11: Add h264 decoder element
New decoder implementation based on dxva2 on d3d11 APIs. The DPB
management implementation is taken from Chromium.
2019-12-31 02:13:48 +00:00
Seungha Yang
7bfdeaf161 d3d11: Add support for Array typed texture memory
A ID3D11Texture2D memory can consist of multiple planes with array.
For array typed memory, GstD3D11Allocator will allocate new GstD3D11Memory
with increased reference count to the ID3D11Texture2D but different array index.
2019-12-31 02:13:48 +00:00
Stéphane Cerveau
add7878e14 bad: use of g_value_dup_string
Use helper method to get string from GValue.
2019-12-30 14:13:03 +00:00
Seungha Yang
81dde0f5b8 d3d11: Aggregate d3d11 memory usage query for dynamic-usage type decision
Even if one of downstream d3d11 elements can support dynamic-usage memory,
another one might not support it. Also, to support dynamic-usage,
both upstream and downstream d3d11device must be the same object.
2019-12-28 05:43:44 +00:00
Seungha Yang
d731bcb18a d3d11colorconvert: Remove device type dependent behavior
If d3d11colorconvert element is configured, do color space conversion
regardless of the device type whether it's S/W emulation or real H/W.
Since d3d11colorconvert is no more a child of d3d11videosinkbin,
we don't need this behavior. Note that previous code was added to
avoid color space conversion from d3d11videosink if no hardware
device is available (S/W emulation of d3d11 is too slow).
2019-12-28 05:43:44 +00:00
Seungha Yang
46186356ca d3d11colorconverter: Fix unmatched lock/unlock pair 2019-12-28 05:43:44 +00:00
Seungha Yang
ab80c771df d3d11upload: Don't hard overwrite caps features to support d3d11 memory passthrough
d3d11upload should be able to support upstream d3d11 memory, not only system memory.

Fix for following pipeline
d3d11upload ! "video/x-raw(memory:D3D11Memory)" ! d3d11videosink
2019-12-28 05:43:44 +00:00