Commit graph

4400 commits

Author SHA1 Message Date
Víctor Manuel Jáquez Leal
f987cd8df9 va: caps: Add image formats in raw caps only for non-iHD.
This plugin, for decoders more concretely, assumes that a VA config
can do certain color conversions when mapping frames onto CPU's
memory.

This assumption was valid for i965 and Gallium drivers which generates
valid outputs in bitstreams testers (v.gr. fluster). Nonetheless, iHD,
even when it generates acceptable rendered frames, output's MD5 of
tests weren't valid.

This patch append the image formats, for color conversion when mapping
to memory, for non-iHD drivers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2128>
2021-04-02 17:18:10 +02:00
Seungha Yang
5acd08f4c1 d3d11: Fix for UYVY/VYUY format rendering
Don't assume that non-native DXGI formats support RTV and/or SRV.
We are mapping UYVY and VYUY formats to DXGI_FORMAT_R8G8_B8G8_UNORM
which doesn't support render target view

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2133>
2021-04-01 16:07:33 +09:00
Yinhang Liu
7740ab9b86 msdkh265enc: add dblk-idc property
The SDK can support deblocking reference structure [1], so add a new
property to enable this feature.

[1] https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#mfxextcodingoption2

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2122>
2021-03-31 02:27:10 +00:00
Yinhang Liu
d99e97af80 msdkh264enc: add dblk-idc property
The SDK can support deblocking reference structure [1], so add a new
property to enable this feature.

[1] https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#mfxextcodingoption2

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2122>
2021-03-31 02:27:10 +00:00
Seungha Yang
791f1da7b8 nvcodec: nvsldec: Refactor graphics api resource handling
* Move GL context object to GstNVDecoder object, and remove
  duplicated handling of each codec decoder element
* Don't create GL context too early. We can create it only if
  we need to negotiate with downstream gl element.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2089>
2021-03-29 15:10:54 +00:00
Haihao Xiang
be1f66a491 msdkenc{h264,h265}: add intra-refresh-type property
The SDK allows user to specify the intra refresh type which can improve
error resilience without significant impact on encoded bitstream size
caused by I frames [1]

[1] https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#mfxextcodingoption2

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2090>
2021-03-29 01:36:57 +00:00
Seungha Yang
f369ca1b32 d3d11vp9dec: Remove debug dump functions
Existing debug messages are not quite useful

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2117>
2021-03-28 18:00:39 +09:00
Seungha Yang
bcdd2c4984 d3d11decoder: Move zero-copy decision logic into decoder object
Get rid of all duplicated code for zero-copy decision and output buffer
allocation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2117>
2021-03-28 18:00:28 +09:00
Seungha Yang
03f1ff2042 d3d11vp9dec: Ignore show_frame flag in output_picture()
baseclass will not call output_picture() if it shouldn't be ouputted.
Note that the show_frame flag can be zero when show_existing_frame is set

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2114>
2021-03-26 14:50:18 +00:00
Seungha Yang
c9518780fd d3d11decoder: Do not hardcode the limit minimum resolution to 64
Decoder should be able to support lower resolution than 64x64

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2113>
2021-03-26 22:04:19 +09:00
Seungha Yang
f74ed67ba0 d3d11videosink: Remove DirectWrite related dead code
It's now not enabled since we moved core part to gst-libs

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2108>
2021-03-25 14:43:54 +00:00
Seungha Yang
cc7bb86aa6 mfvideoenc: Don't pass 0/1 framerate to MFT
Some MFT implementations do not accept 0/1 framerate and it will
result in encoder open failure. If framerate is unknown,
we will use arbitrary 25/1 framerate value.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2106>
2021-03-25 03:27:24 +09:00
Seungha Yang
90181642a3 d3d11decoder: Resurrect zero-copy for fixed-size DPB pool
Enable zero-copy if downstream proposed pool and therefore decoder
can know the amount of buffer required by downstream.
Otherwise decoder will copy when our DPB pool has no sufficient
buffers for later decoding operation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Seungha Yang
b392ed82ba d3d11decoder: Implement array-of-texture DPB again
Re-implementation of array-of-texture based on d3d11 memory pool.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Seungha Yang
39b9f79e11 d3d11: Implement memory pool
Major changes:
* GstD3D11Allocator: This allocator is now device-independent object
  which can allocate GstD3D11Memory object for any GstD3D11Device.
  User can get this object via gst_allocator_find(GST_D3D11_MEMORY_NAME)
* GstD3D11PoolAllocator: A new allocator implementation for texture pool.
  From now on GstD3D11BufferPool will make use of this memory pool allocator
  to avoid frequent texture reallocation. That usually happens because
  of buffer copy (gst_buffer_make_writable for example)

In addition to that, GstD3D11BufferPool will provide GstBuffer with
GstVideoMeta, because CPU access to a GstD3D11Memory without GstVideoMeta
is almost impossible since GPU drivers needs padding for stride alignment.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Seungha Yang
4e7e390cab d3d11decoder: Temporarily remove zero-copy related code
We will re-implement it based on memory pool

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Víctor Manuel Jáquez Leal
316ddddc16 va: postproc: Set one buffer in pools as minimum.
Because some elements, such as videorate check that minimum are
different of maximum number of buffers in the proposed pool, since
they might hold one or more buffers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2102>
2021-03-24 11:52:06 +00:00
Seungha Yang
e006366206 d3d11decoder: Enable high precision clock if needed
We've been doing retry with 1ms sleep if DecoderBeginFrame()
returned E_PENDING which means application should call
DecoderBeginFrame() again because GPU is busy.
The 1ms sleep() during retry would result in usually about 15ms delay
in reality because of bad clock precision on Windows.
To improve throughput performance, this commit will enable
high precision clock only for NVIDIA platform since
DecoderBeginFrame() call on the other GPU vendors seems to
succeed without retry.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2099>
2021-03-23 12:41:43 +00:00
Víctor Manuel Jáquez Leal
32fa99d3e0 va: postproc: disable passthrough as soon as possible
After the VA filter creation, when changing the element's state from NULL
to READY, immediatly checks for any filter operation requested by the user.
If any, the passthrough mode is disabled early, so there's no need for a
future renegotiation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2094>
2021-03-22 09:34:15 +01:00
Víctor Manuel Jáquez Leal
594c1450da va: postproc: rename function to gst_va_vpp_update_passthrough
Since it's widely used, a proper name will reflect its importance.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2094>
2021-03-22 09:33:58 +01:00
Matthew Waters
640a65bf96 gst: don't use volatile to mean atomic
volatile is not sufficient to provide atomic guarantees and real atomics
should be used instead.  GCC 11 has started warning about using volatile
with atomic operations.

https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1719

Discovered in https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/issues/868

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2098>
2021-03-22 14:34:36 +11:00
Seungha Yang
e22befad2e mfvideoenc: Enable Direct3D multi-thread protection
As documented by MS. See also
https://docs.microsoft.com/en-us/windows/win32/medfound/supporting-direct3d-11-video-decoding-in-media-foundation#open-a-device-handle

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 17:54:04 +09:00
Seungha Yang
824b0ce0f4 Revert "d3d11: Enable native multi-thread protection layer and make use of it"
This reverts commit 872b7f503c.

Native multi-thread protection layer seems to be consuming more CPU
resource than application side protection approach in some cases

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 17:53:58 +09:00
Seungha Yang
872b7f503c d3d11: Enable native multi-thread protection layer and make use of it
... instead of our own GRecMutex locking. In this way, any other
Direct3D11 client (MediaFoundation for example) can safely call
any Direct3D11 API even when we are sharing our Direct3D11 device
with others.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2092>
2021-03-18 16:37:37 +00:00
Seungha Yang
80792e12d4 d3d11: Use render-target and shader-resource bind flags by default
Even if bind flags is not needed by an element, other element
might need such bind flags.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2091>
2021-03-17 22:17:22 +09:00
He Junyan
805ff6346a va: vpp: Fix features lost in transform_caps().
When we transform the caps from the sink to src, or vice versa, the
"caps" passed to us may only contain parts of the features. Which
makes our vpp lose some feature in caps and get a negotiation error.
The correct way should be:
Cleaning the format and resolution of that caps, but adding all VA,
DMA features to it, making it a full feature caps. Then, clipping it
with the pad template.

fixes: #1551

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2081>
2021-03-17 10:25:11 +01:00
Seungha Yang
3f08fcb4ff d3d11decoder: WARNING if ID3D11VideoDevice is unavailable, not ERROR
gst_d3d11_decoder_new() method is also used for device capability
checking during plugin init. Although we are checking hardware
flag prior to that, it doesn't guarantee ID3D11VideoDevice interface.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2088>
2021-03-16 12:39:23 +00:00
Seungha Yang
1d6cdbdc90 mediafoundation: Fix resource leak
IMFActivate would hold its internal objects unless user call ShutdownObject(),
even if we release the IMFActivate. Here internal objects may
include Direct3D objects, such as texture, device handle for example.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2087>
2021-03-16 09:47:41 +00:00
Seungha Yang
58e451325b d3d11colorconvert: Fix caps leak
GstBaseTransform::fixate_caps() takes ownership of passed
othercaps argument.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2086>
2021-03-16 08:39:46 +00:00
Víctor Manuel Jáquez Leal
191b6cf5d4 va: postproc: synchronize segment with stream time
This is required to use dynamic controllable parameters.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2082>
2021-03-16 06:30:06 +00:00
Seungha Yang
309a940614 d3d11decoder: Do negotiation again per forward/reverse playback mode change
For reverse playback, we are always copying decoded
frame to downstream buffer. So the pool size can be
and need to be large enough.

In case that forward playback, however, we need to restrict
the max pool size for performance reason. Otherwise decoder
will keep copying decoded texture to downstream buffer pool
if decoding is faster than downstream throughput
performance and also there are queue element between them.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2083>
2021-03-16 05:54:41 +00:00
Seungha Yang
3a99517f7c d3d11videosink: Avoid switching conversion tool during playback
Decoder might be able to copy decoded texture to the other buffer pool
during playback depending on context. In that case, copied one
has no D3D11_BIND_DECODER bind flag.

If we used ID3D11VideoProcessor previously for decoder texture,
and incoming texture supports ID3D11VideoProcessor as well even if it has no
D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example),
allow zero-copying instead of using our fallback texture.

Frequent conversion tool change (between ID3D11VideoProcessor and generic shader)
might result in inconsistent image quality.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 21:00:22 +09:00
Víctor Manuel Jáquez Leal
c74b230579 va: postproc: only drop filters if they change
Currently, at every frame the filters array is recreated. This is not
optimal, since it should be only rebuilt if the VA filter's related
properties change. This patches does that by using a flag.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2078>
2021-03-15 06:10:27 +00:00
Seungha Yang
72345875b1 d3d11decoder: Refactor device open step and negotiation
* Remove redundant method arguments
* Don't allocate staging texture if downstream supports d3d11 memory

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 17:34:20 +09:00
Seungha Yang
347d9ceb4e d3d11decoder: Move profile GUID handling into decoder object
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 16:31:31 +09:00
Seungha Yang
1c1cfc4ba7 d3d11decoder: Get rid of private struct
Completely hide member variables

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 16:30:19 +09:00
Seungha Yang
78e4251750 Revert "d3d11vp9dec: Add support for internal frame resizing"
This reverts commit 58a4c33a0e

We should use ID3D11VideoProcessor instead of shader
to avoid copy. We need to revisit this topic later

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 16:30:15 +09:00
Seungha Yang
615f52f2f7 d3d11device: Hold ID3D11VideoDevice and ID3D11VideoContext object
... instead of QueryInterface-ing per elements. Note that
ID3D11VideoDevice and ID3D11VideoContext objects might not be available
if device doesn't support video interface.
So GstD3D11Device object will create those objects only when requested.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 14:44:55 +09:00
Seungha Yang
b08310f748 d3d11: Run gst-indent for all C++ code
Since all d3d11 plugin implementation code are C++, we need to
run gst-indent manually. This is preparation for later
"gst-indent sys/d3d11/*.cpp" run.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 13:40:12 +09:00
Seungha Yang
5b3e316039 d3d11: Port to C++
Direct3D11 objects are COM, and most COM C APIs are verbose
(C++ is a little better). So, by using C++ APIs, we can make code
shorter and more readable.
Moreover, "ComPtr" helper class (which is C++ only) can be
utilized, that is very helpful for avoiding error-prone COM refcounting
issue/leak.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 13:05:22 +09:00
Víctor Manuel Jáquez Leal
0b2848fc32 va: postproc: update passthrough and reconfigure pads
Added helper function _update_passthrough() which will define and set
the pass-through mode of the filter, and it'll either reconfigure both
pads or it will just mark the src pad for renegotiation or nothing at
all.

There are cases where both pads have to be reconfigured (direction
changed, for example), other when just src pad has to (filters
updated) or none (changing to ready state).

The requirement of renegotiation depends on the need to enable/disable
its VA buffer pools.

This patch sets pass-through mode by default, so the buffer pools
aren't allocated if no filtering/direction operations are defined,
which is the correct behavior.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2074>
2021-03-12 12:34:46 +01:00
Seungha Yang
9da6da20e7 nvh264sldec: Reopen decoder object if larger DPB size is required
Equivalent to the d3d11h264dec fix
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1839

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2059>
2021-03-07 16:51:59 +09:00
Seungha Yang
a29c8935b7 d3d11: Fix an HLSL compiler warning
warning X3578: Output value 'main' is not completely initialized

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2052>
2021-03-04 20:34:23 +00:00
Seungha Yang
3730ea3366 d3d11h264dec: Keep track of actually configured DPB size
... instead of the largest we ever seen.
Note that d3d11h264dec element holds previously configured DPB size
for later decoder object re-open decision.

This is to fix below case:
1) Initial SPS, required DPB size is 6
  - decoder object is opened with DPB size 6
  - max_dpb_size is now 6
2) SPS update with resolution change, required DPB size is 1
  - decoder object is re-opened with DPB size 1
  - max_dpb_size should be updated to 1, but it didn't happen (BUG)
3) SPS update without resolution change, only required DPB size is updated to 6
  - decoder object should be re-opened but didn't happen
    because we didn't update max_dpb_size at 2).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2056>
2021-03-04 17:57:14 +09:00
Benjamin Gaignard
5e8daa1c5c v4l2codecs: h264: retrieve interlaced information
Lets the decoder knows if the frames are interlaced or not.
Provide this information to the driver while filling reference
pictures fields in slice params structure

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00
Nicolas Dufresne
84daea6be2 v4l2codecs: h264: Reduce controls for subsequent slices
Only the SLICE_PARAMS and PRED_WEIGHTS are needed for the second and
following slices.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00
Ezequiel Garcia
1c082b79e1 v4l2codecs: h264: Implement optional scaling matrix
The new H.264 uAPI requires that all drivers support
scaling matrix only as an option, when a non-flat
scaling matrix is provided in the bitstream headers.

Take advantage of this and avoid passing the scaling
matrix if not needed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00
Ezequiel Garcia
78756ecba9 v4l2codecs: h264: Only set SPS control if needed
Given V4L2 controls are cached in V4L2, there is no need
to set them if they don't change. Set the SPS control
only if a new sequence was received by the parser.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00
Ezequiel Garcia
ae9411e334 v4l2codecs: h264: Only slice-based need SLICE_PARAMS and PRED_WEIGHTS
Frame-based decoding mode doesn't require SLICE_PARAMS and
PRED_WEIGHTS controls.

Moreover, if the driver doesn't support these two controls, trying
to set them will fail. Fix this by only setting these on
slice-based decoding mode.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00
Ezequiel Garcia
010565eb7f v4l2codecs: h264: Add API checks
Check that the V4L2 H264 controls' sizes match
our expectation. If not, then probably there's an API
mismatch which will cause errors or decoding corruption.

Also, print a warning if the kernel version is too old.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00