Commit graph

4143 commits

Author SHA1 Message Date
Nicolas Dufresne
f330b5ae62 h265decoder: Sync with the H264 implementation
This ensures that we get the last reference to picture being outputed,
avoiding GstBuffer structure copies and simplifying the buffer management.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1713>
2020-10-21 09:05:57 -04:00
Seungha Yang
f615677051 d3d11decoder: Directly access ID3D11VideoDecoderOutputView for decoding
Decoder output view is stored in GstD3D11Memory object instead of
wrapper struct now. So qdata is no more required.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1712>
2020-10-20 17:56:57 +09:00
Seungha Yang
a4f324d668 d3d11memory: Implement ID3D11VideoDecoderOutputView pool
Similar to texture-array pool, we can reuse decoder output view
since the life time of output view is identical to that of texture-array.
In this way, we can avoid frequent output view alloc/free.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1712>
2020-10-20 17:56:52 +09:00
Seungha Yang
04376397eb d3d11memory: Move to GArray to store texture-array status
The size D3D11_REQ_TEXTURE2D_ARRAY_AXIS_DIMENSION is 2048
which is too large in practice especially for a texture
of dpb

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1712>
2020-10-20 01:14:27 +09:00
Víctor Manuel Jáquez Leal
bcd43acd42 va: Add VP9 decoder
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1702>
2020-10-19 13:15:06 +02:00
Víctor Manuel Jáquez Leal
5e3c7f2092 codecs: vp9decoder: Pass parser as new_sequence() parameter.
In order to know the chroma format, besides profile, subsampling_x and
subsampling_y are needed (Spec 7.2.2 Color config semantics). These values are
in GstVp9Parser but not in  GstVp9Framehdr.

Also, bit_depth is available in parser but not frame header. Evenmore, those
values are copied to picture structure later.

In case of VA-API, to configure the pipeline, it is require to know the chroma
format and depth.

It is possible to know chroma and depth through caps coming from vp9parser, but
it requires string parsing. It would be less error prone to get these values
through the parser structure at new_sequence() virtual method.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1700>
2020-10-16 17:50:42 +00:00
Seungha Yang
6e30cbb3fc d3d11: Introduce d3d11compositor element
Add new video composition element which is equivalent to compositor
and glvideomixer elements. When d3d11 decoder elements are used,
d3d11compositor can do efficient graphics memory handling
(zero copying or at least copying memory on GPU memory space).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1323>
2020-10-16 17:02:15 +00:00
Seungha Yang
36f7966a5e d3d11shader: Allow drawing without shader resource view
... for the case that we are rendering on target without input texture.
For example, we might want to draw arbitrary shape on render target view
without shader resource view.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1323>
2020-10-16 17:02:15 +00:00
Seungha Yang
eecb130bd9 d3d11convert: Add new subclasses for only color convert or resize
New d3d11colorconvert and d3d11scale elements will perform only
colorspace conversion and rescale, respectively. Those new elements
would be useful when only colorspace conversion or rescale is required
and the other part should be done by another elements.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1323>
2020-10-16 17:02:15 +00:00
Seungha Yang
4c19962997 d3d11colorconverter: Allow setting alpha value to use
... used for reordering case for now. In other words, non-alpha formats
such as NV12 is not supported case yet.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1323>
2020-10-16 17:02:15 +00:00
Seungha Yang
ca87289e56 d3d11colorconverter: Add support conversion with blending
This is pre-work for d3d11compositor support

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1323>
2020-10-16 17:02:15 +00:00
Seungha Yang
94ac3cc94d d3d11colorconverter: Add method to support updating destination rect
It's equivalent to GST_VIDEO_CONVERTER_OPT_DEST_* options of GstVideoConverter

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1323>
2020-10-16 17:02:15 +00:00
Seungha Yang
5c7caf70e1 d3d11: Clarify target rect to be updated
Rename internal methods to clarify which rect (i.e., input or output)
should be updated

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1323>
2020-10-16 17:02:15 +00:00
Julian Bouzas
32b0afb608 nvcodec: Report latency in decoder based on max-display-delay
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Julian Bouzas
1d4b6299dc nvcodec: Add max-display-delay decoder property
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Julian Bouzas
8cf1b7a154 nvcodec: Fix compiler error if OpenGL is not enabled
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Julian Bouzas
36030158c4 nvcodec: Add missing CUDAMemory src caps in h264 decoder
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Julian Bouzas
45cc831efd nvcodec: Add missing CUDAMemory sink caps in h264 and h265 encoders
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Julian Bouzas
07c349768a nvcodec: Fix description of cudadownload element
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
6e1c1178c2 nvcodec: Add CUDA video scale element
Add new element for video resizing using CUDA

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
fe83a12b10 nvcodec: Add CUDA video convert element
Add new element for colorspace conversion using CUDA.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
592a8d5400 nvcodec: Add generic CUDA video convert object
Introducing generic video convert object similar to video-converter
but using CUDA.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
4cc73ff9d6 nvcodec: Add support runtime CUDA kernel source compilation
Add util functions for runtime CUDA kernel source compilation
using NVRTC library. Like other nvcodec dependent libraries,
NVRTC library will be loaded via g_module_open.

Note that the NVRTC library naming is not g_module_open friendly
on Windows.
(i.e., nvrtc64_{CUDA major version}{CUDA minor version}.dll).
So users can specify the dll name using GST_NVCODEC_NVRTC_LIBNAME
environment.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
dd995bd9d4 nvcodec: Add CUDA upload/download elements with base class for CUDA filters
Similar to glupload/gldownload elements but for CUDA memory.
It will help transfer memory between system and nvidia GPU

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
b1ac8bafc9 nvcodec: Peer direct access support
If support direct access each other, use device to device memory copy
without staging host memory

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
d16a9237c3 cudacontext: Enable direct CUDA memory access over multiple GPUs
If each device context can access each other, enable peer access
for better interoperability.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
a8e9d616d6 nvenc: Support CUDA buffer pool
When upstream support CUDA memory (only nvdec for now), we will create
CUDA buffer pool.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
8635d12929 nvdec: Support CUDA buffer pool
If downstream can accept CUDA memory caps feature (currently nvenc only),
always CUDA memory is preferred.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
cf5ef5635f nvcodec: Add CUDA specific memory and bufferpool
Introducing CUDA buffer pool with generic CUDA memory support.
Likewise GL memory, any elements which are able to access CUDA device
memory directly can map this CUDA memory without upload/download
overhead via the "GST_MAP_CUDA" map flag.
Also usual GstMemory map/unmap is also possible with internal staging memory.

For staging, CUDA Host allocated memory is used (see CuMemAllocHost API).
The memory is allowing system access but has lower overhead
during GPU upload/download than normal system memory.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
He Junyan
edc6e0fa1d d3d11: vp8dec: No need to check show_frame flag when output_picture.
The VP8 base class has already handled it for us.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1670>
2020-10-15 19:01:54 +00:00
Seungha Yang
a31a6608fe codecs: vp9decoder: handle the show_frame check in base class
Same as vp8 decoder update https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1670

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1697>
2020-10-15 18:36:54 +00:00
Nicolas Dufresne
0f5fc9646e v4l2codecs: vp8: Allow baseclass from skipping frames
In preparation for !1670, this will allow the base class from skipping frames
that should not be displayed. Previously it would complain about unordered
decoding taking place in the driver.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1696>
2020-10-15 13:15:02 -04:00
Nicolas Dufresne
a3e6d9fc24 v4l2codecs: decoder: Unmark previously pending request
requests are executed in order, so while dequeuing sink buffers for previous
request, also mark these request as no longer pending. This will allow reusing
the request later.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1696>
2020-10-15 13:15:02 -04:00
Nicolas Dufresne
2e3bc60422 v4l2codecs: decoder: Properly remove pending requests
Pass the pointer instead of NULL in order to find and remove properly any
pending request from the queue. This coding error was leading to use after
free in error and early exit cases.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1696>
2020-10-15 13:14:57 -04:00
He Junyan
4af33eda76 va: basedec: Create the other pool anyway.
Fix a bug in _create_other_pool(). The old way of checking the
base->other_pool make that other_pool never be changed until the
gst_va_base_dec_stop() to stop the current decoding context.
But in some stream, the resolution may change during the decoding
process, and we need to re-negotiate the buffer pool. Then, the
old other_pool can not be clean correctly and the new correct one
can not be created.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1692>
2020-10-15 13:22:33 +00:00
He Junyan
ee739003e8 va: basedec: Should unmap src frame when dst frame map failing.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1692>
2020-10-15 13:22:33 +00:00
He Junyan
21f4c31a52 va: bufferpool: use release_buffer to clean the mem.
The current bufferpool wastes all pre-allocate buffers when the
buffer pool is actived.
The pool->priv->size is 0 for va buffer pool. And every time, the
reset_buffer() will clean all mem and make the buffer size 0, that
can cache the gst_buffer in the buffer pool.
But when the buffer pool is activing, the default_start() just
allocate the buffer and release_buffer() immediately, all the pre
allocated buffers and surfaces are destroyed because of
gst_buffer_get_size (buffer) != pool->priv->size.
We need to use release_buffer() to do the clean job at the pool
start time.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1686>
2020-10-15 12:02:29 +08:00
Víctor Manuel Jáquez Leal
18444fd993 va: basedec: copy frames logic to decide_allocation()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1673>
2020-10-10 17:18:04 +00:00
Víctor Manuel Jáquez Leal
5658c4182a va: basedec: refactor the other video pool instantiation
Just a code clean up

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1673>
2020-10-10 17:18:04 +00:00
Víctor Manuel Jáquez Leal
a6398eca17 va: basedec: add gstvabasedec helper
This is a helper for all decoders.

It is not an abstract subclass, just merely a helper that avoids code
duplication among the decoders.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1673>
2020-10-10 17:18:04 +00:00
Víctor Manuel Jáquez Leal
a5dcb35ad6 va: vp8dec: add element documentation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1673>
2020-10-10 17:18:04 +00:00
Víctor Manuel Jáquez Leal
a0ba72ed50 va: h264dec: set video alignment definition earlier
This patch renames need_cropping variable to need_videoalign which is clearer
with its function. And now GstVideoAlignment is part of GstVaH264Dec structure,
so it can be set earlier.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1673>
2020-10-10 17:18:04 +00:00
Víctor Manuel Jáquez Leal
fbbf9c629a va: allocator: add _set_format() and _get_format()
Since allocators keep an available memory queue to reuse, video format and usage
hint are now persistant while allocator's memories are around.

This patch adds _set_format() and _get_format() for both VA allocators.

_set_format() validates if given format can be used or reused. If no allocated
surface previously it creates a dummy one to fetch its offsets and
strides. Updated info is returned to callee.

GstVaPool uses _set_format() at config to verify the allocator capacity and to
get the surfaces offsets and strides, which are going to be used by the video
meta.

Allocator extracted caps are compared with caps from config and if they have
different strides or offsets, force_videometa is set.

A new bufferpool method gst_va_pool_requires_video_meta() is added return the
value of force_videometa. This value is checked in order to know if decoders
need to copy the surface if downstream doesn't announce video meta support.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1667>
2020-10-09 16:34:57 +02:00
Víctor Manuel Jáquez Leal
482e93b4d8 va: caps: added gst_caps_is_raw()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1667>
2020-10-09 16:34:57 +02:00
Víctor Manuel Jáquez Leal
6a8fa67f42 va: pool: call parent's start() method
Without preallocating buffers and memories a deadlock in pool allocator is
highly probably since it might hit the case were buffer is returned to the pool
but their memories are still hold by a copy downstream, without other
preallocated buffers available.

This kind of a hack, where buffer_reset() follow the normal path if it's called
from start().

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1667>
2020-10-09 16:34:57 +02:00
Víctor Manuel Jáquez Leal
8f936baffe va: pool: fix log's object
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1667>
2020-10-09 16:34:57 +02:00
Víctor Manuel Jáquez Leal
26ee5b1e4e va: allocator: remove noisy log message
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1667>
2020-10-09 16:34:57 +02:00
Víctor Manuel Jáquez Leal
9254045b3e va: allocator: add a todo for dmabuf_memories_setup()
It would be nice to add a surface pool for this type of surface allocation in
order to have a better control of them.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1667>
2020-10-09 16:34:56 +02:00
Víctor Manuel Jáquez Leal
a299dff9d4 va: allocator: add a surface counter
Every time a new surface is created the counter increases by one, and when it is
destroyed (or will be destroyed in case of GstVaAllocator), the counter is
decreased. Then, at allocator dispose, it is warning if the counter is not zero.

This counter will be also used to check if the allocator can change its
configuration if the counter is zero or can not.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1667>
2020-10-09 16:34:56 +02:00
Víctor Manuel Jáquez Leal
30281af83f va: allocator: remove GstVideoInfo from GstVaBufferSurface
Don't store it them anymore since it is related with the negotiated stream and
not the concrete buffer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1667>
2020-10-09 16:34:56 +02:00