Commit graph

134 commits

Author SHA1 Message Date
Seungha Yang
8f96361c81 nvcodec: nvsldec: Add support for CUDA memory
Add CUDA memory support. Note that zero copying is not supported yet

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1720>
2020-10-27 18:31:42 +00:00
Nicolas Dufresne
f22fc190e7 h265decoder: Add support for l0/l1
Add support for reference list needed for VA-API and some V4L2 decoders.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1713>
2020-10-21 09:05:57 -04:00
Nicolas Dufresne
f330b5ae62 h265decoder: Sync with the H264 implementation
This ensures that we get the last reference to picture being outputed,
avoiding GstBuffer structure copies and simplifying the buffer management.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1713>
2020-10-21 09:05:57 -04:00
Julian Bouzas
32b0afb608 nvcodec: Report latency in decoder based on max-display-delay
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Julian Bouzas
1d4b6299dc nvcodec: Add max-display-delay decoder property
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Julian Bouzas
8cf1b7a154 nvcodec: Fix compiler error if OpenGL is not enabled
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Julian Bouzas
36030158c4 nvcodec: Add missing CUDAMemory src caps in h264 decoder
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Julian Bouzas
45cc831efd nvcodec: Add missing CUDAMemory sink caps in h264 and h265 encoders
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Julian Bouzas
07c349768a nvcodec: Fix description of cudadownload element
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
6e1c1178c2 nvcodec: Add CUDA video scale element
Add new element for video resizing using CUDA

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
fe83a12b10 nvcodec: Add CUDA video convert element
Add new element for colorspace conversion using CUDA.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
592a8d5400 nvcodec: Add generic CUDA video convert object
Introducing generic video convert object similar to video-converter
but using CUDA.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
4cc73ff9d6 nvcodec: Add support runtime CUDA kernel source compilation
Add util functions for runtime CUDA kernel source compilation
using NVRTC library. Like other nvcodec dependent libraries,
NVRTC library will be loaded via g_module_open.

Note that the NVRTC library naming is not g_module_open friendly
on Windows.
(i.e., nvrtc64_{CUDA major version}{CUDA minor version}.dll).
So users can specify the dll name using GST_NVCODEC_NVRTC_LIBNAME
environment.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
dd995bd9d4 nvcodec: Add CUDA upload/download elements with base class for CUDA filters
Similar to glupload/gldownload elements but for CUDA memory.
It will help transfer memory between system and nvidia GPU

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
b1ac8bafc9 nvcodec: Peer direct access support
If support direct access each other, use device to device memory copy
without staging host memory

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
d16a9237c3 cudacontext: Enable direct CUDA memory access over multiple GPUs
If each device context can access each other, enable peer access
for better interoperability.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
a8e9d616d6 nvenc: Support CUDA buffer pool
When upstream support CUDA memory (only nvdec for now), we will create
CUDA buffer pool.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
8635d12929 nvdec: Support CUDA buffer pool
If downstream can accept CUDA memory caps feature (currently nvenc only),
always CUDA memory is preferred.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Seungha Yang
cf5ef5635f nvcodec: Add CUDA specific memory and bufferpool
Introducing CUDA buffer pool with generic CUDA memory support.
Likewise GL memory, any elements which are able to access CUDA device
memory directly can map this CUDA memory without upload/download
overhead via the "GST_MAP_CUDA" map flag.
Also usual GstMemory map/unmap is also possible with internal staging memory.

For staging, CUDA Host allocated memory is used (see CuMemAllocHost API).
The memory is allowing system access but has lower overhead
during GPU upload/download than normal system memory.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1633>
2020-10-16 15:56:49 +00:00
Sebastian Dröge
64039cdf84 gst: Update for gst_video_transfer_function_*() function renaming
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1557>
2020-09-07 12:14:47 +03:00
Matthew Waters
d1667da0c3 build: update for gl pkg-config file split
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1462>
2020-08-07 20:22:41 +10:00
Seungha Yang
e921a07715 codecs: h264decoder: Pass GstVideoCodecFrame to output_picture()
All subclasses are retrieving list to get target output frame, which
can be done by baseclass. And pass the ownership of the GstH264Picture
to subclass so that subclass can clear implementation dependent resources
before finishing the frame.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1449>
2020-07-21 17:18:42 +09:00
Seungha Yang
7a024a740f codecs: h264decoder: Don't attach extra ref of GstH264Picture to GstVideoCodecFrame
The lifecycle of GstH264Picture is being managed by our DPB
implementation. If it's still required, subclass can do that
by itself in the new_picture() method.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1449>
2020-07-21 17:18:42 +09:00
Seungha Yang
76793ffabc nvcodec: Update for documentation
* Add Since marks
* Make use of GST_PARAM_CONDITIONALLY_AVAILABLE flag
* Add documentation template caps
2020-07-02 17:21:24 +02:00
Seungha Yang
48ca7c7e93 nvh265sldec: Fix possible invalid memory access
Fix Coverity issues.

CID 1464959, 1464960, 1464961, 1464962

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1374>
2020-06-25 13:58:57 +00:00
Seungha Yang
133b902044 nvh264sldec: Remove useless double space
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1357>
2020-06-24 16:25:58 +00:00
Seungha Yang
711e964572 nvcodec: Add H265 stateless codec implementation
Add a new GstCodecs based H265 decoder element

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1357>
2020-06-24 16:25:58 +00:00
Seungha Yang
1e544f741a nvcodec: Move common methods to nvdecoder
... and remove all #ifdef from nvh264sldec implementation.
New helper methods will take care of OpenGL specific ones.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1357>
2020-06-24 16:25:58 +00:00
Seungha Yang
1f452c0577 nvh264sldec: Declare OpenGL related objects unconditionally
GstGLDisplay and GstGLContext are subclass of GstObject so we can
remove #ifdef for such object. This is prework for nvh265sldec.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1357>
2020-06-24 16:25:58 +00:00
Seungha Yang
82ddb0600b nvh264sldec: Fix for possible wrong device selction
decoder should select assigned CUDA device id

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1357>
2020-06-24 16:25:58 +00:00
Mathieu Duponchelle
ad49ae42f7 docs: mark more types as plugin API 2020-06-23 12:10:19 -04:00
Seungha Yang
e75049eb84 nvh264sldec: Fix wrong scaling list matrix scan order
Quatization matrix of NVDEC should be raster scan order but
h264parser stores it in zig-zag scan order. We need to convert
the matrix.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1290>
2020-05-23 01:06:28 +09:00
Víctor Manuel Jáquez Leal
d2aae6bb96 codecs: h264decoder: ref pic lists as decode_slice parameters
Pass reference picture lists to decode_slice() vmethods

Change gstv4l2codech264dec and gstnvh264dec accordingly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1238>
2020-05-19 16:57:09 +00:00
Tim-Philipp Müller
9539eecf20 nvcodec: fix compiler warning in certain setups
Fixes gstnvh264dec.c:648:12: warning: unused variable ‘ret’
compiler warning on raspbian where HAVE_NVCODEC_GST_GL is
not defined.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1239>
2020-05-03 11:08:29 +01:00
Seungha Yang
28dc4a1dd8 nvh264sldec: Allow constrained-baseline and baseline profiles
Like d3d11h264dec, we need to relax the condition for profiles of nvh264sldec

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1228>
2020-04-30 02:36:34 +09:00
Seungha Yang
4d585c64d1 nvcodec: Add suppport for environment based primary h264 decoder implementation
Introduce GST_USE_NV_STATELESS_CODEC environment to allow user to select
primary nvidia decoder implementation. In case the environment
GST_USE_NV_STATELESS_CODEC=h264 was set, old nvidia h264 decoder element
will not be registered. Instead, both nvh264dec and nvh264sldec
factory name will create gstcodecs based new nvidia h264 decoder element.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1198>
2020-04-24 09:23:10 +00:00
Seungha Yang
9c21923f04 nvcodec: Add H264 stateless codec implementation
Introduce GstH264Decoder based Nvidia H.264 decoder element.
Similar the element factory name of to v4l2 stateless codec,
this element can be configured with factory name "gstnvh264sldec".
Note that "sl" in the name stands for "stateless"

For now, existing nvh264dec covers more profile and formats
(e.g., interlaced stream) than this implementation.
However, this implementation allows us to control lower level
parameters such as decoded picture buffer management and therefore
we can get a chance to improve performance in terms of latency.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1198>
2020-04-24 09:23:10 +00:00
Seungha Yang
b4efdeba11 nvdec: Don't hardcode DPB size
Too many decode surface would waste GPU memory. Also it seems to be
introducing additional latency depending on stream. Since nvcodec
sdk version 9.0, CUVID parser API has been providing the minimum
required number of surface. By using it, we can save GPU memory
and reduce possible latency.
2020-04-09 16:30:58 +09:00
Seungha Yang
bd706edc52 nvh265enc: Update for video-hdr struct change
See the change of -base https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/594
2020-04-01 05:18:11 +00:00
Seungha Yang
22eab50907 nvdec: Add fallback for CUDA/OpenGL interop failure
It happens when local OpenGL context belongs to non-nvidia GPU.
2020-03-19 13:58:09 +09:00
Nirbheek Chauhan
266dc41596 nvcodec: Mark class data as may-be-leaked to quiet the leaks tracer
The class data with the caps in it will be leaked if the element is
registered but never instantiated. There is no way around this. Mark
the caps as such so that the leaks tracer does not warn about it.

This is the same as pad template caps getting leaked, which are also
marked as may-be-leaked. These objects are initialized exactly once,
and are 'global' data.
2020-02-12 00:00:51 +05:30
Nirbheek Chauhan
3ca87d9988 nvcodec: Fix crash in decoder on 32-bit Windows
Same fix as 1a7ea45ffd, but I didn't
test the decoder so I missed that the function pointers here weren't
using the correct calling convention too.
2020-02-06 13:39:52 +00:00
Sebastian Dröge
5b8ff98f96 nvdec: Don't leak template caps when registering elements with old NVIDIA driver 2020-02-05 09:49:20 +00:00
Seungha Yang
3f4a84bd32 nvenc: Query maximum supported API version
We've been using NvEncodeAPICreateInstance method to find the supported API
version, but that seems to be insufficient since there is a case
where plugin failed in opening encoding session even if NvEncodeAPICreateInstance
succeeded. Asking driver about the version would be the most certain way.
2020-02-03 14:15:28 +00:00
Nicolas Dufresne
d393232bc2 nvdec: Do not map GStreamer discont to CUVid discont
Setting the CUVID_PKT_DISCONTINUITY implies clearing any past information
about the stream in the decoder. The GStreamer discont flag is used for
discontinuity caused by a seek, for first buffer and if a buffer was
dropped. In the first two cases, the parsers and demuxers should ensure we
start from a synchronization point, so it's unlikely that delta will be
matched against the wrong state.

For packet lost, the discontinuity flag will prevent the decoder from doing
any concealment, with a result that ca be much worst visually, or freeze the
playback until an IDR is met. It's better to let the decoder handle that for
us.

Removing this flag, also workaround a but in NVidia parser that makes it
ignore our ENDOFFRAME flag and increase the latency by one frame.
2020-01-25 13:39:03 +00:00
Nicolas Dufresne
a28ce16b3f nvdec: Tell the parser we have complete pictures
This sets the CUVID_PKT_ENDOFPICTURE flag in order to inform the decoder that
we have a complete picture. This should remove one frame latency otherwise
introduce by NVidia parser.
2020-01-25 13:39:03 +00:00
Seungha Yang
a10f26aa3a nvenc: Do not access to broken encode session
If an encode session failed in initializing, the encode
session would be broken and the next nvenc API will cause crash.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1179
2020-01-21 16:34:41 +09:00
Nirbheek Chauhan
bda687344b nvcodec: Print debug info when initializing nvenc
We weren't printing the return value.
2020-01-20 17:15:55 +05:30
Nirbheek Chauhan
1a7ea45ffd nvcodec: Fix crash on 32-bit Windows
We weren't using the correct calling convention when calling CUDA and
CUVID APIs. `CUDAAPI` is `__stdcall` on Windows. This was working fine
on x64 because `__stdcall` is ignored and there's no special calling
convention. However, on x86, we need to use `__stdcall`.
2020-01-20 17:15:55 +05:30
Nirbheek Chauhan
7e93ae0638 nvcodec: cuda.h only needs glib.h, not gst.h
Just a nitpick. Also, force the compiler to use our stub header
instead of searching for it in the include paths.
2020-01-20 15:10:51 +05:30