Commit graph

3674 commits

Author SHA1 Message Date
Matthew Waters
4d528776b6 decklink: fix timestamp tracking when pausing
play->pause->sleep()->play would result in late frames being scheduled
2019-10-14 19:22:36 +11:00
Haihao Xiang
e5279cd97a msdk: return the right pointer
The first channel in memory for MFX_FOURCC_RGB4 (VA_FOURCC_ARGB or
GST_VIDEO_FORMAT_BGRA) is B, not A. In MSDK, channle B is used to access
data for RGB4 surface. In addition, the returned pointers for
MFX_FOURCC_AYUV and MFX_FOURCC_Y410 in gst_msdk_video_memory_map_full
were wrong too before this fix.
2019-10-11 14:07:17 +00:00
Josep Torra
f2c75d548d msdkenc: handle bitrate changes in playing state
When the bitrate is changed in playing state the encoder issues a reconfig
that drains and recreates the underlaying hw encoder instance.
With this set of changes we ensure that all this work is only made when
the bitrate did actually change. It also tries to reuse the vpp buffer
pool and fixes the pool leak spotted when testing this feature.
2019-10-08 06:37:48 -07:00
Haihao Xiang
4525f8707f msdkh265enc: add the missing profile string 2019-10-08 09:40:06 +08:00
Seungha Yang
5e8755cf07 d3d11window: Protect registration/creation of WIN32 window
Querying existing window class info and creation of window object
should be protected since they are not thread safe.
2019-10-04 21:15:25 +09:00
Seungha Yang
44cb5b4ff7 d3d11videosink: Don't try to post error message by d3d11window self
The d3d11window isn't GstElement. To post error message, proxy it
to d3d11videosink instead.
2019-10-04 21:15:15 +09:00
Víctor Manuel Jáquez Leal
5140bf1a1f msdkenc: guard MFX_FOURCC_Y410 with MFX version 1027 2019-10-03 13:50:15 +00:00
Haihao Xiang
71ff8236b5 msdkdec: set lockable flag
When postpone_free_surface is TRUE, the output buffer is not writable,
however the base decoder needs a writable buffer as output buffer,
otherwise it will make a copy of the output buffer. As the underlying
memory is always lockable, so we may set the LOCKABLE flag for this buffer
to avoid buffer copy in the base class.

The refcount of the output buffer is 1 when postpone_free_surface is
FALSE, so needn't set the LOCKABLE flag for this case.
2019-09-30 18:15:38 +00:00
Haihao Xiang
83100b3504 msdkh265enc: add support for Y410 in sink pad
Sample pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=Y410 !
msdkh265enc low-power=1 ! filesink location=output.h265
2019-09-30 10:44:50 +08:00
Haihao Xiang
0507547646 msdkh265enc: add support for VUYA in sink pad
sample pipelie:
gst-launch-1.0 videotestsrc ! video/x-raw,format=VUYA !
msdkh265enc low-power=1 ! filesink location=output.h265
2019-09-30 10:44:38 +08:00
Haihao Xiang
85914997f5 msdkenc: clean up code by using switch
It is convenient for us to add support for new formats in future, there
is no change in functionality.
2019-09-30 10:42:41 +08:00
Haihao Xiang
0c1c2ce45a msdk: remove some dead code 2019-09-30 09:10:31 +08:00
Seungha Yang
22d8d77b13 d3d11videosink: Use pixel aspect ratio to setup window draw area
... instead of calculated display ratio from given PAR and DAR.

d3d11window calculates output display ratio
to decide padding area per window resize event. In the formula,
actual PAR is required to handle both 1:1 PAR and non-1:1 PAR.
2019-09-26 10:23:46 +00:00
Haihao Xiang
ce4f9ae531 msdk: fix for mfx frame alloc response
Both MSDK and this plugin use mfxFrameAllocResponse for video and DMABuf
memory, it is possible that some GST buffers are still in use when calling
gst_msdk_frame_free, so add a reference count in the wrapper of
mfxFrameAllocResponse (GstMsdkAllocResponse) to make sure the underlying
mfx resources are still available if the corresponding buffer pool is in
use.

In addtion, currently all allocators for input or output share the same
mfxFrameAllocResponse pointer in an element, so it is possible that
the content of mfxFrameAllocResponse is updated for a new caps then all
GST buffers allocated from an old allocator will use this new content of
mfxFrameAllocResponse, which will result in unexpected behavior. In this
fix, we save the the content of mfxFrameAllocResponse in the corresponding
tructure to avoid such issue

Sample pipeline:

gst-launch-1.0 filesrc location=vp9_multi_resolutions.ivf ! ivfparse ! msdkvp9dec !
msdkvpp ! video/x-raw\(memory:DMABuf\),format=NV12 ! glimagesink
2019-09-23 09:58:28 +08:00
Haihao Xiang
e50011c064 msdk: don't cache mfxFrameAllocResponse pointer
Otherwise it is possible that different wrappers share the same
mfxFrameAllocResponse pointer, so instead of caching the pointer, we may
cache the content of mfxFrameAllocResponse
2019-09-21 19:51:38 +00:00
Haihao Xiang
f840e304f3 msdkdec: postpone surface free for VC1
For a skipped frame in VC1, MSDK returns the mfx surface of the reference
frame, so we have to make sure the corresponding surface for the
reference frame is not freed. In this fix, we postpone surface free because
we don't know whether a surface is referenced

Before this fix, the error is like as below:

New clock: GstSystemClock
0:00:00.181793130 23098 0x55f8a9d622d0 ERROR                msdkdec
gstmsdkdec.c:622:gst_msdkdec_finish_task:<msdkvc1dec0> Couldn't find the
cached MSDK surface

Sample pipeline:
gst-launch-1.0 filesrc location=input_has_skipped_frame.wmv ! asfdemux !
vc1parse ! msdkvc1dec ! glimagesink
2019-09-21 18:05:42 +00:00
Haihao Xiang
c689c94458 msdkdec: release the surface if this surface is not in use
If the surface is not in use, we may release it even if GST_FLOW_OK is going
to be returned, which may avoid the issue of failing to get surface
available

This fixes the regression caused by commit c05acf4
2019-09-21 17:34:23 +00:00
Haihao Xiang
8daac1c09a msdk: adjust the stride align
GstAllocationParams::align is set to 31 in msdkdec/msdken/msdkvpp, hence
the stride align should be greater than or equal to 31, otherwise it
will result in issue
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/861
(msdk: "GStreamer-CRITICAL: gst_buffer_resize_range failed" SPAM),

In addition, the stride should match the pitch alignment in the media driver,
otherwise it will result in some issues when a buffer is shared between
different elements, e.g. the NV12 issue mentioned in commit 3f2314a, which
can be reproduced by `gst-launch-1.0 vidoetestsrc ! msdkvpp !
video/x-raw\(memory:DMABuf\),format=NV12 ! glimagesink`

Fixed https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/861
2019-09-21 16:59:10 +00:00
Seungha Yang
e0b2d34237 d3d11: Use g_cond_broadcast() to wake up waiting threads
gst_d3d11_device_thread_add() can be called concurrently from different
threads so the cond signal should reach to all waiting threads.
2019-09-18 18:47:44 +09:00
Haihao Xiang
0b0d5afb53 msdkenc: remove unneeded label 2019-09-16 10:09:00 +00:00
Matthew Waters
52cd9846eb applemedia/videocache: remove unimplemented/used upload function 2019-09-16 03:15:39 +00:00
Haihao Xiang
c05acf40a8 msdkdec: set pointer to NULL to avoid freeing an used surface
The surface should be cached in the surface list when GST_FLOW_OK is
going to be returned

This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1051
2019-09-13 17:32:19 +00:00
Haihao Xiang
38c55de132 msdk: free mfx frames to avoid memory leaks in encoder and vpp 2019-09-13 16:54:21 +00:00
Haihao Xiang
2d4b0a5649 msdkdec: unref the caps to avoid memory leaks 2019-09-13 16:54:21 +00:00
Haihao Xiang
cc6ca4ec61 msdkvpp: free msdk surfaces to avoid memory leak 2019-09-13 16:54:21 +00:00
Haihao Xiang
93f72a5041 msdkvpp: unref buffer pools
Unref the pools first in caps re-negotiation to avoid memory leak
2019-09-13 16:54:21 +00:00
Haihao Xiang
3b171f70af msdk: fix memory leaks in msdk allocators 2019-09-13 16:54:21 +00:00
Fuwei Tang
9b2dc96b99 msdkenc: work-around to avoid zero fps in MediaSDK structure 2019-09-13 16:28:00 +00:00
Fuwei Tang
11bbd6f721 msdkenc: add an extra surface for hevc encoding
For some hevc 10bit 4K encoding cases, the encoding process may be
slow, and MediaSDK surface can't be released in time before one other
available surface is needed. So add an extra surface for hevc encoding
to avoid this issue.
2019-09-12 02:20:11 +00:00
Seungha Yang
52dfbbe5da nvenc: Early terminate handle_frame if the last flow was not GST_FLOW_OK
If the last flow was not GST_FLOW_OK, the encoding thread is not running
and there is nothing to pop from GAsyncQueue (this causes deadlock).

To prevent deadlock, just return the handle_frame without further encoding
process if the last flow was not GST_FLOW_OK. Note that the last flow
will be cleared per FLUSH_STOP and STREAM_START event.
2019-09-11 15:21:03 +00:00
Seungha Yang
68a51abdcd nvenc: Add support VUYA format
The addition is very simple. Map NV_ENC_BUFFER_FORMAT_AYUV format
to GST_VIDEO_FORMAT_VUYA and add a condition for the VUYA format.
2019-09-11 14:33:54 +00:00
Seungha Yang
14b9a1cffd nvdec: Add support for mpeg4 video decoding with codec_data
Decoder should handle codec_data of mpeg4 video which includes essential
config data.
2019-09-11 14:00:48 +00:00
Seungha Yang
af77988b9f nvenc: Reduce the number of pre-allocated device memory
The hard-coded upper bound 32 (or 48 depending on resolution) might waste
GPU memory and high resolution encoding causes OUT-OF-MEMORY allocation error
quite easily. This commit calculates the number of required pre-allocated
device memory based on encoding options and it can reduce the amount of device memory
used by nvenc.
2019-09-11 11:44:03 +00:00
Seungha Yang
e3508a4f26 nvdec: Update plugin description and fix typo
Use consistent description with nvenc, and fix typo s/devide/device/g
2019-09-11 15:16:45 +09:00
Seungha Yang
1cbb23cf79 nvenc: Adjust DTS when bframe is enabled
NVDEC driver always uses input timestamp without adjustment
even if bframe encoding was enabled.
So DTS can be larger than PTS when bframe was enabled.
To ensure PTS >= DTS, we should adjust the timestamp manually
based on the PTS difference between the first
encoded frame and the second one. That's also the maximum PTS/DTS
difference.
2019-09-11 13:18:12 +09:00
Seungha Yang
83a1c7a9a6 nvenc: Add qp-{min,max,const}-{i,p,b} properties
This new properties allows more detailed target QP value setting
2019-09-11 13:18:12 +09:00
Seungha Yang
d3a909ccdd nvenc: Add properties to support bframe encoding if device supports it
Note that bframe encoding capability varies with GPU architecture
2019-09-11 13:18:12 +09:00
Seungha Yang
94f2843774 nvenc: Refactoring internal buffer pool structure
To support rc-lookahead and bframe encoding, nvenc needs one more
staging queue, because NvEncEncodePicture can return NV_ENC_ERR_NEED_MORE_INPUT
but which was not considered so far.
As documented by NVENC programming guide, pending buffers should wait
other inputs until NvEncEncodePicture returns success.

New encoding flow is
- Submit raw picture buffer to encoder with NvEncEncodePicture
- The submitted input/output buffer pair will be queued to pending_queue
  - If NvEncEncodePicture returned success, then move all pair in pending_queue
    to final stage
  - Otherwise, wait more input raw pictures.

Another change is dropping NV_ENC_LOCK_INPUT_BUFFER usage.
So now nvenc always uses CUDA memory input buffer. As a result,
both opengl and system memory handling are unified.
2019-09-11 13:18:12 +09:00
Seungha Yang
e73acbaa5c nvenc: Remove pointless iteration and cleanup some code
* The number of iteration is always one so the iteration is useless
and that makes code complicated.
* Also defining named structure can code mroe readable.
* g_free is null safe
2019-09-11 13:18:12 +09:00
Seungha Yang
81272eaa82 nvenc: Add more rate-control options
New rate-control modes are introduced (if device can support)
* cbr-ld-hr: CBR low-delay high quality
* cbr-hq: CBR high quality
* vbr-hq: VBR high quality

Also, various configurable rate-control related properties are added.
2019-09-11 13:18:12 +09:00
Seungha Yang
ea19a7c715 nvenc: Add support for weighted prediction option
Note that this property will be exposed only if the device
supports the weighted prediction.
2019-09-11 13:18:12 +09:00
Seungha Yang
d05cbdbd72 nvenc: Add property for AUD insertion
Make AUD insertion configurable option
2019-09-11 13:18:12 +09:00
Seungha Yang
b3b723462e nvenc: Refactor class hierarchy to handle device capability dependent options
Introducing new dynamic class between GstNvBaseEncClass and
each subclass to be able to access device specific properties and
capabilities from each subclass implementation side.
2019-09-11 13:18:09 +09:00
Mathieu Duponchelle
f402cb07f9 msdk: actually use the include dir we compute 2019-09-11 00:14:08 +00:00
Marc Leeman
3ef503346a nvcodec: minor spell corrects in log messages 2019-09-10 23:13:17 +00:00
Seungha Yang
54c5ebbd53 msdk: Remove all DMABuf caps features on Windows
Add new macro for sink/src pad template to ensure no DMABuf caps
features are exposed on Windows. Some DMABuf caps features
were not handled by the commit 9ec62418c3
2019-09-10 13:29:11 +00:00
Seungha Yang
eb3a117548 msdkdec: Do not use video memory on Windows
Like msdkenc, do not use video memory by default on Windows.
2019-09-10 13:29:11 +00:00
Seungha Yang
d4112d3d7b msdk: Allow video and system memory share among buffers
gst_buffer_make_writable() requires exclusive reference to the
GstMemory so the _make_writable() for the msdk buffer will result
to fallback system memory copy, because the msdk memory were initialized
with GST_MEMORY_FLAG_NO_SHARE flag.

Note that, disable sharing GstMemory brings high overhead but actually
the msdk memory objects can be shared over multiple buffers.
If the memory is not shareable, newly added GstAllocator::mem_copy will
create copied msdk memory.
2019-09-10 13:29:11 +00:00
Haihao Xiang
04b77a8c9d msdkh265dec: remove the requirement on profile
Sometimes a HEVC/H265 stream doesn't have a valid profile but MSDK can
handle this stream. Like vaapih265dec, msdkh265dec may advertise the sink
caps without profile
2019-09-10 12:54:39 +00:00
Haihao Xiang
18728310c9 msdkdec: output the decoded frame immediately if decoded order is required
DecodedOrder was deprecated in msdk-2017 version, but some customers
still use this for low-latency streaming of non-b-frame encoded streams,
which needs to output the frame at once
2019-09-04 05:25:13 +00:00
Matthew Waters
82e23a27f7 decklinkaudiosink: Drop late buffers
Asking decklink to render audio data seems to be based entirely on
the sample counts which completely disregards the timestamps
we pass to decklink.  As a result, we need to explicitly check
for late buffers and drop them ourselves.
2019-09-02 11:09:02 +00:00
Seungha Yang
09fd34dbb0 nvenc: Add support runtime resolution change freely
Do not restrict allowed maximum resolution depending on the
initial resolution. If new resolution is larger than previous one,
just re-init encode session.
2019-09-02 10:59:03 +09:00
Matthew Waters
02814a43da decklink: fix macos werror build
../sys/decklink/gstdecklink.cpp:1703:7: error: format specifies type 'long' but the argument has type 'int64_t' (aka 'long long') [-Werror,-Wformat]
      persistent_id);
      ^~~~~~~~~~~~~~
/Library/Frameworks/GStreamer.framework/Versions/1.0/include/gstreamer-1.0/gst/gstinfo.h:1070:87: note: expanded from macro 'GST_DEBUG'
#define GST_DEBUG(...)                  GST_CAT_LEVEL_LOG (GST_CAT_DEFAULT, GST_LEVEL_DEBUG,   NULL, __VA_ARGS__)
                                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
/Library/Frameworks/GStreamer.framework/Versions/1.0/include/gstreamer-1.0/gst/gstinfo.h:646:31: note: expanded from macro 'GST_CAT_LEVEL_LOG'
        (GObject *) (object), __VA_ARGS__);                             \
                              ^~~~~~~~~~~
2019-08-30 14:16:54 +10:00
Seungha Yang
fa83f086be nvdec: Check flow return of the only current handle_frame() to fix seeking issue
Due to uncleared last flow, decoding after seek was never possible
(last_ret == GST_FLOW_FLUSHING).
nvdec dose not need to keep track of the previous flow return,
and actually the interest is data/even flow of the current handle_frame().
2019-08-30 11:27:27 +09:00
Seungha Yang
be3a3da829 nvdec: Fallback to system memory if OpenGL context could not support PBO memory
If the environment could not support OpenGL PBO memory, nvdec will do negotiation
with system memory as fallback.
2019-08-30 01:36:46 +09:00
Seungha Yang
069fe93452 nvdec: Add support dynamic output format change
Implementing ::negotiate() method to support runtime output format
change. If downstream was reconfigured, baseclass will invoke
::negotiate() method, and nvdec should update output memory
type depending on downstream caps.
2019-08-30 01:36:46 +09:00
Seungha Yang
39f800c449 nvdec: Re-negotiate whenever output format is changed
Input stream might be silently changed without ::set_format() call.
Since nvdec has internal parser, nvdec element can figure out the format change
by itself.
2019-08-30 01:36:41 +09:00
Seungha Yang
f4f8941a91 nvdec: Add support 4:4:4 and 4:2:0 12bit decoding
Depending on GPU architecture, HEVC decoder can support
4:4:4 format up to 12 bitdepth. This commit covers VP9 4:2:0 12 bits
decoding also.
2019-08-29 13:39:59 +00:00
Seungha Yang
ff9838fd3d nvenc: Add support for old drivers which could not understand SDK version 9.0
Add helper functions to support old drivers
with our previous SDK version 8.1
2019-08-29 13:39:59 +00:00
Seungha Yang
afebb15d99 nvenc: Use consistent snake case convention 2019-08-29 13:39:59 +00:00
Seungha Yang
1010b9f567 nvcodec: Bump SDK header to version 9.0
The latest Turing architecture (e.g., RTX serise) can support
decoding HEVC 4:4:4 format up to 12bits.
2019-08-29 13:39:59 +00:00
Seungha Yang
338a32b672 nvenc: Port to GstCudaGraphicsResource
Register openGL resource only once per memory. Also if upstream
provides the registered information, reuse the information
instead of doing it again. This can improve performance dramatically
depending on system since the resource registration might cause
high overhead.
2019-08-29 18:45:25 +09:00
Seungha Yang
d0846f8eab nvdec: Port to GstCudaGraphicsResource
Make it possible to share registered graphics resource among nvidia encoders
and decoders.
2019-08-29 18:05:51 +09:00
Seungha Yang
da075b94a9 cudautils: Add GstCudaGraphicsResource structure for better openGL interoperability
Introduce GstCudaGraphicsResource structure to represent registered
CUDA graphics resources and to enable sharing the information among
nvdec and nvenc. This structure can reduce the number of resource
registration which cause high overhead.
2019-08-29 18:04:33 +09:00
Seungha Yang
8dc2b4a393 nvdec: Port to openGL PBO memory
For openGL interoperability, nvdec uses cuGraphicsGLRegisterImage API
which is to register openGL texture image.
Meanwhile nvenc uses cuGraphicsGLRegisterBuffer API to registure openGL buffer object.
That means two kinds of graphics resources are registered per memory
when nvdec/nvenc are configured at the same time.
The graphics resource registration brings possibly high overhead
so the registration should be performed only once per resource
from optimization point of view.
2019-08-29 18:04:33 +09:00
Seungha Yang
9bfd6d13e6 nvdec: Filter openGL API version to use
To ensure PBO buffer, openGL API >= 3 is required.
2019-08-29 18:04:29 +09:00
Haihao Xiang
4841c2759d msdk: avoid reading data from freed memory
Both g_list_delete_link and g_list_remove remove an element and free it,
so l->next is invalid (catched by valgrind) after calling g_list_delete_link
or g_list_remove
2019-08-28 14:51:54 +00:00
Haihao Xiang
ffa82e07a7 msdkvpp: don't return GST_FLOW_ERROR for MFX_WRN_INCOMPATIBLE_VIDEO_PARAM
Returning MFX_WRN_INCOMPATIBLE_VIDEO_PARAM means MSDK detects some
incompatible parameters but it is resolved, and we may not regard
MFX_WRN_INCOMPATIBLE_VIDEO_PARAM as a fatal error. In this fix,
GST_FLOW_OK is returned but with a warning message so that a pipeline
may run to the end.
2019-08-28 09:37:06 +00:00
Matthew Waters
c941db0c2e ahcsrc: #define GST_USE_UNSTABLE_API for phtography inteface
Fixes werror build:

In file included from ../sys/androidmedia/gstahcsrc.c:70:
../gst-libs/gst/interfaces/photography.h:27:2: error: "The GstPhotography interface is unstable API and may change in future." [-Werror,-W#warnings]
#warning "The GstPhotography interface is unstable API and may change in future."
 ^
../gst-libs/gst/interfaces/photography.h:28:2: error: "You can define GST_USE_UNSTABLE_API to avoid this warning."  [-Werror,-W#warnings]
#warning "You can define GST_USE_UNSTABLE_API to avoid this warning."
 ^
2019-08-28 16:12:11 +10:00
Matthew Waters
6e9d0ac61b uvch264: fix werror build with clang
../subprojects/gst-plugins-bad/sys/uvch264/gstuvch264deviceprovider.c:48:1: warning: unused function 'GST__UVC_H264_DEVICE' [-Wunused-function]
G_DECLARE_FINAL_TYPE (GstUvcH264Device, gst_uvc_h264_device, GST_,
^
/usr/include/glib-2.0/gobject/gtype.h:1405:33: note: expanded from macro 'G_DECLARE_FINAL_TYPE'
  static inline ModuleObjName * MODULE##_##OBJ_NAME (gpointer ptr) {                                     \
                                ^
<scratch space>:75:1: note: expanded from here
GST__UVC_H264_DEVICE
^
../subprojects/gst-plugins-bad/sys/uvch264/gstuvch264deviceprovider.c:48:1: warning: unused function 'GST__IS_UVC_H264_DEVICE' [-Wunused-function]
/usr/include/glib-2.0/gobject/gtype.h:1407:26: note: expanded from macro 'G_DECLARE_FINAL_TYPE'
  static inline gboolean MODULE##_IS_##OBJ_NAME (gpointer ptr) {                                         \
                         ^
<scratch space>:78:1: note: expanded from here
GST__IS_UVC_H264_DEVICE
^
../subprojects/gst-plugins-bad/sys/uvch264/gstuvch264deviceprovider.c:139:1: warning: unused function 'GST_UVC_H264_DEVICE_PROVIDER' [-Wunused-function]
G_DECLARE_FINAL_TYPE (GstUvcH264DeviceProvider, gst_uvc_h264_device_provider, GST, UVC_H264_DEVICE_PROVIDER, GstDeviceProvider)
^
/usr/include/glib-2.0/gobject/gtype.h:1405:33: note: expanded from macro 'G_DECLARE_FINAL_TYPE'
  static inline ModuleObjName * MODULE##_##OBJ_NAME (gpointer ptr) {                                     \
                                ^
<scratch space>:137:1: note: expanded from here
GST_UVC_H264_DEVICE_PROVIDER
^
../subprojects/gst-plugins-bad/sys/uvch264/gstuvch264deviceprovider.c:139:1: warning: unused function 'GST_IS_UVC_H264_DEVICE_PROVIDER' [-Wunused-function]
/usr/include/glib-2.0/gobject/gtype.h:1407:26: note: expanded from macro 'G_DECLARE_FINAL_TYPE'
  static inline gboolean MODULE##_IS_##OBJ_NAME (gpointer ptr) {                                         \
                         ^
<scratch space>:140:1: note: expanded from here
GST_IS_UVC_H264_DEVICE_PROVIDER
^
2019-08-28 02:47:18 +00:00
Nirbheek Chauhan
733f5b2851 wasapi: Move to CoInitializeEx for COM initialization
CoInitialize is not allowed when targeting UWP and causes a Windows
Application Certification Kit (WACK) error.
2019-08-27 19:10:36 +00:00
Xavier Claessens
121711e4ac amc: Do not skip decoders that have no profile levels 2019-08-27 02:23:41 +00:00
Haihao Xiang
d5fd9bc22b msdkvpp: support video-direction property
video-direction property is common property in gstreamer. In addition,
both mirroring & rotation properties are marked as deprecated,
video-direction will override mirroring & rotation properties when they
are set explicitly

Fix https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1058
2019-08-26 16:39:39 +00:00
Yan Wang
f92b5614db msdkdec: Fix buffer allocation based on frame.
gst_msdkdec_finish_task() may release all frames in
GstVideoDecoder object. In this case, allocate_output_buffer()
cannot get the oldest frame to allocate buffer.
So gst_msdkdec_handle_frame() should return GST_FLOW_OK for
letting gst_video_decoder_decode_frame() to send a new frame
for decoding.

Fixes #664.
Fixes #665.
2019-08-26 13:59:16 +00:00
Yeongjin Jeong
e0042a99bf msdk: Don't use VA_RT_FORMAT_YUV420_10 if it's undefined
../sys/msdk/gstmsdkallocator_libva.c:99:16: error: ‘VA_RT_FORMAT_YUV420_10’

The minimum required version seems to VA_API_VERSION >= 1.2.0
2019-08-25 09:40:13 +00:00
gla
c5738c6125 dshowsrcwrapper: fix regression on device selection
Do not take device_name if a device has been specified. Do not take device_index into account if a device or a device name has been specified.
2019-08-23 09:53:59 +00:00
U. Artie Eoff
bd3caf0c04 msdk: vpp: rotate output frame
When vpp rotation is 90 or 270, the output frame
should be rotated, too.

Example:
  gst-launch-1.0 -vf videotestsrc \
   ! video/x-raw,width=720,height=480 \
   ! msdkvpp rotation=90 ! vaapisink
2019-08-22 18:17:30 +00:00
Seungha Yang
807e311ae8 nvdec: Always response QUERY_CONTEXT even if openGL is unavailable on the system
nvdec can response for the CUDA context type query regardless of openGL
availability.
2019-08-21 14:14:07 +09:00
Xavier Claessens
83883718ee amc: Print error when failing to register listener 2019-08-20 08:53:55 -04:00
Xavier Claessens
9ab76c73ea amc: Do not use g_log() for criticals 2019-08-20 08:53:55 -04:00
Xavier Claessens
06fc7880bb amc: crop values are not mandatory in format
Android documentation has example code how to compute width and height
when crop values are present.
https://developer.android.com/reference/android/media/MediaCodec#accessing-raw-video-bytebuffers-on-older-devices
2019-08-20 08:53:55 -04:00
Xavier Claessens
a604796c41 amc: Remove unused gst_amc_surface_texture_set_default_buffer_size() 2019-08-20 08:53:55 -04:00
Xavier Claessens
50cdaf1036 amc: Fix matrix constness in _get_transform_matrix() 2019-08-20 08:53:55 -04:00
Xavier Claessens
ad7c4f5f27 amc: Select between encoder/decoder at GstAmcCodec construct time
Magical 0/1 values where passed to gst_amc_codec_configure() flags
argument. It's more natural to have a boolean is gst_amc_codec_new().
2019-08-20 08:53:55 -04:00
Xavier Claessens
0fe0270244 amc: Remove gst_amc_format_contains_key()
It is not needed, we can just try to get the key and ignore error.
NdkMediaFormat doesn't have that method.
2019-08-20 08:53:55 -04:00
Xavier Claessens
7fcf3ebf07 amc: Turn GstAmcSurfaceTexture into a base class with JNI implementation 2019-08-20 08:53:55 -04:00
Xavier Claessens
29ef89983c amc: Create JNI wrapper for MediaCodecList
There is no NdkMediaCodecList API yet, but it is still better to isolate
JNI code. This will facilitate porting to a native API if Google ever
release one.
2019-08-20 08:53:55 -04:00
Xavier Claessens
515398a9ff amc: Move MediaCodec JNI wrapper into its own module
This will facilitate adding another implementation based on
NdkMediaCodec instead of JNI.
2019-08-20 08:53:55 -04:00
Seungha Yang
4f60117db9 nvdec: Fix possible null object unref
gst_query_get_n_allocation_pools > 0 does not guarantee that
the N th internal array has GstBufferPool object. So users should
check the returned GstBufferPool object from
gst_query_parse_nth_allocation_pool.
2019-08-20 10:14:54 +09:00
Seungha Yang
eab564d857 nvcodec: Use default flag for CUDA stream creation
Since nvdec/nvenc engine is running on default stream,
non-default CUDA stream should be synchronized with default
stream eventually.
2019-08-19 07:13:26 +00:00
Seungha Yang
ca6657367c nvenc: Use non default CUDA stream and async operation
Use CUDA async operation if possible with non default CUDA stream
2019-08-19 01:18:52 +00:00
Seungha Yang
5615e9258f nvdec: Don't use default CUDA stream
Async CUDA operation with default stream (NULL CUstream) is not much
beneficial than blocking operation since all CUDA operations which belong
to the CUDA context will be synchronized with the default stream's operation.
Note that CUDA stream will share all resources of the corresponding CUDA context
but which can help parallel operation similar to the relation between thread and process
2019-08-19 01:18:52 +00:00
Seungha Yang
20d8f54e63 nvdec: Push/Pop CUDA context around library API call 2019-08-19 01:18:52 +00:00
Seungha Yang
f7b2b1b99d nvdec: Fix timestamp mismatch on draining frames
The internal decoding state must be GST_NVDEC_STATE_PARSE before
calling CuvidParseVideoData(). Otherwise, nvdec will be confused
on decode callback as if the frame is decoding only frame and
the input timestamp of corresponding frame will be ignored.
Eventually one decoded frame will have non-increased PTS.
2019-08-18 15:52:32 +09:00
Seungha Yang
b64733972e nvdec: Do not access nvdec object from destroy function of qdata
The destroy callback can be called just before the fìnalization of
GstMiniObject. So the nvdec object might be destroyed already.
Instead, store the GstCudaContext with increased ref to safely
unregister the CUDA resource.
2019-08-16 19:40:31 +09:00
Seungha Yang
736df5604f d3dvideosink: Fix crash on WinProc handler
... caused by null pointer dereference. The d3dvideosink object might
not available yet on the handler.
2019-08-15 17:55:29 +09:00
Seungha Yang
58afcf09b4 meson: d3d11: Remove unnecessary dependency
d3d11 never use any API of gstreamer-allocators-1.0
2019-08-15 16:55:51 +09:00
Seungha Yang
5909a495a7 d3d11videosink: Take into account pixel aspect ratio
Fix unexpected cropping with non 1:1 pixel aspect-ratio.

The actual buffer width/height should be passed to gst_d3d11_window_render(),
instead of the calculated resolution. The width/height
values are parameters for copying d3d11 video memory.
Also, aspect-ratio should be considered on resize callback
to decide render rectangle size.
2019-08-15 16:55:37 +09:00
Xavier Claessens
9b9e39be24 amc: Fix crash when a sync_meta survives its sink
_amc_gl_free() could be called after the GstAmcVideoDec has been
finalized, in the case downstream still has a ref to a buffer.
2019-08-14 16:37:19 +00:00
Ignacio Casal Quinteiro
1181436545 wasapi: fix symbol redefinition build error 2019-08-12 07:20:43 +00:00
Seungha Yang
e6d21d048a nvenc: Add support YV12 format
YV12 format is supported by Nvidia NVENC without manual conversion.
So nvenc is exposing YV12 format at sinkpad template but there is some
missing point around uploading the memory to GPU.
2019-08-09 11:43:22 +09:00
Seungha Yang
8dbaed0af7 nvh265enc: Enable HDR related SEI nal insertion
If upstream provides the HDR related information, create SEI message
nals and pass them to NVENC.
2019-08-08 23:18:14 +09:00
Fuwei Tang
df59db988b msdkdec: fix decoding issue for interlaced streams
Currently h264parser produces a field or a frame for
alignment=au for interlaced streams, but the flag
MFX_BITSTREAM_COMPLETE_FRAME needs a complete frame
or complementary field pair of data, this results in
broken images being output.

Some patches have been sent out to fix h264parser,
but they are pending on some unfinished work. In
order to make gstreamer-msdk decoding work properly
for interlaced streams before h264parser is fixed,
this flag will be removed temporarily and will be
added back once h264parser if fixed.

Related to:
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/399
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/228
2019-08-08 07:55:41 +08:00
Seungha Yang
f3e12a0b56 nvh265enc: Add support YUV 444 10bits encoding
Note that h264 encoder does not support the YUV 444 10bits format
2019-08-08 00:46:16 +09:00
Seungha Yang
fa5e6f546b nvenc: Remove unnecessary constraint from YUV420 10bits capability decision
YUV444 capability shouldn't be applied to YUV420 10 bits format
2019-08-08 00:46:12 +09:00
Sebastian Dröge
99af098a7b decklinkvideosrc: Retrieve mode of the ancillary data from the frame
Instead of using the information we stored ourselves for the video frame
itself. Which was also the wrong one: it was the mode from the property,
not the autodetected one.

This fixes vanc extraction with mode=auto
2019-08-07 10:27:42 +00:00
Sebastian Dröge
2b3a14ff4e decklinkvideosrc: Also set the INTERLACED buffer flag on non-TFF buffers 2019-08-07 10:27:42 +00:00
Aaron Boxer
7dbcddff61 msdkdec: no need to cache output state info 2019-08-07 11:37:28 +03:00
Seungha Yang
cc4d0e91e3 nvenc: Fix broken RGB format support
Add missing format check introduced by the commit 7de4dbdeb2
2019-08-07 07:27:36 +00:00
Seungha Yang
9d0545d1a2 nvcodec: Wrap CUDA API return check with gst_cuda_result
The gst_cuda_result macro function is more helpful for debugging
than previous cuda_OK because gst_cuda_result prints the function
and line number. If the CUDA API return was not CUDA_SUCCESS,
gst_cuda_result will print WARNING level debug message with
error name, error text strings.
2019-08-07 00:59:36 +00:00
Seungha Yang
d69b590683 nvdec: Port to GstCUDAContext
... and drop CUvideoctxlock usage. The CUvideoctxlock basically
has the identical role of cuda context push/pop but nvdec specific
way. Since we can share the CUDA context among encoders and decoders,
use CUDA context directly for accessing GPU API.
2019-08-07 00:59:36 +00:00
Seungha Yang
5cf0351418 nvenc: Port to GstCudaContext
... and add support CUDA context sharing similar to glcontext sharing.
Multiple CUDA context per GPU is not the best practice. The context
sharing method is very similar to that of glcontext. The difference
is that there can be multiple context object on a pipeline since
the CUDA context is created per GPU id. For example, a pipeline
has nvh264dec (uses GPU #0) and nvh264device0dec (uses GPU #1),
then two CUDA context will propagated to all pipeline.
2019-08-07 00:59:36 +00:00
Seungha Yang
094e4a9f5c nvcodec: Introduce NVIDA CUDA helpers
New object and helper functions can remove duplicated code
from nvenc/nvdec. Also this is prework for CUDA device context sharing
among nvdec(s)/nvenc(s).
2019-08-07 00:59:36 +00:00
Sebastian Dröge
ab8bd0aa44 decklinkaudiosrc/decklinkvideosrc: Do nothing in BaseSrc::negotiate() and always set caps in ::create()
We don't support negotiation with downstream but simply set caps based
on the buffers we receive. This prevents renegotiation to other formats,
and negotiation to NTSC in mode=auto in the beginning until the first
buffer is received.

As side-effect of this, also remove various other caps handling code
that was working around the behaviour of the default
BaseSrc::negotiate().
2019-08-06 19:05:38 +00:00
Seungha Yang
7de4dbdeb2 nvenc: Return profile compatible input formats from GstVideoEncoder::getcaps
Do not accept any input formats which could not be supported
by downstream requested codec profiles.
2019-08-06 15:03:22 +00:00
Seungha Yang
9e81f8e700 nvenc: Fix caps negotiation failure on unspecified interlace-mode
During GstVideoInfo conversion from GstCaps, interlace-mode is
inferred to progressive so unspecified interlace-mode should not cause any
negotiation issue. Simly set GST_PAD_FLAG_ACCEPT_INTERSECT flag
on sinkpad to fix issue.
2019-08-06 15:03:22 +00:00
Seungha Yang
b43d0f785c nvenc: Remove unused member variables
Supported interlace-mode and codec profiles are checked
during plugin init and those values are never used.
2019-08-06 15:03:22 +00:00
Haihao Xiang
ef81c9d6c4 msdkdec: Update frame info from video parameters
A 10bit stream may have different depth values for Luma and Chroma, and
MSDK requires the frame info must match the corresponding video
parameters
2019-08-06 09:20:21 +00:00
Seungha Yang
f7f9f327cd nvdec: Respect upstream provided timestamp
Decoder sometimes reports nonincreasing timestamp.
Use input frame's timestamp like other decoder elements.
2019-08-05 20:32:39 +00:00
Seungha Yang
e68bfd7566 nvenc: Add support RGB 8/10bits formats
BGRA/RGBA/RGB10A2/BGR10A2 formats can be supported by nvenc.
Depending on device, supported format can be different.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1038
2019-08-05 18:55:28 +00:00
Seungha Yang
c99b160b50 nvdec: Use upstream framerate if possible
Encoded bitstream might not have valid framerate. If upstream
provided non-variable-framerate (i.e., fps_n > 0 and fps_d > 0)
use upstream framerate instead of parsed one.
2019-08-05 15:32:43 +00:00
Seungha Yang
158b4d8649 nvenc: Fix crash with unspecified framerate
Nvidia driver seems to calculating floating point framerate
without validation. This causes crash both on linux and Windows.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1012
2019-08-05 15:32:43 +00:00
Aaron Boxer
40f7001bef msdkdec: no need to cache allocation_caps 2019-08-05 13:32:15 +00:00
Seungha Yang
2a76807c9a configure: Update for nvcodec dependency change
nvcodec is compilable without external dependency
2019-07-31 15:36:04 +00:00
Seungha Yang
f1cbab7cfd nvdec: Fix build warning error
gstnvdec.c:1222:3: error: implicit declaration of function ‘memset’ [-Werror=implicit-function-declaration]
   memset (&type_info, 0, sizeof (type_info));
   ^~~~~~
2019-07-31 15:36:04 +00:00
Seungha Yang
4fa5a82762 nvenc: Fix build error with x86 msvc
__stdcall is accepted or ignored by the compiler on x64 but x86
is not the case. So the function definition should be consistent
with declaration.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1039
2019-07-30 19:12:46 +09:00
Seungha Yang
0445ed6ba5 nvenc: Fix deadlock when pad_push return was not GST_FLOW_OK
Encoding thread is terminated without any notification so
upstream streaming thread is locked because there is nothing
to pop from GAsyncQueue. If downstream returns error,
we need put SHUTDOWN_COOKIE to GAsyncQueue for chain function
can wakeup.
2019-07-30 17:49:25 +09:00
Seungha Yang
3faf439347 nvcodec: Fix broken ABI in cuda stub header to fix nvenc with opengl
Fix the broken ABI introduced by the commit 367e742e5d
From CUDA Toolkit 3.2, size_t has been used in CUDA_MEMCPY2D structure
instead of unsigned int.
2019-07-30 11:13:18 +09:00
Aaron Boxer
6c7a07bc2a msdk: enable 32 bit build on windows 2019-07-26 11:42:55 +02:00
Seungha Yang
694f91da88 nvdec: Make OpenGL dependency optional
By adding system memory support for nvdec, both en/decoder
in the nvcodec plugin are able to be usable regardless of
OpenGL dependency. Besides, the direct use of system memory
might have less overhead than OpenGL memory depending on use cases.
(e.g., transcoding using S/W encoder)
2019-07-26 00:01:23 +00:00
Sebastian Dröge
e326260173 decklink: Make sure to return a value from all code paths
False warning from MSVC, or it does not understand that
g_assert_not_reached() does not return.

...\gst-plugins-bad-1.0-1.17.0.1\sys\decklink\gstdecklink.cpp(1647) : warning C4715: 'gst_decklink_configure_duplex_mode': not all control paths return a value
2019-07-25 12:29:22 +03:00
Sebastian Dröge
b4b911da4d decklinksrc: Reset timestamp observations on format change
We will usually get timestamps starting from 0 again and due to the
format change the clock of the input might also be different.
2019-07-25 12:29:22 +03:00
Seungha Yang
733c109ce9 nvcodec: Clean up pointless return values around plugin init
Any plugin which returned FALSE from plugin_init will be blacklisted
so the plugin will be unusable even if an user install required runtime
dependency next time. So that's the reason why nvcodec returns TRUE always.

This commit is to remove possible misreading code.
2019-07-25 08:47:50 +00:00
Seungha Yang
7b9045d846 nvcodec: Change log level for g_module_open failure
Since we build nvcodec plugin without external CUDA dependency,
CUDA and en/decoder library loading failure can be natural behavior.

Emit error only when the module was opend but required symbols are missing.
2019-07-25 08:47:50 +00:00
Seungha Yang
e5a98cf9d8 nvdec: Add support for 10bits 4:2:0 decoding
This commit includes h265 main-10 profile support if the device can
decode it.

Note that since h264 10bits decoding is not supported by nvidia GPU for now,
the additional code path for h264 high-10 profile is a preparation for
the future Nvidia's enhancement.
2019-07-25 08:06:26 +00:00
Seungha Yang
d692350fc3 nvdec: Specify supported profiles of h264/h265 codec
See more details about supported formats at
nvidia codec sdk document "NVDEC_VideoDecoder_API_ProgGuide.pdf"
Table 1. Hardware Video Decoder Capabilities.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/926
2019-07-25 08:06:26 +00:00
Seungha Yang
c8640e23f4 nvdec: Skip draining before creating internal parser
GstVideoDecoder::drain/flush can be called at very initial state
with stream-start and flush-stop event, respectively.
Draning with NULL CUvideoparser seems to unsafe and that eventually
failed to handle it.
2019-07-25 07:11:04 +00:00
Aaron Boxer
96dc150b8b msdkdec: improve spelling and grammar of comments 2019-07-24 12:30:31 +02:00
Haihao Xiang
20f61db2e5 msdkdec: make sure mfx frame width/height meets MSDK's requirement
It is possible that the output region size (e.g. 192x144) is different
from the coded picture size (e.g. 192x256). We may adjust the alignment
parameters so that the padding is respected in GstVideoInfo and use
GstVideoInfo to calculate mfx frame width and height

This fixes the error below when decoding a stream which has different
output region size and coded picture size

0:00:00.057726900 28634 0x55df6c3220a0 ERROR                msdkdec
gstmsdkdec.c:1065:gst_msdkdec_handle_frame:<msdkh265dec0>
DecodeFrameAsync failed (failed to allocate memory)

Sample pipeline:

gst-launch-1.0 filesrc location=output.h265 ! h265parse ! msdkh265dec !
glimagesink
2019-07-24 08:41:13 +08:00
Haihao Xiang
8145a2236d msdkdec: remove unneeded code
Before calling gst_msdkdec_create_buffer_pool, the alignment parameters
have been adjusted.
2019-07-24 08:17:29 +08:00
Seungha Yang
367e742e5d nvcodec: Drop system installed cuda.h dependency
... and add our stub cuda header.

Newly introduced stub cuda.h file is defining minimal types in order to
build nvcodec plugin without system installed CUDA toolkit dependency.
This will make cross-compile possible.
2019-07-23 16:32:31 +09:00
Seungha Yang
a2ada54265 nvcodec: Keep requested rank for default device
Fix for default encoder and decoder element factory to make them have
higher rank than the others.
2019-07-23 10:28:52 +09:00
Seungha Yang
92afa74939 nvenc: Register elements per GPU device with capability check
* By this commit, if there are more than one device,
nvenc element factory will be created per
device like nvh264device{device-id}enc and nvh265device{device-id}enc
in addition to nvh264enc and nvh265enc, so that the element factory
can expose the exact capability of the device for the codec.

* Each element factory will have fixed cuda-device-id
which is determined during plugin initialization
depending on the capability of corresponding device.
(e.g., when only the second device can encode h265 among two GPU,
then nvh265enc will choose "1" (zero-based numbering)
as it's target cuda-device-id. As we have element factory
per GPU device, "cuda-device-id" property is changed to read-only.

* nvh265enc gains ability to encoding
4:4:4 8bits, 4:2:0 10 bits formats and up to 8K resolution
depending on device capability.
Additionally, I420 GLMemory input is supported by nvenc.
2019-07-22 21:01:41 +00:00
Seungha Yang
0239152bca nvdec: Create CUDA context with registered device id
Only the default device has been used by NVDEC so far.
This commit make it possible to use registered device id.
To simplify device id selection, GstNvDecCudaContext usage is removed.
2019-07-22 17:39:45 +00:00
Seungha Yang
1df2f13d0c nvdec: Register elements per device/codec with capability check
By this commit, each codec has its own element factory so the
nvdec element factory is removed. Also, if there are more than one device,
additional nvdec element factory will be created per
device like nvh264device{device-id}dec, so that the element factory
can expose the exact capability of the device for the codec.
2019-07-22 17:39:45 +00:00
Seungha Yang
9ec62418c3 msdk: Do not expose DMA buffer caps feature on Windows
On Windows, DMA buffer is not supported. PadTemplate with actually
supported feature seems to more make sense.
2019-07-22 16:13:43 +00:00
Seungha Yang
afe3c7e3ef nvcodec: Drop cudaGL.h dependency
nvcodec does not use any type/define/enum in cudaGL.h.
2019-07-22 23:11:14 +09:00
Seungha Yang
48a6641717 nvdec: Fix video stuttering issue with VP9
Address nvidia driver specific behavior to avoid unexpected frame mismatch
between GStreamer and NVDEC.
2019-07-19 18:44:32 +09:00
Seungha Yang
8018fa2526 nvdec: Drop async queue and handle data on callback of CUvideoparser
Callbacks of CUvideoparser is called on the streaming thread.
So the use of async queue has no benefit.

Make control flow straightforward instead of long while/switch loop.
2019-07-19 18:44:32 +09:00
Seungha Yang
8753561015 nvdec: Port to color_{primaries,transfer,matrix}_to_iso
... and update the color information only when upstream was not provided
the information.
2019-07-17 06:34:21 +00:00
Seungha Yang
e01c68524f nvenc: Specify colorimetry related VUI parameters
Set the colorimetry config for the information to be embedded in encodec bitstream.
2019-07-17 14:45:05 +09:00
Seungha Yang
c64cdf2ff3 kmssink: Fix implicit declaration build error
ffs() and strcmp() require string.h

gstkmssink.c:255:28: error: implicit declaration of function ‘ffs’ [-Werror=implicit-function-declaration]
       crtc_id = res->crtcs[ffs (crtcs_for_connector) - 1];
                            ^~~

gstkmssink.c:590:10: error: implicit declaration of function ‘strcmp’ [-Werror=implicit-function-declaration]
     if (!strcmp (property->name, prop_name)) {
          ^~~~~~
2019-07-17 01:05:32 +09:00
Sebastian Dröge
bf9ecd65cf decklinkvideosrc: Don't report that we have signal until we know for sure
Previously we would've reported that there is signal unless we know for
sure that we don't have signal. For example signal would've been
reported before the device is even opened.

Now keep track whether the signal state is unknown or not and report no
signal if we don't know yet. As before, only send an INFO message about
signal recovery if we actually had a signal loss before.
2019-07-15 12:06:25 +03:00
Seungha Yang
8862abd7c6 nvdec: Fix possible frame drop on EOS
On eos, baseclass videoencoder call finish() vfunc instead of drain()
2019-07-09 20:52:23 +09:00
Ray Tiley
b00f6c4f81 decklinkvideosrc: remove g_print
Causes a lot of output :)
2019-07-09 08:38:18 +00:00
Marc Leeman
489ff8604f nvcodec: do a generic cuda tests before going into version specifics 2019-07-08 10:37:46 +00:00
Seungha Yang
c18fda03d9 nvdec,nvenc: Port to dynamic library loading
... and put them into new nvcodec plugin.

* nvcodec plugin
Now each nvenc and nvdec element is moved to be a part of nvcodec plugin
for better interoperability.
Additionally, cuda runtime API header dependencies
(i.e., cuda_runtime_api.h and cuda_gl_interop.h) are removed.
Note that cuda runtime APIs have prefix "cuda". Since 1.16 release with
Windows support, only "cuda.h" and "cudaGL.h" dependent symbols have
been used except for some defined types. However, those types could be
replaced with other types which were defined by "cuda.h".

* dynamic library loading
CUDA library will be opened with g_module_open() instead of build-time linking.
On Windows, nvcuda.dll is installed to system path by CUDA Toolkit
installer, and on *nix, user should ensure that libcuda.so.1 can be
loadable (i.e., via LD_LIBRARY_PATH or default dlopen path)
Therefore, NVIDIA_VIDEO_CODEC_SDK_PATH env build time dependency for Windows
is removed.
2019-07-08 10:37:46 +00:00
Seungha Yang
5c3879ace6 d3d11videosink: Add new Direct3D11 video render plugin
Direct3D11 was shipped as part of Windows7 and it's obviously
primary graphics API on Windows.

This plugin includes HDR10 rendering if following requirements are satisfied
* IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h)
* Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space
* Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-07-08 08:31:47 +00:00
Haihao Xiang
f5b1c75d4c msdk: add msdkvp9enc element 2019-07-07 15:16:28 +00:00
Haihao Xiang
ba7f3f48c2 msdk: workaround for MFX_FOURCC_VP9_SEGMAP surface
MFX_FOURCC_VP9_SEGMAP surface in MSDK is an internal surface however
MSDK still call the external allocator for this surface, so this plugin
has to return UNSUPPORTED and force MSDK allocates surface using the
internal allocator.

See https://github.com/Intel-Media-SDK/MediaSDK/issues/762 for details
2019-07-07 15:16:28 +00:00
Haihao Xiang
1221898404 msdkenc: allow encode element requires extra frames
The call of MFXVideoENCODE_EncodeFrameAsync may not generate output and
the function returns MFX_ERR_MORE_DATA with NULL sync point, the input
frame is cached in this case, so it is possible that all allocated
frames go into the surfaces_used list after calling
MFXVideoENCODE_EncodeFrameAsync a few times, then the encoder will fail
to get an available surface before releasing used frames

This patch adds a new field of num_extra_frames to GstMsdkEnc and allows
encode element requires extra frames, the default value is 0.

This patch is the preparation for msdkvp9enc element.
2019-07-07 15:16:28 +00:00
Haihao Xiang
98e4967337 msdk: don't share context between msdkvpp and msdkenc
msdkenc supports CSC implicitly, so it is possible that two VPP
processes are required when a pipeline contains msdkvpp and msdkenc.
Before this fix, msdkvpp and msdkenc may share the same context, hence
the same mfx session, which results in MFX_ERR_UNDEFINED_BEHAVIOR
in MSDK because a mfx session has at most one VPP process only

This fixes the broken pipelines below:

gst-launch-1.0 videotestsrc ! video/x-raw,format=I420 ! msdkh264enc ! \
msdkh264dec ! msdkvpp ! video/x-raw,format=YUY2 ! fakesink

gst-launch-1.0 videotestsrc ! msdkvpp ! video/x-raw,format=YUY2 ! \
msdkh264enc ! fakesink
2019-06-30 06:08:31 +00:00
U. Artie Eoff
b756b13ee2 msdkmjpegdec: support 422 output
MSDK supports JPEG YUY2 (422 chroma) output color
format. The color format of input bitstream is
described by JPEGChromaFormat and JPEGColorFormat
fields in the mfxInfoMFX structure which is filled
in by the MFXVideoDECODE_DecodeHeader function.

To obtain lossless decoded output from 422 encoded
JPEGs, we must set the output color format in the
FourCC and ChromaFormat fields in the mfxFrameInfo
structure to the appropriate values at post_configure
so that they are propagated through to the srcpad
caps accordingly.
2019-06-29 17:40:06 +00:00
U. Artie Eoff
70bfd34b41 msdkdec: add post_configure virtual method
A post_configure virtual method is added to allow
codec subclasses to adjust the initialized parameters
after MFXVideoDECODE_DecodeHeader is called from the
gstmsdkdec::gst_msdkdec_handle_frame function.

This is useful if codecs want to adjust the output
parameters based on the codec-specific decoding
options that are present in the mfxInfoMFX structure
after MFXVideoDECODE_DecodeHeader initializes them.
2019-06-29 17:40:06 +00:00
Haihao Xiang
6117a2c6bc msdkvp9dec: add support for VP9 444
The output formats are VUYA for 8bit 444 and Y410 for 10bit 444.
2019-06-29 18:10:23 +02:00
Haihao Xiang
b483f3499a msdkh265dec: add support for main-444-10/main-444-10-intra profile
The output format is Y410
2019-06-29 17:33:57 +02:00
Haihao Xiang
9da737d6aa msdk: return a right pointer for Y410 when mapping a frame 2019-06-29 17:33:57 +02:00
Haihao Xiang
ba24af0490 msdk: set right BitDepth and Shift for Y410 mfx frame
BitDepth is 10 and Shitf must be set to 0 when creating Y410 mfx
frame in MSDK
2019-06-29 17:33:57 +02:00
Haihao Xiang
84a46f04a7 msdk: set parameters in mfxFrameData for a MFX_FOURCC_Y410 frame 2019-06-29 17:33:57 +02:00
Haihao Xiang
bc3a43bdb4 msdk: map MFX_FOURCC_Y410 to VA_FOURCC_Y410 2019-06-29 17:33:57 +02:00
Haihao Xiang
365f6cf0c8 msdk: map GST_VIDEO_FORMAT_Y410 to VA_FOURCC_Y410 2019-06-29 17:33:57 +02:00
Haihao Xiang
9b43a5621f msdk: map GST_VIDEO_FORMAT_Y410 to MFX_FOURCC_Y410 2019-06-29 17:33:57 +02:00
Haihao Xiang
49ae5c2de4 msdkh265dec: add support for main-444 profile
The output format is VUYA
2019-06-29 17:33:57 +02:00
Haihao Xiang
1ee2adc580 msdkh265dec: add support for main-422-10/main-422-10-intra 10bit
The ouput format is Y210
2019-06-29 14:15:32 +00:00
Haihao Xiang
a3f8faa682 msdk: set right BitDepth and Shift for Y210 mfx frame
BitDepth is 10 and Shitf must be set to 1 when creating Y210 mfx
frame in MSDK
2019-06-29 14:15:32 +00:00
Haihao Xiang
32fce730ed msdk: set parameters in mfxFrameData for a MFX_FOURCC_Y210 frame 2019-06-29 14:15:32 +00:00
Haihao Xiang
c763a6f157 msdk: map MFX_FOURCC_Y210 to VA_FOURCC_Y210 2019-06-29 14:15:32 +00:00
Haihao Xiang
47bb53a80a msdk: map GST_VIDEO_FORMAT_Y210 to VA_FOURCC_Y210 2019-06-29 14:15:32 +00:00
Haihao Xiang
24da8ec47b msdk: map GST_VIDEO_FORMAT_Y210 to MFX_FOURCC_Y210 2019-06-29 14:15:32 +00:00
Haihao Xiang
56d6df08f9 msdkh265dec: add support for main-422-10/main-422-10-intra 8bit
The output format is YUY2
2019-06-29 14:15:32 +00:00
Jan Schmidt
60c095900f uvch264: Fix autotools build.
Add gstuvch264deviceprovider.c to the Makefile.am missed
in MR 387
2019-06-28 15:46:22 +10:00
Seungha Yang
a93bf98245 meson: bluez: Early terminate configure on Windows
This plugin is for linux bluetooth stack. So the early termination can save
configure time on Windows (i.e., we can avoid glib subproject fallback)
2019-06-25 12:45:56 +00:00
Sebastian Dröge
907dc6df68 decklink: Correctly ensure >=16 byte alignment for the buffers we allocate
We'll ensure at least 64 byte alignment for AVX2 but 16 byte alignment
is what is required by the decklink SDK.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/986
2019-06-24 05:17:46 +00:00
Sebastian Dröge
5e4d10e296 decklinkvideosrc: Notify about signal loss even when dropping no-signal frames
Otherwise the application has no way of knowing that signal loss
happened other than noticing a gap between actual frames.
2019-06-20 10:11:08 +03:00
Matthew Waters
38c3d4ca7b applemedia: add the '-fobjc-arc' for darwin/ios
We use ARC so we need to signal that to the compiler.
2019-06-17 15:02:15 +10:00
Thibault Saunier
ee108d0ed2 uvch264: Implement device provider 2019-06-16 02:30:53 +00:00
Thibault Saunier
2d3de7afa4 uvch264: Factor out checking if v4l2device is uvc compatible 2019-06-16 02:30:53 +00:00
Arun Raghavan
428b8f3457 avdtpsrc: Honour initial transport volume setting before connection
We make the binding go from the avdtpsrc to the transport, so that any
initial setting before a connection made is used.
2019-06-12 10:18:27 +02:00
Thibault Saunier
c13b53d221 uvch264src: Make sure we set our segment
We were not setting self->segment and we are using it
when notifying downstream that we handled a REQUEST_KEY_UNIT
event, leading to all sort of criticals.
2019-06-11 12:14:15 -04:00
Tim-Philipp Müller
4ff14c0e74 Remove VDPAU plugin
It's been replaced by NVENC/NVDEC and even NVIDIA doesn't
support VDPAU any longer and hasn't for quite some time.

The plugin has been unmaintained and unsupported for a very
long time, and given the track record over the last 10 years
it seems highly unlikely anyone is going to make it work well,
not to mention adding plumbing for proper zero-copy or
gst-gl integration.

Closes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/828
2019-06-06 18:45:09 +01:00
Haihao Xiang
1ec231b85d msdk: return a right pointer for VUYA format
The first channel in memory is V for VUYA format, note
GST_VIDEO_FORMAT_VUYA is mapped to MFX_FOURCC_AYUV in this plugin
2019-05-31 14:51:35 +08:00
Tim-Philipp Müller
7853700b50 meson: add more plugins to plugins list
Makes sure their path gets added to the uninstalled environment
and makes sure they get included in the docs.
2019-05-30 20:41:57 +02:00
Mathieu Duponchelle
da6afdec9c doc: remove xml from comments 2019-05-29 22:58:08 +02:00
Nicolas Dufresne
112baf404e kmssink: Fixup all errno tracing
All DRM ioctl uses errno to report the error and simply returns -1
when some error occured. This patch fixes all usage of the return
value instead of errno to trace the error type and moves to g_strerror
instead of string.h strerror in order to be consistent with the rest
of GStreamer.
2019-05-26 12:17:29 +02:00
Seungha Yang
7d0646d7fe meson: Add nvenc and nvdec to plugin list when available 2019-05-18 02:47:37 +00:00
Haihao Xiang
9f588a97dc msdk: fix the typo for gst_msdkvp9dec_debug 2019-05-16 12:18:30 +08:00
Thibault Saunier
47a49f3381 docs: Build documentation with hotdoc 2019-05-13 17:00:00 -04:00
Thibault Saunier
7fe3f36ac8 Minor documentation fixes 2019-05-13 11:36:27 -04:00
Seungha Yang
a512f5b6cd nvenc: Respect display aspect ratio and set to encoding parameter
It should be written in VUI parameter for proper rendering
2019-05-09 15:42:19 +09:00
Seungha Yang
9d6e2e364c meson: Allow CUDA_PATH fallback on linux
That's what we've supported via autotools build

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/966
2019-05-08 19:42:42 +09:00
Seungha Yang
cae6b7112d nvh264enc: Fix broken Y444 format encoding with system memory Y444 support
separateColourPlaneFlag is mapped to separate_colour_plane_flag which
means Y, U and V planes are separately processed as monochrome sampled pictures.
So encoder shouldn't set that flag for normal 4:4:4 encoding.

Also for 4:4:4 encoding, NV_ENC_H264_PROFILE_HIGH_444_GUID profile must be
explicitly set.
2019-05-06 17:30:32 +09:00
Sebastian Dröge
6354769d89 decklink: Add support for parsing/outputting AFD/Bar 2019-05-03 21:20:49 +00:00
Sebastian Dröge
ba21c2106b decklink: Allow VANC to be used for all modes 2019-05-03 21:20:49 +00:00
Sebastian Dröge
88f9724114 decklink: Add support for widescreen NTSC/PAL
Same as non-widescreen but with a different pixel-aspect-ratio.
2019-05-03 21:20:49 +00:00
Roman Shpuntov
1465a7ecdd androidmedia: added path /system/vendor/etc to dependency 2019-05-03 19:10:22 +07:00
Niels De Graef
11a2df46b3 nvenc/nvdec: Add NVIDIA SDK headers to noinst_HEADERS
The tarballs that were being spun for 1.16 don't contain these headers
due to this small oversight, so let's add them.
2019-05-02 22:31:00 +00:00
Marcos Kintschner
dd7839ca43 Fixed segtotal value being always 2 due to an unused variable
The 'MAX' expression used to set segtotal always returned 2 because the unused and unitialized variable buffer_frame_count was always 0
2019-04-30 21:25:12 -03:00
Haihao Xiang
5b369e6e97 msdk: fix the build error with libva 2.4.0
This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/949
2019-04-17 13:10:10 +08:00
Peter Seiderer
670481bdeb msdkdec: fix error handling in case of unsupported hardware
Check the return value of gst_msdk_context_ensure_context and
abort in case of failure.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/945
2019-04-10 15:54:31 +00:00
Peter Seiderer
e766aaf2b1 msdkenc: fix error handling in case of unsupported hardware
Check the return value of gst_msdk_context_ensure_context and
abort in case of failure.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/945
2019-04-10 15:54:31 +00:00
Peter Seiderer
9b19968a66 msdk: fix error handling in case of unsupported hardware
Fix double gst_object_unref for GstMsdkContext.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/945
2019-04-10 15:54:31 +00:00
Haihao Xiang
e81a22d741 msdkvpp: add BGR10A2_LE format in the src pad
example pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=P010_10LE ! msdkvpp ! \
video/x-raw,format=BGR10A2_LE ! fakesink
2019-04-09 14:26:02 +00:00
Haihao Xiang
21f07e95f5 msdk: set some parameters in mfxFrameData for a MFX_FOURCC_A2RGB10 frame 2019-04-09 14:26:02 +00:00
Haihao Xiang
c40ecbe7f5 msdk: map MFX_FOURCC_A2RGB10 to VA_FOURCC_A2R10G10B10 2019-04-09 14:26:02 +00:00
Haihao Xiang
95fdc2e984 msdk: map GST_VIDEO_FORMAT_BGR10A2_LE to VA_FOURCC_A2R10G10B10 2019-04-09 14:26:02 +00:00
Haihao Xiang
53fd4a1564 msdk: map GST_VIDEO_FORMAT_BGR10A2_LE to MFX_FOURCC_A2RGB10 2019-04-09 14:26:02 +00:00
Peter Körner
58b5d5e10d decklinkvideosrc: add missing break in set_property-code 2019-04-09 12:19:58 +02:00
Shi Yan
f769758e9a Increase nvenc's encoding height to 4096
According to https://developer.nvidia.com/nvidia-video-codec-sdk

the minimum resolution nvenc supports should be 4096x4096
2019-04-04 20:19:42 +00:00
Haihao Xiang
8e4d7c3642 msdkdec: add postinit_decoder
The workaround for https://github.com/Intel-Media-SDK/MediaSDK/issues/1139
is required for vp8 only, so move this workaround to the corresponding
postinit_decoder function

The pipeline below works with this change

gst-launch-1.0 filesrc location=SA10104.vc1 ! \
'video/x-wmv,profile=(string)advanced',width=720,height=480,framerate=14/1 ! \
msdkvc1dec ! fakesink
2019-03-29 13:32:06 +08:00
Haihao Xiang
01051e7dfd msdkdec: don't set unknown picture struct to progressive for vc1
MFXVideoDECODE_DecodeHeader only parses the sequence layer for VC1, so
the structure is unknown for a stream with interlace flag set in the
sequence layer. If forcing the struct to progressive in this plugin,
MediaSDK will fail to decode such streams.
2019-03-29 13:13:07 +08:00
Haihao Xiang
5b0247db33 msdkvc1dec: add the preinit_decoder function 2019-03-29 12:57:51 +08:00
Seungha Yang
aaee282051 nvenc: Don't leak CUDA device memory
Allocated device memory should be freed with cuMemFree
2019-03-26 11:54:16 +00:00
Seungha Yang
d0f2f262aa nvenc: Ensure unmap resources before finalizing them
Mapped resource via NvEncMapInputResource() will result to unregister fail.
It can happen when paused to ready state change without EOS event.
2019-03-26 11:54:16 +00:00
Seungha Yang
60983fcea9 nvenc: Don't release stream lock on GstVideoEncoder::stop()
baseclass didn't take the stream lock during stop. It's valid only for
GstVideoEncoder::finish()

Closes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/912
2019-03-25 22:26:35 +09:00
Haihao Xiang
ac1408874b msdkvpp: add VUYA format in the src and sink pads 2019-03-25 09:50:33 +00:00
Haihao Xiang
8389e88840 msdk: return the right pointer for GST_VIDEO_FORMAT_VUYA frame
The first component for GST_VIDEO_FORMAT_VUYA in memory is V.
2019-03-25 09:50:33 +00:00
Haihao Xiang
24d12a4abf msdk: set some parameters in mfxFrameData for a MFX_FOURCC_AYUV frame 2019-03-25 09:50:33 +00:00
Haihao Xiang
5a5d707cdd msdk: map MFX_FOURCC_AYUV to VA_FOURCC_AYUV 2019-03-25 09:50:33 +00:00
Haihao Xiang
cd8696d2b2 msdk: map GST_VIDEO_FORMAT_VUYA to VA_FOURCC_AYUV 2019-03-25 09:50:33 +00:00
Haihao Xiang
f7863cc741 msdk: map GST_VIDEO_FORMAT_VUYA to MFX_FOURCC_AYUV 2019-03-25 09:50:33 +00:00
Haihao Xiang
28a1b0c418 msdkdec: avoid infinite loop
It is possible MFXVideoDECODE_DecodeFrameAsync returns MFX_ERR_INCOMPATIBLE_VIDEO_PARAM
and this error can't be recovered by retrying MFXVideoDECODE_DecodeFrameAsync
in some cases, so we need to limit the number of retries to avoid infinite loop.

This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/909
2019-03-25 09:10:38 +00:00
Haihao Xiang
2e3022c553 msdk: relicense the plugin to LGPL
Some files licensed under LGPL were included, so relicense the plugin
from BSD to LPGL. In addition, this patch removes the README per the
discussion in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/853

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/853
2019-03-17 18:15:46 +00:00
Nirbheek Chauhan
48d897f65f nvenc: Fix build when cuda is found but nvenc is disabled
tests\check\meson.build:21:5: ERROR:  Unknown variable "use_nvenc_gl".

Also do the same change in nvdec, just to be defensive about future
issues.
2019-03-11 14:45:14 +05:30
Seungha Yang
01a402e4ec nvenc: More clean up on stop for encoder to be reusable
Encoder will not configure internal pool again if input_state exist

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/912
2019-03-10 13:58:42 +09:00
Seungha Yang
74848770a5 nvenc: Ensure drain all frames on finish
To drain all queued encoding items, encoder should gracefully
wait the encoding thread without stealing queued items.
Otherwise, some input frames can be dropped.
2019-03-10 13:58:42 +09:00
Yeongjin Jeong
8e860fffd9 nvenc: Fix GValue leaks
GValue should be freed with g_value_unset()
2019-03-09 15:17:30 +00:00
Seungha Yang
ebe01995ff nvdec/nvenc: Add CUDA Toolkit 10.1 support
Update to support the latest CUDA Toolkit version 10.1
2019-03-07 21:27:44 +09:00
Peter Körner
18d3d31dc8 decklink: document duplex and keyer behaviour 2019-03-04 09:34:44 +00:00
Peter Körner
58dadd1ad0 decklink: implement duplex-mode property 2019-03-04 09:34:44 +00:00
Nirbheek Chauhan
8b738932e0 build: Don't forget to dist applemedia/avfdeviceprovider.h
Closes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/908
2019-03-01 16:20:05 +00:00
Seungha Yang
1d17e3891d mskd: Don't use MFX_FOURCC_RGB565 if it's undefined
../subprojects/gst-plugins-bad/sys/msdk/msdk.c(61): error C2065: 'MFX_FOURCC_RGB565'

The minimum required version for the format seems to MFX_VERSION >= 1028
2019-03-01 07:03:01 +00:00
Haihao Xiang
4b8e3b215a msdkdec: fix for small resolution 2019-02-28 11:51:06 +00:00
Haihao Xiang
18d410b81a msdkdec: fix for resolution change
Returning MFX_ERR_INCOMPATIBLE_VIDEO_PARAM from
MFXVideoDECODE_DecodeFrameAsync means the allocated mfx surface is not
suitable for the current frame, we need a new mfx surface and try
MFXVideoDECODE_DecodeFrameAsync again.
2019-02-28 11:08:24 +00:00
Haihao Xiang
a751b33072 msdkdec: force the alignment of width/height to 16 for vp8/vp9
MSDK library requires 16 alignment for vp8/vp9, otherwise a pipeline
for vp8/vp9 decoding might fail.

example pipeline:
gst-launch-1.0 filesrc location=vp8_1280x720.webm ! matroskademux ! \
msdkvp8dec ! fakesink

0:00:00.150565444 10657 0x55c8484036d0 ERROR                msdkdec
gstmsdkdec.c:1056:gst_msdkdec_handle_frame:<msdkvp8dec0>
DecodeFrameAsync failed (invalid video parameters)
2019-02-27 15:56:29 +00:00
Haihao Xiang
0806f94e1c msdkvpp: add RGB16 format in the sink pad
Note: MSDK doesn't support RGB16 output, hence don't add RGB16 format in
the src pad
2019-02-27 15:22:43 +00:00
Haihao Xiang
f39bcd3920 msdk: set some parameters in mfxFrameData for a RGB16 frame 2019-02-27 15:22:43 +00:00
Haihao Xiang
52b5a7e206 msdk: map MFX_FOURCC_RGB565 to VA_FOURCC_RGB565 2019-02-27 15:22:43 +00:00
Haihao Xiang
a09cff84ce msdk: map GST_VIDEO_FORMAT_RGB16 to VA_FOURCC_RGB565 2019-02-27 15:22:43 +00:00
Haihao Xiang
197f7f569b msdk: map GST_VIDEO_FORMAT_RGB16 to MFX_FOURCC_RGB565 2019-02-27 15:22:43 +00:00
Haihao Xiang
6287016a74 msdkdec: release the occupied surface for MFX_WRN_DEVICE_BUSY
When MFXVideoDECODE_DecodeFrameAsync () returns MFX_WRN_DEVICE_BUSY with
an output surface, a new input surface is required when retrying
MFXVideoDECODE_DecodeFrameAsync ().

This fixes the out-of-surface issue mentioned in
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/890
2019-02-24 19:13:57 +00:00