Commit graph

4495 commits

Author SHA1 Message Date
Seungha Yang
ad8529caf7 d3d11: Disable D3D11Debug and DXGIDebug layer for UWP build
WACK (Windows App Certification Kit) doesn't seem to be happy with
the DXGIGetDebugInterface1 symbol.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1421>
2020-07-08 21:11:31 +00:00
Haihao Xiang
9e977832c1 msdkh265enc: let MSDK select the encoding mode by default
MSDK may support lowpower and non-lowpower modes, some features are
available only under one of the two modes, which is hard to know for
user, so let MSDK select the mode by default.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1405>
2020-07-06 14:43:31 +00:00
Matthew Waters
c94675f1d4 decklinkvideosink: write the cdp timecode data correctly
We were mixing up the tens part with the unit parts all over the place.

e.g. 12 seconds would be encoded as 0x21 instead of the correct 0x12

Aligns the code with the same change applied to ccconverter.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1400>
2020-07-03 06:54:46 +00:00
Nicolas Dufresne
1bef43f9d4 v4l2decoder: Track pending request
With the asynchronous slice decoding, we only queue up to 2 slices
per frames. That side effect is that now we are dequeuing bitstream
buffers in both decoding and presentation order. This would lead to
a bitstream buffer from a previous frame being dequeued instead of
the expected last slice buffer and lead to us trying to queue an
already queued bitstream buffer.

We now fix this by tracking pending requests. As request are executed
in decoding order, we marking a request done, we can effectively
dequeue bitstream buffer from all previous request, as they have been
executed already.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1395>
2020-07-02 12:21:51 -04:00
Nicolas Dufresne
a88e63dd2f v4l2decoder: Improve debug tracing
Add some missing traces and move per-slice operation to TRACE level to
reduce the noise level.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1395>
2020-07-02 12:21:51 -04:00
Nicolas Dufresne
d5a205cff4 v4l2decoder: Convert request pool to GstQueueArray
The decoder is not being access from multiple threads, instead it is
always protected by the streaming lock. For this reason, a
GstAtomicQueue for the request pool is overkill and may even introduce
unneeded overhead. Use a GstQueueArray in replacement, the
GstQueueArray is a good fit since the number of item is predictable and
unlikely to vary at run-time.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1395>
2020-07-02 12:21:51 -04:00
Nicolas Dufresne
a2eb1b57ff v4l2slh264dec: Wait on previous pending request in slice mode
In slice mode, we'll do one request per slice. In order to recycle
bitstream buffer, and not run-out, wait for the last pending
request to complete and mark it done.

We only wait after having queued the current slice in order to reduce
that potential driver starvation and maintain performance (using dual
buffering).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1395>
2020-07-02 12:21:51 -04:00
Nicolas Dufresne
b20c6fe6c4 v4l2slh264dec: Renew bitstream buffer after submitting slice
Submitting a slice actually clears the bitstream buffer. Ensure we
have a newly allocated bitstream buffer for the next slice.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1395>
2020-07-02 12:21:51 -04:00
Nicolas Dufresne
bc1a0323a9 v4l2slh264dec: Factor out bitstream allocation
No functional changes.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1395>
2020-07-02 12:21:51 -04:00
Nicolas Dufresne
779f331bd4 v4l2slh264dec: Add a helper to ensure output buffer
In preparation of multi-slice decoding, we will decode multiple
slices into the same buffer. This will ensure we have a buffer to
decode to, queued into the driver.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1395>
2020-07-02 12:21:51 -04:00
Nicolas Dufresne
d65f7de650 v4l2slh264dec: Factor out request wait
This will be reused to wait for previous slices to be complete
when dealing with following slices (in slice decoding mode).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1395>
2020-07-02 12:21:51 -04:00
Nicolas Dufresne
176a860169 v4l2slh264dec: Remove double return in submit_bitstream()
This is code cleanup, no functional changes.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1395>
2020-07-02 12:21:51 -04:00
Nicolas Dufresne
1f48e60bde v4l2slh264dec: Fix typo in debug trace
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1395>
2020-07-02 12:21:51 -04:00
Mathieu Duponchelle
b33f10e7e2 docs: remove gst prefix from plugin titles 2020-07-02 18:10:21 +02:00
Seungha Yang
8d0dc4fdd2 plugins: Update for documentation of Windows plugins
* Add Since marks
* Make use of GST_PARAM_CONDITIONALLY_AVAILABLE flag
2020-07-02 17:21:29 +02:00
Seungha Yang
76793ffabc nvcodec: Update for documentation
* Add Since marks
* Make use of GST_PARAM_CONDITIONALLY_AVAILABLE flag
* Add documentation template caps
2020-07-02 17:21:24 +02:00
Seungha Yang
73ddf27898 dxgiscreencapsrc: Add missing debug category init
This should've been addressed in the commit 9042062072
but I was missed it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1383>
2020-06-30 02:51:14 +09:00
Víctor Manuel Jáquez Leal
79d11c2022 va: VA-API H.264 decoder and infrastructure
New plugin with an element for H.264 decoding with VA-API. This novel
approach, different from gstreamer-vaapi, uses gstcodecs library for
state handling.

The code is expected to looks cleaner because it uses VA-API without
further layers or wrappers.

* It uses the first supported DRM device as default VA display (other
  displays will be supported through user's GstContext)
* Requires libva >= 1.6
* No multiview/stereo profiles neither interlaced streams because
  gstcodecs doesn't handle them yet
* It is incompatible with gstreamer-vaapi
* Even if memory:VAMemory is exposed, it is not handled yet by any
  other element
* Caps templates are generated dynamically querying VAAPI, but YV12
  and I420 are added for system memory caps because they seem to be
  supported for all the drivers when downloading frames onto main
  memory, as they are used by xvimagesink and others, avoiding color
  conversion.
* Surfaces aren't bounded to context, so they can grow beyond the DBP
  size, allowing smooth reverse playback.
* There isn't yet error handling and recovery.
* 10-bit H.264 streams aren't supported by libva.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1379>
2020-06-28 11:47:35 +02:00
Seungha Yang
117f593cff d3d11h265dec: Fix possible invalid memory access
The number of element to copy should be num_tile_columns_minus1
and num_tile_rows_minus1.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1374>
2020-06-25 13:58:57 +00:00
Seungha Yang
48ca7c7e93 nvh265sldec: Fix possible invalid memory access
Fix Coverity issues.

CID 1464959, 1464960, 1464961, 1464962

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1374>
2020-06-25 13:58:57 +00:00
Seungha Yang
82189d6859 d3d11videosink: Disable d3d11videosink depending on supported feature level
Current shader code is not compatible with HLSL profile "ps_4_0_level_9_3"
or lower. So d3dcompiler cannot compile our shader code in that case.
Note that VirtualBox is one known driver which doesn't support currently
implemented shader code.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1343>
2020-06-25 07:55:47 +00:00
Kevin King
eebfed0726 avfvideosrc: wait for permissions request dialog callback
otherwise gstreamer gives up on transitioning the pipeline before the
user has accepted

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1370>
2020-06-24 18:56:53 +00:00
Seungha Yang
133b902044 nvh264sldec: Remove useless double space
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1357>
2020-06-24 16:25:58 +00:00
Seungha Yang
711e964572 nvcodec: Add H265 stateless codec implementation
Add a new GstCodecs based H265 decoder element

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1357>
2020-06-24 16:25:58 +00:00
Seungha Yang
1e544f741a nvcodec: Move common methods to nvdecoder
... and remove all #ifdef from nvh264sldec implementation.
New helper methods will take care of OpenGL specific ones.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1357>
2020-06-24 16:25:58 +00:00
Seungha Yang
1f452c0577 nvh264sldec: Declare OpenGL related objects unconditionally
GstGLDisplay and GstGLContext are subclass of GstObject so we can
remove #ifdef for such object. This is prework for nvh265sldec.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1357>
2020-06-24 16:25:58 +00:00
Seungha Yang
82ddb0600b nvh264sldec: Fix for possible wrong device selction
decoder should select assigned CUDA device id

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1357>
2020-06-24 16:25:58 +00:00
Mathieu Duponchelle
ad49ae42f7 docs: mark more types as plugin API 2020-06-23 12:10:19 -04:00
Seungha Yang
863c663b1d d3d11window_win32: Chain up mouse event to parent window
Unlike key event, mouse event will not be chained up to parent window
by DefWindowProc

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1361>
2020-06-22 15:09:22 +00:00
Seungha Yang
9042062072 dxgiscreencapsrc: Load HLSL compiler library using g_module_open
Depending on OS version, available d3dcompiler library name is different.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1355>
2020-06-22 14:36:16 +00:00
Seungha Yang
96a5760751 dxgiscreencapsrc: Reorganize debug category usage
Use the same debug category in all dxgiscreencap related source code

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1355>
2020-06-22 14:36:16 +00:00
Seungha Yang
4aa7d3d589 winscreencap: Remove trailing whitespaces
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1355>
2020-06-22 14:36:16 +00:00
Seungha Yang
3dd2b6f20b d3d11device: Print supported DXGI format for debugging
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1355>
2020-06-22 14:36:16 +00:00
Seungha Yang
8fe1aae96c d3d11: Load HLSL compiler library using g_module_open
Depending on OS version, available d3dcompiler library name is different.
But for UWP, we can still use the current way

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1355>
2020-06-22 14:36:16 +00:00
Seungha Yang
9b8eec39b6 d3d11shader: Add missing config.h include
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1355>
2020-06-22 14:36:16 +00:00
Seungha Yang
4997cde699 mediafoundation: Add VP9 encoder element
Some Intel GPUs support hardware accelerated VP9 encoding and
Microsoft provides software VP9 encoding implementation as well.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1295>
2020-06-22 07:58:03 +00:00
Seungha Yang
1a68da54b6 mfvideosrc: Add support for jpeg on Win32 application
Enable reading jpeg data from webcam if it's supported.
Note that this would be enabled only for Win32.
For UWP, we need to research more about how to support jpeg.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1342>
2020-06-17 17:45:32 +09:00
Seungha Yang
f508c8b988 mfvideosrc: Fix wrong casting
Don't cast ISoftwareBitmap to IMFMediaBuffer

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1342>
2020-06-17 17:37:05 +09:00
Seungha Yang
86e3df9493 mfvideosrc: Add support YUY2 format for UWP
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1342>
2020-06-17 17:37:05 +09:00
Seungha Yang
8d7f537782 mediafoundation: Drop IMFCaptureEngine implementation
It was introduced for later use of its enhanced feature over IMFSourceReader
such as taking photo with video preview, audio/video capturing at
the same time, etc. But currently it's not our use case, and it would
be maintenance burden.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1342>
2020-06-17 17:37:05 +09:00
Seungha Yang
f899728dd4 d3d11: Don't assume response of context query has valid d3d11 device context
Peer elements should return FALSE if d3d11 device context is unavailable
but it might happen for some reason (e.g., wrong implementation or so)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1347>
2020-06-16 11:23:34 +00:00
Seungha Yang
bc99ea1e30 d3d11decoder: Disable zero-copy for blacklisted device
Should enable it for verified devices. For now, Xbox is blacklisted

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1339>
2020-06-16 10:49:02 +00:00
Seungha Yang
1144cdc63a d3d11decoder: Adjust alignment constraint for Xbox device
XBox doesn't seem to support 128 bytes alignment for 4K HEVC

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1339>
2020-06-16 10:49:02 +00:00
Seungha Yang
fe361ecc25 d3d11window: Do not configure video processor for Xbox device
Disable video processor for Xbox until it's verified

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1339>
2020-06-16 10:49:02 +00:00
Seungha Yang
d12c0c4c7b d3d11utils: Add a helper method for checking Xbox device
Required for some cases to work around device specific issue

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1339>
2020-06-16 10:49:02 +00:00
Seungha Yang
b5c8889d4f d3d11videosink: Use GPU memory copy if possible
Even if fallback buffer is required (e.g., shader resource view is unavailable),
use direct GPU memory copy if possible. It must be much faster than
system memory copy approach.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1339>
2020-06-16 10:49:02 +00:00
Seungha Yang
73b39407a6 d3d11videosink: Ensure shader resource view of fallback buffer
SRV must be configured for color conversion

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1339>
2020-06-16 10:49:02 +00:00
Seungha Yang
4471788e4a d3d11window: Fix typo "configureed"
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1339>
2020-06-16 10:49:02 +00:00
Jun-ichi OKADA
d43c5b6485 winscreencap: Add dxgiscreencapsrc element.
This element uses the Desktop Duplication API to capture the desktop screen at high speed.
It supports Windows 8 or later.

It has the following features compared to other elements:
 * Runs faster.
 * It works in High DPI environment.
 * Draws an accurate mouse cursor.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/863>
2020-06-16 10:30:13 +09:00
Xu Guangxin
610e477565 msdkdec: hold a reference for the surfaces locked by msdk
previous code releases GstBuffer too earlier. so we will see

ERROR                default gstmsdkvideomemory.c:77:gst_msdk_video_allocator_get_surface: failed to get surface available
ERROR         msdkbufferpool gstmsdkbufferpool.c:270:gst_msdk_buffer_pool_alloc_buffer:<msdkbufferpool0> failed to create new MSDK memory

We need to hold GstBuffer reference for msdk if the surfaced locked by msdk.

step to reproduce.
1. ffmpeg -f lavfi -i testsrc=duration=10:size=320x240:rate=30 -pix_fmt yuv420p -c:v libx265 test.265
2. GST_GL_PLATFORM=egl  gst-launch-1.0 -v filesrc location=test.265  ! h265parse ! msdkh265dec  ! queue ! glimagesink

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1277>
2020-06-15 02:46:53 +00:00
Seungha Yang
2e1bec7ff6 wasapi2: Fallback to IAudioClient interface if IAudioClient3 API is unavailable
When default device is selected, IAudioClient3 API doesn't look like
available.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1332>
2020-06-11 12:40:44 +00:00
Seungha Yang
7ab51e85ab wasapi: Fix possible deadlock while downwards state change
IAudioClient::Stop() doesn't seem to wake up the event handle,
then read() or write() could be blocked forever by WaitForSingleObject.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1329>
2020-06-11 11:40:26 +00:00
Nicolas Dufresne
483539f45c v4l2slh264dec: Fix reading mode and start code type
These two controls are not pointer based, so we don't need to pass any size or
pointer and need to copy the values afterward. This fixes H264 decoding
regression with Hantro and RKVDEC drivers.

Fixes 037730a787

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1330>
2020-06-09 10:52:44 -04:00
Seungha Yang
52a82f5b84 mftransform: Fix deadlock when MFT requests processing output twice
This sequence of event/data flow might happen

1) Initially we have one pending output event
  1-1) Then, process the pending output data
2) No pending input event, then we should wait new pending input event
  2-1) Wakeup by new pending event (but it's pending output event)

In above case, MFT will not report new pending input event
if pending output is not processed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1325>
2020-06-08 23:46:43 +09:00
Seungha Yang
309c679c43 mftransform: Add some debug log
Add some trace level log for debugging

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1325>
2020-06-08 23:42:22 +09:00
Seungha Yang
a6a4267c7b wasapi2: Add device provider implementation
Similar to device provider implementation of wasapi plugin,
this implementation supports only static probing.
But we can implement runtime device add/remove/update
monitoring using DeviceWatcher interface later.

See https://docs.microsoft.com/en-us/uwp/api/windows.devices.enumeration.devicewatcher

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1264>
2020-06-08 03:10:05 +00:00
Seungha Yang
2778457678 wasapi2: Introduce new WASAPI plugin
Add a new wasapi implementation mainly to support UWP application.
Basically the core logic of this plugin is almost identical to
existing wasapi plugin, but main target is Windows 10 (+ UWP).
Since this plugin uses WinRT APIs, this plugin most likely might not
work Windows 8 or lower.

Compared with existing wasapi plugin, additional features of this plugin are
* Fully compatible with both Windows 10 desktop and UWP application
* Supports automatic stream routing (auto fallback when device was removed)
* Support device level mute/volume control

But some features of existing wasapi plugin are not implemented
in this plugin yet
* Exclusive streaming mode is not supported
* Loopback feature is not implemented
* Cross-compile is not possible with current mingw toolchain
  (meaning that MSVC and Windows 10 SDK are required to build this plugin)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1264>
2020-06-08 03:10:05 +00:00
Seungha Yang
0f74785b8e mfvideoenc: Set PAR to output IMFMediaType
We've set it to input IMFMediaType but not for output.
So, if PAR is not 1:1, the input IMFMediaType will be accepted
by MFT (default is 1:1).
The PAR of input/output IMFMediaType must be identical

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1322>
2020-06-06 21:19:48 +09:00
Seungha Yang
cec6858401 mftransform: Don't try to drain if MFT is not running
Otherwise MFT will be blocked forever as no event can be generated by
IMFMediaEventGenerator.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1322>
2020-06-06 21:03:05 +09:00
Mathieu Duponchelle
a048ce81d4 plugins: uddate gst_type_mark_as_plugin_api() calls 2020-06-06 00:40:42 +02:00
Seungha Yang
a38ac7a865 d3d11overlaycompositor: Fix wrong Y position calculation
The Y coordinate of vertex and screen/image are opposite

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1319>
2020-06-05 13:47:49 +00:00
Sebastian Dröge
74f2f733be plugins: Use gst_type_mark_as_plugin_api() for all non-element plugin types 2020-06-04 13:33:16 -04:00
Seungha Yang
c4a2fd7683 d3dvideosink: Use secondary rank
d3dvideosink will be replaced by d3d11videosink

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1311>
2020-06-03 17:57:40 +09:00
Seungha Yang
116d7453cb d3d11videosink: Assign primary rank
d3d11videosink has an advantage over d3dvideosink, such as
* Zero-copy playback with d3d11 decoders
* HDR rendering with 10-bit format/swapchain support
* UWP support
* Any system memory alignment/padding can be supported
* User can select target GPU device
And old d3dvideosink's functionality (e.g., navigation event, overlaycomposition)
can be covered by d3d11videosink

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1311>
2020-06-03 17:57:41 +09:00
Haihao Xiang
f685893ed2 msdkh265enc: add support 12-bit 420 encoding
P016 is used for 12-bit encoding in MediaSDK, so the Shift flag is set
in the mfx parameters

Sample pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=P012_LE ! msdkh265enc ! \
fakesink

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1174>
2020-06-01 05:04:30 +00:00
Xu Guangxin
52be289847 msdkvpp: fix "failed to create new MSDK memory"
all msdk output surfaces come from out_alloc_resp, so the buffer count is not resizable.
we need set min_buffers, max_buffers to same size.

steps to reproduce
1. ffmpeg -f lavfi -i testsrc=duration=10:size=320x240:rate=30:decimals=3 -pix_fmt yuv420p -c:v libx265 ~/bits/hevc/test.265
2. GST_GL_PLATFORM=egl gst-launch-1.0 -v filesrc location=~/bits/hevc/test.265  ! h265parse ! msdkh265dec  ! msdkvpp ! queue ! glimagesink
you will see error like this:
ERROR                default gstmsdkvideomemory.c:77:gst_msdk_video_allocator_get_surface: failed to get surface available
ERROR         msdkbufferpool gstmsdkbufferpool.c:270:gst_msdk_buffer_pool_alloc_buffer:<msdkbufferpool2> failed to create new MSDK memory
ERROR                msdkvpp gstmsdkvpp.c:297:create_output_buffer:<msdkvpp0> failed to create output video buffer
ERROR                msdkdec gstmsdkdec.c:699:gst_msdkdec_finish_task:<msdkh265dec0> Failed to finish frame
ERROR                msdkdec gstmsdkdec.c:1085:gst_msdkdec_handle_frame:<msdkh265dec0> Failed to finish a task

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1278>
2020-06-01 02:09:04 +00:00
Xu Guangxin
275518f661 msdkvpp: hold GstBuffer ref count for locked surfaces
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1278>
2020-06-01 02:09:04 +00:00
Seungha Yang
12532cc8bb mediafoundation: Use core dispatcher of current view instead of main view
Main view might be hidden depending on application's view tree.
In that case, ICoreApplication object doesn't return get_MainView() method

Note that nothing about this behavior was documented by Microsoft
https://docs.microsoft.com/en-us/uwp/api/windows.applicationmodel.core.coreapplication.mainview

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1310>
2020-05-29 23:01:08 +09:00
Seungha Yang
2dc689c2c8 mediafoundation: Fix undeclared identifier error on UWP build
Some symbols are not available in case of UWP

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1306>
2020-05-28 12:04:43 +00:00
Seungha Yang
7b5f99b47a mediafoundation: Add support MP3 audio encoding
Add MediaFoundation MP3 encoder

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1280>
2020-05-27 10:34:47 +00:00
Seungha Yang
cee619486a mediafoundation: Add support for AAC encoding
Add MediaFoundation AAC encoder element.

Before Windows 10, mono and stereo channels were supported audio channels
configuration by AAC encoder MFT. However, on Windows 10,
5.1 channels support was introduced.

To expose correct range of support format by this element
whatever the OS version is, this element will enumerate
all the supported format by the AAC encoder MFT
and then will configure sink/src templates while plugin init.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1280>
2020-05-27 10:34:47 +00:00
Seungha Yang
18f5bdee16 mfutils: Move IMediaType release function to common utility
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1280>
2020-05-27 10:34:47 +00:00
Seungha Yang
c8469644dd mediafoundation: Add util function to dump IMFAttributes values
It would be useful for debugging.

Reference: https://docs.microsoft.com/en-us/windows/win32/medfound/media-type-debugging-code
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1280>
2020-05-27 10:34:47 +00:00
Seungha Yang
bd8a415747 d3d11window_win32: Create internal window on parent window's thread
If parent and child windows are running on different thread,
there is always a chance to cause deadlock as DefWindowProc() call
from child window thread might be blocked until the message
is handled by parent's window procedure.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1299>
2020-05-27 05:08:48 +09:00
Seungha Yang
afe941249e d3d11window: Add unprepare method to clear internal resource
GObject::dispose method can be called multiple times. As win32 d3d11window
has an internal thread and because GObject::dispose method could be called from the
thread, it might cause problems such as trying to join self-thread

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1299>
2020-05-27 04:59:50 +09:00
Seungha Yang
8ce4980273 mfsourceobject: Remove useless null check for string
We can pass null for the value of string type property.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1241>
2020-05-25 15:09:20 +00:00
Seungha Yang
50b36ce257 mediafoundation: Use G_BEGIN_DECLS/G_END_DECLS pair everywhere
... instead of extern "c" {} block.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1241>
2020-05-25 15:09:20 +00:00
Seungha Yang
c4cb51c63d mediafoundation: Add support video capture on UWP app
New video capture implementation using WinRT Media APIs for UWP app.
Due to the strict permission policy of UWP, device enumeration and
open should be done via new WinRT APIs and to get permission from users,
it will invoke permission dialog on UI.
Strictly saying, this implementation is not a part of MediaFoundation
but structurally it's very similar to MediaFoundation API.
So we can avoid some code duplication by adding this implementation
into MediaFoundation plugin.

This implementation requires UniversalApiContract version >= 6.0
which is part of Windows 10 version 1803 (Redstone 4)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1241>
2020-05-25 15:09:20 +00:00
Seungha Yang
e580676747 mfsourceobject: Move device name, path, and index to public struct
... so that subclass can access each value and update them.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1241>
2020-05-25 15:09:20 +00:00
Seungha Yang
6ae478946c mediafoundation: Fix typo in source object impl.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1241>
2020-05-25 15:09:20 +00:00
Seungha Yang
3efc8f5de9 mftransform: Clear unused output IMediaSample
If MFT doesn't produce encoded output, need to free allocated
output sample and buffer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1297>
2020-05-25 10:25:37 +00:00
Seungha Yang
b93fbdd3cd mfvideoenc: Fix huge memory leak
Subclass must unref passed GstVideoCodecFrame on GstVideoEncoder::handle_frame()

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1294>
2020-05-24 19:12:28 +09:00
Seungha Yang
e75049eb84 nvh264sldec: Fix wrong scaling list matrix scan order
Quatization matrix of NVDEC should be raster scan order but
h264parser stores it in zig-zag scan order. We need to convert
the matrix.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1290>
2020-05-23 01:06:28 +09:00
Nicolas Dufresne
d16367f046 v4l2codecs: h264: Add missing break
There was a missing break for the 4:4:4 case which would break the sizeimage
calculation. We don't currently have hardware that supports 4:4:4, so this
code wasn't tested. This was detected by Coverity.

CID 1463592 1463591

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1283>
2020-05-21 20:12:07 +00:00
Matthew Waters
8403d97587 amc/videodec: only retrieve the stride/slice-height for raw output
When outputting to a surface, these values may not exist.

As found on a Google Pixel 3.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1284>
2020-05-21 12:40:21 +00:00
Seungha Yang
dac36cb98f mediafoundation: Add device provider implementation
Only static device probing is supported for now

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1236>
2020-05-20 11:06:09 +00:00
Seungha Yang
ba45e3f82b mfsourceobject: Store selected device path, name and index
Update path, name and index with selected device so that checked by
get_property() after constructed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1236>
2020-05-20 11:06:09 +00:00
Nicolas Dufresne
2160239548 v4l2slh264dec: Request large enough bitstream buffer
The Cedrus driver would otherwise choose 1KB buffer, which is too small.
This follows what some drivers do, which is simply to use the size a
packed raw image would have. Specifications do not really guaranty any minimum
compression ratio.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1268>
2020-05-19 17:51:11 +00:00
Nicolas Dufresne
037730a787 v4l2slh264dec: Add slice based decoder support
This adds support for slice based decoder like the Allwinner/Cedrus driver. In
order to keep things efficient, we hold the sink buffer until we reach the end
of the picture. Note that as we don't know which one is last, we lazy queue the
slices. This effectively introduces one slice latency.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1268>
2020-05-19 17:51:11 +00:00
Nicolas Dufresne
661f0c161e v4l2codecdec: Fix error handling
If none of the format the HW produce is supported, the fiter will be NULL,
which would lead to assertion when trying to release it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1268>
2020-05-19 17:51:11 +00:00
Nicolas Dufresne
26264163de v4l2decoder: Add legacy non-multiplanar support
The Cedrus driver uses the lagacy buffer type (non-mplane). This automatically
detect and use the right v4l2_buf_type. This also affect code using
v4l2_buffer and v4l2_format structures.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1268>
2020-05-19 17:51:11 +00:00
Nicolas Dufresne
14b0b68354 v4l2codecs: Update kernel headers
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1268>
2020-05-19 17:51:11 +00:00
Seungha Yang
24f30b2e6e d3d11convert: Fix fallback texture setup when resolution is not even number
When texture format is semi-planar, resolution should be even number,
and add missing P016 format handling

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1275>
2020-05-19 17:25:05 +00:00
Seungha Yang
a966cc20bc d3d11convert: Fix fallback texture copy
Fix texture copy when input texture has non-zero subresource index

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1275>
2020-05-19 17:25:05 +00:00
Seungha Yang
a24d4e654d d3d11: Add support for video rescale and rename element to d3d11convert
GstD3D11ColorConverter implementation is able to rescale video as well.
By doing colorspace conversion and rescale at once, we can save
one cycle of shader pipeline which will can save GPU resource.

Since this element can support color space conversion and rescale,
it's renamed as d3d11convert

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1275>
2020-05-19 17:25:05 +00:00
Seungha Yang
0e842c74b6 d3d11: Move scoring util method for colorspace conversion to colorconvert element
It's used only by colorconvert element.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1275>
2020-05-19 17:25:05 +00:00
Víctor Manuel Jáquez Leal
d2aae6bb96 codecs: h264decoder: ref pic lists as decode_slice parameters
Pass reference picture lists to decode_slice() vmethods

Change gstv4l2codech264dec and gstnvh264dec accordingly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1238>
2020-05-19 16:57:09 +00:00
Arun Raghavan
d0776faf47 opensles: Remove hard-coded buffer-/latency-time values
These were originally required in early Android versions, but are no
longer needed.
2020-05-16 19:23:06 +00:00
Seungha Yang
8afc283b51 mediafoundation: Refactor GstMFSourceObject implementation
* Move CoInitializeEx/CoUninitialize pair into thread function in order to
  ensure MTA COM thread
* Move common code to baseclass

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1269>
2020-05-15 09:03:13 +00:00
Seungha Yang
c29c71ae9d mediafoundation: Remove COM thread constraints from GstMFTransform object
Move CoInitializeEx/CoUninitialize pair into our dedicated thread so that
we can ensure COM thread is MTA. This will remove thread constraints
around plugin init.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1269>
2020-05-15 09:03:13 +00:00
Matthew Waters
b3cbdb1d26 amcvideodec: fix sync meta copying not taking a reference
Fixup for
9b9e39be24: amc: Fix crash when a sync_meta survives its sink
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/603

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1272>
2020-05-15 13:54:56 +10:00
Ting-Wei Lan
d10e2381c7 v4l2codecs: Fix compilation error on FreeBSD
This commit does the following things to fix compilation on FreeBSD:

1. Add required typedefs to linux/types-compat.h.
2. Remove unnecessary include linux/ioctl.h and replace linux/types.h
   with linux/types-compat.h. Both files do not exist on FreeBSD.
3. Check the header including makedev macro. FreeBSD does not have
   sys/sysmacros.h, and including it unconditionally causes error.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1259>
2020-05-11 15:17:31 +00:00
Haihao Xiang
98600942e4 msdkvpp: clear the parameters after closing the session
Otherwise the stale values are used for the new process.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1159>
2020-05-11 02:16:44 +00:00
Xu Guangxin
cab1ddf48b msdk: bufferpool: set alignment to video meta
else gst_video_meta_validate_alignment will report error like
"videometa gstvideometa.c:416:gst_video_meta_validate_alignment: Stride of plane 0 defined in meta (384) is different from the one computed from the alignment (320)"

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1224>
2020-05-07 05:45:36 +00:00
Nicolas Dufresne
5ece17ecbd v4l2slvp8dec: Flip the meaning of segment_feature_mode
In section 9.3.4 a), segment_feature_mode have 0 for absolute and 1 for delta,
while in 19.2, it says the opposite. But the reference code, which usually
rules over the text state that 1 means absolute:

 if (hdr->update_data)
 {
     hdr->abs = bool_get_bit(bool);

And uses it with that meaning to decide weither to override the existing value
or just add the detla. This fixes multiple decoding issues.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1245>
2020-05-05 17:57:08 +00:00
Nicolas Dufresne
11c62eb015 v4l2slvp8dec: Copy header version
This field was not copied.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1245>
2020-05-05 17:57:08 +00:00
Nicolas Dufresne
8c56822936 v4l2slvp8dec: Add debugging for reference frames
This simply trace the frame number of the references used for decoding.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1245>
2020-05-05 17:57:08 +00:00
Nicolas Dufresne
a449815f95 v4l2slvp8dec: Ensure width/height is always set
Our parser strictly read the bitstream. As it's known from DXVA that always
having a valid width/height might be needed, use the cached width/height
instead of the value from the parser. This didn't impact Hantro driver.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1245>
2020-05-05 17:57:08 +00:00
Nicolas Dufresne
ffd74fb5ec v4l2slvp8dec: Fix debug category name
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1245>
2020-05-05 17:57:08 +00:00
Seungha Yang
d9c552802e d3d11vp8dec: Remove useless assign operation
It's already been done in the line just above.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1246>
2020-05-05 16:37:14 +00:00
Seungha Yang
a4fbb49ef3 d3d11window: Fix processor output view resource leak
Because this object holds reference to native d3d11 device handle internally,
related native d3d11 objects would be leaked as well.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1243>
2020-05-05 20:49:32 +09:00
Tim-Philipp Müller
9539eecf20 nvcodec: fix compiler warning in certain setups
Fixes gstnvh264dec.c:648:12: warning: unused variable ‘ret’
compiler warning on raspbian where HAVE_NVCODEC_GST_GL is
not defined.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1239>
2020-05-03 11:08:29 +01:00
Seungha Yang
ed147a3b48 d3d11h264dec: Rearrange the order of supported profiles in caps
constrained-baseline profile is much preferred than baseline.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1229>
2020-04-29 18:13:42 +00:00
Seungha Yang
28dc4a1dd8 nvh264sldec: Allow constrained-baseline and baseline profiles
Like d3d11h264dec, we need to relax the condition for profiles of nvh264sldec

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1228>
2020-04-30 02:36:34 +09:00
Seungha Yang
7db29fc382 d3d11h264dec: Add constrained-baseline and baseline profile
Both profiles are expected to be supported profile by DXVA hardware

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1212>
2020-04-29 14:24:02 +00:00
Nicolas Dufresne
bf9f82c8c3 v4l2codecs: Factor out plugin registration
This introduce a common place for generic functions and factor out the plugin
registration code. This code is nearly identical between implementation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1216>
2020-04-28 22:53:11 +00:00
Nicolas Dufresne
de2cccaa47 v4l2decoder: Relax width/height return value check
The driver adjust the width/height to coded size. This was not an issue for
H264, as the coded size is in the bitstream, but is an issue with VP8.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1216>
2020-04-28 22:53:11 +00:00
Nicolas Dufresne
72c0b3e5ae v4l2codecs: Add VP8 decoder
This is derived from the H264 decoder, some boiler plate will be factored out
in the following commits.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1216>
2020-04-28 22:53:11 +00:00
Joshua M. Doe
bfc605a666 mfvideosrc: add GRAY16_LE format support
FOURCC "Y16 " is used by FLIR Boson USB Video Class (UVC) camera

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1222>
2020-04-28 15:14:50 -04:00
Seungha Yang
3804dfb28d d3d11device: Add fallback for device creation
D3D11_CREATE_DEVICE_DEBUG flag will be used while creating d3d11 device
to activate debug layer. However, if system doesn't support the
debug layer for some reason, we should try to create d3d11 device
without the flag. Debug layer should be optional for device creation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1217>
2020-04-28 17:33:56 +00:00
Seungha Yang
912b2d7af9 mediafoundation: Disable querying supprted maximum resolution
Initializing MFT for checking supported maximum resolution is too
slow.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/760>
2020-04-28 14:37:31 +00:00
Seungha Yang
565817abd8 mediafoundation: Add h265 encoder
Add Media Foundation HEVC encoder

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/760>
2020-04-28 14:37:31 +00:00
Seungha Yang
9625d19279 mediafoundation: Add h264 encoder
Add Media Foundation h264 encoder. If hardware encoders are available
on system, they will have higher rank than software encoder.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/760>
2020-04-28 14:37:31 +00:00
Seungha Yang
eece89042a mediafoundation: Introduce Microsoft Media Foundation plugin
The Microsoft Media Foundation (MF) is the successor of DirectShow.
This commit includes two kinds of video capture implementation,
one uses IMFSourceReader interface which is available since Windows Vista
and the other is based on IMFCaptureEngine interface which is available
since Windows 8.
Note that this new video source element cannot be used in UWP app
for now, since device activation using those APIs are not allowed by MS.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/760>
2020-04-28 14:37:31 +00:00
Nicolas Dufresne
ed6236f551 d3d11vp8dec: Remove unused parameters
Minor cleanup removing unused picture parameters in two internal functions.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1221>
2020-04-28 06:21:53 +00:00
Seungha Yang
4d585c64d1 nvcodec: Add suppport for environment based primary h264 decoder implementation
Introduce GST_USE_NV_STATELESS_CODEC environment to allow user to select
primary nvidia decoder implementation. In case the environment
GST_USE_NV_STATELESS_CODEC=h264 was set, old nvidia h264 decoder element
will not be registered. Instead, both nvh264dec and nvh264sldec
factory name will create gstcodecs based new nvidia h264 decoder element.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1198>
2020-04-24 09:23:10 +00:00
Seungha Yang
9c21923f04 nvcodec: Add H264 stateless codec implementation
Introduce GstH264Decoder based Nvidia H.264 decoder element.
Similar the element factory name of to v4l2 stateless codec,
this element can be configured with factory name "gstnvh264sldec".
Note that "sl" in the name stands for "stateless"

For now, existing nvh264dec covers more profile and formats
(e.g., interlaced stream) than this implementation.
However, this implementation allows us to control lower level
parameters such as decoded picture buffer management and therefore
we can get a chance to improve performance in terms of latency.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1198>
2020-04-24 09:23:10 +00:00
Seungha Yang
ac850fd25d v4l2slh264dec: Support avc and avc3 formats
h264decoder baseclass provides parsed sps/pps (from codec data) for
subclass. Also, since current implementation is putting start code prefix
manually, the format of stream should not be matter for subclass.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1214>
2020-04-23 23:38:03 +00:00
Seungha Yang
2fe473fa50 d3d11decoder: Fix wrong return type from output_picture vfunc
Return type should be GstFlowReturn, not gboolean

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1213>
2020-04-23 22:06:24 +00:00
Seungha Yang
15365aba70 d3d11decoder: Add VP8 decoder element
New d3d11 VP8 decoder implementation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1079>
2020-04-22 23:20:44 +00:00
Seungha Yang
5965b6ef85 d3d11decoder: Add helper methods for negotiation and decide_allocation
The implementation for all codecs is almost the same.
No need to duplicate code.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1079>
2020-04-22 23:20:44 +00:00
Seungha Yang
09a8d2db05 kmssink: Add support for P010 and P016 formats
Support high bit-depth YUV formats

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1188>
2020-04-22 22:39:04 +00:00
Seungha Yang
c5abdf8a63 d3d11window: Fix videoprocessor leak
Clear all objects in dispose function
2020-04-21 15:32:30 +09:00
Seungha Yang
64a015d51f d3d11device: Drop floating reference count
... and use gst_object_unref instead of g_object_unref for tracer
to be happy.
2020-04-21 15:32:26 +09:00
Víctor Manuel Jáquez Leal
e96435dfea v4l2codecs: allocator: use gst_clear_object
So tracer can do its job.
2020-04-20 12:23:32 +00:00
Stéphane Cerveau
040077dbc8 kmssink: fix memory leak on failing allowed caps
When drm does not find any format, the allowed_caps is
empty one and need to be unref by the caller.

Fix #1268
2020-04-16 16:10:56 +02:00
Nicolas Dufresne
63940feaaf kmssink: Save last metadata at the same time as the last buffer
The render width/height and the vinfo was only saved upon renegotiation. This
fixes the problem by saving this metadata at the same time the buffer is
saved. The saved copy of this is needed for expose() and drain() virtual functions.
This fixes various assertion that happens on drain query.
2020-04-15 16:26:23 +00:00
Víctor Manuel Jáquez Leal
691fdb85c0 v4l2codecs: fix v4l2codecdevice get type
Currently the GType of v4l2codecdevice is hardcoded to zero, but it
rather should be delivered by the GType system.
2020-04-10 21:05:19 +02:00
Jordan Petridis
87fc038f67
gstmsdkdec: fix logical operation that misses parenthesis
in C, & is weaker than the ! operator and clang is giving the following
error about it.

```
../subprojects/gst-plugins-bad/sys/msdk/gstmsdkdec.c:731:7: error: logical not is only applied to the left hand side of this bitwise operator [-Werror,-Wlogical-not-parentheses]
  if (!gst_msdk_context_get_job_type (thiz->context) & GST_MSDK_JOB_DECODER) {
      ^                                              ~
../subprojects/gst-plugins-bad/sys/msdk/gstmsdkdec.c:731:7: note: add parentheses after the '!' to evaluate the bitwise operator first
  if (!gst_msdk_context_get_job_type (thiz->context) & GST_MSDK_JOB_DECODER) {
      ^
       (                                                                   )
../subprojects/gst-plugins-bad/sys/msdk/gstmsdkdec.c:731:7: note: add parentheses around left hand side expression to silence this warning
  if (!gst_msdk_context_get_job_type (thiz->context) & GST_MSDK_JOB_DECODER) {
      ^
      (                                             )
1 error generated.
```
2020-04-10 18:22:21 +03:00
Seungha Yang
430848548d meson: kmssink: Allow fallback only on linux
Otherwise fallback will waste meson configure time on non-linux
2020-04-09 18:36:12 +00:00
Seungha Yang
b4efdeba11 nvdec: Don't hardcode DPB size
Too many decode surface would waste GPU memory. Also it seems to be
introducing additional latency depending on stream. Since nvcodec
sdk version 9.0, CUVID parser API has been providing the minimum
required number of surface. By using it, we can save GPU memory
and reduce possible latency.
2020-04-09 16:30:58 +09:00
Nicolas Dufresne
8f0ceba251 v4l2codecs: Fix FD leak during device enumeration
This was revealed by Coverity.

CID 1461248
2020-04-08 20:25:44 +00:00
Nicolas Dufresne
f5da12bcf5 v4l2decoder: Fix file descriptor leak
A copy paste error was leading to file descriptor leak. This was detected by
Coverity.

CID 1461285
2020-04-07 17:10:08 -04:00
Haihao Xiang
00baed3ebe msdkvp9dec: add support for VP9 12bit
The output formats are P012_LE for 12bit 420 and Y412_LE for 12bit 444.
2020-04-07 00:39:35 +00:00
Nirbheek Chauhan
387b6df948 meson: Don't use get_option('buildtype')
We should directly check the values of the `debug` and `optimization`
options instead.

`get_option('buildtype')` will return `'custom'` for most combinations
of `-Doptimization` and `-Ddebug`, but those two will always be set
correctly if only `-Dbuildtype` is set. So we should look at those
options directly.

For the two-way mapping between `buildtype` and `optimization`
+ `debug`, see this table:
https://mesonbuild.com/Builtin-options.html#build-type-options
2020-04-03 17:07:47 +05:30
Stéphane Cerveau
1f62fe1cbc kms: add fallback on libdrm project if available. 2020-04-03 10:53:01 +00:00
Haihao Xiang
4cc5321010 msdkh265dec: add support for main-444-12, main-444-12-intra profiles
The video format is Y412_LE
2020-04-03 01:19:45 +00:00
Haihao Xiang
4d756c07b0 msdk: map Y412_LE to VA_FOURCC_Y416
In media driver, VA_FOURCC_Y416 is used for packed 12 bits 4:4:4:4 YUV
format, the corresponding RT format is VA_RT_FORMAT_YUV442_12
2020-04-03 01:19:45 +00:00
Haihao Xiang
55ed9458b5 msdk: map MFX_FOURCC_Y416 to VA_FOURCC_Y416
Y416 is used for packed 12 bits 4:4:4:4 YUV format in media driver, the
RT format is VA_RT_FORMAT_YUV444_12
2020-04-03 01:19:45 +00:00
Haihao Xiang
e769df9689 msdk: map Y412_LE to MFX_FOURCC_Y416
MFX_FOURCC_Y416 is used for packed 12 bits 4:4:4:4 YUV format in
MediaSDK
2020-04-03 01:19:45 +00:00
Haihao Xiang
3de690ceda msdkh265dec: add support for main-422-12, main-422-12-intra profiles
The video format is Y212_LE
2020-04-03 01:19:45 +00:00
Haihao Xiang
01edef09a3 msdk: map Y212_LE to VA_FOURCC_Y216
In media driver, VA_FOURCC_Y216 is used for packed 12 bits 4:2:2 YUV
format, the corresponding RT format is VA_RT_FORMAT_YUV422_12
2020-04-03 01:19:45 +00:00
Haihao Xiang
34177985e2 msdk: map MFX_FOURCC_Y216 to VA_FOURCC_Y216
In media driver, Y216 is used for packed 12 bits 4:2:2 format YUV
format, so the RT format is VA_RT_FORMAT_YUV422_12.
2020-04-03 01:19:45 +00:00
Haihao Xiang
fb200ccfdd msdk: map Y212_LE to MFX_FOURCC_Y216
MFX_FOURCC_Y216 is used for packed 12 bits 422 YUV format in MediaSDK
2020-04-03 01:19:45 +00:00
Seungha Yang
950aa3d1db msdkh265enc: Fix for wrong parser free function 2020-04-03 00:42:23 +09:00
Seungha Yang
71cf93c361 msdkh264enc: Configure parser and SEI array only if it's required 2020-04-02 09:20:11 +00:00
Seungha Yang
206fe1534d msdkh265enc: Add support for CEA708 closed caption insertion
Functionally identical to that of msdkh264enc
2020-04-02 09:20:11 +00:00
Seungha Yang
fc9f7a6ade d3d11: Update for video-hdr struct change
See the change of -base https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/594
2020-04-01 05:18:11 +00:00
Seungha Yang
bd706edc52 nvh265enc: Update for video-hdr struct change
See the change of -base https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/594
2020-04-01 05:18:11 +00:00
Nicolas Dufresne
4cb871eb53 v4l2codecs: Only build this plugin on Linux
This is not useful on any other OSs, it will also avoid potential build
failure as this code uses Linux specific calls.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
7254a18f0f v4l2codecs: Add plugin dependency
This ensure that the registry cache get updated when a meaningful change is
made in /dev for files named media*.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
e70993bf43 v4l2codecs: Wait for buffers to come back
This code add required mechanism to try and allocate (not implemented yet)
otherwise wait for more buffers. This also comes with mechanism to terminate
the wait on flush or PAUSED_TO_READY transitions.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
09a9ffcda6 v4l2codecs: Implement flushing sequence
This simply consit of cycling through STREAMOFF/STREAMON with stateless
decoders.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
f69283819a v4l2codecs: allocator: Add method to wait for more buffers
This add function to wait for buffers to get back into the pool along with a
set_flushing() method to allow unblocking this wait.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
bc0a8ff40d v4l2codecs: pool: Create new buffer when pool is empty
This simply create an empty GstBuffer when the pool is empty. This way it's up
to the allocator to grow or wait if we ran out of memory.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
ee26c9c9a1 v4l2slh264dec: Improve end_picture() robustness
Use a goto to ensure that for all cases we cleanup the current picture state.
And move the src buffer allocation higher, so we don't queue a bitstream
buffer if we don't have a picture buffer to decode into.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
390cbe1f00 v4l2slh264dec: Add output format negotiation
This allow negotiating the output format through caps. Some drivers can
pipeline the decoder buffer through an image processor. This only support
colorspace conversion for now.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
6494d7b056 v4l2format: Convert between V4L2 and GST video format
This will be needed in the output format negotiation.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
2a1836f183 v4l2codecs: Read driver provided stride
This implements driver stride support but only for single allocation buffers.
This code is imported from the original v4l2 plugin and adapted to the new
helper context.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
21652a8e52 v4l2h264dec: Copy frames when GstVideoMeta is not supported
In some case, when downstream does not support GstVideoMeta, we need to
normalize the stride and offset of the buffer so that downstream can render
properly with a GstVideoMeta. This code is not called when GstVideoMeta is
supported downstream.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
3c2d25eebf v4l2codec: Add initial GstVideoMeta support
In this patch we strictly set the GstVideoMeta width/height to the coded width
and height. Further patches will add stride support and frame copying when
downstream does not support GstVideoMeta.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
f6690420ce v4l2slh264dec: Prevent spurious renegotiation
Don't let downstream cause a renegotiation at random point in time. This would
lead to spurious renegotiation and the decoder state may not be recoverable.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
b47929a74d v4l2slh264dec: Hold on picture buffers
These buffers should not be reused for decoding until they are no longer
referenced.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
b107c20198 v4l2codecs: Fix bytesused value
Pass the actual amount of bytes we have copied into the bitstream buffer. Also
unmap the memory before queuing.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
0d740a184e v4l2codech264dec: Implement finish frame
This enables the request to be processed by the HW (STREAMON) and to be pushed
downstream for further processing.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
3cc6643050 v4l2codecdecoder: Add support for dequeuing buffers
This includings polling of the request and streamon/streamoff.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
4305771178 v4l2codech264dec: Implement queuing
We now pass the controls, associated to a request, queue the bitstream, qeueue
a picture buffer to decode into and finally queue the request. This now runs
until the buffer pool is exhausted. The next step will be to dequeue.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
a307499437 v4l2decoder: Add helpers to queue buffer and requests 2020-03-31 09:34:05 -04:00
Nicolas Dufresne
db787adfa1 v4l2codec: Add getter for buffer index in allocator and pool 2020-03-31 09:34:05 -04:00
Nicolas Dufresne
527f040bfc v4l2h264codecdec: Copy bitstream parameter and data
In this patch we fill the control structure with the bitstream paramter and
copy the bitstream data into V4L2 memory. Slice paramters are only the subset
of what Hantro needs, without any support for interlaced content.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
2bac52a6fb v4l2slh264dec: Use the newly introduced allocator / pool
We now allocate and free the memory needed for streaming.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
c120f8b041 v4l2codec: Add allocator and pool implementation
This is a pooling allocator and the buffer pool does nothing other then
reusing the GstBuffer structure. Note that the pool is an internal pool, so
the start/stop/set_config virtual functions are not implemented.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
d476e6d689 v4l2codec: decoder: Add allocation helper 2020-03-31 09:34:05 -04:00
Nicolas Dufresne
7ae57631d4 v4l2codecs: Implement H264 format negotiation 2020-03-31 09:34:05 -04:00
Nicolas Dufresne
020a019fd6 v4l2codecs: Add initial formats helpers 2020-03-31 09:34:05 -04:00
Nicolas Dufresne
a007e2dd9d v4l2codecs: Add skeleton of H264 decoder
This introduces the skeleton of the H264 decoder. The plugin will list the
devices and register a subclass of the GstV4L2CodecH264Dec base class. The
subclass will pick the required specific information from the GstV4L2Device
stored in the subclass structure.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
b7762fc332 v4l2codec: Add GstV4L2Deocder helper object
This is a GstObject which will be used to hold on media and video device file
descriptor and provide abstracted ioctl calls with these descriptor. At the
moment this helper contains just enough to enumerate the supported format.
This part will be used by the plugin to register the CODEC specific elements..
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
dd2b0302b3 v4l2codecs: Copy all needed Linux kernel headers
Most of the features we need are very early or not expose yet in the uAPI.
Using an internal copy ensure that we everything we need is defined avoiding
to add load of checks and conditionnal code.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
6d9c98ae5a v4l2codecs: Add device enumeration
This introduces a GstV4L2CodecDevice structure and helper to retrieve a
list of CODEC device drivers. In order to find the device driver we
enumerate all media devices with UDEV. We then get the media controller
topology and locate a entity with function encoder or decoder and make
sure it is linked to two V4L2 IO entity pointing to the same device
node.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
6958fb6d59 v4l2codecs: Introdude a V4L2 CODECs Accelerator plugin
This plugin will support new CODECs uAPI in the Linux kernel. These
consist of an accelerator interface similar to DXVA, NVDEC, VDPAU and
VAAPI.
2020-03-31 09:34:05 -04:00
Seungha Yang
882531aa54 msdkh264enc: Port to h264parser API for frame packing SEI injection
Create frame packing SEI memory only once per set_format() and
reuse it if possible.
2020-03-31 08:30:50 +00:00
Haihao Xiang
326b755332 msdkh265enc: support 8-bit 422 encoding
The media driver can support HEVC 8-bit 422 encoding for non-lowpower
mode since ICL[1], so VPP is not needed for this case.

Sample pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=YUY2 ! msdkh265enc ! \
filesink location=output.h265

[1] https://github.com/intel/media-driver#decodingencoding-features
2020-03-31 09:27:03 +08:00
Haihao Xiang
7c91c8c980 msdkh265enc: add support for Y210 in sink pad
Sample pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=Y210 !
msdkh265enc ! filesink location=output.h265
2020-03-31 09:27:03 +08:00
Haihao Xiang
fbd1d0cecb msdk: add a helper function to get codename of the platform
The features supported in MSDK vary from platform to platform. We may
support some features based on the codename of the platform in future.
2020-03-31 09:27:03 +08:00
Seungha Yang
afb042cbb9 msdkenc: Try to find corresponding codec frame for encoded output buffer
The input and output buffers should be matched as much as possible
so that various metadata and its ordering to be preserved.
2020-03-30 07:59:10 +00:00
Seungha Yang
c0787ed1cd msdkh264enc: Add support for CEA708 closed caption insertion
Currently supported caption format is CEA708_RAW
2020-03-30 07:59:10 +00:00
Seungha Yang
429de2c6d8 d3d11decoder: Add support for array of texture DPB
DXVA supports two kinds of texture structure for DPB, one is
"1) texture array" and the other is "2) array of texture".
1) is a type of texture which is single ID3D11Texture2D object having
ArraySize greater than one. So the ID3D11Texture2D itself is a set of texture.
Each sub texture of this type mush have identical resolution, format and so on,
and the number of sub texture in a texture array is fixed.

2) is an array of usual ID3D11Texture2D object. That means each
ID3D11Texture2D is independent each other and might have different resolution as well.
Moreover, we can modify the number of frames of the array dynamically.
This type is more flexible than "1) texture array" in terms of dynamic
behavior and also this type of texture can be used for shader resource view
but "1) texture array" couldn't be.

If "2) array of texture" is supported by driver, DXVA spec is saying that
it's preferred format over "1) texture array" in terms of performance.
2020-03-30 03:40:16 +00:00
Haihao Xiang
d796707dd4 msdkh265enc: slice size control
Add max-slice-size property to set the maximum slice size
2020-03-30 00:20:11 +00:00
Seungha Yang
3208e3d371 Revert "d3d11decoder: Check decoder status report"
This reverts commit 418e6991c1.

Not all drivers seem to be friendly to this API.
Revert to avoid incomprehensible crash with Nvidia.
2020-03-28 12:12:03 +00:00
Seungha Yang
b2a650cf28 d3d11window: Always reuse swapchain
DXGI format can be updated via ResizeBuffers()
2020-03-28 11:45:48 +00:00
Seungha Yang
f9f6916aa1 d3d11window: Reorganize display colorspace selection
The set of supported color space by DXGI is not full combination of
our colorimetry. That means we should convert color space to one
of supported color space by DXGI. This commit modifies the color space
selection step so that d3d11window can find the best matching DXGI color space
first and then the selected input/output color space will be referenced
by shader and/or d3d11videoprocessor.
2020-03-28 11:45:48 +00:00
James Westman
2ac834b51a applemedia: Add some properties to GstAvfDevice
Adds properties to the devices listed in GstDeviceMonitor by the
applemedia plugin.

These properties are:
- device.api (always set to "avf")
- avf.unique_id
- avf.model_id
- avf.manufacturer (except on iOS)
- avf.has_flash
- avf.has_torch

Everything except device.api is taken directly from the AVCaptureDevice object
provided by AVFoundation.
2020-03-23 12:05:19 +00:00
Haihao Xiang
49604d8fbe msdkh265dec: add support for main-12, main-12-intra profiles
The video format is P012_LE
2020-03-23 06:31:30 +00:00
Haihao Xiang
94fa88b6da msdk: map P012_LE to VA_FOURCC_P016
In media driver, VA_FOURCC_P016 is used for semi-planar 12 bits YUV
format, the corresponding RT format is VA_RT_FORMAT_YUV420_12
2020-03-23 06:31:29 +00:00
Haihao Xiang
c5763f66c9 msdk: map MFX_FOURCC_P016 to VA_FOURCC_P016
P016 is used for semi-planar 12 bits format in the media driver, so the
RT format is VA_RT_FORMAT_YUV420_12
2020-03-23 06:31:29 +00:00
Haihao Xiang
4d31299c12 msdk: map P012_LE to MFX_FOURCC_P016
MFX_FOURCC_P016 is used for semi planar 12 bits YUV format in MediaSDK
2020-03-23 06:31:29 +00:00
Haihao Xiang
4d70add5b6 msdkvp9enc: add support for 10 bits 444 input
The input format is Y410 in sink pad and the corresponding profile in
src pad is profile 3
2020-03-23 08:20:13 +08:00
Haihao Xiang
e7d4158547 msdkvp9enc: add support for 8 bits 444 input
The input format is VUYA in sink pad and the corresponding profile in
src pad is profile 1
2020-03-23 08:15:21 +08:00
Seungha Yang
2e5fee17a3 msdkbufferpool: Fix build warning on Windows
gstmsdkbufferpool.c(274): warning C4101: 'fd': unreferenced local variable
2020-03-22 22:30:15 +09:00
U. Artie Eoff
09d41b0b7d msdkvpp: add YV12 format to src pad
VPP YV12 output format support added since
MFX_VERSION 1032.

https://github.com/Intel-Media-SDK/MediaSDK/pull/2027
2020-03-19 09:31:59 -07:00
Seungha Yang
22eab50907 nvdec: Add fallback for CUDA/OpenGL interop failure
It happens when local OpenGL context belongs to non-nvidia GPU.
2020-03-19 13:58:09 +09:00
Philippe Normand
273e32d5aa decklink: Remove spurious gtk-doc marker 2020-03-15 10:46:54 +00:00
Haihao Xiang
c02f603262 msdk: meson: check whether mfxExtVP9Param is defined
User reported MFX_VERSION is 1026 however the build is broken[1], so add
the check for mfxExtVP9Param to make sure it can be build without
compiler error.

In addtion, it fixes a stupid typo (#endif') introduced by me.

[1] https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/988#note_408093
2020-03-13 01:29:04 +00:00
rubenrua
6c3f092afc asfmux: Fix typo in property description
s/milisecs/milliseconds/g
2020-03-12 18:38:11 +01:00
Seungha Yang
58a4c33a0e d3d11vp9dec: Add support for internal frame resizing
VP9 codec allows resizing reference frame by spec. Handling this case
is a bit tricky especially when the resizing happens on non-keyframe,
because pre-allocated decoder textures (i.e., dpb) have negotiated
resolution and to change resolution meanwhile decoding on non-keyframe,
each texture might need to be re-created, copied to new dpb somehow,
and re-negotiated with downstream.

Due to the complicated requirement of negotiation driven
resizing handling, this commit adds shader into d3d11decoder object
to resize only corresponding frames. Note that if the resolution change
is detected on keyframe, decoder will re-negotiate with downstream.
2020-03-09 23:44:12 +09:00
Seungha Yang
c10b5f06a5 d3d11decoder: Set GstVideoAlignment to downstream d3d11 buffer pool
To copy decoder output texture to another d3d11 texture, the downstream
texture needs to be aligned too.
2020-03-09 23:44:12 +09:00
Seungha Yang
5c51601cdf d3d11decoder: Use aligned resolution for staging texture
Not only any textures for decoder output view, any destination texture
which would be copied from decoder output texture need to be aligned too.
Otherwise driver sometimes crashed/hung (not sure why).
2020-03-09 23:44:12 +09:00
Seungha Yang
5074cecc82 d3d11colorconverter: Add support cropping
Vertex buffer will be updated if input texture resolution is different
from GstVideoInfo or when an user requests specific crop area.
2020-03-09 23:44:06 +09:00
Seungha Yang
c6c83ca493 d3d11bufferpool: Add padding space for semi-planar YUV formats
Resolution of NV12, P010, and P016 formats must be multiple of two.
Otherwise texture cannot be created. Instead of doing this alignment
per API consumer side, do this in buffer pool for simplicity.
2020-03-09 10:14:53 +09:00
Seungha Yang
5609d8751c h265decoder: Pass max_dpb_size to new_sequence vfunc
same as we are doing in h264decoder
2020-03-08 16:11:35 +09:00
Seungha Yang
b70a192220 d3d11h264dec: Make use of max_dpb_size passed by baseclass
Most likely this modification can save GPU memory.
2020-03-08 16:03:35 +09:00
Seungha Yang
7c153b6e93 d3d11h265dec: Fix scaling list parsing
Scaling list can be signalled via sps or pps. Decoder should check
both of them.
2020-03-07 12:57:38 +09:00
Seungha Yang
cfb6283613 d3d11h265dec: Properly signal the ucNumDeltaPocsOfRefRpsIdx
ucNumDeltaPocsOfRefRpsIdx should be the NumDeltaPocs[RefRpsIdx].
2020-03-07 12:20:03 +09:00
Seungha Yang
83b056cb77 h265decoder: Port to nal type classification macro
... and remove namespaceless macro methods from baseclass
2020-03-05 23:22:39 +09:00
Nicolas Dufresne
020ee62c59 d3d11: Use gst_video_decoder_get_frame()
Now that the system_frame_number is saved on the pictures we can use
gst_video_decoder_get_frame() helper instead of getting the full list
and looping over it.
2020-03-05 03:06:16 +00:00
Nicolas Dufresne
c6b8157dd6 codecs: Pass the max_dpb_size to new_segment virtual
On new_segment, the decoder is expected to negotiate. The decoder may want to
pre-allocate the needed buffers. Pass the max_dpb_size as this is needed to
determin how many buffers should be allocated.
2020-03-05 03:06:16 +00:00
Nicolas Dufresne
00d04784d3 Move CODEC base classes into it's own library
This introduce a library which contains a set of base classes which
handles the parsing and the state tracking for the purpose of decoding
different CODECs. Currently H264, H265 and VP9 are supported. These
bases classes are used to decode with low level decoding API like DXVA,
NVDEC, VDPAU, VAAPI and V4L2 State Less decoders. The new library is
named gstreamer-codecs-1.0 / libgstcodecs.
2020-03-05 03:06:16 +00:00
Seungha Yang
be3de5a3db d3d11vp9dec: Add support for reverse playback
This commit moves parsing code for superframe and frame header into
handle_frame() method, and removes parse() implementation from vp9decoder
baseclass.

The combination of
- multiple frames are packed in a given input buffer (i.e., superframe)
- reverse playback
seems to be complicated and also it doesn't work as intended in some case
2020-03-03 12:15:33 +00:00
Matthew Read
3e2d86ea18 openslessink: Allow openslessink to handle 48kHz streams.
The most common audio sample rate in AV streams is 48kHz, and the most
common device output sample rate is 48kHz. This allows handing of 48kHz
input streams without resampling.

Remove comments about avoiding the use of 48kHz.
2020-03-03 08:59:01 +00:00
Heinrich Kruger
03cf2d42a5 decklink: Add support for 2K DCI video modes
Extend the video modes supported by the decklink plugin to include 2K
DCI video modes.
2020-03-03 08:05:56 +00:00
Heinrich Kruger
796fffa441 decklink: Update Decklink SDK to 10.11.4
This change is needed to support 2K DCI video modes.

Version 10.8 of the Decklink SDK supported DCI video modes for output
only. This updated version drops that restriction.

The current latest version of the Decklink SDK is 11.5, however
the gstreamer decklink plugin is not compatible with API changes
introduced in version 11 of the SDK. Therefore I have opted to upgrade
to the latest 10.x version instead.
2020-03-03 08:05:56 +00:00
Seungha Yang
9b7c20bfca d3d11videosink: Clarify the meaning of various width and height variables
* Remove redundant variables for width/height and par from GstD3D11Window.
  GstVideoInfo holds all the values.
* Don't need to pass par to gst_d3d11_window_prepare().
  It will be parsed from caps again
* Remove duplicated math

Fixing regression of the commit 9dada90108
2020-03-02 22:01:43 +09:00
Yeongjin Jeong
25e9ee10b0 d3d11vp9dec: Port to GstVP9SuperframeInfo
The vp9parser is now exposed new API for parsing superframe info.
2020-03-02 01:55:36 +09:00
Seungha Yang
9575d835ba d3d11decoder: Do not print warning message if retry count is in expected range
gst_d3d11_result() will print warning message when HRESULT != S_OK.
However, since the retry is trivial stuff, check hr == E_PENDING first
and do not warn it.
2020-02-28 13:21:04 +09:00
Seungha Yang
418e6991c1 d3d11decoder: Check decoder status report
... and if h/w decoder reports error, increase error count.
2020-02-28 13:21:00 +09:00
Seungha Yang
f2e322197e d3d11window: Set DXGI_PRESENT_ALLOW_TEARING only in fullscreen mode
The DXGI_PRESENT_ALLOW_TEARING flag might cause unexpected tearing
side effect. Setting it in fullscreen mode only seems to be
the correct usage as in the Microsoft's direct3d examples.
2020-02-26 13:18:16 +09:00
Seungha Yang
4643dc1c12 d3d11decoder: Ensure the written bitstream buffer size is 128 bytes aligned
DXVA spec is saying that the size of bitstream buffer provided by hardware decoder
should be 128 bytes aligned. And also the host software decoder should
align the size of written buffer to 128 bytes. That means if the slice
(or frame in case of VP9) size is not aligned with 128 bytes,
the rest of non 128 bytes aligned memory should be zero-padded.

In addition to aligning implementation, some variables are renamed
to be more intuitive by this commit.
2020-02-24 14:46:27 +00:00
Haihao Xiang
acc14225c9 msdk: libva: Don't set the hint if MFX_VERSION is lower than 1025
MFX_MEMTYPE_VIDEO_MEMORY_ENCODER_TARGET is defined since MFX_VERSION
1025.
2020-02-24 01:26:58 +00:00
Seungha Yang
fbe7917a94 d3d11decoder: Add padding space on decoder output view when it's not aligned
Most H/W decoders have required alignment and dxva is also the case.
2020-02-20 17:32:42 +09:00
Seungha Yang
fe72bf6053 d3d11decoder: Register elements per GPU device with capability check
This implementation is similar to what we've done for nvcodec plugin.
Since supported resolution, profiles, and formats are device dependent ones,
single template caps cannot represent them, so this modification
will help autoplugging and fallback.

Note that the legacy gpu list and list of resolution to query were
taken from chromium's code.
2020-02-18 11:58:45 +00:00
Seungha Yang
13586bc77a d3d11device: Fix typo
s/vender/vendor
2020-02-18 11:58:45 +00:00
Seungha Yang
8ead80eecd d3d11device: Adjust debug level for when _new() fails
gst_d3d11_device_new might be used to enumerate device.
2020-02-18 11:58:45 +00:00
Seungha Yang
36fb790243 d3d11videosink: Ensure upload staging texture to fallback render texture
gst_video_frame_copy will copy input frame to stating texture
of fallback frame. Then, we need to map fallback texture with GST_MAP_D3D11
flag to upload the staging texture to render texture. Otherwise
the render texture wouldn't be updated.
2020-02-16 21:29:08 +09:00
Seungha Yang
9bf4746e2f d3d11decoder: Fix copying decoder view to staging
Source texture (decoder view) might be larger than destination (staging) texture.
In that case, D3D11_BOX structure should be passed to CopySubresourceRegion method
in order to specify the exact target area.
2020-02-13 21:25:15 +09:00
Seungha Yang
f6cdb91f55 d3d11window: Fix for broken dirty rect drawing on Windows 7
DXGI_SWAP_EFFECT_DISCARD cannot be used with dirty rect drawing feature
of IDXGISwapChain1::Present().
Note that IDXGISwapChain1 interface is available on Platform Update for Windows 7
and DXGI_SWAP_EFFECT_FLIP_SEQUENTIAL is also the case.
2020-02-12 22:38:53 +09:00
Seungha Yang
4383b387b7 d3d11window: Fix for dxva decoder output view rendering
Use resolution specified in caps for input_rect instead of
passed width and height value. The width and height might be modified
ones by d3d11videosink, then frame resolution might be different.
2020-02-12 12:34:58 +00:00
Seungha Yang
a39a5bf131 d3d11decoder: Refactor decoding process
* Move decoding process to handle_frame
* Remove GstVideoDecoder::parse implementation
* Clarify flush/drain/finish usage

In forward playback case, have_frame() call will be followed by
handle_frame() but reverse playback is not the case.
To ensure GstVideoCodecFrame, the decoding process should be placed inside
of handle_frame(), instead of parse().

Since we don't support alignment=nal, the parse() implementation is not worth.
In order to fix broken reverse playback, let's remove the parse()
implementation and revisit it when adding alignment=nal support.
2020-02-12 12:34:58 +00:00
Seungha Yang
3e78afbe0a d3d11decoder: Move handle_frame implementation to baseclass
... and remove unused start, stop method from subclass.

Current implementation does not require subclass specific behavior
for the handle_frame() method.
2020-02-12 12:34:58 +00:00
Seungha Yang
6da90b59f4 d3d11videosink: Remove max size condition from pool
Actually our buffer pool size and the number of backbuffer are
independent. In case of reverse playback, upstream might request
a lot of buffers (up to GOP size).
2020-02-12 12:34:58 +00:00
Nirbheek Chauhan
266dc41596 nvcodec: Mark class data as may-be-leaked to quiet the leaks tracer
The class data with the caps in it will be leaked if the element is
registered but never instantiated. There is no way around this. Mark
the caps as such so that the leaks tracer does not warn about it.

This is the same as pad template caps getting leaked, which are also
marked as may-be-leaked. These objects are initialized exactly once,
and are 'global' data.
2020-02-12 00:00:51 +05:30
Haihao Xiang
50ae5061f9 msdkvp9enc: output raw vp9 stream instead of IVF stream
video/x-vp9 is required in the src pad, however the output includes a
IVF header, which makes the pipeline below doesn't work

  gst-launch-1.0 videotestsrc ! msdkvp9enc ! msdkvp9dec ! fakesink

Since mfx 1.26, the VP9 encoder supports bitstream without IVF header,
so in this patch, the mfx version is checked and msdkvp9enc is enabled
only if mfx 1.26+ is available
2020-02-10 06:46:28 +00:00
Jan Schmidt
6c1e5ab311 androidmedia: Support float i-frame-interval
Android 25 added support for i-frame-interval to be a floating
point value. Store the property as a float and use the newer
version when it's available.
2020-02-09 02:19:12 +11:00
Jan Schmidt
29e3d09014 androidmedia: Allow dynamic bitrate changes on Android >= 19
Android 19 added an API for dynamically changing the bitrate in a running
codec.

Also make it so that even when not update-able at runtime, parameters will at least
be stored so that they take effect the next the codec is restarted.
2020-02-09 02:19:12 +11:00
Jan Schmidt
1b8bf1be01 androidmedia: Handle force-keyunit requests
Use API from Android 19 to request a keyframe from the MediaCodec
when indicated by the base class.
2020-02-09 02:19:12 +11:00
Jan Schmidt
cfe318ea03 androidmedia: Permit Codec surface to be NULL
The AMC encoder wrapper doesn't support input surfaces yet,
and passes NULL when configuring the underlying codec.

This was broken in commit 7fcf3e
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1102
2020-02-09 02:19:12 +11:00
Nirbheek Chauhan
3ca87d9988 nvcodec: Fix crash in decoder on 32-bit Windows
Same fix as 1a7ea45ffd, but I didn't
test the decoder so I missed that the function pointers here weren't
using the correct calling convention too.
2020-02-06 13:39:52 +00:00
Seungha Yang
478fb29974 d3d11window: Clear cached buffer per new caps
d3d11window holds one buffer to redraw client area per resize event.

When the input format is being changed, this buffer should be cleared
to avoid mismatch beween newly configured shader/videoprocessor and
the format of previously cached buffer.
2020-02-06 00:46:46 +09:00
Sebastian Dröge
5b8ff98f96 nvdec: Don't leak template caps when registering elements with old NVIDIA driver 2020-02-05 09:49:20 +00:00
Seungha Yang
371b181292 d3d11decoder: Use consistent resolution between output caps and video meta
h264/h265 decoded buffer might have crop area then we need to
adjust video meta based on the padding space
2020-02-05 00:52:48 +00:00
Seungha Yang
2f32f30b62 d3d11memory: Add a method to specify padding space 2020-02-05 00:52:48 +00:00
Seungha Yang
f852ce01e4 d3d11decoder: Add support for zero-copy playback
When downstream support d3d11 memory with forward playback case,
expose decoder output view memory objects without copying.
2020-02-05 00:52:48 +00:00
Seungha Yang
69f7f958a0 d3d11decoder: Create decoder output view whenever it's required
Whatever the reason, buffer in pool might be freed then we need to
configure decoder output views again.
2020-02-05 00:52:48 +00:00
Seungha Yang
567575e33d d3d11decoder: Need to zero initilized for g_once
A vairable to be used for g_once, it should be zero initialized
2020-02-05 00:52:48 +00:00
Seungha Yang
8ff667e463 d3d11allocator: Work as if buffer pool when running on texture array mode
Because the size of texture array cannot be updated dynamically,
allocator should block the allocation request. This cannot be
done at buffer pool side if this d3d11 memory is shared among
multiple buffer objects. Note that setting NO_SHARE flag to
d3d11 memory is very inefficient. It would cause most likey
copy of the d3d11 texture.
2020-02-05 00:52:48 +00:00
Seungha Yang
4d02858f66 d3d11videosink: Fix fallback buffer copy
Since we don't use dynamic texture now, cpu access to the fallback
texture should not happen.
2020-02-05 00:52:48 +00:00
Seungha Yang
2aa9f0bd6c d3d11window: Don't create swapchain again per caps change
Creating swapchain is relatively heavy operation. If output dxgi format
is not being chagned, we don't need to destroy and create swachain again.
2020-02-05 00:52:48 +00:00
Seungha Yang
c1d2d9171d d3d11window: Invoke initial resize method from baseclass
... instead of calling from subclass in order for baseclass to handle
more things between swapchain creation and resource creation.
2020-02-05 00:52:48 +00:00
Seungha Yang
7aad9187e4 d3d11videosink: Use ID3D11VideoProcessor interface
...for color space conversion if available

ID3D11VideoProcessor is equivalent to DXVA-HD video processor
which might use specialized blocks for video processing
instead of general GPU resource. In addition to that feature,
we need to use this API for color space conversion of DXVA2 decoder
output memory, because any d3d11 texture arrays that were
created with D3D11_BIND_DECODER cannot be used for shader resource.

This is prework for d3d11decoder zero-copy rendering and also
for conditional HDR tone-map support.
Note that some Intel platform is known to support tone-mapping
at the driver level using this API on Windows 10.
2020-02-05 00:52:48 +00:00
Seungha Yang
122a9b93eb d3d11: Add video processor object
ID3D11VideoProcessor interface provides various image conversion
methods. Note that it's analogous to VAAPI VPP.
2020-02-05 00:52:48 +00:00
Seungha Yang
a967db3b20 d3d11format: Add util methods for mapping DXGI color space with ours
Move color space mapping and hdr10 metadata conversion methods to
d3d11format in order to reuse the code.
2020-02-05 00:52:48 +00:00
Seungha Yang
3f4a84bd32 nvenc: Query maximum supported API version
We've been using NvEncodeAPICreateInstance method to find the supported API
version, but that seems to be insufficient since there is a case
where plugin failed in opening encoding session even if NvEncodeAPICreateInstance
succeeded. Asking driver about the version would be the most certain way.
2020-02-03 14:15:28 +00:00
Nicolas Dufresne
115b65d1ab kmssink: Fix crash with force-modesetting=1
This is a master regression, we would allocate a bo without having
created the allocator yet. As of now, we lazily create the allocator.
2020-01-31 10:01:51 -05:00
Seungha Yang
5d160a8968 msdkdec: Fix GstMsdkContext leak 2020-01-29 15:35:21 +09:00
Seungha Yang
a8fff639c8 msdk: Clear reference counted object in dispose() method
Follow GObject's memory management model
2020-01-29 15:35:21 +09:00
Haihao Xiang
ac9c9d8efc msdk: use cached response for DMABuf when the frame size is same
User is seeing corrupted display when running `videotestsrc !
video/x-raw,format=NV12,width=xxx,height=xxx ! msdkh265enc ! msdkh265dec
! glimagesink` with changed frame size, e.g. from 1920x1080 to 1920x240

The root cause is a same dmabuf fd is used for frames with
different size, which causes some unexpected result. This patch requires
cached response is used for frames with same size only for DMABuf, so a
dmabuf fd can't be used for frames with different size any more.
2020-01-28 03:46:04 +00:00
Seungha Yang
e97ef8a562 d3d11window_win32: Let DXGI choose client area
Don't specify the resolution of backbuffer. Then dxgi will let us know the
actual client area. When upstream resolution is chagned, updating the size
of backbuffer without the consideration for client size would cause mismatch
between them.
2020-01-26 12:13:24 +00:00
Nicolas Dufresne
d393232bc2 nvdec: Do not map GStreamer discont to CUVid discont
Setting the CUVID_PKT_DISCONTINUITY implies clearing any past information
about the stream in the decoder. The GStreamer discont flag is used for
discontinuity caused by a seek, for first buffer and if a buffer was
dropped. In the first two cases, the parsers and demuxers should ensure we
start from a synchronization point, so it's unlikely that delta will be
matched against the wrong state.

For packet lost, the discontinuity flag will prevent the decoder from doing
any concealment, with a result that ca be much worst visually, or freeze the
playback until an IDR is met. It's better to let the decoder handle that for
us.

Removing this flag, also workaround a but in NVidia parser that makes it
ignore our ENDOFFRAME flag and increase the latency by one frame.
2020-01-25 13:39:03 +00:00
Nicolas Dufresne
a28ce16b3f nvdec: Tell the parser we have complete pictures
This sets the CUVID_PKT_ENDOFPICTURE flag in order to inform the decoder that
we have a complete picture. This should remove one frame latency otherwise
introduce by NVidia parser.
2020-01-25 13:39:03 +00:00
Seungha Yang
20d85c95d0 d3d11window_corewindow: Always call methods of CoreWindow interface from UI thread
Like swapchain panel implementation, most methods of CoreWindow
should be called from UI thread.
2020-01-23 03:49:19 +00:00
Haihao Xiang
e3ebebc20b msdk: Fix compiler warning
This patch fixed compiler warning below:

[1/4] Compiling C object 'sys/msdk/dc44ea0@@gstmsdk@sha/gstmsdkvpp.c.o'.
../../gst-plugins-bad/sys/msdk/gstmsdkvpp.c: In function
‘gst_msdkvpp_context_prepare’:
../../gst-plugins-bad/sys/msdk/gstmsdkvpp.c:214:7: warning: suggest
parentheses around operand of ‘!’ or change ‘&’ to ‘&&’ or ‘!’ to ‘~’
[-Wparentheses]
2020-01-21 18:37:10 +00:00
Seungha Yang
a10f26aa3a nvenc: Do not access to broken encode session
If an encode session failed in initializing, the encode
session would be broken and the next nvenc API will cause crash.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1179
2020-01-21 16:34:41 +09:00
Nirbheek Chauhan
1bcf44bb1f msdk: Fix increasing memory usage in dynamic pipelines
Our context is non-persistent, and we propagate it throughout the
pipeline. This means that if we try to reuse any gstmsdk element by
removing it from the pipeline and then re-adding it, we'll clone the
mfxSession and create a new gstmsdk context as a child of the old one
inside `gst_msdk_context_new_with_parent()`.

Normally this only allocates a few KB inside the driver, but on
Windows it seems to allocate tens of MBs which leads to linearly
increasing memory usage for each PLAYING->NULL->PLAYING state cycle
for the process. The contexts will only be freed when the pipeline
itself goes to `NULL`, which would defeat the purpose of dynamic
pipelines.

Essentially, we need to optimize the case in which the element is
removed from the pipeline and re-added and the same context is re-set
on it. To detect that case, we set the context on `old_context`, and
compare it to the new one when preparing the context. If they're the
same, we don't need to do anything.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/946
2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
6834a12112 msdk: Reorganize context preparation code
Split it out into a separate function with early exits to make the
flow clearer, and document what the function is doing clearly.
No functional changes.
2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
e83d5fd8d6 msdk: Fix warning about unused variable on Windows 2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
c0d778c2c7 msdk: Use gst_clear_object()
`gst_object_replace()` is not supposed to be used for unreffing and
NULLing objects.
2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
bda687344b nvcodec: Print debug info when initializing nvenc
We weren't printing the return value.
2020-01-20 17:15:55 +05:30
Nirbheek Chauhan
1a7ea45ffd nvcodec: Fix crash on 32-bit Windows
We weren't using the correct calling convention when calling CUDA and
CUVID APIs. `CUDAAPI` is `__stdcall` on Windows. This was working fine
on x64 because `__stdcall` is ignored and there's no special calling
convention. However, on x86, we need to use `__stdcall`.
2020-01-20 17:15:55 +05:30
Nirbheek Chauhan
7e93ae0638 nvcodec: cuda.h only needs glib.h, not gst.h
Just a nitpick. Also, force the compiler to use our stub header
instead of searching for it in the include paths.
2020-01-20 15:10:51 +05:30
Josep Torra
ef37446460 msdkdec: align frame list using decoded timestamp
Before this change decoder used the oldest frame in the list to pair it
with the decoded surface. This only works when there's a perfect stream
like HEADERS,SYNCPOINT,DELTA...

When playing RTSP streams we can get imperfect streams like HEADERS,
DELTA,SYNCPOINT,DELTA... In this case decoder drops the frames
between HEADERS and SYNCPOINT which leads into using wrong PTS on
the output frames.

With this change we inject the input PTS into the bitstream and use it
to align the internal frame list with the actually decoded position.

Fixes playback with:
```
gst-launch-1.0 rtspsrc location=... latency=0 drop-on-latency=1 ! ...
```
2020-01-19 04:26:15 +00:00
Seungha Yang
e8d527df93 nvenc: Query supported minimum resolution
Hard-coded 16x16 resolution is likely to differ from the device's support
in most cases. If we can use NV_ENC_CAPS_WIDTH_MIN and NV_ENC_CAPS_HEIGHT_MIN,
update pad template with returned value.
2020-01-16 15:24:03 +00:00
Seungha Yang
58a663c1e5 nvcodec: Bump SDK header to version 9.1
Update header to query minimum resolution of encoder and to control
the number of reference frame if it's supported
2020-01-16 15:24:03 +00:00
Haihao Xiang
52abbeeefb msdkdec: support transform_meta virtual method
Allow the base class to copy GstVideoRegionOfInterestMeta data to the
output buffer when calling gst_video_decoder_finish_frame
2020-01-15 00:47:12 +00:00
Haihao Xiang
8512624a41 msdkenc: set ROI region for msdk{h264, h265}enc
A reconfig is needed when ROI is changed, otherwise the ROI parameters won't
take effect
2020-01-15 00:47:12 +00:00
Haihao Xiang
84e234a8c7 msdkenc: add set_extra_params virtual method
set_extra_params is added to allow sub class to add extra mfx parameters
for changed input frame or meta data
2020-01-15 00:47:12 +00:00
Haihao Xiang
0e2fc39f24 msdkenc: add need_reconfig virtual method
need_reconfig is added to allow sub class requires a reconfig when
the input frame or the MetaData (e.g. GstVideoRegionOfInterestMeta)
attached to the input frame is changed.
2020-01-15 00:47:12 +00:00
Seungha Yang
6e73e762c5 d3d11h265dec: Fix wrong NoRaslOutputFlag setting
... and handle EOS and EOB nals.

Only the first CRA picture should be associated with NoRaslOutputFlag
as the comment in code.
2020-01-14 08:47:30 +00:00
Nirbheek Chauhan
b7d91711f7 ipcpipeline: Minimal fixes that allow building with MSVC 2020-01-14 09:23:02 +05:30
Nirbheek Chauhan
c8db7a9127 ipcpipeline: Rework compiler checks
`pipe()` isn't used since 15927b6511,
and `socketpair()` from `#include <sys/socket.h>` is used only in the
examples. In practice, you can use probably also use anything that
allows you to create fd pairs, such as named pipes or anonymous pipes.

We use the cross-platform GstPollFD API in the plugin.
2020-01-14 09:23:02 +05:30
Seungha Yang
96f0f4b613 d3d11memory: Always use native DXGI format if device support it
Use consistent memory layout between dxva and other shader use case.
For example, use DXGI_FORMAT_NV12 texture format instead of
two textures with DXGI_FORMAT_R8_UNORM and DXGI_FORMAT_R8G8_UNORM.
2020-01-13 01:58:08 +00:00
Seungha Yang
9ee40679b5 Revert "d3d11: Add support for D3D11_USAGE_DYNAMIC"
This reverts commit ddd13fc7c0

Dynamic usage can reduce the number of copy per frame but make
things complicated and the benefit seems to not significant.
Also since we don't provide _map() method for the dynamic usage,
application cannot read buffers which make "last-sample" property
unusable in case of d3d11videosink.
2020-01-13 01:58:08 +00:00
Seungha Yang
e188893963 d3d11: Don't register decoders if unavailable
DXVA requires a hardware interface but may not be available,
such as in the case of VMs or when the GPU vendor does not provide a decoder interface.
2020-01-11 17:42:04 +09:00
Seungha Yang
616082d14a d3d11decoder: Don't return not initialized object from _new() method
... and change some debug levels since initialization failure might not be fatal.
2020-01-11 17:42:04 +09:00
Nicolas Dufresne
6d6e897762 decklink: Silence no driver / no SO messages
This g_once() is called everywhere, even in provider. This cause
spurious error when device monitor is used. Just silence or remove
the spurious logs.
2020-01-10 08:00:05 +00:00
Haihao Xiang
bdf9a76226 Revert "msdkdec: make sure to use video memory on Linux"
Commit a1584b6 caused big performance drop if the downstream element
is not a msdk element because it is very slow to read data from video
memory directly.

This reverts commit a1584b6f99.
2020-01-10 01:36:40 +00:00
Seungha Yang
49a1f022fd d3d11colorconverter: Handle P016_LE format
P016 format is no different than P010. Not much things to add code.
2020-01-09 16:29:47 +00:00
Seungha Yang
bbab229905 d3d11colorconverter: Add support for YUV to YUV conversion 2020-01-09 16:29:47 +00:00
Seungha Yang
a122f305f9 d3d11colorconverter: Add support for RGB to YUV conversion
... and remove code for RGBx since it's not supported format
by our d3d11 implementation for now.
2020-01-09 16:29:47 +00:00
Seungha Yang
da50cc4d92 d3d11format: Remove invalid format from supported format list 2020-01-09 16:29:47 +00:00
Sebastian Dröge
e4389146d9 decklinkvideosink: Always configure 10 bit YUV for VANC frames
If 8 bit are required by the device/mode then it will be converted internally
by the SDK, but the SDK won't automatically convert from 8 to 10 bit. As
such, always use 10 bit VANC.

Some devices require configuring also a 10 bit video format when using
10 bit VANC is required but those would fail regardless and the
application would have to configure the correct video format.

With newer versions of the SDK this information can be retrieved via the
BMDDeckLinkVANCRequires10BitYUVVideoFrames flag but we don't use a new
enough SDK version yet to extract this information.
2020-01-09 12:54:04 +02:00
Seungha Yang
e4daa2ef43 d3d11: Add support for Universal Windows Platform
Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2020-01-06 20:14:51 +09:00
Seungha Yang
43a8eb9e92 d3d11decoder: Fix build on non-desktop target
Although the target platform of D3D11 decoding API are both desktop and UWP app,
DXVA header is blocked by "WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)"
which is meaning that that's only for desktop app.
To workaround this inconsistent annoyingness, we need to define WINAPI_PARTITION_DESKTOP
regardless of target WinAPI partition.
2020-01-06 18:45:56 +09:00
Haihao Xiang
9064de27f9 msdkvp9enc: fix 10bit encoding
The codec profile should be consistent with the frame fourcc code, this
fixes pipeline below:

gst-launch-1.0 videotestsrc ! \
video/x-raw,width=320,height=240,format=P010_10LE ! msdkvp9enc ! \
fakesink
2020-01-06 03:54:57 +00:00
Haihao Xiang
da22ff3846 msdkvp9enc: fix width and height
The frame width and height is rounded up to 128 and 32 since commit
8daac1c, so the width, height for initialization should be rounded up to
128 and 32 too because the MSDK VP9 encoder will do some check on width
and height.

Sample pipeline:
gst-launch-1.0 videotestsrc ! \
video/x-raw,width=320,height=240,format=NV12 ! msdkvp9enc ! fakesink
2020-01-06 03:54:57 +00:00
Haihao Xiang
d6fbec5a44 msdk: make sure the found response meets the request
Otherwise the re-used msdk response doesn't have enough frames so it is
possible the pipeline will be broken
2020-01-06 02:16:53 +00:00
Julien Isorce
f578eab6fd msdkenc: renegotiate when video info changes
Renegotiation was implemented for bitrate change. We can re-use
the same sequence when video info changes except that this can be
executed right away when receiving the new input format. I.e. no
need to wait for the next call to handle_frame.
2020-01-06 01:13:28 +00:00
Philippe Normand
9f1fbd3649 decklink: Fix crash when probing without driver
If there is no decklink hardware/driver, the devices list is empty (NULL), so
this needs to be checked before iterating over the list.
2020-01-05 16:20:18 +00:00
Julien Isorce
4e6a1b9634 msdkdec: trigger renegotiation if video info changes
Useful when framerate changes. Previously it was only checking
for resolution change but renego should happen if any video
info changes.
2020-01-03 10:19:57 -08:00
Julien Isorce
a1584b6f99 msdkdec: make sure to use video memory on Linux
The block that sets use_video_memory flag is after the
the condition `if gst_msdk_context_prepare` but it
always returns false when there is no other msdk elements.
So the decoder ends up with use_video_memory as FALSE.
Note that msdkvpp always set use_video_memory as TRUE.

When use_video_memory is FALSE then the msdkdec allocates
the output frames with posix_memalign (see gstmsdksystemmemory.c).
The result is then copied back to the GstVideoPool's buffers
(or to the downstream pool's buffers if any).
When use_video_memory is TRUE then the msdkdec uses vaCreateSurfaces
to create vaapi surfaces for the hw decoder to decode into
(see gstmsdkvideomemory.c). The result is then copied to either
the internal GstVideoPool and to the downstream pool if any.
(vaDeriveImage/vaMapBuffer is used in order to read the surfaces)
2020-01-03 07:08:23 +00:00
Seungha Yang
b88af2e4fe msdkdec: Correct return value of GstVideoDecoder::flush()
Use boolean instead of GstFlowReturn as declared.
Note that since base class does not check return value of GstVideoDecoder::flush(),
this would not cause any change of behavior.
2020-01-02 12:22:07 +09:00
Haihao Xiang
a819f05851 msdkdec: free unlocked msdk surface before output buffer allocation
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/924
is trying to use video memory for decoding on Linux, which reveals a
hidden bug in msdkdec.

For video memory, it is possible that a locked mfx surface is not used
indeed and it will be un-locked later in MSDK, so we have to check the
associated MSDK surface to find out and free un-used surfaces, otherwise
it is easy to exhaust all pre-allocated mfx surfaces and get errors below:

0:00:00.777324879 27290 0x564b65a510a0 ERROR                default
gstmsdkvideomemory.c:77:gst_msdk_video_allocator_get_surface: failed to
get surface available
0:00:00.777429079 27290 0x564b65a510a0 ERROR         msdkbufferpool
gstmsdkbufferpool.c:260:gst_msdk_buffer_pool_alloc_buffer:<msdkbufferpool0>
failed to create new MSDK memory

Note the sample code in MSDK does similar thing in
CBuffering::SyncFrameSurfaces()
2020-01-02 00:43:36 +00:00
Seungha Yang
a139c8c7e8 d3d11: Add h265 decoder element
Some DPB management implementation is taken from gstreamer-vaapi
2019-12-31 02:13:48 +00:00
Seungha Yang
0e7b6526b8 d3d11: Add vp9 decoder element
Based on gstreamer-vaapi and Chromium implemetation.
2019-12-31 02:13:48 +00:00
Seungha Yang
586390b1ba d3d11: Add h264 decoder element
New decoder implementation based on dxva2 on d3d11 APIs. The DPB
management implementation is taken from Chromium.
2019-12-31 02:13:48 +00:00
Seungha Yang
7bfdeaf161 d3d11: Add support for Array typed texture memory
A ID3D11Texture2D memory can consist of multiple planes with array.
For array typed memory, GstD3D11Allocator will allocate new GstD3D11Memory
with increased reference count to the ID3D11Texture2D but different array index.
2019-12-31 02:13:48 +00:00
Stéphane Cerveau
add7878e14 bad: use of g_value_dup_string
Use helper method to get string from GValue.
2019-12-30 14:13:03 +00:00
Seungha Yang
81dde0f5b8 d3d11: Aggregate d3d11 memory usage query for dynamic-usage type decision
Even if one of downstream d3d11 elements can support dynamic-usage memory,
another one might not support it. Also, to support dynamic-usage,
both upstream and downstream d3d11device must be the same object.
2019-12-28 05:43:44 +00:00
Seungha Yang
d731bcb18a d3d11colorconvert: Remove device type dependent behavior
If d3d11colorconvert element is configured, do color space conversion
regardless of the device type whether it's S/W emulation or real H/W.
Since d3d11colorconvert is no more a child of d3d11videosinkbin,
we don't need this behavior. Note that previous code was added to
avoid color space conversion from d3d11videosink if no hardware
device is available (S/W emulation of d3d11 is too slow).
2019-12-28 05:43:44 +00:00
Seungha Yang
46186356ca d3d11colorconverter: Fix unmatched lock/unlock pair 2019-12-28 05:43:44 +00:00
Seungha Yang
ab80c771df d3d11upload: Don't hard overwrite caps features to support d3d11 memory passthrough
d3d11upload should be able to support upstream d3d11 memory, not only system memory.

Fix for following pipeline
d3d11upload ! "video/x-raw(memory:D3D11Memory)" ! d3d11videosink
2019-12-28 05:43:44 +00:00
Seungha Yang
5298d95195 d3d11window: Make use of partial presentation with IDXGISwapChain1::Present1
Since we might draw on partial area of backbuffer in case of force-aspect-ratio,
presenting only updated area is more efficient way.

See also https://docs.microsoft.com/ko-kr/windows/win32/direct3ddxgi/dxgi-1-2-presentation-improvements
2019-12-24 20:10:54 +09:00
Seungha Yang
487a41d312 d3d11videosink: Add support for overlay composition
Add d3d11overlaycompositor object to draw overlay image
on render target using Blend method.
2019-12-24 19:00:45 +09:00
Seungha Yang
a5295509af d3d11: Enable dxgi debug layer and always try to pop d3d11/dxgi debug message if possible
Note that dxgi and d3d11 sdk debug will be enabled on debug build
2019-12-24 19:00:40 +09:00
Seungha Yang
49bccf0433 nvcodec: Refactor plugin initialization
Create CUDA context per device, instead of per codec and encoder/decoder.
Allocating CUDA context is heavy operation so we should reuse it
as much as possible.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1130
2019-12-24 08:10:14 +00:00
Seungha Yang
0e013fe539 d3d11videosink: Use dynamic texture for fallback buffer
Upload CPU memory to texture directly by using dynamic usage texture.
This will reduce at least one step of staging copy per frame.
2019-12-20 19:21:02 +09:00
Seungha Yang
65cace22e2 d3d11colorconvert: Set TRANSFER_NEED_DOWNLOAD flag to output memory for later cpu access
Otherwise CPU cannot access texture via gst_memory_map()
2019-12-20 19:21:02 +09:00
Seungha Yang
18e13404a2 d3d11colorconvert: Use shader resource and render target bind flags together
The output of d3d11colorconvert would be used for rendering (i.e., shader resource)
2019-12-20 19:21:02 +09:00
Seungha Yang
62c8d19e01 d3d11window: Change aspect-ratio mode from window thread
Call DXGI API from window thread as much as possible
2019-12-20 19:21:02 +09:00
Seungha Yang
1b687d556a d3d11videosink: Add support for full screen mode
borderless top-most style full screen mode support.
Basically fullscreen toggle mode is disabled by default. To enable it
use "fullscreen-toggle-mode" property to allow fullscreen mode change
by user input and/or property.
2019-12-20 19:21:02 +09:00
Seungha Yang
0788492461 d3d11: Use GRecMutex to protect immediate context and dxgi API call
In some cases, rendering and dxgi (e.g., swapchain) APIs should be
called from window message pump thread, but current design (dedicated d3d11 thread)
make it impossible. To solve it, change concurrency model to locking based one
from single-thread model.
2019-12-20 19:21:03 +09:00
Seungha Yang
a0a85cd80c d3d11window: Prefer to use flip-sequential over discard
flip-sequential is more efficient than discard
2019-12-20 11:15:12 +09:00
Seungha Yang
1424de0a4a d3d11window: Don't hold backbuffer interface
We don't need to hold it
2019-12-20 11:15:12 +09:00
Seungha Yang
32d618c677 d3d11window: Use CreateSwapChainForHwnd if available
That's recommended way from MS and CreateSwapChainForHwnd supports
more options than CreateSwapChain
2019-12-20 11:15:12 +09:00
Seungha Yang
9fd0b62f2d d3d11: Use configuration file for dxgi header version check 2019-12-20 11:15:12 +09:00
Seungha Yang
23b47a7ec1 d3d11window: Use allow-tearing mode if device supports it
As the recommendation from MS.
2019-12-20 11:15:12 +09:00
Seungha Yang
72b6b3557d d3d11window: Disable fullscreen mode change by alt + enter
Disable full screen mode change until proper handling is implemented
2019-12-20 11:15:12 +09:00
Seungha Yang
9dada90108 d3d11videosink: Remove resizing window hack and unify resizing flow
In earlier implementation of d3d11videosink where no shader was implemented,
the aspect ratio and render size were adjusted by manipulating the backbuffer size
with unintuitive formula. Since now we do color conversion and resize using
shader, we can remove the hack.
2019-12-20 11:15:12 +09:00
Seungha Yang
61cb6b2bbe d3d11: Pass GstD3D11Memory object to ensure_{shader_resource,render_target}_view methods
The method name indicates it should be d3d11memory, so passing
GstD3D11Memory seems to make more sense than GstMemory.
2019-12-20 11:15:12 +09:00
Seungha Yang
7c041f1934 d3d11videosink: Don't specify crop meta to allocation query
d3d11videosink could not handle it for now
2019-12-20 11:15:12 +09:00
Aaron Boxer
c8193b1615 d3dvideosink: hold class lock for entire duration of class destruction
This avoids a race condition currently when temporarily releasing the
lock and then re-acquiring.
2019-12-18 13:15:06 -05:00
Aaron Boxer
7cb6dbefba d3dvideosink: improve concurrency on hidden window create/destroy 2019-12-18 13:15:06 -05:00
Aaron Boxer
eff0117b5a d3dvideosink: use thread pool to handle events from hidden window event queue
window event queue now does not lock on the class lock, so we can now shut
it down without releasing the class lock, thus avoiding a potential race when
stopping the sink.
2019-12-18 13:15:06 -05:00
Aaron Boxer
027eb5ef20 d3dvideosink: improve concurrency on internal window create/destroy
Remove timeout and rely on condition variable instead to indicate thread
start.
2019-12-18 13:15:06 -05:00
Aaron Boxer
adfbce336a d3dvideosink: only warn about HWND already set if new HWND is not NULL 2019-12-18 13:15:06 -05:00
Aaron Boxer
84403dbae3 d3dvideosink: add more null checks 2019-12-18 13:15:05 -05:00
Aaron Boxer
06862d7ff5 d3dvideosink: check ref count and device before resetting display 2019-12-18 13:15:05 -05:00
Aaron Boxer
85c1550b92 d3dvideosink: use class lock when checking d3d formats, and cache format list 2019-12-18 13:15:05 -05:00
Aaron Boxer
d7af30bab9 d3dvideosink: destroy device just before final d3d release call
and free overlays after swap chain is released
2019-12-18 13:15:05 -05:00
Aaron Boxer
5e3520a302 d3dvideosink: add two forward declarations 2019-12-18 13:15:05 -05:00
Aaron Boxer
49211bdf74 d3dvideosink: use class lock when setting pool config 2019-12-18 13:15:05 -05:00
Aaron Boxer
fb6c2e8533 d3dvideosink: use class lock when allocating pool buffer 2019-12-18 13:15:05 -05:00
Aaron Boxer
52254ad31c d3dvideosink: use class lock when creating overlay textures 2019-12-18 13:15:05 -05:00
Aaron Boxer
51f823c98a d3dvideosinnk: check d3d device exists before creating overlay vertex buffer 2019-12-18 13:15:05 -05:00
Aaron Boxer
8710eb0656 d3dvideosink: pass hidden window hwnd to GST_DEBUG, rather than NULL 2019-12-18 13:15:05 -05:00
Aaron Boxer
54ce3d5011 d3dvideosink: fix typo 2019-12-18 13:15:05 -05:00
Seungha Yang
64eb1d46b5 d3d11window: Always draw to internal window
... and use SetParent() WIN32 API when external window is used.
Depending on DXGI swap effect, the external window might not be
reusable by another backend. To preserve the external window's property
and setting, drawing to internal window seems to be safer way.
2019-12-16 22:24:29 +09:00
Seungha Yang
638c6d6931 d3d11window: Always set error reason to GError object 2019-12-16 18:21:02 +09:00
Julien Isorce
93bc74284f msdkdec: call finalize on the parent class
Otherwise GstVideoDecoder is not finalized and
resources are leaked.

Somehow GST_TRACERS="leaks" GST_DEBUG="GST_TRACER:7" did not catch it.

Valgrind output:

==31645== 22,480 (1,400 direct, 21,080 indirect) bytes in 5 blocks are definitely lost in loss record 5,042 of 5,049
==31645==    at 0x4C2FB0F: malloc
==31645==    by 0x51D9E88: g_malloc
==31645==    by 0x51FA7B5: g_slice_alloc
==31645==    by 0x51FAC68: g_slice_alloc0
==31645==    by 0x58D9984: g_type_create_instance
==31645==    by 0x58BA344: g_object_new_with_properties
==31645==    by 0x58BADA0: g_object_new
==31645==    by 0x8ECA966: gst_video_decoder_init
==31645==    by 0x58D99E7: g_type_create_instance
==31645==    by 0x58BA344: g_object_new_with_properties
2019-12-13 15:36:26 -08:00
Roman Shpuntov
5da2938244 avfvideosrc: element requests camera permissions even with capture-screen property is true
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1137

macOS has different dialogs for camera capture and screen capture.
No need to request screen capture permissions, the system detect
screen capture automatically and create request dialog.
2019-12-11 17:31:03 +00:00
Seungha Yang
a400126889 d3d11window: Initialize surface rect with given external window handle
Unlike internal window, resize event might not happen with external window.
2019-12-11 23:04:55 +09:00
Seungha Yang
a8ec409b84 d3d11device: Report alive d3d11 objects at the end
It would be useful for debugging d3d11 object leak.
2019-12-11 23:04:49 +09:00
Seungha Yang
59295b4271 d3d11: Fix ID3D11SamplerState leak 2019-12-11 12:57:46 +00:00
Stéphane Cerveau
6bc0e9527e remove various useless linefeed in logs 2019-12-11 10:51:29 +01:00
Haihao Xiang
b18be3562c msdk: ignore MFX_ERR_UNDEFINED_BEHAVIOR when loading a plugin
An issue can be seen when using msdkh265enc with bitrate change in
playing state. The root cause is the corresponding plugin is loaded
again.

Returning MFX_ERR_UNDEFINED_BEHAVIOR from MSDK just means the plugin has
been loaded, so we may ignore this error when doing configuation again
in the sub class, otherwise the pipeline will be interrupted
2019-12-10 17:23:53 +00:00
Seungha Yang
d963d442c5 d3d11videosink: Fix broken fallback rendering
Make fallback rendering work
2019-12-10 04:06:28 +00:00
Seungha Yang
abc1443cf0 d3d11videosink: Add debug message for fallback reason 2019-12-10 04:06:28 +00:00
Seungha Yang
9835289239 d3d11window: Do not check shader resource view if not doing conversion
If d3d11window does not convert format internally, shader resource view
is not required. Note that shader resource view is used for
color conversion using shader but when conversion is not required,
we just copy input input texture to backbuffer.
2019-12-10 04:06:28 +00:00
Julien Isorce
b681d6f62e msdkvpp: check mfx surface nullity for the input buffer
In theory it should not happen but it happened to me
in some cases where it failed to allocate some video
buffers so this was a consequence of a corner case.
Better to be safe than sorry.
2019-12-10 03:00:13 +00:00
Seungha Yang
8154b22a88 d3d11window: Clear old swapchain per prepare
_prepare() might be called multiple times for given window, so clear
old swapchain if any.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1138
2019-12-10 10:58:28 +09:00
Julien Isorce
3bae199863 msdkdec: fix assertion 'frame->ref_count > 0' failed
Can happen if the oldest frame is the current frame
and if gst_video_decoder_finish_frame failed in which
case the current is unref and then drop instead of
just drop.
This patch also removes some assumptions, it was strange
to call unref and finish_frame in gst_msdkdec_finish_task.
In principle when owning a frame, the code should either
unref, or drop or finish.
2019-12-09 16:10:51 +00:00
Seungha Yang
5bf10d8781 d3d11window: Always set dxgi colorspace if possible
Not only for the 10 bits RGB case with HDR metadata, set proper color space
to swapchain in order to make color representaion correct
2019-12-09 12:22:15 +00:00
Matthew Waters
526afac736 vtdec: add support for outputing vulkan images 2019-12-09 01:49:31 +00:00
Matthew Waters
acac78c1ce applemedia: rename IOSurfaceMemory -> IOSurfaceGLMemory 2019-12-09 01:49:30 +00:00
Matthew Waters
0560946c82 videotexturecache: implement GL specifics as a subclass 2019-12-09 01:49:30 +00:00
Matthew Waters
ff5ba2f126 applemedia/videocache: make a GObject subclass 2019-12-09 01:49:30 +00:00
Matthew Waters
27f4fb060d avfvideosrc: don't leak texture caches
If decide_allocation is called multiple times, then we would continually
overwrite the previous cache without freeing
2019-12-09 01:49:30 +00:00
Aaron Boxer
e9891dfe14 d3d11: translate windows error codes to human readable strings
This will help with debugging.
2019-12-07 16:09:03 -06:00
Xavier Claessens
da584995f6 mlaudiosink: Fix crash when stopping pipeline
This is a bug in LuminOS 0.97.0.
2019-12-06 15:29:29 +00:00
Xavier Claessens
380ab8123b Add mlaudiosink element 2019-12-06 15:29:29 +00:00
Sebastian Dröge
f776e2df96 decklink: Return new references to the devices from get_devices()
Otherwise the caller will free the devices we store internally.
2019-12-05 21:22:18 +02:00
Seungha Yang
4eac9d0ae6 d3dvideosink: Fix warning for unhandled enum in switch
d3dhelpers.c:135:3: warning: enumeration value 'D3DFMT_D32_LOCKABLE' not handled in switch [-Wswitch]
2019-12-05 04:31:54 +00:00
Seungha Yang
199caccc41 d3d11: Initialize debug categories of non-GstElement implementation in plugin init
That's the way to expose debug category to --gst-debug-help
2019-12-05 02:29:18 +00:00
Seungha Yang
71befc477f d3d11window: Invoke resize task with high priority
On resize event, swapchain should be configured with higher priority
than the other tasks since it's directly related to visual artifacts.
2019-12-05 02:29:18 +00:00
Seungha Yang
d2b85683de d3d11device: Add gst_d3d11_device_thread_add_full method
I would be used to invoke GPU task with specified priority.
2019-12-05 02:29:18 +00:00
Seungha Yang
7cab47ddda d3d11videosinkbin: Drop d3d11 color convert element
d3d11videosink can convert color space now, so the conversion element
seems to be redundant.
2019-12-05 02:29:18 +00:00
Seungha Yang
ca3ddf7848 d3d11videosink: Add color conversion support
Draw to back buffer texture directly. It would reduce the number of
copy at least once when color conversion is required.
2019-12-05 02:29:18 +00:00
Seungha Yang
622733ed0d d3d11converter: Skip setup converter if input and output formats are equal 2019-12-05 02:29:18 +00:00
Seungha Yang
c9cb08fcd4 d3d11colorconvert: Split color space converter to reuse code 2019-12-05 02:29:18 +00:00
Seungha Yang
42ca123f0f d3d11: Split shader to reuse code 2019-12-05 02:29:18 +00:00
Seungha Yang
59d9589722 d3d11videosink: Draw window with cached texture on resize
This would render nicer than presenting scene with broken aspect ratio,
especially in case of low framerate.
2019-12-05 02:29:18 +00:00
Seungha Yang
ddd13fc7c0 d3d11: Add support for D3D11_USAGE_DYNAMIC
D3D11 dynamic texture is a special memory type, which is mainly used for
frequent CPU write access to the texture. For now, this texture type
does not support gst_memory_{map,unmap}
2019-12-05 02:29:18 +00:00
Seungha Yang
5ce4748cc0 d3d11device: Add debug message for CreateTexture2D failure 2019-12-05 02:29:18 +00:00
Seungha Yang
914d2df79e d3d11device: Fix misreading debug message 2019-12-05 02:29:18 +00:00
Seungha Yang
71ae632b25 d3d11colorconvert: Disable color conversion with software rasterizer
It's much slower than our CPU based color-converter in most case.
2019-12-05 02:29:18 +00:00
Seungha Yang
79e88b76da d3d11videosink: Handle context query
Propagate d3d11device via context query.
2019-12-05 02:29:18 +00:00
Seungha Yang
5edf64d655 d3d11colorconvert: Avoid copy in/out texture if possible
When input and/or output d3d11memory has its own view, use them for
rendering to avoid texture copy
2019-12-05 02:29:18 +00:00
Seungha Yang
da3545e8c8 d3d11memory: Allocate texture with required views
Store required resource views with d3d11 texture when it requested.
2019-12-05 02:29:18 +00:00
Seungha Yang
4e4fb8a577 d3d11: Add d3d11videosinkbin element
New wrapper element to support d3d11 memory upload, color conversion, and
rendering at once.
2019-12-05 02:29:18 +00:00
Seungha Yang
79ac41ec5b d3d11: Add color space conversion element using shader
Add d3d11colorconvert element for color space conversion.
Currently {RGB, YUV} to RGB conversion is supported.
2019-12-05 02:29:18 +00:00
Seungha Yang
01903c47f7 d3d11memory: Register mininal compare function for GstD3D11AllocationParams
The d3d11 allocation param is stored in GstStructure so it should
provide compare function for later use such as gst_structure_is_equal()
2019-12-05 02:29:18 +00:00
Seungha Yang
0778fa8043 d3d11window: Set render target whenever drawing
Device context's render target might be updated for offscreen rendering
2019-12-05 02:29:18 +00:00
Seungha Yang
a687f06b59 d3d11videosink: Specify d3d11 memory caps feature on sinkpad template 2019-12-05 02:29:18 +00:00
Seungha Yang
c17c1346ee d3d11: Introduce d3d11 upload/download element
That's equivalent to glupload and gldownload elements but for d3d11
2019-12-05 02:29:18 +00:00
Seungha Yang
6c3311a39e d3d11: Refactor d3d11 memory and dxgi format usage
* Create staging texture only when the CPU access is requested.
Note that we should avoid the CPU access to d3d11 memory as mush as possible.
Incoming d3d11upload and d3d11download will take this GPU memory upload/download.

* Upload/Download texture memory from/to staging only if it needed, similar to
GstGL PBO implementation.

* Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder).
Because I420_* formats are not supported formats by dxgi, each plane should
be handled likewise GstGL separately, but NV12/P10 formats might be supported ones.
So we decide the number of d3d11memory per GstBuffer for video memory depending on
OS version and dxgi format. For instance, if NV12 is supported by OS,
only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit.
One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data
to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes.
Such behavior will be controlled via configuration of GstD3D11BufferPool and
default configuration is separate resources per plane.
2019-12-05 02:29:18 +00:00
Seungha Yang
d4c75f32dd d3d11: Add method for querying d3d11 memory type
... and fix misreading "private" annotation.
2019-12-05 02:29:18 +00:00
Seungha Yang
8da5237e22 d3d11device: Store selected d3d feature level
Depending on selected feature level, d3d11 API usage can be different.
Instead of querying the selected feature level by user whenever required,
store it once by d3d11device.
2019-12-05 02:29:18 +00:00
Seungha Yang
4ff0e62b72 d3d11: Don't share d3d11 device context unconditionally without checking adapter
Do not accept any GstD3D11Device context which has different adapter
index from the required one. For example, if a d3d11 element is expecting
d3d11 device with adapter 1 (i.e., the second GPU), any d3d11 device
context having different adapter could not be shared with
the d3d11 element.
2019-12-05 02:29:18 +00:00
Seungha Yang
7ae4105c6e d3d11device: Don't allow auto adapter selection
Change to d3d11device to be created with explicit target adapter index,
and expose some readonly device properties
2019-12-05 02:29:18 +00:00
Seungha Yang
494ec13316 d3d11: Rename native handle getter and context utils functions
Make them consistent with cuda context utils functions.
Put in-only parameter before all in-out parameters, and add _handle()
suffix to native handle getter functions.
2019-12-05 02:29:18 +00:00
Aaron Boxer
d3426cf15b d3dvideosink: use parent buffer pool to release buffers
In certain cases, the sink's buffer pool will not call the parent's
release_buffer method, so the pool does not clean up properly
after the buffer is released.
2019-12-04 23:00:35 +00:00
Sebastian Dröge
8ba4d1a480 decklink: Add simple device provider for Decklink devices 2019-12-03 09:16:22 +00:00
o0Ignition0o
d29012a760 avfvideosrc: Explicitly request device video permissions for macOS 10.14+
Since macOS Mojave (10.14), video permissions have to be explicitly
granted by a user in order to open a video device such as a camera.
This commit adds a check for the current permission status, and tries
to request for permission if applicable.
2019-12-02 08:52:05 +01:00
Nirbheek Chauhan
d6471b0251 wasapisrc: Correctly handle BUFFERFLAGS_SILENT
We need to ignore the data we get from WASAPI in this case and write
out silence (zeroes).

Initially reported at https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/808
2019-11-28 08:59:41 +00:00
Nirbheek Chauhan
08d5bdc7b5 wasapisrc: Try harder to avoid debug output in the hot loop
The whole `src_read()` function is a hot loop since the ringbuffer
thread is waiting on us, and printing to the console from inside it
can easily cause us to miss our deadline.

F.ex., if you had GST_DEBUG=3 and we accidentally missed a device
period, we'd trigger the "reported glitch" warning, which would cause
us to miss another device period, and so on. Let's reduce the log
level so that GST_DEBUG=3 is more usable, and only print buffer flag
info when it's actually relevant.
2019-11-28 08:59:41 +00:00
Nirbheek Chauhan
6cbff552fe wasapisrc: Fix capturing from some buggy audio drivers
Some audio drivers return varying amounts of data per ::GetBuffer
call, instead of following the device period that they've told us
about in `src_prepare()`.

Previously, we would just drop those extra buffers hoping that the
extra buffers were temporary (f.ex., a startup 'burst' of audio data).
However, it seems that some audio drivers, particularly on older
Windows versions (such as Windows 10 1703 and older) consistently
return varying amounts of data.

Use GstAdapter to smooth that out, and hope that the audio driver is
locally varying but globally periodic.

Initially reported in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/808
2019-11-28 08:59:41 +00:00
Nirbheek Chauhan
5d9c060ca7 wasapisrc: Clarify that nBlockAlign is actually bpf
bpf = bytes per frame.
2019-11-28 08:59:41 +00:00
Nirbheek Chauhan
6d27c0ac08 wasapisrc: Fix glitching and clock skew issues
We were miscalculating the device period, i.e. the number of frames
we'll get from WASAPI in each IAudioClient::GetBuffer call, due to
a calculation mistake (truncate instead of round).

For example, on my machine when the aux input is set to 44.1KHz, the
reported device period is 101587, which comes out to 447.998 frames
per ::GetBuffer call. In reality we will, of course, get 448 frames
per call, but we were truncating, so we expected 447 and were
discarding one frame every time. This led to glitching, and skew over
time.

Interestingly, I can only see this with 44.1Khz. 48Khz/96Khz are fine,
because the device period is a more 'even' number.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/806
2019-11-28 08:59:41 +00:00
Seungha Yang
00cb881431 msdk: Fix mixed declarations warning
msdk.c:194:3: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
2019-11-27 12:25:40 +09:00
Julien Isorce
2af672d00a msdkdec: log an error if leaking surfaces in finalize
The for loop in gst_msdkdec_handle_frame is error prone
about how it manages surfaces. Because sometimes it sets
the surface variable to NULL and sometimes it needs to free
it right away. So better to print an error if surfaces are
leaked to help with any change around the loop.
2019-11-25 17:14:20 +00:00
Aaron Boxer
348494fd34 msdk: query hardware when checking MFX availability
msdk plugin is not used for sofware encode/decode as there are better
solutions available. Also, with MFX_IMPL_AUTO_ANY, if software decode
is not supported, the plugin will still load, but will then fail when trying to
run the (autoplugged) pipeline. With MFX_IMPL_HARDWARE_ANY,
the plugin fails and a better software decoder is auto-plugged.
2019-11-22 05:18:42 +00:00
Aaron Boxer
2f1256ada7 msdk: query platform when opening session 2019-11-22 05:18:42 +00:00
Seungha Yang
0cf67c3be7 nvenc: Fix crash when nvenc was reused then freed without encoding
GstNvBaseEnc::n_bufs was set from the previous encoding session
but it wasn't cleared after stop. That might result to invalid memory
access at the next start (no encoded data) and then stop sequence.
Instead of defining a variable for array length, use GArray::len
directly to avoid such confusion.
2019-11-22 03:02:57 +00:00
Seungha Yang
aef414375a nvenc: Remove unused code path
refilling queue would not happen
2019-11-22 03:02:57 +00:00
Julien Isorce
a7dc407041 msdkdec: fix surface leak in msdkdec_handle_frame
Can be reproduced with:
  videotestsrc ! x264enc key-int-max=$N ! \
  h264parse ! msdkh264dec ! fakesink sync=1

It happens with any gop size but the smaller is the distance N
between key frames, the quicker it is leaking.

Fixes #1023
2019-11-19 09:14:51 -08:00
Andrew Branson
397a9edb35 androidmedia: Add new effects and scene modes to Camera parameters 2019-11-19 07:54:16 +01:00
Aaron Boxer
842bd66bd8 d3dvideosink: use sink dimensions when calculating overlay scaling 2019-11-18 03:32:57 +00:00
Aaron Boxer
20f3f63540 d3dvideosink: use explicit system memory feature in overlay composition caps 2019-11-18 02:29:45 +00:00
Jochen Henneberg
3ecd8666d9 msdk: Fixes for meson include directory setup
In case of pkg-config we need to create the include directories object
from the path using include_directories(). For INTELMEDIASDKROOT or
MFX_HOME we need to add the alternate include path ./include/mfx as
Intel MediaSDK now puts the headers there.
2019-11-17 17:39:44 +00:00
Seungha Yang
e52ed98197 d3d11window: Fix subclass procedure recursion
subclass window precedure should be cleared if it's no more used.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1115
2019-11-14 01:26:09 +00:00
Aaron Boxer
bfe2447a3b d3d11videosink: only destroy loop if external_win_id is false
Loop was created with this condition, so it should also be
destroyed under the same condition
2019-11-12 19:54:46 -06:00
Nicolas Dufresne
80b599c81b kmssink: Do not drain if imported buffer are from KMS
This adds a check to avoid draining when the imported buffers are in
fact own by kmssink. This happens since we export our kms buffer as
DMABuf. They are not really imported back as we pre-fill the cache,
but uses the same format as if they were external. This fixes
performance issues seen with videocrop2-test (found in -good).
2019-11-11 22:20:28 +00:00
Nicolas Dufresne
8574154cc8 kmssink: Avoid drain on caps changes
Draining systematically on caps changes was a hack. Instead, properly
save the render information used to render last_render, and use that
information to drain. This fixes performance issues met with video crop
meta and per frame caps changes.
2019-11-11 22:20:28 +00:00
Nicolas Dufresne
a83e0036ea kmssink: Ensure we have an allocator before importing
This fixes cases where the kms allocator API was called with a null
pointer.
2019-11-11 22:20:28 +00:00
Xavier Claessens
7d31f7a9e5 amc: Change plugin name if it's built for Magic Leap 2019-11-08 13:40:14 -05:00
Xavier Claessens
90223a2702 amc: Add MLSDK implementation 2019-11-08 13:40:14 -05:00
Haihao Xiang
dca3e47bce msdk: remove msdkvp8enc
MediaSDK doesn't support vp8 encode which is not going to be
implemented [1], so remove msdkvp8enc from this plugin

[1]: https://github.com/Intel-Media-SDK/MediaSDK/issues/947
2019-11-08 08:44:46 +00:00
Niels De Graef
d8f61515d8 Don't pass default GLib marshallers for signals
By passing NULL to `g_signal_new` instead of a marshaller, GLib will
actually internally optimize the signal (if the marshaller is available
in GLib itself) by also setting the valist marshaller. This makes the
signal emission a bit more performant than the regular marshalling,
which still needs to box into `GValue` and call libffi in case of a
generic marshaller.

Note that for custom marshallers, one would use
`g_signal_set_va_marshaller()` with the valist marshaller instead.
2019-11-06 14:27:46 +00:00
Ignacio Casal Quinteiro
a87ea01992 wasapi: minor cleanup 2019-11-06 08:18:59 +00:00
Aaron Boxer
86b01bb291 d3dvideosink: support OverlayComposition for GPU overlay compositing 2019-11-06 01:05:33 +00:00
Aaron Boxer
6d3429af34 documentation: fixed a heap o' typos 2019-11-05 09:11:25 -05:00
Roman Shpuntov
f84ebbd95f vtdec: fix leak of CMFormatDescription
There was consideration for replacing the CMFormatDescription used
on format changes in `set_format()` however on shutdown, we were
leaking the CMFormatDescription at the end of processing.

'https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1106'
2019-11-04 12:12:57 +00:00
Roman Shpuntov
b0543d6ea5 vtdec: memory leaks bugfix
release 'videoDecoderSpecification' variable in 'gst_vtdec_create_session' function.
release 'extensions' variable in 'create_format_description_from_codec_data' function.
2019-11-02 13:58:56 +00:00
Haihao Xiang
1dd3ef24b1 msdkmjpegenc: convert UYVY to YUY2 instead of NV12
Before this fix, the chroma subsampling of the output is 4:2:0. It is
4:2:2 with this fix, which is better for UYVY input
2019-11-02 09:52:36 +00:00
Haihao Xiang
e1b8b81e84 msdkmjpegenc: don't need conversion for BGRA and YUY2
jpeg encoder in MSDK can accept BGRA and YUY2 input.
2019-11-02 09:52:36 +00:00
Haihao Xiang
fbeaa37b79 msdkenc: add need_conversion method
In future, a sub class of GstMsdkEncClass may decide a native format by
using this method, e.g. JPEG encoder may accept YUY2 input, however the
current implemation needs a conversion from YUY2 to NV12 before encoding.

In addtion, a sub class may choose a format for encoding if the input
format is not supported by MSDK, e.g. the current implemation does
UYVY->NV12 if the input format is UYVY. We may do UYVY->YUY2 for JPEG
encoder in future
2019-11-02 09:52:36 +00:00
Haihao Xiang
afce02b392 msdk: support for MFX_FOURCC_BGR4 frame allocation
MFX_FOURCC_BGR4 is mapped to VA_FOURCC_ABGR and JPEG encoder needs a
MFX_FOURCC_BGR4 frame for internal usage when the input format is
MFX_FOURCC_RGB4

This is a preparation for supporting native formats of JPEG encoder
2019-11-02 09:52:36 +00:00
Víctor Manuel Jáquez Leal
10d4c0c511 msdkdec: use decoder base class packetized flag
Instead of using a proxy of `is_packetized` flag this patch
replaces it with the accessor to that flag in decoder base class,
avoiding probable mismatches.
2019-10-29 14:58:54 +00:00
Haihao Xiang
cd883427e9 msdkdec: add parse callback for non-packetized input
commit 55c0d720 added the capability to handle non-packetized bitstream,
and there is a loop to handle multiple frames in a non-packetized buffer
in gst_msdkdec_handle_frame. However it is possible that a
non-packetized buffer still contains valid data but there is no long any
pending unfinished frame. Currently gst_video_decoder_decode_frame is
invoked to send a new frame with new input data, the situaltion is
repeated till an EOS is received. An application has to exit when
receiving an EOS, however there is still valid data in a
non-packetezied input buffer, hence some frames are dropped.

This fix adds a parse callback for non-packeteized input, a new frame
will be sent to the subclass as soon as the input buffer has valid data

This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/665
2019-10-29 14:58:54 +00:00
Haihao Xiang
a2cafcfc3e msdkh265enc: Add support for tiled encoding
Add num-tile-rows and num-tile-cols properties to set the number of rows
and columns
2019-10-27 18:40:15 +00:00
Haihao Xiang
9fcd3bdd2b msdk: support P010_10LE in DMABuf mode for VP9/HEVC encoding 2019-10-25 16:54:57 +00:00
Cheng-Chang Wu
b5e506d12b androidmedia: Make sure that the first member of GstAmcSurfaceTextureJNI is correct
It's a subclass of GstAmcSurfaceTexture so should have that as first
struct member or otherwise it won't work.

Fixes #1105
2019-10-25 13:56:13 +00:00
Cheng-Chang Wu
e71f620eb0 androidmedia: Call JNI method with the actual Java object instead of our wrapper struct
Fixes #1105
2019-10-25 13:56:13 +00:00
Haihao Xiang
351a3934ec msdkvpp: add support Y410 and Y210 formats
And clean up some macros
2019-10-24 13:57:51 +00:00
U. Artie Eoff
b7ee6dc49a msdkvpp: allow cropping via properties
Add crop-left, crop-right, crop-top and crop-bottom
properties to msdkvpp and apply to input surface at
vpp transform run-time.

Implements #1097
2019-10-24 12:44:49 +00:00
Javier Celaya
8684dffe50 Fix get_supported_types function name 2019-10-16 06:42:37 +00:00
Javier Celaya
f267adeb4a Fix releasing local refs 2019-10-16 06:42:37 +00:00
Javier Celaya
3f0d60515a Fix names of MediaCodecInfo inner classes 2019-10-16 06:42:37 +00:00
Javier Celaya
26ea6d7fbb Fix getting MediaCodecList static methods 2019-10-16 06:42:37 +00:00
Julien Isorce
5de00b9ffb msdkenc: close encoder upon bitrate change
Upon bitrate change, make sure to close the encoder otherwise
the encoder is not re-initialized and the target bitrate is
never reached, and the encoder was flushed at each frame
from this moment.

Regression introduced in f2b35abcab which replaced the call
that was closing the encoder by an early return to avoid
re-initialization.
2019-10-15 15:53:20 -07:00
Seungha Yang
76654539b9 wasapi: Fix build warnings
gstwasapiutil.c(173) : warning C4715: 'gst_wasapi_device_role_to_erole': not all control paths return a value
gstwasapiutil.c(188) : warning C4715: 'gst_wasapi_erole_to_device_role': not all control paths return a value
2019-10-14 14:43:59 +00:00
Seungha Yang
baaa965ebc wasapi: Don't cast GstDeviceProvider to GstElement
The GstDeviceProvider isn't subclass of GstElement.

(gst-device-monitor-1.0:49356): GLib-GObject-WARNING **: 20:21:18.651:
invalid cast from 'GstWasapiDeviceProvider' to 'GstElement'
2019-10-14 14:43:59 +00:00
Tim-Philipp Müller
f218ec2794 Remove autotools build system 2019-10-14 13:54:27 +01:00
Matthew Waters
4d528776b6 decklink: fix timestamp tracking when pausing
play->pause->sleep()->play would result in late frames being scheduled
2019-10-14 19:22:36 +11:00
Haihao Xiang
e5279cd97a msdk: return the right pointer
The first channel in memory for MFX_FOURCC_RGB4 (VA_FOURCC_ARGB or
GST_VIDEO_FORMAT_BGRA) is B, not A. In MSDK, channle B is used to access
data for RGB4 surface. In addition, the returned pointers for
MFX_FOURCC_AYUV and MFX_FOURCC_Y410 in gst_msdk_video_memory_map_full
were wrong too before this fix.
2019-10-11 14:07:17 +00:00
Josep Torra
f2c75d548d msdkenc: handle bitrate changes in playing state
When the bitrate is changed in playing state the encoder issues a reconfig
that drains and recreates the underlaying hw encoder instance.
With this set of changes we ensure that all this work is only made when
the bitrate did actually change. It also tries to reuse the vpp buffer
pool and fixes the pool leak spotted when testing this feature.
2019-10-08 06:37:48 -07:00
Haihao Xiang
4525f8707f msdkh265enc: add the missing profile string 2019-10-08 09:40:06 +08:00
Seungha Yang
5e8755cf07 d3d11window: Protect registration/creation of WIN32 window
Querying existing window class info and creation of window object
should be protected since they are not thread safe.
2019-10-04 21:15:25 +09:00
Seungha Yang
44cb5b4ff7 d3d11videosink: Don't try to post error message by d3d11window self
The d3d11window isn't GstElement. To post error message, proxy it
to d3d11videosink instead.
2019-10-04 21:15:15 +09:00
Víctor Manuel Jáquez Leal
5140bf1a1f msdkenc: guard MFX_FOURCC_Y410 with MFX version 1027 2019-10-03 13:50:15 +00:00
Haihao Xiang
71ff8236b5 msdkdec: set lockable flag
When postpone_free_surface is TRUE, the output buffer is not writable,
however the base decoder needs a writable buffer as output buffer,
otherwise it will make a copy of the output buffer. As the underlying
memory is always lockable, so we may set the LOCKABLE flag for this buffer
to avoid buffer copy in the base class.

The refcount of the output buffer is 1 when postpone_free_surface is
FALSE, so needn't set the LOCKABLE flag for this case.
2019-09-30 18:15:38 +00:00
Haihao Xiang
83100b3504 msdkh265enc: add support for Y410 in sink pad
Sample pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=Y410 !
msdkh265enc low-power=1 ! filesink location=output.h265
2019-09-30 10:44:50 +08:00
Haihao Xiang
0507547646 msdkh265enc: add support for VUYA in sink pad
sample pipelie:
gst-launch-1.0 videotestsrc ! video/x-raw,format=VUYA !
msdkh265enc low-power=1 ! filesink location=output.h265
2019-09-30 10:44:38 +08:00
Haihao Xiang
85914997f5 msdkenc: clean up code by using switch
It is convenient for us to add support for new formats in future, there
is no change in functionality.
2019-09-30 10:42:41 +08:00
Haihao Xiang
0c1c2ce45a msdk: remove some dead code 2019-09-30 09:10:31 +08:00
Seungha Yang
22d8d77b13 d3d11videosink: Use pixel aspect ratio to setup window draw area
... instead of calculated display ratio from given PAR and DAR.

d3d11window calculates output display ratio
to decide padding area per window resize event. In the formula,
actual PAR is required to handle both 1:1 PAR and non-1:1 PAR.
2019-09-26 10:23:46 +00:00
Haihao Xiang
ce4f9ae531 msdk: fix for mfx frame alloc response
Both MSDK and this plugin use mfxFrameAllocResponse for video and DMABuf
memory, it is possible that some GST buffers are still in use when calling
gst_msdk_frame_free, so add a reference count in the wrapper of
mfxFrameAllocResponse (GstMsdkAllocResponse) to make sure the underlying
mfx resources are still available if the corresponding buffer pool is in
use.

In addtion, currently all allocators for input or output share the same
mfxFrameAllocResponse pointer in an element, so it is possible that
the content of mfxFrameAllocResponse is updated for a new caps then all
GST buffers allocated from an old allocator will use this new content of
mfxFrameAllocResponse, which will result in unexpected behavior. In this
fix, we save the the content of mfxFrameAllocResponse in the corresponding
tructure to avoid such issue

Sample pipeline:

gst-launch-1.0 filesrc location=vp9_multi_resolutions.ivf ! ivfparse ! msdkvp9dec !
msdkvpp ! video/x-raw\(memory:DMABuf\),format=NV12 ! glimagesink
2019-09-23 09:58:28 +08:00
Haihao Xiang
e50011c064 msdk: don't cache mfxFrameAllocResponse pointer
Otherwise it is possible that different wrappers share the same
mfxFrameAllocResponse pointer, so instead of caching the pointer, we may
cache the content of mfxFrameAllocResponse
2019-09-21 19:51:38 +00:00
Haihao Xiang
f840e304f3 msdkdec: postpone surface free for VC1
For a skipped frame in VC1, MSDK returns the mfx surface of the reference
frame, so we have to make sure the corresponding surface for the
reference frame is not freed. In this fix, we postpone surface free because
we don't know whether a surface is referenced

Before this fix, the error is like as below:

New clock: GstSystemClock
0:00:00.181793130 23098 0x55f8a9d622d0 ERROR                msdkdec
gstmsdkdec.c:622:gst_msdkdec_finish_task:<msdkvc1dec0> Couldn't find the
cached MSDK surface

Sample pipeline:
gst-launch-1.0 filesrc location=input_has_skipped_frame.wmv ! asfdemux !
vc1parse ! msdkvc1dec ! glimagesink
2019-09-21 18:05:42 +00:00
Haihao Xiang
c689c94458 msdkdec: release the surface if this surface is not in use
If the surface is not in use, we may release it even if GST_FLOW_OK is going
to be returned, which may avoid the issue of failing to get surface
available

This fixes the regression caused by commit c05acf4
2019-09-21 17:34:23 +00:00
Haihao Xiang
8daac1c09a msdk: adjust the stride align
GstAllocationParams::align is set to 31 in msdkdec/msdken/msdkvpp, hence
the stride align should be greater than or equal to 31, otherwise it
will result in issue
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/861
(msdk: "GStreamer-CRITICAL: gst_buffer_resize_range failed" SPAM),

In addition, the stride should match the pitch alignment in the media driver,
otherwise it will result in some issues when a buffer is shared between
different elements, e.g. the NV12 issue mentioned in commit 3f2314a, which
can be reproduced by `gst-launch-1.0 vidoetestsrc ! msdkvpp !
video/x-raw\(memory:DMABuf\),format=NV12 ! glimagesink`

Fixed https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/861
2019-09-21 16:59:10 +00:00
Seungha Yang
e0b2d34237 d3d11: Use g_cond_broadcast() to wake up waiting threads
gst_d3d11_device_thread_add() can be called concurrently from different
threads so the cond signal should reach to all waiting threads.
2019-09-18 18:47:44 +09:00
Haihao Xiang
0b0d5afb53 msdkenc: remove unneeded label 2019-09-16 10:09:00 +00:00
Matthew Waters
52cd9846eb applemedia/videocache: remove unimplemented/used upload function 2019-09-16 03:15:39 +00:00
Haihao Xiang
c05acf40a8 msdkdec: set pointer to NULL to avoid freeing an used surface
The surface should be cached in the surface list when GST_FLOW_OK is
going to be returned

This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1051
2019-09-13 17:32:19 +00:00
Haihao Xiang
38c55de132 msdk: free mfx frames to avoid memory leaks in encoder and vpp 2019-09-13 16:54:21 +00:00
Haihao Xiang
2d4b0a5649 msdkdec: unref the caps to avoid memory leaks 2019-09-13 16:54:21 +00:00
Haihao Xiang
cc6ca4ec61 msdkvpp: free msdk surfaces to avoid memory leak 2019-09-13 16:54:21 +00:00
Haihao Xiang
93f72a5041 msdkvpp: unref buffer pools
Unref the pools first in caps re-negotiation to avoid memory leak
2019-09-13 16:54:21 +00:00
Haihao Xiang
3b171f70af msdk: fix memory leaks in msdk allocators 2019-09-13 16:54:21 +00:00
Fuwei Tang
9b2dc96b99 msdkenc: work-around to avoid zero fps in MediaSDK structure 2019-09-13 16:28:00 +00:00
Fuwei Tang
11bbd6f721 msdkenc: add an extra surface for hevc encoding
For some hevc 10bit 4K encoding cases, the encoding process may be
slow, and MediaSDK surface can't be released in time before one other
available surface is needed. So add an extra surface for hevc encoding
to avoid this issue.
2019-09-12 02:20:11 +00:00
Seungha Yang
52dfbbe5da nvenc: Early terminate handle_frame if the last flow was not GST_FLOW_OK
If the last flow was not GST_FLOW_OK, the encoding thread is not running
and there is nothing to pop from GAsyncQueue (this causes deadlock).

To prevent deadlock, just return the handle_frame without further encoding
process if the last flow was not GST_FLOW_OK. Note that the last flow
will be cleared per FLUSH_STOP and STREAM_START event.
2019-09-11 15:21:03 +00:00
Seungha Yang
68a51abdcd nvenc: Add support VUYA format
The addition is very simple. Map NV_ENC_BUFFER_FORMAT_AYUV format
to GST_VIDEO_FORMAT_VUYA and add a condition for the VUYA format.
2019-09-11 14:33:54 +00:00
Seungha Yang
14b9a1cffd nvdec: Add support for mpeg4 video decoding with codec_data
Decoder should handle codec_data of mpeg4 video which includes essential
config data.
2019-09-11 14:00:48 +00:00
Seungha Yang
af77988b9f nvenc: Reduce the number of pre-allocated device memory
The hard-coded upper bound 32 (or 48 depending on resolution) might waste
GPU memory and high resolution encoding causes OUT-OF-MEMORY allocation error
quite easily. This commit calculates the number of required pre-allocated
device memory based on encoding options and it can reduce the amount of device memory
used by nvenc.
2019-09-11 11:44:03 +00:00
Seungha Yang
e3508a4f26 nvdec: Update plugin description and fix typo
Use consistent description with nvenc, and fix typo s/devide/device/g
2019-09-11 15:16:45 +09:00
Seungha Yang
1cbb23cf79 nvenc: Adjust DTS when bframe is enabled
NVDEC driver always uses input timestamp without adjustment
even if bframe encoding was enabled.
So DTS can be larger than PTS when bframe was enabled.
To ensure PTS >= DTS, we should adjust the timestamp manually
based on the PTS difference between the first
encoded frame and the second one. That's also the maximum PTS/DTS
difference.
2019-09-11 13:18:12 +09:00
Seungha Yang
83a1c7a9a6 nvenc: Add qp-{min,max,const}-{i,p,b} properties
This new properties allows more detailed target QP value setting
2019-09-11 13:18:12 +09:00
Seungha Yang
d3a909ccdd nvenc: Add properties to support bframe encoding if device supports it
Note that bframe encoding capability varies with GPU architecture
2019-09-11 13:18:12 +09:00
Seungha Yang
94f2843774 nvenc: Refactoring internal buffer pool structure
To support rc-lookahead and bframe encoding, nvenc needs one more
staging queue, because NvEncEncodePicture can return NV_ENC_ERR_NEED_MORE_INPUT
but which was not considered so far.
As documented by NVENC programming guide, pending buffers should wait
other inputs until NvEncEncodePicture returns success.

New encoding flow is
- Submit raw picture buffer to encoder with NvEncEncodePicture
- The submitted input/output buffer pair will be queued to pending_queue
  - If NvEncEncodePicture returned success, then move all pair in pending_queue
    to final stage
  - Otherwise, wait more input raw pictures.

Another change is dropping NV_ENC_LOCK_INPUT_BUFFER usage.
So now nvenc always uses CUDA memory input buffer. As a result,
both opengl and system memory handling are unified.
2019-09-11 13:18:12 +09:00
Seungha Yang
e73acbaa5c nvenc: Remove pointless iteration and cleanup some code
* The number of iteration is always one so the iteration is useless
and that makes code complicated.
* Also defining named structure can code mroe readable.
* g_free is null safe
2019-09-11 13:18:12 +09:00
Seungha Yang
81272eaa82 nvenc: Add more rate-control options
New rate-control modes are introduced (if device can support)
* cbr-ld-hr: CBR low-delay high quality
* cbr-hq: CBR high quality
* vbr-hq: VBR high quality

Also, various configurable rate-control related properties are added.
2019-09-11 13:18:12 +09:00
Seungha Yang
ea19a7c715 nvenc: Add support for weighted prediction option
Note that this property will be exposed only if the device
supports the weighted prediction.
2019-09-11 13:18:12 +09:00
Seungha Yang
d05cbdbd72 nvenc: Add property for AUD insertion
Make AUD insertion configurable option
2019-09-11 13:18:12 +09:00
Seungha Yang
b3b723462e nvenc: Refactor class hierarchy to handle device capability dependent options
Introducing new dynamic class between GstNvBaseEncClass and
each subclass to be able to access device specific properties and
capabilities from each subclass implementation side.
2019-09-11 13:18:09 +09:00
Mathieu Duponchelle
f402cb07f9 msdk: actually use the include dir we compute 2019-09-11 00:14:08 +00:00
Marc Leeman
3ef503346a nvcodec: minor spell corrects in log messages 2019-09-10 23:13:17 +00:00
Seungha Yang
54c5ebbd53 msdk: Remove all DMABuf caps features on Windows
Add new macro for sink/src pad template to ensure no DMABuf caps
features are exposed on Windows. Some DMABuf caps features
were not handled by the commit 9ec62418c3
2019-09-10 13:29:11 +00:00
Seungha Yang
eb3a117548 msdkdec: Do not use video memory on Windows
Like msdkenc, do not use video memory by default on Windows.
2019-09-10 13:29:11 +00:00
Seungha Yang
d4112d3d7b msdk: Allow video and system memory share among buffers
gst_buffer_make_writable() requires exclusive reference to the
GstMemory so the _make_writable() for the msdk buffer will result
to fallback system memory copy, because the msdk memory were initialized
with GST_MEMORY_FLAG_NO_SHARE flag.

Note that, disable sharing GstMemory brings high overhead but actually
the msdk memory objects can be shared over multiple buffers.
If the memory is not shareable, newly added GstAllocator::mem_copy will
create copied msdk memory.
2019-09-10 13:29:11 +00:00
Haihao Xiang
04b77a8c9d msdkh265dec: remove the requirement on profile
Sometimes a HEVC/H265 stream doesn't have a valid profile but MSDK can
handle this stream. Like vaapih265dec, msdkh265dec may advertise the sink
caps without profile
2019-09-10 12:54:39 +00:00
Haihao Xiang
18728310c9 msdkdec: output the decoded frame immediately if decoded order is required
DecodedOrder was deprecated in msdk-2017 version, but some customers
still use this for low-latency streaming of non-b-frame encoded streams,
which needs to output the frame at once
2019-09-04 05:25:13 +00:00
Matthew Waters
82e23a27f7 decklinkaudiosink: Drop late buffers
Asking decklink to render audio data seems to be based entirely on
the sample counts which completely disregards the timestamps
we pass to decklink.  As a result, we need to explicitly check
for late buffers and drop them ourselves.
2019-09-02 11:09:02 +00:00
Seungha Yang
09fd34dbb0 nvenc: Add support runtime resolution change freely
Do not restrict allowed maximum resolution depending on the
initial resolution. If new resolution is larger than previous one,
just re-init encode session.
2019-09-02 10:59:03 +09:00
Matthew Waters
02814a43da decklink: fix macos werror build
../sys/decklink/gstdecklink.cpp:1703:7: error: format specifies type 'long' but the argument has type 'int64_t' (aka 'long long') [-Werror,-Wformat]
      persistent_id);
      ^~~~~~~~~~~~~~
/Library/Frameworks/GStreamer.framework/Versions/1.0/include/gstreamer-1.0/gst/gstinfo.h:1070:87: note: expanded from macro 'GST_DEBUG'
#define GST_DEBUG(...)                  GST_CAT_LEVEL_LOG (GST_CAT_DEFAULT, GST_LEVEL_DEBUG,   NULL, __VA_ARGS__)
                                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
/Library/Frameworks/GStreamer.framework/Versions/1.0/include/gstreamer-1.0/gst/gstinfo.h:646:31: note: expanded from macro 'GST_CAT_LEVEL_LOG'
        (GObject *) (object), __VA_ARGS__);                             \
                              ^~~~~~~~~~~
2019-08-30 14:16:54 +10:00
Seungha Yang
fa83f086be nvdec: Check flow return of the only current handle_frame() to fix seeking issue
Due to uncleared last flow, decoding after seek was never possible
(last_ret == GST_FLOW_FLUSHING).
nvdec dose not need to keep track of the previous flow return,
and actually the interest is data/even flow of the current handle_frame().
2019-08-30 11:27:27 +09:00
Seungha Yang
be3a3da829 nvdec: Fallback to system memory if OpenGL context could not support PBO memory
If the environment could not support OpenGL PBO memory, nvdec will do negotiation
with system memory as fallback.
2019-08-30 01:36:46 +09:00
Seungha Yang
069fe93452 nvdec: Add support dynamic output format change
Implementing ::negotiate() method to support runtime output format
change. If downstream was reconfigured, baseclass will invoke
::negotiate() method, and nvdec should update output memory
type depending on downstream caps.
2019-08-30 01:36:46 +09:00
Seungha Yang
39f800c449 nvdec: Re-negotiate whenever output format is changed
Input stream might be silently changed without ::set_format() call.
Since nvdec has internal parser, nvdec element can figure out the format change
by itself.
2019-08-30 01:36:41 +09:00
Seungha Yang
f4f8941a91 nvdec: Add support 4:4:4 and 4:2:0 12bit decoding
Depending on GPU architecture, HEVC decoder can support
4:4:4 format up to 12 bitdepth. This commit covers VP9 4:2:0 12 bits
decoding also.
2019-08-29 13:39:59 +00:00
Seungha Yang
ff9838fd3d nvenc: Add support for old drivers which could not understand SDK version 9.0
Add helper functions to support old drivers
with our previous SDK version 8.1
2019-08-29 13:39:59 +00:00
Seungha Yang
afebb15d99 nvenc: Use consistent snake case convention 2019-08-29 13:39:59 +00:00
Seungha Yang
1010b9f567 nvcodec: Bump SDK header to version 9.0
The latest Turing architecture (e.g., RTX serise) can support
decoding HEVC 4:4:4 format up to 12bits.
2019-08-29 13:39:59 +00:00
Seungha Yang
338a32b672 nvenc: Port to GstCudaGraphicsResource
Register openGL resource only once per memory. Also if upstream
provides the registered information, reuse the information
instead of doing it again. This can improve performance dramatically
depending on system since the resource registration might cause
high overhead.
2019-08-29 18:45:25 +09:00
Seungha Yang
d0846f8eab nvdec: Port to GstCudaGraphicsResource
Make it possible to share registered graphics resource among nvidia encoders
and decoders.
2019-08-29 18:05:51 +09:00
Seungha Yang
da075b94a9 cudautils: Add GstCudaGraphicsResource structure for better openGL interoperability
Introduce GstCudaGraphicsResource structure to represent registered
CUDA graphics resources and to enable sharing the information among
nvdec and nvenc. This structure can reduce the number of resource
registration which cause high overhead.
2019-08-29 18:04:33 +09:00
Seungha Yang
8dc2b4a393 nvdec: Port to openGL PBO memory
For openGL interoperability, nvdec uses cuGraphicsGLRegisterImage API
which is to register openGL texture image.
Meanwhile nvenc uses cuGraphicsGLRegisterBuffer API to registure openGL buffer object.
That means two kinds of graphics resources are registered per memory
when nvdec/nvenc are configured at the same time.
The graphics resource registration brings possibly high overhead
so the registration should be performed only once per resource
from optimization point of view.
2019-08-29 18:04:33 +09:00
Seungha Yang
9bfd6d13e6 nvdec: Filter openGL API version to use
To ensure PBO buffer, openGL API >= 3 is required.
2019-08-29 18:04:29 +09:00
Haihao Xiang
4841c2759d msdk: avoid reading data from freed memory
Both g_list_delete_link and g_list_remove remove an element and free it,
so l->next is invalid (catched by valgrind) after calling g_list_delete_link
or g_list_remove
2019-08-28 14:51:54 +00:00
Haihao Xiang
ffa82e07a7 msdkvpp: don't return GST_FLOW_ERROR for MFX_WRN_INCOMPATIBLE_VIDEO_PARAM
Returning MFX_WRN_INCOMPATIBLE_VIDEO_PARAM means MSDK detects some
incompatible parameters but it is resolved, and we may not regard
MFX_WRN_INCOMPATIBLE_VIDEO_PARAM as a fatal error. In this fix,
GST_FLOW_OK is returned but with a warning message so that a pipeline
may run to the end.
2019-08-28 09:37:06 +00:00
Matthew Waters
c941db0c2e ahcsrc: #define GST_USE_UNSTABLE_API for phtography inteface
Fixes werror build:

In file included from ../sys/androidmedia/gstahcsrc.c:70:
../gst-libs/gst/interfaces/photography.h:27:2: error: "The GstPhotography interface is unstable API and may change in future." [-Werror,-W#warnings]
#warning "The GstPhotography interface is unstable API and may change in future."
 ^
../gst-libs/gst/interfaces/photography.h:28:2: error: "You can define GST_USE_UNSTABLE_API to avoid this warning."  [-Werror,-W#warnings]
#warning "You can define GST_USE_UNSTABLE_API to avoid this warning."
 ^
2019-08-28 16:12:11 +10:00
Matthew Waters
6e9d0ac61b uvch264: fix werror build with clang
../subprojects/gst-plugins-bad/sys/uvch264/gstuvch264deviceprovider.c:48:1: warning: unused function 'GST__UVC_H264_DEVICE' [-Wunused-function]
G_DECLARE_FINAL_TYPE (GstUvcH264Device, gst_uvc_h264_device, GST_,
^
/usr/include/glib-2.0/gobject/gtype.h:1405:33: note: expanded from macro 'G_DECLARE_FINAL_TYPE'
  static inline ModuleObjName * MODULE##_##OBJ_NAME (gpointer ptr) {                                     \
                                ^
<scratch space>:75:1: note: expanded from here
GST__UVC_H264_DEVICE
^
../subprojects/gst-plugins-bad/sys/uvch264/gstuvch264deviceprovider.c:48:1: warning: unused function 'GST__IS_UVC_H264_DEVICE' [-Wunused-function]
/usr/include/glib-2.0/gobject/gtype.h:1407:26: note: expanded from macro 'G_DECLARE_FINAL_TYPE'
  static inline gboolean MODULE##_IS_##OBJ_NAME (gpointer ptr) {                                         \
                         ^
<scratch space>:78:1: note: expanded from here
GST__IS_UVC_H264_DEVICE
^
../subprojects/gst-plugins-bad/sys/uvch264/gstuvch264deviceprovider.c:139:1: warning: unused function 'GST_UVC_H264_DEVICE_PROVIDER' [-Wunused-function]
G_DECLARE_FINAL_TYPE (GstUvcH264DeviceProvider, gst_uvc_h264_device_provider, GST, UVC_H264_DEVICE_PROVIDER, GstDeviceProvider)
^
/usr/include/glib-2.0/gobject/gtype.h:1405:33: note: expanded from macro 'G_DECLARE_FINAL_TYPE'
  static inline ModuleObjName * MODULE##_##OBJ_NAME (gpointer ptr) {                                     \
                                ^
<scratch space>:137:1: note: expanded from here
GST_UVC_H264_DEVICE_PROVIDER
^
../subprojects/gst-plugins-bad/sys/uvch264/gstuvch264deviceprovider.c:139:1: warning: unused function 'GST_IS_UVC_H264_DEVICE_PROVIDER' [-Wunused-function]
/usr/include/glib-2.0/gobject/gtype.h:1407:26: note: expanded from macro 'G_DECLARE_FINAL_TYPE'
  static inline gboolean MODULE##_IS_##OBJ_NAME (gpointer ptr) {                                         \
                         ^
<scratch space>:140:1: note: expanded from here
GST_IS_UVC_H264_DEVICE_PROVIDER
^
2019-08-28 02:47:18 +00:00
Nirbheek Chauhan
733f5b2851 wasapi: Move to CoInitializeEx for COM initialization
CoInitialize is not allowed when targeting UWP and causes a Windows
Application Certification Kit (WACK) error.
2019-08-27 19:10:36 +00:00
Xavier Claessens
121711e4ac amc: Do not skip decoders that have no profile levels 2019-08-27 02:23:41 +00:00
Haihao Xiang
d5fd9bc22b msdkvpp: support video-direction property
video-direction property is common property in gstreamer. In addition,
both mirroring & rotation properties are marked as deprecated,
video-direction will override mirroring & rotation properties when they
are set explicitly

Fix https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1058
2019-08-26 16:39:39 +00:00
Yan Wang
f92b5614db msdkdec: Fix buffer allocation based on frame.
gst_msdkdec_finish_task() may release all frames in
GstVideoDecoder object. In this case, allocate_output_buffer()
cannot get the oldest frame to allocate buffer.
So gst_msdkdec_handle_frame() should return GST_FLOW_OK for
letting gst_video_decoder_decode_frame() to send a new frame
for decoding.

Fixes #664.
Fixes #665.
2019-08-26 13:59:16 +00:00
Yeongjin Jeong
e0042a99bf msdk: Don't use VA_RT_FORMAT_YUV420_10 if it's undefined
../sys/msdk/gstmsdkallocator_libva.c:99:16: error: ‘VA_RT_FORMAT_YUV420_10’

The minimum required version seems to VA_API_VERSION >= 1.2.0
2019-08-25 09:40:13 +00:00
gla
c5738c6125 dshowsrcwrapper: fix regression on device selection
Do not take device_name if a device has been specified. Do not take device_index into account if a device or a device name has been specified.
2019-08-23 09:53:59 +00:00
U. Artie Eoff
bd3caf0c04 msdk: vpp: rotate output frame
When vpp rotation is 90 or 270, the output frame
should be rotated, too.

Example:
  gst-launch-1.0 -vf videotestsrc \
   ! video/x-raw,width=720,height=480 \
   ! msdkvpp rotation=90 ! vaapisink
2019-08-22 18:17:30 +00:00
Seungha Yang
807e311ae8 nvdec: Always response QUERY_CONTEXT even if openGL is unavailable on the system
nvdec can response for the CUDA context type query regardless of openGL
availability.
2019-08-21 14:14:07 +09:00
Xavier Claessens
83883718ee amc: Print error when failing to register listener 2019-08-20 08:53:55 -04:00
Xavier Claessens
9ab76c73ea amc: Do not use g_log() for criticals 2019-08-20 08:53:55 -04:00
Xavier Claessens
06fc7880bb amc: crop values are not mandatory in format
Android documentation has example code how to compute width and height
when crop values are present.
https://developer.android.com/reference/android/media/MediaCodec#accessing-raw-video-bytebuffers-on-older-devices
2019-08-20 08:53:55 -04:00
Xavier Claessens
a604796c41 amc: Remove unused gst_amc_surface_texture_set_default_buffer_size() 2019-08-20 08:53:55 -04:00
Xavier Claessens
50cdaf1036 amc: Fix matrix constness in _get_transform_matrix() 2019-08-20 08:53:55 -04:00
Xavier Claessens
ad7c4f5f27 amc: Select between encoder/decoder at GstAmcCodec construct time
Magical 0/1 values where passed to gst_amc_codec_configure() flags
argument. It's more natural to have a boolean is gst_amc_codec_new().
2019-08-20 08:53:55 -04:00
Xavier Claessens
0fe0270244 amc: Remove gst_amc_format_contains_key()
It is not needed, we can just try to get the key and ignore error.
NdkMediaFormat doesn't have that method.
2019-08-20 08:53:55 -04:00
Xavier Claessens
7fcf3ebf07 amc: Turn GstAmcSurfaceTexture into a base class with JNI implementation 2019-08-20 08:53:55 -04:00
Xavier Claessens
29ef89983c amc: Create JNI wrapper for MediaCodecList
There is no NdkMediaCodecList API yet, but it is still better to isolate
JNI code. This will facilitate porting to a native API if Google ever
release one.
2019-08-20 08:53:55 -04:00
Xavier Claessens
515398a9ff amc: Move MediaCodec JNI wrapper into its own module
This will facilitate adding another implementation based on
NdkMediaCodec instead of JNI.
2019-08-20 08:53:55 -04:00
Seungha Yang
4f60117db9 nvdec: Fix possible null object unref
gst_query_get_n_allocation_pools > 0 does not guarantee that
the N th internal array has GstBufferPool object. So users should
check the returned GstBufferPool object from
gst_query_parse_nth_allocation_pool.
2019-08-20 10:14:54 +09:00
Seungha Yang
eab564d857 nvcodec: Use default flag for CUDA stream creation
Since nvdec/nvenc engine is running on default stream,
non-default CUDA stream should be synchronized with default
stream eventually.
2019-08-19 07:13:26 +00:00
Seungha Yang
ca6657367c nvenc: Use non default CUDA stream and async operation
Use CUDA async operation if possible with non default CUDA stream
2019-08-19 01:18:52 +00:00
Seungha Yang
5615e9258f nvdec: Don't use default CUDA stream
Async CUDA operation with default stream (NULL CUstream) is not much
beneficial than blocking operation since all CUDA operations which belong
to the CUDA context will be synchronized with the default stream's operation.
Note that CUDA stream will share all resources of the corresponding CUDA context
but which can help parallel operation similar to the relation between thread and process
2019-08-19 01:18:52 +00:00
Seungha Yang
20d8f54e63 nvdec: Push/Pop CUDA context around library API call 2019-08-19 01:18:52 +00:00
Seungha Yang
f7b2b1b99d nvdec: Fix timestamp mismatch on draining frames
The internal decoding state must be GST_NVDEC_STATE_PARSE before
calling CuvidParseVideoData(). Otherwise, nvdec will be confused
on decode callback as if the frame is decoding only frame and
the input timestamp of corresponding frame will be ignored.
Eventually one decoded frame will have non-increased PTS.
2019-08-18 15:52:32 +09:00
Seungha Yang
b64733972e nvdec: Do not access nvdec object from destroy function of qdata
The destroy callback can be called just before the fìnalization of
GstMiniObject. So the nvdec object might be destroyed already.
Instead, store the GstCudaContext with increased ref to safely
unregister the CUDA resource.
2019-08-16 19:40:31 +09:00
Seungha Yang
736df5604f d3dvideosink: Fix crash on WinProc handler
... caused by null pointer dereference. The d3dvideosink object might
not available yet on the handler.
2019-08-15 17:55:29 +09:00
Seungha Yang
58afcf09b4 meson: d3d11: Remove unnecessary dependency
d3d11 never use any API of gstreamer-allocators-1.0
2019-08-15 16:55:51 +09:00
Seungha Yang
5909a495a7 d3d11videosink: Take into account pixel aspect ratio
Fix unexpected cropping with non 1:1 pixel aspect-ratio.

The actual buffer width/height should be passed to gst_d3d11_window_render(),
instead of the calculated resolution. The width/height
values are parameters for copying d3d11 video memory.
Also, aspect-ratio should be considered on resize callback
to decide render rectangle size.
2019-08-15 16:55:37 +09:00
Xavier Claessens
9b9e39be24 amc: Fix crash when a sync_meta survives its sink
_amc_gl_free() could be called after the GstAmcVideoDec has been
finalized, in the case downstream still has a ref to a buffer.
2019-08-14 16:37:19 +00:00
Ignacio Casal Quinteiro
1181436545 wasapi: fix symbol redefinition build error 2019-08-12 07:20:43 +00:00
Seungha Yang
e6d21d048a nvenc: Add support YV12 format
YV12 format is supported by Nvidia NVENC without manual conversion.
So nvenc is exposing YV12 format at sinkpad template but there is some
missing point around uploading the memory to GPU.
2019-08-09 11:43:22 +09:00
Seungha Yang
8dbaed0af7 nvh265enc: Enable HDR related SEI nal insertion
If upstream provides the HDR related information, create SEI message
nals and pass them to NVENC.
2019-08-08 23:18:14 +09:00
Fuwei Tang
df59db988b msdkdec: fix decoding issue for interlaced streams
Currently h264parser produces a field or a frame for
alignment=au for interlaced streams, but the flag
MFX_BITSTREAM_COMPLETE_FRAME needs a complete frame
or complementary field pair of data, this results in
broken images being output.

Some patches have been sent out to fix h264parser,
but they are pending on some unfinished work. In
order to make gstreamer-msdk decoding work properly
for interlaced streams before h264parser is fixed,
this flag will be removed temporarily and will be
added back once h264parser if fixed.

Related to:
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/399
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/228
2019-08-08 07:55:41 +08:00
Seungha Yang
f3e12a0b56 nvh265enc: Add support YUV 444 10bits encoding
Note that h264 encoder does not support the YUV 444 10bits format
2019-08-08 00:46:16 +09:00
Seungha Yang
fa5e6f546b nvenc: Remove unnecessary constraint from YUV420 10bits capability decision
YUV444 capability shouldn't be applied to YUV420 10 bits format
2019-08-08 00:46:12 +09:00
Sebastian Dröge
99af098a7b decklinkvideosrc: Retrieve mode of the ancillary data from the frame
Instead of using the information we stored ourselves for the video frame
itself. Which was also the wrong one: it was the mode from the property,
not the autodetected one.

This fixes vanc extraction with mode=auto
2019-08-07 10:27:42 +00:00
Sebastian Dröge
2b3a14ff4e decklinkvideosrc: Also set the INTERLACED buffer flag on non-TFF buffers 2019-08-07 10:27:42 +00:00
Aaron Boxer
7dbcddff61 msdkdec: no need to cache output state info 2019-08-07 11:37:28 +03:00
Seungha Yang
cc4d0e91e3 nvenc: Fix broken RGB format support
Add missing format check introduced by the commit 7de4dbdeb2
2019-08-07 07:27:36 +00:00
Seungha Yang
9d0545d1a2 nvcodec: Wrap CUDA API return check with gst_cuda_result
The gst_cuda_result macro function is more helpful for debugging
than previous cuda_OK because gst_cuda_result prints the function
and line number. If the CUDA API return was not CUDA_SUCCESS,
gst_cuda_result will print WARNING level debug message with
error name, error text strings.
2019-08-07 00:59:36 +00:00
Seungha Yang
d69b590683 nvdec: Port to GstCUDAContext
... and drop CUvideoctxlock usage. The CUvideoctxlock basically
has the identical role of cuda context push/pop but nvdec specific
way. Since we can share the CUDA context among encoders and decoders,
use CUDA context directly for accessing GPU API.
2019-08-07 00:59:36 +00:00
Seungha Yang
5cf0351418 nvenc: Port to GstCudaContext
... and add support CUDA context sharing similar to glcontext sharing.
Multiple CUDA context per GPU is not the best practice. The context
sharing method is very similar to that of glcontext. The difference
is that there can be multiple context object on a pipeline since
the CUDA context is created per GPU id. For example, a pipeline
has nvh264dec (uses GPU #0) and nvh264device0dec (uses GPU #1),
then two CUDA context will propagated to all pipeline.
2019-08-07 00:59:36 +00:00
Seungha Yang
094e4a9f5c nvcodec: Introduce NVIDA CUDA helpers
New object and helper functions can remove duplicated code
from nvenc/nvdec. Also this is prework for CUDA device context sharing
among nvdec(s)/nvenc(s).
2019-08-07 00:59:36 +00:00
Sebastian Dröge
ab8bd0aa44 decklinkaudiosrc/decklinkvideosrc: Do nothing in BaseSrc::negotiate() and always set caps in ::create()
We don't support negotiation with downstream but simply set caps based
on the buffers we receive. This prevents renegotiation to other formats,
and negotiation to NTSC in mode=auto in the beginning until the first
buffer is received.

As side-effect of this, also remove various other caps handling code
that was working around the behaviour of the default
BaseSrc::negotiate().
2019-08-06 19:05:38 +00:00
Seungha Yang
7de4dbdeb2 nvenc: Return profile compatible input formats from GstVideoEncoder::getcaps
Do not accept any input formats which could not be supported
by downstream requested codec profiles.
2019-08-06 15:03:22 +00:00
Seungha Yang
9e81f8e700 nvenc: Fix caps negotiation failure on unspecified interlace-mode
During GstVideoInfo conversion from GstCaps, interlace-mode is
inferred to progressive so unspecified interlace-mode should not cause any
negotiation issue. Simly set GST_PAD_FLAG_ACCEPT_INTERSECT flag
on sinkpad to fix issue.
2019-08-06 15:03:22 +00:00
Seungha Yang
b43d0f785c nvenc: Remove unused member variables
Supported interlace-mode and codec profiles are checked
during plugin init and those values are never used.
2019-08-06 15:03:22 +00:00
Haihao Xiang
ef81c9d6c4 msdkdec: Update frame info from video parameters
A 10bit stream may have different depth values for Luma and Chroma, and
MSDK requires the frame info must match the corresponding video
parameters
2019-08-06 09:20:21 +00:00
Seungha Yang
f7f9f327cd nvdec: Respect upstream provided timestamp
Decoder sometimes reports nonincreasing timestamp.
Use input frame's timestamp like other decoder elements.
2019-08-05 20:32:39 +00:00
Seungha Yang
e68bfd7566 nvenc: Add support RGB 8/10bits formats
BGRA/RGBA/RGB10A2/BGR10A2 formats can be supported by nvenc.
Depending on device, supported format can be different.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1038
2019-08-05 18:55:28 +00:00
Seungha Yang
c99b160b50 nvdec: Use upstream framerate if possible
Encoded bitstream might not have valid framerate. If upstream
provided non-variable-framerate (i.e., fps_n > 0 and fps_d > 0)
use upstream framerate instead of parsed one.
2019-08-05 15:32:43 +00:00
Seungha Yang
158b4d8649 nvenc: Fix crash with unspecified framerate
Nvidia driver seems to calculating floating point framerate
without validation. This causes crash both on linux and Windows.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1012
2019-08-05 15:32:43 +00:00
Aaron Boxer
40f7001bef msdkdec: no need to cache allocation_caps 2019-08-05 13:32:15 +00:00
Seungha Yang
2a76807c9a configure: Update for nvcodec dependency change
nvcodec is compilable without external dependency
2019-07-31 15:36:04 +00:00
Seungha Yang
f1cbab7cfd nvdec: Fix build warning error
gstnvdec.c:1222:3: error: implicit declaration of function ‘memset’ [-Werror=implicit-function-declaration]
   memset (&type_info, 0, sizeof (type_info));
   ^~~~~~
2019-07-31 15:36:04 +00:00
Seungha Yang
4fa5a82762 nvenc: Fix build error with x86 msvc
__stdcall is accepted or ignored by the compiler on x64 but x86
is not the case. So the function definition should be consistent
with declaration.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1039
2019-07-30 19:12:46 +09:00
Seungha Yang
0445ed6ba5 nvenc: Fix deadlock when pad_push return was not GST_FLOW_OK
Encoding thread is terminated without any notification so
upstream streaming thread is locked because there is nothing
to pop from GAsyncQueue. If downstream returns error,
we need put SHUTDOWN_COOKIE to GAsyncQueue for chain function
can wakeup.
2019-07-30 17:49:25 +09:00
Seungha Yang
3faf439347 nvcodec: Fix broken ABI in cuda stub header to fix nvenc with opengl
Fix the broken ABI introduced by the commit 367e742e5d
From CUDA Toolkit 3.2, size_t has been used in CUDA_MEMCPY2D structure
instead of unsigned int.
2019-07-30 11:13:18 +09:00
Aaron Boxer
6c7a07bc2a msdk: enable 32 bit build on windows 2019-07-26 11:42:55 +02:00