Commit graph

330 commits

Author SHA1 Message Date
Seungha Yang
0d2664c668 d3d11decoder: Enable zero-copy for Qualcomm
Qualcomm GPU works fine with current implementation now.
Noticeable difference between when it was disabled and current
d3d11 implementation is that we now support GstD3D11Memory
pool, so there will be no more frequent re-binding decoder surface anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2377>
2021-07-05 12:17:55 +00:00
Seungha Yang
65a78effdb d3d11vp9dec: Fix for incorrect use_prev_in_find_mv_refs setting
Set use_prev_in_find_mv_refs depending on context. The value seems
to be used by AMD and Qualcomm (Intel and NVIDIA doesn't make difference
as per test)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2376>
2021-07-05 15:02:53 +09:00
Seungha Yang
ee543bdacf d3d11compositor: Add scaling policy to support PAR-aware scaling
Identical to https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1156
but for D3D11.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2263>
2021-06-30 06:10:29 +00:00
Seungha Yang
44e3399bf8 d3d11: Add AV1 decoder
Introduce Direct3D11/DXVA AV1 decoder element

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2365>
2021-06-29 07:12:27 +00:00
Seungha Yang
7c94b9c4b0 d3d11: Add support for GRAY and more YUV formats
By this commit, following formats will be newly supported by d3d11 elements

* Y444_{8, 12, 16}LE formats:
  Similar to other planar formats. Such Y444 variants are not supported
  by Direct3D11 natively, but we can simply map each plane by
  using R8 and/or R16 texture.
* P012_LE:
  It is not different from P016_LE, but defining P012 and P016 separately
  for more explicit signalling. Note that DXVA uses P016 texture
  for 12bits encoded bitstreams.
* GRAY:
  This format is required for some codecs (e.g., AV1) if monochrome
  is supported
* 4:2:0 planar 12bits (I420_12LE) and 4:2:2 planar 8, 10, 12bits
  formats (Y42B, I422_10LE, and I422_12LE)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2346>
2021-06-23 15:35:36 +00:00
Seungha Yang
a63539b213 d3d11decoder: Don't print error log when no DPB texture is available
... but we are flushing. The condition is quite expected situation
when pipeline is in the middle of seeking operation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2320>
2021-06-14 07:02:20 +00:00
Seungha Yang
74f81a1a13 d3d11: Add support for YV12 and NV21 formats
Handle UV swapped 4:2:0 8bits formats

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2290>
2021-06-03 17:00:49 +00:00
Seungha Yang
73067bfe0c d3d11window_win32: Ensure closing internal HWND from window thread
Window handle must be closed from its own message thread

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2302>
2021-06-03 15:20:27 +00:00
Seungha Yang
b1541a7470 d3d11: Suppress some warning logs
We uses gst_d3d11_device_new() for enumerating device which can
fail for some reason. Don't print warning log for the case.
And decoding capability check is the same case as well.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2286>
2021-05-29 10:30:28 +00:00
Seungha Yang
fed06df5f6 d3d11compositor: Reuse converter on alpha update
... instead of creating converter object

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2276>
2021-05-28 21:44:12 +09:00
Seungha Yang
0d34a0233b d3d11converter: Introduce config to be extensible
Add a config argument like that of GstVideoConverter so that
we can add more options without modifying existing methods

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2276>
2021-05-28 21:44:10 +09:00
Seungha Yang
f3331652f2 d3d11shader: Don't hold state object in GstD3D11Quad
We might want to update state object

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2276>
2021-05-28 21:44:07 +09:00
Seungha Yang
da33921f8d d3d11compositor: Fix caps update handling
New caps is applied only when previous buffer is consumed if any.
So, the lastest given caps might not be corresponding to the current buffer
to be handled.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2284>
2021-05-26 16:42:39 +09:00
Seungha Yang
8132958b3b d3d11desktopdupsrc: Add support for desktop size/rotation mode change
Re-negotiates with updated size on desktop size
(i.e., resolution, scaling factor), and rotation mode change

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2268>
2021-05-20 17:05:24 +00:00
Seungha Yang
4872b41448 d3d11compositor: Fix missing D3D11 prefix
Fix typo, no functional change

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2262>
2021-05-20 00:51:08 +09:00
Seungha Yang
1039d70d18 d3d11desktopdup: Don't ignore error DXGI_ERROR_UNSUPPORTED
Although Microsoft's DXGIDesktopDuplication example is considering
the DXGI_ERROR_UNSUPPORTED as an expected error
(See https://github.com/microsoft/Windows-classic-samples/tree/master/Samples/DXGIDesktopDuplication)
it might not be recoverable error if application is
run against a discrete GPU
(See https://docs.microsoft.com/en-US/troubleshoot/windows-client/shell-experience/error-when-dda-capable-app-is-against-gpu)

Do early error out if the error happens while opening device,
instead of retrying it forever.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2208>
2021-04-30 05:44:39 +00:00
Jakub Adam
fb8ed9e5d2 d3d11desktopdup: Support desktop switches
Before creating output duplication interface, call SetThreadDesktop()
with HDESK of the current input desktop in case a desktop switch has
occurred.

This allows d3d11desktopdupsrc to capture Windows User Account Control
(UAC) prompts, which appear on a separate secure desktop. Otherwise
IDXGIOutput1::DuplicateOutput() will return E_ACCESSDENIED and the
element won't produce any frames as long as the UAC screen is active.

Note that in order to access secure desktop the application still has to
run at LOCAL_SYSTEM privileges. For GStreamer applications running with
regular user privileges this change has no effect.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2209>
2021-04-30 04:47:36 +00:00
Seungha Yang
957034d71a d3d11: Handle device change
If incoming buffer holds other d3d11 device, and user wants any device
(i.e., adapter index wasn't specified explicitly) update our device
with that of buffer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-28 16:26:58 +09:00
Seungha Yang
227aaed8d4 d3d11videosink: Delay window setup as much as possible
... so that videosink can handle device update with
d3d11 device of the first buffer

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-28 16:26:53 +09:00
Seungha Yang
70c96cf0dd d3d11: Don't accept buffer pool which holds different device
At the moment, d3d11 plugin doesn't support texture sharing between
different device

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-28 00:25:57 +09:00
Seungha Yang
41218dacec d3d11decoder: Run gst-indent
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-28 00:25:56 +09:00
Seungha Yang
00e1561bf2 d3d11decoder: Set flushing to internal pool on flush event
d3d11 decoders use internal pool for DPB texture and
Gst*Decoder::new_picture() will be blocked if internal pool is full.
We should be able to unblock in on flush-start event as expected.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2192>
2021-04-24 01:37:01 +09:00
Seungha Yang
e4fe65a03e d3d11: Fix wrong GstD3D11BufferPool type check
Fix typos

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2190>
2021-04-23 16:54:15 +09:00
Seungha Yang
094bfbedfc d3d11: Remove d3d11videosink wrapper bin
Drop d3d11videosink wrapper bin and handle texture upload
in d3d11videosink.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-20 02:01:30 +09:00
Seungha Yang
41879b20c2 d3d11: pluginutils: Fix wrong gst_memory_unmap() on _map() failure
It was obvious typo

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2164>
2021-04-15 00:05:03 +09:00
Seungha Yang
6eb9856f59 codecs: vp9decoder: Pass GstVideoCodecFrame to duplicate_picture()
... and fix picture duplication logic for vavp9dec

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
2021-04-07 19:32:29 +00:00
Seungha Yang
1f769839c0 codecs: vp9decoder: Port to GstVp9StatefulParser
Use newly implemented VP9 parser. Since new GstVp9FrameHeader
struct holds all the information of the stream, baseclass will not
pass parser object to new_sequence() method anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
2021-04-07 19:32:29 +00:00
Seungha Yang
5acd08f4c1 d3d11: Fix for UYVY/VYUY format rendering
Don't assume that non-native DXGI formats support RTV and/or SRV.
We are mapping UYVY and VYUY formats to DXGI_FORMAT_R8G8_B8G8_UNORM
which doesn't support render target view

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2133>
2021-04-01 16:07:33 +09:00
Seungha Yang
f369ca1b32 d3d11vp9dec: Remove debug dump functions
Existing debug messages are not quite useful

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2117>
2021-03-28 18:00:39 +09:00
Seungha Yang
bcdd2c4984 d3d11decoder: Move zero-copy decision logic into decoder object
Get rid of all duplicated code for zero-copy decision and output buffer
allocation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2117>
2021-03-28 18:00:28 +09:00
Seungha Yang
03f1ff2042 d3d11vp9dec: Ignore show_frame flag in output_picture()
baseclass will not call output_picture() if it shouldn't be ouputted.
Note that the show_frame flag can be zero when show_existing_frame is set

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2114>
2021-03-26 14:50:18 +00:00
Seungha Yang
c9518780fd d3d11decoder: Do not hardcode the limit minimum resolution to 64
Decoder should be able to support lower resolution than 64x64

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2113>
2021-03-26 22:04:19 +09:00
Seungha Yang
f74ed67ba0 d3d11videosink: Remove DirectWrite related dead code
It's now not enabled since we moved core part to gst-libs

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2108>
2021-03-25 14:43:54 +00:00
Seungha Yang
90181642a3 d3d11decoder: Resurrect zero-copy for fixed-size DPB pool
Enable zero-copy if downstream proposed pool and therefore decoder
can know the amount of buffer required by downstream.
Otherwise decoder will copy when our DPB pool has no sufficient
buffers for later decoding operation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Seungha Yang
b392ed82ba d3d11decoder: Implement array-of-texture DPB again
Re-implementation of array-of-texture based on d3d11 memory pool.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Seungha Yang
39b9f79e11 d3d11: Implement memory pool
Major changes:
* GstD3D11Allocator: This allocator is now device-independent object
  which can allocate GstD3D11Memory object for any GstD3D11Device.
  User can get this object via gst_allocator_find(GST_D3D11_MEMORY_NAME)
* GstD3D11PoolAllocator: A new allocator implementation for texture pool.
  From now on GstD3D11BufferPool will make use of this memory pool allocator
  to avoid frequent texture reallocation. That usually happens because
  of buffer copy (gst_buffer_make_writable for example)

In addition to that, GstD3D11BufferPool will provide GstBuffer with
GstVideoMeta, because CPU access to a GstD3D11Memory without GstVideoMeta
is almost impossible since GPU drivers needs padding for stride alignment.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Seungha Yang
4e7e390cab d3d11decoder: Temporarily remove zero-copy related code
We will re-implement it based on memory pool

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Seungha Yang
e006366206 d3d11decoder: Enable high precision clock if needed
We've been doing retry with 1ms sleep if DecoderBeginFrame()
returned E_PENDING which means application should call
DecoderBeginFrame() again because GPU is busy.
The 1ms sleep() during retry would result in usually about 15ms delay
in reality because of bad clock precision on Windows.
To improve throughput performance, this commit will enable
high precision clock only for NVIDIA platform since
DecoderBeginFrame() call on the other GPU vendors seems to
succeed without retry.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2099>
2021-03-23 12:41:43 +00:00
Matthew Waters
640a65bf96 gst: don't use volatile to mean atomic
volatile is not sufficient to provide atomic guarantees and real atomics
should be used instead.  GCC 11 has started warning about using volatile
with atomic operations.

https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1719

Discovered in https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/issues/868

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2098>
2021-03-22 14:34:36 +11:00
Seungha Yang
824b0ce0f4 Revert "d3d11: Enable native multi-thread protection layer and make use of it"
This reverts commit 872b7f503c.

Native multi-thread protection layer seems to be consuming more CPU
resource than application side protection approach in some cases

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 17:53:58 +09:00
Seungha Yang
872b7f503c d3d11: Enable native multi-thread protection layer and make use of it
... instead of our own GRecMutex locking. In this way, any other
Direct3D11 client (MediaFoundation for example) can safely call
any Direct3D11 API even when we are sharing our Direct3D11 device
with others.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2092>
2021-03-18 16:37:37 +00:00
Seungha Yang
80792e12d4 d3d11: Use render-target and shader-resource bind flags by default
Even if bind flags is not needed by an element, other element
might need such bind flags.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2091>
2021-03-17 22:17:22 +09:00
Seungha Yang
3f08fcb4ff d3d11decoder: WARNING if ID3D11VideoDevice is unavailable, not ERROR
gst_d3d11_decoder_new() method is also used for device capability
checking during plugin init. Although we are checking hardware
flag prior to that, it doesn't guarantee ID3D11VideoDevice interface.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2088>
2021-03-16 12:39:23 +00:00
Seungha Yang
58e451325b d3d11colorconvert: Fix caps leak
GstBaseTransform::fixate_caps() takes ownership of passed
othercaps argument.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2086>
2021-03-16 08:39:46 +00:00
Seungha Yang
309a940614 d3d11decoder: Do negotiation again per forward/reverse playback mode change
For reverse playback, we are always copying decoded
frame to downstream buffer. So the pool size can be
and need to be large enough.

In case that forward playback, however, we need to restrict
the max pool size for performance reason. Otherwise decoder
will keep copying decoded texture to downstream buffer pool
if decoding is faster than downstream throughput
performance and also there are queue element between them.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2083>
2021-03-16 05:54:41 +00:00
Seungha Yang
3a99517f7c d3d11videosink: Avoid switching conversion tool during playback
Decoder might be able to copy decoded texture to the other buffer pool
during playback depending on context. In that case, copied one
has no D3D11_BIND_DECODER bind flag.

If we used ID3D11VideoProcessor previously for decoder texture,
and incoming texture supports ID3D11VideoProcessor as well even if it has no
D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example),
allow zero-copying instead of using our fallback texture.

Frequent conversion tool change (between ID3D11VideoProcessor and generic shader)
might result in inconsistent image quality.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 21:00:22 +09:00
Seungha Yang
72345875b1 d3d11decoder: Refactor device open step and negotiation
* Remove redundant method arguments
* Don't allocate staging texture if downstream supports d3d11 memory

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 17:34:20 +09:00
Seungha Yang
347d9ceb4e d3d11decoder: Move profile GUID handling into decoder object
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 16:31:31 +09:00
Seungha Yang
1c1cfc4ba7 d3d11decoder: Get rid of private struct
Completely hide member variables

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 16:30:19 +09:00
Seungha Yang
78e4251750 Revert "d3d11vp9dec: Add support for internal frame resizing"
This reverts commit 58a4c33a0e

We should use ID3D11VideoProcessor instead of shader
to avoid copy. We need to revisit this topic later

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 16:30:15 +09:00