Commit graph

330 commits

Author SHA1 Message Date
Seungha Yang
5074cecc82 d3d11colorconverter: Add support cropping
Vertex buffer will be updated if input texture resolution is different
from GstVideoInfo or when an user requests specific crop area.
2020-03-09 23:44:06 +09:00
Seungha Yang
c6c83ca493 d3d11bufferpool: Add padding space for semi-planar YUV formats
Resolution of NV12, P010, and P016 formats must be multiple of two.
Otherwise texture cannot be created. Instead of doing this alignment
per API consumer side, do this in buffer pool for simplicity.
2020-03-09 10:14:53 +09:00
Seungha Yang
5609d8751c h265decoder: Pass max_dpb_size to new_sequence vfunc
same as we are doing in h264decoder
2020-03-08 16:11:35 +09:00
Seungha Yang
b70a192220 d3d11h264dec: Make use of max_dpb_size passed by baseclass
Most likely this modification can save GPU memory.
2020-03-08 16:03:35 +09:00
Seungha Yang
7c153b6e93 d3d11h265dec: Fix scaling list parsing
Scaling list can be signalled via sps or pps. Decoder should check
both of them.
2020-03-07 12:57:38 +09:00
Seungha Yang
cfb6283613 d3d11h265dec: Properly signal the ucNumDeltaPocsOfRefRpsIdx
ucNumDeltaPocsOfRefRpsIdx should be the NumDeltaPocs[RefRpsIdx].
2020-03-07 12:20:03 +09:00
Seungha Yang
83b056cb77 h265decoder: Port to nal type classification macro
... and remove namespaceless macro methods from baseclass
2020-03-05 23:22:39 +09:00
Nicolas Dufresne
020ee62c59 d3d11: Use gst_video_decoder_get_frame()
Now that the system_frame_number is saved on the pictures we can use
gst_video_decoder_get_frame() helper instead of getting the full list
and looping over it.
2020-03-05 03:06:16 +00:00
Nicolas Dufresne
c6b8157dd6 codecs: Pass the max_dpb_size to new_segment virtual
On new_segment, the decoder is expected to negotiate. The decoder may want to
pre-allocate the needed buffers. Pass the max_dpb_size as this is needed to
determin how many buffers should be allocated.
2020-03-05 03:06:16 +00:00
Nicolas Dufresne
00d04784d3 Move CODEC base classes into it's own library
This introduce a library which contains a set of base classes which
handles the parsing and the state tracking for the purpose of decoding
different CODECs. Currently H264, H265 and VP9 are supported. These
bases classes are used to decode with low level decoding API like DXVA,
NVDEC, VDPAU, VAAPI and V4L2 State Less decoders. The new library is
named gstreamer-codecs-1.0 / libgstcodecs.
2020-03-05 03:06:16 +00:00
Seungha Yang
be3de5a3db d3d11vp9dec: Add support for reverse playback
This commit moves parsing code for superframe and frame header into
handle_frame() method, and removes parse() implementation from vp9decoder
baseclass.

The combination of
- multiple frames are packed in a given input buffer (i.e., superframe)
- reverse playback
seems to be complicated and also it doesn't work as intended in some case
2020-03-03 12:15:33 +00:00
Seungha Yang
9b7c20bfca d3d11videosink: Clarify the meaning of various width and height variables
* Remove redundant variables for width/height and par from GstD3D11Window.
  GstVideoInfo holds all the values.
* Don't need to pass par to gst_d3d11_window_prepare().
  It will be parsed from caps again
* Remove duplicated math

Fixing regression of the commit 9dada90108
2020-03-02 22:01:43 +09:00
Yeongjin Jeong
25e9ee10b0 d3d11vp9dec: Port to GstVP9SuperframeInfo
The vp9parser is now exposed new API for parsing superframe info.
2020-03-02 01:55:36 +09:00
Seungha Yang
9575d835ba d3d11decoder: Do not print warning message if retry count is in expected range
gst_d3d11_result() will print warning message when HRESULT != S_OK.
However, since the retry is trivial stuff, check hr == E_PENDING first
and do not warn it.
2020-02-28 13:21:04 +09:00
Seungha Yang
418e6991c1 d3d11decoder: Check decoder status report
... and if h/w decoder reports error, increase error count.
2020-02-28 13:21:00 +09:00
Seungha Yang
f2e322197e d3d11window: Set DXGI_PRESENT_ALLOW_TEARING only in fullscreen mode
The DXGI_PRESENT_ALLOW_TEARING flag might cause unexpected tearing
side effect. Setting it in fullscreen mode only seems to be
the correct usage as in the Microsoft's direct3d examples.
2020-02-26 13:18:16 +09:00
Seungha Yang
4643dc1c12 d3d11decoder: Ensure the written bitstream buffer size is 128 bytes aligned
DXVA spec is saying that the size of bitstream buffer provided by hardware decoder
should be 128 bytes aligned. And also the host software decoder should
align the size of written buffer to 128 bytes. That means if the slice
(or frame in case of VP9) size is not aligned with 128 bytes,
the rest of non 128 bytes aligned memory should be zero-padded.

In addition to aligning implementation, some variables are renamed
to be more intuitive by this commit.
2020-02-24 14:46:27 +00:00
Seungha Yang
fbe7917a94 d3d11decoder: Add padding space on decoder output view when it's not aligned
Most H/W decoders have required alignment and dxva is also the case.
2020-02-20 17:32:42 +09:00
Seungha Yang
fe72bf6053 d3d11decoder: Register elements per GPU device with capability check
This implementation is similar to what we've done for nvcodec plugin.
Since supported resolution, profiles, and formats are device dependent ones,
single template caps cannot represent them, so this modification
will help autoplugging and fallback.

Note that the legacy gpu list and list of resolution to query were
taken from chromium's code.
2020-02-18 11:58:45 +00:00
Seungha Yang
13586bc77a d3d11device: Fix typo
s/vender/vendor
2020-02-18 11:58:45 +00:00
Seungha Yang
8ead80eecd d3d11device: Adjust debug level for when _new() fails
gst_d3d11_device_new might be used to enumerate device.
2020-02-18 11:58:45 +00:00
Seungha Yang
36fb790243 d3d11videosink: Ensure upload staging texture to fallback render texture
gst_video_frame_copy will copy input frame to stating texture
of fallback frame. Then, we need to map fallback texture with GST_MAP_D3D11
flag to upload the staging texture to render texture. Otherwise
the render texture wouldn't be updated.
2020-02-16 21:29:08 +09:00
Seungha Yang
9bf4746e2f d3d11decoder: Fix copying decoder view to staging
Source texture (decoder view) might be larger than destination (staging) texture.
In that case, D3D11_BOX structure should be passed to CopySubresourceRegion method
in order to specify the exact target area.
2020-02-13 21:25:15 +09:00
Seungha Yang
f6cdb91f55 d3d11window: Fix for broken dirty rect drawing on Windows 7
DXGI_SWAP_EFFECT_DISCARD cannot be used with dirty rect drawing feature
of IDXGISwapChain1::Present().
Note that IDXGISwapChain1 interface is available on Platform Update for Windows 7
and DXGI_SWAP_EFFECT_FLIP_SEQUENTIAL is also the case.
2020-02-12 22:38:53 +09:00
Seungha Yang
4383b387b7 d3d11window: Fix for dxva decoder output view rendering
Use resolution specified in caps for input_rect instead of
passed width and height value. The width and height might be modified
ones by d3d11videosink, then frame resolution might be different.
2020-02-12 12:34:58 +00:00
Seungha Yang
a39a5bf131 d3d11decoder: Refactor decoding process
* Move decoding process to handle_frame
* Remove GstVideoDecoder::parse implementation
* Clarify flush/drain/finish usage

In forward playback case, have_frame() call will be followed by
handle_frame() but reverse playback is not the case.
To ensure GstVideoCodecFrame, the decoding process should be placed inside
of handle_frame(), instead of parse().

Since we don't support alignment=nal, the parse() implementation is not worth.
In order to fix broken reverse playback, let's remove the parse()
implementation and revisit it when adding alignment=nal support.
2020-02-12 12:34:58 +00:00
Seungha Yang
3e78afbe0a d3d11decoder: Move handle_frame implementation to baseclass
... and remove unused start, stop method from subclass.

Current implementation does not require subclass specific behavior
for the handle_frame() method.
2020-02-12 12:34:58 +00:00
Seungha Yang
6da90b59f4 d3d11videosink: Remove max size condition from pool
Actually our buffer pool size and the number of backbuffer are
independent. In case of reverse playback, upstream might request
a lot of buffers (up to GOP size).
2020-02-12 12:34:58 +00:00
Seungha Yang
478fb29974 d3d11window: Clear cached buffer per new caps
d3d11window holds one buffer to redraw client area per resize event.

When the input format is being changed, this buffer should be cleared
to avoid mismatch beween newly configured shader/videoprocessor and
the format of previously cached buffer.
2020-02-06 00:46:46 +09:00
Seungha Yang
371b181292 d3d11decoder: Use consistent resolution between output caps and video meta
h264/h265 decoded buffer might have crop area then we need to
adjust video meta based on the padding space
2020-02-05 00:52:48 +00:00
Seungha Yang
2f32f30b62 d3d11memory: Add a method to specify padding space 2020-02-05 00:52:48 +00:00
Seungha Yang
f852ce01e4 d3d11decoder: Add support for zero-copy playback
When downstream support d3d11 memory with forward playback case,
expose decoder output view memory objects without copying.
2020-02-05 00:52:48 +00:00
Seungha Yang
69f7f958a0 d3d11decoder: Create decoder output view whenever it's required
Whatever the reason, buffer in pool might be freed then we need to
configure decoder output views again.
2020-02-05 00:52:48 +00:00
Seungha Yang
567575e33d d3d11decoder: Need to zero initilized for g_once
A vairable to be used for g_once, it should be zero initialized
2020-02-05 00:52:48 +00:00
Seungha Yang
8ff667e463 d3d11allocator: Work as if buffer pool when running on texture array mode
Because the size of texture array cannot be updated dynamically,
allocator should block the allocation request. This cannot be
done at buffer pool side if this d3d11 memory is shared among
multiple buffer objects. Note that setting NO_SHARE flag to
d3d11 memory is very inefficient. It would cause most likey
copy of the d3d11 texture.
2020-02-05 00:52:48 +00:00
Seungha Yang
4d02858f66 d3d11videosink: Fix fallback buffer copy
Since we don't use dynamic texture now, cpu access to the fallback
texture should not happen.
2020-02-05 00:52:48 +00:00
Seungha Yang
2aa9f0bd6c d3d11window: Don't create swapchain again per caps change
Creating swapchain is relatively heavy operation. If output dxgi format
is not being chagned, we don't need to destroy and create swachain again.
2020-02-05 00:52:48 +00:00
Seungha Yang
c1d2d9171d d3d11window: Invoke initial resize method from baseclass
... instead of calling from subclass in order for baseclass to handle
more things between swapchain creation and resource creation.
2020-02-05 00:52:48 +00:00
Seungha Yang
7aad9187e4 d3d11videosink: Use ID3D11VideoProcessor interface
...for color space conversion if available

ID3D11VideoProcessor is equivalent to DXVA-HD video processor
which might use specialized blocks for video processing
instead of general GPU resource. In addition to that feature,
we need to use this API for color space conversion of DXVA2 decoder
output memory, because any d3d11 texture arrays that were
created with D3D11_BIND_DECODER cannot be used for shader resource.

This is prework for d3d11decoder zero-copy rendering and also
for conditional HDR tone-map support.
Note that some Intel platform is known to support tone-mapping
at the driver level using this API on Windows 10.
2020-02-05 00:52:48 +00:00
Seungha Yang
122a9b93eb d3d11: Add video processor object
ID3D11VideoProcessor interface provides various image conversion
methods. Note that it's analogous to VAAPI VPP.
2020-02-05 00:52:48 +00:00
Seungha Yang
a967db3b20 d3d11format: Add util methods for mapping DXGI color space with ours
Move color space mapping and hdr10 metadata conversion methods to
d3d11format in order to reuse the code.
2020-02-05 00:52:48 +00:00
Seungha Yang
e97ef8a562 d3d11window_win32: Let DXGI choose client area
Don't specify the resolution of backbuffer. Then dxgi will let us know the
actual client area. When upstream resolution is chagned, updating the size
of backbuffer without the consideration for client size would cause mismatch
between them.
2020-01-26 12:13:24 +00:00
Seungha Yang
20d85c95d0 d3d11window_corewindow: Always call methods of CoreWindow interface from UI thread
Like swapchain panel implementation, most methods of CoreWindow
should be called from UI thread.
2020-01-23 03:49:19 +00:00
Seungha Yang
6e73e762c5 d3d11h265dec: Fix wrong NoRaslOutputFlag setting
... and handle EOS and EOB nals.

Only the first CRA picture should be associated with NoRaslOutputFlag
as the comment in code.
2020-01-14 08:47:30 +00:00
Seungha Yang
96f0f4b613 d3d11memory: Always use native DXGI format if device support it
Use consistent memory layout between dxva and other shader use case.
For example, use DXGI_FORMAT_NV12 texture format instead of
two textures with DXGI_FORMAT_R8_UNORM and DXGI_FORMAT_R8G8_UNORM.
2020-01-13 01:58:08 +00:00
Seungha Yang
9ee40679b5 Revert "d3d11: Add support for D3D11_USAGE_DYNAMIC"
This reverts commit ddd13fc7c0

Dynamic usage can reduce the number of copy per frame but make
things complicated and the benefit seems to not significant.
Also since we don't provide _map() method for the dynamic usage,
application cannot read buffers which make "last-sample" property
unusable in case of d3d11videosink.
2020-01-13 01:58:08 +00:00
Seungha Yang
e188893963 d3d11: Don't register decoders if unavailable
DXVA requires a hardware interface but may not be available,
such as in the case of VMs or when the GPU vendor does not provide a decoder interface.
2020-01-11 17:42:04 +09:00
Seungha Yang
616082d14a d3d11decoder: Don't return not initialized object from _new() method
... and change some debug levels since initialization failure might not be fatal.
2020-01-11 17:42:04 +09:00
Seungha Yang
49a1f022fd d3d11colorconverter: Handle P016_LE format
P016 format is no different than P010. Not much things to add code.
2020-01-09 16:29:47 +00:00
Seungha Yang
bbab229905 d3d11colorconverter: Add support for YUV to YUV conversion 2020-01-09 16:29:47 +00:00
Seungha Yang
a122f305f9 d3d11colorconverter: Add support for RGB to YUV conversion
... and remove code for RGBx since it's not supported format
by our d3d11 implementation for now.
2020-01-09 16:29:47 +00:00
Seungha Yang
da50cc4d92 d3d11format: Remove invalid format from supported format list 2020-01-09 16:29:47 +00:00
Seungha Yang
e4daa2ef43 d3d11: Add support for Universal Windows Platform
Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2020-01-06 20:14:51 +09:00
Seungha Yang
43a8eb9e92 d3d11decoder: Fix build on non-desktop target
Although the target platform of D3D11 decoding API are both desktop and UWP app,
DXVA header is blocked by "WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)"
which is meaning that that's only for desktop app.
To workaround this inconsistent annoyingness, we need to define WINAPI_PARTITION_DESKTOP
regardless of target WinAPI partition.
2020-01-06 18:45:56 +09:00
Seungha Yang
a139c8c7e8 d3d11: Add h265 decoder element
Some DPB management implementation is taken from gstreamer-vaapi
2019-12-31 02:13:48 +00:00
Seungha Yang
0e7b6526b8 d3d11: Add vp9 decoder element
Based on gstreamer-vaapi and Chromium implemetation.
2019-12-31 02:13:48 +00:00
Seungha Yang
586390b1ba d3d11: Add h264 decoder element
New decoder implementation based on dxva2 on d3d11 APIs. The DPB
management implementation is taken from Chromium.
2019-12-31 02:13:48 +00:00
Seungha Yang
7bfdeaf161 d3d11: Add support for Array typed texture memory
A ID3D11Texture2D memory can consist of multiple planes with array.
For array typed memory, GstD3D11Allocator will allocate new GstD3D11Memory
with increased reference count to the ID3D11Texture2D but different array index.
2019-12-31 02:13:48 +00:00
Seungha Yang
81dde0f5b8 d3d11: Aggregate d3d11 memory usage query for dynamic-usage type decision
Even if one of downstream d3d11 elements can support dynamic-usage memory,
another one might not support it. Also, to support dynamic-usage,
both upstream and downstream d3d11device must be the same object.
2019-12-28 05:43:44 +00:00
Seungha Yang
d731bcb18a d3d11colorconvert: Remove device type dependent behavior
If d3d11colorconvert element is configured, do color space conversion
regardless of the device type whether it's S/W emulation or real H/W.
Since d3d11colorconvert is no more a child of d3d11videosinkbin,
we don't need this behavior. Note that previous code was added to
avoid color space conversion from d3d11videosink if no hardware
device is available (S/W emulation of d3d11 is too slow).
2019-12-28 05:43:44 +00:00
Seungha Yang
46186356ca d3d11colorconverter: Fix unmatched lock/unlock pair 2019-12-28 05:43:44 +00:00
Seungha Yang
ab80c771df d3d11upload: Don't hard overwrite caps features to support d3d11 memory passthrough
d3d11upload should be able to support upstream d3d11 memory, not only system memory.

Fix for following pipeline
d3d11upload ! "video/x-raw(memory:D3D11Memory)" ! d3d11videosink
2019-12-28 05:43:44 +00:00
Seungha Yang
5298d95195 d3d11window: Make use of partial presentation with IDXGISwapChain1::Present1
Since we might draw on partial area of backbuffer in case of force-aspect-ratio,
presenting only updated area is more efficient way.

See also https://docs.microsoft.com/ko-kr/windows/win32/direct3ddxgi/dxgi-1-2-presentation-improvements
2019-12-24 20:10:54 +09:00
Seungha Yang
487a41d312 d3d11videosink: Add support for overlay composition
Add d3d11overlaycompositor object to draw overlay image
on render target using Blend method.
2019-12-24 19:00:45 +09:00
Seungha Yang
a5295509af d3d11: Enable dxgi debug layer and always try to pop d3d11/dxgi debug message if possible
Note that dxgi and d3d11 sdk debug will be enabled on debug build
2019-12-24 19:00:40 +09:00
Seungha Yang
0e013fe539 d3d11videosink: Use dynamic texture for fallback buffer
Upload CPU memory to texture directly by using dynamic usage texture.
This will reduce at least one step of staging copy per frame.
2019-12-20 19:21:02 +09:00
Seungha Yang
65cace22e2 d3d11colorconvert: Set TRANSFER_NEED_DOWNLOAD flag to output memory for later cpu access
Otherwise CPU cannot access texture via gst_memory_map()
2019-12-20 19:21:02 +09:00
Seungha Yang
18e13404a2 d3d11colorconvert: Use shader resource and render target bind flags together
The output of d3d11colorconvert would be used for rendering (i.e., shader resource)
2019-12-20 19:21:02 +09:00
Seungha Yang
62c8d19e01 d3d11window: Change aspect-ratio mode from window thread
Call DXGI API from window thread as much as possible
2019-12-20 19:21:02 +09:00
Seungha Yang
1b687d556a d3d11videosink: Add support for full screen mode
borderless top-most style full screen mode support.
Basically fullscreen toggle mode is disabled by default. To enable it
use "fullscreen-toggle-mode" property to allow fullscreen mode change
by user input and/or property.
2019-12-20 19:21:02 +09:00
Seungha Yang
0788492461 d3d11: Use GRecMutex to protect immediate context and dxgi API call
In some cases, rendering and dxgi (e.g., swapchain) APIs should be
called from window message pump thread, but current design (dedicated d3d11 thread)
make it impossible. To solve it, change concurrency model to locking based one
from single-thread model.
2019-12-20 19:21:03 +09:00
Seungha Yang
a0a85cd80c d3d11window: Prefer to use flip-sequential over discard
flip-sequential is more efficient than discard
2019-12-20 11:15:12 +09:00
Seungha Yang
1424de0a4a d3d11window: Don't hold backbuffer interface
We don't need to hold it
2019-12-20 11:15:12 +09:00
Seungha Yang
32d618c677 d3d11window: Use CreateSwapChainForHwnd if available
That's recommended way from MS and CreateSwapChainForHwnd supports
more options than CreateSwapChain
2019-12-20 11:15:12 +09:00
Seungha Yang
9fd0b62f2d d3d11: Use configuration file for dxgi header version check 2019-12-20 11:15:12 +09:00
Seungha Yang
23b47a7ec1 d3d11window: Use allow-tearing mode if device supports it
As the recommendation from MS.
2019-12-20 11:15:12 +09:00
Seungha Yang
72b6b3557d d3d11window: Disable fullscreen mode change by alt + enter
Disable full screen mode change until proper handling is implemented
2019-12-20 11:15:12 +09:00
Seungha Yang
9dada90108 d3d11videosink: Remove resizing window hack and unify resizing flow
In earlier implementation of d3d11videosink where no shader was implemented,
the aspect ratio and render size were adjusted by manipulating the backbuffer size
with unintuitive formula. Since now we do color conversion and resize using
shader, we can remove the hack.
2019-12-20 11:15:12 +09:00
Seungha Yang
61cb6b2bbe d3d11: Pass GstD3D11Memory object to ensure_{shader_resource,render_target}_view methods
The method name indicates it should be d3d11memory, so passing
GstD3D11Memory seems to make more sense than GstMemory.
2019-12-20 11:15:12 +09:00
Seungha Yang
7c041f1934 d3d11videosink: Don't specify crop meta to allocation query
d3d11videosink could not handle it for now
2019-12-20 11:15:12 +09:00
Seungha Yang
64eb1d46b5 d3d11window: Always draw to internal window
... and use SetParent() WIN32 API when external window is used.
Depending on DXGI swap effect, the external window might not be
reusable by another backend. To preserve the external window's property
and setting, drawing to internal window seems to be safer way.
2019-12-16 22:24:29 +09:00
Seungha Yang
638c6d6931 d3d11window: Always set error reason to GError object 2019-12-16 18:21:02 +09:00
Seungha Yang
a400126889 d3d11window: Initialize surface rect with given external window handle
Unlike internal window, resize event might not happen with external window.
2019-12-11 23:04:55 +09:00
Seungha Yang
a8ec409b84 d3d11device: Report alive d3d11 objects at the end
It would be useful for debugging d3d11 object leak.
2019-12-11 23:04:49 +09:00
Seungha Yang
59295b4271 d3d11: Fix ID3D11SamplerState leak 2019-12-11 12:57:46 +00:00
Seungha Yang
d963d442c5 d3d11videosink: Fix broken fallback rendering
Make fallback rendering work
2019-12-10 04:06:28 +00:00
Seungha Yang
abc1443cf0 d3d11videosink: Add debug message for fallback reason 2019-12-10 04:06:28 +00:00
Seungha Yang
9835289239 d3d11window: Do not check shader resource view if not doing conversion
If d3d11window does not convert format internally, shader resource view
is not required. Note that shader resource view is used for
color conversion using shader but when conversion is not required,
we just copy input input texture to backbuffer.
2019-12-10 04:06:28 +00:00
Seungha Yang
8154b22a88 d3d11window: Clear old swapchain per prepare
_prepare() might be called multiple times for given window, so clear
old swapchain if any.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1138
2019-12-10 10:58:28 +09:00
Seungha Yang
5bf10d8781 d3d11window: Always set dxgi colorspace if possible
Not only for the 10 bits RGB case with HDR metadata, set proper color space
to swapchain in order to make color representaion correct
2019-12-09 12:22:15 +00:00
Aaron Boxer
e9891dfe14 d3d11: translate windows error codes to human readable strings
This will help with debugging.
2019-12-07 16:09:03 -06:00
Seungha Yang
199caccc41 d3d11: Initialize debug categories of non-GstElement implementation in plugin init
That's the way to expose debug category to --gst-debug-help
2019-12-05 02:29:18 +00:00
Seungha Yang
71befc477f d3d11window: Invoke resize task with high priority
On resize event, swapchain should be configured with higher priority
than the other tasks since it's directly related to visual artifacts.
2019-12-05 02:29:18 +00:00
Seungha Yang
d2b85683de d3d11device: Add gst_d3d11_device_thread_add_full method
I would be used to invoke GPU task with specified priority.
2019-12-05 02:29:18 +00:00
Seungha Yang
7cab47ddda d3d11videosinkbin: Drop d3d11 color convert element
d3d11videosink can convert color space now, so the conversion element
seems to be redundant.
2019-12-05 02:29:18 +00:00
Seungha Yang
ca3ddf7848 d3d11videosink: Add color conversion support
Draw to back buffer texture directly. It would reduce the number of
copy at least once when color conversion is required.
2019-12-05 02:29:18 +00:00
Seungha Yang
622733ed0d d3d11converter: Skip setup converter if input and output formats are equal 2019-12-05 02:29:18 +00:00
Seungha Yang
c9cb08fcd4 d3d11colorconvert: Split color space converter to reuse code 2019-12-05 02:29:18 +00:00
Seungha Yang
42ca123f0f d3d11: Split shader to reuse code 2019-12-05 02:29:18 +00:00
Seungha Yang
59d9589722 d3d11videosink: Draw window with cached texture on resize
This would render nicer than presenting scene with broken aspect ratio,
especially in case of low framerate.
2019-12-05 02:29:18 +00:00
Seungha Yang
ddd13fc7c0 d3d11: Add support for D3D11_USAGE_DYNAMIC
D3D11 dynamic texture is a special memory type, which is mainly used for
frequent CPU write access to the texture. For now, this texture type
does not support gst_memory_{map,unmap}
2019-12-05 02:29:18 +00:00
Seungha Yang
5ce4748cc0 d3d11device: Add debug message for CreateTexture2D failure 2019-12-05 02:29:18 +00:00
Seungha Yang
914d2df79e d3d11device: Fix misreading debug message 2019-12-05 02:29:18 +00:00
Seungha Yang
71ae632b25 d3d11colorconvert: Disable color conversion with software rasterizer
It's much slower than our CPU based color-converter in most case.
2019-12-05 02:29:18 +00:00
Seungha Yang
79e88b76da d3d11videosink: Handle context query
Propagate d3d11device via context query.
2019-12-05 02:29:18 +00:00
Seungha Yang
5edf64d655 d3d11colorconvert: Avoid copy in/out texture if possible
When input and/or output d3d11memory has its own view, use them for
rendering to avoid texture copy
2019-12-05 02:29:18 +00:00
Seungha Yang
da3545e8c8 d3d11memory: Allocate texture with required views
Store required resource views with d3d11 texture when it requested.
2019-12-05 02:29:18 +00:00
Seungha Yang
4e4fb8a577 d3d11: Add d3d11videosinkbin element
New wrapper element to support d3d11 memory upload, color conversion, and
rendering at once.
2019-12-05 02:29:18 +00:00
Seungha Yang
79ac41ec5b d3d11: Add color space conversion element using shader
Add d3d11colorconvert element for color space conversion.
Currently {RGB, YUV} to RGB conversion is supported.
2019-12-05 02:29:18 +00:00
Seungha Yang
01903c47f7 d3d11memory: Register mininal compare function for GstD3D11AllocationParams
The d3d11 allocation param is stored in GstStructure so it should
provide compare function for later use such as gst_structure_is_equal()
2019-12-05 02:29:18 +00:00
Seungha Yang
0778fa8043 d3d11window: Set render target whenever drawing
Device context's render target might be updated for offscreen rendering
2019-12-05 02:29:18 +00:00
Seungha Yang
a687f06b59 d3d11videosink: Specify d3d11 memory caps feature on sinkpad template 2019-12-05 02:29:18 +00:00
Seungha Yang
c17c1346ee d3d11: Introduce d3d11 upload/download element
That's equivalent to glupload and gldownload elements but for d3d11
2019-12-05 02:29:18 +00:00
Seungha Yang
6c3311a39e d3d11: Refactor d3d11 memory and dxgi format usage
* Create staging texture only when the CPU access is requested.
Note that we should avoid the CPU access to d3d11 memory as mush as possible.
Incoming d3d11upload and d3d11download will take this GPU memory upload/download.

* Upload/Download texture memory from/to staging only if it needed, similar to
GstGL PBO implementation.

* Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder).
Because I420_* formats are not supported formats by dxgi, each plane should
be handled likewise GstGL separately, but NV12/P10 formats might be supported ones.
So we decide the number of d3d11memory per GstBuffer for video memory depending on
OS version and dxgi format. For instance, if NV12 is supported by OS,
only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit.
One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data
to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes.
Such behavior will be controlled via configuration of GstD3D11BufferPool and
default configuration is separate resources per plane.
2019-12-05 02:29:18 +00:00
Seungha Yang
d4c75f32dd d3d11: Add method for querying d3d11 memory type
... and fix misreading "private" annotation.
2019-12-05 02:29:18 +00:00
Seungha Yang
8da5237e22 d3d11device: Store selected d3d feature level
Depending on selected feature level, d3d11 API usage can be different.
Instead of querying the selected feature level by user whenever required,
store it once by d3d11device.
2019-12-05 02:29:18 +00:00
Seungha Yang
4ff0e62b72 d3d11: Don't share d3d11 device context unconditionally without checking adapter
Do not accept any GstD3D11Device context which has different adapter
index from the required one. For example, if a d3d11 element is expecting
d3d11 device with adapter 1 (i.e., the second GPU), any d3d11 device
context having different adapter could not be shared with
the d3d11 element.
2019-12-05 02:29:18 +00:00
Seungha Yang
7ae4105c6e d3d11device: Don't allow auto adapter selection
Change to d3d11device to be created with explicit target adapter index,
and expose some readonly device properties
2019-12-05 02:29:18 +00:00
Seungha Yang
494ec13316 d3d11: Rename native handle getter and context utils functions
Make them consistent with cuda context utils functions.
Put in-only parameter before all in-out parameters, and add _handle()
suffix to native handle getter functions.
2019-12-05 02:29:18 +00:00
Seungha Yang
e52ed98197 d3d11window: Fix subclass procedure recursion
subclass window precedure should be cleared if it's no more used.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1115
2019-11-14 01:26:09 +00:00
Aaron Boxer
bfe2447a3b d3d11videosink: only destroy loop if external_win_id is false
Loop was created with this condition, so it should also be
destroyed under the same condition
2019-11-12 19:54:46 -06:00
Niels De Graef
d8f61515d8 Don't pass default GLib marshallers for signals
By passing NULL to `g_signal_new` instead of a marshaller, GLib will
actually internally optimize the signal (if the marshaller is available
in GLib itself) by also setting the valist marshaller. This makes the
signal emission a bit more performant than the regular marshalling,
which still needs to box into `GValue` and call libffi in case of a
generic marshaller.

Note that for custom marshallers, one would use
`g_signal_set_va_marshaller()` with the valist marshaller instead.
2019-11-06 14:27:46 +00:00
Aaron Boxer
6d3429af34 documentation: fixed a heap o' typos 2019-11-05 09:11:25 -05:00
Seungha Yang
5e8755cf07 d3d11window: Protect registration/creation of WIN32 window
Querying existing window class info and creation of window object
should be protected since they are not thread safe.
2019-10-04 21:15:25 +09:00
Seungha Yang
44cb5b4ff7 d3d11videosink: Don't try to post error message by d3d11window self
The d3d11window isn't GstElement. To post error message, proxy it
to d3d11videosink instead.
2019-10-04 21:15:15 +09:00
Seungha Yang
22d8d77b13 d3d11videosink: Use pixel aspect ratio to setup window draw area
... instead of calculated display ratio from given PAR and DAR.

d3d11window calculates output display ratio
to decide padding area per window resize event. In the formula,
actual PAR is required to handle both 1:1 PAR and non-1:1 PAR.
2019-09-26 10:23:46 +00:00
Seungha Yang
e0b2d34237 d3d11: Use g_cond_broadcast() to wake up waiting threads
gst_d3d11_device_thread_add() can be called concurrently from different
threads so the cond signal should reach to all waiting threads.
2019-09-18 18:47:44 +09:00
Seungha Yang
58afcf09b4 meson: d3d11: Remove unnecessary dependency
d3d11 never use any API of gstreamer-allocators-1.0
2019-08-15 16:55:51 +09:00
Seungha Yang
5909a495a7 d3d11videosink: Take into account pixel aspect ratio
Fix unexpected cropping with non 1:1 pixel aspect-ratio.

The actual buffer width/height should be passed to gst_d3d11_window_render(),
instead of the calculated resolution. The width/height
values are parameters for copying d3d11 video memory.
Also, aspect-ratio should be considered on resize callback
to decide render rectangle size.
2019-08-15 16:55:37 +09:00
Seungha Yang
5c3879ace6 d3d11videosink: Add new Direct3D11 video render plugin
Direct3D11 was shipped as part of Windows7 and it's obviously
primary graphics API on Windows.

This plugin includes HDR10 rendering if following requirements are satisfied
* IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h)
* Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space
* Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-07-08 08:31:47 +00:00