Commit graph

34 commits

Author SHA1 Message Date
Seungha Yang
b5c8889d4f d3d11videosink: Use GPU memory copy if possible
Even if fallback buffer is required (e.g., shader resource view is unavailable),
use direct GPU memory copy if possible. It must be much faster than
system memory copy approach.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1339>
2020-06-16 10:49:02 +00:00
Seungha Yang
73b39407a6 d3d11videosink: Ensure shader resource view of fallback buffer
SRV must be configured for color conversion

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1339>
2020-06-16 10:49:02 +00:00
Seungha Yang
afe941249e d3d11window: Add unprepare method to clear internal resource
GObject::dispose method can be called multiple times. As win32 d3d11window
has an internal thread and because GObject::dispose method could be called from the
thread, it might cause problems such as trying to join self-thread

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1299>
2020-05-27 04:59:50 +09:00
Seungha Yang
9b7c20bfca d3d11videosink: Clarify the meaning of various width and height variables
* Remove redundant variables for width/height and par from GstD3D11Window.
  GstVideoInfo holds all the values.
* Don't need to pass par to gst_d3d11_window_prepare().
  It will be parsed from caps again
* Remove duplicated math

Fixing regression of the commit 9dada90108
2020-03-02 22:01:43 +09:00
Seungha Yang
36fb790243 d3d11videosink: Ensure upload staging texture to fallback render texture
gst_video_frame_copy will copy input frame to stating texture
of fallback frame. Then, we need to map fallback texture with GST_MAP_D3D11
flag to upload the staging texture to render texture. Otherwise
the render texture wouldn't be updated.
2020-02-16 21:29:08 +09:00
Seungha Yang
6da90b59f4 d3d11videosink: Remove max size condition from pool
Actually our buffer pool size and the number of backbuffer are
independent. In case of reverse playback, upstream might request
a lot of buffers (up to GOP size).
2020-02-12 12:34:58 +00:00
Seungha Yang
4d02858f66 d3d11videosink: Fix fallback buffer copy
Since we don't use dynamic texture now, cpu access to the fallback
texture should not happen.
2020-02-05 00:52:48 +00:00
Seungha Yang
7aad9187e4 d3d11videosink: Use ID3D11VideoProcessor interface
...for color space conversion if available

ID3D11VideoProcessor is equivalent to DXVA-HD video processor
which might use specialized blocks for video processing
instead of general GPU resource. In addition to that feature,
we need to use this API for color space conversion of DXVA2 decoder
output memory, because any d3d11 texture arrays that were
created with D3D11_BIND_DECODER cannot be used for shader resource.

This is prework for d3d11decoder zero-copy rendering and also
for conditional HDR tone-map support.
Note that some Intel platform is known to support tone-mapping
at the driver level using this API on Windows 10.
2020-02-05 00:52:48 +00:00
Seungha Yang
96f0f4b613 d3d11memory: Always use native DXGI format if device support it
Use consistent memory layout between dxva and other shader use case.
For example, use DXGI_FORMAT_NV12 texture format instead of
two textures with DXGI_FORMAT_R8_UNORM and DXGI_FORMAT_R8G8_UNORM.
2020-01-13 01:58:08 +00:00
Seungha Yang
9ee40679b5 Revert "d3d11: Add support for D3D11_USAGE_DYNAMIC"
This reverts commit ddd13fc7c0

Dynamic usage can reduce the number of copy per frame but make
things complicated and the benefit seems to not significant.
Also since we don't provide _map() method for the dynamic usage,
application cannot read buffers which make "last-sample" property
unusable in case of d3d11videosink.
2020-01-13 01:58:08 +00:00
Seungha Yang
e4daa2ef43 d3d11: Add support for Universal Windows Platform
Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2020-01-06 20:14:51 +09:00
Seungha Yang
81dde0f5b8 d3d11: Aggregate d3d11 memory usage query for dynamic-usage type decision
Even if one of downstream d3d11 elements can support dynamic-usage memory,
another one might not support it. Also, to support dynamic-usage,
both upstream and downstream d3d11device must be the same object.
2019-12-28 05:43:44 +00:00
Seungha Yang
487a41d312 d3d11videosink: Add support for overlay composition
Add d3d11overlaycompositor object to draw overlay image
on render target using Blend method.
2019-12-24 19:00:45 +09:00
Seungha Yang
a5295509af d3d11: Enable dxgi debug layer and always try to pop d3d11/dxgi debug message if possible
Note that dxgi and d3d11 sdk debug will be enabled on debug build
2019-12-24 19:00:40 +09:00
Seungha Yang
0e013fe539 d3d11videosink: Use dynamic texture for fallback buffer
Upload CPU memory to texture directly by using dynamic usage texture.
This will reduce at least one step of staging copy per frame.
2019-12-20 19:21:02 +09:00
Seungha Yang
1b687d556a d3d11videosink: Add support for full screen mode
borderless top-most style full screen mode support.
Basically fullscreen toggle mode is disabled by default. To enable it
use "fullscreen-toggle-mode" property to allow fullscreen mode change
by user input and/or property.
2019-12-20 19:21:02 +09:00
Seungha Yang
9dada90108 d3d11videosink: Remove resizing window hack and unify resizing flow
In earlier implementation of d3d11videosink where no shader was implemented,
the aspect ratio and render size were adjusted by manipulating the backbuffer size
with unintuitive formula. Since now we do color conversion and resize using
shader, we can remove the hack.
2019-12-20 11:15:12 +09:00
Seungha Yang
61cb6b2bbe d3d11: Pass GstD3D11Memory object to ensure_{shader_resource,render_target}_view methods
The method name indicates it should be d3d11memory, so passing
GstD3D11Memory seems to make more sense than GstMemory.
2019-12-20 11:15:12 +09:00
Seungha Yang
7c041f1934 d3d11videosink: Don't specify crop meta to allocation query
d3d11videosink could not handle it for now
2019-12-20 11:15:12 +09:00
Seungha Yang
d963d442c5 d3d11videosink: Fix broken fallback rendering
Make fallback rendering work
2019-12-10 04:06:28 +00:00
Seungha Yang
abc1443cf0 d3d11videosink: Add debug message for fallback reason 2019-12-10 04:06:28 +00:00
Seungha Yang
9835289239 d3d11window: Do not check shader resource view if not doing conversion
If d3d11window does not convert format internally, shader resource view
is not required. Note that shader resource view is used for
color conversion using shader but when conversion is not required,
we just copy input input texture to backbuffer.
2019-12-10 04:06:28 +00:00
Seungha Yang
ca3ddf7848 d3d11videosink: Add color conversion support
Draw to back buffer texture directly. It would reduce the number of
copy at least once when color conversion is required.
2019-12-05 02:29:18 +00:00
Seungha Yang
59d9589722 d3d11videosink: Draw window with cached texture on resize
This would render nicer than presenting scene with broken aspect ratio,
especially in case of low framerate.
2019-12-05 02:29:18 +00:00
Seungha Yang
ddd13fc7c0 d3d11: Add support for D3D11_USAGE_DYNAMIC
D3D11 dynamic texture is a special memory type, which is mainly used for
frequent CPU write access to the texture. For now, this texture type
does not support gst_memory_{map,unmap}
2019-12-05 02:29:18 +00:00
Seungha Yang
79e88b76da d3d11videosink: Handle context query
Propagate d3d11device via context query.
2019-12-05 02:29:18 +00:00
Seungha Yang
a687f06b59 d3d11videosink: Specify d3d11 memory caps feature on sinkpad template 2019-12-05 02:29:18 +00:00
Seungha Yang
6c3311a39e d3d11: Refactor d3d11 memory and dxgi format usage
* Create staging texture only when the CPU access is requested.
Note that we should avoid the CPU access to d3d11 memory as mush as possible.
Incoming d3d11upload and d3d11download will take this GPU memory upload/download.

* Upload/Download texture memory from/to staging only if it needed, similar to
GstGL PBO implementation.

* Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder).
Because I420_* formats are not supported formats by dxgi, each plane should
be handled likewise GstGL separately, but NV12/P10 formats might be supported ones.
So we decide the number of d3d11memory per GstBuffer for video memory depending on
OS version and dxgi format. For instance, if NV12 is supported by OS,
only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit.
One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data
to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes.
Such behavior will be controlled via configuration of GstD3D11BufferPool and
default configuration is separate resources per plane.
2019-12-05 02:29:18 +00:00
Seungha Yang
4ff0e62b72 d3d11: Don't share d3d11 device context unconditionally without checking adapter
Do not accept any GstD3D11Device context which has different adapter
index from the required one. For example, if a d3d11 element is expecting
d3d11 device with adapter 1 (i.e., the second GPU), any d3d11 device
context having different adapter could not be shared with
the d3d11 element.
2019-12-05 02:29:18 +00:00
Seungha Yang
494ec13316 d3d11: Rename native handle getter and context utils functions
Make them consistent with cuda context utils functions.
Put in-only parameter before all in-out parameters, and add _handle()
suffix to native handle getter functions.
2019-12-05 02:29:18 +00:00
Seungha Yang
44cb5b4ff7 d3d11videosink: Don't try to post error message by d3d11window self
The d3d11window isn't GstElement. To post error message, proxy it
to d3d11videosink instead.
2019-10-04 21:15:15 +09:00
Seungha Yang
22d8d77b13 d3d11videosink: Use pixel aspect ratio to setup window draw area
... instead of calculated display ratio from given PAR and DAR.

d3d11window calculates output display ratio
to decide padding area per window resize event. In the formula,
actual PAR is required to handle both 1:1 PAR and non-1:1 PAR.
2019-09-26 10:23:46 +00:00
Seungha Yang
5909a495a7 d3d11videosink: Take into account pixel aspect ratio
Fix unexpected cropping with non 1:1 pixel aspect-ratio.

The actual buffer width/height should be passed to gst_d3d11_window_render(),
instead of the calculated resolution. The width/height
values are parameters for copying d3d11 video memory.
Also, aspect-ratio should be considered on resize callback
to decide render rectangle size.
2019-08-15 16:55:37 +09:00
Seungha Yang
5c3879ace6 d3d11videosink: Add new Direct3D11 video render plugin
Direct3D11 was shipped as part of Windows7 and it's obviously
primary graphics API on Windows.

This plugin includes HDR10 rendering if following requirements are satisfied
* IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h)
* Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space
* Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-07-08 08:31:47 +00:00