mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2024-11-06 01:19:38 +00:00
Code Issues 1 Projects Releases Wiki Activity
gstreamer/subprojects/gst-plugins-bad/sys/d3d11/gstd3d11window.cpp

1175 lines
36 KiB
C++
Raw Normal View History

d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
/*
* GStreamer
* Copyright (C) 2019 Seungha Yang <seungha.yang@navercorp.com>
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Library General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Library General Public License for more details.
*
* You should have received a copy of the GNU Library General Public
* License along with this library; if not, write to the
* Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
* Boston, MA 02110-1301, USA.
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include "gstd3d11window.h"
d3d11: Move core methods to gst-libs Move d3d11 device, memory, buffer pool and minimal method to gst-libs so that other plugins can access d3d11 resource. Since Direct3D is primary graphics API on Windows, we need this infrastructure for various plugins can share GPU resource without downloading GPU memory. Note that this implementation is public only for -bad scope for now. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2020-12-20 17:47:45 +00:00
#include "gstd3d11pluginutils.h"
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11: Allow building UWP features with Desktop features if possible WINAPI_PARTITION_DESKTOP and WINAPI_PARTITION_APP can coexist. Although UWP only binaries should be used for production stage, this change will be useful for development stage Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1962>
2021-01-18 10:17:14 +00:00
#if GST_D3D11_WINAPI_APP
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
/* workaround for GetCurrentTime collision */
#ifdef GetCurrentTime
#undef GetCurrentTime
#endif
#include <windows.ui.xaml.h>
#include <windows.applicationmodel.core.h>
d3d11videosink: Add a property to support rendering statistics data on window Add a new property "render-stats" to allow rendering statistics data on window for debugging and/or development purpose. Text rendering will be accelerated by GPU since this implementation uses Direct2D/DirectWrite API and Direct3D inter-op for minimal overhead. Specifically, text data will be rendered on swapchain backbuffer directly without any copy/allocation of extra texture. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1830>
2020-11-21 19:39:57 +00:00
#endif
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
#include <wrl.h>
d3d11: Get rid of "extern "C"" wrapping for GST_DEBUG_CATEGORY_EXTERN Instead, change the file defining debug category to cpp Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2525>
2021-09-10 15:43:26 +00:00
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
/* *INDENT-OFF* */
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
using namespace Microsoft::WRL;
d3d11: Get rid of "extern "C"" wrapping for GST_DEBUG_CATEGORY_EXTERN Instead, change the file defining debug category to cpp Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2525>
2021-09-10 15:43:26 +00:00
/* *INDENT-ON* */
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
GST_DEBUG_CATEGORY_EXTERN (gst_d3d11_window_debug);
#define GST_CAT_DEFAULT gst_d3d11_window_debug
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
enum
{
PROP_0,
PROP_D3D11_DEVICE,
PROP_FORCE_ASPECT_RATIO,
PROP_ENABLE_NAVIGATION_EVENTS,
PROP_FULLSCREEN_TOGGLE_MODE,
PROP_FULLSCREEN,
PROP_WINDOW_HANDLE,
d3d11videosink: Add a property to support rendering statistics data on window Add a new property "render-stats" to allow rendering statistics data on window for debugging and/or development purpose. Text rendering will be accelerated by GPU since this implementation uses Direct2D/DirectWrite API and Direct3D inter-op for minimal overhead. Specifically, text data will be rendered on swapchain backbuffer directly without any copy/allocation of extra texture. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1830>
2020-11-21 19:39:57 +00:00
PROP_RENDER_STATS,
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
};
#define DEFAULT_ENABLE_NAVIGATION_EVENTS TRUE
#define DEFAULT_FORCE_ASPECT_RATIO TRUE
#define DEFAULT_FULLSCREEN_TOGGLE_MODE GST_D3D11_WINDOW_FULLSCREEN_TOGGLE_MODE_NONE
#define DEFAULT_FULLSCREEN FALSE
d3d11videosink: Add a property to support rendering statistics data on window Add a new property "render-stats" to allow rendering statistics data on window for debugging and/or development purpose. Text rendering will be accelerated by GPU since this implementation uses Direct2D/DirectWrite API and Direct3D inter-op for minimal overhead. Specifically, text data will be rendered on swapchain backbuffer directly without any copy/allocation of extra texture. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1830>
2020-11-21 19:39:57 +00:00
#define DEFAULT_RENDER_STATS FALSE
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
enum
{
SIGNAL_KEY_EVENT,
SIGNAL_MOUSE_EVENT,
SIGNAL_LAST
};
static guint d3d11_window_signals[SIGNAL_LAST] = { 0, };
GType
gst_d3d11_window_fullscreen_toggle_mode_type (void)
{
gst: don't use volatile to mean atomic volatile is not sufficient to provide atomic guarantees and real atomics should be used instead. GCC 11 has started warning about using volatile with atomic operations. https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1719 Discovered in https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/issues/868 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2098>
2021-03-22 03:34:36 +00:00
static gsize mode_type = 0;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
if (g_once_init_enter (&mode_type)) {
static const GFlagsValue mode_types[] = {
{GST_D3D11_WINDOW_FULLSCREEN_TOGGLE_MODE_NONE,
"GST_D3D11_WINDOW_FULLSCREEN_TOGGLE_MODE_NONE", "none"},
{GST_D3D11_WINDOW_FULLSCREEN_TOGGLE_MODE_ALT_ENTER,
"GST_D3D11_WINDOW_FULLSCREEN_TOGGLE_MODE_ALT_ENTER", "alt-enter"},
{GST_D3D11_WINDOW_FULLSCREEN_TOGGLE_MODE_PROPERTY,
"GST_D3D11_WINDOW_FULLSCREEN_TOGGLE_MODE_PROPERTY", "property"},
{0, NULL, NULL},
};
GType tmp = g_flags_register_static ("GstD3D11WindowFullscreenToggleMode",
mode_types);
g_once_init_leave (&mode_type, tmp);
}
return (GType) mode_type;
}
#define gst_d3d11_window_parent_class parent_class
d3d11videosink: Remove DirectWrite related dead code It's now not enabled since we moved core part to gst-libs Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2108>
2021-03-25 12:17:07 +00:00
G_DEFINE_ABSTRACT_TYPE (GstD3D11Window, gst_d3d11_window, GST_TYPE_OBJECT);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
static void gst_d3d11_window_set_property (GObject * object, guint prop_id,
const GValue * value, GParamSpec * pspec);
static void gst_d3d11_window_get_property (GObject * object, guint prop_id,
GValue * value, GParamSpec * pspec);
static void gst_d3d11_window_dispose (GObject * object);
static GstFlowReturn gst_d3d111_window_present (GstD3D11Window * self,
d3d11videosink: Remove DirectWrite related dead code It's now not enabled since we moved core part to gst-libs Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2108>
2021-03-25 12:17:07 +00:00
GstBuffer * buffer, ID3D11VideoProcessorOutputView * pov,
ID3D11RenderTargetView * rtv);
d3d11window: Invoke initial resize method from baseclass ... instead of calling from subclass in order for baseclass to handle more things between swapchain creation and resource creation.
2020-01-10 12:45:43 +00:00
static void gst_d3d11_window_on_resize_default (GstD3D11Window * window,
guint width, guint height);
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
static gboolean gst_d3d11_window_prepare_default (GstD3D11Window * window,
guint display_width, guint display_height, GstCaps * caps,
gboolean * video_processor_available, GError ** error);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
static void
gst_d3d11_window_class_init (GstD3D11WindowClass * klass)
{
GObjectClass *gobject_class = G_OBJECT_CLASS (klass);
gobject_class->set_property = gst_d3d11_window_set_property;
gobject_class->get_property = gst_d3d11_window_get_property;
gobject_class->dispose = gst_d3d11_window_dispose;
d3d11window: Invoke initial resize method from baseclass ... instead of calling from subclass in order for baseclass to handle more things between swapchain creation and resource creation.
2020-01-10 12:45:43 +00:00
klass->on_resize = GST_DEBUG_FUNCPTR (gst_d3d11_window_on_resize_default);
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
klass->prepare = GST_DEBUG_FUNCPTR (gst_d3d11_window_prepare_default);
d3d11window: Invoke initial resize method from baseclass ... instead of calling from subclass in order for baseclass to handle more things between swapchain creation and resource creation.
2020-01-10 12:45:43 +00:00
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
g_object_class_install_property (gobject_class, PROP_D3D11_DEVICE,
g_param_spec_object ("d3d11device", "D3D11 Device",
"GstD3D11Device object for creating swapchain",
GST_TYPE_D3D11_DEVICE,
(GParamFlags) (G_PARAM_WRITABLE | G_PARAM_CONSTRUCT_ONLY |
G_PARAM_STATIC_STRINGS)));
g_object_class_install_property (gobject_class, PROP_FORCE_ASPECT_RATIO,
g_param_spec_boolean ("force-aspect-ratio",
"Force aspect ratio",
"When enabled, scaling will respect original aspect ratio",
DEFAULT_FORCE_ASPECT_RATIO,
(GParamFlags) (G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)));
g_object_class_install_property (gobject_class, PROP_ENABLE_NAVIGATION_EVENTS,
g_param_spec_boolean ("enable-navigation-events",
"Enable navigation events",
"When enabled, signals for navigation events are emitted",
DEFAULT_ENABLE_NAVIGATION_EVENTS,
(GParamFlags) (G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)));
g_object_class_install_property (gobject_class, PROP_FULLSCREEN_TOGGLE_MODE,
g_param_spec_flags ("fullscreen-toggle-mode",
"Full screen toggle mode",
"Full screen toggle mode used to trigger fullscreen mode change",
GST_D3D11_WINDOW_TOGGLE_MODE_GET_TYPE, DEFAULT_FULLSCREEN_TOGGLE_MODE,
(GParamFlags) (G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)));
g_object_class_install_property (gobject_class, PROP_FULLSCREEN,
g_param_spec_boolean ("fullscreen",
"fullscreen",
"Ignored when \"fullscreen-toggle-mode\" does not include \"property\"",
DEFAULT_FULLSCREEN,
(GParamFlags) (G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)));
g_object_class_install_property (gobject_class, PROP_WINDOW_HANDLE,
g_param_spec_pointer ("window-handle",
"Window Handle", "External Window Handle",
(GParamFlags) (G_PARAM_WRITABLE | G_PARAM_CONSTRUCT_ONLY |
G_PARAM_STATIC_STRINGS)));
d3d11_window_signals[SIGNAL_KEY_EVENT] =
g_signal_new ("key-event", G_TYPE_FROM_CLASS (klass),
G_SIGNAL_RUN_LAST, 0, NULL, NULL, NULL,
G_TYPE_NONE, 2, G_TYPE_STRING, G_TYPE_STRING);
d3d11_window_signals[SIGNAL_MOUSE_EVENT] =
g_signal_new ("mouse-event", G_TYPE_FROM_CLASS (klass),
G_SIGNAL_RUN_LAST, 0, NULL, NULL, NULL,
G_TYPE_NONE, 4, G_TYPE_STRING, G_TYPE_INT, G_TYPE_DOUBLE, G_TYPE_DOUBLE);
}
static void
gst_d3d11_window_init (GstD3D11Window * self)
{
self->force_aspect_ratio = DEFAULT_FORCE_ASPECT_RATIO;
self->enable_navigation_events = DEFAULT_ENABLE_NAVIGATION_EVENTS;
self->fullscreen_toggle_mode = GST_D3D11_WINDOW_FULLSCREEN_TOGGLE_MODE_NONE;
self->fullscreen = DEFAULT_FULLSCREEN;
d3d11videosink: Add a property to support rendering statistics data on window Add a new property "render-stats" to allow rendering statistics data on window for debugging and/or development purpose. Text rendering will be accelerated by GPU since this implementation uses Direct2D/DirectWrite API and Direct3D inter-op for minimal overhead. Specifically, text data will be rendered on swapchain backbuffer directly without any copy/allocation of extra texture. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1830>
2020-11-21 19:39:57 +00:00
self->render_stats = DEFAULT_RENDER_STATS;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
}
static void
gst_d3d11_window_set_property (GObject * object, guint prop_id,
const GValue * value, GParamSpec * pspec)
{
GstD3D11Window *self = GST_D3D11_WINDOW (object);
GstD3D11WindowClass *klass = GST_D3D11_WINDOW_GET_CLASS (object);
switch (prop_id) {
case PROP_D3D11_DEVICE:
self->device = (GstD3D11Device *) g_value_dup_object (value);
break;
case PROP_FORCE_ASPECT_RATIO:
{
self->force_aspect_ratio = g_value_get_boolean (value);
if (self->swap_chain)
klass->update_swap_chain (self);
break;
}
case PROP_ENABLE_NAVIGATION_EVENTS:
self->enable_navigation_events = g_value_get_boolean (value);
break;
case PROP_FULLSCREEN_TOGGLE_MODE:
self->fullscreen_toggle_mode =
(GstD3D11WindowFullscreenToggleMode) g_value_get_flags (value);
break;
case PROP_FULLSCREEN:
{
self->requested_fullscreen = g_value_get_boolean (value);
if (self->swap_chain)
klass->change_fullscreen_mode (self);
break;
}
case PROP_WINDOW_HANDLE:
self->external_handle = (guintptr) g_value_get_pointer (value);
break;
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
static void
gst_d3d11_window_get_property (GObject * object, guint prop_id,
GValue * value, GParamSpec * pspec)
{
GstD3D11Window *self = GST_D3D11_WINDOW (object);
switch (prop_id) {
case PROP_ENABLE_NAVIGATION_EVENTS:
g_value_set_boolean (value, self->enable_navigation_events);
break;
case PROP_FORCE_ASPECT_RATIO:
g_value_set_boolean (value, self->force_aspect_ratio);
break;
case PROP_FULLSCREEN_TOGGLE_MODE:
g_value_set_flags (value, self->fullscreen_toggle_mode);
break;
case PROP_FULLSCREEN:
g_value_set_boolean (value, self->fullscreen);
break;
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
static void
gst_d3d11_window_release_resources (GstD3D11Device * device,
GstD3D11Window * window)
{
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
GST_D3D11_CLEAR_COM (window->rtv);
GST_D3D11_CLEAR_COM (window->pov);
GST_D3D11_CLEAR_COM (window->swap_chain);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
}
static void
gst_d3d11_window_dispose (GObject * object)
{
GstD3D11Window *self = GST_D3D11_WINDOW (object);
if (self->device) {
gst_d3d11_window_release_resources (self->device, self);
}
d3d11window: Fix videoprocessor leak Clear all objects in dispose function
2020-04-21 06:30:03 +00:00
g_clear_pointer (&self->processor, gst_d3d11_video_processor_free);
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
g_clear_pointer (&self->converter, gst_d3d11_converter_free);
d3d11window: Fix videoprocessor leak Clear all objects in dispose function
2020-04-21 06:30:03 +00:00
g_clear_pointer (&self->compositor, gst_d3d11_overlay_compositor_free);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
gst_clear_buffer (&self->cached_buffer);
gst_clear_object (&self->device);
G_OBJECT_CLASS (parent_class)->dispose (object);
}
d3d11window: Invoke initial resize method from baseclass ... instead of calling from subclass in order for baseclass to handle more things between swapchain creation and resource creation.
2020-01-10 12:45:43 +00:00
static void
gst_d3d11_window_on_resize_default (GstD3D11Window * window, guint width,
guint height)
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
{
HRESULT hr;
ID3D11Device *device_handle;
D3D11_TEXTURE2D_DESC desc;
DXGI_SWAP_CHAIN_DESC swap_desc;
ID3D11Texture2D *backbuffer = NULL;
GstVideoRectangle src_rect, dst_rect, rst_rect;
IDXGISwapChain *swap_chain;
gst_d3d11_device_lock (window->device);
if (!window->swap_chain)
goto done;
device_handle = gst_d3d11_device_get_device_handle (window->device);
swap_chain = window->swap_chain;
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
GST_D3D11_CLEAR_COM (window->rtv);
GST_D3D11_CLEAR_COM (window->pov);
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
swap_chain->GetDesc (&swap_desc);
d3d11window: Always reuse swapchain DXGI format can be updated via ResizeBuffers()
2020-03-26 12:20:20 +00:00
hr = swap_chain->ResizeBuffers (0, width, height, window->dxgi_format,
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
swap_desc.Flags);
if (!gst_d3d11_result (hr, window->device)) {
GST_ERROR_OBJECT (window, "Couldn't resize buffers, hr: 0x%x", (guint) hr);
goto done;
}
d3d11window: Misc code cleanup * Remove unnecessary upcasting. We are now dealing with C++ class objects and don't need explicit C-style casting in C++ world * Use helper macro IID_PPV_ARGS() everywhere. It will make code a little short. * Use ComPtr smart pointer instead of calling manual IUnknown::Release() Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2461>
2021-08-09 17:48:01 +00:00
hr = swap_chain->GetBuffer (0, IID_PPV_ARGS (&backbuffer));
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
if (!gst_d3d11_result (hr, window->device)) {
GST_ERROR_OBJECT (window,
"Cannot get backbuffer from swapchain, hr: 0x%x", (guint) hr);
goto done;
}
backbuffer->GetDesc (&desc);
window->surface_width = desc.Width;
window->surface_height = desc.Height;
{
dst_rect.x = 0;
dst_rect.y = 0;
dst_rect.w = window->surface_width;
dst_rect.h = window->surface_height;
if (window->force_aspect_ratio) {
d3d11videosink: Clarify the meaning of various width and height variables * Remove redundant variables for width/height and par from GstD3D11Window. GstVideoInfo holds all the values. * Don't need to pass par to gst_d3d11_window_prepare(). It will be parsed from caps again * Remove duplicated math Fixing regression of the commit 9dada901083fec3276cb2df58785db14b87e8f22
2020-03-02 11:55:29 +00:00
src_rect.x = 0;
src_rect.y = 0;
src_rect.w = GST_VIDEO_INFO_WIDTH (&window->render_info);
src_rect.h = GST_VIDEO_INFO_HEIGHT (&window->render_info);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
gst_video_sink_center_rect (src_rect, dst_rect, &rst_rect, TRUE);
} else {
rst_rect = dst_rect;
}
}
window->render_rect.left = rst_rect.x;
window->render_rect.top = rst_rect.y;
window->render_rect.right = rst_rect.x + rst_rect.w;
window->render_rect.bottom = rst_rect.y + rst_rect.h;
GST_LOG_OBJECT (window,
"New client area %dx%d, render rect x: %d, y: %d, %dx%d",
desc.Width, desc.Height, rst_rect.x, rst_rect.y, rst_rect.w, rst_rect.h);
d3d11window: Misc code cleanup * Remove unnecessary upcasting. We are now dealing with C++ class objects and don't need explicit C-style casting in C++ world * Use helper macro IID_PPV_ARGS() everywhere. It will make code a little short. * Use ComPtr smart pointer instead of calling manual IUnknown::Release() Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2461>
2021-08-09 17:48:01 +00:00
hr = device_handle->CreateRenderTargetView (backbuffer, NULL, &window->rtv);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
if (!gst_d3d11_result (hr, window->device)) {
GST_ERROR_OBJECT (window, "Cannot create render target view, hr: 0x%x",
(guint) hr);
goto done;
}
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
if (window->processor) {
D3D11_VIDEO_PROCESSOR_OUTPUT_VIEW_DESC pov_desc;
pov_desc.ViewDimension = D3D11_VPOV_DIMENSION_TEXTURE2D;
pov_desc.Texture2D.MipSlice = 0;
if (!gst_d3d11_video_processor_create_output_view (window->processor,
d3d11window: Misc code cleanup * Remove unnecessary upcasting. We are now dealing with C++ class objects and don't need explicit C-style casting in C++ world * Use helper macro IID_PPV_ARGS() everywhere. It will make code a little short. * Use ComPtr smart pointer instead of calling manual IUnknown::Release() Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2461>
2021-08-09 17:48:01 +00:00
&pov_desc, backbuffer, &window->pov))
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
goto done;
}
d3d11videosink: Add a property to support rendering statistics data on window Add a new property "render-stats" to allow rendering statistics data on window for debugging and/or development purpose. Text rendering will be accelerated by GPU since this implementation uses Direct2D/DirectWrite API and Direct3D inter-op for minimal overhead. Specifically, text data will be rendered on swapchain backbuffer directly without any copy/allocation of extra texture. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1830>
2020-11-21 19:39:57 +00:00
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
window->first_present = TRUE;
/* redraw the last scene if cached buffer exits */
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
if (window->cached_buffer) {
d3d11videosink: Remove DirectWrite related dead code It's now not enabled since we moved core part to gst-libs Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2108>
2021-03-25 12:17:07 +00:00
gst_d3d111_window_present (window, window->cached_buffer,
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
window->pov, window->rtv);
}
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
done:
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
GST_D3D11_CLEAR_COM (backbuffer);
Revert "d3d11: Enable native multi-thread protection layer and make use of it" This reverts commit 872b7f503c49442e559f6a381416c6a84b76a3c6. Native multi-thread protection layer seems to be consuming more CPU resource than application side protection approach in some cases Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 07:15:35 +00:00
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
gst_d3d11_device_unlock (window->device);
}
void
gst_d3d11_window_on_key_event (GstD3D11Window * window, const gchar * event,
const gchar * key)
{
g_return_if_fail (GST_IS_D3D11_WINDOW (window));
if (!window->enable_navigation_events)
return;
g_signal_emit (window, d3d11_window_signals[SIGNAL_KEY_EVENT], 0, event, key);
}
void
gst_d3d11_window_on_mouse_event (GstD3D11Window * window, const gchar * event,
gint button, gdouble x, gdouble y)
{
g_return_if_fail (GST_IS_D3D11_WINDOW (window));
if (!window->enable_navigation_events)
return;
g_signal_emit (window, d3d11_window_signals[SIGNAL_MOUSE_EVENT], 0,
event, button, x, y);
}
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
typedef struct
{
DXGI_FORMAT dxgi_format;
GstVideoFormat gst_format;
gboolean supported;
} GstD3D11WindowDisplayFormat;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
gboolean
d3d11videosink: Clarify the meaning of various width and height variables * Remove redundant variables for width/height and par from GstD3D11Window. GstVideoInfo holds all the values. * Don't need to pass par to gst_d3d11_window_prepare(). It will be parsed from caps again * Remove duplicated math Fixing regression of the commit 9dada901083fec3276cb2df58785db14b87e8f22
2020-03-02 11:55:29 +00:00
gst_d3d11_window_prepare (GstD3D11Window * window, guint display_width,
guint display_height, GstCaps * caps, gboolean * video_processor_available,
GError ** error)
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
{
GstD3D11WindowClass *klass;
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
g_return_val_if_fail (GST_IS_D3D11_WINDOW (window), FALSE);
klass = GST_D3D11_WINDOW_GET_CLASS (window);
g_assert (klass->prepare != NULL);
GST_DEBUG_OBJECT (window, "Prepare window, display resolution %dx%d, caps %"
GST_PTR_FORMAT, display_width, display_height, caps);
return klass->prepare (window, display_width, display_height, caps,
video_processor_available, error);
}
static gboolean
gst_d3d11_window_prepare_default (GstD3D11Window * window, guint display_width,
guint display_height, GstCaps * caps, gboolean * video_processor_available,
GError ** error)
{
GstD3D11WindowClass *klass;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
guint swapchain_flags = 0;
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
ID3D11Device *device_handle;
guint i;
guint num_supported_format = 0;
HRESULT hr;
UINT display_flags =
D3D11_FORMAT_SUPPORT_TEXTURE2D | D3D11_FORMAT_SUPPORT_DISPLAY;
UINT supported_flags = 0;
GstD3D11WindowDisplayFormat formats[] = {
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
{DXGI_FORMAT_R8G8B8A8_UNORM, GST_VIDEO_FORMAT_RGBA, FALSE},
{DXGI_FORMAT_B8G8R8A8_UNORM, GST_VIDEO_FORMAT_BGRA, FALSE},
{DXGI_FORMAT_R10G10B10A2_UNORM, GST_VIDEO_FORMAT_RGB10A2_LE, FALSE},
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
};
const GstD3D11WindowDisplayFormat *chosen_format = NULL;
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
const GstDxgiColorSpace *chosen_colorspace = NULL;
d3d11: Fix wrong preprocessing blocks Missed in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2051>
2021-03-02 11:37:04 +00:00
#if (GST_D3D11_DXGI_HEADER_VERSION >= 4)
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
gboolean have_hdr10 = FALSE;
DXGI_COLOR_SPACE_TYPE native_colorspace_type =
DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709;
#endif
d3d11: Fix wrong preprocessing blocks Missed in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2051>
2021-03-02 11:37:04 +00:00
#if (GST_D3D11_DXGI_HEADER_VERSION >= 5)
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
DXGI_HDR_METADATA_HDR10 hdr10_metadata = { 0, };
#endif
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
/* Step 1: Clear old resources and objects */
d3d11window: Clear cached buffer per new caps d3d11window holds one buffer to redraw client area per resize event. When the input format is being changed, this buffer should be cleared to avoid mismatch beween newly configured shader/videoprocessor and the format of previously cached buffer.
2020-02-05 13:12:15 +00:00
gst_clear_buffer (&window->cached_buffer);
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
g_clear_pointer (&window->processor, gst_d3d11_video_processor_free);
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
g_clear_pointer (&window->converter, gst_d3d11_converter_free);
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
g_clear_pointer (&window->compositor, gst_d3d11_overlay_compositor_free);
d3d11videosink: Avoid switching conversion tool during playback Decoder might be able to copy decoded texture to the other buffer pool during playback depending on context. In that case, copied one has no D3D11_BIND_DECODER bind flag. If we used ID3D11VideoProcessor previously for decoder texture, and incoming texture supports ID3D11VideoProcessor as well even if it has no D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example), allow zero-copying instead of using our fallback texture. Frequent conversion tool change (between ID3D11VideoProcessor and generic shader) might result in inconsistent image quality. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 10:48:56 +00:00
window->processor_in_use = FALSE;
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
/* Step 2: Decide display color format
* If upstream format is 10bits, try DXGI_FORMAT_R10G10B10A2_UNORM first
* Otherwise, use DXGI_FORMAT_B8G8R8A8_UNORM or DXGI_FORMAT_B8G8R8A8_UNORM
*/
gst_video_info_from_caps (&window->info, caps);
device_handle = gst_d3d11_device_get_device_handle (window->device);
for (i = 0; i < G_N_ELEMENTS (formats); i++) {
hr = device_handle->CheckFormatSupport (formats[i].dxgi_format,
&supported_flags);
if (SUCCEEDED (hr) && (supported_flags & display_flags) == display_flags) {
GST_DEBUG_OBJECT (window, "Device supports format %s (DXGI_FORMAT %d)",
gst_video_format_to_string (formats[i].gst_format),
formats[i].dxgi_format);
formats[i].supported = TRUE;
num_supported_format++;
}
}
d3d11window: Clear cached buffer per new caps d3d11window holds one buffer to redraw client area per resize event. When the input format is being changed, this buffer should be cleared to avoid mismatch beween newly configured shader/videoprocessor and the format of previously cached buffer.
2020-02-05 13:12:15 +00:00
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
if (num_supported_format == 0) {
GST_ERROR_OBJECT (window, "Cannot determine render format");
g_set_error (error, GST_RESOURCE_ERROR, GST_RESOURCE_ERROR_FAILED,
"Cannot determine render format");
return FALSE;
}
d3d11videosink: Remove DirectWrite related dead code It's now not enabled since we moved core part to gst-libs Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2108>
2021-03-25 12:17:07 +00:00
for (i = 0; i < GST_VIDEO_INFO_N_COMPONENTS (&window->info); i++) {
if (GST_VIDEO_INFO_COMP_DEPTH (&window->info, i) > 8) {
if (formats[2].supported) {
chosen_format = &formats[2];
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
}
d3d11videosink: Remove DirectWrite related dead code It's now not enabled since we moved core part to gst-libs Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2108>
2021-03-25 12:17:07 +00:00
break;
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
}
}
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
if (!chosen_format) {
/* prefer native format over conversion */
for (i = 0; i < 2; i++) {
if (formats[i].supported &&
formats[i].gst_format == GST_VIDEO_INFO_FORMAT (&window->info)) {
chosen_format = &formats[i];
break;
}
}
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
/* choose any color space then */
if (!chosen_format) {
for (i = 0; i < G_N_ELEMENTS (formats); i++) {
if (formats[i].supported) {
chosen_format = &formats[i];
break;
}
}
}
}
g_assert (chosen_format != NULL);
d3d11window: Fix typo in debug message Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1430>
2021-12-09 10:51:04 +00:00
GST_DEBUG_OBJECT (window, "chosen render format %s (DXGI_FORMAT %d)",
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
gst_video_format_to_string (chosen_format->gst_format),
chosen_format->dxgi_format);
/* Step 3: Create swapchain
* (or reuse old swapchain if the format is not changed) */
window->allow_tearing = FALSE;
d3d11: d3d11device: Remove "allow-tearing" property Plugin can query DXGI_FEATURE_PRESENT_ALLOW_TEARING without d3d11device help Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1098>
2021-10-08 08:16:02 +00:00
#if (GST_D3D11_DXGI_HEADER_VERSION >= 5)
{
ComPtr < IDXGIFactory5 > factory5;
IDXGIFactory1 *factory_handle;
BOOL allow_tearing = FALSE;
factory_handle = gst_d3d11_device_get_dxgi_factory_handle (window->device);
hr = factory_handle->QueryInterface (IID_PPV_ARGS (&factory5));
if (SUCCEEDED (hr)) {
hr = factory5->CheckFeatureSupport (DXGI_FEATURE_PRESENT_ALLOW_TEARING,
(void *) &allow_tearing, sizeof (allow_tearing));
}
if (SUCCEEDED (hr) && allow_tearing)
window->allow_tearing = allow_tearing;
}
#endif
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
if (window->allow_tearing) {
GST_DEBUG_OBJECT (window, "device support tearning");
swapchain_flags |= DXGI_SWAP_CHAIN_FLAG_ALLOW_TEARING;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
}
Revert "d3d11: Enable native multi-thread protection layer and make use of it" This reverts commit 872b7f503c49442e559f6a381416c6a84b76a3c6. Native multi-thread protection layer seems to be consuming more CPU resource than application side protection approach in some cases Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 07:15:35 +00:00
gst_d3d11_device_lock (window->device);
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
window->dxgi_format = chosen_format->dxgi_format;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
klass = GST_D3D11_WINDOW_GET_CLASS (window);
if (!window->swap_chain &&
!klass->create_swap_chain (window, window->dxgi_format,
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
display_width, display_height, swapchain_flags,
&window->swap_chain)) {
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
GST_ERROR_OBJECT (window, "Cannot create swapchain");
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
g_set_error (error, GST_RESOURCE_ERROR, GST_RESOURCE_ERROR_FAILED,
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
"Cannot create swapchain");
d3d11: Protect ID3D11VideoContext with lock Likewise d3d11 immediate context (i.e., ID3D11DeviceContext), ID3D11VideoContext API is not thread safe. It must be protected therefore. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1856>
2020-12-03 18:40:17 +00:00
goto error;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
}
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
/* this rect struct will be used to calculate render area */
window->render_rect.left = 0;
window->render_rect.top = 0;
window->render_rect.right = display_width;
window->render_rect.bottom = display_height;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
window->input_rect.left = 0;
window->input_rect.top = 0;
window->input_rect.right = GST_VIDEO_INFO_WIDTH (&window->info);
window->input_rect.bottom = GST_VIDEO_INFO_HEIGHT (&window->info);
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
/* Step 4: Decide render color space and set it on converter/processor */
d3d11: Fix wrong preprocessing blocks Missed in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2051>
2021-03-02 11:37:04 +00:00
#if (GST_D3D11_DXGI_HEADER_VERSION >= 5)
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
{
GstVideoMasteringDisplayInfo minfo;
GstVideoContentLightLevel cll;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
if (gst_video_mastering_display_info_from_caps (&minfo, caps) &&
gst_video_content_light_level_from_caps (&cll, caps)) {
d3d11window: Misc code cleanup * Remove unnecessary upcasting. We are now dealing with C++ class objects and don't need explicit C-style casting in C++ world * Use helper macro IID_PPV_ARGS() everywhere. It will make code a little short. * Use ComPtr smart pointer instead of calling manual IUnknown::Release() Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2461>
2021-08-09 17:48:01 +00:00
ComPtr < IDXGISwapChain4 > swapchain4;
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
HRESULT hr;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11window: Misc code cleanup * Remove unnecessary upcasting. We are now dealing with C++ class objects and don't need explicit C-style casting in C++ world * Use helper macro IID_PPV_ARGS() everywhere. It will make code a little short. * Use ComPtr smart pointer instead of calling manual IUnknown::Release() Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2461>
2021-08-09 17:48:01 +00:00
hr = window->swap_chain->QueryInterface (IID_PPV_ARGS (&swapchain4));
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
if (gst_d3d11_result (hr, window->device)) {
GST_DEBUG_OBJECT (window, "Have HDR metadata, set to DXGI swapchain");
gst_d3d11_hdr_meta_data_to_dxgi (&minfo, &cll, &hdr10_metadata);
hr = swapchain4->SetHDRMetaData (DXGI_HDR_METADATA_TYPE_HDR10,
sizeof (DXGI_HDR_METADATA_HDR10), &hdr10_metadata);
if (!gst_d3d11_result (hr, window->device)) {
GST_WARNING_OBJECT (window, "Couldn't set HDR metadata, hr 0x%x",
(guint) hr);
} else {
have_hdr10 = TRUE;
}
}
}
}
#endif
/* Step 5: Choose display color space */
gst_video_info_set_format (&window->render_info,
chosen_format->gst_format, display_width, display_height);
/* preserve upstream colorimetry */
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
window->render_info.colorimetry.primaries =
window->info.colorimetry.primaries;
window->render_info.colorimetry.transfer = window->info.colorimetry.transfer;
d3d11window: Prefer full color range for display target colorspace We don't need to preserve input color range for transformed target color space. Also some GPUs doesn't seem to be happy with 16-235 color range for RGB color space. Also, since our default display target color space is DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709, choosing full color range would make more sense. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1814>
2020-11-16 11:44:06 +00:00
/* prefer FULL range RGB. STUDIO range doesn't seem to be well supported
* color space by GPUs and we don't need to preserve color range for
* target display color space type */
window->render_info.colorimetry.range = GST_VIDEO_COLOR_RANGE_0_255;
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
d3d11: Fix wrong preprocessing blocks Missed in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2051>
2021-03-02 11:37:04 +00:00
#if (GST_D3D11_DXGI_HEADER_VERSION >= 4)
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
{
d3d11window: Misc code cleanup * Remove unnecessary upcasting. We are now dealing with C++ class objects and don't need explicit C-style casting in C++ world * Use helper macro IID_PPV_ARGS() everywhere. It will make code a little short. * Use ComPtr smart pointer instead of calling manual IUnknown::Release() Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2461>
2021-08-09 17:48:01 +00:00
ComPtr < IDXGISwapChain3 > swapchain3;
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
HRESULT hr;
d3d11window: Misc code cleanup * Remove unnecessary upcasting. We are now dealing with C++ class objects and don't need explicit C-style casting in C++ world * Use helper macro IID_PPV_ARGS() everywhere. It will make code a little short. * Use ComPtr smart pointer instead of calling manual IUnknown::Release() Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2461>
2021-08-09 17:48:01 +00:00
hr = window->swap_chain->QueryInterface (IID_PPV_ARGS (&swapchain3));
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
if (gst_d3d11_result (hr, window->device)) {
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
chosen_colorspace =
gst_d3d11_find_swap_chain_color_space (&window->render_info,
d3d11window: Remove hack related to color space selection Use input GstVideoColorPrimaries without any special case handling otherwise rendered image color would be very wrong. The hack was added to work around an issue that some Intel driver couldn't handle wide color gamut image without HDR10 metadata, specifically PQ image. But device capability can be checked via a method added in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1723 so there's no issue now. Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1175 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1430>
2021-12-09 10:55:04 +00:00
swapchain3.Get ());
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
if (chosen_colorspace) {
native_colorspace_type =
(DXGI_COLOR_SPACE_TYPE) chosen_colorspace->dxgi_color_space_type;
hr = swapchain3->SetColorSpace1 (native_colorspace_type);
if (!gst_d3d11_result (hr, window->device)) {
GST_WARNING_OBJECT (window, "Failed to set colorspace %d, hr: 0x%x",
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
native_colorspace_type, (guint) hr);
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
chosen_colorspace = NULL;
native_colorspace_type = DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709;
} else {
GST_DEBUG_OBJECT (window,
"Set colorspace %d", native_colorspace_type);
/* update with selected display color space */
window->render_info.colorimetry.primaries =
chosen_colorspace->primaries;
window->render_info.colorimetry.transfer =
chosen_colorspace->transfer;
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
window->render_info.colorimetry.range = chosen_colorspace->range;
window->render_info.colorimetry.matrix = chosen_colorspace->matrix;
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
}
}
}
}
#endif
/* otherwise, use most common DXGI_COLOR_SPACE_RGB_FULL_G22_NONE_P709
* color space */
if (!chosen_colorspace) {
GST_DEBUG_OBJECT (window, "No selected render color space, use BT709");
window->render_info.colorimetry.primaries = GST_VIDEO_COLOR_PRIMARIES_BT709;
window->render_info.colorimetry.transfer = GST_VIDEO_TRANSFER_BT709;
window->render_info.colorimetry.range = GST_VIDEO_COLOR_RANGE_0_255;
}
d3d11: Fix wrong preprocessing blocks Missed in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2051>
2021-03-02 11:37:04 +00:00
#if (GST_D3D11_DXGI_HEADER_VERSION >= 4)
d3d11window: Use ID3D11VideoProcessor only if device supports corresponding conversion ... and drop support for ID3D11VideoProcessor if device doesn't support ID3D11VideoContext1 interface and therefore we cannot query conversion supportability. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1743>
2020-10-30 18:28:55 +00:00
if (chosen_colorspace) {
const GstDxgiColorSpace *in_color_space =
gst_d3d11_video_info_to_dxgi_color_space (&window->info);
const GstD3D11Format *in_format =
gst_d3d11_device_format_from_gst (window->device,
GST_VIDEO_INFO_FORMAT (&window->info));
gboolean hardware = FALSE;
GstD3D11VideoProcessor *processor = NULL;
d3d11window: Disable d3d11 video processor for HDR10 stream. d3d11 video processor is showing buggy behaviors in various cases. And this HDR10 is one case of them. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1454>
2020-07-21 14:44:03 +00:00
d3d11window: Use ID3D11VideoProcessor only if device supports corresponding conversion ... and drop support for ID3D11VideoProcessor if device doesn't support ID3D11VideoContext1 interface and therefore we cannot query conversion supportability. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1743>
2020-10-30 18:28:55 +00:00
if (in_color_space && in_format &&
in_format->dxgi_format != DXGI_FORMAT_UNKNOWN) {
g_object_get (window->device, "hardware", &hardware, NULL);
}
if (hardware) {
processor =
d3d11window: Do not configure video processor for Xbox device Disable video processor for Xbox until it's verified Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1339>
2020-06-12 11:18:53 +00:00
gst_d3d11_video_processor_new (window->device,
GST_VIDEO_INFO_WIDTH (&window->info),
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
GST_VIDEO_INFO_HEIGHT (&window->info), display_width, display_height);
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
}
d3d11window: Use ID3D11VideoProcessor only if device supports corresponding conversion ... and drop support for ID3D11VideoProcessor if device doesn't support ID3D11VideoContext1 interface and therefore we cannot query conversion supportability. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1743>
2020-10-30 18:28:55 +00:00
if (processor) {
DXGI_FORMAT in_dxgi_format = in_format->dxgi_format;
DXGI_FORMAT out_dxgi_format = chosen_format->dxgi_format;
DXGI_COLOR_SPACE_TYPE in_dxgi_color_space =
(DXGI_COLOR_SPACE_TYPE) in_color_space->dxgi_color_space_type;
DXGI_COLOR_SPACE_TYPE out_dxgi_color_space = native_colorspace_type;
if (!gst_d3d11_video_processor_check_format_conversion (processor,
in_dxgi_format, in_dxgi_color_space, out_dxgi_format,
out_dxgi_color_space)) {
GST_DEBUG_OBJECT (window, "Conversion is not supported by device");
gst_d3d11_video_processor_free (processor);
processor = NULL;
} else {
GST_DEBUG_OBJECT (window, "video processor supports conversion");
gst_d3d11_video_processor_set_input_dxgi_color_space (processor,
in_dxgi_color_space);
gst_d3d11_video_processor_set_output_dxgi_color_space (processor,
out_dxgi_color_space);
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
d3d11: Fix wrong preprocessing blocks Missed in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2051>
2021-03-02 11:37:04 +00:00
#if (GST_D3D11_DXGI_HEADER_VERSION >= 5)
d3d11window: Use ID3D11VideoProcessor only if device supports corresponding conversion ... and drop support for ID3D11VideoProcessor if device doesn't support ID3D11VideoContext1 interface and therefore we cannot query conversion supportability. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1743>
2020-10-30 18:28:55 +00:00
if (have_hdr10) {
GST_DEBUG_OBJECT (window, "Set HDR metadata on video processor");
gst_d3d11_video_processor_set_input_hdr10_metadata (processor,
&hdr10_metadata);
gst_d3d11_video_processor_set_output_hdr10_metadata (processor,
&hdr10_metadata);
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
}
#endif
}
d3d11window: Use ID3D11VideoProcessor only if device supports corresponding conversion ... and drop support for ID3D11VideoProcessor if device doesn't support ID3D11VideoContext1 interface and therefore we cannot query conversion supportability. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1743>
2020-10-30 18:28:55 +00:00
window->processor = processor;
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
}
}
d3d11window: Use ID3D11VideoProcessor only if device supports corresponding conversion ... and drop support for ID3D11VideoProcessor if device doesn't support ID3D11VideoContext1 interface and therefore we cannot query conversion supportability. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1743>
2020-10-30 18:28:55 +00:00
#endif
*video_processor_available = !!window->processor;
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
/* configure shader even if video processor is available for fallback */
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
window->converter =
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
gst_d3d11_converter_new (window->device, &window->info,
d3d11converter: Introduce config to be extensible Add a config argument like that of GstVideoConverter so that we can add more options without modifying existing methods Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2276>
2021-05-17 16:24:29 +00:00
&window->render_info, nullptr);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
if (!window->converter) {
GST_ERROR_OBJECT (window, "Cannot create converter");
g_set_error (error, GST_RESOURCE_ERROR, GST_RESOURCE_ERROR_FAILED,
"Cannot create converter");
d3d11: Protect ID3D11VideoContext with lock Likewise d3d11 immediate context (i.e., ID3D11DeviceContext), ID3D11VideoContext API is not thread safe. It must be protected therefore. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1856>
2020-12-03 18:40:17 +00:00
goto error;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
}
window->compositor =
gst_d3d11_overlay_compositor_new (window->device, &window->render_info);
if (!window->compositor) {
GST_ERROR_OBJECT (window, "Cannot create overlay compositor");
g_set_error (error, GST_RESOURCE_ERROR, GST_RESOURCE_ERROR_FAILED,
"Cannot create overlay compositor");
d3d11: Protect ID3D11VideoContext with lock Likewise d3d11 immediate context (i.e., ID3D11DeviceContext), ID3D11VideoContext API is not thread safe. It must be protected therefore. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1856>
2020-12-03 18:40:17 +00:00
goto error;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
}
Revert "d3d11: Enable native multi-thread protection layer and make use of it" This reverts commit 872b7f503c49442e559f6a381416c6a84b76a3c6. Native multi-thread protection layer seems to be consuming more CPU resource than application side protection approach in some cases Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 07:15:35 +00:00
gst_d3d11_device_unlock (window->device);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11window: Invoke initial resize method from baseclass ... instead of calling from subclass in order for baseclass to handle more things between swapchain creation and resource creation.
2020-01-10 12:45:43 +00:00
/* call resize to allocated resources */
d3d11videosink: Clarify the meaning of various width and height variables * Remove redundant variables for width/height and par from GstD3D11Window. GstVideoInfo holds all the values. * Don't need to pass par to gst_d3d11_window_prepare(). It will be parsed from caps again * Remove duplicated math Fixing regression of the commit 9dada901083fec3276cb2df58785db14b87e8f22
2020-03-02 11:55:29 +00:00
klass->on_resize (window, display_width, display_height);
d3d11window: Invoke initial resize method from baseclass ... instead of calling from subclass in order for baseclass to handle more things between swapchain creation and resource creation.
2020-01-10 12:45:43 +00:00
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
if (window->requested_fullscreen != window->fullscreen) {
klass->change_fullscreen_mode (window);
}
GST_DEBUG_OBJECT (window, "New swap chain 0x%p created", window->swap_chain);
return TRUE;
d3d11: Protect ID3D11VideoContext with lock Likewise d3d11 immediate context (i.e., ID3D11DeviceContext), ID3D11VideoContext API is not thread safe. It must be protected therefore. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1856>
2020-12-03 18:40:17 +00:00
error:
Revert "d3d11: Enable native multi-thread protection layer and make use of it" This reverts commit 872b7f503c49442e559f6a381416c6a84b76a3c6. Native multi-thread protection layer seems to be consuming more CPU resource than application side protection approach in some cases Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 07:15:35 +00:00
gst_d3d11_device_unlock (window->device);
d3d11: Protect ID3D11VideoContext with lock Likewise d3d11 immediate context (i.e., ID3D11DeviceContext), ID3D11VideoContext API is not thread safe. It must be protected therefore. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1856>
2020-12-03 18:40:17 +00:00
return FALSE;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
}
void
gst_d3d11_window_show (GstD3D11Window * window)
{
GstD3D11WindowClass *klass;
g_return_if_fail (GST_IS_D3D11_WINDOW (window));
klass = GST_D3D11_WINDOW_GET_CLASS (window);
if (klass->show)
klass->show (window);
}
void
d3d11videosink: Add support for GstVideoOverlay::set_render_rectangle Inspired by an MR https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2382 The idea is that we can make use of MoveWindow() in WIN32 d3d11window implementation safely because WIN32 d3d11window implementation creates internal HWND even when external HWND is set and then subclassing is used to draw on internal HWND in any case. So the coordinates passed to MoveWindow() will be relative to parent HWND, and it meets well to the concept of set_render_rectangle(). On MoveWindow() event, WM_SIZE event will be generated by OS and then GstD3D11WindowWin32 implementation will update render area including swapchain correspondingly, as if it's normal window move/resize case. But in case of UWP (CoreWindow or SwapChainPanel), we need more research to meet expected behavior of set_render_rectangle() Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1416 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2450>
2021-07-30 15:59:14 +00:00
gst_d3d11_window_set_render_rectangle (GstD3D11Window * window,
const GstVideoRectangle * rect)
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
{
d3d11videosink: Add support for GstVideoOverlay::set_render_rectangle Inspired by an MR https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2382 The idea is that we can make use of MoveWindow() in WIN32 d3d11window implementation safely because WIN32 d3d11window implementation creates internal HWND even when external HWND is set and then subclassing is used to draw on internal HWND in any case. So the coordinates passed to MoveWindow() will be relative to parent HWND, and it meets well to the concept of set_render_rectangle(). On MoveWindow() event, WM_SIZE event will be generated by OS and then GstD3D11WindowWin32 implementation will update render area including swapchain correspondingly, as if it's normal window move/resize case. But in case of UWP (CoreWindow or SwapChainPanel), we need more research to meet expected behavior of set_render_rectangle() Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1416 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2450>
2021-07-30 15:59:14 +00:00
GstD3D11WindowClass *klass;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
g_return_if_fail (GST_IS_D3D11_WINDOW (window));
d3d11videosink: Add support for GstVideoOverlay::set_render_rectangle Inspired by an MR https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2382 The idea is that we can make use of MoveWindow() in WIN32 d3d11window implementation safely because WIN32 d3d11window implementation creates internal HWND even when external HWND is set and then subclassing is used to draw on internal HWND in any case. So the coordinates passed to MoveWindow() will be relative to parent HWND, and it meets well to the concept of set_render_rectangle(). On MoveWindow() event, WM_SIZE event will be generated by OS and then GstD3D11WindowWin32 implementation will update render area including swapchain correspondingly, as if it's normal window move/resize case. But in case of UWP (CoreWindow or SwapChainPanel), we need more research to meet expected behavior of set_render_rectangle() Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1416 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2450>
2021-07-30 15:59:14 +00:00
klass = GST_D3D11_WINDOW_GET_CLASS (window);
if (klass->set_render_rectangle)
klass->set_render_rectangle (window, rect);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
}
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
void
gst_d3d11_window_set_title (GstD3D11Window * window, const gchar * title)
{
GstD3D11WindowClass *klass;
g_return_if_fail (GST_IS_D3D11_WINDOW (window));
klass = GST_D3D11_WINDOW_GET_CLASS (window);
if (klass->set_title)
klass->set_title (window, title);
}
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
static gboolean
gst_d3d11_window_buffer_ensure_processor_input (GstD3D11Window * self,
GstBuffer * buffer, ID3D11VideoProcessorInputView ** in_view)
{
GstD3D11Memory *mem;
d3d11: Privatize d3d11memory implementation Hide most of symbols of GstD3D11Memory object. GstD3D11Memory is one of primary resource for imcoming d3d11 library and it's expected to be a extensible feature. Hiding implementation detail would be helpful for later use case. Summary of this commit: * Now all native Direct3D11 resources are private of GstD3D11Memory. To access native resources, getter methods need to be used or generic map (e.g., gst_memory_map) API should be called apart from some exceptional case such as d3d11decoder case. * Various helper methods are added for GstBuffer related operations and in order to remove duplicated code. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1892>
2020-12-19 17:39:40 +00:00
ID3D11VideoProcessorInputView *piv;
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
if (!self->processor)
return FALSE;
if (gst_buffer_n_memory (buffer) != 1)
return FALSE;
mem = (GstD3D11Memory *) gst_buffer_peek_memory (buffer, 0);
d3d11: Privatize d3d11memory implementation Hide most of symbols of GstD3D11Memory object. GstD3D11Memory is one of primary resource for imcoming d3d11 library and it's expected to be a extensible feature. Hiding implementation detail would be helpful for later use case. Summary of this commit: * Now all native Direct3D11 resources are private of GstD3D11Memory. To access native resources, getter methods need to be used or generic map (e.g., gst_memory_map) API should be called apart from some exceptional case such as d3d11decoder case. * Various helper methods are added for GstBuffer related operations and in order to remove duplicated code. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1892>
2020-12-19 17:39:40 +00:00
piv = gst_d3d11_video_processor_get_input_view (self->processor, mem);
if (!piv) {
GST_LOG_OBJECT (self, "Failed to get processor input view");
d3d11window: Reuse ID3D11VideoProcessorInputView if possible GstMemory object could be disposed if GstBuffer is not allocated by GstD3D11BufferPool such as via gst_buffer_copy() and/or gst_buffer_make_writable(). So attaching qdata on GstMemory object would cause unnecessary view alloc/free. By using view pool which is implemented in GstD3D11Allocator, we can avoid redundant view alloc/free. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1716>
2020-10-23 07:59:00 +00:00
return FALSE;
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
}
d3d11: Privatize d3d11memory implementation Hide most of symbols of GstD3D11Memory object. GstD3D11Memory is one of primary resource for imcoming d3d11 library and it's expected to be a extensible feature. Hiding implementation detail would be helpful for later use case. Summary of this commit: * Now all native Direct3D11 resources are private of GstD3D11Memory. To access native resources, getter methods need to be used or generic map (e.g., gst_memory_map) API should be called apart from some exceptional case such as d3d11decoder case. * Various helper methods are added for GstBuffer related operations and in order to remove duplicated code. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1892>
2020-12-19 17:39:40 +00:00
*in_view = piv;
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
return TRUE;
}
d3d11videosink: Avoid switching conversion tool during playback Decoder might be able to copy decoded texture to the other buffer pool during playback depending on context. In that case, copied one has no D3D11_BIND_DECODER bind flag. If we used ID3D11VideoProcessor previously for decoder texture, and incoming texture supports ID3D11VideoProcessor as well even if it has no D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example), allow zero-copying instead of using our fallback texture. Frequent conversion tool change (between ID3D11VideoProcessor and generic shader) might result in inconsistent image quality. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 10:48:56 +00:00
static gboolean
gst_d3d11_window_do_processor (GstD3D11Window * self,
d3d11videosink: Add support for crop meta ... when upstream element is d3d11. Note that, if upstream element is not d3d11, crop meta is almost pointless since d3d11videosink will upload video frame to GPU memory in any case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/933>
2021-09-18 14:37:20 +00:00
ID3D11VideoProcessorInputView * piv, ID3D11VideoProcessorOutputView * pov,
RECT * input_rect)
d3d11videosink: Avoid switching conversion tool during playback Decoder might be able to copy decoded texture to the other buffer pool during playback depending on context. In that case, copied one has no D3D11_BIND_DECODER bind flag. If we used ID3D11VideoProcessor previously for decoder texture, and incoming texture supports ID3D11VideoProcessor as well even if it has no D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example), allow zero-copying instead of using our fallback texture. Frequent conversion tool change (between ID3D11VideoProcessor and generic shader) might result in inconsistent image quality. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 10:48:56 +00:00
{
gboolean ret;
ret = gst_d3d11_video_processor_render_unlocked (self->processor,
d3d11videosink: Add support for crop meta ... when upstream element is d3d11. Note that, if upstream element is not d3d11, crop meta is almost pointless since d3d11videosink will upload video frame to GPU memory in any case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/933>
2021-09-18 14:37:20 +00:00
input_rect, piv, &self->render_rect, pov);
d3d11videosink: Avoid switching conversion tool during playback Decoder might be able to copy decoded texture to the other buffer pool during playback depending on context. In that case, copied one has no D3D11_BIND_DECODER bind flag. If we used ID3D11VideoProcessor previously for decoder texture, and incoming texture supports ID3D11VideoProcessor as well even if it has no D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example), allow zero-copying instead of using our fallback texture. Frequent conversion tool change (between ID3D11VideoProcessor and generic shader) might result in inconsistent image quality. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 10:48:56 +00:00
if (!ret) {
GST_ERROR_OBJECT (self, "Couldn't render to backbuffer using processor");
} else {
GST_TRACE_OBJECT (self, "Rendered using processor");
self->processor_in_use = TRUE;
}
return ret;
}
static gboolean
gst_d3d11_window_do_convert (GstD3D11Window * self,
ID3D11ShaderResourceView * srv[GST_VIDEO_MAX_PLANES],
d3d11videosink: Add support for crop meta ... when upstream element is d3d11. Note that, if upstream element is not d3d11, crop meta is almost pointless since d3d11videosink will upload video frame to GPU memory in any case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/933>
2021-09-18 14:37:20 +00:00
ID3D11RenderTargetView * rtv, RECT * input_rect)
d3d11videosink: Avoid switching conversion tool during playback Decoder might be able to copy decoded texture to the other buffer pool during playback depending on context. In that case, copied one has no D3D11_BIND_DECODER bind flag. If we used ID3D11VideoProcessor previously for decoder texture, and incoming texture supports ID3D11VideoProcessor as well even if it has no D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example), allow zero-copying instead of using our fallback texture. Frequent conversion tool change (between ID3D11VideoProcessor and generic shader) might result in inconsistent image quality. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 10:48:56 +00:00
{
d3d11videosink: Add support for crop meta ... when upstream element is d3d11. Note that, if upstream element is not d3d11, crop meta is almost pointless since d3d11videosink will upload video frame to GPU memory in any case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/933>
2021-09-18 14:37:20 +00:00
if (!gst_d3d11_converter_update_src_rect (self->converter, input_rect)) {
GST_ERROR_OBJECT (self, "Failed to update src rect");
return FALSE;
}
d3d11videosink: Avoid switching conversion tool during playback Decoder might be able to copy decoded texture to the other buffer pool during playback depending on context. In that case, copied one has no D3D11_BIND_DECODER bind flag. If we used ID3D11VideoProcessor previously for decoder texture, and incoming texture supports ID3D11VideoProcessor as well even if it has no D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example), allow zero-copying instead of using our fallback texture. Frequent conversion tool change (between ID3D11VideoProcessor and generic shader) might result in inconsistent image quality. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 10:48:56 +00:00
if (!gst_d3d11_converter_convert_unlocked (self->converter,
srv, &rtv, NULL, NULL)) {
GST_ERROR_OBJECT (self, "Couldn't render to backbuffer using converter");
return FALSE;
} else {
GST_TRACE_OBJECT (self, "Rendered using converter");
}
return TRUE;
}
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
static GstFlowReturn
d3d11videosink: Add a property to support rendering statistics data on window Add a new property "render-stats" to allow rendering statistics data on window for debugging and/or development purpose. Text rendering will be accelerated by GPU since this implementation uses Direct2D/DirectWrite API and Direct3D inter-op for minimal overhead. Specifically, text data will be rendered on swapchain backbuffer directly without any copy/allocation of extra texture. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1830>
2020-11-21 19:39:57 +00:00
gst_d3d111_window_present (GstD3D11Window * self, GstBuffer * buffer,
d3d11videosink: Remove DirectWrite related dead code It's now not enabled since we moved core part to gst-libs Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2108>
2021-03-25 12:17:07 +00:00
ID3D11VideoProcessorOutputView * pov, ID3D11RenderTargetView * rtv)
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
{
GstD3D11WindowClass *klass = GST_D3D11_WINDOW_GET_CLASS (self);
GstFlowReturn ret = GST_FLOW_OK;
guint present_flags = 0;
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
if (!buffer)
return GST_FLOW_OK;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
{
d3d11: Privatize d3d11memory implementation Hide most of symbols of GstD3D11Memory object. GstD3D11Memory is one of primary resource for imcoming d3d11 library and it's expected to be a extensible feature. Hiding implementation detail would be helpful for later use case. Summary of this commit: * Now all native Direct3D11 resources are private of GstD3D11Memory. To access native resources, getter methods need to be used or generic map (e.g., gst_memory_map) API should be called apart from some exceptional case such as d3d11decoder case. * Various helper methods are added for GstBuffer related operations and in order to remove duplicated code. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1892>
2020-12-19 17:39:40 +00:00
GstMapInfo infos[GST_VIDEO_MAX_PLANES];
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
ID3D11ShaderResourceView *srv[GST_VIDEO_MAX_PLANES];
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
ID3D11VideoProcessorInputView *piv = NULL;
d3d11: Privatize d3d11memory implementation Hide most of symbols of GstD3D11Memory object. GstD3D11Memory is one of primary resource for imcoming d3d11 library and it's expected to be a extensible feature. Hiding implementation detail would be helpful for later use case. Summary of this commit: * Now all native Direct3D11 resources are private of GstD3D11Memory. To access native resources, getter methods need to be used or generic map (e.g., gst_memory_map) API should be called apart from some exceptional case such as d3d11decoder case. * Various helper methods are added for GstBuffer related operations and in order to remove duplicated code. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1892>
2020-12-19 17:39:40 +00:00
ID3D11Device *device_handle =
gst_d3d11_device_get_device_handle (self->device);
d3d11videosink: Avoid switching conversion tool during playback Decoder might be able to copy decoded texture to the other buffer pool during playback depending on context. In that case, copied one has no D3D11_BIND_DECODER bind flag. If we used ID3D11VideoProcessor previously for decoder texture, and incoming texture supports ID3D11VideoProcessor as well even if it has no D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example), allow zero-copying instead of using our fallback texture. Frequent conversion tool change (between ID3D11VideoProcessor and generic shader) might result in inconsistent image quality. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 10:48:56 +00:00
gboolean can_convert = FALSE;
gboolean can_process = FALSE;
gboolean convert_ret = FALSE;
d3d11videosink: Add support for crop meta ... when upstream element is d3d11. Note that, if upstream element is not d3d11, crop meta is almost pointless since d3d11videosink will upload video frame to GPU memory in any case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/933>
2021-09-18 14:37:20 +00:00
RECT input_rect = self->input_rect;
GstVideoCropMeta *crop_meta;
d3d11: Privatize d3d11memory implementation Hide most of symbols of GstD3D11Memory object. GstD3D11Memory is one of primary resource for imcoming d3d11 library and it's expected to be a extensible feature. Hiding implementation detail would be helpful for later use case. Summary of this commit: * Now all native Direct3D11 resources are private of GstD3D11Memory. To access native resources, getter methods need to be used or generic map (e.g., gst_memory_map) API should be called apart from some exceptional case such as d3d11decoder case. * Various helper methods are added for GstBuffer related operations and in order to remove duplicated code. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1892>
2020-12-19 17:39:40 +00:00
/* Map memory in any case so that we can upload pending stage texture */
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
if (!gst_d3d11_buffer_map (buffer, device_handle, infos, GST_MAP_READ)) {
d3d11: Privatize d3d11memory implementation Hide most of symbols of GstD3D11Memory object. GstD3D11Memory is one of primary resource for imcoming d3d11 library and it's expected to be a extensible feature. Hiding implementation detail would be helpful for later use case. Summary of this commit: * Now all native Direct3D11 resources are private of GstD3D11Memory. To access native resources, getter methods need to be used or generic map (e.g., gst_memory_map) API should be called apart from some exceptional case such as d3d11decoder case. * Various helper methods are added for GstBuffer related operations and in order to remove duplicated code. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1892>
2020-12-19 17:39:40 +00:00
GST_ERROR_OBJECT (self, "Couldn't map buffer");
return GST_FLOW_ERROR;
}
d3d11videosink: Avoid switching conversion tool during playback Decoder might be able to copy decoded texture to the other buffer pool during playback depending on context. In that case, copied one has no D3D11_BIND_DECODER bind flag. If we used ID3D11VideoProcessor previously for decoder texture, and incoming texture supports ID3D11VideoProcessor as well even if it has no D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example), allow zero-copying instead of using our fallback texture. Frequent conversion tool change (between ID3D11VideoProcessor and generic shader) might result in inconsistent image quality. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 10:48:56 +00:00
can_convert = gst_d3d11_buffer_get_shader_resource_view (buffer, srv);
if (pov) {
can_process = gst_d3d11_window_buffer_ensure_processor_input (self,
buffer, &piv);
}
if (!can_convert && !can_process) {
GST_ERROR_OBJECT (self, "Input texture cannot be used for converter");
return GST_FLOW_ERROR;
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
}
d3d11videosink: Add support for crop meta ... when upstream element is d3d11. Note that, if upstream element is not d3d11, crop meta is almost pointless since d3d11videosink will upload video frame to GPU memory in any case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/933>
2021-09-18 14:37:20 +00:00
crop_meta = gst_buffer_get_video_crop_meta (buffer);
/* Do minimal validate */
if (crop_meta) {
ID3D11Texture2D *texture = (ID3D11Texture2D *) infos[0].data;
D3D11_TEXTURE2D_DESC desc = { 0, };
texture->GetDesc (&desc);
if (desc.Width < crop_meta->x + crop_meta->width ||
desc.Height < crop_meta->y + crop_meta->height) {
GST_WARNING_OBJECT (self, "Invalid crop meta, ignore");
crop_meta = nullptr;
}
}
if (crop_meta) {
input_rect.left = crop_meta->x;
input_rect.right = crop_meta->x + crop_meta->width;
input_rect.top = crop_meta->y;
input_rect.bottom = crop_meta->y + crop_meta->height;
}
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
if (self->first_present) {
d3d11: Clarify target rect to be updated Rename internal methods to clarify which rect (i.e., input or output) should be updated Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1323>
2020-06-02 15:46:13 +00:00
D3D11_VIEWPORT viewport;
viewport.TopLeftX = self->render_rect.left;
viewport.TopLeftY = self->render_rect.top;
viewport.Width = self->render_rect.right - self->render_rect.left;
viewport.Height = self->render_rect.bottom - self->render_rect.top;
viewport.MinDepth = 0.0f;
viewport.MaxDepth = 1.0f;
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
gst_d3d11_converter_update_viewport (self->converter, &viewport);
d3d11: Clarify target rect to be updated Rename internal methods to clarify which rect (i.e., input or output) should be updated Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1323>
2020-06-02 15:46:13 +00:00
gst_d3d11_overlay_compositor_update_viewport (self->compositor,
&viewport);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
}
d3d11videosink: Avoid switching conversion tool during playback Decoder might be able to copy decoded texture to the other buffer pool during playback depending on context. In that case, copied one has no D3D11_BIND_DECODER bind flag. If we used ID3D11VideoProcessor previously for decoder texture, and incoming texture supports ID3D11VideoProcessor as well even if it has no D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example), allow zero-copying instead of using our fallback texture. Frequent conversion tool change (between ID3D11VideoProcessor and generic shader) might result in inconsistent image quality. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 10:48:56 +00:00
/* Converter preference order
* 1) If this texture can be converted via processor, and we used processor
* previously, use processor
* 2) If SRV is available, use converter
* 3) otherwise, use processor
*/
if (can_process && self->processor_in_use) {
d3d11videosink: Add support for crop meta ... when upstream element is d3d11. Note that, if upstream element is not d3d11, crop meta is almost pointless since d3d11videosink will upload video frame to GPU memory in any case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/933>
2021-09-18 14:37:20 +00:00
convert_ret = gst_d3d11_window_do_processor (self, piv, pov, &input_rect);
d3d11videosink: Avoid switching conversion tool during playback Decoder might be able to copy decoded texture to the other buffer pool during playback depending on context. In that case, copied one has no D3D11_BIND_DECODER bind flag. If we used ID3D11VideoProcessor previously for decoder texture, and incoming texture supports ID3D11VideoProcessor as well even if it has no D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example), allow zero-copying instead of using our fallback texture. Frequent conversion tool change (between ID3D11VideoProcessor and generic shader) might result in inconsistent image quality. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 10:48:56 +00:00
} else if (can_convert) {
d3d11videosink: Add support for crop meta ... when upstream element is d3d11. Note that, if upstream element is not d3d11, crop meta is almost pointless since d3d11videosink will upload video frame to GPU memory in any case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/933>
2021-09-18 14:37:20 +00:00
convert_ret = gst_d3d11_window_do_convert (self, srv, rtv, &input_rect);
d3d11videosink: Avoid switching conversion tool during playback Decoder might be able to copy decoded texture to the other buffer pool during playback depending on context. In that case, copied one has no D3D11_BIND_DECODER bind flag. If we used ID3D11VideoProcessor previously for decoder texture, and incoming texture supports ID3D11VideoProcessor as well even if it has no D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example), allow zero-copying instead of using our fallback texture. Frequent conversion tool change (between ID3D11VideoProcessor and generic shader) might result in inconsistent image quality. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 10:48:56 +00:00
} else if (can_process) {
d3d11videosink: Add support for crop meta ... when upstream element is d3d11. Note that, if upstream element is not d3d11, crop meta is almost pointless since d3d11videosink will upload video frame to GPU memory in any case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/933>
2021-09-18 14:37:20 +00:00
convert_ret = gst_d3d11_window_do_processor (self, piv, pov, &input_rect);
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
} else {
d3d11videosink: Avoid switching conversion tool during playback Decoder might be able to copy decoded texture to the other buffer pool during playback depending on context. In that case, copied one has no D3D11_BIND_DECODER bind flag. If we used ID3D11VideoProcessor previously for decoder texture, and incoming texture supports ID3D11VideoProcessor as well even if it has no D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example), allow zero-copying instead of using our fallback texture. Frequent conversion tool change (between ID3D11VideoProcessor and generic shader) might result in inconsistent image quality. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 10:48:56 +00:00
g_assert_not_reached ();
ret = GST_FLOW_ERROR;
goto unmap_and_out;
}
if (!convert_ret) {
ret = GST_FLOW_ERROR;
goto unmap_and_out;
d3d11videosink: Use ID3D11VideoProcessor interface ...for color space conversion if available ID3D11VideoProcessor is equivalent to DXVA-HD video processor which might use specialized blocks for video processing instead of general GPU resource. In addition to that feature, we need to use this API for color space conversion of DXVA2 decoder output memory, because any d3d11 texture arrays that were created with D3D11_BIND_DECODER cannot be used for shader resource. This is prework for d3d11decoder zero-copy rendering and also for conditional HDR tone-map support. Note that some Intel platform is known to support tone-mapping at the driver level using this API on Windows 10.
2020-01-29 12:10:00 +00:00
}
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
gst_d3d11_overlay_compositor_upload (self->compositor, buffer);
gst_d3d11_overlay_compositor_draw_unlocked (self->compositor, &rtv);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11: Fix wrong preprocessing blocks Missed in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2051>
2021-03-02 11:37:04 +00:00
#if (GST_D3D11_DXGI_HEADER_VERSION >= 5)
d3d11window: Set DXGI_PRESENT_ALLOW_TEARING only in fullscreen mode The DXGI_PRESENT_ALLOW_TEARING flag might cause unexpected tearing side effect. Setting it in fullscreen mode only seems to be the correct usage as in the Microsoft's direct3d examples.
2020-02-26 04:11:30 +00:00
if (self->allow_tearing && self->fullscreen) {
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
present_flags |= DXGI_PRESENT_ALLOW_TEARING;
}
#endif
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
if (klass->present)
ret = klass->present (self, present_flags);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
self->first_present = FALSE;
d3d11: Privatize d3d11memory implementation Hide most of symbols of GstD3D11Memory object. GstD3D11Memory is one of primary resource for imcoming d3d11 library and it's expected to be a extensible feature. Hiding implementation detail would be helpful for later use case. Summary of this commit: * Now all native Direct3D11 resources are private of GstD3D11Memory. To access native resources, getter methods need to be used or generic map (e.g., gst_memory_map) API should be called apart from some exceptional case such as d3d11decoder case. * Various helper methods are added for GstBuffer related operations and in order to remove duplicated code. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1892>
2020-12-19 17:39:40 +00:00
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
unmap_and_out:
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
gst_d3d11_buffer_unmap (buffer, infos);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
}
return ret;
}
GstFlowReturn
d3d11videosink: Fix warning around GstVideoOverlay::expose() When expose() is called, d3d11videosink needs to redraw using cached buffer, so gst_d3d11_window_render() should allow null buffer. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2450>
2021-08-02 07:22:06 +00:00
gst_d3d11_window_render (GstD3D11Window * window, GstBuffer * buffer)
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
{
GstMemory *mem;
GstFlowReturn ret;
g_return_val_if_fail (GST_IS_D3D11_WINDOW (window), GST_FLOW_ERROR);
d3d11videosink: Fix warning around GstVideoOverlay::expose() When expose() is called, d3d11videosink needs to redraw using cached buffer, so gst_d3d11_window_render() should allow null buffer. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2450>
2021-08-02 07:22:06 +00:00
if (buffer) {
mem = gst_buffer_peek_memory (buffer, 0);
if (!gst_is_d3d11_memory (mem)) {
GST_ERROR_OBJECT (window, "Invalid buffer");
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11videosink: Fix warning around GstVideoOverlay::expose() When expose() is called, d3d11videosink needs to redraw using cached buffer, so gst_d3d11_window_render() should allow null buffer. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2450>
2021-08-02 07:22:06 +00:00
return GST_FLOW_ERROR;
}
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
}
Revert "d3d11: Enable native multi-thread protection layer and make use of it" This reverts commit 872b7f503c49442e559f6a381416c6a84b76a3c6. Native multi-thread protection layer seems to be consuming more CPU resource than application side protection approach in some cases Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 07:15:35 +00:00
gst_d3d11_device_lock (window->device);
d3d11videosink: Fix warning around GstVideoOverlay::expose() When expose() is called, d3d11videosink needs to redraw using cached buffer, so gst_d3d11_window_render() should allow null buffer. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2450>
2021-08-02 07:22:06 +00:00
if (buffer)
gst_buffer_replace (&window->cached_buffer, buffer);
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
d3d11videosink: Remove DirectWrite related dead code It's now not enabled since we moved core part to gst-libs Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2108>
2021-03-25 12:17:07 +00:00
ret = gst_d3d111_window_present (window, window->cached_buffer,
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
window->pov, window->rtv);
Revert "d3d11: Enable native multi-thread protection layer and make use of it" This reverts commit 872b7f503c49442e559f6a381416c6a84b76a3c6. Native multi-thread protection layer seems to be consuming more CPU resource than application side protection approach in some cases Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 07:15:35 +00:00
gst_d3d11_device_unlock (window->device);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
return ret;
}
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
GstFlowReturn
gst_d3d11_window_render_on_shared_handle (GstD3D11Window * window,
GstBuffer * buffer, HANDLE shared_handle, guint texture_misc_flags,
guint64 acquire_key, guint64 release_key)
{
GstD3D11WindowClass *klass;
GstMemory *mem;
GstFlowReturn ret = GST_FLOW_OK;
GstD3D11WindowSharedHandleData data = { NULL, };
ID3D11VideoProcessorOutputView *pov = NULL;
ID3D11RenderTargetView *rtv = NULL;
g_return_val_if_fail (GST_IS_D3D11_WINDOW (window), GST_FLOW_ERROR);
klass = GST_D3D11_WINDOW_GET_CLASS (window);
g_assert (klass->open_shared_handle != NULL);
g_assert (klass->release_shared_handle != NULL);
mem = gst_buffer_peek_memory (buffer, 0);
if (!gst_is_d3d11_memory (mem)) {
GST_ERROR_OBJECT (window, "Invalid buffer");
return GST_FLOW_ERROR;
}
data.shared_handle = shared_handle;
data.texture_misc_flags = texture_misc_flags;
data.acquire_key = acquire_key;
data.release_key = release_key;
Revert "d3d11: Enable native multi-thread protection layer and make use of it" This reverts commit 872b7f503c49442e559f6a381416c6a84b76a3c6. Native multi-thread protection layer seems to be consuming more CPU resource than application side protection approach in some cases Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 07:15:35 +00:00
gst_d3d11_device_lock (window->device);
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
if (!klass->open_shared_handle (window, &data)) {
GST_ERROR_OBJECT (window, "Couldn't open shared handle");
Revert "d3d11: Enable native multi-thread protection layer and make use of it" This reverts commit 872b7f503c49442e559f6a381416c6a84b76a3c6. Native multi-thread protection layer seems to be consuming more CPU resource than application side protection approach in some cases Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 07:15:35 +00:00
gst_d3d11_device_unlock (window->device);
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
return GST_FLOW_OK;
}
if (data.fallback_rtv) {
rtv = data.fallback_rtv;
pov = data.fallback_pov;
} else {
rtv = data.rtv;
pov = data.pov;
}
d3d11videosink: Remove DirectWrite related dead code It's now not enabled since we moved core part to gst-libs Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2108>
2021-03-25 12:17:07 +00:00
ret = gst_d3d111_window_present (window, buffer, pov, rtv);
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
klass->release_shared_handle (window, &data);
Revert "d3d11: Enable native multi-thread protection layer and make use of it" This reverts commit 872b7f503c49442e559f6a381416c6a84b76a3c6. Native multi-thread protection layer seems to be consuming more CPU resource than application side protection approach in some cases Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 07:15:35 +00:00
gst_d3d11_device_unlock (window->device);
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
return ret;
}
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
gboolean
gst_d3d11_window_unlock (GstD3D11Window * window)
{
GstD3D11WindowClass *klass;
gboolean ret = TRUE;
g_return_val_if_fail (GST_IS_D3D11_WINDOW (window), FALSE);
klass = GST_D3D11_WINDOW_GET_CLASS (window);
if (klass->unlock)
ret = klass->unlock (window);
return ret;
}
gboolean
gst_d3d11_window_unlock_stop (GstD3D11Window * window)
{
GstD3D11WindowClass *klass;
gboolean ret = TRUE;
g_return_val_if_fail (GST_IS_D3D11_WINDOW (window), FALSE);
klass = GST_D3D11_WINDOW_GET_CLASS (window);
if (klass->unlock_stop)
ret = klass->unlock_stop (window);
Revert "d3d11: Enable native multi-thread protection layer and make use of it" This reverts commit 872b7f503c49442e559f6a381416c6a84b76a3c6. Native multi-thread protection layer seems to be consuming more CPU resource than application side protection approach in some cases Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 07:15:35 +00:00
gst_d3d11_device_lock (window->device);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
gst_clear_buffer (&window->cached_buffer);
Revert "d3d11: Enable native multi-thread protection layer and make use of it" This reverts commit 872b7f503c49442e559f6a381416c6a84b76a3c6. Native multi-thread protection layer seems to be consuming more CPU resource than application side protection approach in some cases Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 07:15:35 +00:00
gst_d3d11_device_unlock (window->device);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
return ret;
}
d3d11window: Add unprepare method to clear internal resource GObject::dispose method can be called multiple times. As win32 d3d11window has an internal thread and because GObject::dispose method could be called from the thread, it might cause problems such as trying to join self-thread Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1299>
2020-05-26 16:52:59 +00:00
void
gst_d3d11_window_unprepare (GstD3D11Window * window)
{
GstD3D11WindowClass *klass;
g_return_if_fail (GST_IS_D3D11_WINDOW (window));
klass = GST_D3D11_WINDOW_GET_CLASS (window);
if (klass->unprepare)
klass->unprepare (window);
}
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
GstD3D11WindowNativeType
gst_d3d11_window_get_native_type_from_handle (guintptr handle)
{
if (!handle)
return GST_D3D11_WINDOW_NATIVE_TYPE_NONE;
#if (!GST_D3D11_WINAPI_ONLY_APP)
if (IsWindow ((HWND) handle))
return GST_D3D11_WINDOW_NATIVE_TYPE_HWND;
d3d11: Allow building UWP features with Desktop features if possible WINAPI_PARTITION_DESKTOP and WINAPI_PARTITION_APP can coexist. Although UWP only binaries should be used for production stage, this change will be useful for development stage Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1962>
2021-01-18 10:17:14 +00:00
#endif
#if GST_D3D11_WINAPI_ONLY_APP
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
{
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
/* *INDENT-OFF* */
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
ComPtr<IInspectable> window = reinterpret_cast<IInspectable*> (handle);
ComPtr<ABI::Windows::UI::Core::ICoreWindow> core_window;
ComPtr<ABI::Windows::UI::Xaml::Controls::ISwapChainPanel> panel;
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
/* *INDENT-ON* */
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
if (SUCCEEDED (window.As (&core_window)))
return GST_D3D11_WINDOW_NATIVE_TYPE_CORE_WINDOW;
if (SUCCEEDED (window.As (&panel)))
return GST_D3D11_WINDOW_NATIVE_TYPE_SWAP_CHAIN_PANEL;
}
#endif
return GST_D3D11_WINDOW_NATIVE_TYPE_NONE;
}
const gchar *
gst_d3d11_window_get_native_type_to_string (GstD3D11WindowNativeType type)
{
switch (type) {
case GST_D3D11_WINDOW_NATIVE_TYPE_NONE:
return "none";
case GST_D3D11_WINDOW_NATIVE_TYPE_HWND:
return "hwnd";
case GST_D3D11_WINDOW_NATIVE_TYPE_CORE_WINDOW:
return "core-window";
case GST_D3D11_WINDOW_NATIVE_TYPE_SWAP_CHAIN_PANEL:
return "swap-chain-panel";
default:
break;
}
return "none";
d3d11: Run gst-indent for all C++ code Since all d3d11 plugin implementation code are C++, we need to run gst-indent manually. This is preparation for later "gst-indent sys/d3d11/*.cpp" run. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 04:01:37 +00:00
}
Reference in a new issue Copy permalink