mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2024-12-02 14:36:41 +00:00
Code Issues 1 Projects Releases Wiki Activity
gstreamer/subprojects/gst-plugins-bad/sys/d3d11/gstd3d11videosink.cpp

1570 lines
48 KiB
C++
Raw Normal View History

d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
/* GStreamer
* Copyright (C) 2019 Seungha Yang <seungha.yang@navercorp.com>
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
* Copyright (C) 2020 Seungha Yang <seungha@centricular.com>
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Library General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Library General Public License for more details.
*
* You should have received a copy of the GNU Library General Public
* License along with this library; if not, write to the
* Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
* Boston, MA 02110-1301, USA.
*/
d3d11: Documentation update * Update class metadata * for wrapper bin elements to be distinguishable from internal element. * D3D11 -> Direct3D11 for consistency * Add missing Since mark everywhere * Update plugin cache Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2029>
2021-02-21 08:38:38 +00:00
/**
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
* SECTION:element-d3d11videosink
* @title: d3d11videosink
d3d11: Documentation update * Update class metadata * for wrapper bin elements to be distinguishable from internal element. * D3D11 -> Direct3D11 for consistency * Add missing Since mark everywhere * Update plugin cache Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2029>
2021-02-21 08:38:38 +00:00
*
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
* Direct3D11 based video render element
d3d11: Documentation update * Update class metadata * for wrapper bin elements to be distinguishable from internal element. * D3D11 -> Direct3D11 for consistency * Add missing Since mark everywhere * Update plugin cache Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2029>
2021-02-21 08:38:38 +00:00
*
* ## Example launch line
* ```
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
* gst-launch-1.0 videotestsrc ! d3d11upload ! d3d11videosink
d3d11: Documentation update * Update class metadata * for wrapper bin elements to be distinguishable from internal element. * D3D11 -> Direct3D11 for consistency * Add missing Since mark everywhere * Update plugin cache Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2029>
2021-02-21 08:38:38 +00:00
* ```
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
* This pipeline will display test video stream on screen via d3d11videosink
d3d11: Documentation update * Update class metadata * for wrapper bin elements to be distinguishable from internal element. * D3D11 -> Direct3D11 for consistency * Add missing Since mark everywhere * Update plugin cache Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2029>
2021-02-21 08:38:38 +00:00
*
* Since: 1.18
*
*/
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include "gstd3d11videosink.h"
d3d11: Move core methods to gst-libs Move d3d11 device, memory, buffer pool and minimal method to gst-libs so that other plugins can access d3d11 resource. Since Direct3D is primary graphics API on Windows, we need this infrastructure for various plugins can share GPU resource without downloading GPU memory. Note that this implementation is public only for -bad scope for now. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2020-12-20 17:47:45 +00:00
#include "gstd3d11pluginutils.h"
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
#include <string>
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11: Allow building UWP features with Desktop features if possible WINAPI_PARTITION_DESKTOP and WINAPI_PARTITION_APP can coexist. Although UWP only binaries should be used for production stage, this change will be useful for development stage Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1962>
2021-01-18 10:17:14 +00:00
#if GST_D3D11_WINAPI_APP
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
#include "gstd3d11window_corewindow.h"
#include "gstd3d11window_swapchainpanel.h"
d3d11: Allow building UWP features with Desktop features if possible WINAPI_PARTITION_DESKTOP and WINAPI_PARTITION_APP can coexist. Although UWP only binaries should be used for production stage, this change will be useful for development stage Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1962>
2021-01-18 10:17:14 +00:00
#endif
#if (!GST_D3D11_WINAPI_ONLY_APP)
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
#include "gstd3d11window_win32.h"
#endif
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
#include "gstd3d11window_dummy.h"
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
enum
{
PROP_0,
PROP_ADAPTER,
PROP_FORCE_ASPECT_RATIO,
d3d11videosink: Add support for full screen mode borderless top-most style full screen mode support. Basically fullscreen toggle mode is disabled by default. To enable it use "fullscreen-toggle-mode" property to allow fullscreen mode change by user input and/or property.
2019-12-15 10:29:10 +00:00
PROP_ENABLE_NAVIGATION_EVENTS,
PROP_FULLSCREEN_TOGGLE_MODE,
PROP_FULLSCREEN,
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
PROP_DRAW_ON_SHARED_TEXTURE,
d3d11videosink: Add support for rotation Adding "rotate-method" property Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1396 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:22:10 +00:00
PROP_ROTATE_METHOD,
d3d11videosink: Add gamma-mode and primaries-mode properties Allows controlling gamma remap and/or chromatic adaptation behavior. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2831>
2022-08-02 16:47:46 +00:00
PROP_GAMMA_MODE,
PROP_PRIMARIES_MODE,
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
PROP_DISPLAY_FORMAT,
d3d11videosink: Add "emit-present" property Controls the decision for "present" signal use, and allows delayed "present" signal handler install via the property Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3176>
2022-10-13 15:17:52 +00:00
PROP_EMIT_PRESENT,
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
};
#define DEFAULT_ADAPTER -1
#define DEFAULT_FORCE_ASPECT_RATIO TRUE
#define DEFAULT_ENABLE_NAVIGATION_EVENTS TRUE
d3d11videosink: Add support for full screen mode borderless top-most style full screen mode support. Basically fullscreen toggle mode is disabled by default. To enable it use "fullscreen-toggle-mode" property to allow fullscreen mode change by user input and/or property.
2019-12-15 10:29:10 +00:00
#define DEFAULT_FULLSCREEN_TOGGLE_MODE GST_D3D11_WINDOW_FULLSCREEN_TOGGLE_MODE_NONE
#define DEFAULT_FULLSCREEN FALSE
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
#define DEFAULT_DRAW_ON_SHARED_TEXTURE FALSE
d3d11videosink: Add gamma-mode and primaries-mode properties Allows controlling gamma remap and/or chromatic adaptation behavior. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2831>
2022-08-02 16:47:46 +00:00
#define DEFAULT_GAMMA_MODE GST_VIDEO_GAMMA_MODE_NONE
#define DEFAULT_PRIMARIES_MODE GST_VIDEO_PRIMARIES_MODE_NONE
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
#define DEFAULT_DISPLAY_FORMAT DXGI_FORMAT_UNKNOWN
d3d11videosink: Add "emit-present" property Controls the decision for "present" signal use, and allows delayed "present" signal handler install via the property Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3176>
2022-10-13 15:17:52 +00:00
#define DEFAULT_EMIT_PRESENT FALSE
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
d3d11: Update for documentation * Use GST_PARAM_DOC_SHOW_DEFAULT flags for GPU ID related properties * Add doc caps * Increase deinterlace max resolution Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3250>
2022-10-21 17:49:00 +00:00
/**
* GstD3D11VideoSinkDisplayFormat:
*
* Swapchain's DXGI format
*
* Since: 1.22
*/
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
#define GST_TYPE_D3D11_VIDEO_SINK_DISPLAY_FORMAT (gst_d3d11_video_sink_display_format_type())
static GType
gst_d3d11_video_sink_display_format_type (void)
{
static GType format_type = 0;
GST_D3D11_CALL_ONCE_BEGIN {
static const GEnumValue format_types[] = {
d3d11: Update for documentation * Use GST_PARAM_DOC_SHOW_DEFAULT flags for GPU ID related properties * Add doc caps * Increase deinterlace max resolution Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3250>
2022-10-21 17:49:00 +00:00
/**
* GstD3D11VideoSinkDisplayFormat::unknown:
*
* Since: 1.22
*/
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
{DXGI_FORMAT_UNKNOWN, "DXGI_FORMAT_UNKNOWN", "unknown"},
d3d11: Update for documentation * Use GST_PARAM_DOC_SHOW_DEFAULT flags for GPU ID related properties * Add doc caps * Increase deinterlace max resolution Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3250>
2022-10-21 17:49:00 +00:00
/**
* GstD3D11VideoSinkDisplayFormat::r10g10b10a2-unorm:
*
* Since: 1.22
*/
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
{DXGI_FORMAT_R10G10B10A2_UNORM,
"DXGI_FORMAT_R10G10B10A2_UNORM", "r10g10b10a2-unorm"},
d3d11: Update for documentation * Use GST_PARAM_DOC_SHOW_DEFAULT flags for GPU ID related properties * Add doc caps * Increase deinterlace max resolution Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3250>
2022-10-21 17:49:00 +00:00
/**
* GstD3D11VideoSinkDisplayFormat::r8g8b8a8-unorm:
*
* Since: 1.22
*/
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
{DXGI_FORMAT_R8G8B8A8_UNORM,
"DXGI_FORMAT_R8G8B8A8_UNORM", "r8g8b8a8-unorm"},
d3d11: Update for documentation * Use GST_PARAM_DOC_SHOW_DEFAULT flags for GPU ID related properties * Add doc caps * Increase deinterlace max resolution Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3250>
2022-10-21 17:49:00 +00:00
/**
* GstD3D11VideoSinkDisplayFormat::b8g8r8a8-unorm:
*
* Since: 1.22
*/
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
{DXGI_FORMAT_B8G8R8A8_UNORM,
"DXGI_FORMAT_B8G8R8A8_UNORM", "b8g8r8a8-unorm"},
{0, nullptr, nullptr},
};
format_type = g_enum_register_static ("GstD3D11VideoSinkDisplayFormat",
format_types);
} GST_D3D11_CALL_ONCE_END;
return format_type;
}
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
enum
{
/* signals */
SIGNAL_BEGIN_DRAW,
d3d11videosink: Add present signal The "present" signal will be emitted just before the IDXGISwapChain::Present() call. The client can perform additional GPU operation with given GstD3D11Device object and ID3D11RenderTargetView handle. Or, the client can read back the scene to be displayed on window using the signal. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-19 11:25:31 +00:00
SIGNAL_PRESENT,
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
/* actions */
SIGNAL_DRAW,
LAST_SIGNAL
};
static guint gst_d3d11_video_sink_signals[LAST_SIGNAL] = { 0, };
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11: Don't use hardcoded maximum resolution value Maximum supported texture dimension is pre-defined based on feature level and it couldn't be INT_MAX in any case. See also https://docs.microsoft.com/en-us/windows/win32/direct3d11/overviews-direct3d-11-devices-downlevel-intro Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1964>
2021-01-19 17:29:43 +00:00
static GstStaticCaps pad_template_caps =
d3d11videosink: Specify d3d11 memory caps feature on sinkpad template
2019-08-24 08:43:43 +00:00
GST_STATIC_CAPS (GST_VIDEO_CAPS_MAKE_WITH_FEATURES
d3d11: Don't use hardcoded maximum resolution value Maximum supported texture dimension is pre-defined based on feature level and it couldn't be INT_MAX in any case. See also https://docs.microsoft.com/en-us/windows/win32/direct3d11/overviews-direct3d-11-devices-downlevel-intro Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1964>
2021-01-19 17:29:43 +00:00
(GST_CAPS_FEATURE_MEMORY_D3D11_MEMORY, GST_D3D11_SINK_FORMATS) "; "
GST_VIDEO_CAPS_MAKE_WITH_FEATURES
(GST_CAPS_FEATURE_MEMORY_D3D11_MEMORY ","
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
GST_CAPS_FEATURE_META_GST_VIDEO_OVERLAY_COMPOSITION,
GST_D3D11_SINK_FORMATS) ";"
GST_VIDEO_CAPS_MAKE (GST_D3D11_SINK_FORMATS) "; "
GST_VIDEO_CAPS_MAKE_WITH_FEATURES
(GST_CAPS_FEATURE_MEMORY_SYSTEM_MEMORY ","
d3d11: Don't use hardcoded maximum resolution value Maximum supported texture dimension is pre-defined based on feature level and it couldn't be INT_MAX in any case. See also https://docs.microsoft.com/en-us/windows/win32/direct3d11/overviews-direct3d-11-devices-downlevel-intro Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1964>
2021-01-19 17:29:43 +00:00
GST_CAPS_FEATURE_META_GST_VIDEO_OVERLAY_COMPOSITION,
GST_D3D11_SINK_FORMATS));
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
GST_DEBUG_CATEGORY (d3d11_video_sink_debug);
#define GST_CAT_DEFAULT d3d11_video_sink_debug
d3d11: Move to G_DECLARE_FINAL_TYPE ... and remove unnecessary forward declaration. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1627>
2020-09-25 18:27:39 +00:00
struct _GstD3D11VideoSink
{
GstVideoSink parent;
GstD3D11Device *device;
GstD3D11Window *window;
gint video_width;
gint video_height;
GstVideoInfo info;
guintptr window_id;
d3d11videosink: Delay window setup as much as possible ... so that videosink can handle device update with d3d11 device of the first buffer Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-23 10:29:55 +00:00
gboolean caps_updated;
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
GstBuffer *prepared_buffer;
GstBufferPool *pool;
d3d11videosink: Delay window setup as much as possible ... so that videosink can handle device update with d3d11 device of the first buffer Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-23 10:29:55 +00:00
d3d11: Move to G_DECLARE_FINAL_TYPE ... and remove unnecessary forward declaration. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1627>
2020-09-25 18:27:39 +00:00
/* properties */
gint adapter;
gboolean force_aspect_ratio;
gboolean enable_navigation_events;
GstD3D11WindowFullscreenToggleMode fullscreen_toggle_mode;
gboolean fullscreen;
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
gboolean draw_on_shared_texture;
d3d11videosink: Add gamma-mode and primaries-mode properties Allows controlling gamma remap and/or chromatic adaptation behavior. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2831>
2022-08-02 16:47:46 +00:00
GstVideoGammaMode gamma_mode;
GstVideoPrimariesMode primaries_mode;
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
DXGI_FORMAT display_format;
d3d11videosink: Add "emit-present" property Controls the decision for "present" signal use, and allows delayed "present" signal handler install via the property Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3176>
2022-10-13 15:17:52 +00:00
gboolean emit_present;
d3d11: Move to G_DECLARE_FINAL_TYPE ... and remove unnecessary forward declaration. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1627>
2020-09-25 18:27:39 +00:00
/* saved render rectangle until we have a window */
GstVideoRectangle render_rect;
gboolean pending_render_rect;
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
/* For drawing on user texture */
gboolean drawing;
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
CRITICAL_SECTION lock;
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
gchar *title;
d3d11videosink: Add support for rotation Adding "rotate-method" property Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1396 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:22:10 +00:00
/* method configured via property */
GstVideoOrientationMethod method;
/* method parsed from tag */
GstVideoOrientationMethod tag_method;
/* method currently selected based on "method" and "tag_method" */
GstVideoOrientationMethod selected_method;
d3d11: Move to G_DECLARE_FINAL_TYPE ... and remove unnecessary forward declaration. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1627>
2020-09-25 18:27:39 +00:00
};
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
static void gst_d3d11_videosink_set_property (GObject * object, guint prop_id,
const GValue * value, GParamSpec * pspec);
static void gst_d3d11_videosink_get_property (GObject * object, guint prop_id,
GValue * value, GParamSpec * pspec);
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
static void gst_d3d11_video_sink_finalize (GObject * object);
static gboolean
gst_d3d11_video_sink_draw_action (GstD3D11VideoSink * self,
gpointer shared_handle, guint texture_misc_flags, guint64 acquire_key,
guint64 release_key);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
static void
gst_d3d11_video_sink_video_overlay_init (GstVideoOverlayInterface * iface);
static void
gst_d3d11_video_sink_navigation_init (GstNavigationInterface * iface);
static void gst_d3d11_video_sink_set_context (GstElement * element,
GstContext * context);
static GstCaps *gst_d3d11_video_sink_get_caps (GstBaseSink * sink,
GstCaps * filter);
static gboolean gst_d3d11_video_sink_set_caps (GstBaseSink * sink,
GstCaps * caps);
static gboolean gst_d3d11_video_sink_start (GstBaseSink * sink);
static gboolean gst_d3d11_video_sink_stop (GstBaseSink * sink);
static gboolean gst_d3d11_video_sink_propose_allocation (GstBaseSink * sink,
GstQuery * query);
d3d11videosink: Handle context query Propagate d3d11device via context query.
2019-07-18 14:12:13 +00:00
static gboolean gst_d3d11_video_sink_query (GstBaseSink * sink,
GstQuery * query);
d3d11videosink: Draw window with cached texture on resize This would render nicer than presenting scene with broken aspect ratio, especially in case of low framerate.
2019-12-02 14:27:42 +00:00
static gboolean gst_d3d11_video_sink_unlock (GstBaseSink * sink);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
static gboolean gst_d3d11_video_sink_unlock_stop (GstBaseSink * sink);
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
static gboolean gst_d3d11_video_sink_event (GstBaseSink * sink,
GstEvent * event);
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
static GstFlowReturn gst_d3d11_video_sink_prepare (GstBaseSink * sink,
GstBuffer * buffer);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
static GstFlowReturn
gst_d3d11_video_sink_show_frame (GstVideoSink * sink, GstBuffer * buf);
d3d11videosink: Prepare window once streaming started ... instead of READY state. READY state is too early for setting overlay window handle especially playbin/playsink scenario since playsink will set given overlay handle on videosink once READY state change of videosink is ensured. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1893>
2020-12-20 13:12:44 +00:00
static gboolean gst_d3d11_video_sink_prepare_window (GstD3D11VideoSink * self);
d3d11videosink: Add support for rotation Adding "rotate-method" property Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1396 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:22:10 +00:00
static void gst_d3d11_video_sink_set_orientation (GstD3D11VideoSink * self,
GstVideoOrientationMethod method, gboolean from_tag);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
#define gst_d3d11_video_sink_parent_class parent_class
G_DEFINE_TYPE_WITH_CODE (GstD3D11VideoSink, gst_d3d11_video_sink,
GST_TYPE_VIDEO_SINK,
G_IMPLEMENT_INTERFACE (GST_TYPE_VIDEO_OVERLAY,
gst_d3d11_video_sink_video_overlay_init);
G_IMPLEMENT_INTERFACE (GST_TYPE_NAVIGATION,
gst_d3d11_video_sink_navigation_init);
GST_DEBUG_CATEGORY_INIT (d3d11_video_sink_debug,
"d3d11videosink", 0, "Direct3D11 Video Sink"));
static void
gst_d3d11_video_sink_class_init (GstD3D11VideoSinkClass * klass)
{
GObjectClass *gobject_class = G_OBJECT_CLASS (klass);
GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
GstBaseSinkClass *basesink_class = GST_BASE_SINK_CLASS (klass);
GstVideoSinkClass *videosink_class = GST_VIDEO_SINK_CLASS (klass);
d3d11: Don't use hardcoded maximum resolution value Maximum supported texture dimension is pre-defined based on feature level and it couldn't be INT_MAX in any case. See also https://docs.microsoft.com/en-us/windows/win32/direct3d11/overviews-direct3d-11-devices-downlevel-intro Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1964>
2021-01-19 17:29:43 +00:00
GstCaps *caps;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
gobject_class->set_property = gst_d3d11_videosink_set_property;
gobject_class->get_property = gst_d3d11_videosink_get_property;
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
gobject_class->finalize = gst_d3d11_video_sink_finalize;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
g_object_class_install_property (gobject_class, PROP_ADAPTER,
g_param_spec_int ("adapter", "Adapter",
"Adapter index for creating device (-1 for default)",
-1, G_MAXINT32, DEFAULT_ADAPTER,
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
(GParamFlags) (G_PARAM_READWRITE | GST_PARAM_MUTABLE_READY |
G_PARAM_STATIC_STRINGS)));
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
g_object_class_install_property (gobject_class, PROP_FORCE_ASPECT_RATIO,
g_param_spec_boolean ("force-aspect-ratio",
"Force aspect ratio",
"When enabled, scaling will respect original aspect ratio",
DEFAULT_FORCE_ASPECT_RATIO,
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
(GParamFlags) (G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)));
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
g_object_class_install_property (gobject_class, PROP_ENABLE_NAVIGATION_EVENTS,
g_param_spec_boolean ("enable-navigation-events",
"Enable navigation events",
"When enabled, navigation events are sent upstream",
DEFAULT_ENABLE_NAVIGATION_EVENTS,
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
(GParamFlags) (G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)));
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Add support for full screen mode borderless top-most style full screen mode support. Basically fullscreen toggle mode is disabled by default. To enable it use "fullscreen-toggle-mode" property to allow fullscreen mode change by user input and/or property.
2019-12-15 10:29:10 +00:00
g_object_class_install_property (gobject_class, PROP_FULLSCREEN_TOGGLE_MODE,
g_param_spec_flags ("fullscreen-toggle-mode",
"Full screen toggle mode",
"Full screen toggle mode used to trigger fullscreen mode change",
GST_D3D11_WINDOW_TOGGLE_MODE_GET_TYPE, DEFAULT_FULLSCREEN_TOGGLE_MODE,
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
(GParamFlags) (G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)));
d3d11videosink: Add support for full screen mode borderless top-most style full screen mode support. Basically fullscreen toggle mode is disabled by default. To enable it use "fullscreen-toggle-mode" property to allow fullscreen mode change by user input and/or property.
2019-12-15 10:29:10 +00:00
g_object_class_install_property (gobject_class, PROP_FULLSCREEN,
g_param_spec_boolean ("fullscreen",
"fullscreen",
"Ignored when \"fullscreen-toggle-mode\" does not include \"property\"",
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
DEFAULT_FULLSCREEN,
(GParamFlags) (G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)));
d3d11videosink: Add support for full screen mode borderless top-most style full screen mode support. Basically fullscreen toggle mode is disabled by default. To enable it use "fullscreen-toggle-mode" property to allow fullscreen mode change by user input and/or property.
2019-12-15 10:29:10 +00:00
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
/**
* GstD3D11VideoSink:draw-on-shared-texture:
*
d3d11: Documentation update * Update class metadata * for wrapper bin elements to be distinguishable from internal element. * D3D11 -> Direct3D11 for consistency * Add missing Since mark everywhere * Update plugin cache Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2029>
2021-02-21 08:38:38 +00:00
* Instruct the sink to draw on a shared texture provided by user.
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
* User must watch #d3d11videosink::begin-draw signal and should call
* #d3d11videosink::draw method on the #d3d11videosink::begin-draw
d3d11: Documentation update * Update class metadata * for wrapper bin elements to be distinguishable from internal element. * D3D11 -> Direct3D11 for consistency * Add missing Since mark everywhere * Update plugin cache Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2029>
2021-02-21 08:38:38 +00:00
* signal handler.
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
*
* Currently supported formats for user texture are:
* - DXGI_FORMAT_R8G8B8A8_UNORM
* - DXGI_FORMAT_B8G8R8A8_UNORM
* - DXGI_FORMAT_R10G10B10A2_UNORM
*
* Since: 1.20
*/
g_object_class_install_property (gobject_class, PROP_DRAW_ON_SHARED_TEXTURE,
g_param_spec_boolean ("draw-on-shared-texture",
"Draw on shared texture",
"Draw on user provided shared texture instead of window. "
"When enabled, user can pass application's own texture to sink "
"by using \"draw\" action signal on \"begin-draw\" signal handler, "
"so that sink can draw video data on application's texture. "
"Supported texture formats for user texture are "
"DXGI_FORMAT_R8G8B8A8_UNORM, DXGI_FORMAT_B8G8R8A8_UNORM, and "
"DXGI_FORMAT_R10G10B10A2_UNORM.",
DEFAULT_DRAW_ON_SHARED_TEXTURE,
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
(GParamFlags) (G_PARAM_READWRITE | GST_PARAM_MUTABLE_READY |
G_PARAM_STATIC_STRINGS)));
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
d3d11videosink: Add support for rotation Adding "rotate-method" property Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1396 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:22:10 +00:00
/**
* GstD3D11VideoSink:rotate-method:
*
* Video rotation/flip method to use
*
* Since: 1.22
*/
g_object_class_install_property (gobject_class, PROP_ROTATE_METHOD,
g_param_spec_enum ("rotate-method", "Rotate Method",
"Rotate method to use",
GST_TYPE_VIDEO_ORIENTATION_METHOD, GST_VIDEO_ORIENTATION_IDENTITY,
(GParamFlags) (G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)));
d3d11videosink: Add gamma-mode and primaries-mode properties Allows controlling gamma remap and/or chromatic adaptation behavior. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2831>
2022-08-02 16:47:46 +00:00
/**
* GstD3D11VideoSink:gamma-mode:
*
* Gamma conversion mode
*
* Since: 1.22
*/
g_object_class_install_property (gobject_class, PROP_GAMMA_MODE,
g_param_spec_enum ("gamma-mode", "Gamma mode",
"Gamma conversion mode", GST_TYPE_VIDEO_GAMMA_MODE,
DEFAULT_GAMMA_MODE, (GParamFlags) (GST_PARAM_MUTABLE_READY |
G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)));
/**
* GstD3D11VideoSink:primaries-mode:
*
* Primaries conversion mode
*
* Since: 1.22
*/
g_object_class_install_property (gobject_class, PROP_PRIMARIES_MODE,
g_param_spec_enum ("primaries-mode", "Primaries Mode",
"Primaries conversion mode", GST_TYPE_VIDEO_PRIMARIES_MODE,
DEFAULT_PRIMARIES_MODE, (GParamFlags) (GST_PARAM_MUTABLE_READY |
G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS)));
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
/**
* GstD3D11VideoSink:display-format:
*
* Swapchain display format
*
* Since: 1.22
*/
g_object_class_install_property (gobject_class, PROP_DISPLAY_FORMAT,
g_param_spec_enum ("display-format", "Display Format",
"Swapchain display format", GST_TYPE_D3D11_VIDEO_SINK_DISPLAY_FORMAT,
DEFAULT_DISPLAY_FORMAT, (GParamFlags) (G_PARAM_READWRITE |
GST_PARAM_MUTABLE_READY | G_PARAM_STATIC_STRINGS)));
d3d11videosink: Add "emit-present" property Controls the decision for "present" signal use, and allows delayed "present" signal handler install via the property Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3176>
2022-10-13 15:17:52 +00:00
/**
* GstD3D11VideoSink:emit-present:
*
* Emits "present" signal
*
* Since: 1.22
*/
g_object_class_install_property (gobject_class, PROP_EMIT_PRESENT,
g_param_spec_boolean ("emit-present", "Emit present",
"Emits present signal", DEFAULT_EMIT_PRESENT,
(GParamFlags) (G_PARAM_READWRITE | GST_PARAM_MUTABLE_READY |
G_PARAM_STATIC_STRINGS)));
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
/**
d3d11: Documentation update * Update class metadata * for wrapper bin elements to be distinguishable from internal element. * D3D11 -> Direct3D11 for consistency * Add missing Since mark everywhere * Update plugin cache Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2029>
2021-02-21 08:38:38 +00:00
* GstD3D11VideoSink::begin-draw:
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
* @videosink: the #d3d11videosink
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
*
d3d11: Documentation update * Update class metadata * for wrapper bin elements to be distinguishable from internal element. * D3D11 -> Direct3D11 for consistency * Add missing Since mark everywhere * Update plugin cache Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2029>
2021-02-21 08:38:38 +00:00
* Emitted when sink has a texture to draw. Application needs to invoke
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
* #d3d11videosink::draw action signal before returning from
* #d3d11videosink::begin-draw signal handler.
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
*
* Since: 1.20
*/
gst_d3d11_video_sink_signals[SIGNAL_BEGIN_DRAW] =
g_signal_new ("begin-draw", G_TYPE_FROM_CLASS (klass), G_SIGNAL_RUN_LAST,
G_STRUCT_OFFSET (GstD3D11VideoSinkClass, begin_draw),
NULL, NULL, NULL, G_TYPE_NONE, 0, G_TYPE_NONE);
/**
d3d11: Documentation update * Update class metadata * for wrapper bin elements to be distinguishable from internal element. * D3D11 -> Direct3D11 for consistency * Add missing Since mark everywhere * Update plugin cache Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2029>
2021-02-21 08:38:38 +00:00
* GstD3D11VideoSink::draw:
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
* @videosink: the #d3d11videosink
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
* @shard_handle: a pointer to HANDLE
* @texture_misc_flags: a D3D11_RESOURCE_MISC_FLAG value
* @acquire_key: a key value used for IDXGIKeyedMutex::AcquireSync
* @release_key: a key value used for IDXGIKeyedMutex::ReleaseSync
*
d3d11: Documentation update * Update class metadata * for wrapper bin elements to be distinguishable from internal element. * D3D11 -> Direct3D11 for consistency * Add missing Since mark everywhere * Update plugin cache Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2029>
2021-02-21 08:38:38 +00:00
* Draws on a shared texture. @shard_handle must be a valid pointer to
* a HANDLE which was obtained via IDXGIResource::GetSharedHandle or
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
* IDXGIResource1::CreateSharedHandle.
*
d3d11: Documentation update * Update class metadata * for wrapper bin elements to be distinguishable from internal element. * D3D11 -> Direct3D11 for consistency * Add missing Since mark everywhere * Update plugin cache Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2029>
2021-02-21 08:38:38 +00:00
* If the texture was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag,
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
* caller must specify valid @acquire_key and @release_key.
* Otherwise (i.e., created with D3D11_RESOURCE_MISC_SHARED flag),
* @acquire_key and @release_key will be ignored.
*
* Since: 1.20
*/
gst_d3d11_video_sink_signals[SIGNAL_DRAW] =
g_signal_new ("draw", G_TYPE_FROM_CLASS (klass),
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
(GSignalFlags) (G_SIGNAL_RUN_LAST | G_SIGNAL_ACTION),
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
G_STRUCT_OFFSET (GstD3D11VideoSinkClass, draw), NULL, NULL, NULL,
G_TYPE_BOOLEAN, 4, G_TYPE_POINTER, G_TYPE_UINT, G_TYPE_UINT64,
G_TYPE_UINT64);
d3d11videosink: Add present signal The "present" signal will be emitted just before the IDXGISwapChain::Present() call. The client can perform additional GPU operation with given GstD3D11Device object and ID3D11RenderTargetView handle. Or, the client can read back the scene to be displayed on window using the signal. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-19 11:25:31 +00:00
/**
* GstD3D11VideoSink::present
* @videosink: the #GstD3D11VideoSink
d3d11: Update for documentation * Use GST_PARAM_DOC_SHOW_DEFAULT flags for GPU ID related properties * Add doc caps * Increase deinterlace max resolution Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3250>
2022-10-21 17:49:00 +00:00
* @device: a GstD3D11Device object
d3d11videosink: Add present signal The "present" signal will be emitted just before the IDXGISwapChain::Present() call. The client can perform additional GPU operation with given GstD3D11Device object and ID3D11RenderTargetView handle. Or, the client can read back the scene to be displayed on window using the signal. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-19 11:25:31 +00:00
* @render_target: a ID3D11RenderTargetView handle of swapchain's backbuffer
*
* Emitted just before presenting a texture via the IDXGISwapChain::Present.
* The client can perform additional rendering on the given @render_target,
* or can read the content already rendered on the swapchain's backbuffer.
*
d3d11: Update for documentation * Use GST_PARAM_DOC_SHOW_DEFAULT flags for GPU ID related properties * Add doc caps * Increase deinterlace max resolution Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3250>
2022-10-21 17:49:00 +00:00
* This signal will be emitted with gst_d3d11_device_lock taken and
d3d11videosink: Add present signal The "present" signal will be emitted just before the IDXGISwapChain::Present() call. The client can perform additional GPU operation with given GstD3D11Device object and ID3D11RenderTargetView handle. Or, the client can read back the scene to be displayed on window using the signal. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-19 11:25:31 +00:00
* client should perform GPU operation from the thread where this signal
* emitted.
*
* Since: 1.22
*/
gst_d3d11_video_sink_signals[SIGNAL_PRESENT] =
g_signal_new ("present", G_TYPE_FROM_CLASS (klass),
G_SIGNAL_RUN_LAST, 0, nullptr, nullptr, nullptr,
d3d11: Update for documentation * Use GST_PARAM_DOC_SHOW_DEFAULT flags for GPU ID related properties * Add doc caps * Increase deinterlace max resolution Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3250>
2022-10-21 17:49:00 +00:00
G_TYPE_NONE, 2, GST_TYPE_OBJECT, G_TYPE_POINTER);
d3d11videosink: Add present signal The "present" signal will be emitted just before the IDXGISwapChain::Present() call. The client can perform additional GPU operation with given GstD3D11Device object and ID3D11RenderTargetView handle. Or, the client can read back the scene to be displayed on window using the signal. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-19 11:25:31 +00:00
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
element_class->set_context =
GST_DEBUG_FUNCPTR (gst_d3d11_video_sink_set_context);
gst_element_class_set_static_metadata (element_class,
"Direct3D11 video sink", "Sink/Video",
"A Direct3D11 based videosink",
"Seungha Yang <seungha.yang@navercorp.com>");
d3d11: Don't use hardcoded maximum resolution value Maximum supported texture dimension is pre-defined based on feature level and it couldn't be INT_MAX in any case. See also https://docs.microsoft.com/en-us/windows/win32/direct3d11/overviews-direct3d-11-devices-downlevel-intro Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1964>
2021-01-19 17:29:43 +00:00
caps = gst_d3d11_get_updated_template_caps (&pad_template_caps);
gst_element_class_add_pad_template (element_class,
gst_pad_template_new ("sink", GST_PAD_SINK, GST_PAD_ALWAYS, caps));
gst_caps_unref (caps);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
basesink_class->get_caps = GST_DEBUG_FUNCPTR (gst_d3d11_video_sink_get_caps);
basesink_class->set_caps = GST_DEBUG_FUNCPTR (gst_d3d11_video_sink_set_caps);
basesink_class->start = GST_DEBUG_FUNCPTR (gst_d3d11_video_sink_start);
basesink_class->stop = GST_DEBUG_FUNCPTR (gst_d3d11_video_sink_stop);
basesink_class->propose_allocation =
GST_DEBUG_FUNCPTR (gst_d3d11_video_sink_propose_allocation);
d3d11videosink: Handle context query Propagate d3d11device via context query.
2019-07-18 14:12:13 +00:00
basesink_class->query = GST_DEBUG_FUNCPTR (gst_d3d11_video_sink_query);
d3d11videosink: Draw window with cached texture on resize This would render nicer than presenting scene with broken aspect ratio, especially in case of low framerate.
2019-12-02 14:27:42 +00:00
basesink_class->unlock = GST_DEBUG_FUNCPTR (gst_d3d11_video_sink_unlock);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
basesink_class->unlock_stop =
GST_DEBUG_FUNCPTR (gst_d3d11_video_sink_unlock_stop);
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
basesink_class->event = GST_DEBUG_FUNCPTR (gst_d3d11_video_sink_event);
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
basesink_class->prepare = GST_DEBUG_FUNCPTR (gst_d3d11_video_sink_prepare);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
videosink_class->show_frame =
GST_DEBUG_FUNCPTR (gst_d3d11_video_sink_show_frame);
plugins: Update for documentation of Windows plugins * Add Since marks * Make use of GST_PARAM_CONDITIONALLY_AVAILABLE flag
2020-06-08 15:20:08 +00:00
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
klass->draw = gst_d3d11_video_sink_draw_action;
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
gst_type_mark_as_plugin_api (GST_D3D11_WINDOW_TOGGLE_MODE_GET_TYPE,
(GstPluginAPIFlags) 0);
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
gst_type_mark_as_plugin_api (GST_TYPE_D3D11_VIDEO_SINK_DISPLAY_FORMAT,
(GstPluginAPIFlags) 0);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
static void
gst_d3d11_video_sink_init (GstD3D11VideoSink * self)
{
self->adapter = DEFAULT_ADAPTER;
self->force_aspect_ratio = DEFAULT_FORCE_ASPECT_RATIO;
self->enable_navigation_events = DEFAULT_ENABLE_NAVIGATION_EVENTS;
d3d11videosink: Add support for full screen mode borderless top-most style full screen mode support. Basically fullscreen toggle mode is disabled by default. To enable it use "fullscreen-toggle-mode" property to allow fullscreen mode change by user input and/or property.
2019-12-15 10:29:10 +00:00
self->fullscreen_toggle_mode = DEFAULT_FULLSCREEN_TOGGLE_MODE;
self->fullscreen = DEFAULT_FULLSCREEN;
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
self->draw_on_shared_texture = DEFAULT_DRAW_ON_SHARED_TEXTURE;
d3d11videosink: Add gamma-mode and primaries-mode properties Allows controlling gamma remap and/or chromatic adaptation behavior. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2831>
2022-08-02 16:47:46 +00:00
self->gamma_mode = DEFAULT_GAMMA_MODE;
self->primaries_mode = DEFAULT_PRIMARIES_MODE;
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
self->display_format = DEFAULT_DISPLAY_FORMAT;
d3d11videosink: Add "emit-present" property Controls the decision for "present" signal use, and allows delayed "present" signal handler install via the property Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3176>
2022-10-13 15:17:52 +00:00
self->emit_present = DEFAULT_EMIT_PRESENT;
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
InitializeCriticalSection (&self->lock);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
static void
gst_d3d11_videosink_set_property (GObject * object, guint prop_id,
const GValue * value, GParamSpec * pspec)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (object);
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
GstD3D11CSLockGuard lk (&self->lock);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
switch (prop_id) {
case PROP_ADAPTER:
self->adapter = g_value_get_int (value);
break;
case PROP_FORCE_ASPECT_RATIO:
self->force_aspect_ratio = g_value_get_boolean (value);
if (self->window)
g_object_set (self->window,
"force-aspect-ratio", self->force_aspect_ratio, NULL);
break;
case PROP_ENABLE_NAVIGATION_EVENTS:
self->enable_navigation_events = g_value_get_boolean (value);
if (self->window) {
g_object_set (self->window,
"enable-navigation-events", self->enable_navigation_events, NULL);
}
break;
d3d11videosink: Add support for full screen mode borderless top-most style full screen mode support. Basically fullscreen toggle mode is disabled by default. To enable it use "fullscreen-toggle-mode" property to allow fullscreen mode change by user input and/or property.
2019-12-15 10:29:10 +00:00
case PROP_FULLSCREEN_TOGGLE_MODE:
d3d11: Port to C++ Direct3D11 objects are COM, and most COM C APIs are verbose (C++ is a little better). So, by using C++ APIs, we can make code shorter and more readable. Moreover, "ComPtr" helper class (which is C++ only) can be utilized, that is very helpful for avoiding error-prone COM refcounting issue/leak. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-13 08:40:57 +00:00
self->fullscreen_toggle_mode =
(GstD3D11WindowFullscreenToggleMode) g_value_get_flags (value);
d3d11videosink: Add support for full screen mode borderless top-most style full screen mode support. Basically fullscreen toggle mode is disabled by default. To enable it use "fullscreen-toggle-mode" property to allow fullscreen mode change by user input and/or property.
2019-12-15 10:29:10 +00:00
if (self->window) {
g_object_set (self->window,
"fullscreen-toggle-mode", self->fullscreen_toggle_mode, NULL);
}
break;
case PROP_FULLSCREEN:
self->fullscreen = g_value_get_boolean (value);
if (self->window) {
g_object_set (self->window, "fullscreen", self->fullscreen, NULL);
}
break;
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
case PROP_DRAW_ON_SHARED_TEXTURE:
self->draw_on_shared_texture = g_value_get_boolean (value);
break;
d3d11videosink: Add support for rotation Adding "rotate-method" property Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1396 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:22:10 +00:00
case PROP_ROTATE_METHOD:
gst_d3d11_video_sink_set_orientation (self,
(GstVideoOrientationMethod) g_value_get_enum (value), FALSE);
break;
d3d11videosink: Add gamma-mode and primaries-mode properties Allows controlling gamma remap and/or chromatic adaptation behavior. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2831>
2022-08-02 16:47:46 +00:00
case PROP_GAMMA_MODE:
self->gamma_mode = (GstVideoGammaMode) g_value_get_enum (value);
break;
case PROP_PRIMARIES_MODE:
self->primaries_mode = (GstVideoPrimariesMode) g_value_get_enum (value);
break;
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
case PROP_DISPLAY_FORMAT:
self->display_format = (DXGI_FORMAT) g_value_get_enum (value);
break;
d3d11videosink: Add "emit-present" property Controls the decision for "present" signal use, and allows delayed "present" signal handler install via the property Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3176>
2022-10-13 15:17:52 +00:00
case PROP_EMIT_PRESENT:
self->emit_present = g_value_get_boolean (value);
break;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
static void
gst_d3d11_videosink_get_property (GObject * object, guint prop_id,
GValue * value, GParamSpec * pspec)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (object);
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
GstD3D11CSLockGuard lk (&self->lock);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
switch (prop_id) {
case PROP_ADAPTER:
g_value_set_int (value, self->adapter);
break;
case PROP_FORCE_ASPECT_RATIO:
g_value_set_boolean (value, self->force_aspect_ratio);
break;
case PROP_ENABLE_NAVIGATION_EVENTS:
g_value_set_boolean (value, self->enable_navigation_events);
break;
d3d11videosink: Add support for full screen mode borderless top-most style full screen mode support. Basically fullscreen toggle mode is disabled by default. To enable it use "fullscreen-toggle-mode" property to allow fullscreen mode change by user input and/or property.
2019-12-15 10:29:10 +00:00
case PROP_FULLSCREEN_TOGGLE_MODE:
g_value_set_flags (value, self->fullscreen_toggle_mode);
break;
case PROP_FULLSCREEN:
if (self->window) {
g_object_get_property (G_OBJECT (self->window), pspec->name, value);
} else {
g_value_set_boolean (value, self->fullscreen);
}
break;
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
case PROP_DRAW_ON_SHARED_TEXTURE:
g_value_set_boolean (value, self->draw_on_shared_texture);
break;
d3d11videosink: Add support for rotation Adding "rotate-method" property Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1396 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:22:10 +00:00
case PROP_ROTATE_METHOD:
g_value_set_enum (value, self->method);
break;
d3d11videosink: Add gamma-mode and primaries-mode properties Allows controlling gamma remap and/or chromatic adaptation behavior. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2831>
2022-08-02 16:47:46 +00:00
case PROP_GAMMA_MODE:
g_value_set_enum (value, self->gamma_mode);
break;
case PROP_PRIMARIES_MODE:
g_value_set_enum (value, self->primaries_mode);
break;
d3d11videosink: Add display-format property Make swapchian's display format configurable, since some DXGI formats we can use for swapchain are not API interop compatible. For instance, BGRA format should be used for Direct2D interop. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-23 15:13:21 +00:00
case PROP_DISPLAY_FORMAT:
g_value_set_enum (value, self->display_format);
break;
d3d11videosink: Add "emit-present" property Controls the decision for "present" signal use, and allows delayed "present" signal handler install via the property Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3176>
2022-10-13 15:17:52 +00:00
case PROP_EMIT_PRESENT:
g_value_set_boolean (value, self->emit_present);
break;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
default:
G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
break;
}
}
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
static void
gst_d3d11_video_sink_finalize (GObject * object)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (object);
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
DeleteCriticalSection (&self->lock);
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
g_free (self->title);
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
G_OBJECT_CLASS (parent_class)->finalize (object);
}
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
static void
gst_d3d11_video_sink_set_context (GstElement * element, GstContext * context)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (element);
d3d11: Don't share d3d11 device context unconditionally without checking adapter Do not accept any GstD3D11Device context which has different adapter index from the required one. For example, if a d3d11 element is expecting d3d11 device with adapter 1 (i.e., the second GPU), any d3d11 device context having different adapter could not be shared with the d3d11 element.
2019-08-23 12:41:39 +00:00
gst_d3d11_handle_set_context (element, context, self->adapter, &self->device);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
GST_ELEMENT_CLASS (parent_class)->set_context (element, context);
}
d3d11: Move core methods to gst-libs Move d3d11 device, memory, buffer pool and minimal method to gst-libs so that other plugins can access d3d11 resource. Since Direct3D is primary graphics API on Windows, we need this infrastructure for various plugins can share GPU resource without downloading GPU memory. Note that this implementation is public only for -bad scope for now. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2020-12-20 17:47:45 +00:00
static GstCaps *
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
gst_d3d11_video_sink_get_caps (GstBaseSink * sink, GstCaps * filter)
d3d11: Move core methods to gst-libs Move d3d11 device, memory, buffer pool and minimal method to gst-libs so that other plugins can access d3d11 resource. Since Direct3D is primary graphics API on Windows, we need this infrastructure for various plugins can share GPU resource without downloading GPU memory. Note that this implementation is public only for -bad scope for now. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2020-12-20 17:47:45 +00:00
{
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (sink);
GstCaps *caps = NULL;
d3d11: Move core methods to gst-libs Move d3d11 device, memory, buffer pool and minimal method to gst-libs so that other plugins can access d3d11 resource. Since Direct3D is primary graphics API on Windows, we need this infrastructure for various plugins can share GPU resource without downloading GPU memory. Note that this implementation is public only for -bad scope for now. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2020-12-20 17:47:45 +00:00
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
caps = gst_pad_get_pad_template_caps (GST_VIDEO_SINK_PAD (sink));
d3d11: Move core methods to gst-libs Move d3d11 device, memory, buffer pool and minimal method to gst-libs so that other plugins can access d3d11 resource. Since Direct3D is primary graphics API on Windows, we need this infrastructure for various plugins can share GPU resource without downloading GPU memory. Note that this implementation is public only for -bad scope for now. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2020-12-20 17:47:45 +00:00
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
if (self->device) {
gboolean is_hardware = FALSE;
d3d11: Move core methods to gst-libs Move d3d11 device, memory, buffer pool and minimal method to gst-libs so that other plugins can access d3d11 resource. Since Direct3D is primary graphics API on Windows, we need this infrastructure for various plugins can share GPU resource without downloading GPU memory. Note that this implementation is public only for -bad scope for now. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2020-12-20 17:47:45 +00:00
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
g_object_get (self->device, "hardware", &is_hardware, NULL);
d3d11: Move core methods to gst-libs Move d3d11 device, memory, buffer pool and minimal method to gst-libs so that other plugins can access d3d11 resource. Since Direct3D is primary graphics API on Windows, we need this infrastructure for various plugins can share GPU resource without downloading GPU memory. Note that this implementation is public only for -bad scope for now. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2020-12-20 17:47:45 +00:00
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
/* In case of WARP device, conversion via shader would be inefficient than
* upstream videoconvert. Allow native formats in this case */
if (!is_hardware) {
GValue format_list = G_VALUE_INIT;
GValue format = G_VALUE_INIT;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
g_value_init (&format_list, GST_TYPE_LIST);
g_value_init (&format, G_TYPE_STRING);
d3d11videosink: Add support for overlay composition Add d3d11overlaycompositor object to draw overlay image on render target using Blend method.
2019-12-24 06:54:57 +00:00
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
g_value_set_string (&format, "RGBA");
gst_value_list_append_and_take_value (&format_list, &format);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
format = G_VALUE_INIT;
g_value_init (&format, G_TYPE_STRING);
g_value_set_string (&format, "BGRA");
gst_value_list_append_and_take_value (&format_list, &format);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
caps = gst_caps_make_writable (caps);
gst_caps_set_value (caps, "format", &format_list);
g_value_unset (&format_list);
}
}
if (filter) {
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
GstCaps *isect;
isect = gst_caps_intersect_full (filter, caps, GST_CAPS_INTERSECT_FIRST);
gst_caps_unref (caps);
caps = isect;
}
return caps;
}
static gboolean
gst_d3d11_video_sink_set_caps (GstBaseSink * sink, GstCaps * caps)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (sink);
d3d11videosink: Delay window setup as much as possible ... so that videosink can handle device update with d3d11 device of the first buffer Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-23 10:29:55 +00:00
GST_DEBUG_OBJECT (self, "set caps %" GST_PTR_FORMAT, caps);
/* We will update window on show_frame() */
self->caps_updated = TRUE;
return TRUE;
}
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
static GstFlowReturn
d3d11videosink: Delay window setup as much as possible ... so that videosink can handle device update with d3d11 device of the first buffer Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-23 10:29:55 +00:00
gst_d3d11_video_sink_update_window (GstD3D11VideoSink * self, GstCaps * caps)
{
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
gint video_width, video_height;
gint video_par_n, video_par_d; /* video's PAR */
gint display_par_n = 1, display_par_d = 1; /* display's PAR */
guint num, den;
d3d11videosink: Don't try to post error message by d3d11window self The d3d11window isn't GstElement. To post error message, proxy it to d3d11videosink instead.
2019-10-04 12:15:15 +00:00
GError *error = NULL;
d3d11videosink: Add gamma-mode and primaries-mode properties Allows controlling gamma remap and/or chromatic adaptation behavior. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2831>
2022-08-02 16:47:46 +00:00
GstStructure *config;
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
GstD3D11Window *window;
GstFlowReturn ret = GST_FLOW_OK;
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
GstD3D11AllocationParams *params;
guint bind_flags = D3D11_BIND_SHADER_RESOURCE;
GstD3D11Format device_format;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Delay window setup as much as possible ... so that videosink can handle device update with d3d11 device of the first buffer Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-23 10:29:55 +00:00
GST_DEBUG_OBJECT (self, "Updating window with caps %" GST_PTR_FORMAT, caps);
self->caps_updated = FALSE;
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
if (self->pool) {
gst_buffer_pool_set_active (self->pool, FALSE);
gst_clear_object (&self->pool);
}
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
EnterCriticalSection (&self->lock);
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
if (!gst_d3d11_video_sink_prepare_window (self)) {
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
LeaveCriticalSection (&self->lock);
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
GST_ELEMENT_ERROR (self, RESOURCE, NOT_FOUND, (nullptr),
("Failed to open window."));
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
return GST_FLOW_ERROR;
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
}
d3d11videosink: Prepare window once streaming started ... instead of READY state. READY state is too early for setting overlay window handle especially playbin/playsink scenario since playsink will set given overlay handle on videosink once READY state change of videosink is ensured. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1893>
2020-12-20 13:12:44 +00:00
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
if (!gst_video_info_from_caps (&self->info, caps)) {
GST_DEBUG_OBJECT (self,
"Could not locate image format from caps %" GST_PTR_FORMAT, caps);
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
LeaveCriticalSection (&self->lock);
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
return GST_FLOW_ERROR;
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
}
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
video_width = GST_VIDEO_INFO_WIDTH (&self->info);
video_height = GST_VIDEO_INFO_HEIGHT (&self->info);
video_par_n = GST_VIDEO_INFO_PAR_N (&self->info);
video_par_d = GST_VIDEO_INFO_PAR_D (&self->info);
/* get aspect ratio from caps if it's present, and
* convert video width and height to a display width and height
* using wd / hd = wv / hv * PARv / PARd */
if (!gst_video_calculate_display_ratio (&num, &den, video_width,
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
video_height, video_par_n, video_par_d, display_par_n,
display_par_d)) {
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
LeaveCriticalSection (&self->lock);
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
GST_ELEMENT_ERROR (self, CORE, NEGOTIATION, (nullptr),
("Error calculating the output display ratio of the video."));
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
return GST_FLOW_ERROR;
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
}
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Delay window setup as much as possible ... so that videosink can handle device update with d3d11 device of the first buffer Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-23 10:29:55 +00:00
GST_DEBUG_OBJECT (self,
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
"video width/height: %dx%d, calculated display ratio: %d/%d format: %s",
video_width, video_height, num, den,
gst_video_format_to_string (GST_VIDEO_INFO_FORMAT (&self->info)));
/* now find a width x height that respects this display ratio.
* prefer those that have one of w/h the same as the incoming video
* using wd / hd = num / den
*/
/* start with same height, because of interlaced video
* check hd / den is an integer scale factor, and scale wd with the PAR
*/
if (video_height % den == 0) {
GST_DEBUG_OBJECT (self, "keeping video height");
GST_VIDEO_SINK_WIDTH (self) = (guint)
gst_util_uint64_scale_int (video_height, num, den);
GST_VIDEO_SINK_HEIGHT (self) = video_height;
} else if (video_width % num == 0) {
GST_DEBUG_OBJECT (self, "keeping video width");
GST_VIDEO_SINK_WIDTH (self) = video_width;
GST_VIDEO_SINK_HEIGHT (self) = (guint)
gst_util_uint64_scale_int (video_width, den, num);
} else {
GST_DEBUG_OBJECT (self, "approximating while keeping video height");
GST_VIDEO_SINK_WIDTH (self) = (guint)
gst_util_uint64_scale_int (video_height, num, den);
GST_VIDEO_SINK_HEIGHT (self) = video_height;
}
GST_DEBUG_OBJECT (self, "scaling to %dx%d",
GST_VIDEO_SINK_WIDTH (self), GST_VIDEO_SINK_HEIGHT (self));
d3d11videosink: Take into account pixel aspect ratio Fix unexpected cropping with non 1:1 pixel aspect-ratio. The actual buffer width/height should be passed to gst_d3d11_window_render(), instead of the calculated resolution. The width/height values are parameters for copying d3d11 video memory. Also, aspect-ratio should be considered on resize callback to decide render rectangle size.
2019-08-15 07:20:26 +00:00
self->video_width = video_width;
self->video_height = video_height;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
if (GST_VIDEO_SINK_WIDTH (self) <= 0 || GST_VIDEO_SINK_HEIGHT (self) <= 0) {
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
LeaveCriticalSection (&self->lock);
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
GST_ELEMENT_ERROR (self, CORE, NEGOTIATION, (nullptr),
("Error calculating the output display ratio of the video."));
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
return GST_FLOW_ERROR;
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
}
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Add support for GstVideoOverlay::set_render_rectangle Inspired by an MR https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2382 The idea is that we can make use of MoveWindow() in WIN32 d3d11window implementation safely because WIN32 d3d11window implementation creates internal HWND even when external HWND is set and then subclassing is used to draw on internal HWND in any case. So the coordinates passed to MoveWindow() will be relative to parent HWND, and it meets well to the concept of set_render_rectangle(). On MoveWindow() event, WM_SIZE event will be generated by OS and then GstD3D11WindowWin32 implementation will update render area including swapchain correspondingly, as if it's normal window move/resize case. But in case of UWP (CoreWindow or SwapChainPanel), we need more research to meet expected behavior of set_render_rectangle() Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1416 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2450>
2021-07-30 15:59:14 +00:00
if (self->pending_render_rect) {
GstVideoRectangle rect = self->render_rect;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Add support for GstVideoOverlay::set_render_rectangle Inspired by an MR https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2382 The idea is that we can make use of MoveWindow() in WIN32 d3d11window implementation safely because WIN32 d3d11window implementation creates internal HWND even when external HWND is set and then subclassing is used to draw on internal HWND in any case. So the coordinates passed to MoveWindow() will be relative to parent HWND, and it meets well to the concept of set_render_rectangle(). On MoveWindow() event, WM_SIZE event will be generated by OS and then GstD3D11WindowWin32 implementation will update render area including swapchain correspondingly, as if it's normal window move/resize case. But in case of UWP (CoreWindow or SwapChainPanel), we need more research to meet expected behavior of set_render_rectangle() Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1416 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2450>
2021-07-30 15:59:14 +00:00
self->pending_render_rect = FALSE;
gst_d3d11_window_set_render_rectangle (self->window, &rect);
}
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Add gamma-mode and primaries-mode properties Allows controlling gamma remap and/or chromatic adaptation behavior. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2831>
2022-08-02 16:47:46 +00:00
config = gst_structure_new ("convert-config",
GST_D3D11_CONVERTER_OPT_GAMMA_MODE,
GST_TYPE_VIDEO_GAMMA_MODE, self->gamma_mode,
GST_D3D11_CONVERTER_OPT_PRIMARIES_MODE,
GST_TYPE_VIDEO_PRIMARIES_MODE, self->primaries_mode, nullptr);
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
window = (GstD3D11Window *) gst_object_ref (self->window);
LeaveCriticalSection (&self->lock);
ret = gst_d3d11_window_prepare (window, GST_VIDEO_SINK_WIDTH (self),
GST_VIDEO_SINK_HEIGHT (self), caps, config, self->display_format, &error);
if (ret != GST_FLOW_OK) {
d3d11videosink: Don't try to post error message by d3d11window self The d3d11window isn't GstElement. To post error message, proxy it to d3d11videosink instead.
2019-10-04 12:15:15 +00:00
GstMessage *error_msg;
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
if (ret == GST_FLOW_FLUSHING) {
GstD3D11CSLockGuard lk (&self->lock);
GST_WARNING_OBJECT (self, "Couldn't prepare window but we are flushing");
d3d11videosink: fix race conditions in win32 window One race condition is the fact that the window object can be destroyed while running some routine in the UI thread (such as resizing). To avoid that situation we make UI thread hold a reference on the window object while it's running. Other probpematic case is when the window handle is reused: if we stop and start the pipeline very fast, so the sink creates a new window object that is going to use the same window handle as the previous one. And finally the case when the pipeline is stopped immediatelly right after starting, this one is also handled in this commit. NOTE: a unit test that reproduces this cases have been added in the previous commit. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4417>
2023-03-27 07:20:27 +00:00
gst_d3d11_window_unprepare (self->window);
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
gst_clear_object (&self->window);
gst_object_unref (window);
return GST_FLOW_FLUSHING;
}
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
GST_ERROR_OBJECT (self, "cannot create swapchain");
d3d11videosink: Don't try to post error message by d3d11window self The d3d11window isn't GstElement. To post error message, proxy it to d3d11videosink instead.
2019-10-04 12:15:15 +00:00
error_msg = gst_message_new_error (GST_OBJECT_CAST (self),
error, "Failed to prepare d3d11window");
g_clear_error (&error);
gst_element_post_message (GST_ELEMENT (self), error_msg);
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
gst_object_unref (window);
d3d11videosink: Don't try to post error message by d3d11window self The d3d11window isn't GstElement. To post error message, proxy it to d3d11videosink instead.
2019-10-04 12:15:15 +00:00
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
return GST_FLOW_ERROR;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
if (self->title) {
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
gst_d3d11_window_set_title (window, self->title);
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
g_clear_pointer (&self->title, g_free);
}
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
gst_object_unref (window);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
self->pool = gst_d3d11_buffer_pool_new (self->device);
config = gst_buffer_pool_get_config (self->pool);
if (gst_d3d11_device_get_format (self->device,
GST_VIDEO_INFO_FORMAT (&self->info), &device_format) &&
(device_format.format_support[0] &
(guint) D3D11_FORMAT_SUPPORT_RENDER_TARGET) != 0) {
bind_flags |= D3D11_BIND_RENDER_TARGET;
}
params = gst_d3d11_allocation_params_new (self->device, &self->info,
GST_D3D11_ALLOCATION_FLAG_DEFAULT, bind_flags, 0);
gst_buffer_pool_config_set_d3d11_allocation_params (config, params);
gst_d3d11_allocation_params_free (params);
gst_buffer_pool_config_set_params (config, caps, self->info.size, 0, 0);
if (!gst_buffer_pool_set_config (self->pool, config) ||
!gst_buffer_pool_set_active (self->pool, TRUE)) {
GST_ERROR_OBJECT (self, "Couldn't setup buffer pool");
gst_clear_object (&self->pool);
GST_ELEMENT_ERROR (self, RESOURCE, FAILED, (nullptr),
("Couldn't setup buffer pool"));
return GST_FLOW_ERROR;
}
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
return GST_FLOW_OK;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
static void
gst_d3d11_video_sink_key_event (GstD3D11Window * window, const gchar * event,
const gchar * key, GstD3D11VideoSink * self)
{
d3d11videosink: Early terminate mouse/keyboard event handling ... and add missing null check (plus coding style fix) Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2824>
2022-08-01 19:16:31 +00:00
GstEvent *key_event;
if (!self->enable_navigation_events || !event || !key)
return;
GST_LOG_OBJECT (self, "send key event %s, key %s", event, key);
if (g_strcmp0 ("key-press", event) == 0) {
key_event = gst_navigation_event_new_key_press (key,
GST_NAVIGATION_MODIFIER_NONE);
} else if (g_strcmp0 ("key-release", event) == 0) {
key_event = gst_navigation_event_new_key_release (key,
GST_NAVIGATION_MODIFIER_NONE);
} else {
return;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
d3d11videosink: Early terminate mouse/keyboard event handling ... and add missing null check (plus coding style fix) Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2824>
2022-08-01 19:16:31 +00:00
gst_navigation_send_event_simple (GST_NAVIGATION (self), key_event);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
static void
gst_d3d11_video_mouse_key_event (GstD3D11Window * window, const gchar * event,
gint button, gdouble x, gdouble y, GstD3D11VideoSink * self)
{
d3d11videosink: Early terminate mouse/keyboard event handling ... and add missing null check (plus coding style fix) Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2824>
2022-08-01 19:16:31 +00:00
GstEvent *mouse_event;
if (!self->enable_navigation_events || !event)
return;
GST_LOG_OBJECT (self,
"send mouse event %s, button %d (%.1f, %.1f)", event, button, x, y);
if (g_strcmp0 ("mouse-button-press", event) == 0) {
mouse_event = gst_navigation_event_new_mouse_button_press (button, x, y,
GST_NAVIGATION_MODIFIER_NONE);
} else if (g_strcmp0 ("mouse-button-release", event) == 0) {
mouse_event = gst_navigation_event_new_mouse_button_release (button, x, y,
GST_NAVIGATION_MODIFIER_NONE);
} else if (g_strcmp0 ("mouse-move", event) == 0) {
mouse_event = gst_navigation_event_new_mouse_move (x, y,
GST_NAVIGATION_MODIFIER_NONE);
} else {
return;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
d3d11videosink: Early terminate mouse/keyboard event handling ... and add missing null check (plus coding style fix) Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2824>
2022-08-01 19:16:31 +00:00
gst_navigation_send_event_simple (GST_NAVIGATION (self), mouse_event);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
d3d11videosink: Add present signal The "present" signal will be emitted just before the IDXGISwapChain::Present() call. The client can perform additional GPU operation with given GstD3D11Device object and ID3D11RenderTargetView handle. Or, the client can read back the scene to be displayed on window using the signal. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-19 11:25:31 +00:00
static void
gst_d3d11_video_sink_present (GstD3D11Window * window, GstD3D11Device * device,
ID3D11RenderTargetView * rtv, GstD3D11VideoSink * self)
{
g_signal_emit (self, gst_d3d11_video_sink_signals[SIGNAL_PRESENT], 0,
device, rtv);
}
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
static gboolean
gst_d3d11_video_sink_start (GstBaseSink * sink)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (sink);
GST_DEBUG_OBJECT (self, "Start");
d3d11: Don't share d3d11 device context unconditionally without checking adapter Do not accept any GstD3D11Device context which has different adapter index from the required one. For example, if a d3d11 element is expecting d3d11 device with adapter 1 (i.e., the second GPU), any d3d11 device context having different adapter could not be shared with the d3d11 element.
2019-08-23 12:41:39 +00:00
if (!gst_d3d11_ensure_element_data (GST_ELEMENT_CAST (self), self->adapter,
&self->device)) {
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
GST_ERROR_OBJECT (sink, "Cannot create d3d11device");
return FALSE;
}
d3d11videosink: Prepare window once streaming started ... instead of READY state. READY state is too early for setting overlay window handle especially playbin/playsink scenario since playsink will set given overlay handle on videosink once READY state change of videosink is ensured. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1893>
2020-12-20 13:12:44 +00:00
return TRUE;
}
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
/* called with lock */
d3d11videosink: Prepare window once streaming started ... instead of READY state. READY state is too early for setting overlay window handle especially playbin/playsink scenario since playsink will set given overlay handle on videosink once READY state change of videosink is ensured. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1893>
2020-12-20 13:12:44 +00:00
static gboolean
gst_d3d11_video_sink_prepare_window (GstD3D11VideoSink * self)
{
GstD3D11WindowNativeType window_type = GST_D3D11_WINDOW_NATIVE_TYPE_HWND;
if (self->window)
return TRUE;
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
if (self->draw_on_shared_texture) {
GST_INFO_OBJECT (self,
"Create dummy window for rendering on shared texture");
self->window = gst_d3d11_window_dummy_new (self->device);
d3d11videosink: Fix for force-aspect-ratio setting when rendering on shared texture Set specified force-aspect-ratio value on window object in case of shared texture rendering as well Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1304 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2671>
2022-06-28 08:40:56 +00:00
goto done;
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
}
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
if (!self->window_id)
gst_video_overlay_prepare_window_handle (GST_VIDEO_OVERLAY (self));
if (self->window_id) {
window_type =
gst_d3d11_window_get_native_type_from_handle (self->window_id);
if (window_type != GST_D3D11_WINDOW_NATIVE_TYPE_NONE) {
d3d11videosink: Prepare window once streaming started ... instead of READY state. READY state is too early for setting overlay window handle especially playbin/playsink scenario since playsink will set given overlay handle on videosink once READY state change of videosink is ensured. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1893>
2020-12-20 13:12:44 +00:00
GST_DEBUG_OBJECT (self, "Have window handle %" G_GUINTPTR_FORMAT,
self->window_id);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
gst_video_overlay_got_window_handle (GST_VIDEO_OVERLAY (self),
self->window_id);
}
}
GST_DEBUG_OBJECT (self, "Create window (type: %s)",
gst_d3d11_window_get_native_type_to_string (window_type));
#if GST_D3D11_WINAPI_ONLY_APP
if (window_type != GST_D3D11_WINDOW_NATIVE_TYPE_CORE_WINDOW &&
window_type != GST_D3D11_WINDOW_NATIVE_TYPE_SWAP_CHAIN_PANEL) {
d3d11videosink: Fix build error on UWP gstd3d11videosink.c(662): error C2065: 'sink': undeclared identifier Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1961>
2021-01-18 10:23:30 +00:00
GST_ERROR_OBJECT (self, "Overlay handle must be set before READY state");
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
return FALSE;
}
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
#endif
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
switch (window_type) {
#if (!GST_D3D11_WINAPI_ONLY_APP)
case GST_D3D11_WINDOW_NATIVE_TYPE_HWND:
self->window = gst_d3d11_window_win32_new (self->device, self->window_id);
break;
d3d11: Allow building UWP features with Desktop features if possible WINAPI_PARTITION_DESKTOP and WINAPI_PARTITION_APP can coexist. Although UWP only binaries should be used for production stage, this change will be useful for development stage Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1962>
2021-01-18 10:17:14 +00:00
#endif
#if GST_D3D11_WINAPI_APP
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
case GST_D3D11_WINDOW_NATIVE_TYPE_CORE_WINDOW:
self->window = gst_d3d11_window_core_window_new (self->device,
self->window_id);
break;
case GST_D3D11_WINDOW_NATIVE_TYPE_SWAP_CHAIN_PANEL:
self->window = gst_d3d11_window_swap_chain_panel_new (self->device,
self->window_id);
break;
#endif
default:
break;
}
d3d11videosink: Add color conversion support Draw to back buffer texture directly. It would reduce the number of copy at least once when color conversion is required.
2019-12-03 13:54:26 +00:00
d3d11videosink: Fix for force-aspect-ratio setting when rendering on shared texture Set specified force-aspect-ratio value on window object in case of shared texture rendering as well Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1304 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2671>
2022-06-28 08:40:56 +00:00
done:
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
if (!self->window) {
d3d11videosink: Prepare window once streaming started ... instead of READY state. READY state is too early for setting overlay window handle especially playbin/playsink scenario since playsink will set given overlay handle on videosink once READY state change of videosink is ensured. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1893>
2020-12-20 13:12:44 +00:00
GST_ERROR_OBJECT (self, "Cannot create d3d11window");
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
return FALSE;
d3d11videosink: Add color conversion support Draw to back buffer texture directly. It would reduce the number of copy at least once when color conversion is required.
2019-12-03 13:54:26 +00:00
}
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
g_object_set (self->window,
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
"force-aspect-ratio", self->force_aspect_ratio,
"fullscreen-toggle-mode", self->fullscreen_toggle_mode,
"fullscreen", self->fullscreen,
d3d11videosink: Add present signal The "present" signal will be emitted just before the IDXGISwapChain::Present() call. The client can perform additional GPU operation with given GstD3D11Device object and ID3D11RenderTargetView handle. Or, the client can read back the scene to be displayed on window using the signal. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-19 11:25:31 +00:00
"enable-navigation-events", self->enable_navigation_events,
d3d11videosink: Add "emit-present" property Controls the decision for "present" signal use, and allows delayed "present" signal handler install via the property Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3176>
2022-10-13 15:17:52 +00:00
"emit-present", self->emit_present, nullptr);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Add support for rotation Adding "rotate-method" property Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1396 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:22:10 +00:00
gst_d3d11_window_set_orientation (self->window, self->selected_method);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
g_signal_connect (self->window, "key-event",
G_CALLBACK (gst_d3d11_video_sink_key_event), self);
g_signal_connect (self->window, "mouse-event",
G_CALLBACK (gst_d3d11_video_mouse_key_event), self);
d3d11videosink: Add present signal The "present" signal will be emitted just before the IDXGISwapChain::Present() call. The client can perform additional GPU operation with given GstD3D11Device object and ID3D11RenderTargetView handle. Or, the client can read back the scene to be displayed on window using the signal. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
2022-08-19 11:25:31 +00:00
g_signal_connect (self->window, "present",
G_CALLBACK (gst_d3d11_video_sink_present), self);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
GST_DEBUG_OBJECT (self,
"Have prepared window %" GST_PTR_FORMAT, self->window);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
return TRUE;
}
static gboolean
gst_d3d11_video_sink_stop (GstBaseSink * sink)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (sink);
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
GstD3D11CSLockGuard lk (&self->lock);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
GST_DEBUG_OBJECT (self, "Stop");
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
gst_clear_buffer (&self->prepared_buffer);
if (self->pool) {
gst_buffer_pool_set_active (self->pool, FALSE);
gst_clear_object (&self->pool);
}
d3d11window: Add unprepare method to clear internal resource GObject::dispose method can be called multiple times. As win32 d3d11window has an internal thread and because GObject::dispose method could be called from the thread, it might cause problems such as trying to join self-thread Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1299>
2020-05-26 16:52:59 +00:00
if (self->window)
gst_d3d11_window_unprepare (self->window);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
gst_clear_object (&self->window);
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
gst_clear_object (&self->device);
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
g_clear_pointer (&self->title, g_free);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
return TRUE;
}
static gboolean
gst_d3d11_video_sink_propose_allocation (GstBaseSink * sink, GstQuery * query)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (sink);
GstCaps *caps;
GstBufferPool *pool = NULL;
GstVideoInfo info;
guint size;
gboolean need_pool;
d3d11videosink: Perform propose_allocation() even when we have no configured window In order to support d3d11 device update, d3d11videosink will configure window on the first buffer. So, there might not be configured window when propose_allocation() is required. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/933>
2021-09-18 14:37:59 +00:00
if (!self->device)
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
return FALSE;
gst_query_parse_allocation (query, &caps, &need_pool);
if (caps == NULL)
goto no_caps;
if (!gst_video_info_from_caps (&info, caps))
goto invalid_caps;
/* the normal size of a frame */
size = info.size;
if (need_pool) {
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
GstCapsFeatures *features;
GstStructure *config;
d3d11: Revert staging buffer pool implementation The staging buffer pool implementation was added to improve throughput performance since we can avoid per-frame CPU copy operation via staging texture but it turned out that we can not make it thread safe. See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1182 Reverting the staging texture implementation as it does not show any visible value. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2651>
2022-06-23 12:43:11 +00:00
gboolean is_d3d11 = FALSE;
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
features = gst_caps_get_features (caps, 0);
if (features
&& gst_caps_features_contains (features,
GST_CAPS_FEATURE_MEMORY_D3D11_MEMORY)) {
GST_DEBUG_OBJECT (self, "upstream support d3d11 memory");
pool = gst_d3d11_buffer_pool_new (self->device);
d3d11: Revert staging buffer pool implementation The staging buffer pool implementation was added to improve throughput performance since we can avoid per-frame CPU copy operation via staging texture but it turned out that we can not make it thread safe. See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1182 Reverting the staging texture implementation as it does not show any visible value. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2651>
2022-06-23 12:43:11 +00:00
is_d3d11 = TRUE;
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
} else {
d3d11: Revert staging buffer pool implementation The staging buffer pool implementation was added to improve throughput performance since we can avoid per-frame CPU copy operation via staging texture but it turned out that we can not make it thread safe. See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1182 Reverting the staging texture implementation as it does not show any visible value. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2651>
2022-06-23 12:43:11 +00:00
pool = gst_video_buffer_pool_new ();
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
}
d3d11videosink: Add color conversion support Draw to back buffer texture directly. It would reduce the number of copy at least once when color conversion is required.
2019-12-03 13:54:26 +00:00
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
config = gst_buffer_pool_get_config (pool);
gst_buffer_pool_config_add_option (config,
GST_BUFFER_POOL_OPTION_VIDEO_META);
d3d11: Revert staging buffer pool implementation The staging buffer pool implementation was added to improve throughput performance since we can avoid per-frame CPU copy operation via staging texture but it turned out that we can not make it thread safe. See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1182 Reverting the staging texture implementation as it does not show any visible value. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2651>
2022-06-23 12:43:11 +00:00
if (!is_d3d11) {
gst_buffer_pool_config_add_option (config,
GST_BUFFER_POOL_OPTION_VIDEO_ALIGNMENT);
}
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
size = GST_VIDEO_INFO_SIZE (&info);
if (is_d3d11) {
GstD3D11AllocationParams *d3d11_params;
d3d11_params =
gst_d3d11_allocation_params_new (self->device,
d3d11: Update gst_d3d11_allocation_params_new() interface Define GST_D3D11_ALLOCATION_FLAG_DEFAULT (0) value instead of casting enum type everywhere. And pass D3D11_RESOURCE_MISC_FLAG value Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2646>
2022-06-23 16:21:48 +00:00
&info, GST_D3D11_ALLOCATION_FLAG_DEFAULT, D3D11_BIND_SHADER_RESOURCE,
0);
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
gst_buffer_pool_config_set_d3d11_allocation_params (config, d3d11_params);
gst_d3d11_allocation_params_free (d3d11_params);
}
gst_buffer_pool_config_set_params (config, caps, (guint) size, 2, 0);
if (!gst_buffer_pool_set_config (pool, config)) {
GST_ERROR_OBJECT (pool, "Couldn't set config");
gst_object_unref (pool);
d3d11videosink: Add color conversion support Draw to back buffer texture directly. It would reduce the number of copy at least once when color conversion is required.
2019-12-03 13:54:26 +00:00
d3d11: Add a helper method for d3d11buffferpool setup Remove duplicated code for d3d11buffferpool setup. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1892>
2020-12-19 16:06:24 +00:00
return FALSE;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
d3d11: Add a helper method for d3d11buffferpool setup Remove duplicated code for d3d11buffferpool setup. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1892>
2020-12-19 16:06:24 +00:00
d3d11videosink: Use staging buffer pool for non-d3d11 upstream Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1643>
2022-02-05 16:06:36 +00:00
/* d3d11 buffer pool will update buffer size based on allocated texture,
* get size from config again */
config = gst_buffer_pool_get_config (pool);
gst_buffer_pool_config_get_params (config, nullptr, &size, nullptr,
nullptr);
gst_structure_free (config);
d3d11videosink: Add support for crop meta ... when upstream element is d3d11. Note that, if upstream element is not d3d11, crop meta is almost pointless since d3d11videosink will upload video frame to GPU memory in any case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/933>
2021-09-18 14:37:20 +00:00
d3d11videosink: Use staging buffer pool for non-d3d11 upstream Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1643>
2022-02-05 16:06:36 +00:00
if (is_d3d11) {
d3d11videosink: Add support for crop meta ... when upstream element is d3d11. Note that, if upstream element is not d3d11, crop meta is almost pointless since d3d11videosink will upload video frame to GPU memory in any case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/933>
2021-09-18 14:37:20 +00:00
/* In case of system memory, we will upload video frame to GPU memory,
* (which is copy in any case), so crop meta support for system memory
* is almost pointless */
gst_query_add_allocation_meta (query,
GST_VIDEO_CROP_META_API_TYPE, nullptr);
d3d11bufferpool: Hide buffer_size field from header User can get the required buffer size by using buffer pool config. Since d3d11 implementation is a candidate for public library in the future, we need to hide everything from header as much as possible. Note that the total size of allocated d3d11 texture memory by GPU is not controllable factor. It depends on hardware specific alignment/padding requirement. So, GstD3D11 implementation updates actual buffer size by allocating D3D11 texture, since there's no way to get CPU accessible memory size without allocating real D3D11 texture. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2482>
2021-08-21 15:33:58 +00:00
}
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
d3d11videosink: Don't limit max buffers of buffer pool In some case, especially reverse playback, we would need more than two buffers. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1989>
2021-01-27 17:02:28 +00:00
/* We need at least 2 buffers because we hold on to the last one for redrawing
* on window-resize event */
gst_query_add_allocation_pool (query, pool, size, 2, 0);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
if (pool)
d3d11: Use correct ref/unref methods Those objects are GstObject subclasses Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3479>
2022-11-28 17:06:11 +00:00
gst_object_unref (pool);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
gst_query_add_allocation_meta (query, GST_VIDEO_META_API_TYPE, NULL);
d3d11videosink: Add support for overlay composition Add d3d11overlaycompositor object to draw overlay image on render target using Blend method.
2019-12-24 06:54:57 +00:00
gst_query_add_allocation_meta (query,
GST_VIDEO_OVERLAY_COMPOSITION_META_API_TYPE, NULL);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
return TRUE;
/* ERRORS */
no_caps:
{
GST_WARNING_OBJECT (self, "no caps specified");
return FALSE;
}
invalid_caps:
{
GST_WARNING_OBJECT (self, "invalid caps specified");
return FALSE;
}
return TRUE;
}
d3d11videosink: Handle context query Propagate d3d11device via context query.
2019-07-18 14:12:13 +00:00
static gboolean
gst_d3d11_video_sink_query (GstBaseSink * sink, GstQuery * query)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (sink);
switch (GST_QUERY_TYPE (query)) {
case GST_QUERY_CONTEXT:
if (gst_d3d11_handle_context_query (GST_ELEMENT (self), query,
self->device)) {
return TRUE;
}
break;
default:
break;
}
return GST_BASE_SINK_CLASS (parent_class)->query (sink, query);
}
d3d11videosink: Draw window with cached texture on resize This would render nicer than presenting scene with broken aspect ratio, especially in case of low framerate.
2019-12-02 14:27:42 +00:00
static gboolean
gst_d3d11_video_sink_unlock (GstBaseSink * sink)
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
{
d3d11videosink: Draw window with cached texture on resize This would render nicer than presenting scene with broken aspect ratio, especially in case of low framerate.
2019-12-02 14:27:42 +00:00
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (sink);
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
GstD3D11CSLockGuard lk (&self->lock);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Draw window with cached texture on resize This would render nicer than presenting scene with broken aspect ratio, especially in case of low framerate.
2019-12-02 14:27:42 +00:00
if (self->window)
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
gst_d3d11_window_unlock (self->window);
return TRUE;
}
static gboolean
gst_d3d11_video_sink_unlock_stop (GstBaseSink * sink)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (sink);
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
GstD3D11CSLockGuard lk (&self->lock);
d3d11: Add support for Universal Windows Platform Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2019-12-30 09:58:59 +00:00
if (self->window)
gst_d3d11_window_unlock_stop (self->window);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Draw window with cached texture on resize This would render nicer than presenting scene with broken aspect ratio, especially in case of low framerate.
2019-12-02 14:27:42 +00:00
return TRUE;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
static gboolean
gst_d3d11_video_sink_event (GstBaseSink * sink, GstEvent * event)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (sink);
switch (GST_EVENT_TYPE (event)) {
case GST_EVENT_TAG:{
GstTagList *taglist;
gchar *title = nullptr;
d3d11videosink: Add support for rotation Adding "rotate-method" property Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1396 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:22:10 +00:00
GstVideoOrientationMethod method = GST_VIDEO_ORIENTATION_IDENTITY;
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
gst_event_parse_tag (event, &taglist);
gst_tag_list_get_string (taglist, GST_TAG_TITLE, &title);
if (title) {
const gchar *app_name = g_get_application_name ();
std::string title_string;
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
GstD3D11CSLockGuard lk (&self->lock);
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
if (app_name) {
title_string = std::string (title) + " : " + std::string (app_name);
} else {
title_string = std::string (title);
}
if (self->window) {
gst_d3d11_window_set_title (self->window, title_string.c_str ());
} else {
g_free (self->title);
self->title = g_strdup (title_string.c_str ());
}
g_free (title);
}
d3d11videosink: Add support for rotation Adding "rotate-method" property Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1396 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:22:10 +00:00
if (gst_video_orientation_from_tag (taglist, &method)) {
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
GstD3D11CSLockGuard lk (&self->lock);
d3d11videosink: Add support for rotation Adding "rotate-method" property Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1396 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:22:10 +00:00
gst_d3d11_video_sink_set_orientation (self, method, TRUE);
}
d3d11videosink: Display title of content if possible Update title text of window (currently it's always "Direct3D11 renderer") when we are rendering on internal HWND, not external HWND. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2529>
2021-09-17 17:27:51 +00:00
break;
}
default:
break;
}
return GST_BASE_SINK_CLASS (parent_class)->event (sink, event);
}
d3d11videosink: Add support for rotation Adding "rotate-method" property Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1396 Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:22:10 +00:00
/* called with lock */
static void
gst_d3d11_video_sink_set_orientation (GstD3D11VideoSink * self,
GstVideoOrientationMethod method, gboolean from_tag)
{
if (method == GST_VIDEO_ORIENTATION_CUSTOM) {
GST_WARNING_OBJECT (self, "Unsupported custom orientation");
return;
}
if (from_tag)
self->tag_method = method;
else
self->method = method;
if (self->method == GST_VIDEO_ORIENTATION_AUTO) {
self->selected_method = self->tag_method;
} else {
self->selected_method = self->method;
}
if (self->window)
gst_d3d11_window_set_orientation (self->window, self->selected_method);
}
d3d11: Handle device change If incoming buffer holds other d3d11 device, and user wants any device (i.e., adapter index wasn't specified explicitly) update our device with that of buffer. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-27 12:52:31 +00:00
static void
gst_d3d11_video_sink_check_device_update (GstD3D11VideoSink * self,
GstBuffer * buf)
{
GstMemory *mem;
GstD3D11Memory *dmem;
gboolean update_device = FALSE;
/* We have configured window already, cannot update device */
if (self->window)
return;
mem = gst_buffer_peek_memory (buf, 0);
if (!gst_is_d3d11_memory (mem))
return;
dmem = GST_D3D11_MEMORY_CAST (mem);
/* Same device, nothing to do */
if (dmem->device == self->device)
return;
if (self->adapter < 0) {
update_device = TRUE;
} else {
guint adapter = 0;
g_object_get (dmem->device, "adapter", &adapter, NULL);
/* The same GPU as what user wanted, update */
if (adapter == (guint) self->adapter)
update_device = TRUE;
}
if (!update_device)
return;
GST_INFO_OBJECT (self, "Updating device %" GST_PTR_FORMAT " -> %"
GST_PTR_FORMAT, self->device, dmem->device);
gst_object_unref (self->device);
self->device = (GstD3D11Device *) gst_object_ref (dmem->device);
}
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
static GstFlowReturn
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
gst_d3d11_video_sink_prepare (GstBaseSink * sink, GstBuffer * buffer)
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (sink);
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
GstFlowReturn ret;
d3d11videosink: Add color conversion support Draw to back buffer texture directly. It would reduce the number of copy at least once when color conversion is required.
2019-12-03 13:54:26 +00:00
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
gst_clear_buffer (&self->prepared_buffer);
d3d11: Handle device change If incoming buffer holds other d3d11 device, and user wants any device (i.e., adapter index wasn't specified explicitly) update our device with that of buffer. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-27 12:52:31 +00:00
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
gst_d3d11_video_sink_check_device_update (self, buffer);
d3d11videosink: Delay window setup as much as possible ... so that videosink can handle device update with d3d11 device of the first buffer Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-23 10:29:55 +00:00
if (self->caps_updated || !self->window) {
GstCaps *caps = gst_pad_get_current_caps (GST_BASE_SINK_PAD (sink));
/* shouldn't happen */
if (!caps)
return GST_FLOW_NOT_NEGOTIATED;
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
ret = gst_d3d11_video_sink_update_window (self, caps);
d3d11videosink: Delay window setup as much as possible ... so that videosink can handle device update with d3d11 device of the first buffer Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-23 10:29:55 +00:00
gst_caps_unref (caps);
d3d11videosink: Fix deadlock when parent window is busy Deadlock sequence: * From a streaming thread, d3d11videosink sends synchronous message to the parent window, so that internal (child) window can be constructed on the parent window's thread * App thread (parent window thread) is waiting for pipeline's state change (to GST_STATE_NULL) but streaming thread is blocked and waiting for app thread To avoid the deadlock, GstD3D11WindowWin32 should send message to the parent window asynchronously. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3570>
2022-12-14 16:15:10 +00:00
if (ret != GST_FLOW_OK)
return ret;
d3d11videosink: Delay window setup as much as possible ... so that videosink can handle device update with d3d11 device of the first buffer Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2191>
2021-04-23 10:29:55 +00:00
}
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
if (!gst_is_d3d11_buffer (buffer)) {
GstVideoOverlayCompositionMeta *overlay_meta;
ret = gst_buffer_pool_acquire_buffer (self->pool, &self->prepared_buffer,
nullptr);
if (ret != GST_FLOW_OK)
return ret;
gst_d3d11_buffer_copy_into (self->prepared_buffer, buffer, &self->info);
/* Upload to default texture */
for (guint i = 0; i < gst_buffer_n_memory (self->prepared_buffer); i++) {
GstMemory *mem = gst_buffer_peek_memory (self->prepared_buffer, i);
GstMapInfo info;
if (!gst_memory_map (mem,
&info, (GstMapFlags) (GST_MAP_READ | GST_MAP_D3D11))) {
GST_ERROR_OBJECT (self, "Couldn't map fallback buffer");
gst_clear_buffer (&self->prepared_buffer);
return GST_FLOW_ERROR;
}
gst_memory_unmap (mem, &info);
}
overlay_meta = gst_buffer_get_video_overlay_composition_meta (buffer);
if (overlay_meta) {
gst_buffer_add_video_overlay_composition_meta (self->prepared_buffer,
overlay_meta->overlay);
}
} else {
self->prepared_buffer = gst_buffer_ref (buffer);
}
return GST_FLOW_OK;
}
static GstFlowReturn
gst_d3d11_video_sink_show_frame (GstVideoSink * sink, GstBuffer * buf)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (sink);
GstFlowReturn ret = GST_FLOW_OK;
if (!self->prepared_buffer) {
GST_ERROR_OBJECT (self, "No prepared buffer");
return GST_FLOW_ERROR;
}
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
if (self->draw_on_shared_texture) {
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
GstD3D11CSLockGuard lk (&self->lock);
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
self->drawing = TRUE;
GST_LOG_OBJECT (self, "Begin drawing");
/* Application should call draw method on this callback */
d3d11: Remove d3d11videosink wrapper bin Drop d3d11videosink wrapper bin and handle texture upload in d3d11videosink. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2169>
2021-04-17 11:37:13 +00:00
g_signal_emit (self, gst_d3d11_video_sink_signals[SIGNAL_BEGIN_DRAW], 0,
NULL);
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
GST_LOG_OBJECT (self, "End drawing");
self->drawing = FALSE;
} else {
d3d11videosink: Fix rendering on external handle Partial revert of the commit 068a5c1053ae05b3c7747243948447e764d25aa6. That introduced size mismatch between internal and external HWND Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4010>
2023-02-20 15:27:27 +00:00
gst_d3d11_window_show (self->window);
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
ret = gst_d3d11_window_render (self->window, self->prepared_buffer);
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
}
d3d11videosink: Add a property to support rendering statistics data on window Add a new property "render-stats" to allow rendering statistics data on window for debugging and/or development purpose. Text rendering will be accelerated by GPU since this implementation uses Direct2D/DirectWrite API and Direct3D inter-op for minimal overhead. Specifically, text data will be rendered on swapchain backbuffer directly without any copy/allocation of extra texture. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1830>
2020-11-21 19:39:57 +00:00
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
if (ret == GST_D3D11_WINDOW_FLOW_CLOSED) {
GST_ELEMENT_ERROR (self, RESOURCE, NOT_FOUND,
("Output window was closed"), (NULL));
ret = GST_FLOW_ERROR;
}
return ret;
}
/* VideoOverlay interface */
static void
gst_d3d11_video_sink_set_window_handle (GstVideoOverlay * overlay,
guintptr window_id)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (overlay);
GST_DEBUG ("set window handle %" G_GUINTPTR_FORMAT, window_id);
self->window_id = window_id;
}
static void
gst_d3d11_video_sink_set_render_rectangle (GstVideoOverlay * overlay, gint x,
gint y, gint width, gint height)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (overlay);
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
GstD3D11CSLockGuard lk (&self->lock);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
GST_DEBUG_OBJECT (self,
"render rect x: %d, y: %d, width: %d, height %d", x, y, width, height);
if (self->window) {
d3d11videosink: Add support for GstVideoOverlay::set_render_rectangle Inspired by an MR https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2382 The idea is that we can make use of MoveWindow() in WIN32 d3d11window implementation safely because WIN32 d3d11window implementation creates internal HWND even when external HWND is set and then subclassing is used to draw on internal HWND in any case. So the coordinates passed to MoveWindow() will be relative to parent HWND, and it meets well to the concept of set_render_rectangle(). On MoveWindow() event, WM_SIZE event will be generated by OS and then GstD3D11WindowWin32 implementation will update render area including swapchain correspondingly, as if it's normal window move/resize case. But in case of UWP (CoreWindow or SwapChainPanel), we need more research to meet expected behavior of set_render_rectangle() Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1416 Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2450>
2021-07-30 15:59:14 +00:00
GstVideoRectangle rect;
rect.x = x;
rect.y = y;
rect.w = width;
rect.h = height;
self->render_rect = rect;
gst_d3d11_window_set_render_rectangle (self->window, &rect);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
} else {
self->render_rect.x = x;
self->render_rect.y = y;
self->render_rect.w = width;
self->render_rect.h = height;
self->pending_render_rect = TRUE;
}
}
static void
gst_d3d11_video_sink_expose (GstVideoOverlay * overlay)
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (overlay);
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
GstD3D11CSLockGuard lk (&self->lock);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
d3d11videosink: Protect window with lock at every place Access to the object should be thread safe to support runtime property update Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2708>
2022-07-02 16:18:19 +00:00
if (self->window && self->window->swap_chain)
gst_d3d11_window_render (self->window, nullptr);
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
static void
gst_d3d11_video_sink_video_overlay_init (GstVideoOverlayInterface * iface)
{
iface->set_window_handle = gst_d3d11_video_sink_set_window_handle;
iface->set_render_rectangle = gst_d3d11_video_sink_set_render_rectangle;
iface->expose = gst_d3d11_video_sink_expose;
}
/* Navigation interface */
static void
gst_d3d11_video_sink_navigation_send_event (GstNavigation * navigation,
all: Use new navigation interface and API Use and implement the new navigation interface in all relevant sink elements, and use API functions everywhere instead of directy accessing the event structure. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1633>
2022-02-14 15:08:23 +00:00
GstEvent * event)
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
{
GstD3D11VideoSink *self = GST_D3D11_VIDEO_SINK (navigation);
d3d11videosink: Forward navigation event without modification Current implementation for translating native coordinate and video coordinate is very wrong because d3d11videosink doesn't understand native HWND's coordinate. That should be handled by GstD3D11Window implementation as an enhancement. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2450>
2021-07-30 16:05:47 +00:00
/* TODO: add support for translating native coordinate and video coordinate
* when force-aspect-ratio is set */
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
if (event) {
d3d11videosink: Forward navigation event without modification Current implementation for translating native coordinate and video coordinate is very wrong because d3d11videosink doesn't understand native HWND's coordinate. That should be handled by GstD3D11Window implementation as an enhancement. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2450>
2021-07-30 16:05:47 +00:00
gboolean handled;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
gst_event_ref (event);
handled = gst_pad_push_event (GST_VIDEO_SINK_PAD (self), event);
if (!handled)
gst_element_post_message (GST_ELEMENT_CAST (self),
gst_navigation_message_new_event (GST_OBJECT_CAST (self), event));
gst_event_unref (event);
}
}
static void
gst_d3d11_video_sink_navigation_init (GstNavigationInterface * iface)
{
all: Use new navigation interface and API Use and implement the new navigation interface in all relevant sink elements, and use API functions everywhere instead of directy accessing the event structure. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1633>
2022-02-14 15:08:23 +00:00
iface->send_event_simple = gst_d3d11_video_sink_navigation_send_event;
d3d11videosink: Add new Direct3D11 video render plugin Direct3D11 was shipped as part of Windows7 and it's obviously primary graphics API on Windows. This plugin includes HDR10 rendering if following requirements are satisfied * IDXGISwapChain4::SetHDRMetaData is available (decleared in dxgi1_5.h) * Display can support DXGI_COLOR_SPACE_RGB_FULL_G2084_NONE_P2020 color space * Upstream provides 10 bitdepth format with smpte-st 2084 static metadata
2019-01-30 11:07:29 +00:00
}
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
static gboolean
gst_d3d11_video_sink_draw_action (GstD3D11VideoSink * self,
gpointer shared_handle, guint texture_misc_flags,
guint64 acquire_key, guint64 release_key)
{
GstFlowReturn ret;
g_return_val_if_fail (shared_handle != NULL, FALSE);
if (!self->draw_on_shared_texture) {
GST_ERROR_OBJECT (self, "Invalid draw call, we are drawing on window");
return FALSE;
}
if (!shared_handle) {
GST_ERROR_OBJECT (self, "Invalid handle");
return FALSE;
}
d3d11: Use CRITICAL_SECTION instead of GRecMutex The GRecMutex abstraction (and heap allocation happens in GLib) is unnecessary for this plugin. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2914>
2022-08-18 15:05:17 +00:00
GstD3D11CSLockGuard lk (&self->lock);
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
if (!self->drawing || !self->prepared_buffer) {
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
GST_WARNING_OBJECT (self, "Nothing to draw");
return FALSE;
}
GST_LOG_OBJECT (self, "Drawing on shared handle %p, MiscFlags: 0x%x"
", acquire key: %" G_GUINT64_FORMAT ", release key: %"
G_GUINT64_FORMAT, shared_handle, texture_misc_flags, acquire_key,
release_key);
ret = gst_d3d11_window_render_on_shared_handle (self->window,
d3d11videosink: Move potentially time-consuming operations to ::prepare() Move following tasks to ::prepare() from ::show_frame() * CPU -> GPU upload * GstD3D11Window object setup, including input caps change handling Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3574>
2022-12-15 17:52:08 +00:00
self->prepared_buffer, shared_handle, texture_misc_flags, acquire_key,
d3d11videosink: Add support for drawing on application's own texture Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
2020-12-23 14:49:12 +00:00
release_key);
return ret == GST_FLOW_OK;
}
Reference in a new issue Copy permalink