mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2024-11-08 10:31:05 +00:00
Code Issues 1 Projects Releases Wiki Activity
gstreamer/subprojects/gst-plugins-bad/gst-libs/gst/d3d11/gstd3d11format.h

104 lines
3.6 KiB
C
Raw Normal View History

d3d11: Refactor d3d11 memory and dxgi format usage * Create staging texture only when the CPU access is requested. Note that we should avoid the CPU access to d3d11 memory as mush as possible. Incoming d3d11upload and d3d11download will take this GPU memory upload/download. * Upload/Download texture memory from/to staging only if it needed, similar to GstGL PBO implementation. * Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder). Because I420_* formats are not supported formats by dxgi, each plane should be handled likewise GstGL separately, but NV12/P10 formats might be supported ones. So we decide the number of d3d11memory per GstBuffer for video memory depending on OS version and dxgi format. For instance, if NV12 is supported by OS, only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit. One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes. Such behavior will be controlled via configuration of GstD3D11BufferPool and default configuration is separate resources per plane.
2019-11-06 10:37:33 +00:00
/* GStreamer
* Copyright (C) 2019 Seungha Yang <seungha.yang@navercorp.com>
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Library General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Library General Public License for more details.
*
* You should have received a copy of the GNU Library General Public
* License along with this library; if not, write to the
* Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
* Boston, MA 02110-1301, USA.
*/
d3d11: Install library headers Our Direct3D11 abstraction layer has been improved and it gained good shape from API point of view. Also, On Windows, GstD3D11 has various advantages over GstGL in terms of compatibility/stability/feature/performance. Note that WGL implementation is known to be buggy for some drivers/vendors/scenario (that's a reason why Google implemented ANGLE). Moreover, GstGL is not fully optimized for Windows unfortunately. It's the time to open this interface to application developers for various optimized processing using our Direct3D11 infrastructure. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2646>
2022-06-23 16:25:07 +00:00
#pragma once
d3d11: Refactor d3d11 memory and dxgi format usage * Create staging texture only when the CPU access is requested. Note that we should avoid the CPU access to d3d11 memory as mush as possible. Incoming d3d11upload and d3d11download will take this GPU memory upload/download. * Upload/Download texture memory from/to staging only if it needed, similar to GstGL PBO implementation. * Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder). Because I420_* formats are not supported formats by dxgi, each plane should be handled likewise GstGL separately, but NV12/P10 formats might be supported ones. So we decide the number of d3d11memory per GstBuffer for video memory depending on OS version and dxgi format. For instance, if NV12 is supported by OS, only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit. One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes. Such behavior will be controlled via configuration of GstD3D11BufferPool and default configuration is separate resources per plane.
2019-11-06 10:37:33 +00:00
#include <gst/gst.h>
#include <gst/video/video.h>
d3d11: Move core methods to gst-libs Move d3d11 device, memory, buffer pool and minimal method to gst-libs so that other plugins can access d3d11 resource. Since Direct3D is primary graphics API on Windows, we need this infrastructure for various plugins can share GPU resource without downloading GPU memory. Note that this implementation is public only for -bad scope for now. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2020-12-20 17:47:45 +00:00
#include <gst/d3d11/gstd3d11_fwd.h>
d3d11: Refactor d3d11 memory and dxgi format usage * Create staging texture only when the CPU access is requested. Note that we should avoid the CPU access to d3d11 memory as mush as possible. Incoming d3d11upload and d3d11download will take this GPU memory upload/download. * Upload/Download texture memory from/to staging only if it needed, similar to GstGL PBO implementation. * Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder). Because I420_* formats are not supported formats by dxgi, each plane should be handled likewise GstGL separately, but NV12/P10 formats might be supported ones. So we decide the number of d3d11memory per GstBuffer for video memory depending on OS version and dxgi format. For instance, if NV12 is supported by OS, only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit. One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes. Such behavior will be controlled via configuration of GstD3D11BufferPool and default configuration is separate resources per plane.
2019-11-06 10:37:33 +00:00
d3d11: Move core methods to gst-libs Move d3d11 device, memory, buffer pool and minimal method to gst-libs so that other plugins can access d3d11 resource. Since Direct3D is primary graphics API on Windows, we need this infrastructure for various plugins can share GPU resource without downloading GPU memory. Note that this implementation is public only for -bad scope for now. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2020-12-20 17:47:45 +00:00
G_BEGIN_DECLS
d3d11: Refactor d3d11 memory and dxgi format usage * Create staging texture only when the CPU access is requested. Note that we should avoid the CPU access to d3d11 memory as mush as possible. Incoming d3d11upload and d3d11download will take this GPU memory upload/download. * Upload/Download texture memory from/to staging only if it needed, similar to GstGL PBO implementation. * Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder). Because I420_* formats are not supported formats by dxgi, each plane should be handled likewise GstGL separately, but NV12/P10 formats might be supported ones. So we decide the number of d3d11memory per GstBuffer for video memory depending on OS version and dxgi format. For instance, if NV12 is supported by OS, only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit. One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes. Such behavior will be controlled via configuration of GstD3D11BufferPool and default configuration is separate resources per plane.
2019-11-06 10:37:33 +00:00
d3d11: Add support for packed 8bits 4:2:2 YUV formats Note that newly added formats (YUY2, UYVY, and VYUY) are not supported render target view formats. So such formats can be only input of d3d11convert or d3d11videosink. Another note is that YUY2 format is a very common format for hardware en/decoders on Windows. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1581>
2020-09-17 16:41:35 +00:00
#define GST_D3D11_COMMON_FORMATS \
d3d11: Add support for AYUV, AYUV64, and RGBA64_LE formats Note that AYUV and AYUV64 formats will be used to expand format support, especially some packed YUV formats (e.g., Y410, YUY2) are common DXGI formats used for hardware decoder/encoder on Windows but those formats cannot be used as a render target. We need to handle them differently without pixel shader help, using compute shader for example. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1699>
2022-02-14 13:59:01 +00:00
"RGBA64_LE, RGB10A2_LE, BGRA, RGBA, BGRx, RGBx, VUYA, NV12, NV21, " \
d3d11: Add support for GRAY and more YUV formats By this commit, following formats will be newly supported by d3d11 elements * Y444_{8, 12, 16}LE formats: Similar to other planar formats. Such Y444 variants are not supported by Direct3D11 natively, but we can simply map each plane by using R8 and/or R16 texture. * P012_LE: It is not different from P016_LE, but defining P012 and P016 separately for more explicit signalling. Note that DXVA uses P016 texture for 12bits encoded bitstreams. * GRAY: This format is required for some codecs (e.g., AV1) if monochrome is supported * 4:2:0 planar 12bits (I420_12LE) and 4:2:2 planar 8, 10, 12bits formats (Y42B, I422_10LE, and I422_12LE) Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2346>
2021-06-21 08:13:33 +00:00
"P010_10LE, P012_LE, P016_LE, I420, YV12, I420_10LE, I420_12LE, " \
"Y42B, I422_10LE, I422_12LE, Y444, Y444_10LE, Y444_12LE, Y444_16LE, " \
d3d11: Add support for AYUV, AYUV64, and RGBA64_LE formats Note that AYUV and AYUV64 formats will be used to expand format support, especially some packed YUV formats (e.g., Y410, YUY2) are common DXGI formats used for hardware decoder/encoder on Windows but those formats cannot be used as a render target. We need to handle them differently without pixel shader help, using compute shader for example. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1699>
2022-02-14 13:59:01 +00:00
"GRAY8, GRAY16_LE, AYUV, AYUV64"
d3d11: Add support for packed 8bits 4:2:2 YUV formats Note that newly added formats (YUY2, UYVY, and VYUY) are not supported render target view formats. So such formats can be only input of d3d11convert or d3d11videosink. Another note is that YUY2 format is a very common format for hardware en/decoders on Windows. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1581>
2020-09-17 16:41:35 +00:00
#define GST_D3D11_EXTRA_IN_FORMATS \
d3d11: Disable packed and subsampled YUV formats Direct3D11 sampler doesn't support them very well, and conversion outputs usually result in poor visual quality with our shader code. Should disable support for such formats for now Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2344>
2021-06-20 15:19:17 +00:00
"Y410"
d3d11: Add support for packed 8bits 4:2:2 YUV formats Note that newly added formats (YUY2, UYVY, and VYUY) are not supported render target view formats. So such formats can be only input of d3d11convert or d3d11videosink. Another note is that YUY2 format is a very common format for hardware en/decoders on Windows. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1581>
2020-09-17 16:41:35 +00:00
#define GST_D3D11_SINK_FORMATS \
"{ " GST_D3D11_COMMON_FORMATS " ," GST_D3D11_EXTRA_IN_FORMATS " }"
#define GST_D3D11_SRC_FORMATS \
"{ " GST_D3D11_COMMON_FORMATS " }"
#define GST_D3D11_ALL_FORMATS \
"{ " GST_D3D11_COMMON_FORMATS " ," GST_D3D11_EXTRA_IN_FORMATS " }"
d3d11: Refactor d3d11 memory and dxgi format usage * Create staging texture only when the CPU access is requested. Note that we should avoid the CPU access to d3d11 memory as mush as possible. Incoming d3d11upload and d3d11download will take this GPU memory upload/download. * Upload/Download texture memory from/to staging only if it needed, similar to GstGL PBO implementation. * Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder). Because I420_* formats are not supported formats by dxgi, each plane should be handled likewise GstGL separately, but NV12/P10 formats might be supported ones. So we decide the number of d3d11memory per GstBuffer for video memory depending on OS version and dxgi format. For instance, if NV12 is supported by OS, only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit. One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes. Such behavior will be controlled via configuration of GstD3D11BufferPool and default configuration is separate resources per plane.
2019-11-06 10:37:33 +00:00
struct _GstD3D11Format
{
GstVideoFormat format;
/* direct mapping to dxgi format if applicable */
DXGI_FORMAT dxgi_format;
/* formats for texture processing */
d3d11: Move core methods to gst-libs Move d3d11 device, memory, buffer pool and minimal method to gst-libs so that other plugins can access d3d11 resource. Since Direct3D is primary graphics API on Windows, we need this infrastructure for various plugins can share GPU resource without downloading GPU memory. Note that this implementation is public only for -bad scope for now. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2020-12-20 17:47:45 +00:00
DXGI_FORMAT resource_format[GST_VIDEO_MAX_PLANES];
d3d11: Refactor d3d11 memory and dxgi format usage * Create staging texture only when the CPU access is requested. Note that we should avoid the CPU access to d3d11 memory as mush as possible. Incoming d3d11upload and d3d11download will take this GPU memory upload/download. * Upload/Download texture memory from/to staging only if it needed, similar to GstGL PBO implementation. * Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder). Because I420_* formats are not supported formats by dxgi, each plane should be handled likewise GstGL separately, but NV12/P10 formats might be supported ones. So we decide the number of d3d11memory per GstBuffer for video memory depending on OS version and dxgi format. For instance, if NV12 is supported by OS, only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit. One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes. Such behavior will be controlled via configuration of GstD3D11BufferPool and default configuration is separate resources per plane.
2019-11-06 10:37:33 +00:00
d3d11device: Enhance format enumeration and debugging * Enhance debug log to print human readable D3D11_FORMAT_SUPPORT flags value, instead of packed numeric flagset value. * Only device supported format will be added to format table. Depending on device feature level (i.e., D3D9 feature devices), 16bits formats will not be supported. Although there might be formats we deinfed but not supported, it will not be a major issue in practice since our D3D11 implementation does not support legacy devices already (known limitation) and also old d3dvideosink will be promoted in that case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2441>
2022-05-14 15:56:59 +00:00
/* extra format used for unordered access view (unused) */
d3d11: Reorganize format mapping table ... and handle GST <-> D3D11 format mapping in a single place to make adding format easier Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1699>
2022-02-13 11:32:48 +00:00
DXGI_FORMAT uav_format[GST_VIDEO_MAX_PLANES];
d3d11window: Reorganize display colorspace selection The set of supported color space by DXGI is not full combination of our colorimetry. That means we should convert color space to one of supported color space by DXGI. This commit modifies the color space selection step so that d3d11window can find the best matching DXGI color space first and then the selected input/output color space will be referenced by shader and/or d3d11videoprocessor.
2020-03-09 11:29:17 +00:00
d3d11device: Enhance format enumeration and debugging * Enhance debug log to print human readable D3D11_FORMAT_SUPPORT flags value, instead of packed numeric flagset value. * Only device supported format will be added to format table. Depending on device feature level (i.e., D3D9 feature devices), 16bits formats will not be supported. Although there might be formats we deinfed but not supported, it will not be a major issue in practice since our D3D11 implementation does not support legacy devices already (known limitation) and also old d3dvideosink will be promoted in that case. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2441>
2022-05-14 15:56:59 +00:00
/* D3D11_FORMAT_SUPPORT flags */
guint format_support[GST_VIDEO_MAX_PLANES];
/* D3D11_FORMAT_SUPPORT2 flags (unused) */
guint format_support2[GST_VIDEO_MAX_PLANES];
/*< private >*/
guint padding[GST_PADDING_LARGE];
d3d11: Reorganize format mapping table ... and handle GST <-> D3D11 format mapping in a single place to make adding format easier Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1699>
2022-02-13 11:32:48 +00:00
};
d3d11: Refactor d3d11 memory and dxgi format usage * Create staging texture only when the CPU access is requested. Note that we should avoid the CPU access to d3d11 memory as mush as possible. Incoming d3d11upload and d3d11download will take this GPU memory upload/download. * Upload/Download texture memory from/to staging only if it needed, similar to GstGL PBO implementation. * Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder). Because I420_* formats are not supported formats by dxgi, each plane should be handled likewise GstGL separately, but NV12/P10 formats might be supported ones. So we decide the number of d3d11memory per GstBuffer for video memory depending on OS version and dxgi format. For instance, if NV12 is supported by OS, only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit. One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes. Such behavior will be controlled via configuration of GstD3D11BufferPool and default configuration is separate resources per plane.
2019-11-06 10:37:33 +00:00
d3d11: Move core methods to gst-libs Move d3d11 device, memory, buffer pool and minimal method to gst-libs so that other plugins can access d3d11 resource. Since Direct3D is primary graphics API on Windows, we need this infrastructure for various plugins can share GPU resource without downloading GPU memory. Note that this implementation is public only for -bad scope for now. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2020-12-20 17:47:45 +00:00
GST_D3D11_API
d3d11: Refactor d3d11 memory and dxgi format usage * Create staging texture only when the CPU access is requested. Note that we should avoid the CPU access to d3d11 memory as mush as possible. Incoming d3d11upload and d3d11download will take this GPU memory upload/download. * Upload/Download texture memory from/to staging only if it needed, similar to GstGL PBO implementation. * Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder). Because I420_* formats are not supported formats by dxgi, each plane should be handled likewise GstGL separately, but NV12/P10 formats might be supported ones. So we decide the number of d3d11memory per GstBuffer for video memory depending on OS version and dxgi format. For instance, if NV12 is supported by OS, only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit. One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes. Such behavior will be controlled via configuration of GstD3D11BufferPool and default configuration is separate resources per plane.
2019-11-06 10:37:33 +00:00
gboolean gst_d3d11_dxgi_format_get_size (DXGI_FORMAT format,
guint width,
guint height,
guint pitch,
gsize offset[GST_VIDEO_MAX_PLANES],
gint stride[GST_VIDEO_MAX_PLANES],
gsize *size);
d3d11: Add a method for conversion from DXGI format to GstVideoFormat Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2016>
2021-02-13 21:23:55 +00:00
GST_D3D11_API
GstVideoFormat gst_d3d11_dxgi_format_to_gst (DXGI_FORMAT format);
d3d11: Store device format in struct Holding pointer to struct was unsafe approach because the pointer to d3d11 format will be invalidated once d3d11 device object is released Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1699>
2022-02-12 19:36:39 +00:00
GST_D3D11_API
void gst_d3d11_format_init (GstD3D11Format * format);
d3d11: Reorganize format mapping table ... and handle GST <-> D3D11 format mapping in a single place to make adding format easier Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1699>
2022-02-13 11:32:48 +00:00
GST_D3D11_API
guint gst_d3d11_dxgi_format_get_resource_format (DXGI_FORMAT format,
DXGI_FORMAT resource_format[GST_VIDEO_MAX_PLANES]);
GST_D3D11_API
guint gst_d3d11_dxgi_format_get_alignment (DXGI_FORMAT format);
GST_D3D11_API
const gchar * gst_d3d11_dxgi_format_to_string (DXGI_FORMAT format);
d3d11format: Move DXGI color space mapping method to gst-libs Expose color space mapping method to public so that application can configure GstCaps or GstVideoInfo using DXGI color space Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2776>
2022-07-16 19:48:24 +00:00
GST_D3D11_API
gboolean gst_video_info_to_dxgi_color_space (const GstVideoInfo * info,
DXGI_COLOR_SPACE_TYPE * color_space);
GST_D3D11_API
gboolean gst_video_info_apply_dxgi_color_space (DXGI_COLOR_SPACE_TYPE color_space,
GstVideoInfo * info);
d3d11: Refactor d3d11 memory and dxgi format usage * Create staging texture only when the CPU access is requested. Note that we should avoid the CPU access to d3d11 memory as mush as possible. Incoming d3d11upload and d3d11download will take this GPU memory upload/download. * Upload/Download texture memory from/to staging only if it needed, similar to GstGL PBO implementation. * Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder). Because I420_* formats are not supported formats by dxgi, each plane should be handled likewise GstGL separately, but NV12/P10 formats might be supported ones. So we decide the number of d3d11memory per GstBuffer for video memory depending on OS version and dxgi format. For instance, if NV12 is supported by OS, only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit. One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes. Such behavior will be controlled via configuration of GstD3D11BufferPool and default configuration is separate resources per plane.
2019-11-06 10:37:33 +00:00
G_END_DECLS
Reference in a new issue Copy permalink