gstreamer/subprojects/gst-plugins-bad/sys/qsv/plugin.cpp

290 lines
8.3 KiB
C++
Raw Normal View History

qsv: Introduce H.264 Intel Quick Sync Video Encoder A new implementation of Intel Quick Sync Video plugin. This plugin supports both Windows and Linux but optimization for VA/DMABuf is not implemented yet. This new plugin has some notable differences compared with existing MSDK plugin. * Encoder will expose formats which can be natively supported without internal conversion. This will make encoder control/negotiation flow much simpler and cleaner than that of MSDK plugin. * This plugin includes QSV specific library loading helper, called dispatcher, with QSV SDK headers as a part of this plugin. So, there will be no more SDK version dependent #ifdef in the code and also there will be no more build-time MSDK/oneVPL SDK dependency. * Memory allocator interop between GStreamer and QSV is re-designed and decoupled. Instead of implementing QSV specific allocator/bufferpool, this plugin will make use of generic GStreamer memory allocator/bufferpool (e.g., GstD3D11Allocator and GstD3D11BufferPool). Specifically, GstQsvAllocator object will help interop between GstMemory and mfxFrameAllocator memory abstraction layers. Note that because of the design decision, VA/DMABuf support is not made as a part of this initial commit. We can add the optimization for Linux later once GstVA library exposes allocator/bufferpool implementation as an API like GstD3D11. * Initial encoder implementation supports interop with GstD3D11 infrastructure, including zero-copy encoding with upstream D3D11 element. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1408>
2022-02-06 12:34:43 +00:00
/* GStreamer
* Copyright (C) 2021 Seungha Yang <seungha@centricular.com>
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Library General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Library General Public License for more details.
*
* You should have received a copy of the GNU Library General Public
* License along with this library; if not, write to the
* Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
* Boston, MA 02110-1301, USA.
*/
#ifdef HAVE_CONFIG_H
#include "config.h"
#endif
#include <gst/gst.h>
#include <mfx.h>
#include "gstqsvutils.h"
#include "gstqsvh264dec.h"
qsv: Introduce H.264 Intel Quick Sync Video Encoder A new implementation of Intel Quick Sync Video plugin. This plugin supports both Windows and Linux but optimization for VA/DMABuf is not implemented yet. This new plugin has some notable differences compared with existing MSDK plugin. * Encoder will expose formats which can be natively supported without internal conversion. This will make encoder control/negotiation flow much simpler and cleaner than that of MSDK plugin. * This plugin includes QSV specific library loading helper, called dispatcher, with QSV SDK headers as a part of this plugin. So, there will be no more SDK version dependent #ifdef in the code and also there will be no more build-time MSDK/oneVPL SDK dependency. * Memory allocator interop between GStreamer and QSV is re-designed and decoupled. Instead of implementing QSV specific allocator/bufferpool, this plugin will make use of generic GStreamer memory allocator/bufferpool (e.g., GstD3D11Allocator and GstD3D11BufferPool). Specifically, GstQsvAllocator object will help interop between GstMemory and mfxFrameAllocator memory abstraction layers. Note that because of the design decision, VA/DMABuf support is not made as a part of this initial commit. We can add the optimization for Linux later once GstVA library exposes allocator/bufferpool implementation as an API like GstD3D11. * Initial encoder implementation supports interop with GstD3D11 infrastructure, including zero-copy encoding with upstream D3D11 element. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1408>
2022-02-06 12:34:43 +00:00
#include "gstqsvh264enc.h"
#include "gstqsvh265enc.h"
#include "gstqsvjpegenc.h"
#include "gstqsvvp9enc.h"
qsv: Introduce H.264 Intel Quick Sync Video Encoder A new implementation of Intel Quick Sync Video plugin. This plugin supports both Windows and Linux but optimization for VA/DMABuf is not implemented yet. This new plugin has some notable differences compared with existing MSDK plugin. * Encoder will expose formats which can be natively supported without internal conversion. This will make encoder control/negotiation flow much simpler and cleaner than that of MSDK plugin. * This plugin includes QSV specific library loading helper, called dispatcher, with QSV SDK headers as a part of this plugin. So, there will be no more SDK version dependent #ifdef in the code and also there will be no more build-time MSDK/oneVPL SDK dependency. * Memory allocator interop between GStreamer and QSV is re-designed and decoupled. Instead of implementing QSV specific allocator/bufferpool, this plugin will make use of generic GStreamer memory allocator/bufferpool (e.g., GstD3D11Allocator and GstD3D11BufferPool). Specifically, GstQsvAllocator object will help interop between GstMemory and mfxFrameAllocator memory abstraction layers. Note that because of the design decision, VA/DMABuf support is not made as a part of this initial commit. We can add the optimization for Linux later once GstVA library exposes allocator/bufferpool implementation as an API like GstD3D11. * Initial encoder implementation supports interop with GstD3D11 infrastructure, including zero-copy encoding with upstream D3D11 element. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1408>
2022-02-06 12:34:43 +00:00
#include <string.h>
#ifdef G_OS_WIN32
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <versionhelpers.h>
#include <gst/d3d11/gstd3d11.h>
#else
#include <gst/va/gstvadisplay.h>
#include <gst/va/gstvadisplay_drm.h>
#endif
GST_DEBUG_CATEGORY (gst_qsv_debug);
GST_DEBUG_CATEGORY (gst_qsv_allocator_debug);
GST_DEBUG_CATEGORY (gst_qsv_decoder_debug);
qsv: Introduce H.264 Intel Quick Sync Video Encoder A new implementation of Intel Quick Sync Video plugin. This plugin supports both Windows and Linux but optimization for VA/DMABuf is not implemented yet. This new plugin has some notable differences compared with existing MSDK plugin. * Encoder will expose formats which can be natively supported without internal conversion. This will make encoder control/negotiation flow much simpler and cleaner than that of MSDK plugin. * This plugin includes QSV specific library loading helper, called dispatcher, with QSV SDK headers as a part of this plugin. So, there will be no more SDK version dependent #ifdef in the code and also there will be no more build-time MSDK/oneVPL SDK dependency. * Memory allocator interop between GStreamer and QSV is re-designed and decoupled. Instead of implementing QSV specific allocator/bufferpool, this plugin will make use of generic GStreamer memory allocator/bufferpool (e.g., GstD3D11Allocator and GstD3D11BufferPool). Specifically, GstQsvAllocator object will help interop between GstMemory and mfxFrameAllocator memory abstraction layers. Note that because of the design decision, VA/DMABuf support is not made as a part of this initial commit. We can add the optimization for Linux later once GstVA library exposes allocator/bufferpool implementation as an API like GstD3D11. * Initial encoder implementation supports interop with GstD3D11 infrastructure, including zero-copy encoding with upstream D3D11 element. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1408>
2022-02-06 12:34:43 +00:00
GST_DEBUG_CATEGORY (gst_qsv_encoder_debug);
GST_DEBUG_CATEGORY (gst_qsv_h264_dec_debug);
qsv: Introduce H.264 Intel Quick Sync Video Encoder A new implementation of Intel Quick Sync Video plugin. This plugin supports both Windows and Linux but optimization for VA/DMABuf is not implemented yet. This new plugin has some notable differences compared with existing MSDK plugin. * Encoder will expose formats which can be natively supported without internal conversion. This will make encoder control/negotiation flow much simpler and cleaner than that of MSDK plugin. * This plugin includes QSV specific library loading helper, called dispatcher, with QSV SDK headers as a part of this plugin. So, there will be no more SDK version dependent #ifdef in the code and also there will be no more build-time MSDK/oneVPL SDK dependency. * Memory allocator interop between GStreamer and QSV is re-designed and decoupled. Instead of implementing QSV specific allocator/bufferpool, this plugin will make use of generic GStreamer memory allocator/bufferpool (e.g., GstD3D11Allocator and GstD3D11BufferPool). Specifically, GstQsvAllocator object will help interop between GstMemory and mfxFrameAllocator memory abstraction layers. Note that because of the design decision, VA/DMABuf support is not made as a part of this initial commit. We can add the optimization for Linux later once GstVA library exposes allocator/bufferpool implementation as an API like GstD3D11. * Initial encoder implementation supports interop with GstD3D11 infrastructure, including zero-copy encoding with upstream D3D11 element. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1408>
2022-02-06 12:34:43 +00:00
GST_DEBUG_CATEGORY (gst_qsv_h264_enc_debug);
GST_DEBUG_CATEGORY (gst_qsv_h265_enc_debug);
GST_DEBUG_CATEGORY (gst_qsv_jpeg_enc_debug);
GST_DEBUG_CATEGORY (gst_qsv_vp9_enc_debug);
qsv: Introduce H.264 Intel Quick Sync Video Encoder A new implementation of Intel Quick Sync Video plugin. This plugin supports both Windows and Linux but optimization for VA/DMABuf is not implemented yet. This new plugin has some notable differences compared with existing MSDK plugin. * Encoder will expose formats which can be natively supported without internal conversion. This will make encoder control/negotiation flow much simpler and cleaner than that of MSDK plugin. * This plugin includes QSV specific library loading helper, called dispatcher, with QSV SDK headers as a part of this plugin. So, there will be no more SDK version dependent #ifdef in the code and also there will be no more build-time MSDK/oneVPL SDK dependency. * Memory allocator interop between GStreamer and QSV is re-designed and decoupled. Instead of implementing QSV specific allocator/bufferpool, this plugin will make use of generic GStreamer memory allocator/bufferpool (e.g., GstD3D11Allocator and GstD3D11BufferPool). Specifically, GstQsvAllocator object will help interop between GstMemory and mfxFrameAllocator memory abstraction layers. Note that because of the design decision, VA/DMABuf support is not made as a part of this initial commit. We can add the optimization for Linux later once GstVA library exposes allocator/bufferpool implementation as an API like GstD3D11. * Initial encoder implementation supports interop with GstD3D11 infrastructure, including zero-copy encoding with upstream D3D11 element. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1408>
2022-02-06 12:34:43 +00:00
#define GST_CAT_DEFAULT gst_qsv_debug
#ifdef G_OS_WIN32
#define MFX_ACCEL_MODE MFX_ACCEL_MODE_VIA_D3D11
#else
#define MFX_ACCEL_MODE MFX_ACCEL_MODE_VIA_VAAPI
#endif
#ifdef G_OS_WIN32
static mfxSession
create_session_with_platform_device (mfxLoader loader,
mfxImplDescription * desc, guint impl_index, GstObject ** d3d11_device,
GList ** devices)
{
mfxSession session = nullptr;
mfxStatus status;
GstD3D11Device *selected = nullptr;
GList *list = *devices;
GList *iter;
mfxU16 device_id = 0;
*d3d11_device = nullptr;
status = MFXCreateSession (loader, impl_index, &session);
if (status != MFX_ERR_NONE) {
GST_WARNING ("Failed to create session with index %d, %d (%s)",
impl_index, QSV_STATUS_ARGS (status));
return nullptr;
}
if (desc->ApiVersion.Major >= 2 ||
(desc->ApiVersion.Major == 1 && desc->ApiVersion.Minor >= 19)) {
mfxPlatform platform;
memset (&platform, 0, sizeof (mfxPlatform));
if (MFXVideoCORE_QueryPlatform (session, &platform) == MFX_ERR_NONE) {
device_id = platform.DeviceId;
/* XXX: re-create session, MFXVideoCORE_QueryPlatform() may cause
* later MFXVideoCORE_SetHandle() call failed with
* MFX_ERR_UNDEFINED_BEHAVIOR error */
g_clear_pointer (&session, MFXClose);
status = MFXCreateSession (loader, impl_index, &session);
if (status != MFX_ERR_NONE) {
GST_WARNING ("Failed to re-create session with index %d, %d (%s)",
impl_index, QSV_STATUS_ARGS (status));
return nullptr;
}
}
}
if (device_id) {
for (iter = list; iter; iter = g_list_next (iter)) {
GstD3D11Device *dev = GST_D3D11_DEVICE (iter->data);
guint dev_id;
g_object_get (dev, "device-id", &dev_id, nullptr);
if (dev_id == (guint) device_id) {
selected = dev;
list = g_list_delete_link (list, iter);
break;
}
}
}
if (!selected) {
/* Unknown device id, pick the first device */
selected = GST_D3D11_DEVICE (list->data);
list = g_list_delete_link (list, list);
}
*devices = list;
status = MFXVideoCORE_SetHandle (session, MFX_HANDLE_D3D11_DEVICE,
gst_d3d11_device_get_device_handle (selected));
if (status != MFX_ERR_NONE) {
GST_WARNING ("Failed to set d3d11 device handle, %d (%s)",
QSV_STATUS_ARGS (status));
gst_object_unref (selected);
MFXClose (session);
return nullptr;
}
*d3d11_device = GST_OBJECT (selected);
return session;
}
#else
static mfxSession
create_session_with_platform_device (mfxLoader loader,
mfxImplDescription * desc, guint impl_index, GstObject ** va_display,
GList ** devices)
{
mfxSession session = nullptr;
mfxStatus status;
GstVaDisplay *selected;
GList *list = *devices;
*va_display = nullptr;
status = MFXCreateSession (loader, impl_index, &session);
if (status != MFX_ERR_NONE) {
GST_WARNING ("Failed to create session with index %d, %d (%s)",
impl_index, QSV_STATUS_ARGS (status));
return nullptr;
}
/* XXX: what's the relation between implementation index and VA display ?
* Pick the first available device for now */
selected = GST_VA_DISPLAY (list->data);
list = g_list_delete_link (list, list);
*devices = list;
status = MFXVideoCORE_SetHandle (session, MFX_HANDLE_VA_DISPLAY,
gst_va_display_get_va_dpy (selected));
if (status != MFX_ERR_NONE) {
GST_WARNING ("Failed to set display handle, %d (%s)",
QSV_STATUS_ARGS (status));
gst_object_unref (selected);
MFXClose (session);
return nullptr;
}
*va_display = GST_OBJECT (selected);
return session;
}
#endif
static void
plugin_deinit (gpointer data)
{
gst_qsv_deinit ();
}
static gboolean
plugin_init (GstPlugin * plugin)
{
mfxLoader loader;
guint i = 0;
GList *platform_devices = nullptr;
#ifdef G_OS_WIN32
/* D3D11 Video API is supported since Windows 8.
* Do we want to support old OS (Windows 7 for example) with D3D9 ?? */
if (!IsWindows8OrGreater ())
return TRUE;
#endif
GST_DEBUG_CATEGORY_INIT (gst_qsv_debug, "qsv", 0, "Intel Quick Sync Video");
loader = gst_qsv_get_loader ();
if (!loader)
return TRUE;
platform_devices = gst_qsv_get_platform_devices ();
if (!platform_devices) {
gst_qsv_deinit ();
return TRUE;
}
GST_INFO ("Found %d platform devices", g_list_length (platform_devices));
GST_DEBUG_CATEGORY_INIT (gst_qsv_allocator_debug,
"qsvallocator", 0, "qsvallocator");
GST_DEBUG_CATEGORY_INIT (gst_qsv_decoder_debug,
"qsvdecoder", 0, "qsvdecoder");
GST_DEBUG_CATEGORY_INIT (gst_qsv_encoder_debug,
"qsvencoder", 0, "qsvencoder");
GST_DEBUG_CATEGORY_INIT (gst_qsv_h264_dec_debug,
"qsvh264dec", 0, "qsvh264dec");
qsv: Introduce H.264 Intel Quick Sync Video Encoder A new implementation of Intel Quick Sync Video plugin. This plugin supports both Windows and Linux but optimization for VA/DMABuf is not implemented yet. This new plugin has some notable differences compared with existing MSDK plugin. * Encoder will expose formats which can be natively supported without internal conversion. This will make encoder control/negotiation flow much simpler and cleaner than that of MSDK plugin. * This plugin includes QSV specific library loading helper, called dispatcher, with QSV SDK headers as a part of this plugin. So, there will be no more SDK version dependent #ifdef in the code and also there will be no more build-time MSDK/oneVPL SDK dependency. * Memory allocator interop between GStreamer and QSV is re-designed and decoupled. Instead of implementing QSV specific allocator/bufferpool, this plugin will make use of generic GStreamer memory allocator/bufferpool (e.g., GstD3D11Allocator and GstD3D11BufferPool). Specifically, GstQsvAllocator object will help interop between GstMemory and mfxFrameAllocator memory abstraction layers. Note that because of the design decision, VA/DMABuf support is not made as a part of this initial commit. We can add the optimization for Linux later once GstVA library exposes allocator/bufferpool implementation as an API like GstD3D11. * Initial encoder implementation supports interop with GstD3D11 infrastructure, including zero-copy encoding with upstream D3D11 element. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1408>
2022-02-06 12:34:43 +00:00
GST_DEBUG_CATEGORY_INIT (gst_qsv_h264_enc_debug,
"qsvh264enc", 0, "qsvh264enc");
GST_DEBUG_CATEGORY_INIT (gst_qsv_h265_enc_debug,
"qsvh265enc", 0, "qsvh265enc");
GST_DEBUG_CATEGORY_INIT (gst_qsv_jpeg_enc_debug,
"qsvjpegenc", 0, "qsvjpegenc");
GST_DEBUG_CATEGORY_INIT (gst_qsv_vp9_enc_debug, "qsvvp9enc", 0, "qsvvp9enc");
qsv: Introduce H.264 Intel Quick Sync Video Encoder A new implementation of Intel Quick Sync Video plugin. This plugin supports both Windows and Linux but optimization for VA/DMABuf is not implemented yet. This new plugin has some notable differences compared with existing MSDK plugin. * Encoder will expose formats which can be natively supported without internal conversion. This will make encoder control/negotiation flow much simpler and cleaner than that of MSDK plugin. * This plugin includes QSV specific library loading helper, called dispatcher, with QSV SDK headers as a part of this plugin. So, there will be no more SDK version dependent #ifdef in the code and also there will be no more build-time MSDK/oneVPL SDK dependency. * Memory allocator interop between GStreamer and QSV is re-designed and decoupled. Instead of implementing QSV specific allocator/bufferpool, this plugin will make use of generic GStreamer memory allocator/bufferpool (e.g., GstD3D11Allocator and GstD3D11BufferPool). Specifically, GstQsvAllocator object will help interop between GstMemory and mfxFrameAllocator memory abstraction layers. Note that because of the design decision, VA/DMABuf support is not made as a part of this initial commit. We can add the optimization for Linux later once GstVA library exposes allocator/bufferpool implementation as an API like GstD3D11. * Initial encoder implementation supports interop with GstD3D11 infrastructure, including zero-copy encoding with upstream D3D11 element. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1408>
2022-02-06 12:34:43 +00:00
do {
mfxStatus status = MFX_ERR_NONE;
mfxSession session = nullptr;
mfxImplDescription *desc = nullptr;
GstObject *device = nullptr;
status = MFXEnumImplementations (loader,
i, MFX_IMPLCAPS_IMPLDESCSTRUCTURE, (mfxHDL *) & desc);
if (status != MFX_ERR_NONE)
break;
if ((desc->Impl & MFX_IMPL_TYPE_HARDWARE) == 0)
goto next;
if ((desc->AccelerationMode & MFX_ACCEL_MODE) == 0)
goto next;
session = create_session_with_platform_device (loader, desc, i, &device,
&platform_devices);
if (!session)
goto next;
gst_qsv_h264_dec_register (plugin, GST_RANK_MARGINAL, i, device, session);
qsv: Introduce H.264 Intel Quick Sync Video Encoder A new implementation of Intel Quick Sync Video plugin. This plugin supports both Windows and Linux but optimization for VA/DMABuf is not implemented yet. This new plugin has some notable differences compared with existing MSDK plugin. * Encoder will expose formats which can be natively supported without internal conversion. This will make encoder control/negotiation flow much simpler and cleaner than that of MSDK plugin. * This plugin includes QSV specific library loading helper, called dispatcher, with QSV SDK headers as a part of this plugin. So, there will be no more SDK version dependent #ifdef in the code and also there will be no more build-time MSDK/oneVPL SDK dependency. * Memory allocator interop between GStreamer and QSV is re-designed and decoupled. Instead of implementing QSV specific allocator/bufferpool, this plugin will make use of generic GStreamer memory allocator/bufferpool (e.g., GstD3D11Allocator and GstD3D11BufferPool). Specifically, GstQsvAllocator object will help interop between GstMemory and mfxFrameAllocator memory abstraction layers. Note that because of the design decision, VA/DMABuf support is not made as a part of this initial commit. We can add the optimization for Linux later once GstVA library exposes allocator/bufferpool implementation as an API like GstD3D11. * Initial encoder implementation supports interop with GstD3D11 infrastructure, including zero-copy encoding with upstream D3D11 element. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1408>
2022-02-06 12:34:43 +00:00
gst_qsv_h264_enc_register (plugin, GST_RANK_NONE, i, device, session);
gst_qsv_h265_enc_register (plugin, GST_RANK_NONE, i, device, session);
gst_qsv_jpeg_enc_register (plugin, GST_RANK_NONE, i, device, session);
gst_qsv_vp9_enc_register (plugin, GST_RANK_NONE, i, device, session);
qsv: Introduce H.264 Intel Quick Sync Video Encoder A new implementation of Intel Quick Sync Video plugin. This plugin supports both Windows and Linux but optimization for VA/DMABuf is not implemented yet. This new plugin has some notable differences compared with existing MSDK plugin. * Encoder will expose formats which can be natively supported without internal conversion. This will make encoder control/negotiation flow much simpler and cleaner than that of MSDK plugin. * This plugin includes QSV specific library loading helper, called dispatcher, with QSV SDK headers as a part of this plugin. So, there will be no more SDK version dependent #ifdef in the code and also there will be no more build-time MSDK/oneVPL SDK dependency. * Memory allocator interop between GStreamer and QSV is re-designed and decoupled. Instead of implementing QSV specific allocator/bufferpool, this plugin will make use of generic GStreamer memory allocator/bufferpool (e.g., GstD3D11Allocator and GstD3D11BufferPool). Specifically, GstQsvAllocator object will help interop between GstMemory and mfxFrameAllocator memory abstraction layers. Note that because of the design decision, VA/DMABuf support is not made as a part of this initial commit. We can add the optimization for Linux later once GstVA library exposes allocator/bufferpool implementation as an API like GstD3D11. * Initial encoder implementation supports interop with GstD3D11 infrastructure, including zero-copy encoding with upstream D3D11 element. Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1408>
2022-02-06 12:34:43 +00:00
next:
MFXDispReleaseImplDescription (loader, desc);
g_clear_pointer (&session, MFXClose);
gst_clear_object (&device);
i++;
/* What's the possible maximum number of impl/device ? */
} while (i < 16 && platform_devices != nullptr);
if (platform_devices)
g_list_free_full (platform_devices, (GDestroyNotify) gst_object_unref);
g_object_set_data_full (G_OBJECT (plugin), "plugin-qsv-shutdown",
(gpointer) "shutdown-data", (GDestroyNotify) plugin_deinit);
return TRUE;
}
GST_PLUGIN_DEFINE (GST_VERSION_MAJOR,
GST_VERSION_MINOR,
qsv,
"Intel Quick Sync Video plugin",
plugin_init, VERSION, "LGPL", GST_PACKAGE_NAME, GST_PACKAGE_ORIGIN)