Commit graph

181 commits

Author SHA1 Message Date
Víctor Manuel Jáquez Leal
bb9eb6f477 vkimagebufferpool: add encoding usage as video usage
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7247>
2024-07-29 17:10:15 +02:00
Víctor Manuel Jáquez Leal
541e2e93ca vkimagebufferpool: reset the number of profiles at set_config()
Virtual method set_config() can be called several times, and if the number of
profiles counter isn't reset the pool will reach an error state.

The purpose of number of profiles is to check the number of valid vulkan video
profiles (two in the case of transcoding use-case, for example) so it's local to
set_config() virtual method.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7247>
2024-07-29 17:10:14 +02:00
Víctor Manuel Jáquez Leal
d8e384085a vulkan: encoder and decoder runtime check for driver version 1.3.275
Which is the one checked in meson. See commit 21ee264d65

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7247>
2024-07-29 17:10:14 +02:00
Víctor Manuel Jáquez Leal
2387c3c4aa vkimagebufferpool: no aliased images for video decoding
This fix regression in validation layer introduced by commit 3a2e8d2d19

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7211>
2024-07-22 16:46:53 +00:00
Víctor Manuel Jáquez Leal
423f0d5384 vkdecoder: handle barrier internally for coincide references
This is to avoid a regression in validation layer (introduced by commit
916c4e70cd) when using vulkandownload

VUID-VkImageMemoryBarrier2-srcAccessMask-03914 ..  vkCmdPipelineBarrier2():
pDependencyInfo->pImageMemoryBarriers[1].srcAccessMask (VK_ACCESS_TRANSFER_READ_BIT)
is not supported by stage mask (VK_PIPELINE_STAGE_2_VIDEO_DECODE_BIT_KHR)

since vulkandownload set DPB memories' access mask to
VK_ACCESS_TRANSFER_READ_BIT, while they are retain by the DPB queue, so when
they are used as DPB after been shown, this validation error is raised.

Must of the barrier values are set ignoring the previous state of the vulkan
images.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7211>
2024-07-22 16:46:53 +00:00
Víctor Manuel Jáquez Leal
035e0698cf vkoperation: fix documentation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7203>
2024-07-22 10:49:33 +02:00
Víctor Manuel Jáquez Leal
e91ffae1f0 vkimagebufferpool: reset buffer's access flags
The access flags are kept around the operations, but when the buffer is
released, the access flag should be reset to its original value, since queue
transfers can be done along the pipeline and, when reusing the buffer, the new
queue might not support the latest access flag.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7165>
2024-07-19 14:45:39 +02:00
Víctor Manuel Jáquez Leal
2990cc5f71 vulkan: add source pipeline stage to _operation_add_frame_barrier()
Instead of dragging the last destination pipeline stage as current barrier
source pipeline stage (which isn't a valid semantic) this patch adds a parameter
to gst_vulkan_operation_add_frame_barrier() to set the source pipeline stage to
define the barrier.

The previous logic brought problems particularly with queue transfers, when the
new queue doesn't support the stage set during a previous operation in a
different queue.

Now the operation API is closer to Vulkan semantics.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7165>
2024-07-19 14:45:39 +02:00
Víctor Manuel Jáquez Leal
c7259f3a5b vulkan: remove beta extensions guard for encode operations
This is not needed anymore since encoder operations are not beta anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7179>
2024-07-16 15:08:39 +00:00
Víctor Manuel Jáquez Leal
3a2e8d2d19 vkimagebufferpool: set image creation flags in needed
set ALIAS bit if the usage is for both sampled and storage.

set MUTABLE_FORMAT and EXTENDED_USAGE bits if the image is a multiplane YUV and
uses multiple memories.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6798>
2024-07-11 20:28:04 +02:00
Víctor Manuel Jáquez Leal
0f8f2d0057 vkformat: add gst_vulkan_format_get_map function
This will be used later to compare the format selected by
gst_vulkan_format_from_video_info_2(), to verify if it's multiple memory buffer
or not.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6798>
2024-07-11 20:28:04 +02:00
Víctor Manuel Jáquez Leal
2d17bba6f1 vkimagebufferpool: define a default usage
Define a default usage and use it instead of repeating the same bitwise
addition.

Therefore, when usage is defined as zero, the usage is defined with the
format's supported usage and the default usage, now without the storage
bit, but with color and input attachment bits.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6798>
2024-07-11 10:52:19 +02:00
Víctor Manuel Jáquez Leal
b1eebec485 vkformat: unward G8_B8R8_2PLANE_420_UNORM
Since it exists since VK_VERSION_1_1. It should be ignored by usage flags or the
no_multiplane parameter.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6798>
2024-07-11 10:52:19 +02:00
Nirbheek Chauhan
8619e163bf vk-video: Fix uint64_t string format errors
With clang on macOS:

```
error: format specifies type 'long' but the argument has type 'uint64_t' (aka 'unsigned long long')
...
error: format specifies type 'unsigned long' but the argument has type 'VkImageView' (aka 'unsigned long long')
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
2024-07-07 13:20:12 +00:00
Nirbheek Chauhan
156a016313 meson: Find MoltenVK with the objc++ compiler everywhere
When building for iOS in Cerbero, as of MoltenVK SDK 1.3.283, we have
to statically link to libMoltenVK since it no longer ships a dylib.
This requires linking to libc++, so we find the dep with the objc++
compiler to ensure that meson uses the right linker.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
2024-07-07 13:20:12 +00:00
Nirbheek Chauhan
5ed3e045d8 meson: Fix vulkan library build on iOS
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
2024-07-07 13:20:12 +00:00
Nirbheek Chauhan
5c41d387e3 meson: Use / instead of join_paths for vulkan
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
2024-07-07 13:20:12 +00:00
Nirbheek Chauhan
54a6643986 meson: Fix automagic dependency checks in gstvulkan
Windowing, in particular, was getting silently disabled.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
2024-07-07 13:20:12 +00:00
Stéphane Cerveau
a4c976dd20 vkdecoder: support layered and non dedicated DPB
As NVIDIA Amperium. In this case the each output buffer is also a DPB,
but using a different view layer.

Still pending a validation layer issue:

VUID-VkVideoBeginCodingInfoKHR-flags-07244

Co-authored-by: Victor Jaquez <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6954>
2024-06-19 10:37:32 +00:00
Stéphane Cerveau
f40d947c4a vkoperation: support for query_result_status
query_result_status can be optional so we should not create
the query pool if the queue does not support it,
ie, AMD does not support VK_QUERY_TYPE_RESULT_STATUS_ONLY_KHR

In other use case such as VK_QUERY_TYPE_VIDEO_ENCODE_FEEDBACK_KHR, the
query pool must be created.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7043>
2024-06-18 12:13:41 +00:00
Stéphane Cerveau
71ebb20267 vkphysicaldevice: rename query to query_result_status
As only queryResultStatusSupport can be optional,
the variable name should be more specific.

queryResultStatusSupport reports VK_TRUE if query type
VK_QUERY_TYPE_RESULT_STATUS_ONLY_KHR and use of
VK_QUERY_RESULT_WITH_STATUS_BIT_KHR are supported.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7043>
2024-06-18 12:13:41 +00:00
Matthew Waters
050c622b7f vulkan/swapper: expose choose_queue() in docs
It was missing a doc trigraph.

Also mark input queue argument as nullable.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7023>
2024-06-12 23:37:58 +10:00
Stéphane Cerveau
fe737fc274 vulkan: fix macos build
The VulkanSDK can be downloaded from LunarG website and can
be installed properly in /usr/local following:

https://vulkan.lunarg.com/doc/view/latest/mac/getting_started.html

Fixes partly #2372

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6669>
2024-06-07 03:16:26 +00:00
Matthew Waters
b6d03432b4 vulkan/fullscreenquad: add check for unset video info
So we don't crash when set_info() is not called.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7000>
2024-06-07 01:44:32 +00:00
Matthew Waters
e925f22f33 vulkan/fullscreenquad: allow setting NULL input/output buffer to unset
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7000>
2024-06-07 01:44:32 +00:00
Stéphane Cerveau
a17957a7c8 vulkan: remove remaining GST_VULKAN_HAVE_VIDEO_ENCODERS
Some define use have been forgotten in
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6992

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7001>
2024-06-06 10:32:51 +00:00
Stéphane Cerveau
21ee264d65 vulkan: remove GST_VULKAN_HAVE_VIDEO_ENCODERS
Use 2.3.275 as first supported SDK version

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6992>
2024-06-05 17:50:27 +00:00
Stéphane Cerveau
b9633cb766 vkqueue: remove useless decoder include
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6992>
2024-06-05 17:50:26 +00:00
Matthew Waters
013026b06a vulkan/videofilter: add getters for queue/device/instance
Allows bindings to not pke at structs for this information.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6987>
2024-06-03 00:06:18 +10:00
Matthew Waters
a7a70ca7b6 vulkan/fullscreenquad: add get_queue()
Allows bindings to not poke at the instance struct.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6987>
2024-06-03 00:06:18 +10:00
Matthew Waters
5e182c911c vulkan/fullscreenquad: mark set_info GstVideoInfo as const
It's not modified by the function.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6987>
2024-06-03 00:06:18 +10:00
Matthew Waters
346df4cb3f vulkan: support not having glslc available for building vulkan plugin
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6980>
2024-06-01 12:41:49 +00:00
Stéphane Cerveau
7bbf5fd801 vkutils: do not forget to clear context in case of error
The context is leaking in case of a failing instance open.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6975>
2024-05-31 14:44:09 +00:00
Stéphane Cerveau
c3f3fd7351 vkerror: free the error string after usage
g_set_error already used the var string, can clear it now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6975>
2024-05-31 14:44:09 +00:00
Víctor Manuel Jáquez Leal
e913b4870a vkformat: try UNORM format first and decouple them from colorimetry
From the spec (chapter 34, v1.3.283):

````
UNORM: the components are unsigned normalized values in the range [0, 1]

SRGB: the R, G and B components are unsigned normalized value that represent
      values using sRGB nonlinear encoding, while the A component (if one
      exists) is a regular unsigned normalized value
```

The difference is the storage encoding, the first one is aimed for image
transfers, while the second is for shaders, mostly in the swapchain stage in the
pipeline, and it's done automatically if needed [1].

As far as I have checked, other frameworks (FFmpeg, GTK+), when import or export
images from/to Vulkan, use exclusively UNORM formats, while SRGB formats are
ignored.

My conclusion is that Vulkan formats are related on how bits are stored in
memory rather their transfer functions (colorimetry).

This patch does two interrelated changes:

1. It swaps certain color format maps to try first, in both
gst_vulkan_format_from_video_info() and gst_vulkan_format_from_video_info_2(),
the UNORM formats, when comparing its usage, and later check for SRGB.

2. It removes the code that check for colorimetry in
gst_vulkan_format_from_video_info_2(), since it not storage related.

1. https://community.khronos.org/t/noob-difference-between-unorm-and-srgb/106132/7

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6797>
2024-05-23 17:41:30 +00:00
Stéphane Cerveau
73c64e8182 tests: add vulkan H.265 encode
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6676>
2024-05-03 19:40:17 +00:00
Stéphane Cerveau
5320514076 tests: add Vulkan H.264 encode
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6676>
2024-05-03 19:40:16 +00:00
Víctor Manuel Jáquez Leal
1b1593568f vkencoder: add gstvkencoder helper object
Add a gstvkencoder class to support Vulkan encoder such as H26X
formats.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6676>
2024-05-03 19:40:16 +00:00
Stéphane Cerveau
ddb9b9d9ca vkdecoder: change dstmask in decoder frame barrier
Use of VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT instead of
specific VK_PIPELINE_STAGE_2_VIDEO_DECODE_BIT_KHR

Fix for VUID-vkCmdPipelineBarrier2-srcStageMask-03849

pDependencyInfo->pImageMemoryBarriers[0].srcStageMask
(VK_PIPELINE_STAGE_2_VIDEO_DECODE_BIT_KHR) is not compatible with
the queue family properties
(VK_QUEUE_GRAPHICS_BIT|VK_QUEUE_COMPUTE_BIT|VK_QUEUE_TRANSFER_BIT|
VK_QUEUE_SPARSE_BINDING_BIT|VK_QUEUE_PROTECTED_BIT) of this
command buffer. The Vulkan spec states: The srcStageMask member
of any element of the pMemoryBarriers, pBufferMemoryBarriers, or
pImageMemoryBarriers members of pDependencyInfo must only
include pipeline stages valid for the queue family that was
used to create the command pool that commandBuffer was allocated
 from (
https://www.khronos.org/registry/vulkan/specs/1.3-extensions/
html/vkspec.html#VUID-vkCmdPipelineBarrier2-srcStageMask-03849)

The frame barrier should use a compatible srcStageMask for all
the queues.

Remove reset_pipeline_stage_mask as it is redundant

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6780>
2024-05-02 18:07:25 +00:00
Víctor Manuel Jáquez Leal
1f080391ed vulkan: replace gst_vulkan_queue_create_decoder() with gst_vulkan_decoder_new_from_queue()
The purpose of this refactor is to hide decoding code from public API.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6723>
2024-04-26 16:24:22 +00:00
Víctor Manuel Jáquez Leal
18c32272bd vulkan: conceal unused decoder symbols
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6723>
2024-04-26 16:24:22 +00:00
Víctor Manuel Jáquez Leal
668b395a38 vulkan: conceal decoder from public API
Since we don't want to expose video decoding API outside of GStreamer, the
header is removed from installation and both source files are renamed as
-private.

The header must remain in gst-libs because is referred by GstVulkanQueue,
which's the decoder factory.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6723>
2024-04-26 16:24:22 +00:00
Víctor Manuel Jáquez Leal
d2c8593b2e vkswapper: choose color space according with format
The swapper surfaces contains the color space for each supported format. Instead
of hard coding the color space, it returns the value associated with the
negotiated vulkan format.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6725>
2024-04-24 13:53:18 +00:00
Chris Spencer
c6a3b2ab85 vkbufferpool: correct usage flags type
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6514>
2024-04-03 09:36:44 +00:00
Víctor Manuel Jáquez Leal
b18483a563 vulkan: sink references at instantiation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6398>
2024-04-03 08:48:25 +00:00
Chris Spencer
1032d58187 vkmemory: invalidate non-coherent memory when mapping for read
Mapping non-coherent memory does not implicitly invalidate the host caches.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6310>
2024-03-11 01:55:44 +00:00
Chris Spencer
9412565221 vulkan/operation: use timeline semaphore fallback if sync2 not supported
gst_vulkan_operation_add_dependency_frame does not fall back to the
timeline semaphore implementation if VK_KHR_synchronization2 is compiled
in, but not supported by the driver.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6309>
2024-03-11 00:58:40 +00:00
Chris Spencer
7701e9ffeb vulkan/operation: add missing unlock
gst_vulkan_operation_add_dependency_frame does not release its lock if
support for VK_KHR_timeline_semaphore/VK_KHR_synchronization2 is compiled
in, but not supported by the driver.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6309>
2024-03-11 00:58:40 +00:00
Víctor Manuel Jáquez Leal
2011dec249 vulkan/decoder: fix documentation generation
GstVulkanDecoderParameters is an union, not a struct, and that isn't handled by
hotdoc.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6089>
2024-02-10 17:17:23 +00:00
Víctor Manuel Jáquez Leal
f0c5a853a6 vulkan/decoder: don't initialize function table once
Since it has to be associated with the device and it gets destroyed when the
decoder is freed.

Now it's created when the decoder starts and it's flagged.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6089>
2024-02-10 17:17:23 +00:00