Commit graph

454 commits

Author SHA1 Message Date
Seungha Yang 842805d7c6 av1decoder: Fix DPB size signalling
Take current frame into account for the DPB size signalling

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4845>
2023-06-14 15:16:04 +00:00
Seungha Yang 77b9e1351a mpeg2decoder: Fix DPB size signalling
Take current frame into account for the DPB size signalling

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4845>
2023-06-14 15:16:04 +00:00
Seungha Yang 965e566156 vp9decoder: Fix DPB size signalling
Take current frame into account for the DPB size signalling

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4845>
2023-06-14 15:16:04 +00:00
Seungha Yang 7d2beab9b1 vp8decoder: Fix DPB size signalling
Take current frame into account for the DPB size signalling

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4845>
2023-06-14 15:16:04 +00:00
Seungha Yang a88d90b777 d3d11: Log device removed reason
... and live objects. It could be useful hint for GPU debugging

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4170>
2023-06-14 06:23:12 +00:00
Víctor Manuel Jáquez Leal 06a2b2c7d9 va: dmabuf allocator to use GstVideoInfoDmaDrm
Change the internal GstVideoInfo structure in the GstVaDmabufAllocator to
GstVideoInfoDmaDrm in order to keep track of the exported DRM format by the
driver, and thus removing the DRMModifier quark attached as qdata in the
GstMemory. Though, the exposed API isn't updated yet; that has to go in a
second iteration.

Also this patch clean up some code (remove an unused buffer size assignation)
and fix some typos in documentation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4821>
2023-06-14 01:08:37 +00:00
He Junyan e00e64116a va: Add a helper function to detect the surface modifier
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4821>
2023-06-14 01:08:37 +00:00
He Junyan a01f2f0206 va: Lower the message level for va_export_surface_to_dmabuf()
Some surface formats such as GST_VIDEO_FORMAT_Y42B and GST_VIDEO_FORMAT_RGB
can be created but can not be exported as DMA buffer. You can not say that
this is a driver bug because the driver may never want to share this kind of
surface out of libva.
And this function will be used to detect modifiers later, so the error message
will be annoying.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4821>
2023-06-14 01:08:37 +00:00
He Junyan 7166fd3863 va: Extend the va_create_surfaces() function to accept modifiers
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4821>
2023-06-14 01:08:37 +00:00
He Junyan 994ab957c1 va: Add a helper function to convert video drm info to video info
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4821>
2023-06-14 01:08:36 +00:00
He Junyan 547f3e8622 va: Map drm fourcc into va fourcc in video format
The fourcc defined in va.h and drm_fourcc.h sometimes is not identical.
For example, the I420 format is defined as "I420" in va.h but defined
as "YU12" in drm_fourcc.h.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4821>
2023-06-14 01:08:36 +00:00
Seungha Yang e422a991f3 h265decoder: Calculate DPB size based on level
As specified in "A.4 Tiers and levels", calculate DPB size based
on level in order to avoid over preallocated DPB

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4840>
2023-06-13 02:57:06 +00:00
Seungha Yang 15735a4ef3 d3d11converter: Add support for premultiplied alpha
Adding "src-alpha-mode" and "dest-alpha-mode" properties
and performs premultiplied alpha conversion if needed

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4825>
2023-06-10 18:12:58 +00:00
Seungha Yang 1aa9e74aaf cudadownload: Always download CUDA memory if it's bound to decoder
Decoder bounded CUDA memory is allocated by driver and the pool size
is fixed. Since we don't know how many buffers would be held by
downstream non-CUDA element, we should download such CUDA memory
and release it back to decoder.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4810>
2023-06-08 22:27:06 +00:00
Víctor Manuel Jáquez Leal 53860147c6 va: bump libva version to 1.12
Remove all checks in gstva for all version lower than 1.12

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4781>
2023-06-08 08:50:04 +00:00
Seungha Yang 1e5343f0c6 d3d11memory: Add support for resource sharing via NT handle
Adding gst_d3d11_memory_get_nt_handle() method so that GstD3D11Memory
can hold shareable NT handle and for easy reuse of it

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4774>
2023-06-06 17:54:44 +00:00
Víctor Manuel Jáquez Leal e074cb55f7 va: remove unused parameters in internal function
gst_va_buffer_surface_new() don't use neither format, width nor height.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4768>
2023-06-04 19:16:53 +02:00
Víctor Manuel Jáquez Leal 06d793033d vulkan/format: use basic features (v1) as fallback
if vkGetPhysicalDeviceFormatProperties2 isn't available.

This approach assumes that, for basic features flags symbols,
VkFormatFeatureFlagBits and VkFormatFeatureFlagBits2 are binary equivalent.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4746>
2023-06-01 13:58:47 +00:00
Matthew Waters 0a74a967e3 vulkan/format: don't rely on vulkan header version for function availability
e.g. android will have update vulkan.h headers but old platforms will
not contain the function and will result in a link error.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4746>
2023-06-01 13:58:47 +00:00
Mathieu Duponchelle 211928ca41 webrtc/nice: put usage of OPTION_CONSENT_FRESHNESS behind check
See https://gitlab.freedesktop.org/libnice/libnice/-/merge_requests/257

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4750>
2023-06-01 00:42:28 +00:00
Colin Kinloch 82c449ce00 waylandsink: Emit "map" signal boarder surface is ready
This allows gtkwaylandsink to queue a draw of its gtk widget at the
correct time, avoiding a race.

Signed-off-by: Colin Kinloch <colin.kinloch@collabora.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4722>
2023-05-31 18:57:56 +00:00
Víctor Manuel Jáquez Leal 8b75e5fb57 vkvideoutils: fix code-style
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4716>
2023-05-26 10:08:27 +02:00
Víctor Manuel Jáquez Leal 365454790a vkphysicaldevice: fetch queue family query result support
And merged in a single structure the queue family operations report.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4716>
2023-05-26 10:08:27 +02:00
Víctor Manuel Jáquez Leal e2a40c58bf vkimagebufferpool: chain the correct structure
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4716>
2023-05-26 09:45:07 +02:00
Víctor Manuel Jáquez Leal e9414b9582 vkvideoutils: fix bug for chroma string comparison
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4716>
2023-05-26 09:45:07 +02:00
Víctor Manuel Jáquez Leal 8670a023f1 vkimagebufferpool: prepare new allocated buffers
This means change initial memory undefined layout and access to the ones
required for their usage.

This is only used if VK_KHR_SYNCHRONIZATION_2_EXTENSION_NAME is available.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:30 +00:00
Víctor Manuel Jáquez Leal e177080bec vulkan: number of memories in buffer rather than number of planes
New vulkan formats don't match the number of planes with the number of memories
attached to the buffer. This patch changes the pattern of using planes for
traverse the memories with the number of attached memories.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 565fe3291e vkutils: add gst_vulkan_get_or_create_image_view_with_info()
It's a generalization of the original gst_vulkan_get_or_create_image_view().

The reason for passing the whole VkImageViewCreateInfo structure rather than
just the missing fields, is because VkImageSubresourceRange and
VkComponentMapping can be different and those are most of VkImageViewCreateInfo.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal bb29b4dfec vkdevice: enable device optional extensions
Enable before-hand what the user might use.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal d7bbcffe14 vkformat: add VK_FORMAT_G8_B8R8_2PLANE_420_UNORM
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal b1321f78d3 vkswapper: use gst_vulkan_format_to_video_format()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal d84926b60f vkformat: add gst_vulkan_format_to_video_format()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal ce7256e0d3 vulkan: use gst_vulkan_format_get_aspect()
In order to use it, without depending in a previous calling of
_create_info_from_args(), VkFormat as input parameter to
gst_vulkan_image_memory_init() was added.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal b8e908131d vkformat: add gst_vulkan_format_get_aspect()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 26d00ba0cc vkimagebufferpool: fail if image cannot be allocated
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal d91c17264c vkimagebufferpool: gst_vulkan_image_buffer_pool_config_set_decode_caps()
This is going to be used when the pool is used by a video decoder for
VK_IMAGE_USAGE_VIDEO_DECODE_DST_BIT_KHR, since the frame allocation needs the
VkVideoProfileInfoKHR, and for that here GstCaps is used to wire it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 07271fc648 vkvideoutils: add gst_vulkan_video_profile_{to/from}_caps()
Add this new source file with utils for video profile mapping with GstCaps.
These method is used to pass this information along GStreamer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal c7c5dedd7f vkimagebufferpool: use gst_vulkan_image_memory_alloc_with_image_info()
As the pool will be only used for images with usage
VK_IMAGE_LAYOUT_VIDEO_DECODE_DST_KHR only one layer has to be allocated.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 83e73a2f59 vulkan: find memory type index with requirements
The specification says:

VUID-vkAllocateMemory-pAllocateInfo-01713

must pAllocateInfo->allocationSize be less than or equal to
VkPhysicalDeviceMemoryProperties::memoryHeaps[memindex].size where memindex =
VkPhysicalDeviceMemoryProperties::memoryTypes[pAllocateInfo->memoryTypeIndex].heapIndex
as returned by vkGetPhysicalDeviceMemoryProperties for the VkPhysicalDevice that
device was created from.

Though this can be catch by the validation layer, the requested frame size
depends on the use case so it's better to check this restriction by our code.

This patch also makes use of this new function to find memory type index,
 and removes the unused function to find memory type index, which, as GstVulkan is
considered unstable, we can do it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 4dd0fd6797 vkimagebufferpool: use gst_vulkan_format_from_video_info_2()
To get the number of images and its Vulkan formats for the GStreamer format.
Also it gets the basic image usage by default if it's not defined. It fails if the
video format is not supported.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 35dbd9f92c vkformat: add gst_vulkan_format_from_video_info_2()
The purpose of this function is to get more info about the mapped Vulkan format
from the GStreamer format, since they can be multiple Vulkan formats for one
GStreamer format.

Also a Vulkan format may have certain usage and aspects that must be verified.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 14dd6aac16 vkformat: move gst_vulkan_format_from_video_info()
Moved gst_vulkan_format_from_video_info() from gstvkimagememory to gstvkformat
in order to centralize the video format operations in a single source file.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal b1c166b53c vkimagebufferpool: pass usage and memory properties
Pass, in config structure, usage and memory properties, assigning proper
defaults if they aren't set. This parameters are used in image memory
allocation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 7b62c26139 vkdevice: add gst_vulkan_device_queue_family_indices()
This method will return a GArray with all the queue family indices created by
the device when it's opened. This array will be used by VkImageCreateInfo to
allocate a new Vulkan image.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal ea2bd1882f vkdevice: add programming parameters verification
To `gst_vulkan_device_foreach_queue()`.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 1e2ff519c6 vkbuffermemory: add gst_vulkan_buffer_memory_alloc_with_buffer_info()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 64ba7fb2eb vkimagememory: gst_vulkan_image_memory_alloc_with_image_info()
This new method instead of passing a bunch of variables which will fill
`VkImageCreateInfo`, the method accepts this whole structure, so the caller can
customize it as much as they want.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 5f5249cf73 vkimagememory: look up for Vulkan format using a static table
Replace the switch structure with explicit and extendable static table.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 1458c31031 vkdevice: enable multiple queues per device
Originally the opened device only created one queue of one family queue, to say
graphics one. This approach felt short when other queue family is required not
shared with the graphics queue family, for example video decoding.

This new approach proposes to create those queues with supported families. For
now, only video decoding and encoder are created, if they are available.

In order to hold multiple queues opened, an array of VkDeviceQueueCreateInfo is
held along the live the device object, because it's used to traverse or get the
opened queues.

The algorithm to choose which queues create (or open) is to look for the queue
with more family bits, which also supports the one we are requesting, thus
minimizing the number of global queues of a certain family to create.
Nonetheless, the number of queues to open per family is set to be all of them,
widening the possibility of parallelism.

Also, this commit do a cosmetic refactor the assigning the physical device
nearer where it's used.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00
Víctor Manuel Jáquez Leal 25140499fc vkdevice: enable features from physical device
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4351>
2023-05-19 04:26:29 +00:00