We did several things to enable the new memory logic in msdkdec:
(1) We always use video memory for decoder in linux path;
(2) We give negotiated pool to alloc_pool stored in GstMsdkContext which
will be used in callback mfxFrameAllocator:Alloc to alloc surfaces as
MediaSDK needs, and this pool is also available for decoder itself;
(3) We modify decide_allocation process, that is we make pool negotiaion
before gst_msdk_init_decoder to ensure the pool is decided and ready for
use in mfxFrameAllocator:Alloc callback; then we will consider the case
when we need to do the gpu to cpu copy.
(4) In gst_msdkdec_finish_task, we modify the way for copy following the
logic in (3).
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3439>
Add a pool creation function name as 2 for later use which will create
va pool for video memory in linux and keep system pool for windows.
This gst_msdkdec_create_buffer_pool2 will replace gst_msdkdec_create_buffer_pool
when all the memory allocation modifications are ready in the commits after.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3439>
Rewrite gst_msdk_frame_alloc and name it as xxx_2 before applying it.
It uses negotiated bufferpool stored in GstMsdkContext to allocate buffers
in the callback MfxFrameAllocator:Alloc, then extract VASurface from buffer,
wrap it as mfxMemIDs and pass these IDs to MediaSDK/oneVPL.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3439>
Replace video_copy with memcpy to fix the issue when the sizes of the
src frame and dst frame don't match. Moreover, for Windows, you have to
do the copy first and call gst_msdk_import_to_msdk_surface later.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3231>
Replace video_copy with memcpy to fix the issue when the sizes of the
src frame and dst frame don't match. Moreover, for Windows, you have to
do the copy first and call gst_msdk_import_to_msdk_surface later.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3231>
Currently MSDK context does not support d3d11va. Now introduce d3d11va
device to MSDK context, making it able to create msdk session with d3d11
device and to easily share with upstream and donwstream.
Add environment variable to enable user to choose GPU device in multi-GPU
environment. This variable is only valid when there's no context
returned by upstream or downstream. Otherwise it will use the device
that created by upstream or downstream.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3231>
We use va pool as msdkvpp's bufferpool, which means both va memory
and dma memory will be allocated by va pool. Considering drm modifier
stuff is not ready, we use va memory with higher priortiry than
dma memory when deciding vpp caps.
Besides, this patch removes the specified "interlace-mode" in vpp caps.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3253>
If the buffer is not msdk_buffer, we can try to directly import the
attached memory (i.e. va mem and dmabuf mem) by applying the common
uitl function: import_to_msdk_function ().
Here add a flag "from_qdata" in GstMsdkSurface to handle the cropping case,
we should avoid updating the crop values when msdk_surface is from the
memory's qdata, because the crop info from this surface is the already
updated one.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2498>
When input buffer is of dmabuf memory but not a msdk buffer (i.e., the
allocator is not msdk_allocator), then we can try to get fd of this mem,
create the corresponding va surface and wrap it as mfx surface.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2498>
Msdkdec should use it own pool when the allocation from downstream query
is not any msdk_allocator (i.e. msdk_video_allocator,
msdk_dmabuf_allocator and msdk_system_allocator). Otherwise, when using
pipeline "msdkh264dec ! vah264enc !" to transcode a not 16-bit-aligned
stream (i.e. 1920x1080), the transcoding will fail due to the size
mismatch issue between decoder pool and encoder pool.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2451>
