Add Windows Graphics Capture (WGC) API based screen capture mode.
The conditions where this mode is used:
* Explicitly requested by user (capture-api property)
* To capture specific window
* When DXGI desktop duplication API does not work on hybrid graphics systems
(e.g., multi-gpu laptop)
Full features of this implementation require Windows 11. And Windows 11
SDK is required to build this feature.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3144>
Adding loopback capture mode for specified PID.
Note that this feature requires Windows 10 build 20348
(Windows 11/Windows Server 2022 or later),
and any process loopback related properties will not be exposed
if OS does not support it.
Example launch lines:
* wasapi2src loopback-mode=include-process-tree loopback-target-pid=<PID>
Captures audio generated by an application (specified by PID)
and its child process
* wasapi2src loopback-mode=exclude-process-tree loopback-target-pid=<PID>
Captures desktop audio excluding PID and its child process
Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1278
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3195>
The order of the devices iterator from the SDK is undefined and can
randomly change.
Keep the device-number property for backwards compatibility and
simplicity but prefer the persistent-id property and also use it for the
device provider implementation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3078>
Currently if the user is not able to access the devices under /dev/media*,
either due to no media devices present on the system or simply no permission
to access the device, v4l2codecs initialises with no features or debug messages.
Since calling `GST_DEBUG="v4l2*:7" gst-inspect-1.0 v4l2codecs` is a typical way
to diagnose why element(s) failed to enumerate, we should be more verbose here
when the user is not able to access any /dev/media* device. So print a simple
debug message in this case to aid debugging.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3088>
The AV1 support multi spatial layers within one TU with different
resolutions, and only the highest spatial layer need to be output.
For example, there are two spatial layer, base level is 800x600
and higher level is 1920x1080. We need to decode both because the
higher level needs base layer as reference, but we only need to output
1920x1080 frames here.
The current manner always renegotiates the caps once we detect the
current picture resolution changes, so we renegotiate again and
again between different layers. That's a big waste and has very
low performance. We now only do the renegotiation for the highest
output layer. For other non output layers, we just keep a internal
buffer pool which is big enough to handle the surface allocation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2382>
- Update the docker image we use, starting using the standard one adding
`gtk4-doc` as required by rust plugins
- Update the plugins_doc_caches as required, some more plugins are built
with the new image
- Install ninja from pip as the version from F31 is too old
- Avoid buildings all GSreamer plugins when building the doc as it takes
time and resources for no good reason
- Stop linking to `GInstanceInitFunc` as it is not present in latest GLib
documentation, leading to warnings in hotdoc.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2954>
Handle d3d11 device context in set_context() method with
additional device compatibility check so that only NVIDIA GPU
associated d3d11 device can be configured in the element.
And clear old d3d11 device per set_info() for d3d11 device to be
updated as well.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3018>
If the buffer is not msdk_buffer, we can try to directly import the
attached memory (i.e. va mem and dmabuf mem) by applying the common
uitl function: import_to_msdk_function ().
Here add a flag "from_qdata" in GstMsdkSurface to handle the cropping case,
we should avoid updating the crop values when msdk_surface is from the
memory's qdata, because the crop info from this surface is the already
updated one.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2498>
When input buffer is of dmabuf memory but not a msdk buffer (i.e., the
allocator is not msdk_allocator), then we can try to get fd of this mem,
create the corresponding va surface and wrap it as mfx surface.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2498>
* Private header name is changed to gstd3d11-private.h to follow
naming convention
* Add Since mark everywhere
* Update member variable names to be consistent with the other
object implementations in this library
* Correct outdated documentation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2945>
Current default G_MAXINT is not a correct value under any circumstances.
This creates an issue with screen capture, during which we currently do
not get any framerate info causing G_MAXINT to show up, where elements
downstream can possibly misbehave - for example, `vtenc` causes
a kernel panic.
Replace with 30/1 to avoid such scenarios.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2944>
The current handle_frame() does not return the real error that happens
in decode_scan and decode_frame, which makes the pipeline continue with
the error and may trigger asserting later.
We also return the error when decode_quant_table or decode_huffman_table
fails.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2938>
The "present" signal will be emitted just before the
IDXGISwapChain::Present() call. The client can perform additional
GPU operation with given GstD3D11Device object and
ID3D11RenderTargetView handle. Or, the client can read back
the scene to be displayed on window using the signal.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
This allows an application to provide their own opened DRM device
fd handle to kmssink. For example, an application can lease
multiple fd's from a DRM master to display on different CRTC
outputs at the same time with multiple kmssink instances.
Specifying the fd property is not allowed when driver-name
and/or bus-id properties are specified.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2807>
Radeon mesa gallium driver has a bug which adds P010_10LE sink caps
format. This patch removes formats which arent 420 chroma.
gst_caps_set_format_array() wasn't used because the fix traverse
several structures with potential different formats.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2844>
The function g_array_sized_new() leaves the len to 0, but the slice
implementation assumes it would be set to 4. Sending multiple slices is
not yet support for H.264 as no driver needed it yet, but if that code
was to be used it would have overflowed as the array would never grow as
multiple 0 by 2 always results in 0.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1079>
We should move this functionality to gst-libs so that GstD3D11Converter
can be moved to gst-libs.
Another advantage is that applications can call our
HLSL compiler wrapper method without any worry about OS version
dependent system installed HLSL library.
Note that there are multiple HLSL compiler library versions
on Windows and system installed one would be OS version dependent.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2760>
... and don't use GstD3D11VideoProcessor. Now GstD3D11Converter will
be able to convert using videoprocessor, and texture upload is also supported by
GstD3D11Converter. All the noisy code can be removed therefore.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2697>
* Add videoprocessor feature to d3d11converter, in order to unifiy
conversion flow.
* Add convert_buffer() method to support automatic shader/videoprocessor
selection. The method also supports texture upload if input memory
cannot be used for conversion (e.g., system memory or so)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2697>
Adding nvautogpu{h264,h265}enc class which will accept upstream logical
GPU device object (GstCudaContext or GstD3D11Device) instead of
using pre-assigned GPU instance.
If upstream logical GPU device object is not NVENC compatible
(e.g., D3D11 device of non-NVIDIA GPU) or it's system memory,
then user specified "cuda-device-id" or "adapter-luid" property
will be used for GPU device selection.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2666>
GstCudaMemory supports CPU access via CUDA pinned host memory already
and it would show faster memory transfer performance between
GPU and CPU than copying from/to normal system memory.
If downstream supports video meta, we can passthrough CUDA memory.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2690>
Performs crop, scale, and color space conversion all in
a single render pipeline. Note that cropping related property is not
added in this element (which will make negotiation very complicated),
but user can configure videocrop element for crop meta to be attached
on each buffer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2678>
This patch adds general mechanism for handling specific hacks. In this
case for jpeg decoder in i965 driver, which cannot create surfaces
with fourcc specified.
From jpeg decoder to the allocator, which creates the surfaces,
there's a non-simple path: basedec pseudo-class adds a hacks guint32
which will be set by actual elements (vajpegdec, in this case) and
basedec will always set the hack to the allocator when the allocator
is instantiated.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1575>
Given the supported rt formats in a profile/entrypoint config it's
possible to know the supported JPEG colorspace and subsampling. This
patch adds this information in coded caps to a safer autoplugging
after jpegparser.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1575>
This base class is intented for hardware accelerated decoders, but since
only VA uses it, it will be kept internally in va plugin.
It follows the same logic as the others video decoders in the library but.
as JPEG are independet images, there's no need to handle a DBP so no need
of a picture object. Instead a scan object is added with all the structures
required to decode the image (huffman and quant tables, mcus, etc.).
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1575>
Removing glvideomixer-like nuance (it was initially referenced)
and rewriting element since it's not an optimal design at all
from performance point of view.
* Remove wrapper bin (and internal conversion/upload/download elements)
which will waste CPU/GPU resources. Conversion/blending can be done by the
d3d11compositor element at once.
* Add support YUV blending without RGB conversion.
The RGB <-> YUV conversion is completely unnecessary since YUV textures
support blending as well.
* Remove complicated blending operation properties since it's hard
to use from application point of view. Instead, adding "operator" property
like what compositor element does.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2631>
AVC and HEVC define crop rectangle and the x/y coordinates might
not be zero. This commit will address the non-zero x/y offset coordinates
via GstVideoCropMeta if downstream supports the meta and d3d11 memory.
Otherwise decoder will copy decoded texture into output frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2624>
Currently, video format is decided with downstream caps intersection,
but that's not correct since chroma is not considered. The video
decoders have to decide the output format given the used chroma, not
by the downstream caps negotiation.
This patch changes that. Still, caps feature is selected by caps
negotiation, then, with the preferred caps feature, the output format
is search within that caps feature.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2569>
Rewriting GstD3D11Converter (equivalent to GstVideoConverter)
to optimize some conversion path and clean up.
* Extract YUV <-> RGB conversion matrix building method to
utils. It will be used by other implementation
* Use calculated offset values for YCbCr <-> YPbPr conversion
instead of hardcoded values
* Handle color range adjustment
* Move transform matrix building helper function to utils.
The method will be used by other elements
* Use single constant buffer. Multiple constatne buffer for
conversion pipeline is almost pointless
* Remove lots of duplicated HLSL code and split pixel shader
code path into sampling -> colorspace conversion ->
shader output packing
* Avoid floating point precision error around UV coordinates
* Optimize RGB -> YUV conversion path
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2581>
Copy V4L2_PIX_FMT_P010 define from linux header.
V4L2_PIX_FMT_P010 is the little endian definition of P010 so map
it GST_VIDEO_FORMAT_P010_10LE.
Add it v4l2 default video formats to allows v4l2 decoders to
enumerate and use it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2590>
In case that input is D3D11 texture, QSV seems to work regardless
of the alignment. Actually the alignment requirement seems to make
only sense for system memory.
Other Intel GPU dependent implementations (new VA encoder, and MediaFoundation)
do not require such alignment nor other vendor specific ones (NVENC and AMF)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2540>
Msdkdec should use it own pool when the allocation from downstream query
is not any msdk_allocator (i.e. msdk_video_allocator,
msdk_dmabuf_allocator and msdk_system_allocator). Otherwise, when using
pipeline "msdkh264dec ! vah264enc !" to transcode a not 16-bit-aligned
stream (i.e. 1920x1080), the transcoding will fail due to the size
mismatch issue between decoder pool and encoder pool.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2451>
This is a workaround for pts because oneVPL cannot handle the pts
correctly when there is b-frames. We first cache the input frame pts in
a queue then retrive the smallest one for the output encoded frame as
we always output the coded frame when this frame is displayable.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2089>
gst_amf_encoder_try_output() pushes at most one output buffer downstream
although more may be ready. As a consequence, output samples will keep
queueing up in AMFComponent whenever QueryOutput() returns AMF_REPEAT
(and do_wait is FALSE). This has negative impact on latency when the
video being encoded is a live stream.
In order to avoid it, always retrieve and push all samples available in
AMFComponent's output queue at once.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2536>
Intel DXVA driver crashes sometimes (from GPU thread) if
ID3D11VideoDecoder is released while there are outstanding view objects.
To make sure the object life cycle, holds an ID3D11VideoDecoder refcount
in GstD3D11Memory object.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2504>
We should not reset the input/output_frame_count when some configure
changes. For example, the if resolution changes, the current way just
resets the frame count and make the PTS of the output buffer restart
from the original PTS of the first frame. That causes a lot of QOS
event and drop all the new frames.
We should only reset them when encoder start().
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2489>
d3d11screencapture can miss a cursor shape to draw or draw an outdated cursor shape.
- AcquireNextFrame only provides cursor shape when there is one update
- current d3d11screencapture skips cursor shape when mouse is not drawn
So, if a gstreamer application uses d3d11screencapture with cursor initially not drawn
"show-cursor"=false and then switches this property to true, the cursor will not be
actually drawn until AcquireNextFrame provides a new cursor shape.
This commit makes d3d11screencapture always update the cursor shape information, even
if the mouse is not drawn. d3d11screencapture will always have the latest cursor shape
when requested to draw it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2485>
NVIDIA GPUs have undocumented limitation regarding minimum resolution
and it can be queried via a NVDEC API. However, since we don't want to
bring CUDA/NVDEC API into D3D11, use hardcoded values for now
until we find a nice way for capability check.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2406>
If the stream chroma doesn't match with any video format in the source
caps template (generated from va config surface formats) instead of
return unknown, return the first available format in the template,
assuming that the driver would be capable to do color conversions.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2404>
Instead of using a hard-coded list of preferred formats according the
chroma type, now if now caps are pre-negotiated, from template caps
will choose the first format with the same chroma type. If
pre-negotiated, then it will choose the first format, with same chroma
type, from the first caps structure.
Also all the decoders will check if GST_VIDEO_FORMAT_UNKNOWN is
returned, failing the negotiation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2351>
V4L spec now requires decode_params flags to be set in accordance to the
frame's type. In particular this is required by H.264 decoder of NVIDIA
Tegra SoC to operate properly. Set the flags based on type of parsed
slices.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1757>
GstD3D11ScreenCapture object is pipeline-independent global object
and the object can be shared by multiple src elements,
in order to overcome a limitation of DXGI Desktop Duplication API.
Note that the API allows only single capture session in a process for
a monitor.
Therefore GstD3D11ScreenCapture object must be able to handle a case
where a src element holds different GstD3D11Device object. Which can
happen when GstD3D11Device context is not shared by pipelines.
What's changed:
* Allocates capture texture with D3D11_RESOURCE_MISC_SHARED for the
texture to be able to copied into other device's texture
* Holds additional shader objects per src element and use it when drawing
mouse
Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1197
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2366>
If there weren't any moved/dirty regions in the captured frame, the
viewport of the ID3D11DeviceContext would be left at whatever previous
value it had, which could lead to the cursor being drawn in a wrong
position and/or in an incorrect size.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2362>
Make all codecs consistent so that subclass can know additional DPB
size requirement depending on render-delay configuration regardless
of codec. Note that render-delay feature is not implemented for AV1
yet but it's planned.
Also, consider new_sequence() is mandatory requirement, not optional
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2343>
When we fixup src caps, the current way of handling the HDR fields is not
correct.
1. We trim the HDR fields only when the input caps is not a subset of the
fixup src caps. But in fact, the input caps with HDR fields such as the
"mastering-display-info" can possibly be the subset of the fixup src caps,
if they have all same other fields.
2. We always copy the colorimetry from input caps to src caps if it is
absent. But when hdr-tone-mapping is enabled, the HDR->SDR conversion makes
the colorimetry change. We should use downstream's setting, or just use the
default colorimetry of SDR.
We changes to:
1. If hdr-tone-mapping is enabled, we trim all HDR fields and add a correct
colorimetry.
2. Copy colorimetry from input if it is still absent.
3. Consider the subset replacement.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2244>
GLib guarantees libintl is always present, using proxy-libintl as
last resort. There is no need to mock gettex API any more.
This fix static build on Windows because G_INTL_STATIC_COMPILATION must
be defined before including libintl.h, and glib does it for us as part
as including glib.h.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2028>
Our decoder implementation does not use downstream d3d11 pool for
decoding because of special requirement of D3D11/DXVA. So preallocation
using the downstream buffer pool will waste GPU memory in most cases.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2211>
The current way names the level by the number of B frames it contains, the
less it contains, the higher level it is. So the non ref B frames are in the
lowest layer and the B frames in the highest level refer to I/P frames.
But the widely used way is just the opposite, the ref B frames are in the
lower level and non ref B frames are at the highest level.
The is just a terminology change, and does not have any effect for compression
result and quality.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2149>
timeapi.h is missing in our MinGW toolchain. Include mmsystem.h
header instead, which defines struct and APIs in case of our MinGW
toolchain. Note that in case of native Windows10 SDK (MSVC build),
mmsystem.h will include timeapi.h
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2153>
They are part of gst_dep already and we have to make sure to always have
gst_dep. The order in dependencies matters, because it is also the order
in which Meson will set -I args. We want gstreamer's config.h to take
precedence over glib's private config.h when it's a subproject.
While at it, remove useless fallback args for gmodule/gio dependencies,
only gstreamer core needs it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2031>
Added GstVaFeature enum type, and new parameter for VA allocator's
set_format() and get_format(). Also added a new parameter in VA pool
gst_va_pool_new_with_config() and
gst_buffer_pool_config_set_va_allocation_params().
This new parameter will define if derived images will by used for
buffer mapping.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2057>
By default, the classification is
"Converter/Filter/Colorspace/Scaler/Video/Hardware", but if VA
post-processor driver supports either color balance, skin tone
enhancement, sharpening or noise reduction, "Effect" is added.
Thus, if vapostproc ranking is raised, it can be chosen by
autovideosink.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2066>
In order to other plugins use gstva objects, such as allocators and buffer
pools, this merge request move them from the va plugin to the gstva library.
This objects are not exposed in <gst/va/gstva.h> since they are not expected
to be used by users, only by plugin implementators.
Because of the surface copy design, which is used to implement allocator's
mem_copy() virtual function, depends on the vafilter, which is kept inside
the plugin, memory copy through VAPosproc is disabled and removed temporarly.
Also added some missing parameter validation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2048>
Untabifying header file.
The logging category was moved from the plugin generic category to
the display category. It can argue that video formats hacks are
display dependant.
Added validations for input parameters.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2048>
... and add more encoding options.
QSV API supports dynamic bitrate change without IDR insertion.
That's more efficient way of runtime encoding option update
than starting from new sequence with IDR per bitrate option change.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2039>
Check that `self` and `self->callback` are defined. `self` can be set to
`NULL` in `remove_listener`, and `self->callback` can be set to `NULL`
inside `gst_amc_surface_texture_jni_set_on_frame_available_callback`.
This can cause a segfault since the Java object can outlive the C
object, and call the callback after `remove_listener` is called.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2024>
Produce an error if we try to use the feature of holding capture buffer
but it is not supported by the driver. Ingoring this can lead to stalls
as the driver will run-out of capture buffer to decode into. This
affects slice decoders but also split-field interlaced decoding.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2009>
Some problematic H265 stream may miss the reference frame in the DPB,
and get some message like: "No short term reference picture for xxx".
So there may be empty entries in ref_pic_list0/1 when passing to
decode_slice() function of sub class. We need to check the NULL pointer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2018>
Adding new encoder elements nvd3d11{h264,h265}enc for Direct3D11
input support and re-written nvcuda{h264,h265}enc elements.
Newly writeen elements have some differences compared with old
nv{h264,h265}enc including non-backward compatible changes.
* RGBA is not a supported input format any more:
New elements will support only YUV formats to avoid implicit conversion
done by hardware. Ideally it should be done by upstream element
in order to have more control on it. Moreover, RGBA support can cause
redundant RGBA -> YUV conversion if multiple encoders are
used for the same RGBA input
* Subsampled planar format support is dropped:
I420 and YV12 format are not supported formats for Direct3D11.
Although it's supported in CUDA mode, it's not a hardware friendly
memory layout and it will waste GPU memory since UV planes
will have large padding due to the memory layout requirement of NVENC.
* GL support is dropped: Similar to the RGBA case,
GL support in encoder would be suboptimal if GL input is
used by multiple encoders, because each encoder will copy GL memory
into CUDA memory.
Upstream cudaupload element can be used for GL <-> CUDA
interop instead.
* No more pre-allocation of encoder input surfaces. New implementation
will use input CUDA memory without copy (zero-copy) or
will copy into a NVENC's input buffer struct in case of
system memory input.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1997>
Since the strings are empty for GST_MSDK_CAPS_MAKE_WITH_DMABUF_FEATURE
and GST_MSDK_CAPS_MAKE_WITH_VA_FEATURE, when excuting
gst-inspect-1.0.exe msdkh265enc, there will be convert static caps error
because of the extra semicolon between two empty strings. Now macro
definitions are added to avoid this issue.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2004>
Pass the current frame to the duplicate_picture callback. This makes it easier
to set the frame's output_buffer if we already have one available. Also
documented that unlike VP9, it is not optional to implement this as the
picture will populate the DPB if it is a key-frame. To ensure this, remove the
default implementation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1992>
1. Always set the according GstVaH264EncFrame pointer when GstVideoCodecFrame
pointer is assigned, which can make the logic safe.
2. Fix the forgotten change in _sort_by_frame_num. Its input pointer now is
GstVideoCodecFrame type.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1935>
On GstVideoDecoder::{drain,flush}, we send null packet with
CUVID_PKT_ENDOFSTREAM flag to drain out decoder. Which will
reset CUVID parser as well.
To continue decoding after the drain, the next input buffer
should include sequence headers otherwise CUVID parser will
not report any decodeable frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1911>
Instead of using GstMiniObject to hold H264 frame, now it uses a plain
structure. Besides, instead of holding a reference to
GstVideoCodecFrame, the H264 frame structure is set as a
GstVideoCodecFrame user data.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1856>
When there is vpp scaling downstream, we need to make sure SFC is not
triggered because vpp may fall into passthrough mode which causes
the decoder negotiation to create src caps with vpp scaled width/height.
This patch includes bitstream's original size in first query with
downstream in gst_msdkdec_src_caps, which is the same for what we do for
color format in this query. This is to ensure SFC scaling starts to
work only when downstream directly asks for a different size instead of
through vpp.
Note that here SFC scaling follows the same behavior as msdkvpp:
if user only changes width or height, e.g. dec ! video/x-raw,width=xx !,
the height will be modified to the value which fits the original DAR.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1838>
* Hide GstCudaMemory member variables
* Make GstCudaAllocator object GstCudaContext independent
* Set offset/stride of memory correctly via video meta
* Drop GST_BUFFER_POOL_OPTION_VIDEO_ALIGNMENT support.
This implementation actually does not support custom alignment
because we allocate device memory via cuMemAllocPitch
of which alignment is almost uncontrollable
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1834>
* cudaupload/download
- Specify only formats actually we can deal with
nvcodec elements, not all video formats
- Supports CUDA output for download and input for upload in order
to make passthrough possible, like other upload/download elements.
* cudabasetransform
- Reset conversion element if upstream CUDA memory
holds different CUDA context and the element can accept it.
This is the same behavior as corresponding d3d11 filter elements.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1834>
Currently for copying the coded buffer onto a GStreamer buffer, the
coded buffer is mapped two times: one for getting the size, and later
for do the actual copy. We can avoid this by doing directly in the
element rather than in the general encoder object.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1845>
This a new VA-API implementation of a H264 encoder.
It can control the GOP and parameter settings, while the MV searching,
VCL and the rate control algorithm are implemented by VA drivers and HW.
It supports most of the common usage options in H264, but still lacks
of look ahead, field, B frame weighted prediction, etc.
Co-authored-by: Victor Jaquez <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1051>
The g_queue_clear_full() and g_array_copy() functions in the glib
may not be available for the current glib version check, so we add
helper functions to wrap it.
This should be deleted after the glib version bumps.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1051>
The fd was in different meanings on windows:
POSIX read and write use the fd as a file descriptor.
The gst_poll use the fd as a WSASocket.
This patch use WSASocket as default on windows. This is a temporary measure, because IPC has many different implement. There may be a better way in the future.
See #1044
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1791>
We have the d3d11screencapturesrc element in d3d11 plugin
which is obviously better than this element in terms of performance
and design, so we don't need to make people be confused by two separate elements.
Let's pick the better implementation and remove unnecessary one.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1750>
It was assumed that the kernel parameters would match with the bitstream value
but instead the author when with another set of value. Surprisingly, this
makes no difference with the resulting fluster score.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1748>
If present, add '-lsocket' and '-lnsl' to network_deps.
ext/curl/meson.build: add network_deps to dependencies
gst/festival/meson.build: same
sys/shm/meson.build: same
Fixes linking issues on Illumos distros.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1525>
Note that AYUV and AYUV64 formats will be used to expand format
support, especially some packed YUV formats (e.g., Y410, YUY2)
are common DXGI formats used for hardware decoder/encoder on Windows
but those formats cannot be used as a render target. We need to handle
them differently without pixel shader help, using compute shader
for example.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1699>
Remove all the d3d11 and dxgi header version dependent ifdef
and bump the minimum requirement to d3d11_4.h and dxgi1_6.h.
We are already failing support old Visual Studio (Windows SDK actually)
such as Visual Studio 2015. Note that our MinGW toolchain satisfies
the requirement.
From runtime point of view, this change should be fine since
we are checking OS version with IUnknown::QueryInterface()
everywhere in order to check API availability
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1684>
We make all MSDK encoders declare "memory:VAMemory" feature. Then
the pipeline such as:
gst-launch-1.0 -vf filesrc location=xxx.h264 ! h264parse ! \
vah264dec ! msdkh265enc ! fakesink
will choose VA memory caps between the VA decoder and MSDK encoder.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1087>
The MSDK encoder's query function is not set and it just forwards
all query to its base class. We now need to answer the context
query correctly. Other VA plugins need to query the VA display.
By the way, the current query of "gst.msdk.Context" is also missing.
The other MSDK elements must depend on the bin's context message(
sent in context_propagate()) to set their MsdkContext correctly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1087>
The MSDK VPP's query function is not set and it just forwards
all query to its base class. We now need to answer the context
query correctly. Other VA plugins need to query the VA display.
By the way, the current query of "gst.msdk.Context" is also missing.
The other MSDK elements must depend on the bin's context message(
sent in context_propagate()) to set their MsdkContext correctly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1087>
The MSDK decoder's query function is not set and it just forwards
all query to its base class. We now need to answer the context
query correctly. Other VA plugins need to query the VA display.
By the way, the current query of "gst.msdk.Context" is also missing.
The other MSDK elements must depend on the bin's context message(
sent in context_propagate()) to set their MsdkContext correctly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1087>
We now can use the gst va lib's display to create our MSDK context,
and use its helper functions to simplify our code. The improved logic
is like this:
1. Every MSDK element should use gst_msdk_context_find() to find a MSDK
context from neighbour. If valid, reuse it.
2. Use gst_msdk_ensure_new_context(). It will first query neighbours
about the GstVaDisplay, if found(e.g. some VA element is connected),
use gst_msdk_context_from_external_display() to create a MSDK context.
3. Then, creating the MSDK context from scratch. It creates both the
display and MSDK context.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1087>
The VA display object from VA lib is a common defined object. which
contain the whole display things. It is easier to use, and more important,
we can share it with the other VA plugins and keep all the VA related
plugins working on the same GPU device.
We also delete the useless gst_msdk_context_get_fd() API.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1087>
Our D3D11/DXVA codecs implementation has been verified
during 1.18 and 1.20 development cycle and also via the Fluster
test framework. Similar to the case of nvdec and vtdec,
we can prefer hardware over software in most cases
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1672>
Duplicating a picture what was already a dup was leading to a crash. Rename
the custom picture flags as HOLDS_BUFFER to make its meaning clear. Then save
then ref and store the picture as userdata, so it can be obtained when
duplicating. Finally, mark the doplicated as HOLDS_BUFFER to avoid thinking it
holds a request.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1681>
This patch adds a new parameter: hdr-tone-mapping (same as
vaapipostproc), if the HDR capabilites are availabe in driver, and
it's disabled by default.
If hdr-tone-mapping is enabled then HDR fields in sink caps are
processed in frames from HDR to SDR, removing those hdr fields in
source pad caps too.
hdr-tone-mapping is not enabled if a color conversion is also
requested, since it fails to process in the iHD driver, so far.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1258>
In commit e699aaeb we moved linking of libgudev to the plugin rather
the library, because it's only used in the plugin. But the dependency
check is still done in library.
This patch removes the dependency check in library, and updates the
dependency check in plugin.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1650>
A new implementation of Intel Quick Sync Video plugin.
This plugin supports both Windows and Linux but optimization for
VA/DMABuf is not implemented yet.
This new plugin has some notable differences compared with existing
MSDK plugin.
* Encoder will expose formats which can be natively supported
without internal conversion. This will make encoder
control/negotiation flow much simpler and cleaner than
that of MSDK plugin.
* This plugin includes QSV specific library loading helper,
called dispatcher, with QSV SDK headers as a part of this plugin.
So, there will be no more SDK version dependent #ifdef in the code
and also there will be no more build-time MSDK/oneVPL SDK
dependency.
* Memory allocator interop between GStreamer and QSV is re-designed
and decoupled. Instead of implementing QSV specific allocator/bufferpool,
this plugin will make use of generic GStreamer memory
allocator/bufferpool (e.g., GstD3D11Allocator and GstD3D11BufferPool).
Specifically, GstQsvAllocator object will help interop between
GstMemory and mfxFrameAllocator memory abstraction layers.
Note that because of the design decision, VA/DMABuf support is not made
as a part of this initial commit. We can add the optimization for Linux
later once GstVA library exposes allocator/bufferpool implementation as
an API like GstD3D11.
* Initial encoder implementation supports interop with GstD3D11
infrastructure, including zero-copy encoding with upstream D3D11 element.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1408>
