If the buffer is not msdk_buffer, we can try to directly import the
attached memory (i.e. va mem and dmabuf mem) by applying the common
uitl function: import_to_msdk_function ().
Here add a flag "from_qdata" in GstMsdkSurface to handle the cropping case,
we should avoid updating the crop values when msdk_surface is from the
memory's qdata, because the crop info from this surface is the already
updated one.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2498>
When input buffer is of dmabuf memory but not a msdk buffer (i.e., the
allocator is not msdk_allocator), then we can try to get fd of this mem,
create the corresponding va surface and wrap it as mfx surface.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2498>
We were checking possible bind flags for the DXGI format
of the source texture but that's never applied to
the destination texture desc.
Just use the already configured bind (and misc) flags of source texture
for the destination texture allocation without additional check.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2950>
Setting the content-type property shall override internally detected MIME
types, to make it possible to do as following example (where audio/basic to be
used prior to audio/x-mulaw):
gst-launch-1.0 ... ! mulawenc ! audio/x-mulaw,rate=8000,channels=1 !
curlhttpsink location=<url> content-type=audio/basic
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2732>
* Private header name is changed to gstd3d11-private.h to follow
naming convention
* Add Since mark everywhere
* Update member variable names to be consistent with the other
object implementations in this library
* Correct outdated documentation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2945>
Current default G_MAXINT is not a correct value under any circumstances.
This creates an issue with screen capture, during which we currently do
not get any framerate info causing G_MAXINT to show up, where elements
downstream can possibly misbehave - for example, `vtenc` causes
a kernel panic.
Replace with 30/1 to avoid such scenarios.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2944>
The current handle_frame() does not return the real error that happens
in decode_scan and decode_frame, which makes the pipeline continue with
the error and may trigger asserting later.
We also return the error when decode_quant_table or decode_huffman_table
fails.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2938>
Add an example to show the usage of present singal.
In this example, a text overlay with alpha blended background
will be rendered on swapchain's backbuffer by using
Direct3D11, Direct2D, and DirectWrite APIs.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
The "present" signal will be emitted just before the
IDXGISwapChain::Present() call. The client can perform additional
GPU operation with given GstD3D11Device object and
ID3D11RenderTargetView handle. Or, the client can read back
the scene to be displayed on window using the signal.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2923>
This allows an application to provide their own opened DRM device
fd handle to kmssink. For example, an application can lease
multiple fd's from a DRM master to display on different CRTC
outputs at the same time with multiple kmssink instances.
Specifying the fd property is not allowed when driver-name
and/or bus-id properties are specified.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2807>
Without this change cleanup function for g_autoptr is not defined for
GstPlayMediaInfo, GstPlaySignalAdapter, GstPlayVideoRenderer,
GstPlayVideoOverlayVideoRenderer and GstPlayVisualization. Cleanup
function was defined in gstplay.h, but missing in other header files.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2888>
Ideally new() functions should simply call g_object_new() and not much
else, so let's do that here and handle all the construction properly in
a GObject way.
Now a play object created via g_object_new() is actually usable.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2880>
Ideally new() functions should simply call g_object_new() and not much
else, so let's do that here and handle all the construction properly in
a GObject way.
Now a player object created via g_object_new() is actually usable.
In addition, also fix the video-renderer property so that reading it
returns an object of the correct type.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2880>
This was showing up as a memory leak in GTK's
gstreamer media backend:
40 bytes in 1 blocks are definitely lost in loss record 18,487 of 40,868
at 0x484586F: malloc (vg_replace_malloc.c:381)
by 0x50D5278: g_malloc (gmem.c:125)
by 0x50EDBA5: g_slice_alloc (gslice.c:1072)
by 0x50EFBCC: g_slice_alloc0 (gslice.c:1098)
by 0x51F2F45: g_type_create_instance (gtype.c:1911)
by 0x51DAE37: g_object_new_internal (gobject.c:2011)
by 0x51DC080: g_object_new_with_properties (gobject.c:2181)
by 0x51DCB20: g_object_new (gobject.c:1821)
by 0x9855F86: UnknownInlinedFun (gstplayer-wrapped-video-renderer.c:109)
by 0x9855F86: gst_player_new (gstplayer.c:579)
Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1374
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2875>
Radeon mesa gallium driver has a bug which adds P010_10LE sink caps
format. This patch removes formats which arent 420 chroma.
gst_caps_set_format_array() wasn't used because the fix traverse
several structures with potential different formats.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2844>
GLib's GRecMutex will allocate another heap memory for CRITICAL_SECTION
struct and g_rec_mutex_lock/g_rec_mutex_unlock use WIN32 APIs actually.
We don't need such intermediate function calls and redundant heap allocation.
Just call WIN32 APIs directly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2845>
This is based on gtksink, but similar to waylandsink uses Wayland APIs
directly instead of rendering with Gtk/Cairo primitives.
Note that the long term plan is to move this into the existing extension
in `-good`, which requires the Wayland library to move the as well.
For this reason several files like `gstgtkutils.*` and `gtkgstbasewidget.*`
are straight copies and should be kept in sync.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1515>
With the 2.72 release, glib-networking developers have decided that
TLS certificate validation cannot be implemented correctly by them, so
they've deprecated it.
In a nutshell: a cert can have several validation errors, but there
are no guarantees that the TLS backend will return all those errors,
and things are made even more complicated by the fact that the list of
errors might refer to certs that are added for backwards-compat and
won't actually be used by the TLS library.
Our best option is to ignore the deprecation and pass the warning onto
users so they can make an appropriate security decision regarding
this.
We can't deprecate the tls-validation-flags property because it is
very useful when connecting to RTSP cameras that will never get
updates to fix certificate errors.
Relevant upstream merge requests / issues:
https://gitlab.gnome.org/GNOME/glib/-/merge_requests/2214https://gitlab.gnome.org/GNOME/glib-networking/-/issues/179https://gitlab.gnome.org/GNOME/glib-networking/-/merge_requests/193
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2494>
Depending on device feature level, d3d11 runtime can support
ID3D11Fence which is equivalent to ID3D12Fence.
Waiting using fence has performance-wise benefit over pulling
ID3D11Query status. If ID3D11Fence is not supported by device,
then ID3D11Query will be used instead.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2790>
It may happens that bitstream doesn't provided SPS in decoding order
(like in VPSSPSPPS_A_MainConcept_1 conformance test file).
To be sure that the decoder got the correct SPS parameters process
SPS just before start decoding the frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2575>
While possible defer computataion of pps and sps fields until
slice parsing since it may happens that bitstreams don't encoded
them in expected order.
A example weird ordered bitstreams is VPSSPSPPS_A_MainConcept_1
conformance test.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2575>
The function g_array_sized_new() leaves the len to 0, but the slice
implementation assumes it would be set to 4. Sending multiple slices is
not yet support for H.264 as no driver needed it yet, but if that code
was to be used it would have overflowed as the array would never grow as
multiple 0 by 2 always results in 0.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1079>
We should move this functionality to gst-libs so that GstD3D11Converter
can be moved to gst-libs.
Another advantage is that applications can call our
HLSL compiler wrapper method without any worry about OS version
dependent system installed HLSL library.
Note that there are multiple HLSL compiler library versions
on Windows and system installed one would be OS version dependent.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2760>
We need GStreamer elements to do the bandwidth estimation as this way
they can also control the pacing of the transmission flow as specified
in the [GCC] algorithm for example.
Bandwidth estimator element are placed right before the "RTPSession" as
an "rtp-aux-sender" element. This way they can use the "Transport-wide
Congestion Control" RTCP feedback messages through the "RTPTwcc" custom
events that are sent by the rtpsession.
Applications are responsible to react to the bandwidth estimator element
and set the encoder target bitrate etc... which means that we can not
pass an estimator as an element factory, so a signal as been chosen
instead.
[GCC]: https://datatracker.ietf.org/doc/html/draft-ietf-rmcat-gcc-02
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2562>
There might be a sequence of event and buffer flow:
- Got stream-start/caps/segment events
- Got flush events
- And then buffers with a new segment event
In the above case, stream-start and caps event might not be reached to
peer proxysrc if peer proxysrc is not ready to receive them.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1552>
... and don't use GstD3D11VideoProcessor. Now GstD3D11Converter will
be able to convert using videoprocessor, and texture upload is also supported by
GstD3D11Converter. All the noisy code can be removed therefore.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2697>
* Add videoprocessor feature to d3d11converter, in order to unifiy
conversion flow.
* Add convert_buffer() method to support automatic shader/videoprocessor
selection. The method also supports texture upload if input memory
cannot be used for conversion (e.g., system memory or so)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2697>
Adding nvautogpu{h264,h265}enc class which will accept upstream logical
GPU device object (GstCudaContext or GstD3D11Device) instead of
using pre-assigned GPU instance.
If upstream logical GPU device object is not NVENC compatible
(e.g., D3D11 device of non-NVIDIA GPU) or it's system memory,
then user specified "cuda-device-id" or "adapter-luid" property
will be used for GPU device selection.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2666>
GstCudaMemory supports CPU access via CUDA pinned host memory already
and it would show faster memory transfer performance between
GPU and CPU than copying from/to normal system memory.
If downstream supports video meta, we can passthrough CUDA memory.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2690>
Performs crop, scale, and color space conversion all in
a single render pipeline. Note that cropping related property is not
added in this element (which will make negotiation very complicated),
but user can configure videocrop element for crop meta to be attached
on each buffer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2678>
This example code demonstrates D3D11 device sharing between
application and GStreamer. Application can access texture
using appsink and it can be rendered on application's window without
any copy operation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2646>
Our Direct3D11 abstraction layer has been improved and
it gained good shape from API point of view.
Also, On Windows, GstD3D11 has various advantages over GstGL
in terms of compatibility/stability/feature/performance.
Note that WGL implementation is known to be buggy for some
drivers/vendors/scenario (that's a reason why Google implemented ANGLE).
Moreover, GstGL is not fully optimized for Windows unfortunately.
It's the time to open this interface to application developers
for various optimized processing using our Direct3D11
infrastructure.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2646>
This patch adds general mechanism for handling specific hacks. In this
case for jpeg decoder in i965 driver, which cannot create surfaces
with fourcc specified.
From jpeg decoder to the allocator, which creates the surfaces,
there's a non-simple path: basedec pseudo-class adds a hacks guint32
which will be set by actual elements (vajpegdec, in this case) and
basedec will always set the hack to the allocator when the allocator
is instantiated.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1575>
Given the supported rt formats in a profile/entrypoint config it's
possible to know the supported JPEG colorspace and subsampling. This
patch adds this information in coded caps to a safer autoplugging
after jpegparser.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1575>
This base class is intented for hardware accelerated decoders, but since
only VA uses it, it will be kept internally in va plugin.
It follows the same logic as the others video decoders in the library but.
as JPEG are independet images, there's no need to handle a DBP so no need
of a picture object. Instead a scan object is added with all the structures
required to decode the image (huffman and quant tables, mcus, etc.).
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1575>
Gallium drivers historically have reported strange dmabuf sizes, from always
zero to the whole frame (multiple fds). The simplest solution is to use lseek
SEEK_END to get the prime descriptor size.
Also the allocator raises a warning if both values differ in order to report
it to driver.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2574>
The commit b90d0274 introduces uninitialized width and height when we
consider to change the "pixel-aspect-ratio" for some interlaced stream.
We need to check the resolution in the src caps, and if no resolution
info found, there is no need to consider the aspect ratio.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2630>
Removing glvideomixer-like nuance (it was initially referenced)
and rewriting element since it's not an optimal design at all
from performance point of view.
* Remove wrapper bin (and internal conversion/upload/download elements)
which will waste CPU/GPU resources. Conversion/blending can be done by the
d3d11compositor element at once.
* Add support YUV blending without RGB conversion.
The RGB <-> YUV conversion is completely unnecessary since YUV textures
support blending as well.
* Remove complicated blending operation properties since it's hard
to use from application point of view. Instead, adding "operator" property
like what compositor element does.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2631>
Similar to and inspired by glimagesink and gtkglsink.
Using the Wayland buffer transform API allows to offload
rotate operations to the Wayland compositor. This can have
several advantages:
- The Wayland compositor may be able to use hardware plane
capabilities to do the rotation.
- In case of pre-rotated content on rotated outputs the
rotations may equal out, potentially allowing the
compositor to use hardware planes even if they don't
support rotate operations.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2543>
AVC and HEVC define crop rectangle and the x/y coordinates might
not be zero. This commit will address the non-zero x/y offset coordinates
via GstVideoCropMeta if downstream supports the meta and d3d11 memory.
Otherwise decoder will copy decoded texture into output frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2624>
This allows the reception of streams that don't exactly match
the codec preferences. In particular, the ssrc in the codec preferences
is local sender SSRC, the other side is expected to send a different SSRC.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2615>
Some encoders (e.g. Makito) have H265 field-based interlacing, but then
also specify an 1:2 pixel aspect ratio. That makes it kind-of work with
decoders that don't properly support field-based decoding, but makes us
end up with the wrong aspect ratio if we implement everything properly.
As a workaround, detect 1:2 pixel aspect ratio for field-based
interlacing, and check if making that 1:1 would make the new display
aspect ratio common. In that case, we override it with 1:1.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2577>
Currently, video format is decided with downstream caps intersection,
but that's not correct since chroma is not considered. The video
decoders have to decide the output format given the used chroma, not
by the downstream caps negotiation.
This patch changes that. Still, caps feature is selected by caps
negotiation, then, with the preferred caps feature, the output format
is search within that caps feature.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2569>
Rewriting GstD3D11Converter (equivalent to GstVideoConverter)
to optimize some conversion path and clean up.
* Extract YUV <-> RGB conversion matrix building method to
utils. It will be used by other implementation
* Use calculated offset values for YCbCr <-> YPbPr conversion
instead of hardcoded values
* Handle color range adjustment
* Move transform matrix building helper function to utils.
The method will be used by other elements
* Use single constant buffer. Multiple constatne buffer for
conversion pipeline is almost pointless
* Remove lots of duplicated HLSL code and split pixel shader
code path into sampling -> colorspace conversion ->
shader output packing
* Avoid floating point precision error around UV coordinates
* Optimize RGB -> YUV conversion path
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2581>
Copy V4L2_PIX_FMT_P010 define from linux header.
V4L2_PIX_FMT_P010 is the little endian definition of P010 so map
it GST_VIDEO_FORMAT_P010_10LE.
Add it v4l2 default video formats to allows v4l2 decoders to
enumerate and use it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2590>
In case that input is D3D11 texture, QSV seems to work regardless
of the alignment. Actually the alignment requirement seems to make
only sense for system memory.
Other Intel GPU dependent implementations (new VA encoder, and MediaFoundation)
do not require such alignment nor other vendor specific ones (NVENC and AMF)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2540>
A few header files in -bad contain comments that start with the
/** gtk-doc pattern, but should not actually be parsed (and warned
about as such).
Previously, we were using far-reaching wildcard patterns to avoid
parsing those, but this had the unintended side effect of also
excluding legitimate files, and creating confusion when comments
were not parsed from those.
Switch to excluding specific files instead.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2576>
Msdkdec should use it own pool when the allocation from downstream query
is not any msdk_allocator (i.e. msdk_video_allocator,
msdk_dmabuf_allocator and msdk_system_allocator). Otherwise, when using
pipeline "msdkh264dec ! vah264enc !" to transcode a not 16-bit-aligned
stream (i.e. 1920x1080), the transcoding will fail due to the size
mismatch issue between decoder pool and encoder pool.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2451>
