We did several things to enable the new memory logic in msdkdec:
(1) We always use video memory for decoder in linux path;
(2) We give negotiated pool to alloc_pool stored in GstMsdkContext which
will be used in callback mfxFrameAllocator:Alloc to alloc surfaces as
MediaSDK needs, and this pool is also available for decoder itself;
(3) We modify decide_allocation process, that is we make pool negotiaion
before gst_msdk_init_decoder to ensure the pool is decided and ready for
use in mfxFrameAllocator:Alloc callback; then we will consider the case
when we need to do the gpu to cpu copy.
(4) In gst_msdkdec_finish_task, we modify the way for copy following the
logic in (3).
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3439>
Add a pool creation function name as 2 for later use which will create
va pool for video memory in linux and keep system pool for windows.
This gst_msdkdec_create_buffer_pool2 will replace gst_msdkdec_create_buffer_pool
when all the memory allocation modifications are ready in the commits after.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3439>
Rewrite gst_msdk_frame_alloc and name it as xxx_2 before applying it.
It uses negotiated bufferpool stored in GstMsdkContext to allocate buffers
in the callback MfxFrameAllocator:Alloc, then extract VASurface from buffer,
wrap it as mfxMemIDs and pass these IDs to MediaSDK/oneVPL.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3439>
The `add_candidate` vfunc of the GstWebRTCICE interface gained a GstPromise
argument, which is an ABI break. We're not aware of any external user of this
interface yet so we think it's OK.
This change is useful in cases where the application needs to bubble up errors
from the underlying ICE agent, for instance when the agent was given an invalid
ICE candidate.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3960>
The signal triggers an asynchronous task on the PC thread but in some cases it
can be useful for apps to be notified when the task completed. This method of
the PeerConnection spec also returns a Promise so the interface is now more
coherent with the spec.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3960>
Read and flush console buffer from the console thread immediately,
instead of main thread. Otherwise (if main thread is busy)
the console thread will keep adding idle source and then main thread
will be unresponsive.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4067>
The av1decoder class does not implement the ->parse() virtual function,
and we always need to add the av1parse element before it. So we should
set_needs_format of the decoder to TRUE, then if no parse before it, it
can fail with a "not-negotiated" error early, rather than go on and
generate unexpected error.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4064>
The vp9decoder class does not implement the ->parse() virtual function,
and we always need to add the vp9parse element before it. So we should
set_needs_format of the decoder to TRUE, then if no parse before it, it
can fail with a "not-negotiated" error early, rather than go on and
generate unexpected error.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4064>
The vp8decoder class does not implement the ->parse() virtual function,
it can only accepts frame aligned data. If some element such as filesrc
feed it with unaligned data, the behaviour is undecided. So we should
set_needs_format of the decoder to TRUE, then it can fail with a
"not-negotiated" error early, rather than go on and generate unexpected
error.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4064>
The mpeg2decoder class does not implement the ->parse() virtual function,
and we always need to add the mpegvideoparse element before it. So we should
set_needs_format of the decoder to TRUE, then if no parse before it, it
can fail with a "not-negotiated" error early, rather than go on and
generate unexpected error.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4064>
The h264decoder class does not implement the ->parse() virtual function,
and we always need to add the h264parse element before it. So we should
set_needs_format of the decoder to TRUE, then if no parse before it, it
can fail with a "not-negotiated" error early, rather than go on and
generate unexpected error.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4064>
The h265decoder class does not implement the ->parse() virtual function,
and we always need to add the h265parse element before it. So we should
set_needs_format of the decoder to TRUE, then if no parse before it, it
can fail with a "not-negotiated" error early, rather than go on and
generate unexpected error.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4064>
Raw 608 caps can now contain a "field" field. On the input side it
signifies that the input raw 608 is attached to either field 0 or 1,
on the output side it allows selecting whether to extract the raw 608
data for field 0 or 1 for field-aware formats.
In addition, it is also allowed to use ccconverter to "convert" 608
field 0 to 608 field 1 (and conversely), this is passthrough as the
change only needs to happen in the caps.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4031>
These parameters are not actually `out` parameters but must
be allocated and zero-initialized by the calling function.
Marking them as `out caller-allocates` will cause memory
corruptions when calling these APIs from e.g., Python code.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4051>
The dimension of the overlay texture directly corresponds to the size of the overlay **buffer** which is given by its video meta.
The dimension at which the overlay should be displayed directly correspond to the overlay `render_width`and `render_height`.
This match the behavior of glimagesink
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4046>
It's only malformed data in APP when its length is less than 6 chars,
because it should have at least an id string. Otherwise, if the id string
is not handled, no warning is raised, only a debug message noticing it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3943>
Fixes the following valgrind error:
==616== Conditional jump or move depends on uninitialised value(s)
==616== at 0x4900E34: gst_debug_print_object (gstinfo.c:1143)
==616== by 0x49010B6: gst_info_printf_pointer_extension_func (gstinfo.c:1215)
==616== by 0x4959FDB: __gst_printf_pointer_extension_serialize (printf-extension.c:47)
==616== by 0x495A487: printf_postprocess_args (vasnprintf.c:258)
==616== by 0x495A52C: __gst_vasnprintf (vasnprintf.c:290)
==616== by 0x4959F8F: __gst_vasprintf (printf.c:154)
==616== by 0x4901C1F: gst_debug_message_get (gstinfo.c:791)
==616== by 0x4901C75: _gst_debug_log_preamble (gstinfo.c:1431)
==616== by 0x4903208: gst_debug_log_default (gstinfo.c:1575)
==616== by 0x49020BA: gst_debug_log_full_valist (gstinfo.c:624)
==616== by 0x490211D: gst_debug_log_valist (gstinfo.c:656)
==616== by 0x49021AD: gst_debug_log (gstinfo.c:533)
==616== by 0x48DDC11: gst_buffer_copy_into (gstbuffer.c:693)
==616== by 0x48DF5F1: gst_buffer_copy_with_flags (gstbuffer.c:727)
==616== by 0x48DF640: gst_buffer_copy_deep (gstbuffer.c:756)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4034>
When the QoS stats are reset (e.g. changing the source) the counters for
dropped + rendered frames are reset to zero which result in negative values
for their difference. This results in max-fps getting pegged at an extremely
high value.
```
fpsdisplaysink.c:373:display_current_fps:<fpsdisplaysink0> Updated max-fps to 36840705952231460864.000000
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3989>
Makes "start-bitrate" work without setting "connection-speed" property. Having
another property set as a requirement for this one to work is unexpected.
This commit allows to request some initial bitrate for first segment, then
go into adaptive streaming for the rest of media playback.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3895>
When constructing an output profile using --profile-from, it is useful
to be able to override the top level container profile.
Expose a --container-profile option that applies as an override after
other methods for constructing an output profile have run. If no other
method was used, this will result in an empty top level container.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3977>
Instead of creating new decoder instance per new sequence,
re-use configured decoder instance via cuvidReconfigureDecoder()
API. It will make output surface reusable without re-allocation.
Also, in order for application to be able to reserve higher resolution
output surface, "init-max-width" and "init-max-height" properties are
added to each decoder.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3884>
Call input resource map functions (i.e., nvEncRegisterResource,
nvEncUnregisterResource, nvEncMapInputResource, and
nvEncUnmapInputResource) only once and reuse the mapped resources,
instead of per input frame map/unmap
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3884>
Wrap mapped decoder output surface using GstCudaMemory and
output without any copy operation. Also, for application to be able to
control the number of zero-copyable output surfaces,
"num-output-surfaces" property is added.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3884>
A data offset with an offset smaller than the moof length is wrong
in smooth streaming streams. The samples will not be located and
eventually playback will error out. So compensate assuming data
is in mdat following moof.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3840>
A data offset with an offset smaller than the moof length is wrong
in smooth streaming streams.
The samples will not be located and eventually playback will
error out. So compensate assuming data is in mdat following moof.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3840>
When uridecodebin exposes pads for its streams, we immediately ghost
the relevant (selected) one and let composition send a seek as soon as a
buffer is probed.
This means that sometimes uridecodebin is still linking elements
internally (for non-selected streams) and sees flush events travel down
the elements it is still busy trying to link / forward sticky events to.
This causes all sorts of nasty issues, which can be avoided by simply
blocking all data flow from the source until no-more-pads has been
emitted by uridecodebin (or whatever sub_element is wrapped).
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3971>
This most likely never caused any issues as we don't connect to
no-more-pads in the first place, and the element isn't directly exposed
to the user, but emitting it makes no sense, and we are actually going
to connect to no-more-pads in a subsequent commit.
The call was added in 86b893e54c, a patch
by me in 2013, I have no idea why but I probably didn't have a firm
grasp on what I was doing then.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3971>
this is an issue seen with musl based linux distros e.g. alpine [1]
musl is not going to change this since it breaks ABI/API interfaces
Newer compilers are stringent ( e.g. clang16 ) which can now detect
signature mismatches in function pointers too, existing code warned but
did not error with older clang
Fixes
gstv4l2object.c:544:23: error: incompatible function pointer types assigning to 'gint (*)(gint, ioctl_req_t, ...)' (aka 'int (*)(int, unsigned long, ...)') from 'int (int, int, ...)' [-Wincompatible-function-pointer-types]
v4l2object->ioctl = ioctl;
^ ~~~~~
[1] https://gitlab.alpinelinux.org/alpine/aports/-/issues/7580
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3950>
This fixes a compile error with recent upstream FFmpeg.
The AV_CODEC_CAP_AUTO_THREADS was deprecated and renamed to
AV_CODEC_CAP_OTHER_THREADS in FFmpeg upstream commit
7d09579190de (lavc 58.132.100).
The AV_CODEC_CAP_AUTO_THREADS was finally removed in FFmpeg upstream
commit 10c9a0874cb3 (lavc 59.63.100).
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3951>
The encoder does not support reconfiguration, and only deinitializing it
and then initializing it again causes deadlocks.
Also only reconfigure and drain the encoder if the video info has
actually changed.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3957>
Fixes#1358.
Passing ARGB64/RGBA64 to vtenc caused the encoding to fail
when running on M1 Pro/Max variants with macOS 12.x, so let's
remove these formats from caps when such scenario is detected.
This issue appears to have been fixed OS-side in macOS 13.0.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3912>
This crept in several years ago sadly :(
The usage of accurate seeking should be reserved to use-cases where it is
essential that we seek to that position. This should not be the default.
There is a new option `--acurate-seeks/-a` to be able to force that.
Furthermore, if accurate seeks aren't required, a player should be using the
GST_SEEK_FLAG_KEY_UNIT flag to seek to the closest keyframe and provide the most
reactive experience.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3926>
This was causing incorrect output when seeking, especially
when used with a multithreaded source like `videotestsrc n-threads=2`.
It should now correctly wait for frames still being processed by VT
while vtdec is flushing.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3922>
We are using std::isspace() with one parameter. That function is defined
in the cctype header.
```
win32ipcutils.cpp(34): error C2672: 'std::isspace': no matching overloaded function found
win32ipcutils.cpp(34): error C2780: 'bool std::isspace(_Elem,const std::locale &)': expects 2 arguments - 1 provided
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3933>
When the task already exists, we forgot to free the passed `user_data`.
This wasn't an issue for most C code, which doesn't pass a
`GDestroyNotify`, but bindings such as gstreamer-rs do!
That said, allocating a trampoline in gstreamer-rs just for it to get
thrown away again is awkward. Maybe we need a `gst_pad_resume_task`?
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3920>
The av1C box is optional so dropping parsing does not break anything
fundamentally, and there seems to be no historical record how version 0
even looks like while the comments and the parsing disagreed with each
other.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3882>
Since b76d336549
pads are deactivated when going to READY but in `uridecodebin(3)`, the
sources source pads are activated while in NULL state (when PULL mode is
supported), meaning that we are ending up deactivating those pads in
NULL_TO_READY, breaking the pipeline.
The intent of the commit mentioned above is to ensure that the pads are
deactivated either in PAUSED_TO_READY or READY_TO_READY, so it should
be safe to avoid deactivating in NULL_TO_READY.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3849>
Generating the source element is done when uridecodebin is doing the
READY to PAUSED state change, so it is reasonable to set the new source
element to that state.
This also allows detecting early failures with backing libraries or
hardware (checks done in NULL->READY).
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3857>
Do not store cached EGL images in GstMemory QData. Instead, use a
per-DmabufUpload GHashTable to store cache entries with a weak
reference to the GstMemory.
This allows two glupload elements on separate tee branches to have
their own EGL image cache. For this pipeline:
gst-launch-1.0 v4l2src ! tee name=t \
t. ! queue ! glupload ! fakesink
t. ! queue ! glupload ! fakesink
this gets rid of the occasional critical error message:
GStreamer-CRITICAL **: 08:26:33.194: gst_mini_object_unref: assertion 'GST_MINI_OBJECT_REFCOUNT_VALUE (mini_object) > 0' failed
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3880>
When using qtdemux in a pipeline that should only work as a pure demuxer (not
for actual playback), qtdemux shouldn't emit new GstSegments to correct
the start time (jump to the future) to ensure that the user experiences no
playback delay. By doing so, it's generating the wrong segments when an append
of data from the past happens. When that happens, downstream elements such as
parsers (eg: aacparse) may clip those buffers laying before the GstSegment and
create problems on the GStreamer client app (eg: WebKit).
Getting buffers clipped out because of the wrong GstSegments started becoming
a problen when this commit was introduced:
ab6e49e9cc audioparsers: add back segment clipping to parsers that have lost it
This clipping makes test DASH shaka 35 from MVT tests[1] to fail in
WebKitGTK/WPE (at least) and can potentially cause a number of other problems
in the WebKit Media Source Extensions (MSE) code.
Note that this new behaviour of not emitting new GstSegments only makes sense
when qtdemux is being used as a pure demuxer and not as part of a regular
pipeline. This is why the variant field has been added. When equal to
VARIANT_MSE_BYTESTREAM, it will make qtdemux behave differently in push mode,
taking decisions that meet the expectations for an MSE-like processing mode.
This kind of tweaks have been done in the past for MSS streams, for instance.
That code has been refactored to use VARIANT_MSS_FRAGMENTED now, instead of
its own dedicated boolean flag.
Co-authored by: Alicia Boya García <ntrrgc@gmail.com>
...who suggested to use "variant: mse-bytestream" in the caps to identify that
mode, as proposed in her unmerged patch:
https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/issues/467
[1] https://github.com/rdkcentral/mvt
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3867>
The live playlists should be updated at a defined interval. The problem is that
this interval was used *after* the playlist was finally received and processed,
which resulted in a gradual shift happening in playlist updates.
Instead store and use the time at which playlists were requested to determine
when the next one should be downloaded.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3883>
The scanning is done in a reverse order, the proper full checks to do are
therefore:
* If the position is beyond half a "segment duration", it's in the following
segment
* If the position is within the first half of a segment, it's in that one
* If the segment is the first one and the position is within half a duration
backwards, we consider the position as being within that first segment
Also handle the case where a "partial only" segment doesn't have a reliable
duration, and therefore use the playlist target duration instead.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3883>
The implementation wouldn't work with regular HLS streams (i.e. the final
fallback).
Now that the implementation uses time to search for the starting
segment (instead of just the n-th from the end), we can specify the correct
hold_back fallback value from the RFC
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3883>
Avoid a deadlock if a downstream seeking query happens while the scheduler
thread is holding the manifest lock (for example during a seek back to live).
Instead, do a more elaborate fix where the external calls that need access to a
'manifest' access a copy that's updated during a manually triggered manifest
update callback.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3883>
Rename track_dequeue_data_locked() to
gst_adaptive_demux_track_dequeue_data_locked(), since it's non-static.
Make find_stream_for_track_locked() static since it's only used in the main
gstadaptivedemux.c file.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3883>
gst_adaptive_demux2_stream_finish_download() will already schedule another
fragment download if it can so don't fall through to the retry code that will
also try and schedule a download (triggering an assert).
Fix the logic in general to retry advancing into the live seek range once.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3883>
When calculating the seek range for a live stream, use the same hold-back logic
as when choosing a starting segment, including low-latency segments if
enabled. Permits seeking closer to the live edge when re-synching or catching
up.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3883>
