The original BUNDLE support commit placed a queue after the
rtpfunnel that combines streams, but I don't see a good reason for
it. It has default settings, so if network output is slow might
accidentally store up to 1 second of pending data, increasing
latency.
Remove it in favour of doing any necessary buffering before
webrtcbin. If it turns out that there is a reason for it to
exist, the limits should probably be configurable and small.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3437>
Currently, when rtspsrc property add-reference-timestamp-metadata=true,
a downstream rtph264depay element will attach multiple copies of the
same GstReferenceTimestampMeta to the depayloaded media buffers. This
can have signficant performance impacts further downstream in a pipeline
like the following:
rtspsrc add-reference-timestamp-metadata=true ! rtph264depay ! h264parse ! ... ! rtph264pay ! ...
For example, if there are 10 packet buffers for a frame of RTP H.264
video, each of those packet buffers will contain the same reference
timestamp meta. The rtph264depay element will then attach all 10
metadata to the depayloaded frame. And then later when we payload the
frame buffer again for proxying, we now have 10 more buffers each with
10 instance of the same metadata. Allocating/deallocating 100+ instances
of metadata @ 30fps for multiple streams has a pretty large performance
impact.
Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1578
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3431>
The tile width in pixel is not always available. Notably for
8L128 10bit format, the tile is 8x128 bytes, and the pixel
format is fully packed. That means that the tile contains at
least 6 pixels per line, but it also hold some bits of the
pixel from the same line on the previous or next tile.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3424>
In current tile representation, only tiles with power of two
width and height in bytes are supported. This limitation
prevents adding more complex tiles formats.
In this patch, we deprecate tile_ws and tile_hs from GstVideoFormatInfo and
replace if with an array of GstVideoTileInfo. Each plane tiles are then
described with their pixels width/height, line stride and total size.
The helper gst_video_format_info_get_tile_sizes() that depends on the
deprecated API is also being removed. This can simply be removed as it wasn't
in any stable release yet.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3424>
Setting force_live lets aggregator behave as if it had at least one of
its sinks connected to a live source, which should let us get rid of the
fake live test source hack that is probably present in dozens of
applications by now.
+ Expose API for subclasses to set and get force_live
+ Expose force-live properties in GstVideoAggregator and GstAudioAggregator
+ Adds a simple test
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3008>
The only case where we definitely need to write a new trun is when the
data_offset value does not match the end of the list of entries.
Needing multiple trun atoms is required when interleaving multiple
streams together.
All other cases can be covered by adding more entries to the existing
trun atom.
Fixes playback of fragemented mp4 in ffplay and chrome.
Using e.g. mp4mux fragment-duration=1000 fragment-mode=dash-or-mss
and
mp4mux fragment-duration=1000 fragment-mode=first-moov-then-finalise
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3426>
Attribute's value should use returned value from get_attribute for
VAConfigAttribRTFormat, since VAProfileHEVCMain10, in AMD Mesa Gallium,
can process either VA_RT_FORMAT_420 and VA_RT_FORMAT_420_10, which isn't
considered in gstreamer-vaapi design, where encoder's src pads will
expose only 4:2:0 color formats but no 4:2:0 10bit. So, this is a workaround
for this issue while new vah265enc is released.
Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3397>
This was the intention from the start, just took me a few years *cough* to
actually implement it properly.
Gapless is handled by re-using as much as possible the same decoders and sinks
if present, and only pre-rolling switching at the sources level (with buffering
if/when needed).
In order to enable "gapless" playback, the "next" uri should be set at any time
between the moment the `about-to-finish` signal is emitted and the moment the
current play item is done. Previously this could only be done with the signal
emission.
This new implementation also allows "Instantaneous URI switching". This allows a
much faster way of switching playback entries while re-using as many elements as
possible. To enable this set `instant-uri` property to TRUE, the default being
FALSE.
API: instant-uri properties
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2784>
When dealing with gapless input (i.e. streams with changing group-id in
GST_EVENT_STREAM_START), we need to take into account the elapsed
running-time (if applicable) in order to properly calculate levels and output
time. Without doing this all incoming data from future groups would be
considered as being "late" and would be consumed immediately.
This does **NOT** modify the actual segment and buffer times, and is only used
internally.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2784>
DecodebinInput (and their backing parsebin or identity) are no longer released
when the corresponding sinkpad is unlinked, but when it's released.
The parsebin element will be resetted:
* If incoming caps are incompatible (was the case before)
* Or when unlinking and it was previously pull-based
This opens the way to use decodebin3 with changing inputs (i.e. gapless)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2784>
Introduce the option to have the streams be parsed with `parsebin` for
compatible sources (i.e. which are eligible for buffering in the same way as
before this commit).
By parsing the inputs directly, this allows more accurate buffering control:
* Instead of relying on potential bitrate information coming from somewhere
* and *without* being linked downstream
If `parse-streams` is activated and the stream is eligible for buffering, then a
`multiqueue` will be used on the output of `parsebin` in order to handle the
buffering.
API: `parse-streams`
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2784>
If the incoming streams are already parsed, there is no need to add yet-another
parsebin to process it *IF* that stream is compatible with a decoder or the
decodebin3 output caps.
This only applies if all the following conditions are met:
* The incoming stream can *NOT* do pull-based scheduling
* The incoming stream provides a `GstStream` and `GstStreamCollection`
* The caps are compatible with either the decodebin3 output caps or a decoder
input
If all those conditions are met, a identity element is used instead of a
parsebin element and the same code paths are taken.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2784>
* Instead of creating temporary `PendingPad` structures, always create a
DecodebinInputStream for every pad of parsebin
* Remove never used `pending_stream` field from DecodebinInputStream
* When unblocking a given DecodebinInput (i.e. wrapping a parsebin), also make
sure that other parsebins from the same GstStreamCollection are unblocked
since they come from the same source
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2784>
Make an explicit topology/tree of structures:
* ChildSrcPadInfo is created for each source element source pad
* ChildSrcPadInfo contains the chain of optional elements specific to that
pad (ex: typefind)
* A ChildSrcPadInfo links to one or more OutputSlot, which contain what is
specific to the output (i.e. optional buffering and ghostpad)
* No longer use GObject {set|get}_data() functions to store those structures and
instead make them explicit
* Pass those structures around explicitely in each function/callback
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2784>
The following problem could happen:
* Thread 1 : urisourcebin gets activated from READY->PAUSED
* Thread 2 : some element causes a pad to be added to urisourcebin , which gets
linked downstream, which decides to activate upstream to pull-based.
* That requires "activating" the pads from PUSH to NONE, and then from NONE to PULL
* Thread 1 : the base class state change handlers checks if all pads are
activated
The issue is that since going form PUSH to PULL requires going through NONE,
there is a window during which:
* Thread 1 : The pad was set to NONE (before being set to PULL)
* Thread 2 : The base class activates that pad (to PUSH)
* Thread 1 : The attempt to "activate" to PULL fails (silently or not)
This is very racy, so in order to avoid that, we make sure that we only add pads
once the transition from READY->PAUSED in the parent classes is done.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2784>
This change allow output caps to be updated even though we stay in
streaming state. This is needed so that any upstream updated to fields
like framerate, hdr data, etc. can result in a downstream caps event
being pushed.
Previously, any of these changes was being ignored and the downstream
caps would not reflect it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3328>
In theory, input caps can be updated anytime at non-keyframe or
sequence boundary, such as HDR10 metadata, framerate, aspect-ratio
or so. Those information update might not trigger ::new_sequence()
or subclass may ignore the changes.
By this commit, input state change will be tracked by baseclass
and subclass will be able to know the non-decoding-essential
update by checking the codec specific picture struct
on ::output_picture()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3328>
This reverts commit fcad4cc646.
This was wrong is so many ways.
* The memcmp was badly used (it should use == 0 to check the data is identical,
and not != 0)
* There was no boundary checks on the present stream section_data when passing
it to memcmp.
* The return value should have been TRUE (i.e. we have done all checks, none of
them failed, therefore the section has been seen before)
* stream->section_data would *always* be NULL if the section had already been
processed
Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1559
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3421>
There are cases where upstream will not provide a framerate, or it won't be
fixed. But if there is latency introduced by the decoder we do want to report
it.
Therefore use the framerate stored in the actual decoder, which will have a
default.
Fixes hangs when playing back such streams with decodebin3 (where the multiqueue
will not have been informed of that downstream latency and not grow accordingly)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3391>
Postpone the cleanup of any consecutive sequence of lost frames
which starts at frame 0, until frame 100 is dequeued from driver.
This allows fluster tests JVT/CVWP2_TOSHIBA_E, JVC/CVWP3_TOSHIBA_E
and HEVC/POC_A_Bossen_3 that sends out-of-order frames to successfully
complete (e.g., test of Amphion vpu driver).
Fixes#1569
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3398>
In order to figure out if the "raw" audio contained within the wav
container is actually DTS, wavparse calls the typefinder helper
except that means it runs all typefinders.
Since it only cares about checking for DTS, we should only run the
audio/x-dts typefinder (if present). Commit 858e516 did not really
fix things.
Use the new type helper with the caps to fix this.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3417>
Set udpsrc seqnums on all events sent to the udpsrc's, and before
forwarding events out of rtspsrc set the latest seek seqnum on them if
any.
Also produce a consistent seqnum in rtspsrc from the very beginning.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3409>
Rewriting GstCudaConverter object, since the old implementation was not
well organized and it's hard to add new features.
Moreover, the conversion operations were not very optimized.
Major change of this implementation:
* Remove redundant intermediate conversion operations such as
any RGB -> ARGB(64) conversion or any YUV -> Y444 (or 16bits Y444).
That's not required most of cases. The only required case is
converting 24bits (such as RGB/BGR) packed format to 32bits format
because CUDA texture object does not support sampling 24bits format
* Use normalized sample fetching (i.e., [0, 1] range float value)
and also normalized coordinates system for CUDA texture.
It's consistent with the other graphics APIs such as Direct3D
and OpenGL, that makes sampling operations much easier.
* Support a kind of viewport and adopt math for colorspace conversion
from GstD3D11 implementation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3389>
GstCudaConverter object can do colorspace conversion and scale at once.
Adding new element "cudaconvertscale" to do that, this can
save unnecessary GPU operation if colorspace conversion and
rescale is required for given input stream format.
Most of codes are taken from d3d11convert element
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3389>
Introduce a new API that can return a GstTypeFind * with helper functions
and data set around buffer data.
While at it, drop factory field from GstTypeFindBufHelper. While it was
useful for logging, it was not passed through function arguments and keeping
it for logging would require an additional API increasing the API surface
and making it harder to use.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3296>
Instead of returning a "const gchar" or a "gchar" that should not be freed, always
return a duplicated string as those functions were used together with g_strdup anyway.
This is needed to prepare support for returning modified strings in next commit.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1147>
This is small regression from commit f7abd81a.
When calling `gst_element_query()` no pad is associated with that query, but the
current code always forwards the query to the associated pad, which is NULL in
previous case. This patch checks for the pad before forwarding the query.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3404>
If we don't receive any data from usrsctp, then there will be no src pad
for the stream id and the stream reset will fail to remove the relevant
src pad. Workaround by first attempting to add the relevant src pad, then
almost immediately removing it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3381>
Replace video_copy with memcpy to fix the issue when the sizes of the
src frame and dst frame don't match. Moreover, for Windows, you have to
do the copy first and call gst_msdk_import_to_msdk_surface later.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3231>
Replace video_copy with memcpy to fix the issue when the sizes of the
src frame and dst frame don't match. Moreover, for Windows, you have to
do the copy first and call gst_msdk_import_to_msdk_surface later.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3231>
Currently MSDK context does not support d3d11va. Now introduce d3d11va
device to MSDK context, making it able to create msdk session with d3d11
device and to easily share with upstream and donwstream.
Add environment variable to enable user to choose GPU device in multi-GPU
environment. This variable is only valid when there's no context
returned by upstream or downstream. Otherwise it will use the device
that created by upstream or downstream.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3231>
Add support for more formats so as to run the libvpx high bit depth test suite.
This means the files under CONFIG_VP9_HIGHBITDEPTH
This also allows running the yuv444p 8bit file in the regular 8 bit vp9 suite.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3356>
If an input is malformed (only produces cea608 field 1 cc_data) then
when in passthrough we would effectively be dropping every second cea608
on output as we would not store any unused cea608 data.
Fix by having all code paths go through the framerate conversion code
which will store and retrieve any relevant data across buffers.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3211>
When a tile format is padded and imported as DMABuf, the stride
contains the information about the actual width and height in
number of tiles. This information is needed by the detiling shader
in order accuratly calculate the location of pixels. To fix that,
we also copy the offset and strides into the otuput format and
the converter will ensure that the shader is recompiled whenever
the stride changes.
This fixes video corruptions observed when decoding on MT8195
with videos that aren't not aligned to 64bytes in width.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3365>
