Two RTP Header extensions are very relevant for rtprtxsend/receive.
1. "urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id": will always be removed
2. "urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id": will be written
instead of the "rtp-stream-id" header extension.
Currently it's only a simple replacement of one header extension for
another however a future change would only add the relevant extension
based on some heuristics (like, video frames only on one of the rtp key
frame buffers, or only until the rtx ssrc has been validated by the peer)
in order to reduce the required bandwidth.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1759>
When syncing to an RFC7273 clock this will add the original
reconstructed reference clock timestamp to buffers in form
of a GstReferenceTimestampMeta.
This is useful when we want to process or analyse data based
on the original timestamps untainted by any local adjustments,
for example reconstruct AES67 audio streams with sample accuracy.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1964>
When syncing to an RFC7273 clock this will add the original
reconstructed reference clock timestamp to buffers in form
of a GstReferenceTimestampMeta.
This is useful when we want to process or analyse data based
on the original timestamps untainted by any local adjustments,
for example reconstruct AES67 audio streams with sample accuracy.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1964>
When syncing to an RFC7273 clock this will add the original
reconstructed reference clock timestamp to buffers in form
of a GstReferenceTimestampMeta.
This is useful when we want to process or analyse data based
on the original timestamps untainted by any local adjustments,
for example reconstruct AES67 audio streams with sample accuracy.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1964>
Previously the result of the calculations included inaccuracies caused
by the NTP clock estimation, which caused the timestamps to jitter
+/- 1/clockrate.
By reorganizing the calculations it is possible to get rid of this
inaccuracy and calculate deterministic and exact packet timestamps based
on the actual NTP clock as long as the estimation is not off by more
than 2**31 clockrate units.
The only remaining inaccuracy that is introduced now is caused by the
conversion from the NTP clock to the pipeline clock.
Also split up debug output, demote many messages to the trace debug
level and output more intermediate results.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1955>
This is difficult to encounter in ordinary networks, but is
encountered when using tc-netem to add random delays to packets, and
also when your UDP stream is bonded over multiple links with varying
characteristics.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1952>
Gapless playback is handled by adjusting buffer timestamps & durations
and by adding GstAudioClippingMeta.
Support for "Frankenstein" streams (= poorly stitched together streams)
is also added, so that gapless playback support doesn't prevent those
from being properly played.
Co-authored-by: Sebastian Dröge <sebastian@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1028>
Add properties to control input cropping in the V4L2 device.
The input cropping is applied before composing the result to the
capture buffer. By default the capture size will be set to the same
size as the crop region, but it can be scaled to a different output
frame size if supported by the V4L2 device.
If scaling is not supported, the cropped image will
be composed as is into the top-left corner of the capture buffer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1089>
Get the current crop bounding region from the V4L2 device so
that it can be provided to applications and used to validate
crop settings. Also make the default crop region available so
that it can be used to reset the crop when appropriate.
Uses the selection API when available with fallback to the crop
API for older kernels.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1089>
The gst_v4l2_object_set_crop() is used for removing buffer
alignment padding. Give it a name that better reflects
that usage. This helps to distinguish from cropping of the
input image (e.g. cropping at the image sensor on a captre
device), which can be unrelated to the memory buffer padding,
especially if scaling is involved.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1089>
This patch fixes a seg.fault in gst_structure_new() with warnings as below.
GLib-GObject-WARNING **:
../gobject/gtype.c:4330: type id '0' is invalid
GLib-GObject-WARNING **:
can't peek value table for type '<invalid>' which is not currently referenced
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1918>
The point here is that rtpsession will create a new rtpsource when
the field "rtx-ssrc" is present, and when not doing rtx, that means
a random ssrc will create a new rtpsource that will be included in RTCP
messages for the current session.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1882>
When doing only a single stream of audio/video this hardly matters,
but when doing many at the same time, the fact that you have to get
a hold of the glib global type-system lock every time you process a buffer,
means that there is a limit to how many streams you can process in
parallel.
Luckily the fix is very simple, by doing a cast rather than a full
type-check.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1873>
There is a chance that pool->buffers[index] sets BUFFER_STATE_QUEUED, but
it has not been queued yet which makes pool->buffers[index] still NULL.
At this time, if pool_streamff release all buffers with BUFFER_STATE_QUEUED
state regardless of whether the buffer is NULL or not, it will cause segfault.
To fix this, also check buffer when streamoff release buffer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1842>
When the size of V4L2 capture or output is changes with VIDIOC_S_FMT,
the device is only required to update the compisition window to fit
inside the new frame size. This can result in captured data only being
updated on a portion of the frame after a resize.
Update the composition window to the default value determined by the
V4L2 device driver whenever the format is changed to make sure that
all image data is composed to its full size.
Fixes#765
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1806>
Not having this field is equivalent with it being 1/1 so consider
it like that. The generic caps functions are not aware of these
semantics and would consider the caps different, causing a negotiation
failure when caps are changing from caps with to caps without or the
other way around.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1826>
Not having these fields is equivalent with them being mono/0 so consider
them like that. The generic caps functions are not aware of these
semantics and would consider the caps different, causing a negotiation
failure when caps are changing from caps with to caps without or the
other way around.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1826>
If the tracks element was parsed from the SeekEntry, don't
parse it a second time and recreate tracks, as this
loses any tags that were read using the seek table.
If a genuinely new Tracks element is found, do read that
as it is needed for MSE support.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1798>
Do not maintain similar build instructions within each gst-plugins-*
subproject and the subproject/gstreamer subproject. Use the build
instructions from the mono-repository and link to them via hyperlink.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1743>
LOAD macro relies in m7 being zero for interleaving purposes. Using LOAD
on the m7 register makes it interleave with its new content instead of
with 0.
The effect of this bug was bobbing on some static lines that appeared
over fast-moving content.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1816>
The dynamic resolution changes when
the sequence starts when the decoder detects a coded frame with one or
more of the following parameters different from those previously
established (and reflected by corresponding queries):
1.coded resolution (OUTPUT width and height),
2.visible resolution (selection rectangles),
3.the minimum number of buffers needed for decoding,
4.bit-depth of the bitstream has been changed.
Although gstreamer parser has parsed the stream resolution.
but there are some case that we need to handle resolution change event.
1. bit-depth is different from the negotiated format.
2. the capture buffer count can meet the demand
3. there are some hardware limitations that the decoded resolution may
be larger than the display size. For example, the stream size is
1920x1080, but some vpu may decode it to 1920x1088.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1381>
v4l2videodec do some refactoring so that it can support
dynamic resolution change event.
1.wrap the setup process of capture as a function,
as decoder need setup the capture again when
dynamic resolution change event is received.
2.move the function "remove_padding"
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1381>
This allows downstream of a payloader to know the RTP header's marker
flag without first having to map the buffer and parse the RTP header.
Especially inside RTP header extension implementations this can be
useful to decide which packet corresponds to e.g. the last packet of a
video frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1776>
osxaudiodeviceprovider now probes devices more than once to determine
if the device can function as both an input AND and output device.
Previously, if the device provider detected that a device had any output
capabilities, it was treated solely as an Audio/Sink. This causes issues
that have both input and output capabilities (for example, USB interfaces
for professional audio have both input and output channels). Such devices
are now listed as both an Audio/Sink as well as an Audio/Source.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1385>
The RTP payload seems to be required as it carries the frame count
information. Also, gst_rtp_base_payload_allocate_output_buffer had
the second argument incorrect.
Strangely some devices like Shanling MP4 and Sony XM3 would still
work without this while some like the Sony XM4 do not.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1797>
As this element is single threaded, we only need to stop the objects to
allow changing the format again. Fixes assertion notably on shutdown and
on some other situation where the format may be set twice without
actually activating the element.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1567>
If the video4linux device supports norms but has no norm set, norm is
returned as an uninitialized variable after the ioctl call, leading to
gst_v4l2_tuner_get_norm_by_std_id() returning a random norm from the
supported norms. Catch this case and instead return NULL to indicate
that no norm is setup.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1625>
Each stream may have its own segment timeline
(i.g., different segment.start or segment.base)
depending on edit-list and composition-to-decode atom.
Make sure whether time position of a stream has been actually
far behind than that of current target stream.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1352>
The libjpeg-turbo internal state might not be correctly initialized for
the first frame in a stream, pull the frame stride from gstreamer frame
metadata instead, which is correct even for the first frame, and which
makes this code consistent with the surrounding lines.
Fixes: e6d83d8f96 ("jpegdec: Support libjpeg-turbo colorspace conversion")
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1687>
It is imperative that the libjpeg-turbo state is properly initialized
before jpeg_start_decompress() is called. Make sure cinfo.out_color_space
and cinfo.raw_data_out are set to their final values matching their peer
caps before calling jpeg_start_decompress().
Fixes: e6d83d8f96 ("jpegdec: Support libjpeg-turbo colorspace conversion")
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1687>
Pull out peer caps checking code into gst_jpeg_turbo_parse_ext_fmt_convert().
This code is used by libjpeg-turbo extras to determine whether peer is capable
of handling buffers into which libjpeg-turbo can directly decode data. This
kind of check must be performed before jpeg_start_decompress() is called in
gst_jpeg_dec_prepare_decode() as well as in gst_jpeg_dec_negotiate(), hence
the common code.
This commit does modify the code a little to make it easier to call from both
call sites without much duplication, hence the extra `if (*clrspc)` test.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1687>
This reverts commit 2aa2477208.
The commit is completely wrong, libjpeg-turbo is perfectly capable
of decoding I420 (YUV) to RGB. The test case provided alongside the
aforementioned commit passes without this revert because it decodes
image of JCS_YCrCb color space, so the new `if (clrspc == JCS_RGB)`
condition is false on that image, and the libjpeg-turbo decoding
does not get used. The real bug is hidden by that commit.
The real problem is in the call order of gst_jpeg_dec_prepare_decode()
and gst_jpeg_dec_negotiate(). The gst_jpeg_dec_prepare_decode() calls
jpeg_start_decompress() which sets up internal state of the libjpeg,
however, neither cinfo.out_color_space nor cinfo.raw_data_out are
set correctly yet. Those two are set up in gst_jpeg_dec_negotiate()
which is called a bit later. Therefore, the real fix is the set up
cinfo.out_color_space and cinfo.raw_data_out before calling
jpeg_start_decompress(). This is however a separate patch.
Fixes: 2aa2477208 ("jpegdec: only allow conversions from RGB")
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1687>
Constantly updating the ts_offset results in audiable glitches
when streaming audio using ntp-sync=true. By requiring a minimum
offset before updating ts_offset this can be mitigated. Added a
parameter which can be used to set min_ts_offset in ntp-sync mode.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1409>
Hotdoc should be able to extract and parse comments out of these. Just
need to be careful to only add the glob in directories that actually
contain *.m (objc) and *.mm (objcpp) files.
Also fix some doc comments and remove redundant ones.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1614>
This is listed as a public interface implemented by osxaudio, so we
need to mark it as a plugin API so that it's listed in the
documentation correctly.
This is an ancient symbol, so add it to the symbol index too.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1601>
Starting with libsoup3, there is no attempt to handle thread safety
inside the library, and it was never considered fully safe before
either. Therefore, move all session handling into its own thread.
The libsoup thread has its own context and main loop. When some
request is made or a response needs to be read, an idle source
is created to issue that; the gstreamer thread issuing that waits
for that to be complete. There is a per-src condition variable to
deal with that.
Since the thread/loop needs to be longer-lived than the soup
session itself, a wrapper object is provided to contain them. The
soup session only has a single reference, owned by the wrapper
object.
It is no longer possible to force an external session, since this
does not seem to be used anywhere within gstreamer and would be
tricky to implement; this is because one would not have to provide
just a session, but also the complete thread arrangement made in
the same way as the system currently does internally, in order to
be safe.
Messages are still built gstreamer-side. It is safe to do so until
the message is sent on the session. Headers are also processed on
the gstreamer side, which should likewise be safe.
All requests as well as reads on the libsoup thread are issued
asynchronously. That allows libsoup to schedule things with as
little blocking as possible, and means that concurrent access
to the session is possible, when sharing the session.
Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/947
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1555>
This is due to an unsafe usage of the pad task. We didn't ensure proper
ownership of the task. That race involved the task being released too early,
and was detected, luckily, by the glib mutex implementationt that
reported the mutex being disposed while being locked.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1478>
On Android (especially) and for static builds in general it is safer to link
against libsoup and have the dynamic custom loading disabled. For those cases we
can safely assume the application will use either libsoup2 or libsoup3 and not
both.
Fixes#939
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1536>
When multiple streams are bundled together, there may be more
than one red payload type to handle.
In addition, as the red decoder works by filling in gaps in
the seqnums, there needs to be one rtp_history queue per sequence
domain.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1429>
In redenc, when input buffers have a header for the TWCC extension,
we now add one to our wrapper buffers.
In ulpfecenc we add one in that case to our protection buffers.
This makes TWCC functional when UlpRed is used in webrtcbin.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1414>
In order to allow "level-asymmetry-allowed" we now handle a new
"profile" field, which as the same semantics as the "profile" field in
H.264 stream so that we can force payloaded stream to have the right
format when using the `gst_sdp_media_get_caps_from_media` to set caps
filter after the payloader. This allows a simple negotiation in standard
RTP negotiation based on SDPs (like webrtc) for that particular case,
closely respecting the specs.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1410>
XNextEvent() blocks indefinitely in absence of X11 events, which can
prevent the pipeline from stopping.
This can cause problems when ximagesrc is used in "remote desktop"
scenarios and the GStreamer application itself, through which the user
is viewing and controlling the machine, is the only source of input
events.
Replace the call with non-blocking XCheckTypedEvent().
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1438>
since `gst_caps_replace()` and `gst_pad_set_caps()` both ref the caps and neither of them takes the ownership of the caps -> it must be unreffed in `gst_multi_file_src_set_property()`
to test the leak (on Unix): `echo coucou > /tmp/file.txt && GST_TRACERS=leaks GST_DEBUG="GST_TRACER:7" gst-launch-1.0 multifilesrc location=/tmp/file.txt caps='txt' ! fakesink`
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1436>
Also ignore 0x0 sizes in the fallback case and assume the size can be
anything between 1x1 and MAXxMAX.
This fixes the case where a width=0, height=0 caps are created. Whith
this patch the caps will contain width=[1,MAX], height=[1,MAX].
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1396>
Follow-up on 97d83056b3, only check
for intersection with the current srccaps when checking if a sinkpad
can accept caps.
I must have been lucky in my firefox testing then, and always entered
the code path with audio getting negotiated first, thus not failing
the is_subset check when srccaps had been negotiated as
application/x-rtp, and an accept-caps query was made for the video
caps with a defined extmap.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1384>
If qt asks us to redraw before we have both set a buffer and caps we
would attempt to use the new caps with the old buffer which could result
in bad things happening.
Only update caps from new_caps once the buffer has actually been set.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1346>
A previous patch has caused rtpfunnel to output twcc-related
information downstream, however this leaked into upstream
negotiation (through funnel->srccaps), causing payloader to
negotiate twcc caps even when not prompted to do so by the user.
Fix this by only enforcing that upstream sends us application/x-rtp
caps as was the case originally.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1278>
Negative composition time offsets are only allowed with version 1 of the
box, however we parse it as a signed value also for version 0 boxes as
unfortunately there are such files out there and it's unlikely to have
(valid) huge composition offsets.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1294>
g_sequence_remove_range's end iter is exclusive, so if one
wants to remove that item as well, it should be called with
the next iter.
This could in theory fix an issue where:
* The sequence isn't entirely trimmed, with an old item lingering
* Following FEC packets are immediately discarded because they
arrived later than corresponding media packets, long enough for
seqnums to wrap around
* We now try to reconstruct a media packet with a completely obsolete
FEC packet, chaos ensues.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1341>
We need to hold onto the last buffer until the next buffer arrives.
Before, if a caps change comes we would remove the currently rendering
buffer. if Qt asks use to render something, we would render the dummy
black texture.
Fixes a period of black output when upstream is e.g. changing resolution
as in hls adaptive bitrate scenarios.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1338>
The `gst_v4l2_buffer_pool_dqbuf` function contains this ominous comment:
/* get our GstBuffer with that index from the pool, if the buffer was
* outstanding we have a serious problem.
*/
outbuf = pool->buffers[group->buffer.index];
Unfortunately it is common for buffers in _output_ buffer pools to be
both queued and outstanding at the same time. This can happen if the
upstream element keeps a reference to the buffer, or in an encoder
element itself when it keeps a reference to the input buffer for each
frame.
Since the current code doesn't handle this case properly we can end up
with crashes in other elements such as:
(gst-launch-1.0:32559): CRITICAL **: 17:33:35.740: gst_video_frame_map_id: assertion 'GST_IS_BUFFER (buffer)' failed
and:
(gst-launch-1.0:231): GStreamer-CRITICAL **: 00:16:20.882: write map requested on non-writable buffer
Both these crashes are caused by a race condition related to releasing
the same buffer twice from two different threads. If a buffer is queued
and outstanding this situation is possible:
**Thread 1**
- Calls `gst_buffer_unref` decrementing the reference count to zero.
- The core GstBufferPool object marks the buffer non-outstanding.
- Calls the V4L2 release buffer function.
- If the buffer is _not_ queued:
- Release it back to the free pool (containing non-queued buffers).
**Thread 2**
- Dequeues the queued output buffer.
- Marks the buffer as not queued.
- If the buffer is _not_ outstanding:
- Calls the V4L2 release buffer function.
- Release it back to the free pool (containing non-queued buffers).
If both of these threads run at exactly the same time there is a small
window where the buffer is marked both not outstanding and not queued
but before it has been released. In this case the buffer will be freed
twice causing the above crashes.
Unfortunately the variable recording whether a buffer is outstanding is
part of the core `GstBuffer` object and is managed by `GstBufferPool` so
it's not as straightforward as adding a mutex. Instead we can fix this
by additionally recording the buffer state in `GstV4l2BufferPool`, and
handle "internal" and "external" buffer release separately so we can
detect when a buffer becomes not outstanding.
In the new solution:
- The "external" buffer pool release and the "dqbuf" functions
atomically update the buffer state and determine if a buffer is still
queued or outstanding.
- Subsequent code and a new
`gst_v4l2_buffer_pool_complete_release_buffer` function can proceed to
release (or not) a buffer knowing that it's not racing with another
thread.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1010>
The pipeline flow for receiving looks like this:
rtpsession ! rtpssrcdemux ! session_fec_decoder ! rtpjitterbuffer ! \
rtpptdemux ! stream_fec_decoder ! ...
There are two places where a fec decoder could be placed.
1. As requested from the 'request-fec-decoder' signal: after rtpptdemux
for each ssrc/pt produced
2. after rtpssrcdemux but before rtpjitterbuffer: added for the
rtpst2022-1-fecenc/dec elements,
However, there was some cross-contamination of the elements involved and
the request-fec-decoder signal was also being used to request the fec
decoder for the session_fec_decoder which would then be cached and
re-used for subsequent fec decoder requests. This would cause the same
element to be attempted to be linked to multiple elements in different
places in the pipeline. This would fail and cause all kinds of havoc
usually resulting in a not-linked error being returned upstream and an
error message being posted by the source.
Fix by not using the request-fec-decoder signal for requesting the
session_fec_decoder and instead solely rely on the added properties for
that case.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1300>
