Add 5 new navigation event types for touchscreen events, with the same
naming and meaning as in libinput - touch-down, touch-motion, touch-up,
touch-frame and touch-cancel - as well as constructors and parse
functions for them.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1633>
Add a function to get x/y coordinates from suitable navigation events,
and one to create a copy with given coordinate values.
For e.g. translating event coordinates, this avoids having to either
switch on the event type to select the right parse function, or
having to rely on implementation details of the underlying event
structure.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1633>
This deprecates the current send_event interface, and the wrapper
functions based on it, replacing it with a send_event_simple interface and
wrapper function. Together with the new event constructors, this avoids
implementations having to directly access the underlying structure.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1633>
Since the strings are empty for GST_MSDK_CAPS_MAKE_WITH_DMABUF_FEATURE
and GST_MSDK_CAPS_MAKE_WITH_VA_FEATURE, when excuting
gst-inspect-1.0.exe msdkh265enc, there will be convert static caps error
because of the extra semicolon between two empty strings. Now macro
definitions are added to avoid this issue.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2004>
Pass the current frame to the duplicate_picture callback. This makes it easier
to set the frame's output_buffer if we already have one available. Also
documented that unlike VP9, it is not optional to implement this as the
picture will populate the DPB if it is a key-frame. To ensure this, remove the
default implementation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1992>
The system_frame_number is notably used by V4L2 decoder as a unique
indentifier for the frame that was decoded. This value is used to tell driver
which frame to reference, as V4L2 does not have an efficient mechanism to
otherwise pass back the frames.
For this reason, and because it is more ligical, copy the original
system_frame_number into the duplicate picture instead of using the current
frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1992>
Two RTP Header extensions are very relevant for rtprtxsend/receive.
1. "urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id": will always be removed
2. "urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id": will be written
instead of the "rtp-stream-id" header extension.
Currently it's only a simple replacement of one header extension for
another however a future change would only add the relevant extension
based on some heuristics (like, video frames only on one of the rtp key
frame buffers, or only until the rtx ssrc has been validated by the peer)
in order to reduce the required bandwidth.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1759>
Showing existing keyframe have special meaning in AV1. All the references
frame will be refreshed with the original keyframe information. The refresh
process (7.20) is implemented by saving data from the frame_header into the
state. To fix this special case, load all the relevant information into the
frame_header.
As there is nothing happening in between this and the loading of the key-frame
into the state, this patch also remove the separate API function, using it
internally instead.
Fixes#1090
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1971>
We need to parse the payload type map provided by the offer SDP and
set those values on the payloader, otherwise webrtcbin will create
a recvonly answer SDP and we won't send anything to the browser.
Fixed it for both C and Python sendrecv examples.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1864>
Earlier, the example only supported one negotiation mode:
* Browser client is running, gstreamer starts a call and sends offer
Now these three modes are also supported:
* Browser client is running, gstreamer starts a call and sends an
offer request
* gstreamer connects and waits for browser client to start a call and
send an offer
* gstreamer connects and waits for browser client to start a call and
send an offer request
The following features are still missing:
* Data channel support
* TWCC support + stats logging
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1864>
The documentation for several gst_*_writable_structure functions stated
that they would never return NULL, without making clear that the passed
object is required to be writable. This changes the wording in those
cases to make that requirement more clear.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1784>
When syncing to an RFC7273 clock this will add the original
reconstructed reference clock timestamp to buffers in form
of a GstReferenceTimestampMeta.
This is useful when we want to process or analyse data based
on the original timestamps untainted by any local adjustments,
for example reconstruct AES67 audio streams with sample accuracy.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1964>
When syncing to an RFC7273 clock this will add the original
reconstructed reference clock timestamp to buffers in form
of a GstReferenceTimestampMeta.
This is useful when we want to process or analyse data based
on the original timestamps untainted by any local adjustments,
for example reconstruct AES67 audio streams with sample accuracy.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1964>
When syncing to an RFC7273 clock this will add the original
reconstructed reference clock timestamp to buffers in form
of a GstReferenceTimestampMeta.
This is useful when we want to process or analyse data based
on the original timestamps untainted by any local adjustments,
for example reconstruct AES67 audio streams with sample accuracy.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1964>
We bind transceivers' fec_percentage property to the FEC encoder
percentage property, and with the binding bidirectional a deadlock
was introduced by the latest changes from !1762:
We take hold of the transceiver's object lock, then add the binding
and set the property to its initial value on the encoder, which causes
set_property to deadlock in the transceiver when the binding kicks in.
Changing the binding type to DEFAULT (source to target) is enough
to address the deadlock and still serves the original intent.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1967>
Previously the result of the calculations included inaccuracies caused
by the NTP clock estimation, which caused the timestamps to jitter
+/- 1/clockrate.
By reorganizing the calculations it is possible to get rid of this
inaccuracy and calculate deterministic and exact packet timestamps based
on the actual NTP clock as long as the estimation is not off by more
than 2**31 clockrate units.
The only remaining inaccuracy that is introduced now is caused by the
conversion from the NTP clock to the pipeline clock.
Also split up debug output, demote many messages to the trace debug
level and output more intermediate results.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1955>
This is difficult to encounter in ordinary networks, but is
encountered when using tc-netem to add random delays to packets, and
also when your UDP stream is bonded over multiple links with varying
characteristics.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1952>
Holding previously decoded but not outputted pictures even after
new_sequence is not a safe approach in various aspect.
However, we cannot drain out DPB on new_sequence() unconditionally,
because there is a case where decoder should drop decoded pictures
if NoOutputOfPriorPicsFlag is set.
To detect NoOutputOfPriorPicsFlag before the new_sequence() call,
this patch splits decoding process into two path, one for nal unit parsing
in order to detect NoOutputOfPriorPicsFlag and then each nal unit
will be decoded.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1937>
Gapless playback is handled by adjusting buffer timestamps & durations
and by adding GstAudioClippingMeta.
Support for "Frankenstein" streams (= poorly stitched together streams)
is also added, so that gapless playback support doesn't prevent those
from being properly played.
Co-authored-by: Sebastian Dröge <sebastian@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1028>
1. Always set the according GstVaH264EncFrame pointer when GstVideoCodecFrame
pointer is assigned, which can make the logic safe.
2. Fix the forgotten change in _sort_by_frame_num. Its input pointer now is
GstVideoCodecFrame type.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1935>
Add properties to control input cropping in the V4L2 device.
The input cropping is applied before composing the result to the
capture buffer. By default the capture size will be set to the same
size as the crop region, but it can be scaled to a different output
frame size if supported by the V4L2 device.
If scaling is not supported, the cropped image will
be composed as is into the top-left corner of the capture buffer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1089>
Get the current crop bounding region from the V4L2 device so
that it can be provided to applications and used to validate
crop settings. Also make the default crop region available so
that it can be used to reset the crop when appropriate.
Uses the selection API when available with fallback to the crop
API for older kernels.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1089>
The gst_v4l2_object_set_crop() is used for removing buffer
alignment padding. Give it a name that better reflects
that usage. This helps to distinguish from cropping of the
input image (e.g. cropping at the image sensor on a captre
device), which can be unrelated to the memory buffer padding,
especially if scaling is involved.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1089>
The default query handler would go through typefind, which by default accepts
any CAPS. But once configured, parsebin can't reconfigure itself, it should
therefore pass through the ACCEPT_CAPS query to the first element after
typefind (if any).
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1900>
Don't reconfigure outputs when the select-streams
event is sent from the app, as the selection may
not take effect for some time. Instead, wait
for the pipeline to confirm the new set of
selected streams when it sends the message.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1900>
If we previously had subtitles coming in, the video
may be chained through a text overlay block. Before,
the code would end up trying to link pads that were
already linked and video would not get reconnected
properly.
To fix that, make sure that the candidate
pads are actually unlinked first. If a textoverlay
is present and no longer needed, it will be cleaned
up later in the reconfiguration sequence.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1900>
Requesting a new pad can start a reconfiguration cycle, where
playsink will block all input pads and wait for data on them
before doing internal reconfiguration. If a pad is released,
that reconfiguration might never trigger because it's now waiting
for a pad that doesn't exist any more.
In that case, complete the reconfiguration on pad release.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1180>
Make it posible to configure the element to obtain the timestamps from
reference timestamp meta data instead of using the ntp-offset property,
or estimating its own offset. Currently the only time format supported
is "timestamp/x-unix", i.e. UTC time expressed in the unix time epoch.
In addition the custom event GstNtpOffset has been renamed to
GstOnvifTimestamp, to reflect that it is not necessarily used to convey
the ntp-offset. As a consequence we had to modify a couple of files in
the rtsp-server as well.
Fixes#984
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1683>
This patch fixes a seg.fault in gst_structure_new() with warnings as below.
GLib-GObject-WARNING **:
../gobject/gtype.c:4330: type id '0' is invalid
GLib-GObject-WARNING **:
can't peek value table for type '<invalid>' which is not currently referenced
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1918>
On GstVideoDecoder::{drain,flush}, we send null packet with
CUVID_PKT_ENDOFSTREAM flag to drain out decoder. Which will
reset CUVID parser as well.
To continue decoding after the drain, the next input buffer
should include sequence headers otherwise CUVID parser will
not report any decodeable frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1911>
There could be a case where the new program has the same program number as the
previous one ... but is actually located on a PID previously used for elementary
stream. In that case the program is guaranteed to not be an update of the
previous program but a completely new one.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1893>
We need to be able to look for programs by their PID also. Using a hash table
was a bit sub-par (and overkill) for storing a range of programs.
This is needed because there could potentially be two programs with the same
program id but different PMT PID (while one is being deactivated the new one
would "exist").
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1893>
This commit modifies the interleave calculation to allow growing when incoming
data is before the segment start.
The rationale is that there is no requirement whatsoever for data before the
segment start to be "coherent" on all streams.
For example, a demuxer could rightfully send data from the video stream from the
previous keyframe (potentially quite a bit before the segment start) and the
audio from just before the segment start.
This will activate the same logic as growing the interleave when some streams
haven't received buffers yet.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1892>
* When a stream receives EOS, it will no longer change, we shouldn't take that
stream into account for interleave calculation.
* When streams (re)appear, we do not want to grow the initial interleave values
to excessive values. Instead of setting it to a default of 5s, progressively
grow it to that maximum.
* When the status of input streams change (i.e. going to/from "some haven't
received data yet" and "all have received data"), update the interleave
immediately instead of waiting for (potentially) 5s of data before updating
it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1892>
The point here is that rtpsession will create a new rtpsource when
the field "rtx-ssrc" is present, and when not doing rtx, that means
a random ssrc will create a new rtpsource that will be included in RTCP
messages for the current session.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1882>
Instead of using GstMiniObject to hold H264 frame, now it uses a plain
structure. Besides, instead of holding a reference to
GstVideoCodecFrame, the H264 frame structure is set as a
GstVideoCodecFrame user data.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1856>
According to va_dec_hevc.h, pic_param->st_rps_bits should be set
for accelorater to skip parsing the *short_term_ref_pic_set
(num_short_term_ref_pic_sets) structure.
Also modified fill_picture to get parser info as a parameter,
in order to get slide_hdr->short_term_ref_pic_set_size.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1886>
Fix a small race where a group can receive stream-start
and post a pending buffering message just as another
thread posts a different buffering message, causing them
to be received by the application out of order. In the
worst case, this leads the application receiving a
stale 99% buffering message and going back to buffering
right after the 100% buffering message.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1840>
Apparently GtkSharp expects each object has only one ToggleRef at any
time. Assigning element.Handle into Raw has a consequence that second
ToggleRef attempts to get created but fails on g_object_unref () that
breaks a GObject assertion:
toggle_refs_notify: assertion failed: (tstack.n_toggle_refs == 1)
This is because toggle references should be removed with
g_object_remove_toggle_ref(), not a simple unref().
In order to avoid duplicate toggle references, introduce
ElementFactory.MakeRaw(), which creates a GstElement without its
accompanying C# object. The returned raw pointer can be assigned into
another GLib.Object without trouble.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1885>
And use the output segment position for the outgoing timestamp while it
is. This is needed to delay the calculation of `output_ts_offset` until
we actually have a usable timestamp, as tsmux will output a few initial
packets while `last_ts` is still unset.
Without this, the calculation would use the initial `0` value, which did
not have the intended effect of making VBR mode behave like CBR mode,
but always calculated an offset equal to the selected start time.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1884>
Fix suppression to support release and debug builds.
Here is the debug build call stack:
```
==10707== by 0x48B5520: g_malloc (gmem.c:106)
==10707== by 0x48D19DC: g_slice_alloc (gslice.c:1069)
==10707== by 0x48D3947: g_slist_copy_deep (gslist.c:619)
==10707== by 0x48D38B8: g_slist_copy (gslist.c:567)
==10707== by 0x4ADC90B: gst_debug_remove_with_compare_func (gstinfo.c:1504)
```
In release build `g_slist_copy (gslist.c:567)` got inlined:
```
==15419== by 0x48963E0: g_malloc (gmem.c:106)
==15419== by 0x48AA382: g_slice_alloc (gslice.c:1069)
==15419== by 0x48AB732: g_slist_copy_deep (gslist.c:619)
==15419== by 0x4A39B8F: gst_debug_remove_with_compare_func (gstinfo.c:1504)
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1814>
When doing only a single stream of audio/video this hardly matters,
but when doing many at the same time, the fact that you have to get
a hold of the glib global type-system lock every time you process a buffer,
means that there is a limit to how many streams you can process in
parallel.
Luckily the fix is very simple, by doing a cast rather than a full
type-check.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1873>
There is a chance that pool->buffers[index] sets BUFFER_STATE_QUEUED, but
it has not been queued yet which makes pool->buffers[index] still NULL.
At this time, if pool_streamff release all buffers with BUFFER_STATE_QUEUED
state regardless of whether the buffer is NULL or not, it will cause segfault.
To fix this, also check buffer when streamoff release buffer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1842>
When tunneling over HTTP, if connection on the second channel happens
before the control timer is created we may trigger an assert in
rtsp_ctrl_timeout_remove(). Avoid that by taking the priv->lock before
attaching the client thread to the context.
Fixes#1025
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1867>
When there is vpp scaling downstream, we need to make sure SFC is not
triggered because vpp may fall into passthrough mode which causes
the decoder negotiation to create src caps with vpp scaled width/height.
This patch includes bitstream's original size in first query with
downstream in gst_msdkdec_src_caps, which is the same for what we do for
color format in this query. This is to ensure SFC scaling starts to
work only when downstream directly asks for a different size instead of
through vpp.
Note that here SFC scaling follows the same behavior as msdkvpp:
if user only changes width or height, e.g. dec ! video/x-raw,width=xx !,
the height will be modified to the value which fits the original DAR.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1838>
* Hide GstCudaMemory member variables
* Make GstCudaAllocator object GstCudaContext independent
* Set offset/stride of memory correctly via video meta
* Drop GST_BUFFER_POOL_OPTION_VIDEO_ALIGNMENT support.
This implementation actually does not support custom alignment
because we allocate device memory via cuMemAllocPitch
of which alignment is almost uncontrollable
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1834>
* cudaupload/download
- Specify only formats actually we can deal with
nvcodec elements, not all video formats
- Supports CUDA output for download and input for upload in order
to make passthrough possible, like other upload/download elements.
* cudabasetransform
- Reset conversion element if upstream CUDA memory
holds different CUDA context and the element can accept it.
This is the same behavior as corresponding d3d11 filter elements.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1834>
This was to support very old V4L2 kernel. As we moved to DMABuf and can now
detach buffers on renegotiation, the buffer it tries to fix no longer exist.
The risk to blocking indefinitly the application does still exist though.
Fixes#1070
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1861>
When we negotiate with downstream, We should use the intersected
caps of input and output to decide the alignment and stream format.
The current code just uses the input caps which may lack the stream
format.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1837>
The demux now outputs the AV1 stream in "tu" alignment, so we do not need
to detect the input alignment. But the annex b stream format is not recognized
by the demux, we still need to detect that stream format for the first input.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1837>
Decoders that required frame aligmment and didn't have an associated
alpha decoder were skipped. This is because the parser was constructing
caps based on the software alpha decoder, which specify super-frame
alignment.
Iterate over the caps to filter the one that have a matching codec-alpha, with
the semantic the no codec-alpha field means codec-alpha=false. Then if
everything was removed, callback to the original, so that the first non-alpha
decoder will be picked.
Fixes#820
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1855>
Currently for copying the coded buffer onto a GStreamer buffer, the
coded buffer is mapped two times: one for getting the size, and later
for do the actual copy. We can avoid this by doing directly in the
element rather than in the general encoder object.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1845>
We need to always add the RTX/RED/ULPFEC elements as rtpbin will only
call us once to request aux/fec senders/receivers.
We also need to regenerate the media section of the SDP instead of
blindly copying from the previous offer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1762>
