Commit graph

742 commits

Author SHA1 Message Date
Mathieu Duponchelle
1865899621 rtpbin2: implement jitterbuffer
The jitterbuffer implements both reordering and duplicate packet
handling.

Co-Authored-By: Sebastian Dröge <sebastian@centricular.com>
Co-Authored-By: Matthew Waters <matthew@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 17:35:41 +10:00
Sebastian Dröge
2b4ec75bc5 rtpbin2: Add support for receiving rtcp-mux packets
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 17:35:41 +10:00
Sebastian Dröge
e09ad990fa rtpbin2: Implement support for reduced size RTCP (RFC 5506)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 17:35:41 +10:00
Sebastian Dröge
1e4a966c92 rtpbin2: Add support for sending NACK/PLI and FIR
Co-Authored-By: Matthew Waters <matthew@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 17:35:41 +10:00
Sebastian Dröge
66c9840ad8 rtpbin2: Add handling for receiving NACK/PLI and FIR
Co-Authored-By: Matthew Waters <matthew@centricular.com>
Co-Authored-By: Mathieu Duponchelle <mathieu@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 17:35:41 +10:00
Matthew Waters
2c86f18a99 rtpbin2: add support for RFC 4585 (RTP/AVPF)
Implements the timing rules for RTP/AVPF

Co-Authored-By: Sebastian Dröge <sebastian@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 17:35:41 +10:00
Matthew Waters
27ad26c258 rtp: Initial rtpbin2 element
Can receive and recevie one or more RTP sessions containing multiple
pt/ssrc combinations.

Demultiplexing happens internally instead of relying on separate
elements.

Co-Authored-By: François Laignel <francois@centricular.com>
Co-Authored-By: Mathieu Duponchelle <mathieu@centricular.com>
Co-Authored-By: Sebastian Dröge <sebastian@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 17:35:41 +10:00
Sebastian Dröge
984a9fe5ff rtp: Don't restrict payload types for payloaders
WebRTC uses payload types 35-63 as dynamic payload types too to be able
to place more codec variants into the SDP offer.

Instead of allowing just certain payload types, completely remove any
restrictions and let the user decide. There's technically nothing wrong
with using any payload type, especially when using the encoding-name.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/551

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1587>
2024-05-27 09:34:16 +00:00
Liam
b4fd6cf362 aws: Add system-defined metadata options to both sinks
Add to awss3sink and awss3putobjectsink elements the following
paramerters which are set on the uploaded S3 objects:

* cache-control;
* content-encoding; and
* content-language

Bugfix: Set the content-type and content-disposition values in the S3
putobject call. Previously the params were defined on the element but
unused.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1585>
2024-05-27 10:25:22 +03:00
Tim-Philipp Müller
4f74cb7958 rtp: klv: add test for fragmented payloads with packet loss
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1580>
2024-05-26 12:34:44 +03:00
Tim-Philipp Müller
b6e24668a7 rtp: klv: add unit test with some packet loss
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1580>
2024-05-26 12:34:44 +03:00
Tim-Philipp Müller
92a1e222f4 rtp: tests: add functionality to drop RTP packets after payloading
Add ExpectedPacket::drop() to flag RTP packets that should not
be forwarded to the depayloader.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1580>
2024-05-26 12:34:44 +03:00
Tim-Philipp Müller
de71e9dadd rtp: tests: print rtp timestamp mismatch minus the initial offset
Unit tests specify a 0-based offset, so printing that plus the
random initial offset on failure is just needlessly confusing,
so subtract the initial offset when printing expected/actual
values. The real values are still printed as part of the assert.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1580>
2024-05-26 12:34:44 +03:00
Tim-Philipp Müller
be7da027f8 rtp: klv: add some basic tests
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1580>
2024-05-26 12:34:44 +03:00
Tim-Philipp Müller
1e33926dc5 fixup: klv payloader indentation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1580>
2024-05-26 12:34:44 +03:00
Tim-Philipp Müller
c2f67bd3c9 fixup: klv depay: debug log indentation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1580>
2024-05-26 12:34:44 +03:00
Tim-Philipp Müller
e7d0e0702a fixup: payloader
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1580>
2024-05-26 12:34:44 +03:00
Tim-Philipp Müller
566e6443f4 rtp: Add KLV RTP payloader/depayloader
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1580>
2024-05-25 20:21:50 +03:00
François Laignel
4259d284bd webrtc: add android webrtcsrc example
This commit adds an Android `webrtcsrc` based example with the following
features:

* A first view allows retrieving the producer list from the signaller (peer ids
  are uuids which are too long to tap, especially using an onscreen keyboard).
* Selecting a producer opens a second view. The first available video stream is
  rendered on a native Surface. All the audio streams are rendered using
  `autoaudiosink`.

Available Settings:

* Signaller URI.
* A toggle to prefer hardware decoding for OPUS, otherwise the app defaults to
  raising `opusdec`'s rank. Hardware decoding was moved aside since it was found
  to crash the app on all tested devices (2 smartphones, 1 tv).

**Warning**: in order to ease testing, this demonstration application enables
unencrypted network communication. See `AndroidManifest.xml`.

The application uses the technologies currenlty proposed by Android Studio when
creating a new project:

* Kotlin as the default language, which is fully interoperable with Java and
  uses the same SDK.
* gradle 8.6.
* kotlin dialect for gradle. The structure is mostly the same as the previously
  preferred dialect, for which examples can be found online readily.
* However, JNI code generation still uses Makefiles (instead of CMake) due to
  the need to call [`gstreamer-1.0.mk`] for `gstreamer_android` generation.
  Note: on-going work on that front:
  - https://gitlab.freedesktop.org/gstreamer/cerbero/-/merge_requests/1466
  - https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6794

Current limitations:

* x86 support is currently discarded as `gstreamer_android` libs generation
  fails (observed with `gstreamer-1.0-android-universal-1.24.3`).
* A selector could be added to let the user chose the video streams and
  possibly decide whether to render all audio streams or just select one.

Nice to have:

* Support for the synchronization features of the `webrtc-precise-sync-recv`
  example (NTP clock, RFC 7273).
* It could be nice to use Rust for the specific native code.

[`gstreamer-1.0.mk`]: https://gitlab.freedesktop.org/gstreamer/cerbero/-/blob/main/data/ndk-build/gstreamer-1.0.mk

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1578>
2024-05-24 16:14:13 +00:00
Sebastian Dröge
58e91c154c rtp: basedepay: Reset last used ext seqnum on discontinuities
The ext seqnum counting is reset too so keeping the old one around will
cause problems with timestamping of the next outgoing buffer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1584>
2024-05-24 10:23:06 +03:00
cdelguercio
c99cabfbc5 webrtcsink: Add VP9 parser after the encoder for VP9 too
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1572>
2024-05-23 10:16:59 +03:00
cdelguercio
f5a7de9dc3 webrtcsink: Support av1 via nvav1enc, av1enc, and rav1enc
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1572>
2024-05-23 10:16:59 +03:00
Sebastian Dröge
dcc0b47349 rtp: basepay: Fix header extension negotiation
Only configure header extensions from the source pad caps if they exist
already in the source pad caps, otherwise the configuration will fail.
Extensions that are added via the signals might not exist in the source
pad caps yet and would be added later.

Also, if configuring an existing extension from the new caps fails,
remove it and try to request a new extension for it.

Additionally don't remove extensions from the caps that can't be
provided. No header extensions for them would be added to the packets,
but that's not a problem. Removing them on the other hand would cause
negotiation to fail. This only affects extensions that are already
included in the caps.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1577>
2024-05-20 07:53:50 +00:00
Sebastian Dröge
0d33077df6 rtp: basedepay: Clean up header extension negotiation
If configuring an existing extension from the new caps fails, remove it
and try to request a new extension for it.

Also remove all extensions from the list that are not provided in the
caps, instead of passing RTP packets to all of them anyway.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1577>
2024-05-20 07:53:50 +00:00
Tim-Philipp Müller
16608d2541 rtp: opus: add multichannel depay/pay test
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1571>
2024-05-18 09:29:29 +00:00
Tim-Philipp Müller
bab3498c6a rtp: opus: add multichannel pay/depay test
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1571>
2024-05-18 09:29:29 +00:00
Tim-Philipp Müller
72006215cb rtp: tests: add run_test_pipeline_full() that checks output caps too
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1571>
2024-05-18 09:29:29 +00:00
Tim-Philipp Müller
10e0294d5a rtp: opus: fix payloader caps query handling and add tests
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1571>
2024-05-18 09:29:29 +00:00
Tim-Philipp Müller
61523baa7b rtp: opus: add minimal depayload / re-payload test
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1571>
2024-05-18 09:29:29 +00:00
Tim-Philipp Müller
6f871e6ce2 rtp: opus: add simple payload / depayload test
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1571>
2024-05-18 09:29:29 +00:00
Tim-Philipp Müller
92c0cf1285 rtp: opus: add test for payloader dtx packet handling
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1571>
2024-05-18 09:29:29 +00:00
Tim-Philipp Müller
2585639054 rtp: Add Opus RTP payloader/depayloader
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1571>
2024-05-18 09:29:29 +00:00
Sebastian Dröge
539000574b aws: Update to base32 0.5
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1576>
2024-05-17 07:50:51 +00:00
Robert Ayrapetyan
bac5845be1 webrtc: add support for insecure tls connections
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1553>
2024-05-16 19:34:57 +00:00
Martin Nordholts
9a7f37e2b7 rtpgccbwe: Support linear regression based delay estimation
In our tests, the slope (found with linear regression) on a
history of the (smoothed) accumulated inter-group delays
gives a more stable congestion control. In particular,
low-end devices becomes less sensitive to spikes in
inter-group delay measurements.

This flavour of delay based bandwidth estimation with Google
Congestion Control is also what Chromium is using.

To make it easy to experiment with the new estimator, as
well as add support for new ones in the future, also add
infrastructure for making delay estimator flavour selectable
at runtime.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1566>
2024-05-14 16:25:48 +00:00
Martin Nordholts
71e9c2bb04 rtpgccbwe: Also log self.measure in overuse_filter()
Also log `self.measure` in overuse_filter() since tracking
`self.measure` over time help a lot in making sense of
`self.estimate` (and `amplified_estimate`).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1566>
2024-05-14 16:25:48 +00:00
Martin Nordholts
d9aa0731f4 rtpgccbwe: Rename variable t to amplified_estimate
We normally multiply `self.estimate` with `MAX_DELTAS` (60).
Rename the variables that holds the result of this
calculation to `amplified_estimate` to make the distinction
clearer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1566>
2024-05-14 16:25:48 +00:00
Tamas Levai
71cd80f204 net/quinn: Enable client to keep QUIC conn alive
Co-authored-by: Felician Nemeth <nemethf@tmit.bme.hu>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1568>
2024-05-11 08:51:00 +02:00
Sebastian Dröge
613ed56675 webrtcsink: Add a custom signaller example in Python
This re-implements the default webrtcsink/src signalling protocol in
Python for demonstration purposes.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1569>
2024-05-10 15:59:12 +00:00
Martin Nordholts
a719cbfcc6 rtp: Change RtpBasePay2::ssrc_collision from AtomicU64 to Option<u32>
Rust targets without support for `AtomicU64` is still
somewhat common. Running

    git grep -i 'max_atomic_width: Some(32)' | wc -l

in the Rust compiler repo currently counts to 34 targets.

Change the `RtpBasePay2::ssrc_collision` from `AtomicU64` to
`Mutex<Option<u32>>`. This way we keep support for these
targets.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1562>
2024-05-10 14:23:41 +00:00
Martin Nordholts
aabb011f5a rtpgccbwe: Log effective bitrate in more places
While monitoring and debugging rtpgccbwe, it is very helpful
to get continuous values of what it considers the effective
bitrate. Right now such prints will stop coming once the
algorithm stabilizes. Print it in more places so it keeps
coming. Use the same format to make it simpler to extract
the values by parsing the logs.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1567>
2024-05-10 11:56:51 +00:00
Martin Nordholts
e845e3575c rtpgccbwe: Add mising 'ps' suffix to 'kbps' of 'effective bitrate'
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1567>
2024-05-10 11:56:51 +00:00
Sebastian Dröge
e8e173d0d0 webrtc: Update Signallable interface to new interface definition API
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1570>
2024-05-10 14:13:55 +03:00
Sebastian Dröge
7e09481adc rtp: Add JPEG RTP payloader/depayloader
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1543>
2024-05-10 11:12:49 +03:00
Sanchayan Maity
fe55acb4c9 net/hlssink3: Refactor out HlsBaseSink & hlscmafsink from hlssink3
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1564>
2024-05-09 21:50:32 +05:30
Tamas Levai
5884c00bd0 net/quinn: Improve stream shutdown process
Co-authored-by: Sanchayan Maity <sanchayan@asymptotic.io>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1565>
2024-05-09 16:43:26 +02:00
Tamas Levai
13c3db7857 net/quinn: Port to quinn 0.11 and rustls 0.23
Co-authored-by: Felician Nemeth <nemethf@tmit.bme.hu>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1565>
2024-05-09 13:49:33 +02:00
Martin Nordholts
2b7488a4c8 rtpgccbwe: Log delay and loss target bitrates separately
When debugging rtpgccbwe it is helpful to know if it is
delay based or loss based band-width estimation that puts a
bound on the current target bitrate, so add logs for that.

To minimize the time we need to hold the state lock, perform
the logging after we have released the state lock.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1561>
2024-05-08 19:12:44 +00:00
Mathieu Duponchelle
8861fc493b webrtcsink: improve error when no discovery pipeline runs
If for instance no encoder was found or the RTP plugin was missing,
it is possible that no discovery pipeline will run for a given stream.

Provide a more helpful error message for that case.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/534
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1560>
2024-05-06 11:39:48 +00:00
Sanchayan Maity
3a3cec96ff net/quinn: Add pipeline example
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1558>
2024-05-02 16:39:29 +00:00