Commit graph

57 commits

Author SHA1 Message Date
Nirbheek Chauhan
639f8a24ae webrtc/js: Support renegotiation during a call correctly
When a video track is muted, hide the video element to differentiate
it from a track that is stuck because we stopped receiving RTP data.
Show it again when it is unmuted.

When a video track is removed, remove the video element. It will be
re-added on renegotiation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5045>
2023-07-19 13:01:49 +00:00
Nirbheek Chauhan
57b6c743ef webrtc/js: Remove obsolete mozilla stun server
Mozilla's public stun server is gone. Remove it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5045>
2023-07-19 13:01:49 +00:00
Nirbheek Chauhan
80603746af webrtc/js: Support pressing "enter" to connect
I press "enter" every time which doesn't work and then I click
"Connect", so let's fix that.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5045>
2023-07-19 13:01:49 +00:00
Nirbheek Chauhan
aa1fa50129 webrtc_sendrecv.py: Add AV1 support when creating the offer
Requires svtav1enc at present for simplicity.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4644>
2023-05-17 16:20:36 +00:00
Nirbheek Chauhan
61e536b546 webrtc_sendrecv.py: Fix warnings about gi version
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4644>
2023-05-17 16:20:36 +00:00
Philippe Normand
906b90287c webrtcbin: Relay add-ice-candidate errors from Ice implementation to Application
The `add_candidate` vfunc of the GstWebRTCICE interface gained a GstPromise
argument, which is an ABI break. We're not aware of any external user of this
interface yet so we think it's OK.

This change is useful in cases where the application needs to bubble up errors
from the underlying ICE agent, for instance when the agent was given an invalid
ICE candidate.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3960>
2023-02-27 09:09:47 +00:00
Sebastian Dröge
fc5bad5f75 examples: webrtc: rust: Fix a couple of minor clippy warnings
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3928>
2023-02-10 11:43:00 +00:00
Sebastian Dröge
28ab612a88 examples: webrtc: rust: Update to gstreamer-rs 0.20
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3928>
2023-02-10 11:43:00 +00:00
Nirbheek Chauhan
033a71e405 webrtc examples: Use webrtc.gstreamer.net
Actually just a CNAME to webrtc.nirbheek.in for now, but it allows
replacement / hosting without my involvement, so reduces the bus
factor.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3802>
2023-02-04 13:37:02 +00:00
Matthew Waters
b134433e0b examples/webrtc-sendrecv: add some dot file dumps on async-done and error messages
Just as a helpful thing if debugging is needed

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3823>
2023-01-30 05:22:59 +00:00
Nirbheek Chauhan
32e8ff4e2a webrtc_sendrecv.py: Fix PEP8 warnings in CI lint
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3742>
2023-01-25 16:53:17 +00:00
Nirbheek Chauhan
6a83602601 webrtc_sendrecv.py: Handle LATENCY messages
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3742>
2023-01-25 16:53:17 +00:00
Nirbheek Chauhan
5500c228f6 webrtc_sendrecv.py: Add bus message handling
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3742>
2023-01-25 16:53:17 +00:00
Nirbheek Chauhan
9b2404e76d webrtc_sendrecv.py: Add support for using H264 encoding
Currently only works when we are creating the offer or the offer only
contains H264.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3742>
2023-01-25 16:53:17 +00:00
Nirbheek Chauhan
6f99faa080 webrtc_sendrecv.py: Use sine wave for audio instead of red-noise
Makes it easier to notice when there's packet loss or other audio
distortion.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3742>
2023-01-25 16:53:17 +00:00
Sebastian Dröge
4e86c77270 examples: webrtc: rust: Update dependencies
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3758>
2023-01-20 11:36:57 +00:00
Sebastian Dröge
bf4a3c89cd examples: webrtc: sendrecv: rust: Implement OFFER_REQUEST handling
Allow requesting an offer from the peer if we're joining a call with a
peer, and allow the peer to request an offer from us if waiting for an
incoming call.

This implements all 4 variants the protocol allows for.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3758>
2023-01-20 11:36:57 +00:00
Sebastian Dröge
638465908e examples: webrtc: sendrecv: rust: Allow providing our ID via the commandline
Otherwise it continues to use a random ID as before.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3758>
2023-01-20 11:36:57 +00:00
Sebastian Dröge
541c637910 examples: webrtc: sendrecv: rust: Implement TWCC support in both directions
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3758>
2023-01-20 11:36:57 +00:00
Sebastian Dröge
6541dccaea examples: webrtc: rust: Set keyframe-max-dist=2000 and picture-id-mode=15-bit for VP8 and perfect-timestamps=true for audio
This makes it in sync with the C sendrecv and generally behaves better.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3758>
2023-01-20 11:36:57 +00:00
Sebastian Dröge
083b9f2a6e examples: webrtc: sendrecv: rust: Use the correct payload types if the remote is the offerer
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3758>
2023-01-20 11:36:57 +00:00
Sebastian Dröge
ac1d10f80c gst-examples: Update Rust dependencies
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3750>
2023-01-19 10:40:32 +02:00
Olivier Crête
b7c0e8bc84 webrtc examples: Force regular non-MULTIOPUS
Using MULTIOPUS breaks with most browsers

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3675>
2023-01-04 12:02:25 +00:00
Sebastian Dröge
c739fcbe41 examples: webrtc: Add handling of the LATENCY messages to the Rust examples
Without this the configured latency on the pipeline will be wrong.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3609>
2022-12-20 13:10:27 +02:00
Sebastian Dröge
284d22437e examples: webrtc: Update dependencies
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3609>
2022-12-20 13:06:43 +02:00
Sebastian Dröge
d10981f7b9 examples: webrtc: Add bus handling to the Android and C sendrecv examples
Without a bus, messages will just pile up and errors are not handled at
all. Also without handling the LATENCY messages the latency configured
on the pipeline will be wrong.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3609>
2022-12-20 13:02:08 +02:00
Seungmin Kim
0db1ff532d Change GstSdp.sdp_message_parse_buffer to GstSdp.SDPMessage.new_from_text in examples
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3477>
2022-12-16 10:40:41 +00:00
Guillaume Desmottes
ebfbdf9076 examples: webrtc: fix plugins check
`videoconvert` and `videoscale` are now part of the `videoconvertscale`
plugin, see d11f13f476

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3529>
2022-12-05 17:04:57 +00:00
Jan Schmidt
8177588250 examples/sendrecv: Remove extra unref of webrtcbin
The code now constructs webrtcbin with a floating ref and then
gives it to the pipeline. The extra unref is one too many.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3436>
2022-11-19 19:51:54 +11:00
Jan Schmidt
f2ae481a69 examples/webrtc: Configure payload types
MR 2398 broke the webrtc sendrecv example
by not configuring the payload types, so both audio and video streams
get sent on payload 96.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3434>
2022-11-19 13:12:58 +11:00
Sebastian Dröge
7193a601b3 examples: webrtc: Update to gstreamer-rs 0.19 release
Also update the macOS workaround for gstreamer-gl requiring a
`NSRunLoop` / `NSApp` on the main thread, and update from strucopt to
clap 4.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3255>
2022-10-24 11:50:09 +00:00
Sebastian Dröge
64c376b5b2 webrtc: Add/fix various annotations
And mark string parameters as const.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3194>
2022-10-18 08:56:58 +00:00
Matthew Waters
d586c2cc28 examples/webrtc: don't use factory_make_full() for enums
They are not currently translated into their respective enum values and
will produce an error.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3210>
2022-10-18 01:30:37 +00:00
Nirbheek Chauhan
6a3319c8f2 examples: Support multiple video streams in JS webrtc sendrecv
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3079>
2022-09-27 19:48:56 +00:00
yatinmaan
2c1e61ea16 webrtc: Split WebRTCICE into base classes and implementation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2398>
2022-07-26 13:51:11 +00:00
Sebastian Dröge
d2ecce5862 webrtc: Update dependencies of the Rust examples
And also clean up code a bit while updating to new APIs.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2016>
2022-03-24 12:05:29 +02:00
Nirbheek Chauhan
4ae903d383 webrtc_sendrecv.py: Link pads instead of elements
This was not a problem here because even if we end up accidentally
linking to the wrong pad, things will work out eventually as long as
one pad-added is emitted for each pad that is added.

But it will be a huge problem if someone copies this code and changes
something that requires different handling for different sorts of
pads. The resultant code will be racy. Let's not do this, it's a bad
example.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2008>
2022-03-23 21:04:39 +00:00
Nirbheek Chauhan
0007fa38e0 webrtc-sendrecv: Fix create-answer caps negotiation
We need to parse the payload type map provided by the offer SDP and
set those values on the payloader, otherwise webrtcbin will create
a recvonly answer SDP and we won't send anything to the browser.

Fixed it for both C and Python sendrecv examples.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1864>
2022-03-18 08:16:46 +00:00
Nirbheek Chauhan
3c0d582b7c webrtc_sendrecv.py: Add picture-id-mode to rtpvp8pay
This doesn't just make TWCC stats perform better, it also fixes
stuttery video playback in Chrome.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1864>
2022-03-18 08:16:46 +00:00
Nirbheek Chauhan
e0378f9913 webrtc_sendrecv.py: Print an error on unknown JSON message
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1864>
2022-03-18 08:16:46 +00:00
Nirbheek Chauhan
90da0e1d1e webrtc_sendrecv.py: Add missing copyright headers
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1864>
2022-03-18 08:16:46 +00:00
Nirbheek Chauhan
583408c312 webrtc_sendrecv.py: Implement all negotiation modes
Earlier, the example only supported one negotiation mode:
* Browser client is running, gstreamer starts a call and sends offer

Now these three modes are also supported:
* Browser client is running, gstreamer starts a call and sends an
  offer request
* gstreamer connects and waits for browser client to start a call and
  send an offer
* gstreamer connects and waits for browser client to start a call and
  send an offer request

The following features are still missing:
* Data channel support
* TWCC support + stats logging

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1864>
2022-03-18 08:16:46 +00:00
Nirbheek Chauhan
0b1438cc97 webrtc_sendrecv.py: Make it executable
Why wasn't it already. Tired of typing 'python webrtc_sendrecv.py'.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1864>
2022-03-18 08:16:45 +00:00
Nirbheek Chauhan
2294356d9b webrtc_sendrecv.py: Fix event loop usage for messages
Instead of creating a new loop, we should just be fetching the running
loop, then doing a blocking network call inside the callback, schedule
it on the event loop. This is what the C example does too.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1864>
2022-03-18 08:16:45 +00:00
Nirbheek Chauhan
ee4ca699a0 webrtc_sendrecv.py: Use default arg for server URL
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1864>
2022-03-18 08:16:45 +00:00
Nirbheek Chauhan
4de365b31c webrtc_sendrecv.py: Sync element props with C version
Also add indentation to make it easier to read

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1821>
2022-03-01 16:33:28 +00:00
Nirbheek Chauhan
5ca5a83e75 webrtc_sendrecv.py: Ensure that gst-python overrides are installed
Otherwise fetching of the offer will fail with a cryptic error:

```
Traceback (most recent call last):
  File "/../gstreamer/subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py", line 56, in on_offer_created
    offer = reply['offer']
TypeError: 'Structure' object is not subscriptable
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1821>
2022-03-01 16:33:28 +00:00
Nirbheek Chauhan
e9a02a7380 webrtc_sendrecv.py: Don't try to set state on a None pipe
```
ERROR peer '5762' not found
Traceback (most recent call last):
  File "/../gstreamer/subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py", line 190, in <module>
    res = loop.run_until_complete(c.loop())
  File "/usr/lib64/python3.10/asyncio/base_events.py", line 641, in run_until_complete
    return future.result()
  File "/../gstreamer/subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py", line 155, in loop
    self.close_pipeline()
  File "/../gstreamer/subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py", line 142, in close_pipeline
    self.pipe.set_state(Gst.State.NULL)
AttributeError: 'NoneType' object has no attribute 'set_state'
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1821>
2022-03-01 16:33:28 +00:00
Nirbheek Chauhan
78f8505b9a webrtc_sendrecv.py: Fix SSLError when connecting to websocket server
```
  File "/../gstreamer/subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py", line 189, in <module>
    loop.run_until_complete(c.connect())
  File "/usr/lib64/python3.10/asyncio/base_events.py", line 641, in run_until_complete
    return future.result()
  File "/../gstreamer/subprojects/gst-examples/webrtc/sendrecv/gst/webrtc_sendrecv.py", line 40, in connect
    self.conn = await websockets.connect(self.server, ssl=sslctx)
  File "/home/nirbheek/.local/lib/python3.10/site-packages/websockets/legacy/client.py", line 650, in __await_impl_timeout__
    return await asyncio.wait_for(self.__await_impl__(), self.open_timeout)
  File "/usr/lib64/python3.10/asyncio/tasks.py", line 445, in wait_for
    return fut.result()
  File "/home/nirbheek/.local/lib/python3.10/site-packages/websockets/legacy/client.py", line 654, in __await_impl__
    transport, protocol = await self._create_connection()
  File "/usr/lib64/python3.10/asyncio/base_events.py", line 1080, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "/usr/lib64/python3.10/asyncio/base_events.py", line 1110, in _create_connection_transport
    await waiter
  File "/usr/lib64/python3.10/asyncio/sslproto.py", line 631, in _on_handshake_complete
    raise handshake_exc
  File "/usr/lib64/python3.10/asyncio/sslproto.py", line 676, in _process_write_backlog
    ssldata = self._sslpipe.do_handshake(
  File "/usr/lib64/python3.10/asyncio/sslproto.py", line 116, in do_handshake
    self._sslobj = self._context.wrap_bio(
  File "/usr/lib64/python3.10/ssl.py", line 526, in wrap_bio
    return self.sslobject_class._create(
  File "/usr/lib64/python3.10/ssl.py", line 865, in _create
    sslobj = context._wrap_bio(
ssl.SSLError: Cannot create a client socket with a PROTOCOL_TLS_SERVER context (_ssl.c:801)
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1821>
2022-03-01 16:33:28 +00:00
Nirbheek Chauhan
e453e43e5a webrtc_sendrecv.py: Fix deprecation warning with Python 3.10
asyncio.get_event_loop() will not implicitly create a new event loop
in a future version of Python, so we need to do that explicitly.

```
webrtc_sendrecv.py:188: DeprecationWarning: There is no current event loop
  loop = asyncio.get_event_loop()
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1821>
2022-03-01 16:33:28 +00:00