mirrors/gst-plugins-rs
mirrors/gst-plugins-rs
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs.git synced 2024-12-20 17:16:39 +00:00
Code Issues Projects Releases Wiki Activity

Merge 'webrtcsink' from 020c7e2900

Browse source
This commit is contained in:
Thibault Saunier 2022-10-20 11:51:58 +02:00
commit eb9d0bb824
51 changed files with 16369 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2071
net/webrtc/Cargo.lock generated Normal file
View file

File diff suppressed because it is too large Load diff

7
net/webrtc/Cargo.toml Normal file
View file

@ -0,0 +1,7 @@
[workspace]
members = [
"plugins",
"protocol",
"signalling",
]

373
net/webrtc/LICENSE Normal file
View file

@ -0,0 +1,373 @@
Mozilla Public License Version 2.0
==================================
1. Definitions
--------------
1.1. "Contributor"
means each individual or legal entity that creates, contributes to
the creation of, or owns Covered Software.
1.2. "Contributor Version"
means the combination of the Contributions of others (if any) used
by a Contributor and that particular Contributor's Contribution.
1.3. "Contribution"
means Covered Software of a particular Contributor.
1.4. "Covered Software"
means Source Code Form to which the initial Contributor has attached
the notice in Exhibit A, the Executable Form of such Source Code
Form, and Modifications of such Source Code Form, in each case
including portions thereof.
1.5. "Incompatible With Secondary Licenses"
means
(a) that the initial Contributor has attached the notice described
in Exhibit B to the Covered Software; or
(b) that the Covered Software was made available under the terms of
version 1.1 or earlier of the License, but not also under the
terms of a Secondary License.
1.6. "Executable Form"
means any form of the work other than Source Code Form.
1.7. "Larger Work"
means a work that combines Covered Software with other material, in
a separate file or files, that is not Covered Software.
1.8. "License"
means this document.
1.9. "Licensable"
means having the right to grant, to the maximum extent possible,
whether at the time of the initial grant or subsequently, any and
all of the rights conveyed by this License.
1.10. "Modifications"
means any of the following:
(a) any file in Source Code Form that results from an addition to,
deletion from, or modification of the contents of Covered
Software; or
(b) any new file in Source Code Form that contains any Covered
Software.
1.11. "Patent Claims" of a Contributor
means any patent claim(s), including without limitation, method,
process, and apparatus claims, in any patent Licensable by such
Contributor that would be infringed, but for the grant of the
License, by the making, using, selling, offering for sale, having
made, import, or transfer of either its Contributions or its
Contributor Version.
1.12. "Secondary License"
means either the GNU General Public License, Version 2.0, the GNU
Lesser General Public License, Version 2.1, the GNU Affero General
Public License, Version 3.0, or any later versions of those
licenses.
1.13. "Source Code Form"
means the form of the work preferred for making modifications.
1.14. "You" (or "Your")
means an individual or a legal entity exercising rights under this
License. For legal entities, "You" includes any entity that
controls, is controlled by, or is under common control with You. For
purposes of this definition, "control" means (a) the power, direct
or indirect, to cause the direction or management of such entity,
whether by contract or otherwise, or (b) ownership of more than
fifty percent (50%) of the outstanding shares or beneficial
ownership of such entity.
2. License Grants and Conditions
--------------------------------
2.1. Grants
Each Contributor hereby grants You a world-wide, royalty-free,
non-exclusive license:
(a) under intellectual property rights (other than patent or trademark)
Licensable by such Contributor to use, reproduce, make available,
modify, display, perform, distribute, and otherwise exploit its
Contributions, either on an unmodified basis, with Modifications, or
as part of a Larger Work; and
(b) under Patent Claims of such Contributor to make, use, sell, offer
for sale, have made, import, and otherwise transfer either its
Contributions or its Contributor Version.
2.2. Effective Date
The licenses granted in Section 2.1 with respect to any Contribution
become effective for each Contribution on the date the Contributor first
distributes such Contribution.
2.3. Limitations on Grant Scope
The licenses granted in this Section 2 are the only rights granted under
this License. No additional rights or licenses will be implied from the
distribution or licensing of Covered Software under this License.
Notwithstanding Section 2.1(b) above, no patent license is granted by a
Contributor:
(a) for any code that a Contributor has removed from Covered Software;
or
(b) for infringements caused by: (i) Your and any other third party's
modifications of Covered Software, or (ii) the combination of its
Contributions with other software (except as part of its Contributor
Version); or
(c) under Patent Claims infringed by Covered Software in the absence of
its Contributions.
This License does not grant any rights in the trademarks, service marks,
or logos of any Contributor (except as may be necessary to comply with
the notice requirements in Section 3.4).
2.4. Subsequent Licenses
No Contributor makes additional grants as a result of Your choice to
distribute the Covered Software under a subsequent version of this
License (see Section 10.2) or under the terms of a Secondary License (if
permitted under the terms of Section 3.3).
2.5. Representation
Each Contributor represents that the Contributor believes its
Contributions are its original creation(s) or it has sufficient rights
to grant the rights to its Contributions conveyed by this License.
2.6. Fair Use
This License is not intended to limit any rights You have under
applicable copyright doctrines of fair use, fair dealing, or other
equivalents.
2.7. Conditions
Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted
in Section 2.1.
3. Responsibilities
-------------------
3.1. Distribution of Source Form
All distribution of Covered Software in Source Code Form, including any
Modifications that You create or to which You contribute, must be under
the terms of this License. You must inform recipients that the Source
Code Form of the Covered Software is governed by the terms of this
License, and how they can obtain a copy of this License. You may not
attempt to alter or restrict the recipients' rights in the Source Code
Form.
3.2. Distribution of Executable Form
If You distribute Covered Software in Executable Form then:
(a) such Covered Software must also be made available in Source Code
Form, as described in Section 3.1, and You must inform recipients of
the Executable Form how they can obtain a copy of such Source Code
Form by reasonable means in a timely manner, at a charge no more
than the cost of distribution to the recipient; and
(b) You may distribute such Executable Form under the terms of this
License, or sublicense it under different terms, provided that the
license for the Executable Form does not attempt to limit or alter
the recipients' rights in the Source Code Form under this License.
3.3. Distribution of a Larger Work
You may create and distribute a Larger Work under terms of Your choice,
provided that You also comply with the requirements of this License for
the Covered Software. If the Larger Work is a combination of Covered
Software with a work governed by one or more Secondary Licenses, and the
Covered Software is not Incompatible With Secondary Licenses, this
License permits You to additionally distribute such Covered Software
under the terms of such Secondary License(s), so that the recipient of
the Larger Work may, at their option, further distribute the Covered
Software under the terms of either this License or such Secondary
License(s).
3.4. Notices
You may not remove or alter the substance of any license notices
(including copyright notices, patent notices, disclaimers of warranty,
or limitations of liability) contained within the Source Code Form of
the Covered Software, except that You may alter any license notices to
the extent required to remedy known factual inaccuracies.
3.5. Application of Additional Terms
You may choose to offer, and to charge a fee for, warranty, support,
indemnity or liability obligations to one or more recipients of Covered
Software. However, You may do so only on Your own behalf, and not on
behalf of any Contributor. You must make it absolutely clear that any
such warranty, support, indemnity, or liability obligation is offered by
You alone, and You hereby agree to indemnify every Contributor for any
liability incurred by such Contributor as a result of warranty, support,
indemnity or liability terms You offer. You may include additional
disclaimers of warranty and limitations of liability specific to any
jurisdiction.
4. Inability to Comply Due to Statute or Regulation
---------------------------------------------------
If it is impossible for You to comply with any of the terms of this
License with respect to some or all of the Covered Software due to
statute, judicial order, or regulation then You must: (a) comply with
the terms of this License to the maximum extent possible; and (b)
describe the limitations and the code they affect. Such description must
be placed in a text file included with all distributions of the Covered
Software under this License. Except to the extent prohibited by statute
or regulation, such description must be sufficiently detailed for a
recipient of ordinary skill to be able to understand it.
5. Termination
--------------
5.1. The rights granted under this License will terminate automatically
if You fail to comply with any of its terms. However, if You become
compliant, then the rights granted under this License from a particular
Contributor are reinstated (a) provisionally, unless and until such
Contributor explicitly and finally terminates Your grants, and (b) on an
ongoing basis, if such Contributor fails to notify You of the
non-compliance by some reasonable means prior to 60 days after You have
come back into compliance. Moreover, Your grants from a particular
Contributor are reinstated on an ongoing basis if such Contributor
notifies You of the non-compliance by some reasonable means, this is the
first time You have received notice of non-compliance with this License
from such Contributor, and You become compliant prior to 30 days after
Your receipt of the notice.
5.2. If You initiate litigation against any entity by asserting a patent
infringement claim (excluding declaratory judgment actions,
counter-claims, and cross-claims) alleging that a Contributor Version
directly or indirectly infringes any patent, then the rights granted to
You by any and all Contributors for the Covered Software under Section
2.1 of this License shall terminate.
5.3. In the event of termination under Sections 5.1 or 5.2 above, all
end user license agreements (excluding distributors and resellers) which
have been validly granted by You or Your distributors under this License
prior to termination shall survive termination.
************************************************************************
* *
* 6. Disclaimer of Warranty *
* ------------------------- *
* *
* Covered Software is provided under this License on an "as is" *
* basis, without warranty of any kind, either expressed, implied, or *
* statutory, including, without limitation, warranties that the *
* Covered Software is free of defects, merchantable, fit for a *
* particular purpose or non-infringing. The entire risk as to the *
* quality and performance of the Covered Software is with You. *
* Should any Covered Software prove defective in any respect, You *
* (not any Contributor) assume the cost of any necessary servicing, *
* repair, or correction. This disclaimer of warranty constitutes an *
* essential part of this License. No use of any Covered Software is *
* authorized under this License except under this disclaimer. *
* *
************************************************************************
************************************************************************
* *
* 7. Limitation of Liability *
* -------------------------- *
* *
* Under no circumstances and under no legal theory, whether tort *
* (including negligence), contract, or otherwise, shall any *
* Contributor, or anyone who distributes Covered Software as *
* permitted above, be liable to You for any direct, indirect, *
* special, incidental, or consequential damages of any character *
* including, without limitation, damages for lost profits, loss of *
* goodwill, work stoppage, computer failure or malfunction, or any *
* and all other commercial damages or losses, even if such party *
* shall have been informed of the possibility of such damages. This *
* limitation of liability shall not apply to liability for death or *
* personal injury resulting from such party's negligence to the *
* extent applicable law prohibits such limitation. Some *
* jurisdictions do not allow the exclusion or limitation of *
* incidental or consequential damages, so this exclusion and *
* limitation may not apply to You. *
* *
************************************************************************
8. Litigation
-------------
Any litigation relating to this License may be brought only in the
courts of a jurisdiction where the defendant maintains its principal
place of business and such litigation shall be governed by laws of that
jurisdiction, without reference to its conflict-of-law provisions.
Nothing in this Section shall prevent a party's ability to bring
cross-claims or counter-claims.
9. Miscellaneous
----------------
This License represents the complete agreement concerning the subject
matter hereof. If any provision of this License is held to be
unenforceable, such provision shall be reformed only to the extent
necessary to make it enforceable. Any law or regulation which provides
that the language of a contract shall be construed against the drafter
shall not be used to construe this License against a Contributor.
10. Versions of the License
---------------------------
10.1. New Versions
Mozilla Foundation is the license steward. Except as provided in Section
10.3, no one other than the license steward has the right to modify or
publish new versions of this License. Each version will be given a
distinguishing version number.
10.2. Effect of New Versions
You may distribute the Covered Software under the terms of the version
of the License under which You originally received the Covered Software,
or under the terms of any subsequent version published by the license
steward.
10.3. Modified Versions
If you create software not governed by this License, and you want to
create a new license for such software, you may create and use a
modified version of this License if you rename the license and remove
any references to the name of the license steward (except to note that
such modified license differs from this License).
10.4. Distributing Source Code Form that is Incompatible With Secondary
Licenses
If You choose to distribute Source Code Form that is Incompatible With
Secondary Licenses under the terms of this version of the License, the
notice described in Exhibit B of this License must be attached.
Exhibit A - Source Code Form License Notice
-------------------------------------------
This Source Code Form is subject to the terms of the Mozilla Public
License, v. 2.0. If a copy of the MPL was not distributed with this
file, You can obtain one at http://mozilla.org/MPL/2.0/.
If it is not possible or desirable to put the notice in a particular
file, then You may include the notice in a location (such as a LICENSE
file in a relevant directory) where a recipient would be likely to look
for such a notice.
You may add additional accurate notices of copyright ownership.
Exhibit B - "Incompatible With Secondary Licenses" Notice
---------------------------------------------------------
This Source Code Form is "Incompatible With Secondary Licenses", as
defined by the Mozilla Public License, v. 2.0.

224
net/webrtc/README.md Normal file
View file

@ -0,0 +1,224 @@
# webrtcsink
All-batteries included GStreamer WebRTC producer, that tries its best to do The Right Thing™.
## Use case
The [webrtcbin] element in GStreamer is extremely flexible and powerful, but using
it can be a difficult exercise. When all you want to do is serve a fixed set of streams
to any number of consumers, `webrtcsink` (which wraps `webrtcbin` internally) can be a
useful alternative.
[webrtcbin]: https://gstreamer.freedesktop.org/documentation/webrtc/index.html
## Features
`webrtcsink` implements the following features:
* Built-in signaller: when using the default signalling server, this element will
perform signalling without requiring application interaction.
This makes it usable directly from `gst-launch`.
* Application-provided signalling: `webrtcsink` can be instantiated by an application
with a custom signaller. That signaller must be a GObject, and must implement the
`Signallable` interface as defined [here](plugins/src/webrtcsink/mod.rs). The
[default signaller](plugins/src/signaller/mod.rs) can be used as an example.
An [example project] is also available to use as a boilerplate for implementing
and using a custom signaller.
* Sandboxed consumers: when a consumer is added, its encoder / payloader / webrtcbin
elements run in a separately managed pipeline. This provides a certain level of
sandboxing, as opposed to having those elements running inside the element itself.
It is important to note that at this moment, encoding is not shared between consumers.
While this is not on the roadmap at the moment, nothing in the design prevents
implementing this optimization.
* Congestion control: the element leverages transport-wide congestion control
feedback messages in order to adapt the bitrate of individual consumers' video
encoders to the available bandwidth.
* Configuration: the level of user control over the element is slowly expanding,
consult `gst-inspect-1.0` for more information on the available properties and
signals.
* Packet loss mitigation: webrtcsink now supports sending protection packets for
Forward Error Correction, modulating the amount as a function of the available
bandwidth, and can honor retransmission requests. Both features can be disabled
via properties.
It is important to note that full control over the individual elements used by
`webrtcsink` is *not* on the roadmap, as it will act as a black box in that respect,
for example `webrtcsink` wants to reserve control over the bitrate for congestion
control.
A signal is now available however for the application to provide the initial
configuration for the encoders `webrtcsink` instantiates.
If more granular control is required, applications should use `webrtcbin` directly,
`webrtcsink` will focus on trying to just do the right thing, although it might
expose more interfaces to guide and tune the heuristics it employs.
[example project]: https://github.com/centricular/webrtcsink-custom-signaller
## Building
### Prerequisites
The element has only been tested for now against GStreamer main.
For testing, it is recommended to simply build GStreamer locally and run
in the uninstalled devenv.
> Make sure to install the development packages for some codec libraries
> beforehand, such as libx264, libvpx and libopusenc, exact names depend
> on your distribution.
```
git clone --depth 1 --single-branch --branch main https://gitlab.freedesktop.org/gstreamer/gstreamer
cd gstreamer
meson build
ninja -C build
ninja -C build devenv
```
### Compiling
``` shell
cargo build
```
## Usage
Open three terminals. In the first, run:
``` shell
WEBRTCSINK_SIGNALLING_SERVER_LOG=debug cargo run --bin server
```
In the second, run:
``` shell
python3 -m http.server -d www/
```
In the third, run:
``` shell
export GST_PLUGIN_PATH=$PWD/target/debug:$GST_PLUGIN_PATH
gst-launch-1.0 webrtcsink name=ws videotestsrc ! ws. audiotestsrc ! ws.
```
When the pipeline above is running succesfully, open a browser and
point it to the http server:
``` shell
xdg-open http://127.0.0.1:8000
```
You should see an identifier listed in the left-hand panel, click on
it. You should see a test video stream, and hear a test tone.
## Configuration
The element itself can be configured through its properties, see
`gst-inspect-1.0 webrtcsink` for more information about that, in addition the
default signaller also exposes properties for configuring it, in
particular setting the signalling server address, those properties
can be accessed through the `gst::ChildProxy` interface, for example
with gst-launch:
``` shell
gst-launch-1.0 webrtcsink signaller::address="ws://127.0.0.1:8443" ..
```
The signaller object can not be inspected, refer to [the source code]
for the list of properties.
[the source code]: plugins/src/signaller/imp.rs
### Enable 'navigation' a.k.a user interactivity with the content
`webrtcsink` implements the [`GstNavigation`] interface which allows interacting
with the content, for example move with your mouse, entering keys with the
keyboard, etc... On top of that a `WebRTCDataChannel` based protocol has been
implemented and can be activated with the `enable-data-channel-navigation=true`
property. The [demo](www/) implements the protocol and you can easily test this
feature, using the [`wpesrc`] for example.
As an example, the following pipeline allows you to navigate the GStreamer
documentation inside the video running within your web browser (in
http://127.0.0.1:8000 if you followed previous steps of that readme):
```
gst-launch-1.0 wpesrc location=https://gstreamer.freedesktop.org/documentation/ ! webrtcsink enable-data-channel-navigation=true
```
[`GstNavigation`]: https://gstreamer.freedesktop.org/documentation/video/gstnavigation.html
[`wpesrc`]: https://gstreamer.freedesktop.org/documentation/wpe/wpesrc.html
## Testing congestion control
For the purpose of testing congestion in a reproducible manner, a
[simple tool] has been used, I only used it on Linux but it is documented
as usable on MacOS too. I had to run the client browser on a separate
machine on my local network for congestion to actually be applied, I didn't
look into why that was necessary.
My testing procedure was:
* identify the server machine network interface (eg with `ifconfig` on Linux)
* identify the client machine IP address (eg with `ifconfig` on Linux)
* start the various services as explained in the Usage section (use
`GST_DEBUG=webrtcsink:7` to get detailed logs about congestion control)
* start playback in the client browser
* Run a `comcast` command on the server machine, for instance:
``` shell
/home/meh/go/bin/comcast --device=$SERVER_INTERFACE --target-bw 3000 --target-addr=$CLIENT_IP --target-port=1:65535 --target-proto=udp
```
* Observe the bitrate sharply decreasing, playback should slow down briefly
then catch back up
* Remove the bandwidth limitation, and observe the bitrate eventually increasing
back to a maximum:
``` shell
/home/meh/go/bin/comcast --device=$SERVER_INTERFACE --stop
```
For comparison, the congestion control property can be set to disabled on
webrtcsink, then the above procedure applied again, the expected result is
for playback to simply crawl down to a halt until the bandwidth limitation
is lifted:
``` shell
gst-launch-1.0 webrtcsink congestion-control=disabled
```
[simple tool]: https://github.com/tylertreat/comcast
## Monitoring tool
An example server / client application for monitoring per-consumer stats
can be found [here].
[here]: plugins/examples/README.md
## License
All the rust code in this repository is licensed under the [Mozilla Public License Version 2.0].
Parts of the JavaScript code in the www/ example are licensed under the [Apache License, Version 2.0],
the rest is licensed under the [Mozilla Public License Version 2.0] unless advertised in the
header.
[Mozilla Public License Version 2.0]: http://opensource.org/licenses/MPL-2.0
[Apache License, Version 2.0]: https://www.apache.org/licenses/LICENSE-2.1

68
net/webrtc/plugins/Cargo.toml Normal file
View file

@ -0,0 +1,68 @@
[package]
name = "webrtcsink"
version = "0.1.0"
edition = "2018"
authors = ["Mathieu Duponchelle <mathieu@centricular.com>"]
license = "MPL-2.0"
description = "GStreamer WebRTC sink"
repository = "https://github.com/centricular/webrtcsink/"
build = "build.rs"
[dependencies]
gst = { git="https://gitlab.freedesktop.org/gstreamer/gstreamer-rs", package = "gstreamer", features = ["v1_20", "serde"] }
gst-app = { git="https://gitlab.freedesktop.org/gstreamer/gstreamer-rs", package = "gstreamer-app", features = ["v1_20"] }
gst-video = { git="https://gitlab.freedesktop.org/gstreamer/gstreamer-rs", package = "gstreamer-video", features = ["v1_20", "serde"] }
gst-webrtc = { git="https://gitlab.freedesktop.org/gstreamer/gstreamer-rs", package = "gstreamer-webrtc", features = ["v1_20"] }
gst-sdp = { git="https://gitlab.freedesktop.org/gstreamer/gstreamer-rs", package = "gstreamer-sdp", features = ["v1_20"] }
gst-rtp = { git="https://gitlab.freedesktop.org/gstreamer/gstreamer-rs", package = "gstreamer-rtp", features = ["v1_20"] }
gst-utils = { git="https://gitlab.freedesktop.org/gstreamer/gstreamer-rs", package = "gstreamer-utils" }
once_cell = "1.0"
chrono = { version = "0.4", default-features = false }
smallvec = "1"
anyhow = "1"
thiserror = "1"
futures = "0.3"
async-std = { version = "1", features = ["unstable"] }
async-native-tls = { version = "0.4.0" }
async-tungstenite = { version = "0.17", features = ["async-std-runtime", "async-native-tls"] }
serde = "1"
serde_json = "1"
fastrand = "1.0"
webrtcsink-protocol = { version = "0.1", path="../protocol" }
human_bytes = "0.3.1"
[dev-dependencies]
tracing = { version = "0.1", features = ["log"] }
tracing-subscriber = { version = "0.3", features = ["registry", "env-filter"] }
tracing-log = "0.1"
uuid = { version = "1", features = ["v4"] }
clap = { version = "4", features = ["derive"] }
[lib]
name = "webrtcsink"
crate-type = ["cdylib", "rlib"]
path = "src/lib.rs"
[build-dependencies]
gst-plugin-version-helper = "0.7"
[features]
static = []
capi = []
gst1_22 = ["gst/v1_22", "gst-app/v1_22", "gst-video/v1_22", "gst-webrtc/v1_22", "gst-sdp/v1_22", "gst-rtp/v1_22"]
[package.metadata.capi]
min_version = "0.8.0"
[package.metadata.capi.header]
enabled = false
[package.metadata.capi.library]
install_subdir = "gstreamer-1.0"
versioning = false
[package.metadata.capi.pkg_config]
requires_private = "gstreamer-rtp >= 1.20, gstreamer-webrtc >= 1.20, gstreamer-1.0 >= 1.20, gstreamer-app >= 1.20, gstreamer-video >= 1.20, gstreamer-sdp >= 1.20, gobject-2.0, glib-2.0, gmodule-2.0"
[[example]]
name = "webrtcsink-stats-server"

3
net/webrtc/plugins/build.rs Normal file
View file

@ -0,0 +1,3 @@
fn main() {
gst_plugin_version_helper::info()
}

18
net/webrtc/plugins/examples/README.md Normal file
View file

@ -0,0 +1,18 @@
# webrtcsink examples
Collection (1-sized for now) of webrtcsink examples
## webrtcsink-stats-server
A simple application that instantiates a webrtcsink and serves stats
over websockets.
The application expects a signalling server to be running at `ws://localhost:8443`,
similar to the usage example in the main README.
``` shell
cargo run --example webrtcsink-stats-server
```
Once it is running, follow the instruction in the webrtcsink-stats folder to
run an example client.

235
net/webrtc/plugins/examples/webrtcsink-stats-server.rs Normal file
View file

@ -0,0 +1,235 @@
use std::collections::HashMap;
use std::sync::{Arc, Mutex};
use anyhow::Error;
use async_std::net::{TcpListener, TcpStream};
use async_std::task;
use async_tungstenite::tungstenite::Message as WsMessage;
use clap::Parser;
use futures::channel::mpsc;
use futures::prelude::*;
use gst::glib::Type;
use gst::prelude::*;
use tracing::{debug, info, trace};
use tracing_subscriber::prelude::*;
#[derive(Parser, Debug)]
#[clap(about, version, author)]
/// Program arguments
struct Args {
/// URI of file to serve. Must hold at least one audio and video stream
uri: String,
/// Disable Forward Error Correction
#[clap(long)]
disable_fec: bool,
/// Disable retransmission
#[clap(long)]
disable_retransmission: bool,
/// Disable congestion control
#[clap(long)]
disable_congestion_control: bool,
}
fn serialize_value(val: &gst::glib::Value) -> Option<serde_json::Value> {
match val.type_() {
Type::STRING => Some(val.get::<String>().unwrap().into()),
Type::BOOL => Some(val.get::<bool>().unwrap().into()),
Type::I32 => Some(val.get::<i32>().unwrap().into()),
Type::U32 => Some(val.get::<u32>().unwrap().into()),
Type::I_LONG | Type::I64 => Some(val.get::<i64>().unwrap().into()),
Type::U_LONG | Type::U64 => Some(val.get::<u64>().unwrap().into()),
Type::F32 => Some(val.get::<f32>().unwrap().into()),
Type::F64 => Some(val.get::<f64>().unwrap().into()),
_ => {
if let Ok(s) = val.get::<gst::Structure>() {
serde_json::to_value(
s.iter()
.filter_map(|(name, value)| {
serialize_value(value).map(|value| (name.to_string(), value))
})
.collect::<HashMap<String, serde_json::Value>>(),
)
.ok()
} else if let Ok(a) = val.get::<gst::Array>() {
serde_json::to_value(
a.iter()
.filter_map(|value| serialize_value(value))
.collect::<Vec<serde_json::Value>>(),
)
.ok()
} else if let Some((_klass, values)) = gst::glib::FlagsValue::from_value(val) {
Some(
values
.iter()
.map(|value| value.nick())
.collect::<Vec<&str>>()
.join("+")
.into(),
)
} else if let Ok(value) = val.serialize() {
Some(value.as_str().into())
} else {
None
}
}
}
}
#[derive(Clone)]
struct Listener {
id: uuid::Uuid,
sender: mpsc::Sender<WsMessage>,
}
struct State {
listeners: Vec<Listener>,
}
async fn run(args: Args) -> Result<(), Error> {
tracing_log::LogTracer::init().expect("Failed to set logger");
let env_filter = tracing_subscriber::EnvFilter::try_from_env("WEBRTCSINK_STATS_LOG")
.unwrap_or_else(|_| tracing_subscriber::EnvFilter::new("info"));
let fmt_layer = tracing_subscriber::fmt::layer()
.with_thread_ids(true)
.with_target(true)
.with_span_events(
tracing_subscriber::fmt::format::FmtSpan::NEW
| tracing_subscriber::fmt::format::FmtSpan::CLOSE,
);
let subscriber = tracing_subscriber::Registry::default()
.with(env_filter)
.with(fmt_layer);
tracing::subscriber::set_global_default(subscriber).expect("Failed to set subscriber");
let state = Arc::new(Mutex::new(State { listeners: vec![] }));
let addr = "127.0.0.1:8484".to_string();
// Create the event loop and TCP listener we'll accept connections on.
let try_socket = TcpListener::bind(&addr).await;
let listener = try_socket.expect("Failed to bind");
info!("Listening on: {}", addr);
let pipeline_str = format!(
"webrtcsink name=ws do-retransmission={} do-fec={} congestion-control={} \
uridecodebin name=d uri={} \
d. ! video/x-raw ! queue ! ws.video_0 \
d. ! audio/x-raw ! queue ! ws.audio_0",
!args.disable_retransmission,
!args.disable_fec,
if args.disable_congestion_control {
"disabled"
} else {
"homegrown"
},
args.uri
);
let pipeline = gst::parse_launch(&pipeline_str)?;
let ws = pipeline
.downcast_ref::<gst::Bin>()
.unwrap()
.by_name("ws")
.unwrap();
ws.connect("encoder-setup", false, |values| {
let encoder = values[3].get::<gst::Element>().unwrap();
info!("Encoder: {}", encoder.factory().unwrap().name());
let configured = if let Some(factory) = encoder.factory() {
match factory.name().as_str() {
"does-not-exist" => {
// One could configure a hardware encoder to their liking here,
// and return true to make sure webrtcsink does not do any configuration
// of its own
true
}
_ => false,
}
} else {
false
};
Some(configured.to_value())
});
let ws_clone = ws.downgrade();
let state_clone = state.clone();
task::spawn(async move {
let mut interval = async_std::stream::interval(std::time::Duration::from_millis(100));
while interval.next().await.is_some() {
if let Some(ws) = ws_clone.upgrade() {
let stats = ws.property::<gst::Structure>("stats");
let stats = serialize_value(&stats.to_value()).unwrap();
debug!("Stats: {}", serde_json::to_string_pretty(&stats).unwrap());
let msg = WsMessage::Text(serde_json::to_string(&stats).unwrap());
let listeners = state_clone.lock().unwrap().listeners.clone();
for mut listener in listeners {
if listener.sender.send(msg.clone()).await.is_err() {
let mut state = state_clone.lock().unwrap();
let index = state
.listeners
.iter()
.position(|l| l.id == listener.id)
.unwrap();
state.listeners.remove(index);
}
}
} else {
break;
}
}
});
pipeline.set_state(gst::State::Playing)?;
while let Ok((stream, _)) = listener.accept().await {
task::spawn(accept_connection(state.clone(), stream));
}
Ok(())
}
async fn accept_connection(state: Arc<Mutex<State>>, stream: TcpStream) {
let addr = stream
.peer_addr()
.expect("connected streams should have a peer address");
info!("Peer address: {}", addr);
let mut ws_stream = async_tungstenite::accept_async(stream)
.await
.expect("Error during the websocket handshake occurred");
info!("New WebSocket connection: {}", addr);
let mut state = state.lock().unwrap();
let (sender, mut receiver) = mpsc::channel::<WsMessage>(1000);
state.listeners.push(Listener {
id: uuid::Uuid::new_v4(),
sender,
});
drop(state);
task::spawn(async move {
while let Some(msg) = receiver.next().await {
trace!("Sending to one listener!");
if ws_stream.send(msg).await.is_err() {
info!("Listener errored out");
receiver.close();
}
}
});
}
fn main() -> Result<(), Error> {
gst::init()?;
let args = Args::parse();
task::block_on(run(args))
}

4
net/webrtc/plugins/examples/webrtcsink-stats/.gitignore vendored Normal file
View file

@ -0,0 +1,4 @@
/node_modules/
/dist/
/.vscode/
.DS_Store

20
net/webrtc/plugins/examples/webrtcsink-stats/README.md Normal file
View file

@ -0,0 +1,20 @@
# Example web client for webrtcsink-stats-server
This web client will display live statistics as received through a
websocket connected to a `webrtcsink-stats-server`.
Usage:
``` shell
npm install
npm run dev
```
Then navigate to `http://localhost:3000/`. Once consumers are connected
to the webrtc-sink-stats-server, they should be listed on the page, clicking
on any consumer will show a modal with plots for some of the most interesting
statistics.
The stat server can also be specified through the `remote-url` search parameter,
for example you can access a distant stat server with
`http://localhost:3000?remote-uri=my-remoye.com:72522`.

13
net/webrtc/plugins/examples/webrtcsink-stats/index.html Normal file
View file

@ -0,0 +1,13 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<link rel="icon" href="/favicon.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Svelte + TS + Vite App</title>
</head>
<body>
<div id="app"></div>
<script type="module" src="/src/main.ts"></script>
</body>
</html>

919
net/webrtc/plugins/examples/webrtcsink-stats/package-lock.json generated Normal file
View file

@ -0,0 +1,919 @@
{
"name": "webrtcsink-stats",
"version": "0.0.0",
"lockfileVersion": 1,
"requires": true,
"dependencies": {
"@fortawesome/fontawesome-common-types": {
"version": "0.2.36",
"resolved": "https://registry.npmjs.org/@fortawesome/fontawesome-common-types/-/fontawesome-common-types-0.2.36.tgz",
"integrity": "sha512-a/7BiSgobHAgBWeN7N0w+lAhInrGxksn13uK7231n2m8EDPE3BMCl9NZLTGrj9ZXfCmC6LM0QLqXidIizVQ6yg==",
"dev": true
},
"@fortawesome/free-solid-svg-icons": {
"version": "5.15.4",
"resolved": "https://registry.npmjs.org/@fortawesome/free-solid-svg-icons/-/free-solid-svg-icons-5.15.4.tgz",
"integrity": "sha512-JLmQfz6tdtwxoihXLg6lT78BorrFyCf59SAwBM6qV/0zXyVeDygJVb3fk+j5Qat+Yvcxp1buLTY5iDh1ZSAQ8w==",
"dev": true,
"requires": {
"@fortawesome/fontawesome-common-types": "^0.2.36"
}
},
"@nodelib/fs.scandir": {
"version": "2.1.5",
"resolved": "https://registry.npmjs.org/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz",
"integrity": "sha512-vq24Bq3ym5HEQm2NKCr3yXDwjc7vTsEThRDnkp2DK9p1uqLR+DHurm/NOTo0KG7HYHU7eppKZj3MyqYuMBf62g==",
"dev": true,
"requires": {
"@nodelib/fs.stat": "2.0.5",
"run-parallel": "^1.1.9"
}
},
"@nodelib/fs.stat": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/@nodelib/fs.stat/-/fs.stat-2.0.5.tgz",
"integrity": "sha512-RkhPPp2zrqDAQA/2jNhnztcPAlv64XdhIp7a7454A5ovI7Bukxgt7MX7udwAu3zg1DcpPU0rz3VV1SeaqvY4+A==",
"dev": true
},
"@nodelib/fs.walk": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/@nodelib/fs.walk/-/fs.walk-1.2.8.tgz",
"integrity": "sha512-oGB+UxlgWcgQkgwo8GcEGwemoTFt3FIO9ababBmaGwXIoBKZ+GTy0pP185beGg7Llih/NSHSV2XAs1lnznocSg==",
"dev": true,
"requires": {
"@nodelib/fs.scandir": "2.1.5",
"fastq": "^1.6.0"
}
},
"@rollup/pluginutils": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/@rollup/pluginutils/-/pluginutils-4.1.1.tgz",
"integrity": "sha512-clDjivHqWGXi7u+0d2r2sBi4Ie6VLEAzWMIkvJLnDmxoOhBYOTfzGbOQBA32THHm11/LiJbd01tJUpJsbshSWQ==",
"dev": true,
"requires": {
"estree-walker": "^2.0.1",
"picomatch": "^2.2.2"
}
},
"@sveltejs/vite-plugin-svelte": {
"version": "1.0.0-next.30",
"resolved": "https://registry.npmjs.org/@sveltejs/vite-plugin-svelte/-/vite-plugin-svelte-1.0.0-next.30.tgz",
"integrity": "sha512-YQqdMxjL1VgSFk4/+IY3yLwuRRapPafPiZTiaGEq1psbJYSNYUWx9F1zMm32GMsnogg3zn99mGJOqe3ld3HZSg==",
"dev": true,
"requires": {
"@rollup/pluginutils": "^4.1.1",
"debug": "^4.3.2",
"kleur": "^4.1.4",
"magic-string": "^0.25.7",
"require-relative": "^0.8.7",
"svelte-hmr": "^0.14.7"
}
},
"@tsconfig/svelte": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/@tsconfig/svelte/-/svelte-2.0.1.tgz",
"integrity": "sha512-aqkICXbM1oX5FfgZd2qSSAGdyo/NRxjWCamxoyi3T8iVQnzGge19HhDYzZ6NrVOW7bhcWNSq9XexWFtMzbB24A==",
"dev": true
},
"@types/node": {
"version": "16.11.10",
"resolved": "https://registry.npmjs.org/@types/node/-/node-16.11.10.tgz",
"integrity": "sha512-3aRnHa1KlOEEhJ6+CvyHKK5vE9BcLGjtUpwvqYLRvYNQKMfabu3BwfJaA/SLW8dxe28LsNDjtHwePTuzn3gmOA==",
"dev": true
},
"@types/pug": {
"version": "2.0.5",
"resolved": "https://registry.npmjs.org/@types/pug/-/pug-2.0.5.tgz",
"integrity": "sha512-LOnASQoeNZMkzexRuyqcBBDZ6rS+rQxUMkmj5A0PkhhiSZivLIuz6Hxyr1mkGoEZEkk66faROmpMi4fFkrKsBA==",
"dev": true
},
"@types/sass": {
"version": "1.43.1",
"resolved": "https://registry.npmjs.org/@types/sass/-/sass-1.43.1.tgz",
"integrity": "sha512-BPdoIt1lfJ6B7rw35ncdwBZrAssjcwzI5LByIrYs+tpXlj/CAkuVdRsgZDdP4lq5EjyWzwxZCqAoFyHKFwp32g==",
"dev": true,
"requires": {
"@types/node": "*"
}
},
"ansi-styles": {
"version": "4.3.0",
"resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
"integrity": "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg==",
"dev": true,
"requires": {
"color-convert": "^2.0.1"
}
},
"anymatch": {
"version": "3.1.2",
"resolved": "https://registry.npmjs.org/anymatch/-/anymatch-3.1.2.tgz",
"integrity": "sha512-P43ePfOAIupkguHUycrc4qJ9kz8ZiuOUijaETwX7THt0Y/GNK7v0aa8rY816xWjZ7rJdA5XdMcpVFTKMq+RvWg==",
"dev": true,
"requires": {
"normalize-path": "^3.0.0",
"picomatch": "^2.0.4"
}
},
"balanced-match": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz",
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==",
"dev": true
},
"binary-extensions": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/binary-extensions/-/binary-extensions-2.2.0.tgz",
"integrity": "sha512-jDctJ/IVQbZoJykoeHbhXpOlNBqGNcwXJKJog42E5HDPUwQTSdjCHdihjj0DlnheQ7blbT6dHOafNAiS8ooQKA==",
"dev": true
},
"brace-expansion": {
"version": "1.1.11",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz",
"integrity": "sha512-iCuPHDFgrHX7H2vEI/5xpz07zSHB00TpugqhmYtVmMO6518mCuRMoOYFldEBl0g187ufozdaHgWKcYFb61qGiA==",
"dev": true,
"requires": {
"balanced-match": "^1.0.0",
"concat-map": "0.0.1"
}
},
"braces": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/braces/-/braces-3.0.2.tgz",
"integrity": "sha512-b8um+L1RzM3WDSzvhm6gIz1yfTbBt6YTlcEKAvsmqCZZFw46z626lVj9j1yEPW33H5H+lBQpZMP1k8l+78Ha0A==",
"dev": true,
"requires": {
"fill-range": "^7.0.1"
}
},
"buffer-crc32": {
"version": "0.2.13",
"resolved": "https://registry.npmjs.org/buffer-crc32/-/buffer-crc32-0.2.13.tgz",
"integrity": "sha1-DTM+PwDqxQqhRUq9MO+MKl2ackI=",
"dev": true
},
"callsites": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/callsites/-/callsites-3.1.0.tgz",
"integrity": "sha512-P8BjAsXvZS+VIDUI11hHCQEv74YT67YUi5JJFNWIqL235sBmjX4+qx9Muvls5ivyNENctx46xQLQ3aTuE7ssaQ==",
"dev": true
},
"chalk": {
"version": "4.1.2",
"resolved": "https://registry.npmjs.org/chalk/-/chalk-4.1.2.tgz",
"integrity": "sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==",
"dev": true,
"requires": {
"ansi-styles": "^4.1.0",
"supports-color": "^7.1.0"
}
},
"chokidar": {
"version": "3.5.2",
"resolved": "https://registry.npmjs.org/chokidar/-/chokidar-3.5.2.tgz",
"integrity": "sha512-ekGhOnNVPgT77r4K/U3GDhu+FQ2S8TnK/s2KbIGXi0SZWuwkZ2QNyfWdZW+TVfn84DpEP7rLeCt2UI6bJ8GwbQ==",
"dev": true,
"requires": {
"anymatch": "~3.1.2",
"braces": "~3.0.2",
"fsevents": "~2.3.2",
"glob-parent": "~5.1.2",
"is-binary-path": "~2.1.0",
"is-glob": "~4.0.1",
"normalize-path": "~3.0.0",
"readdirp": "~3.6.0"
}
},
"color-convert": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
"integrity": "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ==",
"dev": true,
"requires": {
"color-name": "~1.1.4"
}
},
"color-name": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz",
"integrity": "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA==",
"dev": true
},
"concat-map": {
"version": "0.0.1",
"resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz",
"integrity": "sha1-2Klr13/Wjfd5OnMDajug1UBdR3s=",
"dev": true
},
"debug": {
"version": "4.3.2",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.3.2.tgz",
"integrity": "sha512-mOp8wKcvj7XxC78zLgw/ZA+6TSgkoE2C/ienthhRD298T7UNwAg9diBpLRxC0mOezLl4B0xV7M0cCO6P/O0Xhw==",
"dev": true,
"requires": {
"ms": "2.1.2"
}
},
"detect-indent": {
"version": "6.1.0",
"resolved": "https://registry.npmjs.org/detect-indent/-/detect-indent-6.1.0.tgz",
"integrity": "sha512-reYkTUJAZb9gUuZ2RvVCNhVHdg62RHnJ7WJl8ftMi4diZ6NWlciOzQN88pUhSELEwflJht4oQDv0F0BMlwaYtA==",
"dev": true
},
"es6-promise": {
"version": "3.3.1",
"resolved": "https://registry.npmjs.org/es6-promise/-/es6-promise-3.3.1.tgz",
"integrity": "sha1-oIzd6EzNvzTQJ6FFG8kdS80ophM=",
"dev": true
},
"esbuild": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.13.15.tgz",
"integrity": "sha512-raCxt02HBKv8RJxE8vkTSCXGIyKHdEdGfUmiYb8wnabnaEmHzyW7DCHb5tEN0xU8ryqg5xw54mcwnYkC4x3AIw==",
"dev": true,
"requires": {
"esbuild-android-arm64": "0.13.15",
"esbuild-darwin-64": "0.13.15",
"esbuild-darwin-arm64": "0.13.15",
"esbuild-freebsd-64": "0.13.15",
"esbuild-freebsd-arm64": "0.13.15",
"esbuild-linux-32": "0.13.15",
"esbuild-linux-64": "0.13.15",
"esbuild-linux-arm": "0.13.15",
"esbuild-linux-arm64": "0.13.15",
"esbuild-linux-mips64le": "0.13.15",
"esbuild-linux-ppc64le": "0.13.15",
"esbuild-netbsd-64": "0.13.15",
"esbuild-openbsd-64": "0.13.15",
"esbuild-sunos-64": "0.13.15",
"esbuild-windows-32": "0.13.15",
"esbuild-windows-64": "0.13.15",
"esbuild-windows-arm64": "0.13.15"
}
},
"esbuild-android-arm64": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-android-arm64/-/esbuild-android-arm64-0.13.15.tgz",
"integrity": "sha512-m602nft/XXeO8YQPUDVoHfjyRVPdPgjyyXOxZ44MK/agewFFkPa8tUo6lAzSWh5Ui5PB4KR9UIFTSBKh/RrCmg==",
"dev": true,
"optional": true
},
"esbuild-darwin-64": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-darwin-64/-/esbuild-darwin-64-0.13.15.tgz",
"integrity": "sha512-ihOQRGs2yyp7t5bArCwnvn2Atr6X4axqPpEdCFPVp7iUj4cVSdisgvEKdNR7yH3JDjW6aQDw40iQFoTqejqxvQ==",
"dev": true,
"optional": true
},
"esbuild-darwin-arm64": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-darwin-arm64/-/esbuild-darwin-arm64-0.13.15.tgz",
"integrity": "sha512-i1FZssTVxUqNlJ6cBTj5YQj4imWy3m49RZRnHhLpefFIh0To05ow9DTrXROTE1urGTQCloFUXTX8QfGJy1P8dQ==",
"dev": true,
"optional": true
},
"esbuild-freebsd-64": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-freebsd-64/-/esbuild-freebsd-64-0.13.15.tgz",
"integrity": "sha512-G3dLBXUI6lC6Z09/x+WtXBXbOYQZ0E8TDBqvn7aMaOCzryJs8LyVXKY4CPnHFXZAbSwkCbqiPuSQ1+HhrNk7EA==",
"dev": true,
"optional": true
},
"esbuild-freebsd-arm64": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-freebsd-arm64/-/esbuild-freebsd-arm64-0.13.15.tgz",
"integrity": "sha512-KJx0fzEDf1uhNOZQStV4ujg30WlnwqUASaGSFPhznLM/bbheu9HhqZ6mJJZM32lkyfGJikw0jg7v3S0oAvtvQQ==",
"dev": true,
"optional": true
},
"esbuild-linux-32": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-linux-32/-/esbuild-linux-32-0.13.15.tgz",
"integrity": "sha512-ZvTBPk0YWCLMCXiFmD5EUtB30zIPvC5Itxz0mdTu/xZBbbHJftQgLWY49wEPSn2T/TxahYCRDWun5smRa0Tu+g==",
"dev": true,
"optional": true
},
"esbuild-linux-64": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-linux-64/-/esbuild-linux-64-0.13.15.tgz",
"integrity": "sha512-eCKzkNSLywNeQTRBxJRQ0jxRCl2YWdMB3+PkWFo2BBQYC5mISLIVIjThNtn6HUNqua1pnvgP5xX0nHbZbPj5oA==",
"dev": true,
"optional": true
},
"esbuild-linux-arm": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-linux-arm/-/esbuild-linux-arm-0.13.15.tgz",
"integrity": "sha512-wUHttDi/ol0tD8ZgUMDH8Ef7IbDX+/UsWJOXaAyTdkT7Yy9ZBqPg8bgB/Dn3CZ9SBpNieozrPRHm0BGww7W/jA==",
"dev": true,
"optional": true
},
"esbuild-linux-arm64": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-linux-arm64/-/esbuild-linux-arm64-0.13.15.tgz",
"integrity": "sha512-bYpuUlN6qYU9slzr/ltyLTR9YTBS7qUDymO8SV7kjeNext61OdmqFAzuVZom+OLW1HPHseBfJ/JfdSlx8oTUoA==",
"dev": true,
"optional": true
},
"esbuild-linux-mips64le": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-linux-mips64le/-/esbuild-linux-mips64le-0.13.15.tgz",
"integrity": "sha512-KlVjIG828uFPyJkO/8gKwy9RbXhCEUeFsCGOJBepUlpa7G8/SeZgncUEz/tOOUJTcWMTmFMtdd3GElGyAtbSWg==",
"dev": true,
"optional": true
},
"esbuild-linux-ppc64le": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-linux-ppc64le/-/esbuild-linux-ppc64le-0.13.15.tgz",
"integrity": "sha512-h6gYF+OsaqEuBjeesTBtUPw0bmiDu7eAeuc2OEH9S6mV9/jPhPdhOWzdeshb0BskRZxPhxPOjqZ+/OqLcxQwEQ==",
"dev": true,
"optional": true
},
"esbuild-netbsd-64": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-netbsd-64/-/esbuild-netbsd-64-0.13.15.tgz",
"integrity": "sha512-3+yE9emwoevLMyvu+iR3rsa+Xwhie7ZEHMGDQ6dkqP/ndFzRHkobHUKTe+NCApSqG5ce2z4rFu+NX/UHnxlh3w==",
"dev": true,
"optional": true
},
"esbuild-openbsd-64": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-openbsd-64/-/esbuild-openbsd-64-0.13.15.tgz",
"integrity": "sha512-wTfvtwYJYAFL1fSs8yHIdf5GEE4NkbtbXtjLWjM3Cw8mmQKqsg8kTiqJ9NJQe5NX/5Qlo7Xd9r1yKMMkHllp5g==",
"dev": true,
"optional": true
},
"esbuild-sunos-64": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-sunos-64/-/esbuild-sunos-64-0.13.15.tgz",
"integrity": "sha512-lbivT9Bx3t1iWWrSnGyBP9ODriEvWDRiweAs69vI+miJoeKwHWOComSRukttbuzjZ8r1q0mQJ8Z7yUsDJ3hKdw==",
"dev": true,
"optional": true
},
"esbuild-windows-32": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-windows-32/-/esbuild-windows-32-0.13.15.tgz",
"integrity": "sha512-fDMEf2g3SsJ599MBr50cY5ve5lP1wyVwTe6aLJsM01KtxyKkB4UT+fc5MXQFn3RLrAIAZOG+tHC+yXObpSn7Nw==",
"dev": true,
"optional": true
},
"esbuild-windows-64": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-windows-64/-/esbuild-windows-64-0.13.15.tgz",
"integrity": "sha512-9aMsPRGDWCd3bGjUIKG/ZOJPKsiztlxl/Q3C1XDswO6eNX/Jtwu4M+jb6YDH9hRSUflQWX0XKAfWzgy5Wk54JQ==",
"dev": true,
"optional": true
},
"esbuild-windows-arm64": {
"version": "0.13.15",
"resolved": "https://registry.npmjs.org/esbuild-windows-arm64/-/esbuild-windows-arm64-0.13.15.tgz",
"integrity": "sha512-zzvyCVVpbwQQATaf3IG8mu1IwGEiDxKkYUdA4FpoCHi1KtPa13jeScYDjlW0Qh+ebWzpKfR2ZwvqAQkSWNcKjA==",
"dev": true,
"optional": true
},
"estree-walker": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/estree-walker/-/estree-walker-2.0.2.tgz",
"integrity": "sha512-Rfkk/Mp/DL7JVje3u18FxFujQlTNR2q6QfMSMB7AvCBx91NGj/ba3kCfza0f6dVDbw7YlRf/nDrn7pQrCCyQ/w==",
"dev": true
},
"fast-glob": {
"version": "3.2.7",
"resolved": "https://registry.npmjs.org/fast-glob/-/fast-glob-3.2.7.tgz",
"integrity": "sha512-rYGMRwip6lUMvYD3BTScMwT1HtAs2d71SMv66Vrxs0IekGZEjhM0pcMfjQPnknBt2zeCwQMEupiN02ZP4DiT1Q==",
"dev": true,
"requires": {
"@nodelib/fs.stat": "^2.0.2",
"@nodelib/fs.walk": "^1.2.3",
"glob-parent": "^5.1.2",
"merge2": "^1.3.0",
"micromatch": "^4.0.4"
}
},
"fastq": {
"version": "1.13.0",
"resolved": "https://registry.npmjs.org/fastq/-/fastq-1.13.0.tgz",
"integrity": "sha512-YpkpUnK8od0o1hmeSc7UUs/eB/vIPWJYjKck2QKIzAf71Vm1AAQ3EbuZB3g2JIy+pg+ERD0vqI79KyZiB2e2Nw==",
"dev": true,
"requires": {
"reusify": "^1.0.4"
}
},
"fill-range": {
"version": "7.0.1",
"resolved": "https://registry.npmjs.org/fill-range/-/fill-range-7.0.1.tgz",
"integrity": "sha512-qOo9F+dMUmC2Lcb4BbVvnKJxTPjCm+RRpe4gDuGrzkL7mEVl/djYSu2OdQ2Pa302N4oqkSg9ir6jaLWJ2USVpQ==",
"dev": true,
"requires": {
"to-regex-range": "^5.0.1"
}
},
"fs.realpath": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/fs.realpath/-/fs.realpath-1.0.0.tgz",
"integrity": "sha1-FQStJSMVjKpA20onh8sBQRmU6k8=",
"dev": true
},
"fsevents": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"dev": true,
"optional": true
},
"function-bind": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.1.tgz",
"integrity": "sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==",
"dev": true
},
"glob": {
"version": "7.2.0",
"resolved": "https://registry.npmjs.org/glob/-/glob-7.2.0.tgz",
"integrity": "sha512-lmLf6gtyrPq8tTjSmrO94wBeQbFR3HbLHbuyD69wuyQkImp2hWqMGB47OX65FBkPffO641IP9jWa1z4ivqG26Q==",
"dev": true,
"requires": {
"fs.realpath": "^1.0.0",
"inflight": "^1.0.4",
"inherits": "2",
"minimatch": "^3.0.4",
"once": "^1.3.0",
"path-is-absolute": "^1.0.0"
}
},
"glob-parent": {
"version": "5.1.2",
"resolved": "https://registry.npmjs.org/glob-parent/-/glob-parent-5.1.2.tgz",
"integrity": "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow==",
"dev": true,
"requires": {
"is-glob": "^4.0.1"
}
},
"graceful-fs": {
"version": "4.2.8",
"resolved": "https://registry.npmjs.org/graceful-fs/-/graceful-fs-4.2.8.tgz",
"integrity": "sha512-qkIilPUYcNhJpd33n0GBXTB1MMPp14TxEsEs0pTrsSVucApsYzW5V+Q8Qxhik6KU3evy+qkAAowTByymK0avdg==",
"dev": true
},
"has": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/has/-/has-1.0.3.tgz",
"integrity": "sha512-f2dvO0VU6Oej7RkWJGrehjbzMAjFp5/VKPp5tTpWIV4JHHZK1/BxbFRtf/siA2SWTe09caDmVtYYzWEIbBS4zw==",
"dev": true,
"requires": {
"function-bind": "^1.1.1"
}
},
"has-flag": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/has-flag/-/has-flag-4.0.0.tgz",
"integrity": "sha512-EykJT/Q1KjTWctppgIAgfSO0tKVuZUjhgMr17kqTumMl6Afv3EISleU7qZUzoXDFTAHTDC4NOoG/ZxU3EvlMPQ==",
"dev": true
},
"import-fresh": {
"version": "3.3.0",
"resolved": "https://registry.npmjs.org/import-fresh/-/import-fresh-3.3.0.tgz",
"integrity": "sha512-veYYhQa+D1QBKznvhUHxb8faxlrwUnxseDAbAp457E0wLNio2bOSKnjYDhMj+YiAq61xrMGhQk9iXVk5FzgQMw==",
"dev": true,
"requires": {
"parent-module": "^1.0.0",
"resolve-from": "^4.0.0"
}
},
"inflight": {
"version": "1.0.6",
"resolved": "https://registry.npmjs.org/inflight/-/inflight-1.0.6.tgz",
"integrity": "sha1-Sb1jMdfQLQwJvJEKEHW6gWW1bfk=",
"dev": true,
"requires": {
"once": "^1.3.0",
"wrappy": "1"
}
},
"inherits": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz",
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"dev": true
},
"is-binary-path": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/is-binary-path/-/is-binary-path-2.1.0.tgz",
"integrity": "sha512-ZMERYes6pDydyuGidse7OsHxtbI7WVeUEozgR/g7rd0xUimYNlvZRE/K2MgZTjWy725IfelLeVcEM97mmtRGXw==",
"dev": true,
"requires": {
"binary-extensions": "^2.0.0"
}
},
"is-core-module": {
"version": "2.8.0",
"resolved": "https://registry.npmjs.org/is-core-module/-/is-core-module-2.8.0.tgz",
"integrity": "sha512-vd15qHsaqrRL7dtH6QNuy0ndJmRDrS9HAM1CAiSifNUFv4x1a0CCVsj18hJ1mShxIG6T2i1sO78MkP56r0nYRw==",
"dev": true,
"requires": {
"has": "^1.0.3"
}
},
"is-extglob": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/is-extglob/-/is-extglob-2.1.1.tgz",
"integrity": "sha1-qIwCU1eR8C7TfHahueqXc8gz+MI=",
"dev": true
},
"is-glob": {
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/is-glob/-/is-glob-4.0.3.tgz",
"integrity": "sha512-xelSayHH36ZgE7ZWhli7pW34hNbNl8Ojv5KVmkJD4hBdD3th8Tfk9vYasLM+mXWOZhFkgZfxhLSnrwRr4elSSg==",
"dev": true,
"requires": {
"is-extglob": "^2.1.1"
}
},
"is-number": {
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/is-number/-/is-number-7.0.0.tgz",
"integrity": "sha512-41Cifkg6e8TylSpdtTpeLVMqvSBEVzTttHvERD741+pnZ8ANv0004MRL43QKPDlK9cGvNp6NZWZUBlbGXYxxng==",
"dev": true
},
"kleur": {
"version": "4.1.4",
"resolved": "https://registry.npmjs.org/kleur/-/kleur-4.1.4.tgz",
"integrity": "sha512-8QADVssbrFjivHWQU7KkMgptGTl6WAcSdlbBPY4uNF+mWr6DGcKrvY2w4FQJoXch7+fKMjj0dRrL75vk3k23OA==",
"dev": true
},
"magic-string": {
"version": "0.25.7",
"resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.25.7.tgz",
"integrity": "sha512-4CrMT5DOHTDk4HYDlzmwu4FVCcIYI8gauveasrdCu2IKIFOJ3f0v/8MDGJCDL9oD2ppz/Av1b0Nj345H9M+XIA==",
"dev": true,
"requires": {
"sourcemap-codec": "^1.4.4"
}
},
"merge2": {
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/merge2/-/merge2-1.4.1.tgz",
"integrity": "sha512-8q7VEgMJW4J8tcfVPy8g09NcQwZdbwFEqhe/WZkoIzjn/3TGDwtOCYtXGxA3O8tPzpczCCDgv+P2P5y00ZJOOg==",
"dev": true
},
"micromatch": {
"version": "4.0.4",
"resolved": "https://registry.npmjs.org/micromatch/-/micromatch-4.0.4.tgz",
"integrity": "sha512-pRmzw/XUcwXGpD9aI9q/0XOwLNygjETJ8y0ao0wdqprrzDa4YnxLcz7fQRZr8voh8V10kGhABbNcHVk5wHgWwg==",
"dev": true,
"requires": {
"braces": "^3.0.1",
"picomatch": "^2.2.3"
}
},
"min-indent": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/min-indent/-/min-indent-1.0.1.tgz",
"integrity": "sha512-I9jwMn07Sy/IwOj3zVkVik2JTvgpaykDZEigL6Rx6N9LbMywwUSMtxET+7lVoDLLd3O3IXwJwvuuns8UB/HeAg==",
"dev": true
},
"minimatch": {
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.0.4.tgz",
"integrity": "sha512-yJHVQEhyqPLUTgt9B83PXu6W3rx4MvvHvSUvToogpwoGDOUQ+yDrR0HRot+yOCdCO7u4hX3pWft6kWBBcqh0UA==",
"dev": true,
"requires": {
"brace-expansion": "^1.1.7"
}
},
"minimist": {
"version": "1.2.5",
"resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.5.tgz",
"integrity": "sha512-FM9nNUYrRBAELZQT3xeZQ7fmMOBg6nWNmJKTcgsJeaLstP/UODVpGsr5OhXhhXg6f+qtJ8uiZ+PUxkDWcgIXLw==",
"dev": true
},
"mkdirp": {
"version": "0.5.5",
"resolved": "https://registry.npmjs.org/mkdirp/-/mkdirp-0.5.5.tgz",
"integrity": "sha512-NKmAlESf6jMGym1++R0Ra7wvhV+wFW63FaSOFPwRahvea0gMUcGUhVeAg/0BC0wiv9ih5NYPB1Wn1UEI1/L+xQ==",
"dev": true,
"requires": {
"minimist": "^1.2.5"
}
},
"moment": {
"version": "2.29.1",
"resolved": "https://registry.npmjs.org/moment/-/moment-2.29.1.tgz",
"integrity": "sha512-kHmoybcPV8Sqy59DwNDY3Jefr64lK/by/da0ViFcuA4DH0vQg5Q6Ze5VimxkfQNSC+Mls/Kx53s7TjP1RhFEDQ=="
},
"mri": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/mri/-/mri-1.2.0.tgz",
"integrity": "sha512-tzzskb3bG8LvYGFF/mDTpq3jpI6Q9wc3LEmBaghu+DdCssd1FakN7Bc0hVNmEyGq1bq3RgfkCb3cmQLpNPOroA==",
"dev": true
},
"ms": {
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.2.tgz",
"integrity": "sha512-sGkPx+VjMtmA6MX27oA4FBFELFCZZ4S4XqeGOXCv68tT+jb3vk/RyaKWP0PTKyWtmLSM0b+adUTEvbs1PEaH2w==",
"dev": true
},
"nanoid": {
"version": "3.1.30",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.1.30.tgz",
"integrity": "sha512-zJpuPDwOv8D2zq2WRoMe1HsfZthVewpel9CAvTfc/2mBD1uUT/agc5f7GHGWXlYkFvi1mVxe4IjvP2HNrop7nQ==",
"dev": true
},
"normalize-path": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/normalize-path/-/normalize-path-3.0.0.tgz",
"integrity": "sha512-6eZs5Ls3WtCisHWp9S2GUy8dqkpGi4BVSz3GaqiE6ezub0512ESztXUwUB6C6IKbQkY2Pnb/mD4WYojCRwcwLA==",
"dev": true
},
"once": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/once/-/once-1.4.0.tgz",
"integrity": "sha1-WDsap3WWHUsROsF9nFC6753Xa9E=",
"dev": true,
"requires": {
"wrappy": "1"
}
},
"parent-module": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/parent-module/-/parent-module-1.0.1.tgz",
"integrity": "sha512-GQ2EWRpQV8/o+Aw8YqtfZZPfNRWZYkbidE9k5rpl/hC3vtHHBfGm2Ifi6qWV+coDGkrUKZAxE3Lot5kcsRlh+g==",
"dev": true,
"requires": {
"callsites": "^3.0.0"
}
},
"path-is-absolute": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/path-is-absolute/-/path-is-absolute-1.0.1.tgz",
"integrity": "sha1-F0uSaHNVNP+8es5r9TpanhtcX18=",
"dev": true
},
"path-parse": {
"version": "1.0.7",
"resolved": "https://registry.npmjs.org/path-parse/-/path-parse-1.0.7.tgz",
"integrity": "sha512-LDJzPVEEEPR+y48z93A0Ed0yXb8pAByGWo/k5YYdYgpY2/2EsOsksJrq7lOHxryrVOn1ejG6oAp8ahvOIQD8sw==",
"dev": true
},
"picocolors": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.0.0.tgz",
"integrity": "sha512-1fygroTLlHu66zi26VoTDv8yRgm0Fccecssto+MhsZ0D/DGW2sm8E8AjW7NU5VVTRt5GxbeZ5qBuJr+HyLYkjQ==",
"dev": true
},
"picomatch": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-2.3.0.tgz",
"integrity": "sha512-lY1Q/PiJGC2zOv/z391WOTD+Z02bCgsFfvxoXXf6h7kv9o+WmsmzYqrAwY63sNgOxE4xEdq0WyUnXfKeBrSvYw==",
"dev": true
},
"postcss": {
"version": "8.4.4",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.4.4.tgz",
"integrity": "sha512-joU6fBsN6EIer28Lj6GDFoC/5yOZzLCfn0zHAn/MYXI7aPt4m4hK5KC5ovEZXy+lnCjmYIbQWngvju2ddyEr8Q==",
"dev": true,
"requires": {
"nanoid": "^3.1.30",
"picocolors": "^1.0.0",
"source-map-js": "^1.0.1"
}
},
"queue-microtask": {
"version": "1.2.3",
"resolved": "https://registry.npmjs.org/queue-microtask/-/queue-microtask-1.2.3.tgz",
"integrity": "sha512-NuaNSa6flKT5JaSYQzJok04JzTL1CA6aGhv5rfLW3PgqA+M2ChpZQnAC8h8i4ZFkBS8X5RqkDBHA7r4hej3K9A==",
"dev": true
},
"readdirp": {
"version": "3.6.0",
"resolved": "https://registry.npmjs.org/readdirp/-/readdirp-3.6.0.tgz",
"integrity": "sha512-hOS089on8RduqdbhvQ5Z37A0ESjsqz6qnRcffsMU3495FuTdqSm+7bhJ29JvIOsBDEEnan5DPu9t3To9VRlMzA==",
"dev": true,
"requires": {
"picomatch": "^2.2.1"
}
},
"require-relative": {
"version": "0.8.7",
"resolved": "https://registry.npmjs.org/require-relative/-/require-relative-0.8.7.tgz",
"integrity": "sha1-eZlTn8ngR6N5KPoZb44VY9q9Nt4=",
"dev": true
},
"resolve": {
"version": "1.20.0",
"resolved": "https://registry.npmjs.org/resolve/-/resolve-1.20.0.tgz",
"integrity": "sha512-wENBPt4ySzg4ybFQW2TT1zMQucPK95HSh/nq2CFTZVOGut2+pQvSsgtda4d26YrYcr067wjbmzOG8byDPBX63A==",
"dev": true,
"requires": {
"is-core-module": "^2.2.0",
"path-parse": "^1.0.6"
}
},
"resolve-from": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/resolve-from/-/resolve-from-4.0.0.tgz",
"integrity": "sha512-pb/MYmXstAkysRFx8piNI1tGFNQIFA3vkE3Gq4EuA1dF6gHp/+vgZqsCGJapvy8N3Q+4o7FwvquPJcnZ7RYy4g==",
"dev": true
},
"reusify": {
"version": "1.0.4",
"resolved": "https://registry.npmjs.org/reusify/-/reusify-1.0.4.tgz",
"integrity": "sha512-U9nH88a3fc/ekCF1l0/UP1IosiuIjyTh7hBvXVMHYgVcfGvt897Xguj2UOLDeI5BG2m7/uwyaLVT6fbtCwTyzw==",
"dev": true
},
"rimraf": {
"version": "2.7.1",
"resolved": "https://registry.npmjs.org/rimraf/-/rimraf-2.7.1.tgz",
"integrity": "sha512-uWjbaKIK3T1OSVptzX7Nl6PvQ3qAGtKEtVRjRuazjfL3Bx5eI409VZSqgND+4UNnmzLVdPj9FqFJNPqBZFve4w==",
"dev": true,
"requires": {
"glob": "^7.1.3"
}
},
"rollup": {
"version": "2.60.1",
"resolved": "https://registry.npmjs.org/rollup/-/rollup-2.60.1.tgz",
"integrity": "sha512-akwfnpjY0rXEDSn1UTVfKXJhPsEBu+imi1gqBA1ZkHGydUnkV/fWCC90P7rDaLEW8KTwBcS1G3N4893Ndz+jwg==",
"dev": true,
"requires": {
"fsevents": "~2.3.2"
}
},
"run-parallel": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/run-parallel/-/run-parallel-1.2.0.tgz",
"integrity": "sha512-5l4VyZR86LZ/lDxZTR6jqL8AFE2S0IFLMP26AbjsLVADxHdhB/c0GUsH+y39UfCi3dzz8OlQuPmnaJOMoDHQBA==",
"dev": true,
"requires": {
"queue-microtask": "^1.2.2"
}
},
"sade": {
"version": "1.7.4",
"resolved": "https://registry.npmjs.org/sade/-/sade-1.7.4.tgz",
"integrity": "sha512-y5yauMD93rX840MwUJr7C1ysLFBgMspsdTo4UVrDg3fXDvtwOyIqykhVAAm6fk/3au77773itJStObgK+LKaiA==",
"dev": true,
"requires": {
"mri": "^1.1.0"
}
},
"sander": {
"version": "0.5.1",
"resolved": "https://registry.npmjs.org/sander/-/sander-0.5.1.tgz",
"integrity": "sha1-dB4kXiMfB8r7b98PEzrfohalAq0=",
"dev": true,
"requires": {
"es6-promise": "^3.1.2",
"graceful-fs": "^4.1.3",
"mkdirp": "^0.5.1",
"rimraf": "^2.5.2"
}
},
"sass": {
"version": "1.43.5",
"resolved": "https://registry.npmjs.org/sass/-/sass-1.43.5.tgz",
"integrity": "sha512-WuNm+eAryMgQluL7Mbq9M4EruyGGMyal7Lu58FfnRMVWxgUzIvI7aSn60iNt3kn5yZBMR7G84fAGDcwqOF5JOg==",
"dev": true,
"requires": {
"chokidar": ">=3.0.0 <4.0.0"
}
},
"sorcery": {
"version": "0.10.0",
"resolved": "https://registry.npmjs.org/sorcery/-/sorcery-0.10.0.tgz",
"integrity": "sha1-iukK19fLBfxZ8asMY3hF1cFaUrc=",
"dev": true,
"requires": {
"buffer-crc32": "^0.2.5",
"minimist": "^1.2.0",
"sander": "^0.5.0",
"sourcemap-codec": "^1.3.0"
}
},
"source-map": {
"version": "0.7.3",
"resolved": "https://registry.npmjs.org/source-map/-/source-map-0.7.3.tgz",
"integrity": "sha512-CkCj6giN3S+n9qrYiBTX5gystlENnRW5jZeNLHpe6aue+SrHcG5VYwujhW9s4dY31mEGsxBDrHR6oI69fTXsaQ==",
"dev": true
},
"source-map-js": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.0.1.tgz",
"integrity": "sha512-4+TN2b3tqOCd/kaGRJ/sTYA0tR0mdXx26ipdolxcwtJVqEnqNYvlCAt1q3ypy4QMlYus+Zh34RNtYLoq2oQ4IA==",
"dev": true
},
"sourcemap-codec": {
"version": "1.4.8",
"resolved": "https://registry.npmjs.org/sourcemap-codec/-/sourcemap-codec-1.4.8.tgz",
"integrity": "sha512-9NykojV5Uih4lgo5So5dtw+f0JgJX30KCNI8gwhz2J9A15wD0Ml6tjHKwf6fTSa6fAdVBdZeNOs9eJ71qCk8vA==",
"dev": true
},
"strip-indent": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/strip-indent/-/strip-indent-3.0.0.tgz",
"integrity": "sha512-laJTa3Jb+VQpaC6DseHhF7dXVqHTfJPCRDaEbid/drOhgitgYku/letMUqOXFoWV0zIIUbjpdH2t+tYj4bQMRQ==",
"dev": true,
"requires": {
"min-indent": "^1.0.0"
}
},
"supports-color": {
"version": "7.2.0",
"resolved": "https://registry.npmjs.org/supports-color/-/supports-color-7.2.0.tgz",
"integrity": "sha512-qpCAvRl9stuOHveKsn7HncJRvv501qIacKzQlO/+Lwxc9+0q2wLyv4Dfvt80/DPn2pqOBsJdDiogXGR9+OvwRw==",
"dev": true,
"requires": {
"has-flag": "^4.0.0"
}
},
"svelte": {
"version": "3.44.2",
"resolved": "https://registry.npmjs.org/svelte/-/svelte-3.44.2.tgz",
"integrity": "sha512-jrZhZtmH3ZMweXg1Q15onb8QlWD+a5T5Oca4C1jYvSURp2oD35h4A5TV6t6MEa93K4LlX6BkafZPdQoFjw/ylA==",
"dev": true
},
"svelte-check": {
"version": "2.2.10",
"resolved": "https://registry.npmjs.org/svelte-check/-/svelte-check-2.2.10.tgz",
"integrity": "sha512-UVLd/N7hUIG2v6dytofsw8MxYn2iS2hpNSglsGz9Z9b8ZfbJ5jayl4Mm1SXhNwiFs5aklG90zSBJtd7NTK8dTg==",
"dev": true,
"requires": {
"chalk": "^4.0.0",
"chokidar": "^3.4.1",
"fast-glob": "^3.2.7",
"import-fresh": "^3.2.1",
"minimist": "^1.2.5",
"sade": "^1.7.4",
"source-map": "^0.7.3",
"svelte-preprocess": "^4.0.0",
"typescript": "*"
}
},
"svelte-fa": {
"version": "2.4.0",
"resolved": "https://registry.npmjs.org/svelte-fa/-/svelte-fa-2.4.0.tgz",
"integrity": "sha512-0bnbMGbsE1LUnlioDcf27tl2O8kjuXlTXMXzIxC7LoIOWmqn0D+zd539HfLiQbdLuOHGTaynwN9V+4ehhEu1Jw==",
"dev": true
},
"svelte-hmr": {
"version": "0.14.7",
"resolved": "https://registry.npmjs.org/svelte-hmr/-/svelte-hmr-0.14.7.tgz",
"integrity": "sha512-pDrzgcWSoMaK6AJkBWkmgIsecW0GChxYZSZieIYfCP0v2oPyx2CYU/zm7TBIcjLVUPP714WxmViE9Thht4etog==",
"dev": true
},
"svelte-preprocess": {
"version": "4.9.8",
"resolved": "https://registry.npmjs.org/svelte-preprocess/-/svelte-preprocess-4.9.8.tgz",
"integrity": "sha512-EQS/oRZzMtYdAprppZxY3HcysKh11w54MgA63ybtL+TAZ4hVqYOnhw41JVJjWN9dhPnNjjLzvbZ2tMhTsla1Og==",
"dev": true,
"requires": {
"@types/pug": "^2.0.4",
"@types/sass": "^1.16.0",
"detect-indent": "^6.0.0",
"magic-string": "^0.25.7",
"sorcery": "^0.10.0",
"strip-indent": "^3.0.0"
}
},
"to-regex-range": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz",
"integrity": "sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ==",
"dev": true,
"requires": {
"is-number": "^7.0.0"
}
},
"tslib": {
"version": "2.3.1",
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.3.1.tgz",
"integrity": "sha512-77EbyPPpMz+FRFRuAFlWMtmgUWGe9UOG2Z25NqCwiIjRhOf5iKGuzSe5P2w1laq+FkRy4p+PCuVkJSGkzTEKVw==",
"dev": true
},
"typescript": {
"version": "4.5.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-4.5.2.tgz",
"integrity": "sha512-5BlMof9H1yGt0P8/WF+wPNw6GfctgGjXp5hkblpyT+8rkASSmkUKMXrxR0Xg8ThVCi/JnHQiKXeBaEwCeQwMFw==",
"dev": true
},
"vite": {
"version": "2.6.14",
"resolved": "https://registry.npmjs.org/vite/-/vite-2.6.14.tgz",
"integrity": "sha512-2HA9xGyi+EhY2MXo0+A2dRsqsAG3eFNEVIo12olkWhOmc8LfiM+eMdrXf+Ruje9gdXgvSqjLI9freec1RUM5EA==",
"dev": true,
"requires": {
"esbuild": "^0.13.2",
"fsevents": "~2.3.2",
"postcss": "^8.3.8",
"resolve": "^1.20.0",
"rollup": "^2.57.0"
}
},
"wrappy": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz",
"integrity": "sha1-tSQ9jz7BqjXxNkYFvA0QNuMKtp8=",
"dev": true
}
}
}

27
net/webrtc/plugins/examples/webrtcsink-stats/package.json Normal file
View file

@ -0,0 +1,27 @@
{
"name": "webrtcsink-stats",
"version": "0.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"serve": "vite preview",
"check": "svelte-check --tsconfig ./tsconfig.json"
},
"devDependencies": {
"@fortawesome/free-solid-svg-icons": "^5.15.4",
"@sveltejs/vite-plugin-svelte": "^1.0.0-next.11",
"@tsconfig/svelte": "^2.0.1",
"sass": "^1.43.5",
"svelte": "^3.37.0",
"svelte-check": "^2.1.0",
"svelte-fa": "^2.4.0",
"svelte-preprocess": "^4.7.2",
"tslib": "^2.2.0",
"typescript": "^4.3.2",
"vite": "^2.6.4"
},
"dependencies": {
"moment": "^2.29.1"
}
}

BIN
net/webrtc/plugins/examples/webrtcsink-stats/public/favicon.ico Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 1.1 KiB

182
net/webrtc/plugins/examples/webrtcsink-stats/src/App.svelte Normal file
View file

@ -0,0 +1,182 @@
<svelte:head>
<script src="https://cdn.plot.ly/plotly-latest.min.js" type="text/javascript"></script>
</svelte:head>
<script lang="ts">
import Home from '@/pages/Home.svelte'
import Header from '@/components/Header.svelte'
import type { ConsumerType } from '@/types/app'
import { WebSocketStatus, MitigationMode } from '@/types/app'
import { onMount, onDestroy } from 'svelte';
let ws: WebSocket | undefined = undefined
let websocketStatus: WebSocketStatus = WebSocketStatus.Connecting
let consumers: Map<string, ConsumerType> = new Map ()
let consumers_array: Array<ConsumerType> = []
let timeout: ReturnType<typeof setTimeout> | undefined = undefined
const updateConsumerStats = (consumer: ConsumerType, stats: Object) => {
let target_bitrate = 0
let fec_percentage = 0
let keyframe_requests = 0
let retransmission_requests = 0
let bitrate_sent = 0
let bitrate_recv = 0
let packet_loss = 0
let delta_of_delta = 0
if (stats["consumer-stats"]["video-encoders"].length > 0) {
let venc = stats["consumer-stats"]["video-encoders"][0]
target_bitrate = venc["bitrate"]
fec_percentage = venc["fec-percentage"]
consumer.video_codec = venc["codec-name"]
let mitigation_mode = MitigationMode.None
for (let mode of venc["mitigation-mode"].split("+")) {
switch (mode) {
case "none": {
mitigation_mode |= MitigationMode.None
break
}
case "downscaled": {
mitigation_mode |= MitigationMode.Downscaled
break
}
case "downsampled": {
mitigation_mode |= MitigationMode.Downsampled
break
}
}
}
consumer.mitigation_mode = mitigation_mode
}
for (let svalue of Object.values(stats)) {
if (svalue["type"] == "transport") {
let twcc_stats = svalue["gst-twcc-stats"]
if (twcc_stats !== undefined) {
bitrate_sent = twcc_stats["bitrate-sent"]
bitrate_recv = twcc_stats["bitrate-recv"]
packet_loss = twcc_stats["packet-loss-pct"]
delta_of_delta = twcc_stats["avg-delta-of-delta"]
}
} else if (svalue["type"] == "outbound-rtp") {
keyframe_requests += svalue["pli-count"]
retransmission_requests += svalue["nack-count"]
}
}
consumer.stats["target_bitrate"] = target_bitrate
consumer.stats["fec_percentage"] = fec_percentage
consumer.stats["bitrate_sent"] = bitrate_sent
consumer.stats["bitrate_recv"] = bitrate_recv
consumer.stats["packet_loss"] = packet_loss
consumer.stats["delta_of_delta"] = delta_of_delta
consumer.stats["keyframe_requests"] = keyframe_requests
consumer.stats["retransmission_requests"] = retransmission_requests
}
const fetchStats = () => {
const urlParams = new URLSearchParams(window.location.search);
var remote_server = urlParams.get('remote-url');
if (!remote_server)
remote_server = "127.0.0.1:8484"
const ws_url = `ws://${remote_server}`;
console.info(`Logging to ${ws_url}`);
ws = new WebSocket(ws_url);
ws.onerror = () => {
websocketStatus = WebSocketStatus.Error
}
ws.onclose = () => {
websocketStatus = WebSocketStatus.Error
consumers = new Map()
consumers_array = []
timeout = setTimeout(fetchStats, 500)
}
ws.onopen = () => {
websocketStatus = WebSocketStatus.Connected
}
ws.onmessage = (event) => {
let stats = JSON.parse(event.data)
// Set is supposed to be buildable from an iterator,
// no idea why the Arra.from is needed ..
let to_remove = new Set(Array.from(consumers.keys()))
for (let [key, value] of Object.entries(stats)) {
let consumer = undefined;
if (consumers.get(key) === undefined) {
consumer = {
id: key,
video_codec: undefined,
mitigation_mode: MitigationMode.None,
stats: new Map([
["target_bitrate", 0],
["fec_percentage", 0],
["bitrate_sent", 0],
["bitrate_recv", 0],
["packet_loss", 0],
["delta_of_delta", 0],
["keyframe_requests", 0],
["retransmission_requests", 0],
]),
}
consumers.set(key, consumer)
} else {
consumer = consumers.get(key)
}
updateConsumerStats(consumer, value)
to_remove.delete(key)
}
for (let key of to_remove) {
consumers.delete(key)
}
consumers_array = Array.from(consumers.values())
}
}
const closeWebSocket = () => {
if (ws != undefined) {
ws.close();
ws = undefined;
}
if (timeout != undefined) {
clearTimeout(timeout)
timeout = undefined
}
}
onMount(fetchStats)
onDestroy(closeWebSocket)
</script>
<Header websocketStatus={ websocketStatus } />
<Home consumers={ consumers_array } />
<style lang="scss">
:root {
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen,
Ubuntu, Cantarell, 'Open Sans', 'Helvetica Neue', sans-serif;
height: 100%;
}
:global(body) {
/* this will apply to <body> */
margin: 0;
height: 100%;
background-color: #fbfbfb;
}
</style>

BIN
net/webrtc/plugins/examples/webrtcsink-stats/src/assets/h264.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 124 KiB

BIN
net/webrtc/plugins/examples/webrtcsink-stats/src/assets/svelte.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 5.1 KiB

BIN
net/webrtc/plugins/examples/webrtcsink-stats/src/assets/vp8.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 74 KiB

BIN
net/webrtc/plugins/examples/webrtcsink-stats/src/assets/vp9.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 70 KiB

32
net/webrtc/plugins/examples/webrtcsink-stats/src/components/Consumer.svelte Normal file
View file

@ -0,0 +1,32 @@
<script lang="ts">
import type { ConsumerType } from '@/types/app'
import EncoderProps from '@/components/EncoderProps.svelte'
export let consumer: ConsumerType
$: id = consumer.id
</script>
<div class="consumer-card" on:click >
<div class="id">{id}</div>
<EncoderProps
consumer={consumer}
/>
</div>
<style lang="scss">
.consumer-card {
word-break: break-all;
width: 150px;
height: 100px;
margin-right: 15px;
background-color: #fff;
padding: 15px;
border-radius: 10px;
box-shadow: rgb(0 0 0 / 24%) 0px 3px 8px;
.id {
font-weight: bold;
margin-bottom: 10px;
}
}
</style>

53
net/webrtc/plugins/examples/webrtcsink-stats/src/components/EncoderProps.svelte Normal file
View file

@ -0,0 +1,53 @@
<script lang="ts">
import type { ConsumerType } from '@/types/app'
import { MitigationMode } from '@/types/app'
import Fa from 'svelte-fa'
import { faExclamationTriangle, faCheckCircle } from '@fortawesome/free-solid-svg-icons';
import vp8_logo from '@/assets/vp8.png'
import vp9_logo from '@/assets/vp9.png'
import h264_logo from '@/assets/h264.png'
export let consumer: ConsumerType
$: video_codec = consumer.video_codec
$: mitigation_mode = consumer.mitigation_mode
</script>
<div class="encoder-props">
<div class="codec">
{#if video_codec == "video/x-vp8"}
<img src={vp8_logo} alt="VP8">
{:else if video_codec == "video/x-vp9"}
<img src={vp9_logo} alt="VP8">
{:else if video_codec == "video/x-h264"}
<img src={h264_logo} alt="VP8">
{/if}
</div>
<div>
{#if mitigation_mode & MitigationMode.Downsampled && mitigation_mode & MitigationMode.Downscaled}
<abbr title="Very congested link, video is downscaled and downsampled">
<Fa icon={faExclamationTriangle} color="tomato" />
</abbr>
{:else if mitigation_mode & MitigationMode.Downscaled}
<abbr title="Congested link, video is downscaled">
<Fa icon={faExclamationTriangle} color="orange" />
</abbr>
{:else}
<abbr title="Link with minimal to no congestion">
<Fa icon={faCheckCircle} color="lightseagreen" />
</abbr>
{/if}
</div>
</div>
<style lang="scss">
.encoder-props {
display: flex;
justify-content: space-evenly;
.codec {
img {
width: 25px;
}
}
}
</style>

48
net/webrtc/plugins/examples/webrtcsink-stats/src/components/Header.svelte Normal file
View file

@ -0,0 +1,48 @@
<script lang="ts">
import { WebSocketStatus } from '@/types/app'
import logo from '@/assets/svelte.png'
import Fa from 'svelte-fa'
import { faSpinner, faExclamationTriangle, faCheckCircle } from '@fortawesome/free-solid-svg-icons';
export let websocketStatus: WebSocketStatus
</script>
<header class="global-header">
<div class="app-name">
<img src={logo} alt="Svelte Logo" class="logo"/>
<div class="title">WebRTC Stats App</div>
<div>
{#if websocketStatus == WebSocketStatus.Connected}
<Fa icon={faCheckCircle} color="lightseagreen" size="1x" />
{:else if websocketStatus == WebSocketStatus.Connecting}
<Fa icon={faSpinner} color="#afaeae" size="1x" spin />
{:else if websocketStatus == WebSocketStatus.Error}
<Fa icon={faExclamationTriangle} color="tomato" size="1x" />
{/if}
</div>
</div>
</header>
<style lang="scss">
.global-header {
background-color: #313131;
height: 56px;
color: white;
padding: 15px;
box-sizing: border-box;
.app-name {
display: flex;
align-items: center;
height: 100%;
.logo {
height: 100%;
margin-right: 5px;
}
.title {
font-weight: bold;
margin-right: 5px;
}
}
}
</style>

50
net/webrtc/plugins/examples/webrtcsink-stats/src/components/Modal.svelte Normal file
View file

@ -0,0 +1,50 @@
<script lang="ts">
import Fa from 'svelte-fa'
import { faTimes } from '@fortawesome/free-solid-svg-icons';
import { createEventDispatcher } from 'svelte';
const dispatch = createEventDispatcher();
</script>
<div class="modal-overlay">
<div class="modal">
<div class="modal-header">
<slot name="title"></slot>
<div class="close-icon" on:click="{() => dispatch('closeModal')}">
<Fa icon={faTimes} color="#afaeae" size="1x" />
</div>
</div>
<slot name="body"></slot>
<slot name="footer"></slot>
</div>
</div>
<style lang="scss">
.modal {
background-color: white;
padding: 15px 0;
border-radius: 10px;
box-shadow: rgb(0 0 0 / 24%) 0px 3px 8px;
&-overlay {
position: absolute;
top: 0;
height: 100vh;
width: 100vw;
display: flex;
align-items: center;
justify-content: center;
background-color: rgb(0, 0, 0, 0.4);
}
&-header {
display: flex;
justify-content: space-between;
padding: 0 15px 10px;
border-bottom: 3px solid #e2e2e2;
}
.close-icon {
cursor: pointer;
}
}
</style>

142
net/webrtc/plugins/examples/webrtcsink-stats/src/components/PlotConsumerModal.svelte Normal file
View file

@ -0,0 +1,142 @@
<svelte:head>
<script src="https://cdn.plot.ly/plotly-latest.min.js" type="text/javascript"></script>
</svelte:head>
<script lang="ts">
import { createEventDispatcher, onMount, onDestroy } from 'svelte';
import Modal from '@/components/Modal.svelte'
import type { ConsumerType } from '@/types/app'
import EncoderProps from '@/components/EncoderProps.svelte'
export let consumer: ConsumerType
$: if (consumer === undefined) {
dispatch('close')
}
$: id = consumer !== undefined ? consumer.id : undefined
const dispatch = createEventDispatcher();
let interval: ReturnType<typeof setInterval> | undefined = undefined
onMount(() => {
let plotDiv = document.getElementById('plotDiv');
let traces = []
let layout = {
legend: {traceorder: 'reversed'},
height: 800,
}
let ctr = 1;
let domain_step = 1.0 / consumer.stats.size
let domain_margin = 0.05
for (let key of consumer.stats.keys()) {
let trace = {
x: [],
y: [],
xaxis: 'x' + ctr,
yaxis: 'y' + ctr,
mode: 'lines',
line: {shape: 'spline'},
name: key
}
traces.push(trace)
layout['xaxis' + ctr] = {
type: 'date',
}
layout['yaxis' + ctr] = {
domain: [(ctr - 1) * domain_step, (ctr * domain_step) - domain_margin],
rangemode: "tozero",
}
ctr += 1
}
Plotly.newPlot(plotDiv, traces, layout);
interval = setInterval(function() {
let time = new Date()
let ctr = 0
let traces = []
let data_update = {
x: [],
y: [],
}
for (let value of Object.values(consumer.stats)) {
data_update.x.push([time])
data_update.y.push([value])
traces.push(ctr)
ctr += 1
}
Plotly.extendTraces(plotDiv, data_update, traces, 600)
}, 50);
});
onDestroy(() => {
console.log ("destroyed")
if (interval !== undefined ) {
clearInterval (interval)
interval = undefined
}
})
</script>
<Modal on:closeModal="{() => dispatch('close')}">
<div slot="body" class="modal-body">
<div class="id">{id}</div>
<EncoderProps
consumer={consumer}
/>
<div id="plotDiv"></div>
</div>
<div slot="footer" class="modal-footer">
<div class="buttons-wrapper">
<button
class="button"
on:click|stopPropagation="{() => dispatch('close')}"
>
Cancel
</button>
</div>
</div>
</Modal>
<style lang="scss">
.modal {
&-body {
width: 1000px;
padding: 20px 15px 10px;
gap: 15px 0;
.id {
font-weight: bold;
margin-bottom: 10px;
}
}
&-footer {
padding: 0 15px;
.buttons-wrapper {
text-align: right;
}
.button {
height: 30px;
padding: 0 10px;
text-align: center;
box-sizing: content-box;
border-radius: 3px;
border: 1px solid #000;
&:active {
background-color: #b9b7b7;
}
}
}
}
</style>

7
net/webrtc/plugins/examples/webrtcsink-stats/src/main.ts Normal file
View file

@ -0,0 +1,7 @@
import App from './App.svelte'
const app = new App({
target: document.getElementById('app')
})
export default app

64
net/webrtc/plugins/examples/webrtcsink-stats/src/pages/Home.svelte Normal file
View file

@ -0,0 +1,64 @@
<script lang="ts">
import type { ConsumerType } from '@/types/app'
import Consumer from '@/components/Consumer.svelte'
import PlotConsumerModal from '@/components/PlotConsumerModal.svelte'
export let consumers: Array<ConsumerType>
let consumerToPlot: ConsumerType | undefined
let showPlotModal = false
/**
* Display the Plot modal
*
* @param {ConsumerType} consumer
*/
const openPlotConsumer = (consumer: ConsumerType) => {
consumerToPlot = consumer
showPlotModal = true
}
/**
* Close the Plot modal
*
*/
const closePlotConsumer = () => {
consumerToPlot = undefined
showPlotModal = false
}
</script>
<main>
<div class="consumer-card-container">
{#each consumers as consumer}
<Consumer
consumer = {consumer}
on:click="{() => { openPlotConsumer(consumer) }}"
/>
{/each}
</div>
</main>
{#if showPlotModal}
<PlotConsumerModal
consumer={consumers.find(consumer => consumer == consumerToPlot)}
on:close="{closePlotConsumer}"
/>
{/if}
<style lang="scss">
main {
padding: 2em;
margin: 0 auto;
width: 100vw;
box-sizing: border-box;
}
.consumer-card {
&-container {
display: flex;
flex-wrap: wrap;
row-gap: 20px;
justify-content: space-evenly;
}
}
</style>

18
net/webrtc/plugins/examples/webrtcsink-stats/src/types/app.ts Normal file
View file

@ -0,0 +1,18 @@
export enum MitigationMode {
None = 0,
Downscaled = 1,
Downsampled = 2,
}
export interface ConsumerType {
id: string,
video_codec: string | undefined,
mitigation_mode: MitigationMode,
stats: Map<string, number>,
}
export enum WebSocketStatus {
Connecting = 0,
Connected = 1,
Error = 2,
}

2
net/webrtc/plugins/examples/webrtcsink-stats/src/vite-env.d.ts vendored Normal file
View file

@ -0,0 +1,2 @@
/// <reference types="svelte" />
/// <reference types="vite/client" />

13
net/webrtc/plugins/examples/webrtcsink-stats/svelte.config.js Normal file
View file

@ -0,0 +1,13 @@
import sveltePreprocess from 'svelte-preprocess'
import * as sass from 'sass'
export default {
// Consult https://github.com/sveltejs/svelte-preprocess
// for more information about preprocessors
preprocess: sveltePreprocess({
sass: {
sync: true,
implementation: sass,
},
})
}

24
net/webrtc/plugins/examples/webrtcsink-stats/tsconfig.json Normal file
View file

@ -0,0 +1,24 @@
{
"extends": "@tsconfig/svelte/tsconfig.json",
"compilerOptions": {
"target": "esnext",
"module": "esnext",
"moduleResolution": "node",
"resolveJsonModule": true,
"baseUrl": ".",
/**
* Typechecking JS in `.svelte` and `.js` files by default.
* Disable checkJs if you'd like to use dynamic types in JS.
* Note that setting allowJs false does not prevent the use
* of JS in `.svelte` files.
*/
"allowJs": true,
"checkJs": true,
"paths": {
"@/*": [
"src/*"
],
}
},
"include": ["src/**/*.d.ts", "src/**/*.ts", "src/**/*.js", "src/**/*.svelte"]
}

13
net/webrtc/plugins/examples/webrtcsink-stats/vite.config.js Normal file
View file

@ -0,0 +1,13 @@
import { defineConfig } from 'vite'
import { svelte } from '@sveltejs/vite-plugin-svelte'
import path from 'path';
// https://vitejs.dev/config/
export default defineConfig({
plugins: [svelte()],
resolve: {
alias: {
'@': path.resolve('/src'),
},
}
})

1370
net/webrtc/plugins/src/gcc/imp.rs Normal file
View file

File diff suppressed because it is too large Load diff

16
net/webrtc/plugins/src/gcc/mod.rs Normal file
View file

@ -0,0 +1,16 @@
use gst::glib;
use gst::prelude::*;
mod imp;
glib::wrapper! {
pub struct BandwidthEstimator(ObjectSubclass<imp::BandwidthEstimator>) @extends gst::Element, gst::Object;
}
pub fn register(plugin: &gst::Plugin) -> Result<(), glib::BoolError> {
gst::Element::register(
Some(plugin),
"rtpgccbwe",
gst::Rank::None,
BandwidthEstimator::static_type(),
)
}

24
net/webrtc/plugins/src/lib.rs Normal file
View file

@ -0,0 +1,24 @@
use gst::glib;
pub mod gcc;
mod signaller;
pub mod webrtcsink;
fn plugin_init(plugin: &gst::Plugin) -> Result<(), glib::BoolError> {
webrtcsink::register(plugin)?;
gcc::register(plugin)?;
Ok(())
}
gst::plugin_define!(
webrtcsink,
env!("CARGO_PKG_DESCRIPTION"),
plugin_init,
concat!(env!("CARGO_PKG_VERSION"), "-", env!("COMMIT_ID")),
"MPL-2.0",
env!("CARGO_PKG_NAME"),
env!("CARGO_PKG_NAME"),
env!("CARGO_PKG_REPOSITORY"),
env!("BUILD_REL_DATE")
);

478
net/webrtc/plugins/src/signaller/imp.rs Normal file
View file

@ -0,0 +1,478 @@
use crate::webrtcsink::WebRTCSink;
use anyhow::{anyhow, Error};
use async_std::task;
use async_tungstenite::tungstenite::Message as WsMessage;
use futures::channel::mpsc;
use futures::prelude::*;
use gst::glib::prelude::*;
use gst::glib::{self, Type};
use gst::prelude::*;
use gst::subclass::prelude::*;
use once_cell::sync::Lazy;
use std::collections::HashMap;
use std::path::PathBuf;
use std::sync::Mutex;
use webrtcsink_protocol as p;
static CAT: Lazy<gst::DebugCategory> = Lazy::new(|| {
gst::DebugCategory::new(
"webrtcsink-signaller",
gst::DebugColorFlags::empty(),
Some("WebRTC sink signaller"),
)
});
#[derive(Default)]
struct State {
/// Sender for the websocket messages
websocket_sender: Option<mpsc::Sender<p::IncomingMessage>>,
send_task_handle: Option<task::JoinHandle<Result<(), Error>>>,
receive_task_handle: Option<task::JoinHandle<()>>,
}
#[derive(Clone)]
struct Settings {
address: Option<String>,
cafile: Option<PathBuf>,
}
impl Default for Settings {
fn default() -> Self {
Self {
address: Some("ws://127.0.0.1:8443".to_string()),
cafile: None,
}
}
}
#[derive(Default)]
pub struct Signaller {
state: Mutex<State>,
settings: Mutex<Settings>,
}
impl Signaller {
async fn connect(&self, element: &WebRTCSink) -> Result<(), Error> {
let settings = self.settings.lock().unwrap().clone();
let connector = if let Some(path) = settings.cafile {
let cert = async_std::fs::read_to_string(&path).await?;
let cert = async_native_tls::Certificate::from_pem(cert.as_bytes())?;
let connector = async_native_tls::TlsConnector::new();
Some(connector.add_root_certificate(cert))
} else {
None
};
let (ws, _) = async_tungstenite::async_std::connect_async_with_tls_connector(
settings.address.unwrap(),
connector,
)
.await?;
gst::info!(CAT, obj: element, "connected");
// Channel for asynchronously sending out websocket message
let (mut ws_sink, mut ws_stream) = ws.split();
// 1000 is completely arbitrary, we simply don't want infinite piling
// up of messages as with unbounded
let (mut websocket_sender, mut websocket_receiver) =
mpsc::channel::<p::IncomingMessage>(1000);
let element_clone = element.downgrade();
let send_task_handle = task::spawn(async move {
while let Some(msg) = websocket_receiver.next().await {
if let Some(element) = element_clone.upgrade() {
gst::trace!(CAT, obj: &element, "Sending websocket message {:?}", msg);
}
ws_sink
.send(WsMessage::Text(serde_json::to_string(&msg).unwrap()))
.await?;
}
if let Some(element) = element_clone.upgrade() {
gst::info!(CAT, obj: &element, "Done sending");
}
ws_sink.send(WsMessage::Close(None)).await?;
ws_sink.close().await?;
Ok::<(), Error>(())
});
let meta = if let Some(meta) = element.property::<Option<gst::Structure>>("meta") {
serialize_value(&meta.to_value())
} else {
None
};
websocket_sender
.send(p::IncomingMessage::SetPeerStatus(p::PeerStatus {
roles: vec![p::PeerRole::Producer],
meta,
peer_id: None,
}))
.await?;
let element_clone = element.downgrade();
let receive_task_handle = task::spawn(async move {
while let Some(msg) = async_std::stream::StreamExt::next(&mut ws_stream).await {
if let Some(element) = element_clone.upgrade() {
match msg {
Ok(WsMessage::Text(msg)) => {
gst::trace!(CAT, obj: &element, "Received message {}", msg);
if let Ok(msg) = serde_json::from_str::<p::OutgoingMessage>(&msg) {
match msg {
p::OutgoingMessage::Welcome { peer_id } => {
gst::info!(
CAT,
obj: &element,
"We are registered with the server, our peer id is {}",
peer_id
);
}
p::OutgoingMessage::StartSession {
session_id,
peer_id,
} => {
if let Err(err) =
element.start_session(&session_id, &peer_id)
{
gst::warning!(CAT, obj: &element, "{}", err);
}
}
p::OutgoingMessage::EndSession(session_info) => {
if let Err(err) =
element.end_session(&session_info.session_id)
{
gst::warning!(CAT, obj: &element, "{}", err);
}
}
p::OutgoingMessage::Peer(p::PeerMessage {
session_id,
peer_message,
}) => match peer_message {
p::PeerMessageInner::Sdp(p::SdpMessage::Answer { sdp }) => {
if let Err(err) = element.handle_sdp(
&session_id,
&gst_webrtc::WebRTCSessionDescription::new(
gst_webrtc::WebRTCSDPType::Answer,
gst_sdp::SDPMessage::parse_buffer(
sdp.as_bytes(),
)
.unwrap(),
),
) {
gst::warning!(CAT, obj: &element, "{}", err);
}
}
p::PeerMessageInner::Sdp(p::SdpMessage::Offer {
..
}) => {
gst::warning!(
CAT,
obj: &element,
"Ignoring offer from peer"
);
}
p::PeerMessageInner::Ice {
candidate,
sdp_m_line_index,
} => {
if let Err(err) = element.handle_ice(
&session_id,
Some(sdp_m_line_index),
None,
&candidate,
) {
gst::warning!(CAT, obj: &element, "{}", err);
}
}
},
_ => {
gst::warning!(
CAT,
obj: &element,
"Ignoring unsupported message {:?}",
msg
);
}
}
} else {
gst::error!(
CAT,
obj: &element,
"Unknown message from server: {}",
msg
);
element.handle_signalling_error(
anyhow!("Unknown message from server: {}", msg).into(),
);
}
}
Ok(WsMessage::Close(reason)) => {
gst::info!(
CAT,
obj: &element,
"websocket connection closed: {:?}",
reason
);
break;
}
Ok(_) => (),
Err(err) => {
element.handle_signalling_error(
anyhow!("Error receiving: {}", err).into(),
);
break;
}
}
} else {
break;
}
}
if let Some(element) = element_clone.upgrade() {
gst::info!(CAT, obj: &element, "Stopped websocket receiving");
}
});
let mut state = self.state.lock().unwrap();
state.websocket_sender = Some(websocket_sender);
state.send_task_handle = Some(send_task_handle);
state.receive_task_handle = Some(receive_task_handle);
Ok(())
}
pub fn start(&self, element: &WebRTCSink) {
let this = self.instance().clone();
let element_clone = element.clone();
task::spawn(async move {
let this = Self::from_instance(&this);
if let Err(err) = this.connect(&element_clone).await {
element_clone.handle_signalling_error(err.into());
}
});
}
pub fn handle_sdp(
&self,
element: &WebRTCSink,
session_id: &str,
sdp: &gst_webrtc::WebRTCSessionDescription,
) {
let state = self.state.lock().unwrap();
let msg = p::IncomingMessage::Peer(p::PeerMessage {
session_id: session_id.to_string(),
peer_message: p::PeerMessageInner::Sdp(p::SdpMessage::Offer {
sdp: sdp.sdp().as_text().unwrap(),
}),
});
if let Some(mut sender) = state.websocket_sender.clone() {
let element = element.downgrade();
task::spawn(async move {
if let Err(err) = sender.send(msg).await {
if let Some(element) = element.upgrade() {
element.handle_signalling_error(anyhow!("Error: {}", err).into());
}
}
});
}
}
pub fn handle_ice(
&self,
element: &WebRTCSink,
session_id: &str,
candidate: &str,
sdp_m_line_index: Option<u32>,
_sdp_mid: Option<String>,
) {
let state = self.state.lock().unwrap();
let msg = p::IncomingMessage::Peer(p::PeerMessage {
session_id: session_id.to_string(),
peer_message: p::PeerMessageInner::Ice {
candidate: candidate.to_string(),
sdp_m_line_index: sdp_m_line_index.unwrap(),
},
});
if let Some(mut sender) = state.websocket_sender.clone() {
let element = element.downgrade();
task::spawn(async move {
if let Err(err) = sender.send(msg).await {
if let Some(element) = element.upgrade() {
element.handle_signalling_error(anyhow!("Error: {}", err).into());
}
}
});
}
}
pub fn stop(&self, element: &WebRTCSink) {
gst::info!(CAT, obj: element, "Stopping now");
let mut state = self.state.lock().unwrap();
let send_task_handle = state.send_task_handle.take();
let receive_task_handle = state.receive_task_handle.take();
if let Some(mut sender) = state.websocket_sender.take() {
task::block_on(async move {
sender.close_channel();
if let Some(handle) = send_task_handle {
if let Err(err) = handle.await {
gst::warning!(CAT, obj: element, "Error while joining send task: {}", err);
}
}
if let Some(handle) = receive_task_handle {
handle.await;
}
});
}
}
pub fn end_session(&self, element: &WebRTCSink, session_id: &str) {
gst::debug!(CAT, obj: element, "Signalling session {} ended", session_id);
let state = self.state.lock().unwrap();
let session_id = session_id.to_string();
let element = element.downgrade();
if let Some(mut sender) = state.websocket_sender.clone() {
task::spawn(async move {
if let Err(err) = sender
.send(p::IncomingMessage::EndSession(p::EndSessionMessage {
session_id: session_id.to_string(),
}))
.await
{
if let Some(element) = element.upgrade() {
element.handle_signalling_error(anyhow!("Error: {}", err).into());
}
}
});
}
}
}
#[glib::object_subclass]
impl ObjectSubclass for Signaller {
const NAME: &'static str = "RsWebRTCSinkSignaller";
type Type = super::Signaller;
type ParentType = glib::Object;
}
impl ObjectImpl for Signaller {
fn properties() -> &'static [glib::ParamSpec] {
static PROPERTIES: Lazy<Vec<glib::ParamSpec>> = Lazy::new(|| {
vec![
glib::ParamSpecString::new(
"address",
"Address",
"Address of the signalling server",
Some("ws://127.0.0.1:8443"),
glib::ParamFlags::READWRITE,
),
glib::ParamSpecString::new(
"cafile",
"CA file",
"Path to a Certificate file to add to the set of roots the TLS connector will trust",
None,
glib::ParamFlags::READWRITE,
),
]
});
PROPERTIES.as_ref()
}
fn set_property(
&self,
_id: usize,
value: &glib::Value,
pspec: &glib::ParamSpec,
) {
match pspec.name() {
"address" => {
let address: Option<_> = value.get().expect("type checked upstream");
if let Some(address) = address {
gst::info!(CAT, "Signaller address set to {}", address);
let mut settings = self.settings.lock().unwrap();
settings.address = Some(address);
} else {
gst::error!(CAT, "address can't be None");
}
}
"cafile" => {
let value: String = value.get().unwrap();
let mut settings = self.settings.lock().unwrap();
settings.cafile = Some(value.into());
}
_ => unimplemented!(),
}
}
fn property(&self, _id: usize, pspec: &glib::ParamSpec) -> glib::Value {
match pspec.name() {
"address" => self.settings.lock().unwrap().address.to_value(),
"cafile" => {
let settings = self.settings.lock().unwrap();
let cafile = settings.cafile.as_ref();
cafile.and_then(|file| file.to_str()).to_value()
}
_ => unimplemented!(),
}
}
}
fn serialize_value(val: &gst::glib::Value) -> Option<serde_json::Value> {
match val.type_() {
Type::STRING => Some(val.get::<String>().unwrap().into()),
Type::BOOL => Some(val.get::<bool>().unwrap().into()),
Type::I32 => Some(val.get::<i32>().unwrap().into()),
Type::U32 => Some(val.get::<u32>().unwrap().into()),
Type::I_LONG | Type::I64 => Some(val.get::<i64>().unwrap().into()),
Type::U_LONG | Type::U64 => Some(val.get::<u64>().unwrap().into()),
Type::F32 => Some(val.get::<f32>().unwrap().into()),
Type::F64 => Some(val.get::<f64>().unwrap().into()),
_ => {
if let Ok(s) = val.get::<gst::Structure>() {
serde_json::to_value(
s.iter()
.filter_map(|(name, value)| {
serialize_value(value).map(|value| (name.to_string(), value))
})
.collect::<HashMap<String, serde_json::Value>>(),
)
.ok()
} else if let Ok(a) = val.get::<gst::Array>() {
serde_json::to_value(
a.iter()
.filter_map(|value| serialize_value(value))
.collect::<Vec<serde_json::Value>>(),
)
.ok()
} else if let Some((_klass, values)) = gst::glib::FlagsValue::from_value(val) {
Some(
values
.iter()
.map(|value| value.nick())
.collect::<Vec<&str>>()
.join("+")
.into(),
)
} else if let Ok(value) = val.serialize() {
Some(value.as_str().into())
} else {
gst::warning!(CAT, "Can't convert {} to json", val.type_().name());
None
}
}
}
}

62
net/webrtc/plugins/src/signaller/mod.rs Normal file
View file

@ -0,0 +1,62 @@
use crate::webrtcsink::{Signallable, WebRTCSink};
use gst::glib;
use gst::subclass::prelude::ObjectSubclassExt;
use std::error::Error;
mod imp;
glib::wrapper! {
pub struct Signaller(ObjectSubclass<imp::Signaller>);
}
unsafe impl Send for Signaller {}
unsafe impl Sync for Signaller {}
impl Signallable for Signaller {
fn start(&mut self, element: &WebRTCSink) -> Result<(), Box<dyn Error>> {
let signaller = imp::Signaller::from_instance(self);
signaller.start(element);
Ok(())
}
fn handle_sdp(
&mut self,
element: &WebRTCSink,
peer_id: &str,
sdp: &gst_webrtc::WebRTCSessionDescription,
) -> Result<(), Box<dyn Error>> {
let signaller = imp::Signaller::from_instance(self);
signaller.handle_sdp(element, peer_id, sdp);
Ok(())
}
fn handle_ice(
&mut self,
element: &WebRTCSink,
session_id: &str,
candidate: &str,
sdp_mline_index: Option<u32>,
sdp_mid: Option<String>,
) -> Result<(), Box<dyn Error>> {
let signaller = imp::Signaller::from_instance(self);
signaller.handle_ice(element, session_id, candidate, sdp_mline_index, sdp_mid);
Ok(())
}
fn stop(&mut self, element: &WebRTCSink) {
let signaller = imp::Signaller::from_instance(self);
signaller.stop(element);
}
fn session_ended(&mut self, element: &WebRTCSink, session_id: &str) {
let signaller = imp::Signaller::from_instance(self);
signaller.end_session(element, session_id);
}
}
impl Default for Signaller {
fn default() -> Self {
glib::Object::new::<Self>(&[])
}
}

420
net/webrtc/plugins/src/webrtcsink/homegrown_cc.rs Normal file
View file

@ -0,0 +1,420 @@
use gst::{
glib::{self, value::FromValue},
prelude::*,
};
use once_cell::sync::Lazy;
use super::imp::VideoEncoder;
static CAT: Lazy<gst::DebugCategory> = Lazy::new(|| {
gst::DebugCategory::new(
"webrtcsink-homegrowncc",
gst::DebugColorFlags::empty(),
Some("WebRTC sink"),
)
});
#[derive(Debug)]
enum IncreaseType {
/// Increase bitrate by value
Additive(f64),
/// Increase bitrate by factor
Multiplicative(f64),
}
#[derive(Debug, Clone, Copy)]
enum ControllerType {
// Running the "delay-based controller"
Delay,
// Running the "loss based controller"
Loss,
}
#[derive(Debug)]
enum CongestionControlOp {
/// Don't update target bitrate
Hold,
/// Decrease target bitrate
Decrease {
factor: f64,
#[allow(dead_code)]
reason: String, // for Debug
},
/// Increase target bitrate, either additively or multiplicatively
Increase(IncreaseType),
}
fn lookup_twcc_stats(stats: &gst::StructureRef) -> Option<gst::Structure> {
for (_, field_value) in stats {
if let Ok(s) = field_value.get::<gst::Structure>() {
if let Ok(type_) = s.get::<gst_webrtc::WebRTCStatsType>("type") {
if (type_ == gst_webrtc::WebRTCStatsType::Transport
|| type_ == gst_webrtc::WebRTCStatsType::CandidatePair)
&& s.has_field("gst-twcc-stats")
{
return Some(s.get::<gst::Structure>("gst-twcc-stats").unwrap());
}
}
}
}
None
}
pub struct CongestionController {
/// Note: The target bitrate applied is the min of
/// target_bitrate_on_delay and target_bitrate_on_loss
///
/// Bitrate target based on delay factor for all video streams.
/// Hasn't been tested with multiple video streams, but
/// current design is simply to divide bitrate equally.
pub target_bitrate_on_delay: i32,
/// Bitrate target based on loss for all video streams.
pub target_bitrate_on_loss: i32,
/// Exponential moving average, updated when bitrate is
/// decreased, discarded when increased again past last
/// congestion window. Smoothing factor hardcoded.
bitrate_ema: Option<f64>,
/// Exponentially weighted moving variance, recursively
/// updated along with bitrate_ema. sqrt'd to obtain standard
/// deviation, used to determine whether to increase bitrate
/// additively or multiplicatively
bitrate_emvar: f64,
/// Used in additive mode to track last control time, influences
/// calculation of added value according to gcc section 5.5
last_update_time: Option<std::time::Instant>,
/// For logging purposes
peer_id: String,
min_bitrate: u32,
max_bitrate: u32,
}
impl CongestionController {
pub fn new(peer_id: &str, min_bitrate: u32, max_bitrate: u32) -> Self {
Self {
target_bitrate_on_delay: 0,
target_bitrate_on_loss: 0,
bitrate_ema: None,
bitrate_emvar: 0.,
last_update_time: None,
peer_id: peer_id.to_string(),
min_bitrate,
max_bitrate,
}
}
fn update_delay(
&mut self,
element: &super::WebRTCSink,
twcc_stats: &gst::StructureRef,
rtt: f64,
) -> CongestionControlOp {
let target_bitrate = f64::min(
self.target_bitrate_on_delay as f64,
self.target_bitrate_on_loss as f64,
);
// Unwrap, all those fields must be there or there's been an API
// break, which qualifies as programming error
let bitrate_sent = twcc_stats.get::<u32>("bitrate-sent").unwrap();
let bitrate_recv = twcc_stats.get::<u32>("bitrate-recv").unwrap();
let delta_of_delta = twcc_stats.get::<i64>("avg-delta-of-delta").unwrap();
let sent_minus_received = bitrate_sent.saturating_sub(bitrate_recv);
let delay_factor = sent_minus_received as f64 / target_bitrate;
let last_update_time = self.last_update_time.replace(std::time::Instant::now());
gst::trace!(
CAT,
obj: element,
"consumer {}: considering stats {}",
self.peer_id,
twcc_stats
);
if delay_factor > 0.1 {
let (factor, reason) = if delay_factor < 0.64 {
(0.96, format!("low delay factor {}", delay_factor))
} else {
(
delay_factor.sqrt().sqrt().clamp(0.8, 0.96),
format!("High delay factor {}", delay_factor),
)
};
CongestionControlOp::Decrease { factor, reason }
} else if delta_of_delta > 1_000_000 {
CongestionControlOp::Decrease {
factor: 0.97,
reason: format!("High delta: {}", delta_of_delta),
}
} else {
CongestionControlOp::Increase(if let Some(ema) = self.bitrate_ema {
let bitrate_stdev = self.bitrate_emvar.sqrt();
gst::trace!(
CAT,
obj: element,
"consumer {}: Old bitrate: {}, ema: {}, stddev: {}",
self.peer_id,
target_bitrate,
ema,
bitrate_stdev,
);
// gcc section 5.5 advises 3 standard deviations, but experiments
// have shown this to be too low, probably related to the rest of
// homegrown algorithm not implementing gcc, revisit when implementing
// the rest of the RFC
if target_bitrate < ema - 7. * bitrate_stdev {
gst::trace!(
CAT,
obj: element,
"consumer {}: below last congestion window",
self.peer_id
);
/* Multiplicative increase */
IncreaseType::Multiplicative(1.03)
} else if target_bitrate > ema + 7. * bitrate_stdev {
gst::trace!(
CAT,
obj: element,
"consumer {}: above last congestion window",
self.peer_id
);
/* We have gone past our last estimated max bandwidth
* network situation may have changed, go back to
* multiplicative increase
*/
self.bitrate_ema.take();
IncreaseType::Multiplicative(1.03)
} else {
let rtt_ms = rtt * 1000.;
let response_time_ms = 100. + rtt_ms;
let time_since_last_update_ms = match last_update_time {
None => 0.,
Some(instant) => {
(self.last_update_time.unwrap() - instant).as_millis() as f64
}
};
// gcc section 5.5 advises 0.95 as the smoothing factor, but that
// seems intuitively much too low, granting disproportionate importance
// to the last measurement. 0.5 seems plenty enough, I don't have maths
// to back that up though :)
let alpha = 0.5 * f64::min(time_since_last_update_ms / response_time_ms, 1.0);
let bits_per_frame = target_bitrate / 30.;
let packets_per_frame = f64::ceil(bits_per_frame / (1200. * 8.));
let avg_packet_size_bits = bits_per_frame / packets_per_frame;
gst::trace!(
CAT,
obj: element,
"consumer {}: still in last congestion window",
self.peer_id,
);
/* Additive increase */
IncreaseType::Additive(f64::max(1000., alpha * avg_packet_size_bits))
}
} else {
/* Multiplicative increase */
gst::trace!(
CAT,
obj: element,
"consumer {}: outside congestion window",
self.peer_id
);
IncreaseType::Multiplicative(1.03)
})
}
}
fn clamp_bitrate(&mut self, bitrate: i32, n_encoders: i32, controller_type: ControllerType) {
match controller_type {
ControllerType::Loss => {
self.target_bitrate_on_loss = bitrate.clamp(
self.min_bitrate as i32 * n_encoders,
self.max_bitrate as i32 * n_encoders,
)
}
ControllerType::Delay => {
self.target_bitrate_on_delay = bitrate.clamp(
self.min_bitrate as i32 * n_encoders,
self.max_bitrate as i32 * n_encoders,
)
}
}
}
fn get_remote_inbound_stats(&self, stats: &gst::StructureRef) -> Vec<gst::Structure> {
let mut inbound_rtp_stats: Vec<gst::Structure> = Default::default();
for (_, field_value) in stats {
if let Ok(s) = field_value.get::<gst::Structure>() {
if let Ok(type_) = s.get::<gst_webrtc::WebRTCStatsType>("type") {
if type_ == gst_webrtc::WebRTCStatsType::RemoteInboundRtp {
inbound_rtp_stats.push(s);
}
}
}
}
inbound_rtp_stats
}
fn lookup_rtt(&self, stats: &gst::StructureRef) -> f64 {
let inbound_rtp_stats = self.get_remote_inbound_stats(stats);
let mut rtt = 0.;
let mut n_rtts = 0u64;
for inbound_stat in &inbound_rtp_stats {
if let Err(err) = (|| -> Result<(), gst::structure::GetError<<<f64 as FromValue>::Checker as glib::value::ValueTypeChecker>::Error>> {
rtt += inbound_stat.get::<f64>("round-trip-time")?;
n_rtts += 1;
Ok(())
})() {
gst::debug!(CAT, "{:?}", err);
}
}
rtt /= f64::max(1., n_rtts as f64);
gst::log!(CAT, "Round trip time: {}", rtt);
rtt
}
pub fn loss_control(
&mut self,
element: &super::WebRTCSink,
stats: &gst::StructureRef,
encoders: &mut Vec<VideoEncoder>,
) {
let loss_percentage = stats.get::<f64>("packet-loss-pct").unwrap();
self.apply_control_op(
element,
encoders,
if loss_percentage > 10. {
CongestionControlOp::Decrease {
factor: ((100. - (0.5 * loss_percentage)) / 100.).clamp(0.7, 0.98),
reason: format!("High loss: {}", loss_percentage),
}
} else if loss_percentage > 2. {
CongestionControlOp::Hold
} else {
CongestionControlOp::Increase(IncreaseType::Multiplicative(1.05))
},
ControllerType::Loss,
);
}
pub fn delay_control(
&mut self,
element: &super::WebRTCSink,
stats: &gst::StructureRef,
encoders: &mut Vec<VideoEncoder>,
) {
if let Some(twcc_stats) = lookup_twcc_stats(stats) {
let op = self.update_delay(element, &twcc_stats, self.lookup_rtt(stats));
self.apply_control_op(element, encoders, op, ControllerType::Delay);
}
}
fn apply_control_op(
&mut self,
element: &super::WebRTCSink,
encoders: &mut Vec<VideoEncoder>,
control_op: CongestionControlOp,
controller_type: ControllerType,
) {
gst::trace!(
CAT,
obj: element,
"consumer {}: applying congestion control operation {:?}",
self.peer_id,
control_op
);
let n_encoders = encoders.len() as i32;
let prev_bitrate = i32::min(self.target_bitrate_on_delay, self.target_bitrate_on_loss);
match &control_op {
CongestionControlOp::Hold => {}
CongestionControlOp::Increase(IncreaseType::Additive(value)) => {
self.clamp_bitrate(
self.target_bitrate_on_delay + *value as i32,
n_encoders,
controller_type,
);
}
CongestionControlOp::Increase(IncreaseType::Multiplicative(factor)) => {
self.clamp_bitrate(
(self.target_bitrate_on_delay as f64 * factor) as i32,
n_encoders,
controller_type,
);
}
CongestionControlOp::Decrease { factor, .. } => {
self.clamp_bitrate(
(self.target_bitrate_on_delay as f64 * factor) as i32,
n_encoders,
controller_type,
);
if let ControllerType::Delay = controller_type {
// Smoothing factor
let alpha = 0.75;
if let Some(ema) = self.bitrate_ema {
let sigma: f64 = (self.target_bitrate_on_delay as f64) - ema;
self.bitrate_ema = Some(ema + (alpha * sigma));
self.bitrate_emvar =
(1. - alpha) * (self.bitrate_emvar + alpha * sigma.powi(2));
} else {
self.bitrate_ema = Some(self.target_bitrate_on_delay as f64);
self.bitrate_emvar = 0.;
}
}
}
}
let target_bitrate =
i32::min(self.target_bitrate_on_delay, self.target_bitrate_on_loss).clamp(
self.min_bitrate as i32 * n_encoders,
self.max_bitrate as i32 * n_encoders,
) / n_encoders;
if target_bitrate != prev_bitrate {
gst::info!(
CAT,
"{:?} {} => {} | on delay {} - on loss {} | min {} - max {}",
control_op,
human_bytes::human_bytes(prev_bitrate),
human_bytes::human_bytes(target_bitrate),
human_bytes::human_bytes(self.target_bitrate_on_delay),
human_bytes::human_bytes(self.target_bitrate_on_loss),
human_bytes::human_bytes(self.min_bitrate),
human_bytes::human_bytes(self.max_bitrate),
);
}
let fec_ratio = {
if target_bitrate <= 2000000 || self.max_bitrate <= 2000000 {
0f64
} else {
(target_bitrate as f64 - 2000000f64) / (self.max_bitrate as f64 - 2000000f64)
}
};
let fec_percentage = (fec_ratio * 50f64) as u32;
for encoder in encoders.iter_mut() {
encoder.set_bitrate(element, target_bitrate);
encoder
.transceiver
.set_property("fec-percentage", fec_percentage);
}
}
}

2852
net/webrtc/plugins/src/webrtcsink/imp.rs Normal file
View file

File diff suppressed because it is too large Load diff

164
net/webrtc/plugins/src/webrtcsink/mod.rs Normal file
View file

@ -0,0 +1,164 @@
use gst::glib;
use gst::prelude::*;
use gst::subclass::prelude::*;
use std::error::Error;
mod homegrown_cc;
mod imp;
glib::wrapper! {
pub struct WebRTCSink(ObjectSubclass<imp::WebRTCSink>) @extends gst::Bin, gst::Element, gst::Object, @implements gst::ChildProxy, gst_video::Navigation;
}
unsafe impl Send for WebRTCSink {}
unsafe impl Sync for WebRTCSink {}
#[derive(thiserror::Error, Debug)]
pub enum WebRTCSinkError {
#[error("no session with id")]
NoSessionWithId(String),
#[error("consumer refused media")]
ConsumerRefusedMedia { session_id: String, media_idx: u32 },
#[error("consumer did not provide valid payload for media")]
ConsumerNoValidPayload { session_id: String, media_idx: u32 },
#[error("SDP mline index is currently mandatory")]
MandatorySdpMlineIndex,
#[error("duplicate session id")]
DuplicateSessionId(String),
#[error("error setting up consumer pipeline")]
SessionPipelineError {
session_id: String,
peer_id: String,
details: String,
},
}
pub trait Signallable: Sync + Send + 'static {
fn start(&mut self, element: &WebRTCSink) -> Result<(), Box<dyn Error>>;
fn handle_sdp(
&mut self,
element: &WebRTCSink,
session_id: &str,
sdp: &gst_webrtc::WebRTCSessionDescription,
) -> Result<(), Box<dyn Error>>;
/// sdp_mid is exposed for future proofing, see
/// https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1174,
/// at the moment sdp_m_line_index will always be Some and sdp_mid will always
/// be None
fn handle_ice(
&mut self,
element: &WebRTCSink,
session_id: &str,
candidate: &str,
sdp_m_line_index: Option<u32>,
sdp_mid: Option<String>,
) -> Result<(), Box<dyn Error>>;
fn session_ended(&mut self, element: &WebRTCSink, session_id: &str);
fn stop(&mut self, element: &WebRTCSink);
}
/// When providing a signaller, we expect it to both be a GObject
/// and be Signallable. This is arguably a bit strange, but exposing
/// a GInterface from rust is at the moment a bit awkward, so I went
/// for a rust interface for now. The reason the signaller needs to be
/// a GObject is to make its properties available through the GstChildProxy
/// interface.
pub trait SignallableObject: AsRef<glib::Object> + Signallable {}
impl<T: AsRef<glib::Object> + Signallable> SignallableObject for T {}
impl Default for WebRTCSink {
fn default() -> Self {
glib::Object::new::<Self>(&[])
}
}
impl WebRTCSink {
pub fn with_signaller(signaller: Box<dyn SignallableObject>) -> Self {
let ret: WebRTCSink = glib::Object::new(&[]);
let ws = imp::WebRTCSink::from_instance(&ret);
ws.set_signaller(signaller).unwrap();
ret
}
pub fn handle_sdp(
&self,
session_id: &str,
sdp: &gst_webrtc::WebRTCSessionDescription,
) -> Result<(), WebRTCSinkError> {
let ws = imp::WebRTCSink::from_instance(self);
ws.handle_sdp(self, session_id, sdp)
}
/// sdp_mid is exposed for future proofing, see
/// https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1174,
/// at the moment sdp_m_line_index must be Some
pub fn handle_ice(
&self,
session_id: &str,
sdp_m_line_index: Option<u32>,
sdp_mid: Option<String>,
candidate: &str,
) -> Result<(), WebRTCSinkError> {
let ws = imp::WebRTCSink::from_instance(self);
ws.handle_ice(self, session_id, sdp_m_line_index, sdp_mid, candidate)
}
pub fn handle_signalling_error(&self, error: Box<dyn Error + Send + Sync>) {
let ws = imp::WebRTCSink::from_instance(self);
ws.handle_signalling_error(self, anyhow::anyhow!(error));
}
pub fn start_session(&self, session_id: &str, peer_id: &str) -> Result<(), WebRTCSinkError> {
let ws = imp::WebRTCSink::from_instance(self);
ws.start_session(self, session_id, peer_id)
}
pub fn end_session(&self, session_id: &str) -> Result<(), WebRTCSinkError> {
let ws = imp::WebRTCSink::from_instance(self);
ws.remove_session(self, session_id, false)
}
}
#[derive(Debug, Eq, PartialEq, Ord, PartialOrd, Hash, Clone, Copy, glib::Enum)]
#[repr(u32)]
#[enum_type(name = "GstWebRTCSinkCongestionControl")]
pub enum WebRTCSinkCongestionControl {
#[enum_value(name = "Disabled: no congestion control is applied", nick = "disabled")]
Disabled,
#[enum_value(name = "Homegrown: simple sender-side heuristic", nick = "homegrown")]
Homegrown,
#[enum_value(name = "Google Congestion Control algorithm", nick = "gcc")]
GoogleCongestionControl,
}
#[glib::flags(name = "GstWebRTCSinkMitigationMode")]
enum WebRTCSinkMitigationMode {
#[flags_value(name = "No mitigation applied", nick = "none")]
NONE = 0b00000000,
#[flags_value(name = "Lowered resolution", nick = "downscaled")]
DOWNSCALED = 0b00000001,
#[flags_value(name = "Lowered framerate", nick = "downsampled")]
DOWNSAMPLED = 0b00000010,
}
pub fn register(plugin: &gst::Plugin) -> Result<(), glib::BoolError> {
gst::Element::register(
Some(plugin),
"webrtcsink",
gst::Rank::None,
WebRTCSink::static_type(),
)
}

12
net/webrtc/protocol/Cargo.toml Normal file
View file

@ -0,0 +1,12 @@
[package]
name="webrtcsink-protocol"
version = "0.1.0"
edition = "2018"
authors = ["Mathieu Duponchelle <mathieu@centricular.com>"]
license = "MPL-2.0"
description = "GStreamer WebRTC sink default protocol"
repository = "https://github.com/centricular/webrtcsink/"
[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"

144
net/webrtc/protocol/src/lib.rs Normal file
View file

@ -0,0 +1,144 @@
/// The default protocol used by the signalling server
use serde::{Deserialize, Serialize};
#[derive(Serialize, Deserialize, Debug, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct Peer {
pub id: String,
#[serde(default)]
pub meta: Option<serde_json::Value>,
}
#[derive(Serialize, Deserialize, Debug, PartialEq)]
#[serde(tag = "type")]
#[serde(rename_all = "camelCase")]
/// Messages sent from the server to peers
pub enum OutgoingMessage {
/// Welcoming message, sets the Peer ID linked to a new connection
Welcome { peer_id: String },
/// Notifies listeners that a peer status has changed
PeerStatusChanged(PeerStatus),
/// Instructs a peer to generate an offer and inform about the session ID
#[serde(rename_all = "camelCase")]
StartSession { peer_id: String, session_id: String },
/// Let consumer know that the requested session is starting with the specified identifier
#[serde(rename_all = "camelCase")]
SessionStarted { peer_id: String, session_id: String },
/// Signals that the session the peer was in was ended
#[serde(rename_all = "camelCase")]
EndSession(EndSessionMessage),
/// Messages directly forwarded from one peer to another
Peer(PeerMessage),
/// Provides the current list of consumer peers
List { producers: Vec<Peer> },
/// Notifies that an error occurred with the peer's current session
Error { details: String },
}
#[derive(Serialize, Deserialize, Debug, PartialEq, Eq, Clone)]
#[serde(rename_all = "camelCase")]
/// Register with a peer type
pub enum PeerRole {
/// Register as a producer
#[serde(rename_all = "camelCase")]
Producer,
/// Register as a listener
#[serde(rename_all = "camelCase")]
Listener,
}
#[derive(Serialize, Deserialize, Debug, PartialEq, Eq, Default, Clone)]
#[serde(rename_all = "camelCase")]
pub struct PeerStatus {
pub roles: Vec<PeerRole>,
pub meta: Option<serde_json::Value>,
#[serde(skip_serializing_if = "Option::is_none")]
#[serde(default)]
pub peer_id: Option<String>,
}
impl PeerStatus {
pub fn producing(&self) -> bool {
self.roles.iter().any(|t| matches!(t, PeerRole::Producer))
}
pub fn listening(&self) -> bool {
self.roles.iter().any(|t| matches!(t, PeerRole::Listener))
}
}
#[derive(Serialize, Deserialize, Debug)]
#[serde(rename_all = "camelCase")]
/// Ask the server to start a session with a producer peer
pub struct StartSessionMessage {
/// Identifies the peer
pub peer_id: String,
}
#[derive(Serialize, Deserialize, Debug, PartialEq, Clone)]
#[serde(tag = "type")]
#[serde(rename_all = "camelCase")]
/// Conveys a SDP
pub enum SdpMessage {
/// Conveys an offer
Offer {
/// The SDP
sdp: String,
},
/// Conveys an answer
Answer {
/// The SDP
sdp: String,
},
}
#[derive(Serialize, Deserialize, Debug, PartialEq, Clone)]
#[serde(rename_all = "camelCase")]
/// Contents of the peer message
pub enum PeerMessageInner {
/// Conveys an ICE candidate
#[serde(rename_all = "camelCase")]
Ice {
/// The candidate string
candidate: String,
/// The mline index the candidate applies to
sdp_m_line_index: u32,
},
Sdp(SdpMessage),
}
#[derive(Serialize, Deserialize, Debug, PartialEq)]
#[serde(rename_all = "camelCase")]
/// Messages directly forwarded from one peer to another
pub struct PeerMessage {
pub session_id: String,
#[serde(flatten)]
pub peer_message: PeerMessageInner,
}
#[derive(Serialize, Deserialize, Debug, PartialEq, Eq)]
#[serde(rename_all = "camelCase")]
/// End a session
pub struct EndSessionMessage {
/// The identifier of the session to end
pub session_id: String,
}
#[derive(Serialize, Deserialize, Debug)]
#[serde(tag = "type")]
#[serde(rename_all = "camelCase")]
/// Messages received by the server from peers
pub enum IncomingMessage {
/// Internal message to let know about new peers
NewPeer,
/// Set current peer status
SetPeerStatus(PeerStatus),
/// Start a session with a producer peer
StartSession(StartSessionMessage),
/// End an existing session
EndSession(EndSessionMessage),
/// Send a message to a peer the sender is currently in session with
Peer(PeerMessage),
/// Retrieve the current list of producers
List,
}

26
net/webrtc/signalling/Cargo.toml Normal file
View file

@ -0,0 +1,26 @@
[package]
name="webrtcsink-signalling"
version = "0.1.0"
edition = "2018"
authors = ["Mathieu Duponchelle <mathieu@centricular.com>"]
license = "MPL-2.0"
description = "GStreamer WebRTC sink signalling server"
repository = "https://github.com/centricular/webrtcsink/"
[dependencies]
anyhow = "1"
async-std = { version = "1", features = ["unstable", "attributes"] }
async-native-tls = "0.4"
async-tungstenite = { version = "0.17", features = ["async-std-runtime", "async-native-tls"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
clap = { version = "4", features = ["derive"] }
tracing = { version = "0.1", features = ["log"] }
tracing-subscriber = { version = "0.3", features = ["registry", "env-filter"] }
tracing-log = "0.1"
futures = "0.3"
uuid = { version = "1", features = ["v4"] }
thiserror = "1"
test-log = { version = "0.2", features = ["trace"], default-features = false }
pin-project-lite = "0.2"
webrtcsink-protocol = { version = "0.1", path="../protocol" }

101
net/webrtc/signalling/src/bin/server.rs Normal file
View file

@ -0,0 +1,101 @@
use async_std::task;
use clap::Parser;
use tracing_subscriber::prelude::*;
use webrtcsink_signalling::handlers::Handler;
use webrtcsink_signalling::server::Server;
use anyhow::Error;
use async_native_tls::TlsAcceptor;
use async_std::fs::File as AsyncFile;
use async_std::net::TcpListener;
use tracing::{info, warn};
#[derive(Parser, Debug)]
#[clap(about, version, author)]
/// Program arguments
struct Args {
/// Address to listen on
#[clap(short, long, default_value = "0.0.0.0")]
host: String,
/// Port to listen on
#[clap(short, long, default_value_t = 8443)]
port: u16,
/// TLS certificate to use
#[clap(short, long)]
cert: Option<String>,
/// password to TLS certificate
#[clap(long)]
cert_password: Option<String>,
}
fn initialize_logging(envvar_name: &str) -> Result<(), Error> {
tracing_log::LogTracer::init()?;
let env_filter = tracing_subscriber::EnvFilter::try_from_env(envvar_name)
.unwrap_or_else(|_| tracing_subscriber::EnvFilter::new("info"));
let fmt_layer = tracing_subscriber::fmt::layer()
.with_thread_ids(true)
.with_target(true)
.with_span_events(
tracing_subscriber::fmt::format::FmtSpan::NEW
| tracing_subscriber::fmt::format::FmtSpan::CLOSE,
);
let subscriber = tracing_subscriber::Registry::default()
.with(env_filter)
.with(fmt_layer);
tracing::subscriber::set_global_default(subscriber)?;
Ok(())
}
fn main() -> Result<(), Error> {
let args = Args::parse();
let server = Server::spawn(|stream| Handler::new(stream));
initialize_logging("WEBRTCSINK_SIGNALLING_SERVER_LOG")?;
task::block_on(async move {
let addr = format!("{}:{}", args.host, args.port);
// Create the event loop and TCP listener we'll accept connections on.
let listener = TcpListener::bind(&addr).await?;
let acceptor = match args.cert {
Some(cert) => {
let key = AsyncFile::open(cert).await?;
Some(TlsAcceptor::new(key, args.cert_password.as_deref().unwrap_or("")).await?)
}
None => None,
};
info!("Listening on: {}", addr);
while let Ok((stream, _)) = listener.accept().await {
let mut server_clone = server.clone();
let address = match stream.peer_addr() {
Ok(address) => address,
Err(err) => {
warn!("Connected peer with no address: {}", err);
continue;
}
};
info!("Accepting connection from {}", address);
if let Some(ref acceptor) = acceptor {
let stream = match acceptor.accept(stream).await {
Ok(stream) => stream,
Err(err) => {
warn!("Failed to accept TLS connection from {}: {}", address, err);
continue;
}
};
task::spawn(async move { server_clone.accept_async(stream).await });
} else {
task::spawn(async move { server_clone.accept_async(stream).await });
}
}
Ok(())
})
}

1421
net/webrtc/signalling/src/handlers/mod.rs Normal file
View file

File diff suppressed because it is too large Load diff

2
net/webrtc/signalling/src/lib.rs Normal file
View file

@ -0,0 +1,2 @@
pub mod handlers;
pub mod server;

218
net/webrtc/signalling/src/server/mod.rs Normal file
View file

@ -0,0 +1,218 @@
use anyhow::Error;
use async_std::task;
use async_tungstenite::tungstenite::Message as WsMessage;
use futures::channel::mpsc;
use futures::prelude::*;
use futures::{AsyncRead, AsyncWrite};
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::pin::Pin;
use std::sync::{Arc, Mutex};
use tracing::{info, instrument, trace, warn};
struct Peer {
receive_task_handle: task::JoinHandle<()>,
send_task_handle: task::JoinHandle<Result<(), Error>>,
sender: mpsc::Sender<String>,
}
struct State {
tx: Option<mpsc::Sender<(String, Option<String>)>>,
peers: HashMap<String, Peer>,
}
#[derive(Clone)]
pub struct Server {
state: Arc<Mutex<State>>,
}
#[derive(thiserror::Error, Debug)]
pub enum ServerError {
#[error("error during handshake {0}")]
Handshake(#[from] async_tungstenite::tungstenite::Error),
}
impl Server {
#[instrument(level = "debug", skip(factory))]
pub fn spawn<
I: for<'a> Deserialize<'a>,
O: Serialize + std::fmt::Debug,
Factory: FnOnce(Pin<Box<dyn Stream<Item = (String, Option<I>)> + Send>>) -> St,
St: Stream<Item = (String, O)>,
>(
factory: Factory,
) -> Self
where
O: Serialize + std::fmt::Debug,
St: Send + Unpin + 'static,
{
let (tx, rx) = mpsc::channel::<(String, Option<String>)>(1000);
let mut handler = factory(Box::pin(rx.filter_map(|(peer_id, msg)| async move {
if let Some(msg) = msg {
match serde_json::from_str::<I>(&msg) {
Ok(msg) => Some((peer_id, Some(msg))),
Err(err) => {
warn!("Failed to parse incoming message: {} ({})", err, msg);
None
}
}
} else {
Some((peer_id, None))
}
})));
let state = Arc::new(Mutex::new(State {
tx: Some(tx),
peers: HashMap::new(),
}));
let state_clone = state.clone();
let _ = task::spawn(async move {
while let Some((peer_id, msg)) = handler.next().await {
match serde_json::to_string(&msg) {
Ok(msg) => {
if let Some(peer) = state_clone.lock().unwrap().peers.get_mut(&peer_id) {
let mut sender = peer.sender.clone();
task::spawn(async move {
let _ = sender.send(msg).await;
});
}
}
Err(err) => {
warn!("Failed to serialize outgoing message: {}", err);
}
}
}
});
Self { state }
}
#[instrument(level = "debug", skip(state))]
fn remove_peer(state: Arc<Mutex<State>>, peer_id: &str) {
if let Some(mut peer) = state.lock().unwrap().peers.remove(peer_id) {
let peer_id = peer_id.to_string();
task::spawn(async move {
peer.sender.close_channel();
if let Err(err) = peer.send_task_handle.await {
trace!(peer_id = %peer_id, "Error while joining send task: {}", err);
}
peer.receive_task_handle.await;
});
}
}
#[instrument(level = "debug", skip(self, stream))]
pub async fn accept_async<S: 'static>(&mut self, stream: S) -> Result<String, ServerError>
where
S: AsyncRead + AsyncWrite + Unpin + Send,
{
let ws = match async_tungstenite::accept_async(stream).await {
Ok(ws) => ws,
Err(err) => {
warn!("Error during the websocket handshake: {}", err);
return Err(ServerError::Handshake(err));
}
};
let this_id = uuid::Uuid::new_v4().to_string();
info!(this_id = %this_id, "New WebSocket connection");
// 1000 is completely arbitrary, we simply don't want infinite piling
// up of messages as with unbounded
let (websocket_sender, mut websocket_receiver) = mpsc::channel::<String>(1000);
let this_id_clone = this_id.clone();
let (mut ws_sink, mut ws_stream) = ws.split();
let send_task_handle = task::spawn(async move {
loop {
match async_std::future::timeout(
std::time::Duration::from_secs(30),
websocket_receiver.next(),
)
.await
{
Ok(Some(msg)) => {
trace!(this_id = %this_id_clone, "sending {}", msg);
ws_sink.send(WsMessage::Text(msg)).await?;
}
Ok(None) => {
break;
}
Err(_) => {
trace!(this_id = %this_id_clone, "timeout, sending ping");
ws_sink.send(WsMessage::Ping(vec![])).await?;
}
}
}
ws_sink.send(WsMessage::Close(None)).await?;
ws_sink.close().await?;
Ok::<(), Error>(())
});
let mut tx = self.state.lock().unwrap().tx.clone();
let this_id_clone = this_id.clone();
let state_clone = self.state.clone();
let receive_task_handle = task::spawn(async move {
if let Some(tx) = tx.as_mut() {
if let Err(err) = tx
.send((
this_id_clone.clone(),
Some(
serde_json::json!({
"type": "newPeer",
})
.to_string(),
),
))
.await
{
warn!(this = %this_id_clone, "Error handling message: {:?}", err);
}
}
while let Some(msg) = ws_stream.next().await {
info!("Received message {msg:?}");
match msg {
Ok(WsMessage::Text(msg)) => {
if let Some(tx) = tx.as_mut() {
if let Err(err) = tx.send((this_id_clone.clone(), Some(msg))).await {
warn!(this = %this_id_clone, "Error handling message: {:?}", err);
}
}
}
Ok(WsMessage::Close(reason)) => {
info!(this_id = %this_id_clone, "connection closed: {:?}", reason);
break;
}
Ok(WsMessage::Pong(_)) => {
continue;
}
Ok(_) => warn!(this_id = %this_id_clone, "Unsupported message type"),
Err(err) => {
warn!(this_id = %this_id_clone, "recv error: {}", err);
break;
}
}
}
if let Some(tx) = tx.as_mut() {
let _ = tx.send((this_id_clone.clone(), None)).await;
}
Self::remove_peer(state_clone, &this_id_clone);
});
self.state.lock().unwrap().peers.insert(
this_id.clone(),
Peer {
receive_task_handle,
send_task_handle,
sender: websocket_sender,
},
);
Ok(this_id)
}
}

38
net/webrtc/www/index.html Normal file
View file

@ -0,0 +1,38 @@
<!DOCTYPE html>
<!--
vim: set sts=2 sw=2 et :
Demo Javascript app for negotiating and streaming a sendrecv webrtc stream
with a GStreamer app. Runs only in passive mode, i.e., responds to offers
with answers, exchanges ICE candidates, and streams.
Author: Nirbheek Chauhan <nirbheek@centricular.com>
-->
<html>
<head>
<style>
.error { color: red; }
</style>
<link rel="stylesheet" type="text/css" href="theme.css">
<script src="https://webrtc.github.io/adapter/adapter-latest.js"></script>
<script src="keyboard.js"></script>
<script src="input.js"></script>
<script src="webrtc.js"></script>
<script>window.onload = setup;</script>
</head>
<body>
<div class="holygrail-body">
<div class="content">
<div id="sessions">
</div>
<div id="image-holder">
<img id="image"></img>
</div>
</div>
<ul class="nav" id="camera-list">
</ul>
</div>
</body>
</html>

482
net/webrtc/www/input.js Normal file
View file

@ -0,0 +1,482 @@
/**
* Copyright 2019 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/*global GamepadManager*/
/*eslint no-unused-vars: ["error", { "vars": "local" }]*/
class Input {
/**
* Input handling for WebRTC web app
*
* @constructor
* @param {Element} [element]
* Video element to attach events to
* @param {function} [send]
* Function used to send input events to server.
*/
constructor(element, send) {
/**
* @type {Element}
*/
this.element = element;
/**
* @type {function}
*/
this.send = send;
/**
* @type {boolean}
*/
this.mouseRelative = false;
/**
* @type {Object}
*/
this.m = null;
/**
* @type {Keyboard}
*/
this.keyboard = null;
/**
* @type {GamepadManager}
*/
this.gamepadManager = null;
/**
* @type {Integer}
*/
this.x = 0;
/**
* @type {Integer}
*/
this.y = 0;
/**
* @type {Integer}
*/
this.lastTouch = 0;
/**
* @type {function}
*/
this.ongamepadconnected = null;
/**
* @type {function}
*/
this.ongamepaddisconneceted = null;
/**
* List of attached listeners, record keeping used to detach all.
* @type {Array}
*/
this.listeners = [];
/**
* @type {function}
*/
this.onresizeend = null;
// internal variables used by resize start/end functions.
this._rtime = null;
this._rtimeout = false;
this._rdelta = 200;
}
/**
* Handles mouse button and motion events and sends them to WebRTC app.
* @param {MouseEvent} event
*/
_mouseButtonMovement(event) {
const down = (event.type === 'mousedown' ? 1 : 0);
var data = {};
if (event.type === 'mousemove' && !this.m) return;
if (!document.pointerLockElement) {
if (this.mouseRelative)
event.target.requestPointerLock();
}
// Hotkey to enable pointer lock, CTRL-SHIFT-LeftButton
if (down && event.button === 0 && event.ctrlKey && event.shiftKey) {
event.target.requestPointerLock();
return;
}
if (document.pointerLockElement) {
// FIXME - mark as relative!
console.warn("FIXME: Make event relative!")
this.x = event.movementX;
this.y = event.movementY;
} else if (event.type === 'mousemove') {
this.x = this._clientToServerX(event.clientX);
this.y = this._clientToServerY(event.clientY);
data["event"] = "MouseMove"
}
if (event.type === 'mousedown') {
data["event"] = "MouseButtonPress";
} else if (event.type === 'mouseup') {
data["event"] = "MouseButtonRelease";
}
if (event.type === 'mousedown' || event.type === 'mouseup') {
data["button"] = event.button + 1;
}
data["x"] = this.x;
data["y"] = this.y;
data["modifier_state"] = this._modifierState(event);
this.send(data);
}
/**
* Handles touch events and sends them to WebRTC app.
* @param {TouchEvent} event
*/
_touch(event) {
var mod_state = this._modifierState(event);
// Use TouchUp for cancelled touch points
if (event.type === 'touchcancel') {
let data = {};
data["event"] = "TouchUp";
data["identifier"] = event.changedTouches[0].identifier;
data["x"] = this._clientToServerX(event.changedTouches[0].clientX);
data["y"] = this._clientToServerY(event.changedTouches[0].clientY);
data["modifier_state"] = mod_state;
this.send(data);
return;
}
if (event.type === 'touchstart') {
var event_name = "TouchDown";
} else if (event.type === 'touchmove') {
var event_name = "TouchMotion";
} else if (event.type === 'touchend') {
var event_name = "TouchUp";
}
for (let touch of event.changedTouches) {
let data = {};
data["event"] = event_name;
data["identifier"] = touch.identifier;
data["x"] = this._clientToServerX(touch.clientX);
data["y"] = this._clientToServerY(touch.clientY);
data["modifier_state"] = mod_state;
if (event.type !== 'touchend') {
if ('force' in touch) {
data["pressure"] = touch.force;
} else {
data["pressure"] = NaN;
}
}
this.send(data);
}
if (event.timeStamp > this.lastTouch) {
let data = {};
data["event"] = "TouchFrame";
data["modifier_state"] = mod_state;
this.send(data);
this.lastTouch = event.timeStamp;
}
event.preventDefault();
}
/**
* Handles mouse wheel events and sends them to WebRTC app.
* @param {MouseEvent} event
*/
_wheel(event) {
let data = {
"event": "MouseScroll",
"x": this.x,
"y": this.y,
"delta_x": -event.deltaX,
"delta_y": -event.deltaY,
"modifier_state": this._modifierState(event),
};
this.send(data);
event.preventDefault();
}
/**
* Captures mouse context menu (right-click) event and prevents event propagation.
* @param {MouseEvent} event
*/
_contextMenu(event) {
event.preventDefault();
}
/**
* Sends WebRTC app command to hide the remote pointer when exiting pointer lock.
*/
_exitPointerLock() {
document.exitPointerLock();
}
/**
* constructs the string representation for the active modifiers on the event
*/
_modifierState(event) {
let masks = []
if (event.altKey) masks.push("alt-mask");
if (event.ctrlKey) masks.push("control-mask");
if (event.metaKey) masks.push("meta-mask");
if (event.shiftKey) masks.push("shift-mask");
return masks.join('+')
}
/**
* Captures display and video dimensions required for computing mouse pointer position.
* This should be fired whenever the window size changes.
*/
_windowMath() {
const windowW = this.element.offsetWidth;
const windowH = this.element.offsetHeight;
const frameW = this.element.videoWidth;
const frameH = this.element.videoHeight;
const multi = Math.min(windowW / frameW, windowH / frameH);
const vpWidth = frameW * multi;
const vpHeight = (frameH * multi);
var elem = this.element;
var offsetLeft = 0;
var offsetTop = 0;
do {
if (!isNaN(elem.offsetLeft)) {
offsetLeft += elem.offsetLeft;
}
if (!isNaN(elem.offsetTop)) {
offsetTop += elem.offsetTop;
}
} while (elem = elem.offsetParent);
this.m = {
mouseMultiX: frameW / vpWidth,
mouseMultiY: frameH / vpHeight,
mouseOffsetX: Math.max((windowW - vpWidth) / 2.0, 0),
mouseOffsetY: Math.max((windowH - vpHeight) / 2.0, 0),
offsetLeft: offsetLeft,
offsetTop: offsetTop,
scrollX: window.scrollX,
scrollY: window.scrollY,
frameW,
frameH,
};
}
/**
* Translates pointer position X based on current window math.
* @param {Integer} clientX
*/
_clientToServerX(clientX) {
var serverX = Math.round((clientX - this.m.mouseOffsetX - this.m.offsetLeft + this.m.scrollX) * this.m.mouseMultiX);
if (serverX === this.m.frameW - 1) serverX = this.m.frameW;
if (serverX > this.m.frameW) serverX = this.m.frameW;
if (serverX < 0) serverX = 0;
return serverX;
}
/**
* Translates pointer position Y based on current window math.
* @param {Integer} clientY
*/
_clientToServerY(clientY) {
let serverY = Math.round((clientY - this.m.mouseOffsetY - this.m.offsetTop + this.m.scrollY) * this.m.mouseMultiY);
if (serverY === this.m.frameH - 1) serverY = this.m.frameH;
if (serverY > this.m.frameH) serverY = this.m.frameH;
if (serverY < 0) serverY = 0;
return serverY;
}
/**
* When fullscreen is entered, request keyboard and pointer lock.
*/
_onFullscreenChange() {
if (document.fullscreenElement !== null) {
// Enter fullscreen
this.requestKeyboardLock();
this.element.requestPointerLock();
}
// Reset local keyboard. When holding to exit full-screen the escape key can get stuck.
this.keyboard.reset();
// Reset stuck keys on server side.
// FIXME: How to implement resetting keyboard with the GstNavigation interface
// this.send("kr");
}
/**
* Called when window is being resized, used to detect when resize ends so new resolution can be sent.
*/
_resizeStart() {
this._rtime = new Date();
if (this._rtimeout === false) {
this._rtimeout = true;
setTimeout(() => { this._resizeEnd() }, this._rdelta);
}
}
/**
* Called in setTimeout loop to detect if window is done being resized.
*/
_resizeEnd() {
if (new Date() - this._rtime < this._rdelta) {
setTimeout(() => { this._resizeEnd() }, this._rdelta);
} else {
this._rtimeout = false;
if (this.onresizeend !== null) {
this.onresizeend();
}
}
}
/**
* Attaches input event handles to docuemnt, window and element.
*/
attach() {
this.listeners.push(addListener(this.element, 'resize', this._windowMath, this));
this.listeners.push(addListener(this.element, 'wheel', this._wheel, this));
this.listeners.push(addListener(this.element, 'contextmenu', this._contextMenu, this));
this.listeners.push(addListener(this.element.parentElement, 'fullscreenchange', this._onFullscreenChange, this));
this.listeners.push(addListener(window, 'resize', this._windowMath, this));
this.listeners.push(addListener(window, 'resize', this._resizeStart, this));
if ('ontouchstart' in window) {
console.warning("FIXME: Enabling mouse pointer display for touch devices.");
} else {
this.listeners.push(addListener(this.element, 'mousemove', this._mouseButtonMovement, this));
this.listeners.push(addListener(this.element, 'mousedown', this._mouseButtonMovement, this));
this.listeners.push(addListener(this.element, 'mouseup', this._mouseButtonMovement, this));
}
this.listeners.push(addListener(this.element, 'touchstart', this._touch, this));
this.listeners.push(addListener(this.element, 'touchend', this._touch, this));
this.listeners.push(addListener(this.element, 'touchmove', this._touch, this));
this.listeners.push(addListener(this.element, 'touchcancel', this._touch, this));
// Adjust for scroll offset
this.listeners.push(addListener(window, 'scroll', () => {
this.m.scrollX = window.scrollX;
this.m.scrollY = window.scrollY;
}, this));
// Using guacamole keyboard because it has the keysym translations.
this.keyboard = new Keyboard(this.element);
this.keyboard.onkeydown = (keysym, state) => {
this.send({"event": "KeyPress", "key": keysym, "modifier_state": state});
};
this.keyboard.onkeyup = (keysym, state) => {
this.send({"event": "KeyRelease", "key": keysym, "modifier_state": state});
};
this._windowMath();
}
detach() {
removeListeners(this.listeners);
this._exitPointerLock();
if (this.keyboard) {
this.keyboard.onkeydown = null;
this.keyboard.onkeyup = null;
this.keyboard.reset();
delete this.keyboard;
// FIXME: How to implement resetting keyboard with the GstNavigation interface
// this.send("kr");
}
}
/**
* Request keyboard lock, must be in fullscreen mode to work.
*/
requestKeyboardLock() {
// event codes: https://www.w3.org/TR/uievents-code/#key-alphanumeric-writing-system
const keys = [
"AltLeft",
"AltRight",
"Tab",
"Escape",
"ContextMenu",
"MetaLeft",
"MetaRight"
];
console.log("requesting keyboard lock");
navigator.keyboard.lock(keys).then(
() => {
console.log("keyboard lock success");
}
).catch(
(e) => {
console.log("keyboard lock failed: ", e);
}
)
}
getWindowResolution() {
return [
parseInt(this.element.offsetWidth * window.devicePixelRatio),
parseInt(this.element.offsetHeight * window.devicePixelRatio)
];
}
}
/**
* Helper function to keep track of attached event listeners.
* @param {Object} obj
* @param {string} name
* @param {function} func
* @param {Object} ctx
*/
function addListener(obj, name, func, ctx) {
const newFunc = ctx ? func.bind(ctx) : func;
obj.addEventListener(name, newFunc);
return [obj, name, newFunc];
}
/**
* Helper function to remove all attached event listeners.
* @param {Array} listeners
*/
function removeListeners(listeners) {
for (const listener of listeners)
listener[0].removeEventListener(listener[1], listener[2]);
}

3302
net/webrtc/www/keyboard.js Normal file
View file

File diff suppressed because it is too large Load diff

141
net/webrtc/www/theme.css Normal file
View file

@ -0,0 +1,141 @@
/* Reset CSS from Eric Meyer */
html, body, div, span, applet, object, iframe,
h1, h2, h3, h4, h5, h6, p, blockquote, pre,
a, abbr, acronym, address, big, cite, code,
del, dfn, em, img, ins, kbd, q, s, samp,
small, strike, strong, sub, sup, tt, var,
b, u, i, center,
dl, dt, dd, ol, ul, li,
fieldset, form, label, legend,
table, caption, tbody, tfoot, thead, tr, th, td,
article, aside, canvas, details, embed,
figure, figcaption, footer, header, hgroup,
menu, nav, output, ruby, section, summary,
time, mark, audio, video {
margin: 0;
padding: 0;
border: 0;
font-size: 100%;
font: inherit;
vertical-align: baseline;
}
/* HTML5 display-role reset for older browsers */
article, aside, details, figcaption, figure,
footer, header, hgroup, menu, nav, section {
display: block;
}
body {
line-height: 1;
}
ol, ul {
list-style: none;
}
blockquote, q {
quotes: none;
}
blockquote:before, blockquote:after,
q:before, q:after {
content: '';
content: none;
}
table {
border-collapse: collapse;
border-spacing: 0;
}
/* Our style */
body{
display: flex;
flex-direction: column;
min-height: 100vh;
background-color: #222;
color: white;
}
.holygrail-body {
flex: 1 0 auto;
display: flex;
}
.holygrail-body .content {
width: 100%;
}
#sessions {
display: flex;
flex-direction: row;
flex-wrap: wrap;
justify-content: space-around;
}
.holygrail-body .nav {
width: 220px;
list-style: none;
text-align: left;
order: -1;
background-color: #333;
margin: 0;
}
@media (max-width: 700px) {
.holygrail-body {
flex-direction: column;
}
.holygrail-body .nav {
width: 100%;
}
}
.session p span {
float: right;
}
.session p {
padding-top: 5px;
padding-bottom: 5px;
}
.stream {
background-color: black;
width: 480px;
}
#camera-list {
text-align: center;
}
.button {
border: none;
padding: 8px;
text-align: center;
text-decoration: none;
display: inline-block;
font-size: 16px;
-webkit-transition-duration: 0.4s; /* Safari */
transition-duration: 0.4s;
cursor: pointer;
margin: 5px auto;
width: 90%;
}
.button1 {
background-color: #222;
color: white;
border: 2px solid #4CAF50;
word-wrap: anywhere;
}
.button1:hover {
background-color: #4CAF50;
color: white;
}
#image-holder {
display: flex;
flex-direction: row;
flex-wrap: wrap;
justify-content: space-around;
}

466
net/webrtc/www/webrtc.js Normal file
View file

@ -0,0 +1,466 @@
/* vim: set sts=4 sw=4 et :
*
* Demo Javascript app for negotiating and streaming a sendrecv webrtc stream
* with a GStreamer app. Runs only in passive mode, i.e., responds to offers
* with answers, exchanges ICE candidates, and streams.
*
* Author: Nirbheek Chauhan <nirbheek@centricular.com>
*/
// Set this to override the automatic detection in websocketServerConnect()
var ws_server;
var ws_port;
// Override with your own STUN servers if you want
var rtc_configuration = {iceServers: [{urls: "stun:stun.l.google.com:19302"},
/* TODO: do not keep these static and in clear text in production,
* and instead use one of the mechanisms discussed in
* https://groups.google.com/forum/#!topic/discuss-webrtc/nn8b6UboqRA
*/
{'urls': 'turn:turn.homeneural.net:3478?transport=udp',
'credential': '1qaz2wsx',
'username': 'test'
}],
/* Uncomment the following line to ensure the turn server is used
* while testing. This should be kept commented out in production,
* as non-relay ice candidates should be preferred
*/
// iceTransportPolicy: "relay",
};
var sessions = {}
/* https://stackoverflow.com/questions/105034/create-guid-uuid-in-javascript */
function getOurId() {
return 'xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx'.replace(/[xy]/g, function(c) {
var r = Math.random() * 16 | 0, v = c == 'x' ? r : (r & 0x3 | 0x8);
return v.toString(16);
});
}
function Uint8ToString(u8a){
var CHUNK_SZ = 0x8000;
var c = [];
for (var i=0; i < u8a.length; i+=CHUNK_SZ) {
c.push(String.fromCharCode.apply(null, u8a.subarray(i, i+CHUNK_SZ)));
}
return c.join("");
}
function Session(our_id, peer_id, closed_callback) {
this.id = null;
this.peer_connection = null;
this.ws_conn = null;
this.peer_id = peer_id;
this.our_id = our_id;
this.closed_callback = closed_callback;
this.data_channel = null;
this.input = null;
this.getVideoElement = function() {
return document.getElementById("stream-" + this.our_id);
};
this.resetState = function() {
if (this.peer_connection) {
this.peer_connection.close();
this.peer_connection = null;
}
var videoElement = this.getVideoElement();
if (videoElement) {
videoElement.pause();
videoElement.src = "";
}
var session_div = document.getElementById("session-" + this.our_id);
if (session_div) {
session_div.parentNode.removeChild(session_div);
}
if (this.ws_conn) {
this.ws_conn.close();
this.ws_conn = null;
}
this.input && this.input.detach();
this.data_channel = null;
};
this.handleIncomingError = function(error) {
this.resetState();
this.closed_callback(this.our_id);
};
this.setStatus = function(text) {
console.log(text);
var span = document.getElementById("status-" + this.our_id);
// Don't set the status if it already contains an error
if (!span.classList.contains('error'))
span.textContent = text;
};
this.setError = function(text) {
console.error(text);
var span = document.getElementById("status-" + this.our_id);
span.textContent = text;
span.classList.add('error');
this.resetState();
this.closed_callback(this.our_id);
};
// Local description was set, send it to peer
this.onLocalDescription = function(desc) {
console.log("Got local description: " + JSON.stringify(desc), this);
var thiz = this;
this.peer_connection.setLocalDescription(desc).then(() => {
this.setStatus("Sending SDP answer");
var sdp = {
'type': 'peer',
'sessionId': this.id,
'sdp': this.peer_connection.localDescription.toJSON()
};
this.ws_conn.send(JSON.stringify(sdp));
}).catch(function(e) {
thiz.setError(e);
});
};
this.onRemoteDescriptionSet = function() {
this.setStatus("Remote SDP set");
this.setStatus("Got SDP offer");
this.peer_connection.createAnswer()
.then(this.onLocalDescription.bind(this)).catch(this.setError);
}
// SDP offer received from peer, set remote description and create an answer
this.onIncomingSDP = function(sdp) {
var thiz = this;
this.peer_connection.setRemoteDescription(sdp)
.then(this.onRemoteDescriptionSet.bind(this))
.catch(function(e) {
thiz.setError(e)
});
};
// ICE candidate received from peer, add it to the peer connection
this.onIncomingICE = function(ice) {
var candidate = new RTCIceCandidate(ice);
var thiz = this;
this.peer_connection.addIceCandidate(candidate).catch(function(e) {
thiz.setError(e)
});
};
this.onServerMessage = function(event) {
console.log("Received " + event.data);
try {
msg = JSON.parse(event.data);
} catch (e) {
if (e instanceof SyntaxError) {
this.handleIncomingError("Error parsing incoming JSON: " + event.data);
} else {
this.handleIncomingError("Unknown error parsing response: " + event.data);
}
return;
}
if (msg.type == "registered") {
this.setStatus("Registered with server");
this.connectPeer();
} else if (msg.type == "sessionStarted") {
this.setStatus("Registered with server");
this.id = msg.sessionId;
} else if (msg.type == "error") {
this.handleIncomingError(msg.details);
} else if (msg.type == "endSession") {
this.resetState();
this.closed_callback(this.our_id);
} else if (msg.type == "peer") {
// Incoming peer message signals the beginning of a call
if (!this.peer_connection)
this.createCall(msg);
if (msg.sdp != null) {
this.onIncomingSDP(msg.sdp);
} else if (msg.ice != null) {
this.onIncomingICE(msg.ice);
} else {
this.handleIncomingError("Unknown incoming JSON: " + msg);
}
}
};
this.streamIsPlaying = function(e) {
this.setStatus("Streaming");
};
this.onServerClose = function(event) {
this.resetState();
this.closed_callback(this.our_id);
};
this.onServerError = function(event) {
this.handleIncomingError('Server error');
};
this.websocketServerConnect = function() {
// Clear errors in the status span
var span = document.getElementById("status-" + this.our_id);
span.classList.remove('error');
span.textContent = '';
console.log("Our ID:", this.our_id);
var ws_port = ws_port || '8443';
if (window.location.protocol.startsWith ("file")) {
var ws_server = ws_server || "127.0.0.1";
} else if (window.location.protocol.startsWith ("http")) {
var ws_server = ws_server || window.location.hostname;
} else {
throw new Error ("Don't know how to connect to the signalling server with uri" + window.location);
}
var ws_url = 'ws://' + ws_server + ':' + ws_port
this.setStatus("Connecting to server " + ws_url);
this.ws_conn = new WebSocket(ws_url);
/* When connected, immediately register with the server */
this.ws_conn.addEventListener('open', (event) => {
this.setStatus("Connecting to the peer");
this.connectPeer();
});
this.ws_conn.addEventListener('error', this.onServerError.bind(this));
this.ws_conn.addEventListener('message', this.onServerMessage.bind(this));
this.ws_conn.addEventListener('close', this.onServerClose.bind(this));
};
this.connectPeer = function() {
this.setStatus("Connecting " + this.peer_id);
this.ws_conn.send(JSON.stringify({
"type": "startSession",
"peerId": this.peer_id
}));
};
this.onRemoteStreamAdded = function(event) {
var videoTracks = event.stream.getVideoTracks();
var audioTracks = event.stream.getAudioTracks();
console.log(videoTracks);
if (videoTracks.length > 0) {
console.log('Incoming stream: ' + videoTracks.length + ' video tracks and ' + audioTracks.length + ' audio tracks');
this.getVideoElement().srcObject = event.stream;
this.getVideoElement().play();
} else {
this.handleIncomingError('Stream with unknown tracks added, resetting');
}
};
this.createCall = function(msg) {
console.log('Creating RTCPeerConnection');
this.peer_connection = new RTCPeerConnection(rtc_configuration);
this.peer_connection.onaddstream = this.onRemoteStreamAdded.bind(this);
this.peer_connection.ondatachannel = (event) => {
console.log(`Data channel created: ${event.channel.label}`);
this.data_channel = event.channel;
video_element = this.getVideoElement();
if (video_element) {
this.input = new Input(video_element, (data) => {
if (this.data_channel) {
console.log(`Navigation data: ${data}`);
this.data_channel.send(JSON.stringify(data));
}
});
}
this.data_channel.onopen = (event) => {
console.log("Receive channel opened, attaching input");
this.input.attach();
}
this.data_channel.onclose = (event) => {
console.info("Receive channel closed");
this.input && this.input.detach();
this.data_channel = null;
}
this.data_channel.onerror = (event) => {
this.input && this.input.detach();
console.warn("Error on receive channel", event.data);
this.data_channel = null;
}
let buffer = [];
this.data_channel.onmessage = (event) => {
if (typeof event.data === 'string' || event.data instanceof String) {
if (event.data == 'BEGIN_IMAGE')
buffer = [];
else if (event.data == 'END_IMAGE') {
var decoder = new TextDecoder("ascii");
var array_buffer = new Uint8Array(buffer);
var str = decoder.decode(array_buffer);
let img = document.getElementById("image");
img.src = 'data:image/png;base64, ' + str;
}
} else {
var i, len = buffer.length
var view = new DataView(event.data);
for (i = 0; i < view.byteLength; i++) {
buffer[len + i] = view.getUint8(i);
}
}
}
}
this.peer_connection.onicecandidate = (event) => {
if (event.candidate == null) {
console.log("ICE Candidate was null, done");
return;
}
this.ws_conn.send(JSON.stringify({
"type": "peer",
"sessionId": this.id,
"ice": event.candidate.toJSON()
}));
};
this.setStatus("Created peer connection for call, waiting for SDP");
};
document.getElementById("stream-" + this.our_id).addEventListener("playing", this.streamIsPlaying.bind(this), false);
this.websocketServerConnect();
}
function startSession() {
var peer_id = document.getElementById("camera-id").value;
if (peer_id === "") {
return;
}
sessions[peer_id] = new Session(peer_id);
}
function session_closed(peer_id) {
sessions[peer_id] = null;
}
function addPeer(peer_id, meta) {
console.log("Meta: ", JSON.stringify(meta));
var nav_ul = document.getElementById("camera-list");
meta = meta ? meta : {"display-name": peer_id};
let display_html = `${meta["display-name"] ? meta["display-name"] : peer_id}<ul>`;
for (const key in meta) {
if (key != "display-name") {
display_html += `<li>- ${key}: ${meta[key]}</li>`;
}
}
display_html += "</ul>"
var li_str = '<li id="peer-' + peer_id + '"><button class="button button1">' + display_html + '</button></li>';
nav_ul.insertAdjacentHTML('beforeend', li_str);
var li = document.getElementById("peer-" + peer_id);
li.onclick = function(e) {
var sessions_div = document.getElementById('sessions');
var our_id = getOurId();
var session_div_str = '<div class="session" id="session-' + our_id + '"><video preload="none" class="stream" id="stream-' + our_id + '"></video><p>Status: <span id="status-' + our_id + '">unknown</span></p></div>'
sessions_div.insertAdjacentHTML('beforeend', session_div_str);
sessions[peer_id] = new Session(our_id, peer_id, session_closed);
}
}
function clearPeers() {
var nav_ul = document.getElementById("camera-list");
while (nav_ul.firstChild) {
nav_ul.removeChild(nav_ul.firstChild);
}
}
function onServerMessage(event) {
console.log("Received " + event.data);
try {
msg = JSON.parse(event.data);
} catch (e) {
if (e instanceof SyntaxError) {
console.error("Error parsing incoming JSON: " + event.data);
} else {
console.error("Unknown error parsing response: " + event.data);
}
return;
}
if (msg.type == "welcome") {
console.info(`Got welcomed with ID ${msg.peer_id}`);
ws_conn.send(JSON.stringify({
"type": "list"
}));
} else if (msg.type == "list") {
clearPeers();
for (i = 0; i < msg.producers.length; i++) {
addPeer(msg.producers[i].id, msg.producers[i].meta);
}
} else if (msg.type == "peerStatusChanged") {
var li = document.getElementById("peer-" + msg.peerId);
if (msg.roles.includes("producer")) {
if (li == null) {
console.error('Adding peer');
addPeer(msg.peerId, msg.meta);
}
} else if (li != null) {
li.parentNode.removeChild(li);
}
} else {
console.error("Unsupported message: ", msg);
}
};
function clearConnection() {
ws_conn.removeEventListener('error', onServerError);
ws_conn.removeEventListener('message', onServerMessage);
ws_conn.removeEventListener('close', onServerClose);
ws_conn = null;
}
function onServerClose(event) {
clearConnection();
clearPeers();
console.log("Close");
window.setTimeout(connect, 1000);
};
function onServerError(event) {
clearConnection();
clearPeers();
console.log("Error", event);
window.setTimeout(connect, 1000);
};
function connect() {
var ws_port = ws_port || '8443';
if (window.location.protocol.startsWith ("file")) {
var ws_server = ws_server || "127.0.0.1";
} else if (window.location.protocol.startsWith ("http")) {
var ws_server = ws_server || window.location.hostname;
} else {
throw new Error ("Don't know how to connect to the signalling server with uri" + window.location);
}
var ws_url = 'ws://' + ws_server + ':' + ws_port
console.log("Connecting listener");
ws_conn = new WebSocket(ws_url);
ws_conn.addEventListener('open', (event) => {
ws_conn.send(JSON.stringify({
"type": "setPeerStatus",
"roles": ["listener"]
}));
});
ws_conn.addEventListener('error', onServerError);
ws_conn.addEventListener('message', onServerMessage);
ws_conn.addEventListener('close', onServerClose);
}
function setup() {
connect();
}