Commit graph

26842 commits

Author SHA1 Message Date
Vedang Patel
551258b2c4 tests: Add tests for CRF Synchronizer element
This adds tests to validate whether the avtpcrfsync element applies the
adjustment correctly.

Also, the infrastructure to include additional source files while compiling
is added. This change is exactly the same as the one in gst-plugins-good.
2020-04-30 23:31:25 +00:00
Vedang Patel
e47fa2006f avtp: Introduce the CRF Check element
This commit introduces the AVTP Clock Reference Format (CRF) Checker
element. This element re-uses the GstAvtpCrfBase class introduced along
with the CRF Synchronizer element.

This element will typically be used along with the avtpsrc element to
ensure that the AVTP timestamp (and H264 timestamp in case of CVF-H264
packets) is "aligned" with the incoming CRF stream. Here, "aligned" means
that the timestamp value should be within 25% of the period of the media
clock recovered from the CRF stream.

The user can also set an option (drop-invalid) in order to drop any packet
whose timestamp is not within the thresholds of the incoming CRF stream.
2020-04-30 23:31:25 +00:00
Vedang Patel
12ad2a4bcd avtp: Introduce the CRF Sync Element
This commit introduces the AVTP Clock Reference Format (CRF) Synchronizer
element. This element implements the AVTP CRF Listener as described in IEEE
1722-2016 Section 10.

CRF is useful in synchronizing events within different systems by
distributing a common clock. This is useful in a scenario where there are
multiple talkers who are sending data to a single listener which is
processing that data. E.g.  CCTV cameras on a network sending AVTP video
streams to a base station to display on the same screen.

It is assumed that all the systems are already time-synchronized with each
other. So, the AVTP Talker essentially adjusts the AVTP Presentation Time
so it's phase-locked with the reference clock provided by the CRF stream.

There are 2 different roles of systems which participate in CRF data
exchange.  A system can either be a CRF Talker, which samples it's own
clock and generates a stream of timestamps to transmit over the network, or
a CRF Listener, the system which receives the generated timestamps and
recovers the media clock from the timestamps. It then adjusts it's own
clock to align with recovered media clock. The timestamps generated by the
talker may not be continuous and the listener might have to interpolate
some timestamps to recover the media clock. The number of timestamps to
interpolate is mentioned in the CRF stream AVTPDU (Refer IEEE 1722-2016
Section 10.4 for AVTPDU structure). Only CRF Listener has been implemented
in this commit.

The CRF Sync element will create a separate thread to listen for the CRF
stream. This thread will calculate and store the average period of the
recovered media clock. The pipeline thread will use this stored period
along with the first timestamp of the latest CRF AVTPDU received to
calculate adjustment for timestamps in the audio/video streams. In case of
CRF AVTPDUs with single timestamp, two consecutive CRF AVTPDUs will be used
to figure out the average period of the recovered media clock.

In case of H264 streams, both AVTP timestamp and H264 timestamp will be
adjusted.

In the future commits, another "CRF Checker" element will be introduced
which will validate the timestamps on the AVTP Listener side. Which is why
a lot of code has been implemented as part of the gstcrfbase class.
2020-04-30 23:31:25 +00:00
Ederson de Souza
3ea0f694de clockselect: Add TAI clock support
Via new value for property clock-id, "tai", it's possible to use
GST_CLOCK_TYPE_TAI as pipeline clock.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1009>
2020-04-30 19:21:37 +00:00
Ederson de Souza
443c01e119 clockselect: Remove non-sense comment
Commentary appears to assume `gst_harness_find_element` return value was
"transfer none", but it is not the case. So, element must be unrefed
before the end.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1009>
2020-04-30 19:21:37 +00:00
krivoguzovVlad
b769af0c4f Update gstsrtobject.c
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/999>
2020-04-30 18:57:13 +00:00
Olivier Crête
d9512dc132 ristrtpdeext: Expose the largest sequence number received
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:32 +00:00
Olivier Crête
f2e8d4dcf2 ristrtpdeext: Update RTP header extension packet to latest spec
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:32 +00:00
Olivier Crête
a602eb7eea ristrtpext: Update RTP header extension packet to latest spec
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:32 +00:00
Olivier Crête
03a60a47b5 rist: Document main profile support
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:32 +00:00
Olivier Crête
15f89cd088 ristsrc: Add ristrtpdeext to the pipeline
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:32 +00:00
Olivier Crête
a0de749814 ristsink: Add ristrtpext to sink
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:32 +00:00
Olivier Crête
f8bb1e0b85 ristsink: Receive RIST seqnum ext and feed it to rtxsend
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:32 +00:00
Olivier Crête
fc76254dfc ristsink: Pass the session id to the on-app-rtcp callback
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:31 +00:00
Olivier Crête
e873780a1f ristrtxsend: Use externally given seqnum extension when available
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:31 +00:00
Olivier Crête
58e31e116b ristrtxsend: Store sent packets with extended seqnum
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:31 +00:00
Olivier Crête
efd78bb8d8 rist: Factor our seqnum extension code
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:31 +00:00
Olivier Crête
1342e4ba43 rist: Add test for rtp ext code
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:31 +00:00
Olivier Crête
59b01048ae rist: Drop packets that are more than G_MAXINT16 seqnum late
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:31 +00:00
Olivier Crête
fa5d206c2c rist: Insert RTP seqnum extension header
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:31 +00:00
Olivier Crête
005bd960ee rist: Add element to remove the header extension
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:31 +00:00
Olivier Crête
7b0377c185 rist: Add element that inserts the RTP header extension
Currently can suppress the TS null packets, but can't insert
the seqnum extension yet.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1153>
2020-04-30 18:31:31 +00:00
Seungha Yang
24367cbff0 h264decoder: Allow frame gap even if it's not allowed by SPS
It's most likely the case where some frames were dropped for some reason
(e.g., broken stream, network loss or so). However, decoder might be able to
decode following frames even if some frames are visually broken.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1232>
2020-05-01 01:55:21 +09:00
Matthew Waters
80ede09193 webrtcbin: only start gathering on local descriptions
If we are in a state where we are answering, we would start gathering
when the offer is set which is incorrect for at least two reasons.

1. Sending ICE candidates before sending an answer is a hard error in
   all of the major browsers and will fail the negotiation.
2. If libnice ever adds the username fragment to the candidate for
   ice-restart hardening, the ice username and fragment would be
   incorrect.

JSEP also hints that the right call flow is to only start gathering when
a local description is set in 4.1.9 setLocalDescription

"This API indirectly controls the candidate gathering process."

as well as hints throughout other sections.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1226>
2020-04-30 14:47:55 +00:00
Seungha Yang
0b102d22ec webrtc: Correct symbol visibility to fix build warning on Windows
GstWebRTCDataChannel is fully internal of plugin

webrtcdatachannel.c(50): warning C4273: 'gst_webrtc_data_channel_get_type': inconsistent dll linkage

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1225>
2020-04-30 10:27:47 +00:00
Thibault Saunier
a3595f7e0f lv2: Namespace global variables and explicitly make them private
And fix a LV2_PORT_GROUPS__rearLeft/LV2_PORT_GROUPS__rearRight typo

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1125>
2020-04-29 19:49:45 +00:00
Debarshi Ray
a0cd455dd0 lv2: Make it build with -fno-common
GCC 10 defaults to -fno-common. This means that global variables shared
across multiple translation units should be declared as 'extern' in
header files and defined in exactly one C file. See:
https://gcc.gnu.org/gcc-10/porting_to.html

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1125

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1125>
2020-04-29 19:49:45 +00:00
Seungha Yang
ed147a3b48 d3d11h264dec: Rearrange the order of supported profiles in caps
constrained-baseline profile is much preferred than baseline.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1229>
2020-04-29 18:13:42 +00:00
Seungha Yang
28dc4a1dd8 nvh264sldec: Allow constrained-baseline and baseline profiles
Like d3d11h264dec, we need to relax the condition for profiles of nvh264sldec

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1228>
2020-04-30 02:36:34 +09:00
Seungha Yang
7db29fc382 d3d11h264dec: Add constrained-baseline and baseline profile
Both profiles are expected to be supported profile by DXVA hardware

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1212>
2020-04-29 14:24:02 +00:00
Nicolas Dufresne
bf9f82c8c3 v4l2codecs: Factor out plugin registration
This introduce a common place for generic functions and factor out the plugin
registration code. This code is nearly identical between implementation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1216>
2020-04-28 22:53:11 +00:00
Nicolas Dufresne
de2cccaa47 v4l2decoder: Relax width/height return value check
The driver adjust the width/height to coded size. This was not an issue for
H264, as the coded size is in the bitstream, but is an issue with VP8.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1216>
2020-04-28 22:53:11 +00:00
Nicolas Dufresne
72c0b3e5ae v4l2codecs: Add VP8 decoder
This is derived from the H264 decoder, some boiler plate will be factored out
in the following commits.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1216>
2020-04-28 22:53:11 +00:00
Joshua M. Doe
bfc605a666 mfvideosrc: add GRAY16_LE format support
FOURCC "Y16 " is used by FLIR Boson USB Video Class (UVC) camera

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1222>
2020-04-28 15:14:50 -04:00
Seungha Yang
3804dfb28d d3d11device: Add fallback for device creation
D3D11_CREATE_DEVICE_DEBUG flag will be used while creating d3d11 device
to activate debug layer. However, if system doesn't support the
debug layer for some reason, we should try to create d3d11 device
without the flag. Debug layer should be optional for device creation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1217>
2020-04-28 17:33:56 +00:00
Seungha Yang
c19d21f464 tests: mfvideosrc: Add unit test
Simple test for reuse scenario

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/760>
2020-04-28 14:37:31 +00:00
Seungha Yang
912b2d7af9 mediafoundation: Disable querying supprted maximum resolution
Initializing MFT for checking supported maximum resolution is too
slow.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/760>
2020-04-28 14:37:31 +00:00
Seungha Yang
565817abd8 mediafoundation: Add h265 encoder
Add Media Foundation HEVC encoder

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/760>
2020-04-28 14:37:31 +00:00
Seungha Yang
9625d19279 mediafoundation: Add h264 encoder
Add Media Foundation h264 encoder. If hardware encoders are available
on system, they will have higher rank than software encoder.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/760>
2020-04-28 14:37:31 +00:00
Seungha Yang
eece89042a mediafoundation: Introduce Microsoft Media Foundation plugin
The Microsoft Media Foundation (MF) is the successor of DirectShow.
This commit includes two kinds of video capture implementation,
one uses IMFSourceReader interface which is available since Windows Vista
and the other is based on IMFCaptureEngine interface which is available
since Windows 8.
Note that this new video source element cannot be used in UWP app
for now, since device activation using those APIs are not allowed by MS.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/760>
2020-04-28 14:37:31 +00:00
He Junyan
9082f103a4 test: h265parser: Add check for high throughput scc.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1121>
2020-04-28 13:54:11 +00:00
He Junyan
bd2fb6cbb9 libs: h265parser: Add High throughput scc extensions profiles' IDC.
It is compitable with scc and we use scc's function to identify it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1121>
2020-04-28 13:54:11 +00:00
He Junyan
be1cdca760 test: h265parser: Add more check or h265 extensions.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1121>
2020-04-28 13:54:11 +00:00
He Junyan
35bf824b6d libs: h265parser: select extension profile by profile idc.
the old manner does not consider the profile idc. The profile idc should
play an more important role in recognizing the profile than the other
information. And there is no need to mix profiles of different extensions
together to find the closest profile when the bits stream is not standard,
different extensions support different features and should not be mixed.

The correct way should be recognize the extension category by profile idc
firstly, and then find the closest profile.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1121>
2020-04-28 13:54:11 +00:00
He Junyan
f4ddccb08e libs: h265parser: rename the FormatRangeExtensionProfile
FormatRangeExtensionProfile declares the common bits used for not
only format range extensions profiles, but also for several different
h265 extension profiles, such as high throughput, screen content
coding extensions, etc. And So the old name is not proper.
We also rename the get_h265_extension_profile function.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1121>
2020-04-28 13:54:11 +00:00
He Junyan
e99b962c0f libs: h265parser: Fix a bug for getting extension profile.
We should use the traget ExtensionProfile's IDC to check the
profile_compatibility_flag, rather than the profile_idc in the
stream. The old profile_compatibility_flag check always return
true. This causes that profiles with same constraint flags but
different profile_idc can't be recognized correctly. For example,
the screen-extended-main-444 profile is always be recognized as
the high-throughput-444 profile.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1121>
2020-04-28 13:54:11 +00:00
Seungha Yang
1d38728d05 h264decoder: Perform low-latency outputting only for live pipeline
Frequent outputting might drop throughput-wise performance.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1211>
2020-04-28 13:23:53 +00:00
Seungha Yang
00ad871628 h264decoder: Output decoded pictures as soon as possible
In case of IDR, any previously decoded pictures must be drained
before the IDR and POC of IDR should be zero. So we can output
IDR immediately. Also, when POC of current picture is expected to be
the next output POC, decoder can output the picture as well
without waiting.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1211>
2020-04-28 13:23:53 +00:00
Nicolas Dufresne
ed6236f551 d3d11vp8dec: Remove unused parameters
Minor cleanup removing unused picture parameters in two internal functions.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1221>
2020-04-28 06:21:53 +00:00
Mathieu Duponchelle
d3be83a26c gsth265parser.h: escape link-like syntax in doc 2020-04-28 00:29:52 +02:00