Commit graph

6498 commits

Author SHA1 Message Date
Ederson de Souza
5592ab1769 docs: Add AVTP elements documentation 2019-07-03 09:59:35 -07:00
Ederson de Souza
45624661ba avtp: Add fragmented packets handling to CVF depayloader
This patch adds to the CVF depayloader the capability to regroup H.264
fragmented FU-A packets.

After all packets are regrouped, they are added to the "stash" of H.264
NAL units that will be sent as soon as an AVTP packet with M bit set is
found (usually, the last fragment).

Unrecognized fragments (such as first fragment seen, but with no Start
bit set) are discarded - and any NAL units on the "stash" are sent
downstream, as if a SEQNUM discontinuty happened.
2019-07-03 09:59:35 -07:00
Ederson de Souza
45d2f5a779 avtp: Introduce AVTP CVF depayloader element
This patch introduces the AVTP Compressed Video Format (CVF) depayloader
specified in IEEE 1722-2016 section 8. Currently, this depayloader only
supports H.264 encapsulation described in section 8.5.

Is also worth noting that only single NAL units are handled: aggregated
and fragmented payloads are not handled.

As stated in AVTP CVF payloader patch, AVTP timestamp is used to define
outgoing buffer DTS, while the H264_TIMESTAMP defines outgoing buffer
PTS.

When an AVTP packet is received, the extracted H.264 NAL unit is added to
a "stash" (the out_buffer) of H.264 NAL units. This "stash" is pushed
downstream as single buffer (with NAL units aggregated according to format
used on GStreamer, based on ISO/IEC 14496-15) as soon as we get the AVTP
packet with M bit set.

This patch groups NAL units using a fixed NAL size lenght, sent downstream
on the `codec_data` capability.

The "stash" of NAL units can be prematurely sent downstream if a
discontinuity (a missing SEQNUM) happens.

This patch reuses the infra provided by gstavtpbasedepayload.c.
2019-07-03 09:59:35 -07:00
Ederson de Souza
b056297eea avtp: Add fragmentation feature to CVF payloader
Based on `mtu` property, the CVF payloader is now capable of properly
fragmenting H.264 NAL units that are bigger than MTU in several AVTP
packets.

AVTP spec defines two methods for fragmenting H.264 packets, but this
patch only generates non-interleaved FU-A fragments.

Usually, only the last NAL unit from a group of NAL units in a single
buffer will be big enough to be fragmented. Nevertheless, only the last
AVTP packet sent for a group of NAL units will have the M bit set (this
means that the AVTP packet for the last fragment will only have the M
bit set if there's no more NAL units in the group).
2019-07-03 09:59:35 -07:00
Ederson de Souza
3b4f3a0b3f avtp: Introduce AVTP CVF payloader element
This patch introduces the AVTP Compressed Video Format (CVF) payloader
specified in IEEE 1722-2016 section 8. Currently, this payload only
supports H.264 encapsulation described in section 8.5.

Is also worth noting that only single NAL units are encapsulated: no
aggregation or fragmentation is performed by the payloader.

An interesting characteristic of CVF H.264 spec is that it defines an
H264_TIMESTAMP, in addition to the AVTP timestamp. The later is
translated to the GST_BUFFER_DTS while the former is translated to the
GST_BUFFER_PTS. From AVTP CVF H.264 spec, it is clear that the AVTP
timestamp is related to the decoding order, while the H264_TIMESTAMP is
an ancillary information to the H.264 decoder.

Upon receiving a buffer containing a group of NAL units, the avtpcvfpay
element will extract each NAL unit and payload them into individual AVTP
packets. The last AVTP packet generated for a group of NAL units will
have the M bit set, so the depayloader is able to properly regroup them.

The exact format of the buffer of NAL units is described on the
'codec_data' capability, which is parsed by the avtpcvfpay, in the same
way done in rtph264pay.

This patch reuses the infra provided by gstavtpbasepayload.c.
2019-07-03 09:59:35 -07:00
Andre Guedes
5abe516c6c avtp: Introduce AVTP source element
This patch introduces the avtpsrc element which implements a typical
network source. The avtpsrc element receives AVTPDUs encapsulated into
Ethernet frames and push them downstream in the GStreamer pipeline.
Implementation if pretty straightforward since the burden is implemented
by GstPushSrc class.

Likewise the avtpsink element, applications that utilize this element
must have CAP_NET_RAW capability since it is required by Linux to open
sockets from AF_PACKET domain.
2019-07-03 09:59:35 -07:00
Andre Guedes
37550226d8 avtp: Introduce AVTP sink element
This patch introduces the avtpsink elements which implements a typical
network sink. Implementation is pretty straightforward since the burden
is implemented by GstBaseSink class.

The avtpsink element defines three new properties: 1) network interface
from where AVTPDU should be transmitted, 2) destination MAC address
(usually a multicast address), and 3) socket priority (SO_PRIORITY).

Socket setup and teardown are done in start/stop virtual methods while
AVTPDU transmission is carried out by render(). AVTPDUs are encapsulated
into Ethernet frames and transmitted to the network via AF_PACKET socket
domain.  Linux requires CAP_NET_RAW capability in order to open an
AF_PACKET socket so the application that utilize this element must have
it. For further info about AF_PACKET socket domain see packet(7).

Finally, AVTPDUs are expected to be transmitted at specific times -
according to the GstBuffer presentation timestamp - so the 'sync'
property from GstBaseSink is set to TRUE by default.
2019-07-03 09:59:35 -07:00
Andre Guedes
6477884a56 avtp: Introduce AAF depayloader element
This patch introduces the AAF depayloader element, the counterpart from
the AAF payloader. As expected, this element inputs AVTPDUs and outputs
audio raw data and supports AAF PCM encapsulation only.

The AAF depayloader srcpad produces a fixed format that is encoded
within the AVTPDU. Once the first AVTPDU is received by the element, the
audio features e.g. sample format, rate, number of channels, are decoded
and the srcpad caps are set accordingly. Also, at this point, the
element pushes a SEGMENT event downstream defining the segment according
to the AVTP presentation time.

All AVTP depayloaders will share some common code. For that reason, this
patch introduces the GstAvtpBaseDepayload abstract class that implements
common depayloader functionalities. AAF-specific functionalities are
implemented in the derived class GstAvtpAafDepay.
2019-07-03 09:59:35 -07:00
Andre Guedes
1e985f02f4 avtp: Introduce AAF payloader element
This patch introduces the AVTP Audio Format (AAF) payloader element from
the AVTP plugin. The element inputs audio raw data and outputs AVTP
packets (aka AVTPDUs), implementing a typical protocol payloader element
from GStreamer.

AAF is one of the available formats to transport audio data in an AVTP
system. AAF is specified in IEEE 1722-2016 section 7 and provides two
encapsulation mode: PCM and AES3. This patch implements PCM
encapsulation mode only.

The AAF payloader working mechanism consists of building the AAF header,
prepending it to the GstBuffer received on the sink pad, and pushing the
buffer downstream. Payloader parameters such as stream ID, maximum
transit time, time uncertainty, and timestamping mode are passed via
element properties. AAF doesn't support all possible sample format and
sampling rate values so the sink pad caps template from the payloader is
a subset of audio/x-raw. Additionally, this patch implements only
"normal" timestamping mode from AAF. "Sparse" mode should be implemented
in future.

Upcoming patches will introduce other AVTP payloader elements that will
have some common code. For that reason, this patch introduces the
GstAvtpBasePayload abstract class that implements common payloader
functionalities, and the GstAvtpAafPay class that extends the
GstAvtpBasePayload class, implementing AAF-specific functionalities.

The AAF payloader element is most likely to be used with the AVTP sink
element (to be introduced by a later patch) but it could also be used
with UDP sink element to implement AVTP over UDP as described in IEEE
1722-2016 Annex J.

This element was inspired by RTP payloader elements.
2019-07-03 09:59:35 -07:00
Andre Guedes
eaeab383bb avtp: AVTP plugin bootstrap code
This patch introduces the bootstrap code from the AVTP plugin (plugin
definition and init) as well as the build system files. Upcoming patches
will introduce payloaders, source and sink elements provided by the AVTP
plugin. These elements can be utilized by a GStreamer pipeline to
implement TSN audio/video applications.

Regarding the plugin build system files, both autotools and meson files
are introduced. The AVTP plugin is landed in ext/ since it has an
external dependency on libavtp, an opensource AVTP packetization
library. For further information about libavtp check [1].

[1] https://github.com/AVnu/libavtp
2019-07-03 09:59:35 -07:00
Juan Navarro
d289608a99 dtlsagent: Clear the certificate upon finalize
Cleaning this up was likely just forgotten
2019-06-25 20:37:57 +02:00
Juan Navarro
8317112883 dtlsdec: Avoid duplicate ref when passing certificate property
The agent itself will take a ref on the property setter, so we'll be
left with two references to the certificate object, when actually there
should be only one
2019-06-25 20:37:38 +02:00
Philippe Normand
3811d13269 wpe: Port for WPEWebKit 2.25.x
When WPEBackend-fdo >= 1.3.0 is detected, the threaded view now relies on the
wpe_fdo_egl_exported_image API instead of the EGLImageKHR-based API which is
going to be deprecated in 2.26. The GLib sources created by the view now use the
default priority as well, the custom priority is no longer required.
2019-06-25 10:16:25 +00:00
Matthew Waters
ec9ea06591 vkswapper: support rescaling to the output size 2019-06-24 16:23:29 +10:00
Matthew Waters
35325fdffb vksink: Don't take vulkan buffer's as input 2019-06-24 16:23:29 +10:00
Philippe Normand
36de11520e webrtc: Fix data-channel send-string doc 2019-06-23 17:03:32 +01:00
Matthew Waters
dcff70d6bf vulkan: remove unused downsample AYUV shader 2019-06-20 05:44:47 +00:00
Mathieu Duponchelle
9023ac1c95 webrtcbin: fix DTLS when receivebin is set to DROP
Regression introduced by b4bdcf15b7

This commit prevents the handshake from reaching dtlsdec when
the receive state of the receive bin is set to DROP (for example
when transceivers are sendonly).

This preserves the intent of the commit, by blocking the bin
at its sinks until the receive state is no longer BLOCK, but
makes sure the handshake still goes through, by only dropping
data at the src pads, as was the case before.
2019-06-19 18:04:14 +00:00
Matthew Waters
d43fea06c5 vkcolorconvert: add support for RGB<->NV12 2019-06-20 01:41:56 +10:00
Matthew Waters
0cb416db11 vkbuffermemory: report requested size of the memory
Rather than using Vulkan's much larger aligned sizes. Fixes multi-planer
video with the GstVideoFrame API.
2019-06-20 01:41:56 +10:00
Matthew Waters
2180fbb498 vulkancolorconvert: support RGB <-> AYUV/YUY2/UYVY 2019-06-20 01:41:56 +10:00
Matthew Waters
dbf60a1738 vktrash: add mini_object_unref destroy function 2019-06-20 01:41:56 +10:00
Matthew Waters
b93de8b0d8 vkfullscreenrender: create descriptor sets later
The desciptor sets may be dependant on the caps
2019-06-20 01:41:56 +10:00
Matthew Waters
2e34dd891d vulkan/shaders: change glslc compilation args based on configurations
Add depfile support so that modifying an #included glsl snippet
rebuilds all the dependant shaders.
2019-06-20 01:41:56 +10:00
Matthew Waters
5363b30f6c vulkan: add a color conversion element
Currently converts between all 4-component RGBA/RGBx formats.
2019-06-20 01:41:56 +10:00
Matthew Waters
31728880fe vkupload: Also implement copying non vulkan memory into vulkan buffers
The only way we can upload things is through our memory so any
non-vulkan memory that appears must be copied into our memory.
2019-06-20 01:41:56 +10:00
Matthew Waters
3fef510c48 vulkan: add download element
Currently only downloads images into a host-visible buffer and
synchronises immediately.
2019-06-20 01:41:56 +10:00
Matthew Waters
dc0c9dea7f vulkan/identity: Split out most rendering code to a base class
A simple base class that renders a 2d fullscreen quad parallel to the
screen surface inside the view frustum.
2019-06-20 01:41:56 +10:00
Matthew Waters
773e914f48 vkupload: fix a structure sType 2019-06-20 01:41:56 +10:00
Matthew Waters
db617a98ad vksink: Retrieve vulkan queue earlier
Allows using the swapper's queue over upstream's queue.  The swapper
will check for the necessary presentation support that upstream may not
consider.
2019-06-20 01:41:56 +10:00
Sebastian Dröge
5ed27c12cc zbar: Include running-time, stream-time and duration in the messages
The timestamp/PTS alone is meaningless without the segment and usually
applications care about the running-time or stream-time.

This also keeps the messages in sync with the spectrum and level
elements.
2019-06-19 13:31:39 +03:00
Seungha Yang
e779160434 tests: Enable hls m3u8 unit test with meson build 2019-06-18 07:14:28 +00:00
Seungha Yang
48657bf846 hlsdemux: Set fragment header uri if exists
To allow downloading fragment hearder, set its uri if there is
available "Media Initialization" parsed from EXT-X-MAP tag

https://bugzilla.gnome.org/show_bug.cgi?id=776928
2019-06-18 07:14:28 +00:00
Seungha Yang
f9dc67c372 hls: m3u8: Parsing EXT-X-MAP tag to store initialization data
EXT-X-MAP tag informs media initialization data,
such as moov box in ISOBMFF case and PAT/PMT for MPEG TS stream.

https://bugzilla.gnome.org/show_bug.cgi?id=776928
2019-06-18 07:14:28 +00:00
Seungha Yang
b45dd51fb1 hlsdemux: Do not clear/advance fragment by finished header downloading
Header data must be forwarded to downstream, but if demux does not finish
to finding type (e.g., ts, mp4 and etc), this header data can be cleared
by _stream_clear_pending_data(). Moreover, although demux finish downloading
header data, still it has fragment date to be downloaded, fragment sequence
shouldn't be advanced yet at that moment.

https://bugzilla.gnome.org/show_bug.cgi?id=776928
2019-06-18 07:14:28 +00:00
Philippe Normand
5105bc8041 wpe: Fix build with -Werror enabled
Including gl.h from WPEThreadedView.h leads to GST_LEVEL_DEFAULT detected as
redefined. The proposed fix is to include config.h from the CPP implementation
file and disable gl.h inclusion in the header, by using forward declarations.
2019-06-05 12:47:16 +01:00
Matthew Waters
f33310df56 vkupload: remove debugging error log 2019-06-04 09:03:44 +00:00
Matthew Waters
ac09f88eb9 vulkan: implement the correct memory barriers necessary
Vulkan validation layers are finally silent!
2019-06-04 09:03:44 +00:00
Matthew Waters
1230a6c723 vulkan: remove uneeded buffer inheritence information
buffer inheritence informationi is only required for secondary command
buffers which we are not creating
2019-06-04 09:03:44 +00:00
Matthew Waters
9593e4e8bb vulkan: add a new image copying element
Copies using the graphics pipeline, an input image and renders to an
output image.
2019-06-04 09:03:44 +00:00
Matthew Waters
80cbbb319d vkupload: add raw->vulkanimage uploader 2019-06-04 09:03:44 +00:00
Matthew Waters
ca38c2f25e vktrash: add more destruction of vulkan types 2019-06-04 09:03:44 +00:00
Matthew Waters
d61e771c37 vulkan: use c99 designated initializers where possible
Makes the code much easier to read and allows removing our terrible
macros for structure initialization.
2019-06-04 09:03:44 +00:00
Matthew Waters
b02493091b vkswapper: support displaying memory:VulkanImage 2019-06-04 09:03:44 +00:00
Matthew Waters
673d775df0 vkupload: implement buffer to image uploader 2019-06-04 09:03:44 +00:00
Matthew Waters
f351f3a759 vulkansink: Respond to queue queries 2019-06-04 09:03:44 +00:00
Matthew Waters
32d217a9df vulkan: add a command pool object for tracking 2019-06-04 09:03:44 +00:00
Matthew Waters
2d0cab20c0 vkswapper: don't output duplicate formats 2019-06-04 09:03:44 +00:00
Matthew Waters
25dd3b32e5 vulkan: make a gstvulkan library out of the existing API 2019-06-04 09:03:44 +00:00
Matthew Waters
8def3b3743 vkbufferpool: Fix multiplaner allocations
Use the plane width/height and the sizes required by vulkan

Fixes allocation of:
videotestsrc ! video/x-raw,format=I420 ! vulkanupload ! fakesink
2019-06-04 09:03:44 +00:00