Everytime a buffer is being provided from baseparse, we are parsing all the data from the beginning.
But since we would have already parsed some of the data in the previous iterations,
it doesnt make much sense to keep parsing the same everytime.
Hence skipping the data which is already read in previous iterations to improve the parsing performance.
https://bugzilla.gnome.org/show_bug.cgi?id=740058
The image capture mutex and the pad object lock would cause a race
if the pad query was made right when the image probe was running.
The image probe needs the capture mutex and the querying would need
the pad object lock.
It might be racy with the image probe thread as it uses the capture
mutex just like the start-capture handler from camerabin. The
start-capture would be waiting for the source's streaming thread
to stop to be able to set the source state to ready while the
probe would be blocked waiting to acquire the capture mutex.
It causes a deadlock.
Don't rely on core implementation details, which are private and
may change. It's also not needed here, the performance impact is
close to none. Also copy buffer before changing its metadata.
Get rid of some indirections and inefficiencies,
just payload things directly which gives us more
control over what memory is allocated where and
how and makes things much simpler. In particular,
we can now allocate the payload header plus the
GstMemory to represent it in one go.
Get rid of now-useless packetizer struct and just
call internal functions directly. Also remove
version property which is now defunct, not least
because we create the packetizer with the
version in the init function before a version
can be set.
Add function to calculate a payload CRC across multiple memories
so we don't have to merge buffers with multiple memories just to
calculate the CRC. Also make CRC calculation function static,
since it's not used outside dataprotocol.h and move special-casing
of length = 0 -> CRC = 0 into CRC function (from caller).
Perhaps more importantly, since payload CRC is off by default:
don't map buffer (and possibly merge memories in the process)
if we are not going to use it to calculate a CRC anyway.
This can happen if this is a live pipeline and no source produced any buffer
and sent no caps until the an output buffer should've been produced according
to the latency.
When this is TRUE, we really have to produce output. This happens
in live mixing mode when we have to output something for the current
time, no matter if we have enough input or not.
Some video bitstreams report a too restrictive set of profiles. If a video
decoder was to strictly follow the indicated profile, it wouldn't support that
stream, whereas it could in theory and in practice. So we should relax the
profile restriction for allowing the decoder to get connected with parser.
https://bugzilla.gnome.org/show_bug.cgi?id=739992
When dealing with random-access content (such as files), we initially
search for the last PCR in order to figure out duration and to handle
other position estimation such as those used in seeking.
Previously, the code looking for that last PCR would search in the last
640kB of the file going forward, and stop at the first PCR encountered.
The problem with that was two-fold:
* It wouldn't really be the last PCR (it would be the first one within
those last 640kB. In case of VBR files, this would put off duration
and seek code slightly.
* It would fail on files with bitrates higher than 52Mbit/s (not common)
Instead this patch modifies that code by:
* Scanning over the last 2048kB (allows to cope with streams up to 160Mbit/s)
* Starts by the end of the file, going over chunks of 300 MPEG-TS packets
* Doesn't stop at the first PCR detected in a chunk, but instead records all
of them, and only stop searching if there was "at least" one PCR within
that chunk
This should improve duration reporting and seeking operations on VBR files
https://bugzilla.gnome.org/show_bug.cgi?id=708532
Sometimes rawparse does not handle the seeking query
properly, the rawparse should send the query upstream
first. For example, upstream could support seeking in
TIME format (but not in BYTE format), so the BYTE format
seeking query that rawparse sends in push mode would
fail.
https://bugzilla.gnome.org/show_bug.cgi?id=722764
Read PNG data chunk in one go by letting the parser
base class know the size we need, so that it doesn't
drip-feed us small chunks of data (causing a lot of
reallocs and memcpy in the process) until we have
everything.
Improves parsing performance of very large PNG files
(65MB) from ~13 seconds to a couple of millisecs.
https://bugzilla.gnome.org/show_bug.cgi?id=736176
This commit add an helper to convert a frame to frame-layer format and
use it to implement these two stream-format conversion:
- asf --> sequence-layer-frame-layer
- asf --> frame-layer
In simple/main profile, we basically have a raw frame, so building a
frame layer isn't too complicated. But in advanced profile, the first
frame-layer should contain sequence-header, entrypoint, and frame and
each keyframe should contain entrypoint, so we have to handle these
carefully.
https://bugzilla.gnome.org/show_bug.cgi?id=738526
Add an helper to check that output stream-format is coherent with
profile and header-format. It also check if we know how to do the
conversion if the input stream-format differs from selected
output-format.
So, in case output stream-format is not allowed, it will now fail at
negotiation rather than in pre_push_frame.
https://bugzilla.gnome.org/show_bug.cgi?id=738526
This commit introduces an helper to convert an ASF frame to BDUs format with
startcodes and use this helper to implements following stream-format
conversions:
- asf --> bdu
- asf --> sequence-layer-bdu
- asf --> sequence-layer-raw-frame
https://bugzilla.gnome.org/show_bug.cgi?id=738526
It add the support of following stream-format conversion:
- bdu --> sequence-layer-bdu
- bdu-frame --> sequence-layer-bdu-frame
- frame-layer --> sequence-layer-frame-layer
For these conversion, the only requirements is to push a sequence-layer
buffer prior to data.
https://bugzilla.gnome.org/show_bug.cgi?id=738526
It prepares the template for stream-format conversion and it implements
the following conversion:
- sequence-layer-bdu --> bdu
- sequence-layer-bdu-frame --> bdu-frame
- sequence-layer-frame-layer --> frame-layer
Work is done in the pre_push_frame() method.
https://bugzilla.gnome.org/show_bug.cgi?id=738526
gstinteraudiosrc.c: In function 'gst_inter_audio_src_create':
gstinteraudiosrc.c:339:27: error: variable 'buffer_samples' set but not used [-Werror=unused-but-set-variable]
guint64 period_samples, buffer_samples;
^
The whole not_linked optimisation is really a bit dodgy here, but
let's leave it in place for now and at least start pushing data
again when a pad got linked later, in which case we should get a
RECONFIGURE event.
Current CLAMP checks both if the value is below 0 or above 255. Considering it
is an unsigned value it can never be less than zero, so that comparison is
unnecessary. Switching to using if just for the upper bound.
CID #1139796
Value from left_luminance is assigned to out_luminance here, but that stored
value is not used before it is overwritten in the next cycle of the loop.
Removing assignation.
CID #1226473
As a consequence, tsdemux won't remove its pads anymore on EOS.
Fixes the case when mpegtsbase is not able to process new packets
after EOS as the corresponding pids aren't known anymore because
the programs were removed and the pes/psi were kept, preventing the
PAT to be parsed again.
https://bugzilla.gnome.org/show_bug.cgi?id=738695
It was using a 24000/24000/48000, but I think it meant to use
24000/32000/48000. Not 100% sure...
https://en.wikipedia.org/wiki/G.722.1 has the list of supported
bitrates. It's not clear whether the "flag" code maps to this,
however.
Coverity 206072
This parses the frame_packing_arragement() payload in SEI message.
This information can be used by decoders to appropriately rearrange the
samples which belong to Stereoscopic and Multiview High profiles.
https://bugzilla.gnome.org/show_bug.cgi?id=685215
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Assume that small backward PCR jumps are just from upstream packet
mis-ordering and don't reset timestamp tracking state - assuming that
things will be OK again shortly.
Make the threshold for detecting discont between sequential buffers
configurable and match the smoothing-latency setting on tsparse
to better cope with data bursts.
When the set-timestamps property is set, use PCRs on the provided
(or autodetected) pcr-pid to apply (or replace) timestamps on the
output buffers, using piece-wise linear interpolation.
This allows tsparse to be used to stream an arbitrary mpeg-ts file,
or to smooth jittery reception timestamps from a network stream.
The reported latency is increased to match the smoothing latency if
necessary.
Otherwise a magic capsfilter after the source is required with
exactly the same caps as the input.
This would've failed before with invalid buffer sizes:
gst-launch-1.0 videotestsrc ! intervideosink intervideosrc ! "video/x-raw,width=640,height=480" ! xvimagesink
Audiomixer blocksize, cant be 0, hence adjusting the minimum value to 1
timeout value of aggregator is defined with MAX of MAXINT64,
but it cannot cross G_MAXLONG * GST_SECOND - 1
Hence changed the max value of the same
https://bugzilla.gnome.org/show_bug.cgi?id=738845
Signal sparse streams properly in stream-start event and force sending
of pending sticky events which have been stored on the pad already and
which otherwise would only be sent on the first buffer or serialized
event (which means very late in case of subtitle streams). Playsink in
playbin waits for stream-start or another serialized event, and if we
don't do this it will wait for the multiqueue to run full before
starting playback, which might take a couple of seconds.
https://bugzilla.gnome.org/show_bug.cgi?id=734040
All pads of a stream are now added at the beginning. In order to cope with
streams that don't get any data (forever or for a long time) we detect gaps
and push out GAP events when needed.
Cleanups and commenting by Jan Schmidt <jan@centricular.com>
https://bugzilla.gnome.org/show_bug.cgi?id=734040
Some VC1 decoder can have different caps according to wmv format, ie
WMV3 or WVC1.
So instead of keeping the first available caps, we interserct with
current WMV format.
https://bugzilla.gnome.org/show_bug.cgi?id=738532
When stream-format is ASF or sequence-layer-raw-frame, we basically have
a raw frame so we can parse it to extract some information such the
keyframe flag. The only requirement is to have a valid sequence-header.
This commit parse the frame header and set the DELTA_UNIT buffer flag in
case the frame is not a keyframe.
https://bugzilla.gnome.org/show_bug.cgi?id=738519
frame-layer header is represented as a sequence of 32 bit unsigned
integer serialized in little-endian byte order, so framesize is on the
first 3 bytes.
SMPTE 421M Annex L.
https://bugzilla.gnome.org/show_bug.cgi?id=738243
Also, strictly speaking, these numbers aren't DLT_*; they are LINKTYPE_* because
libpcap translates from internal OS-specific DLT_ numbering to the portable
LINKTYPE_ number space when writing files.
https://bugzilla.gnome.org/show_bug.cgi?id=738206
GST_BASE_PARSE_FRAME_FLAG_PARSING value is wrong, and the same flag is
not being used presently. Hence changing the value and commenting it out.
This needs to be included in baseparse.h later on
https://bugzilla.gnome.org/show_bug.cgi?id=737411
There are unnecessary definitions for disabling deprecation warnings.
Since GLIB_DISABLE_DEPRECATION_WARNINGS is not needed anymore in these files,
removing the same.
https://bugzilla.gnome.org/show_bug.cgi?id=737559
If a discontinuity in the stream is detected, data is discarded until
a new PES starts. If the first packet after the discontinuity is also
the start of a PES, there is no reason to discard the packets.
https://bugzilla.gnome.org/show_bug.cgi?id=737569
Or not negotiated in the case we would be actually not negotiated
Currently we are getting assertions from
gst_pb_utils_add_codec_description_to_tag_list because of NULL
caps.
https://bugzilla.gnome.org/show_bug.cgi?id=737186
If we don't have a seq_layer_buffer, we also don't have a valid
seq_layer because there are set together in
gst_vc1_parse_handle_seq_layer().
So when output header format is sequence-layer and when we don't have a
seq_layer_buffer, we forge one from seq_hdr.
https://bugzilla.gnome.org/show_bug.cgi?id=736781
Sequence-layer and frame-layer are serialized in little-endian byte
order except for STRUCT_C and framedata fields as described in SMPTE 421M Annex
L.
https://bugzilla.gnome.org/show_bug.cgi?id=736750
packetized mode is being set when framerate is being set
which is not correct. Changing the same by checking the
input segement format. If input segment is in TIME it is
Packetized, and if it is in BYTES it is not.
https://bugzilla.gnome.org/show_bug.cgi?id=736252
This commit fix several issues with sequence layer header forging on
update_caps():
- 0x00000004 unsigned integer is before STRUCT_C.
- Set reserved bits of STRUCT_C to their values for simple/main
profiles in sequence layer header format and ASF header format.
- Sequence layer shall be represented as a sequence of 32 bits unsigned
integers and shall be serialized in little-endian byte order except
for STRUCT_C which shall be serialized in big-endian byte-order.
See SMPTE 421M Annex L for more details about sequence layer format.
https://bugzilla.gnome.org/show_bug.cgi?id=736474
packet_length is defined as a guint16 in the PESHeader structure. This
definition match the specification. But since we add 6 bytes to the
packet_length value (length of start_code + stream_id + packet_length),
we can overflow the guint16 when the value in the PES header is greater
than 65529.
So use a guint32 instead of a guint16 to avoid overflow.
https://bugzilla.gnome.org/show_bug.cgi?id=736490
In gst_data_uri_src_create(), buf cannot be NULL, hence
else if (*buf != NULL) will be invalid so removing the
else if condition and adding a check to unreference buf
in else condition, just in case
https://bugzilla.gnome.org/show_bug.cgi?id=735861
gst_zebra_stripe_transform_frame_ip_planarY
gst_zebra_stripe_transform_frame_ip_YUY2
gst_zebra_stripe_transform_frame_ip_AYUV
all above 3 functions do the same functionality except for offset and pixel stride.
Hence moving the functionality to a single funtion.
https://bugzilla.gnome.org/show_bug.cgi?id=735032
Do more elaborate validation of the input caps: what fields
are required and/or not allowed. Don't assume AVC3 format
input without codec_data field is byte-stream format. Fix
up some now-unreachable code (CID 1232800).
It should try to use bytestream in these cases that the format
is set to _FORMAT_NONE as it seems that is what the 'else' clause
for bytestream can handle (by defaulting to _FORMAT_BYTESTREAM).
gst_pad_get_pad_template_caps() returns a reference which is unreferenced,
so creating a copy using gst_caps_copy() results in a reference leak.
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=734528
32 bit integers are going to overflow, especially the PCR offset to
the first PCR will overflow after about 159 seconds. This makes playback
of streams stop at 159 seconds as suddenly the timestamps are starting
again from 0. Now we have a few more years time until it happens again
and 64 bits are too small.
They are kept until the probes are removed but they will never be
removed as the refcount of the element won't get to 0 because the
probes own references (cyclic refs). As the probes should only be
running as long as the element is running there is no need to
secure a ref for them.
Removes 3 leaked refs of wrappercamerabinsrc
Use the sticky events to compose the streamheader as they are the
ones that are persisted to config new pads linked. Instead of storing
them ourselves rely on the pad storage that already orders it for us
https://bugzilla.gnome.org/show_bug.cgi?id=732596
The notify signal is triggered when caps is changed. But instead of
proxying the fixed caps, we query for the caps. Hence, when we go to
READY state, we endup setting template caps on the proxied caps
filter instead of NULL, which leads to negoitation failure. Correctly
proxy NULL caps if this is the new caps. Fixes not negotiated error
when running in cheese. Also fix a leak of caps string in one of the
trace.
https://bugzilla.gnome.org/show_bug.cgi?id=732741
We can still get OOB events while stopping the watchdog element, and while
stopping it we destroy the main context.
Also let the GSource own a reference to the element for additional safety.
https://bugzilla.gnome.org/show_bug.cgi?id=732554
Always use a GstAdapter when collecting access units (alignment="au")
in either byte-stream or avcC format. This is required to properly
preserve config headers like SPS and PPS when invalid or broken NAL
units are subsequently parsed.
More precisely, this fixes scenario like:
<SPS> <PPS> <invalid-NAL> <slice>
where we used to reset the output frame buffer when an invalid or
broken NAL is parsed, i.e. SPS and PPS NAL units were lost, thus
preventing the next slice unit to be decoded, should this also
represent any valid data.
https://bugzilla.gnome.org/show_bug.cgi?id=732203
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Carefully track cases when skipping broken or invalid NAL units is
necessary. In particular, always allow NAL units to be processed
and let that gst_h264_parse_process_nal() function decide on whether
the current NAL needs to be dropped or not.
This fixes parsing of streams with SEI NAL buffering_period() message
inserted between SPS and PPS, or SPS-Ext NAL following a traditional
SPS NAL unit, among other cases too.
Practical examples from the H.264 AVC conformance suite include
alphaconformanceG, CVSE2_Sony_B, CVSE3_Sony_H, CVSEFDFT3_Sony_E
when parsing in stream-format=byte-stream,alignment=au mode.
https://bugzilla.gnome.org/show_bug.cgi?id=732203
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Improve parser state tracking by introducing new flags reflecting
it: "got-sps", "got-pps" and "got-slice". This is an addition for
robustness purposes.
Older have_sps and have_pps variables are kept because they have
a different meaning. i.e. they are used for deciding on when to
submit updated caps or not, and rather mean "have new SPS/PPS to
be submitted?"
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Use gst_h264_parser_identify_nalu_unchecked() to identify the next
NAL unit. We don't want to parse the full NAL unit, but only the
header bytes and possibly the first RBSP byte for identifying the
first_mb_in_slice syntax element.
Also fix check for failure when returning from that function. The
only success condition for that is GST_H264_PARSER_OK, so use it.
https://bugzilla.gnome.org/show_bug.cgi?id=732154
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
The gst_h264_parse_pps() function dynamically allocates the slice
group ids map array, so that needs to be cleared before parsing a
new PPS NAL unit again, or when it is no longer needed.
Likewise, a clean copy to the internal NAL parser state needs to be
performed so that to avoid a double-free corruption.
https://bugzilla.gnome.org/show_bug.cgi?id=707282
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
The recovery point SEI message helps a decoder in determining if the
decoding process would produce acceptable pictures for display after
the decoder initiates random access or after the encoder indicates
a broken link in the coded video sequence.
This is not used in the h264parse element, but it could help debugging.
https://bugzilla.gnome.org/show_bug.cgi?id=723380
It was previously a mix and match of both variants, introducing just too much
confusion.
The prefix are from now on:
* GstMpegts for structures and type names (and not GstMpegTs)
* gst_mpegts_ for functions (and not gst_mpeg_ts_)
* GST_MPEGTS_ for enums/flags (and not GST_MPEG_TS_)
* GST_TYPE_MPEGTS_ for types (and not GST_TYPE_MPEG_TS_)
The rationale for chosing that is:
* the namespace is shorter/direct (it's mpegts, not mpeg_ts nor mpeg-ts)
* the namespace is one word under Gst
* it's shorter (yah)
In ISO/IEC 14496-15, the minimum size of a HEVCDecoderConfigurationRecord
(i.e., the contents of a hvcC box) is 23 bytes. However, the code in h265parse
checks that the size of this data is not less than 28 bytes, and it refuses to
accept caps if the check fails. The result is that standards-conformant streams
that don't carry any parameter sets in their hvcC boxes won't play.
https://bugzilla.gnome.org//show_bug.cgi?id=731783
When wrapover/reset occur, we end up with a small window of time where
the PTS/DTS will still be using the previous/next time-range.
In order not to return bogus values, return GST_CLOCK_TIME_NONE if the
PTS/DTS value to convert differs by more than 15s against the last seen
PCR
https://bugzilla.gnome.org/show_bug.cgi?id=674536
Using 32bit unsigned values for corrected pcr/offset meant that we
potentially ended up in bogus values
Furthermore, refpcr - refpcroffset could end up being negative, which
PCRTIME_TO_GSTTIME() can't handle (and returned a massive positive value)
Co-Authored by: Thibault Saunier <tsaunier@gnome.org>
From a high level perspective, the new process for seeking h264
streams is as follows:
1) Rewind the stream until we find the first I-slice of a frame,
and mark its offset in the stream.
2) Rewind the stream until we find SPS and PPS informations,
to make sure the subsequent parser is up to date.
3) Accumulate optionnal SEI NAL units on the way.
4) Push the SPS, PPS and SEI units before the new keyframe.
https://bugzilla.gnome.org/show_bug.cgi?id=675132
This should always be set for valid files when we get there,
and checking this avoids having ad hoc checks further down
in several places.
Coverity 1139698
If _set_current_pcr_offset gets called after a flushing seek, we ended
up using the current group for delta calculation ... whereas we should
be using the first group to calculate shifts.
Also add an early exit if there are no changes to apply
When working in push mode, we need to be able to evaluate the duration
based on a single group of observations.
To do that we use the current group values
When handling the PTS/DTS conversion in new groups, there's a possibility
that the PTS might be smaller than the first PCR value observed, due to
re-ordering.
When using the current group, only apply the wraparound correction when we
are certain it is one (i.e. differs by more than a second) and not when it's
just a small difference (like out-of-order PTS).
https://bugzilla.gnome.org/show_bug.cgi?id=731088
When we receive sticky events from upstream, always return TRUE.
Fixes the issue where we receive custom sticky events (such as "uri")
and no pads are created yet.
Since all the other timestamp tracking now gets reset on a discont,
it makes sense to wait for a PCR and timestamp buffers like when
playback first starts
(same as 744c58d71b but for the
position query)
It was only querying in time, but then trying to use dead bytes
to time conversion code.
Coverity 1139677
(Identical to commit 612cdeec80 which
was for resindvd)
When we'd see an unknown stream type, then a SDDS stream.
Then we'd get to the end of the switch with a NULL temp stream
pointer, and dereference it.
Coverity 1139708
Due to mpegts streaming nature some pads are created but are only added
later to the element. This can cause a scenario where the first stream
doesn't have an available decoder (while the next ones still pending
would have) and tsdemux will fail with not-linked as the first stream
added wouldn't be linked.
To avoid this tsdemux needs to add pads to the flowcombiner
when they are created instead of only when adding them to the
element.
