Sometimes rawparse does not handle the seeking query
properly, the rawparse should send the query upstream
first. For example, upstream could support seeking in
TIME format (but not in BYTE format), so the BYTE format
seeking query that rawparse sends in push mode would
fail.
https://bugzilla.gnome.org/show_bug.cgi?id=722764
Read PNG data chunk in one go by letting the parser
base class know the size we need, so that it doesn't
drip-feed us small chunks of data (causing a lot of
reallocs and memcpy in the process) until we have
everything.
Improves parsing performance of very large PNG files
(65MB) from ~13 seconds to a couple of millisecs.
https://bugzilla.gnome.org/show_bug.cgi?id=736176
This commit add an helper to convert a frame to frame-layer format and
use it to implement these two stream-format conversion:
- asf --> sequence-layer-frame-layer
- asf --> frame-layer
In simple/main profile, we basically have a raw frame, so building a
frame layer isn't too complicated. But in advanced profile, the first
frame-layer should contain sequence-header, entrypoint, and frame and
each keyframe should contain entrypoint, so we have to handle these
carefully.
https://bugzilla.gnome.org/show_bug.cgi?id=738526
Add an helper to check that output stream-format is coherent with
profile and header-format. It also check if we know how to do the
conversion if the input stream-format differs from selected
output-format.
So, in case output stream-format is not allowed, it will now fail at
negotiation rather than in pre_push_frame.
https://bugzilla.gnome.org/show_bug.cgi?id=738526
This commit introduces an helper to convert an ASF frame to BDUs format with
startcodes and use this helper to implements following stream-format
conversions:
- asf --> bdu
- asf --> sequence-layer-bdu
- asf --> sequence-layer-raw-frame
https://bugzilla.gnome.org/show_bug.cgi?id=738526
It add the support of following stream-format conversion:
- bdu --> sequence-layer-bdu
- bdu-frame --> sequence-layer-bdu-frame
- frame-layer --> sequence-layer-frame-layer
For these conversion, the only requirements is to push a sequence-layer
buffer prior to data.
https://bugzilla.gnome.org/show_bug.cgi?id=738526
It prepares the template for stream-format conversion and it implements
the following conversion:
- sequence-layer-bdu --> bdu
- sequence-layer-bdu-frame --> bdu-frame
- sequence-layer-frame-layer --> frame-layer
Work is done in the pre_push_frame() method.
https://bugzilla.gnome.org/show_bug.cgi?id=738526
gstinteraudiosrc.c: In function 'gst_inter_audio_src_create':
gstinteraudiosrc.c:339:27: error: variable 'buffer_samples' set but not used [-Werror=unused-but-set-variable]
guint64 period_samples, buffer_samples;
^
The whole not_linked optimisation is really a bit dodgy here, but
let's leave it in place for now and at least start pushing data
again when a pad got linked later, in which case we should get a
RECONFIGURE event.
Current CLAMP checks both if the value is below 0 or above 255. Considering it
is an unsigned value it can never be less than zero, so that comparison is
unnecessary. Switching to using if just for the upper bound.
CID #1139796
Value from left_luminance is assigned to out_luminance here, but that stored
value is not used before it is overwritten in the next cycle of the loop.
Removing assignation.
CID #1226473
As a consequence, tsdemux won't remove its pads anymore on EOS.
Fixes the case when mpegtsbase is not able to process new packets
after EOS as the corresponding pids aren't known anymore because
the programs were removed and the pes/psi were kept, preventing the
PAT to be parsed again.
https://bugzilla.gnome.org/show_bug.cgi?id=738695
It was using a 24000/24000/48000, but I think it meant to use
24000/32000/48000. Not 100% sure...
https://en.wikipedia.org/wiki/G.722.1 has the list of supported
bitrates. It's not clear whether the "flag" code maps to this,
however.
Coverity 206072
This parses the frame_packing_arragement() payload in SEI message.
This information can be used by decoders to appropriately rearrange the
samples which belong to Stereoscopic and Multiview High profiles.
https://bugzilla.gnome.org/show_bug.cgi?id=685215
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Assume that small backward PCR jumps are just from upstream packet
mis-ordering and don't reset timestamp tracking state - assuming that
things will be OK again shortly.
Make the threshold for detecting discont between sequential buffers
configurable and match the smoothing-latency setting on tsparse
to better cope with data bursts.
When the set-timestamps property is set, use PCRs on the provided
(or autodetected) pcr-pid to apply (or replace) timestamps on the
output buffers, using piece-wise linear interpolation.
This allows tsparse to be used to stream an arbitrary mpeg-ts file,
or to smooth jittery reception timestamps from a network stream.
The reported latency is increased to match the smoothing latency if
necessary.
Otherwise a magic capsfilter after the source is required with
exactly the same caps as the input.
This would've failed before with invalid buffer sizes:
gst-launch-1.0 videotestsrc ! intervideosink intervideosrc ! "video/x-raw,width=640,height=480" ! xvimagesink
Audiomixer blocksize, cant be 0, hence adjusting the minimum value to 1
timeout value of aggregator is defined with MAX of MAXINT64,
but it cannot cross G_MAXLONG * GST_SECOND - 1
Hence changed the max value of the same
https://bugzilla.gnome.org/show_bug.cgi?id=738845
Signal sparse streams properly in stream-start event and force sending
of pending sticky events which have been stored on the pad already and
which otherwise would only be sent on the first buffer or serialized
event (which means very late in case of subtitle streams). Playsink in
playbin waits for stream-start or another serialized event, and if we
don't do this it will wait for the multiqueue to run full before
starting playback, which might take a couple of seconds.
https://bugzilla.gnome.org/show_bug.cgi?id=734040
All pads of a stream are now added at the beginning. In order to cope with
streams that don't get any data (forever or for a long time) we detect gaps
and push out GAP events when needed.
Cleanups and commenting by Jan Schmidt <jan@centricular.com>
https://bugzilla.gnome.org/show_bug.cgi?id=734040
Some VC1 decoder can have different caps according to wmv format, ie
WMV3 or WVC1.
So instead of keeping the first available caps, we interserct with
current WMV format.
https://bugzilla.gnome.org/show_bug.cgi?id=738532
When stream-format is ASF or sequence-layer-raw-frame, we basically have
a raw frame so we can parse it to extract some information such the
keyframe flag. The only requirement is to have a valid sequence-header.
This commit parse the frame header and set the DELTA_UNIT buffer flag in
case the frame is not a keyframe.
https://bugzilla.gnome.org/show_bug.cgi?id=738519
frame-layer header is represented as a sequence of 32 bit unsigned
integer serialized in little-endian byte order, so framesize is on the
first 3 bytes.
SMPTE 421M Annex L.
https://bugzilla.gnome.org/show_bug.cgi?id=738243
Also, strictly speaking, these numbers aren't DLT_*; they are LINKTYPE_* because
libpcap translates from internal OS-specific DLT_ numbering to the portable
LINKTYPE_ number space when writing files.
https://bugzilla.gnome.org/show_bug.cgi?id=738206
GST_BASE_PARSE_FRAME_FLAG_PARSING value is wrong, and the same flag is
not being used presently. Hence changing the value and commenting it out.
This needs to be included in baseparse.h later on
https://bugzilla.gnome.org/show_bug.cgi?id=737411
There are unnecessary definitions for disabling deprecation warnings.
Since GLIB_DISABLE_DEPRECATION_WARNINGS is not needed anymore in these files,
removing the same.
https://bugzilla.gnome.org/show_bug.cgi?id=737559
If a discontinuity in the stream is detected, data is discarded until
a new PES starts. If the first packet after the discontinuity is also
the start of a PES, there is no reason to discard the packets.
https://bugzilla.gnome.org/show_bug.cgi?id=737569
Or not negotiated in the case we would be actually not negotiated
Currently we are getting assertions from
gst_pb_utils_add_codec_description_to_tag_list because of NULL
caps.
https://bugzilla.gnome.org/show_bug.cgi?id=737186
If we don't have a seq_layer_buffer, we also don't have a valid
seq_layer because there are set together in
gst_vc1_parse_handle_seq_layer().
So when output header format is sequence-layer and when we don't have a
seq_layer_buffer, we forge one from seq_hdr.
https://bugzilla.gnome.org/show_bug.cgi?id=736781
Sequence-layer and frame-layer are serialized in little-endian byte
order except for STRUCT_C and framedata fields as described in SMPTE 421M Annex
L.
https://bugzilla.gnome.org/show_bug.cgi?id=736750
packetized mode is being set when framerate is being set
which is not correct. Changing the same by checking the
input segement format. If input segment is in TIME it is
Packetized, and if it is in BYTES it is not.
https://bugzilla.gnome.org/show_bug.cgi?id=736252
This commit fix several issues with sequence layer header forging on
update_caps():
- 0x00000004 unsigned integer is before STRUCT_C.
- Set reserved bits of STRUCT_C to their values for simple/main
profiles in sequence layer header format and ASF header format.
- Sequence layer shall be represented as a sequence of 32 bits unsigned
integers and shall be serialized in little-endian byte order except
for STRUCT_C which shall be serialized in big-endian byte-order.
See SMPTE 421M Annex L for more details about sequence layer format.
https://bugzilla.gnome.org/show_bug.cgi?id=736474
packet_length is defined as a guint16 in the PESHeader structure. This
definition match the specification. But since we add 6 bytes to the
packet_length value (length of start_code + stream_id + packet_length),
we can overflow the guint16 when the value in the PES header is greater
than 65529.
So use a guint32 instead of a guint16 to avoid overflow.
https://bugzilla.gnome.org/show_bug.cgi?id=736490
In gst_data_uri_src_create(), buf cannot be NULL, hence
else if (*buf != NULL) will be invalid so removing the
else if condition and adding a check to unreference buf
in else condition, just in case
https://bugzilla.gnome.org/show_bug.cgi?id=735861
gst_zebra_stripe_transform_frame_ip_planarY
gst_zebra_stripe_transform_frame_ip_YUY2
gst_zebra_stripe_transform_frame_ip_AYUV
all above 3 functions do the same functionality except for offset and pixel stride.
Hence moving the functionality to a single funtion.
https://bugzilla.gnome.org/show_bug.cgi?id=735032
Do more elaborate validation of the input caps: what fields
are required and/or not allowed. Don't assume AVC3 format
input without codec_data field is byte-stream format. Fix
up some now-unreachable code (CID 1232800).
It should try to use bytestream in these cases that the format
is set to _FORMAT_NONE as it seems that is what the 'else' clause
for bytestream can handle (by defaulting to _FORMAT_BYTESTREAM).
gst_pad_get_pad_template_caps() returns a reference which is unreferenced,
so creating a copy using gst_caps_copy() results in a reference leak.
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=734528
32 bit integers are going to overflow, especially the PCR offset to
the first PCR will overflow after about 159 seconds. This makes playback
of streams stop at 159 seconds as suddenly the timestamps are starting
again from 0. Now we have a few more years time until it happens again
and 64 bits are too small.
They are kept until the probes are removed but they will never be
removed as the refcount of the element won't get to 0 because the
probes own references (cyclic refs). As the probes should only be
running as long as the element is running there is no need to
secure a ref for them.
Removes 3 leaked refs of wrappercamerabinsrc
Use the sticky events to compose the streamheader as they are the
ones that are persisted to config new pads linked. Instead of storing
them ourselves rely on the pad storage that already orders it for us
https://bugzilla.gnome.org/show_bug.cgi?id=732596
The notify signal is triggered when caps is changed. But instead of
proxying the fixed caps, we query for the caps. Hence, when we go to
READY state, we endup setting template caps on the proxied caps
filter instead of NULL, which leads to negoitation failure. Correctly
proxy NULL caps if this is the new caps. Fixes not negotiated error
when running in cheese. Also fix a leak of caps string in one of the
trace.
https://bugzilla.gnome.org/show_bug.cgi?id=732741
We can still get OOB events while stopping the watchdog element, and while
stopping it we destroy the main context.
Also let the GSource own a reference to the element for additional safety.
https://bugzilla.gnome.org/show_bug.cgi?id=732554
Always use a GstAdapter when collecting access units (alignment="au")
in either byte-stream or avcC format. This is required to properly
preserve config headers like SPS and PPS when invalid or broken NAL
units are subsequently parsed.
More precisely, this fixes scenario like:
<SPS> <PPS> <invalid-NAL> <slice>
where we used to reset the output frame buffer when an invalid or
broken NAL is parsed, i.e. SPS and PPS NAL units were lost, thus
preventing the next slice unit to be decoded, should this also
represent any valid data.
https://bugzilla.gnome.org/show_bug.cgi?id=732203
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Carefully track cases when skipping broken or invalid NAL units is
necessary. In particular, always allow NAL units to be processed
and let that gst_h264_parse_process_nal() function decide on whether
the current NAL needs to be dropped or not.
This fixes parsing of streams with SEI NAL buffering_period() message
inserted between SPS and PPS, or SPS-Ext NAL following a traditional
SPS NAL unit, among other cases too.
Practical examples from the H.264 AVC conformance suite include
alphaconformanceG, CVSE2_Sony_B, CVSE3_Sony_H, CVSEFDFT3_Sony_E
when parsing in stream-format=byte-stream,alignment=au mode.
https://bugzilla.gnome.org/show_bug.cgi?id=732203
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Improve parser state tracking by introducing new flags reflecting
it: "got-sps", "got-pps" and "got-slice". This is an addition for
robustness purposes.
Older have_sps and have_pps variables are kept because they have
a different meaning. i.e. they are used for deciding on when to
submit updated caps or not, and rather mean "have new SPS/PPS to
be submitted?"
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Use gst_h264_parser_identify_nalu_unchecked() to identify the next
NAL unit. We don't want to parse the full NAL unit, but only the
header bytes and possibly the first RBSP byte for identifying the
first_mb_in_slice syntax element.
Also fix check for failure when returning from that function. The
only success condition for that is GST_H264_PARSER_OK, so use it.
https://bugzilla.gnome.org/show_bug.cgi?id=732154
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
The gst_h264_parse_pps() function dynamically allocates the slice
group ids map array, so that needs to be cleared before parsing a
new PPS NAL unit again, or when it is no longer needed.
Likewise, a clean copy to the internal NAL parser state needs to be
performed so that to avoid a double-free corruption.
https://bugzilla.gnome.org/show_bug.cgi?id=707282
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
The recovery point SEI message helps a decoder in determining if the
decoding process would produce acceptable pictures for display after
the decoder initiates random access or after the encoder indicates
a broken link in the coded video sequence.
This is not used in the h264parse element, but it could help debugging.
https://bugzilla.gnome.org/show_bug.cgi?id=723380
It was previously a mix and match of both variants, introducing just too much
confusion.
The prefix are from now on:
* GstMpegts for structures and type names (and not GstMpegTs)
* gst_mpegts_ for functions (and not gst_mpeg_ts_)
* GST_MPEGTS_ for enums/flags (and not GST_MPEG_TS_)
* GST_TYPE_MPEGTS_ for types (and not GST_TYPE_MPEG_TS_)
The rationale for chosing that is:
* the namespace is shorter/direct (it's mpegts, not mpeg_ts nor mpeg-ts)
* the namespace is one word under Gst
* it's shorter (yah)
In ISO/IEC 14496-15, the minimum size of a HEVCDecoderConfigurationRecord
(i.e., the contents of a hvcC box) is 23 bytes. However, the code in h265parse
checks that the size of this data is not less than 28 bytes, and it refuses to
accept caps if the check fails. The result is that standards-conformant streams
that don't carry any parameter sets in their hvcC boxes won't play.
https://bugzilla.gnome.org//show_bug.cgi?id=731783
When wrapover/reset occur, we end up with a small window of time where
the PTS/DTS will still be using the previous/next time-range.
In order not to return bogus values, return GST_CLOCK_TIME_NONE if the
PTS/DTS value to convert differs by more than 15s against the last seen
PCR
https://bugzilla.gnome.org/show_bug.cgi?id=674536
Using 32bit unsigned values for corrected pcr/offset meant that we
potentially ended up in bogus values
Furthermore, refpcr - refpcroffset could end up being negative, which
PCRTIME_TO_GSTTIME() can't handle (and returned a massive positive value)
Co-Authored by: Thibault Saunier <tsaunier@gnome.org>
From a high level perspective, the new process for seeking h264
streams is as follows:
1) Rewind the stream until we find the first I-slice of a frame,
and mark its offset in the stream.
2) Rewind the stream until we find SPS and PPS informations,
to make sure the subsequent parser is up to date.
3) Accumulate optionnal SEI NAL units on the way.
4) Push the SPS, PPS and SEI units before the new keyframe.
https://bugzilla.gnome.org/show_bug.cgi?id=675132
This should always be set for valid files when we get there,
and checking this avoids having ad hoc checks further down
in several places.
Coverity 1139698
If _set_current_pcr_offset gets called after a flushing seek, we ended
up using the current group for delta calculation ... whereas we should
be using the first group to calculate shifts.
Also add an early exit if there are no changes to apply
When working in push mode, we need to be able to evaluate the duration
based on a single group of observations.
To do that we use the current group values
When handling the PTS/DTS conversion in new groups, there's a possibility
that the PTS might be smaller than the first PCR value observed, due to
re-ordering.
When using the current group, only apply the wraparound correction when we
are certain it is one (i.e. differs by more than a second) and not when it's
just a small difference (like out-of-order PTS).
https://bugzilla.gnome.org/show_bug.cgi?id=731088
When we receive sticky events from upstream, always return TRUE.
Fixes the issue where we receive custom sticky events (such as "uri")
and no pads are created yet.
Since all the other timestamp tracking now gets reset on a discont,
it makes sense to wait for a PCR and timestamp buffers like when
playback first starts
(same as 744c58d71b but for the
position query)
It was only querying in time, but then trying to use dead bytes
to time conversion code.
Coverity 1139677
(Identical to commit 612cdeec80 which
was for resindvd)
When we'd see an unknown stream type, then a SDDS stream.
Then we'd get to the end of the switch with a NULL temp stream
pointer, and dereference it.
Coverity 1139708
Due to mpegts streaming nature some pads are created but are only added
later to the element. This can cause a scenario where the first stream
doesn't have an available decoder (while the next ones still pending
would have) and tsdemux will fail with not-linked as the first stream
added wouldn't be linked.
To avoid this tsdemux needs to add pads to the flowcombiner
when they are created instead of only when adding them to the
element.
gstfreeverb.c:781:29: error: using integer absolute value function 'abs' when
argument is of floating point type [-Werror,-Wabsolute-value]
if (abs (out_l2) > 0 || abs (out_r2) > 0)
They are very confusing for people, and more often than not
also just not very accurate. Seeing 'last reviewed: 2005' in
your docs is not very confidence-inspiring. Let's just remove
those comments.
This should not harm regular files, since those are the last 4 bytes of
a normal file.
This allows to handle playback of concatenated mpeg-ps files. Seeking and
duration reporting is still wrong though.
Testing mpegversion when mpegaudioversion was likely meant.
Similar tests in sys/androidmedia/gstamcaudiodec.c also test
mpegaudioversion with the same conditional code.
Coverity 206071
This component is dereferenced, and later code checking for
NULL in particular cases implies it can be NULL. This likely
does not fix the coverity warning as it was seeing another
path setting component to NULL explicitely, but this was
spotted by looking at:
Coverity 1139736
Which is actually OK from what I can see since the actual
dereference of the explicit NULL pointer will not happen
if the condition that led to the NULL pointer assignment
is met, since the assignment and defeference have mutually
exclusive tests.
Detect resolution changes on key frames, and propagate the resulting
caps to the src pad. Only the uncompressed data chunk is decoded, so
avoid using the new VP8 bitstream parsing library for now.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Avoid possible division-by-zero while deriving the presentation timestamp
of the buffer. The base class will take care of any interpolation needs.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
* Search in current pending values first. For CBR streams we can very
easily end up having just one initial observations and then nothing
else (since the bitrate doesn't change).
* Use one group whether we are in that group *OR* if there is only
one group.
* If the group to use isn't closed (points are being accumulated in the
PCROffsetCurrent), use the latest data available for calculation
* If in the unlikelyness that all of this *still* didn't produce more
than one data point, just return the initial offset
While the calculation done in these macros will work with 64bit
integers, they will fail if working with 32bit integers.
Force the scaling up to solve that.
This amazingly didn't introduce major issues up to now, but resulted
in bogus values in debug logs.
Doing a hard flush on the packetizer will drop all observations, which
will eventually break push-based seeking (with BYTES segment) since
we won't know where to seek to anymore (new data would always be
considered as the beginning of the stream).
Turns out glib aborts on allocation failure, so this is pointless.
We'll just ignore Coverity warnings on such constructs.
This reverts commit d347809a82.
While this probably should never happen if callers are well behaved,
this avoids a crash if it does. With a warning about it. Unsure if
it'd be better to not add at all, but it should not happen...
Coverity 1139713
While it will probably not trigger, it should silence a Coverity
warning about the fail code path testing for NULLness before
freeing, where the buffer was already dereferenced. It seems
safest to keep that test, in case future goto fail statements
happen to have a NULL buffer there.
Coverity 1139851
Fixes multiple seeking issues. When doing ACCURATE or normal
non-KEYUNIT seeks, mxfdemux would just send data from the
edit unit that covered the seek position, whether that's
a keyframe or not. Decoders would only output things from
the next keyframe then, which means there's a gap between
the start of the segment and the first decoded data in
some cases. In combination with gst-editing-services this
might result in a frozen picture for the duration of that
gap at the beginning (if videorate fixes up the first
buffer's start timestamp to cover the entire gap), or
a black frame (if no videorate is used and videomixer
fills the gap). Also fixes A/V sync issue when requesting
a KEYUNIT seek.
gst_ts_demux_push_pending_data() will check if it now can activate the
stream and add the pad, we don't have to check that ourselves.
Fixes playback of very short MPEG TS files.
We create our textures (in Desktop GL) with GL_TEXTURE_RECTANGLE,
vaapi attempts to bind our texture to GL_TEXTURE_2D which throws a
GL_INVALID_OPERATION error and as thus, no video.
Also, by moving exclusively to GL_TEXTURE_2D and the npot extension
we also remove a difference between the Desktop GL and GLES2 code.
https://bugzilla.gnome.org/show_bug.cgi?id=712287
Simplify gst_glimage_sink_redisplay which is there
only to ask the window for a redraw.
Put a lock to make sure we are not realeasing
the stored buffer while still drawing the corresponding
texture
Write forward declarations in another way to avoid
repeated typedefs "error: redefinition of typedef".
Raised when using i686-apple-darwin11-llvm-gcc-4.2
It seems that C apple compiler does not support
C11 feature.
https://bugzilla.gnome.org/show_bug.cgi?id=703885
- Make GstGLWindow subclassablerather than specified at compile time.
- Add GstGLWindowX11 for x11 windows and two subclasses, GstGLWindowX11GLX
and GstGLWindwX11EGL for GLX and EGL repectively. (win32 and cocoa
ports to come)
- Also cleanup GL library detection in configure.ac
Just use g_file_get_contents() instead of home-made file loading.
Fixes two issues - one is that we should pass "r" to fopen and
not O_RDONLY, the other is that an incorrect variable was used
to read the file length, leading to an empty shader file.
Spotted by: Wang Xin-yu (王昕宇) <comicfans44@gmail.com>
https://bugzilla.gnome.org/show_bug.cgi?id=702844https://bugzilla.gnome.org/show_bug.cgi?id=702845
Conflicts:
gst/gl/gstglfiltershader.c
'if statement has empty body', which were real bugs and
'comparison of unsigned expression < 0 is always false', which was
only an unneeded comparison.
Settle with 7x7 gaussian convolution kernels, maybe slightly less
accurate than previous 9x9 but fast enough to be able to use it on i915.
About a 20% percent speed gain (again, roughly measured with
videotestsrc and glimagesink sync=false). No noticeable rendering
difference with current effects.
Get rid of buggy and complicated hls conversion code for the sin effect.
The only thing needed was hue anyway and it is easily calculated using
Preucil formula for rgb to polar coordinates conversion.
Now works on i915 (removed all the IF blocks). Still needs some tuning,
I wonder if it will ever work properly.
Add a new convenience function in GstGLFilter that just draws an input
texture to a target texture using a simple shader with just a "tex"
uniform sampler.
Move draw_texture from glfiltersobel to glfilter. Still need to update
other plugins to this.
Rework Sobel a little bit again making it work as the old one:
1. desaturate input texture
2. calculate horizontal convolution for x gradient and vertical
convolution for y gradient at the same time (halves the number of
needed texture lookups)
3. store results in a single texture (red and green channel)
4. calculate remaining convolution (same as above switching vertical and
horizontal)
5. calculate length of gradient using red and green as x and y
components.
Optimize wherever possible, store kernels as constants in the shaders,
remove unneeded uniforms. Restore invert property carefully avoiding
using IF.
Still not sure if "full color" convolution will be needed, glfiltersobel
is to be intended as a demo filter and xray, the only effect which uses
sobel only needs edge intensity. Dropping it for now.
Reimplement sobel in a multipass fully separated convolution:
- calculate x gradient map convolving first horizontally with blurring
kernel and then vertically with differentiating kernel
- calculate y gradient map convolving first vertically with blurring
kernel and then horizonally with differentiating kernel
- calculate length of the gradient vector
Particular care was needed with normalization of the blurring kernel and
with grey level offset of the differentiating one to prevent overflow of
rgb values from the [0.0,1.0] range in intermediate passes.
Now works on i915.
Thanks to Eric Anholt I've finally understood (at least I hope) how to
count texture indirections and save up some. Texture sampling dependent
on the result of some math counts as an indirection phase. Grouped
texture lookups with no math involved count as a single indirection.
Math on the coordinates count as indirection.
So the best thing is to group all the math involving coordinates and
then do all the lookups.
This saves enough indirections to make glfilterblur and glow effect
work, albeit a bit slowly, on i915.
Remove unused uniforms from the laplacian filter. Also remove if
kernel[i] != 0 checks so that it compiles where IF is not available.
Again, big thanks to Eric Anholt for the hints.
Apparently assigning gl_TexCoord to a temp count as an indirection.
Using it directly avoids it and limits indirections to four not
exceeding i915 limit. Now xpro effect works on i915.
Get rid of polar coordinates in the twirl effect. The same can be done
using a rotation matrix, saving alu instructions and, most of all,
avoiding the use of the evil atan() function (which uses IF operators).
Calculate rotation angle in a saner, understandable way.
Works on i915! (Hope it still works elsewhere too as I'm not able to
test at the moment)
Get rid of polar coordinates in the tunnel effect as the same can easily
be done just clamping the radius and multiplying.
Remove the evil atan() call that uses branching and a lot of unneeded alu
instructions. Now works on i915!
Generate a normalized gaussian kernel with given size and standard
deviation on the fly.
Remove "norm_const" uniform from convolution shaders and provide a
normalized kernel instead. Remove norm_offset uniform as it was always
zero, will reintroduce it if really needed in the future. Thanks to Eric
Anholt for suggesting it.
Save some ALU instruction calculating directly the coordinate for
texture lookup instead of summing an offset.
Still exceed maximum indirect texture lookups on i915, the only solution
I see is using a 3x3 kernel.
Reduce the number of register calculating texture lookup offset on the
fly. It was just a simple sequence, no need to store it in a array.
Fixes maximum number of registers exceeded error with i915. Still
exceed maximum indirect texture lookups and maximum ALU instructions.
Maybe we should gave up some blur goodness and use lightly more little
kernels.
Apparently saving up some texture lookup for zero kernel elements is
definitely not worth the use of branching. This way convolution
fragment programs also work where IF operator is not supported (tested
on i915 and nouveau). See also discussion on bug #615696.
Thanks to Eric Anholt for spotting this.
Port blur filter to use the common convolution shaders in
gstgleffectssources.c. This reduces code duplication and, incidentally,
the shaders in the common file were already updated to not use array
constructor and to not depend on #version 120.
First step towards bug #615696 fixing.
Fix some crazy formatting caused by gst-indent previous runs and disable
the script for this file. The best would be to move shaders into
separate files and load them at runtime or hardcode them at compile
time.
For now only identity, mirror and squeeze effects are available.
Maybe some factorization is needed about compilation shader
before to put the other effects since only a copy/past is needed,
at least until effect number 9: heat.
The effects from 10:sepia to 15:glow require more work.
qglwtextureshare now works again. In this example,
the pipeline is src ! glupload ! fakesink.
So in this case the glupload element is a sink in
terms of gl chain.
But the problem is still there if the pipeline is
src ! glupload ! glfilter ! fakesink
(it's the case in sdlshare and cluttershare examples)
because since recent changes about how the gstgldisplay
is transmitted to the gl element, the context is usually
created by the sink in terms of gl chain.
A solution would be to also install this property on glfilter.
The background image needs to be scaled to fit current texture size.
Previously this was done by gdk_pixbuf_scale_simple but that's been
removed.
Create a texture from the background pixbuf with correct dimensions and
use interpolation shader to scale it to the right size. Interpolation
fragment shader doesn't have too much sense if all the textures don't
have the same size so this seemed the most natural place to do the
scaling. It could probably be done with some custom texture mapping
outside the shader but it involved more code.
Fixes bug #599883.
glmixer can be seen as a glfilter except it handles N requested
sink pads.
Each sink pad and the src pad are video/x-raw-gl.
glmixer is responsible for managing different framerates from inputs.
It uses OpenGL context sharing. It means that each input is in its
own OpenGL context shared together and shared with the OpenGL context
of the ouput gl chain.
Also add a glmosaic which is an example of implementation of glmixer.
For now glmosaic is a cube but it will be fixed in the next commits.
For now the glmixer has some weird behaviours in some configurations
but it will be improved in the next commits.
The autotools builds is temporarly broken since those changes
have been made on win32.
Before, a gstgldisplay was instancied by the gl src in terms of gl chain.
And then the next element got it through the first gstglbuffer.
Now, this is done though queries.
All glelements get their ref on a gstgldisplay in READY state.
This rewrite is mainly a first step to be able to share OpenGL context hold
by the gstgldisplay using more complex glelements.
For example, with a glvideomixer. The associated gstgldisplay of each gl chain
of the sink pads will share their OpenGL context.
Add a pkg-config check for opengl and if not found assume opengl-es. If user has
none of both one still get build error later on (there is no pkg-config for
opengl-es).
Add more files to EXTRA dist and build the opengles variant if selected.
Simmilar changes could be done for the winCE backend.
Greedyh operation implemented using OpenGL Shading Language.
We could add other operations later.
Does some good results but still not as expected.
That's why I do not add it yet to the build.
Fixes bug #584877
Before this commit calling "gst_x_overlay_set_xwindow_id" more
than one time, had no effect.
It mainly affects the glimagesink implementation.
But on win32 (and CE), some stuff has to be done to
release the old parent.
And add a switchxoverlay example where the user
can click on left/right part of the main window to
switch the xoverlay.
The external opengl context must be specify when creating
our OpenGL context (glx) or just after (wgl).
When calling glXCreateContext or wglShareLists, the
external opengl context must not be current.
Then our gl context can be current in the gl thread while
the external gl context is current in an other thread.
See tests/examples/clutter/cluttershare.c
In OpenGL 2.x for Embedded System, a lot of basic scene/draw functions
have been removed. It means that everything is made using vertex and
fragment shaders.
I have also added a gstglwindow backend for winCE that uses EGL
(Native Platform Graphics Intercace) (which is a full part of
OpenGL ES specification). It remove the use of wgl/glx functions.
