- update for shaders
- add alpha property
- image placement properties shamelessly borrowed from gdkpixbufoverlay
- image placement properties are GstController able
- use GstGLMemory for the overlay image data
- add support for gles2
Otherwise we could pass on a RGBA formatted buffer and downstream would
misinterpret that as some other video format.
Fixes pipelines of the form
gleffects ! tee ! xvimagesink
Allows callers to properly reference count the buffers used for
rendering.
Fixes a redraw race in glimagesink where the previous buffer
(the one used for redraw operations) is freed as soon as the next
buffer is uploaded.
1. glimagesink uploads in _prepare() to texture n
1.1 glupload holds buffer n
2. glimagesink _render()s texture n
3. glimagesink uploads texture n+1
3.1 glupload free previous buffer which deletes texture n
3.2 glupload holds buffer n+1
4. glwindow resize/expose
5. glimagesink redraws with texture n
The race is that the buffer n (the one used for redrawing) is freed as soon as
the buffer n+1 arrives. There could be any amount of time and number of
redraws between this event and when buffer n+1 is actually rendered and thus
replaces buffer n as the redraw source.
https://bugzilla.gnome.org/show_bug.cgi?id=736740
If EOS or ERROR happens before the download loop thread has reached its
g_cond_wait() call, then the g_cond_signal doesn't have any effect and
the download loop thread stucks later.
https://bugzilla.gnome.org/show_bug.cgi?id=735663
If EOS or ERROR happens before the download loop thread has reached its
g_cond_wait() call, then the g_cond_signal doesn't have any effect and
the download loop thread stucks later.
https://bugzilla.gnome.org/show_bug.cgi?id=735663
If EOS or ERROR happens before the download loop thread has reached its
g_cond_wait() call, then the g_cond_signal doesn't have any effect and
the download loop thread stucks later.
https://bugzilla.gnome.org/show_bug.cgi?id=735663
The internal pad still keeps its EOS flag and event as it can be assigned
after the flush-start/stop pair is sent. The EOS is assigned from the streaming
thread so this is racy.
To be sure to clear it, it has to be done after setting the source to READY to
be sure that its streaming thread isn't running.
https://bugzilla.gnome.org/show_bug.cgi?id=736012
The internal pad still keeps its EOS flag and event as it can be assigned
after the flush-start/stop pair is sent. The EOS is assigned from the streaming
thread so this is racy.
To be sure to clear it, it has to be done after setting the source to READY to
be sure that its streaming thread isn't running.
https://bugzilla.gnome.org/show_bug.cgi?id=736012
The internal pad still keeps its EOS flag and event as it can be assigned
after the flush-start/stop pair is sent. The EOS is assigned from the streaming
thread so this is racy.
To be sure to clear it, it has to be done after setting the source to READY to
be sure that its streaming thread isn't running.
https://bugzilla.gnome.org/show_bug.cgi?id=736012
packetized mode is being set when framerate is being set
which is not correct. Changing the same by checking the
input segement format. If input segment is in TIME it is
Packetized, and if it is in BYTES it is not.
https://bugzilla.gnome.org/show_bug.cgi?id=736252
Previously we only refetched the playlist if downloading a fragment
has failed once. We should also do that if it failed a second or third time,
chances are that the playlist was updated now and contains new URIs.
face detection will be performed only if image standard deviation is
greater that min-stddev. Default min-stddev is 0 for backward
compatibility. This property will avoid to perform face detection on
images with little changes improving cpu usage and reducing false
positives
https://bugzilla.gnome.org/show_bug.cgi?id=730510
* aspect should not be 0 on init
* rename fovy to fov
* add mvp to properties as boxed graphene type
* fix transformation order. scale first
* clear color with 1.0 alpha
https://bugzilla.gnome.org/show_bug.cgi?id=734223
If the language is not specified in the AdaptationSet, use the ContentComponent
node to get it. We only get it if there is only a single ContentComponent as
it doesn't seem clear on what to do if there are multiple entries
https://bugzilla.gnome.org/show_bug.cgi?id=732237
Dynamic pipelines that get and release the sink pads will finalize
the pad without going through gst_gl_mixer_stop() which is where the
upload object is usually freed. Don't leak objects in such case.
Instead always use the low bandwith playlist making things go smoother
as the current heuristic is rather set for normal playback, and
currently it does not behave properly.
https://bugzilla.gnome.org/show_bug.cgi?id=734445
When a seek with a negative rate is requested, find the target
segment where gstsegment.stop belongs in and then download from
this segment backwards until the first segment.
This allows proper reverse playback.
If window is resized, GstStructure pointer values have to be rescaled to
original geometry. A get_surface_dimensions GLWindow class method is added for
this purpose and used in the navigation send_event function.
https://bugzilla.gnome.org/show_bug.cgi?id=703486
When flushing, this will prevent dashdemux from trying to download more
fragments or more chunks of the same fragment before stopping.
Also improves the error handling to not transform everything non-ok into
an error.
https://bugzilla.gnome.org/show_bug.cgi?id=734014
templatematch operates on BGR data. In fact, OpenCV's IplImage always
stores color image data in BGR order -- this isn't documented at all in
the OpenCV source code, but there are hints around the web (see for
example
http://www.cs.iit.edu/~agam/cs512/lect-notes/opencv-intro/opencv-intro.html#SECTION00041000000000000000
and http://www.comp.leeds.ac.uk/vision/opencv/iplimage.html ).
gst_templatematch_load_template loads the template (the image to find)
from disk using OpenCV's cvLoadImage, so it is stored in an IplImage in
BGR order. But in gst_templatematch_chain, no OpenCV conversion
functions are used: the imageData pointer of the IplImage for the video
frame (the image to search in) is just set to point to the raw buffer
data. Without this fix, that raw data is in RGB order, so the call to
cvMatchTemplate ends up comparing the template's Blue channel against
the frame's Red channel, producing very poor results.
Previously changing the template property resulted in an exception
thrown from cvMatchTemplate, because "dist_image" (the intermediate
match-certainty-distribution) was the wrong size (because the
template image size had changed).
Locking has also been added to allow changing the properties (e.g. the
pattern to match) while the pipeline is playing.
* gst_element_post_message is moved outside of the lock, because it will
call into arbitrary user code (otherwise, if that user code calls into
gst_templatematch_set_property on this same thread it would deadlock).
* gst_template_match_load_template: If we fail to load the new template
we still unload the previous template, so this element becomes a no-op
in the pipeline. The alternative would be to keep the previous template;
I believe unloading the previous template is a better choice, because it
is consistent with the state this element would be in if it fails to
load the very first template at start-up.
Thanks to Will Manley for the bulk of this work; any errors are probably
mine.
The early return was bypassing the call to gst_pad_push. With no
filter->template (and thus no filter->cvTemplateImage) the rest of this
function is essentially a no-op (except for the call to gst_pad_push).
This (plus the previous commit) allows templatematch to be
enabled/disabled without removing it entirely from the pipeline, by
setting/unsetting the template property.
Delaying the segment event to when caps are decided can cause issues as
the first thing katedec does on its chain function it doing a segment clip.
It will lead to an assertion if the segment format is undefined
https://bugzilla.gnome.org/show_bug.cgi?id=733226
Properly handle the caps event by configuring the kate decoding lib using the
available streamheaders. This makes it possible to decode kate subtitles when
the stream is seeked before katedec gets the initial buffers that are usually
the streamheaders.
https://bugzilla.gnome.org/show_bug.cgi?id=733226
The headers were never getting reffed when being added to the headers
list, which is later unreffed-and-freed by the caller (e.g.
gst_opus_parse_parse_frame()).
https://bugzilla.gnome.org/show_bug.cgi?id=733013
The expected default behaviour for video sink is to maintain the
aspect ratio. Fix the default value to reflect this. The property
default was already TRUE, but the value was not initially TRUE.
First this is handle by base transform, hence this is a no-op, and if it wasn't it
would lead to a buffer copy being leaked, and then an unreffed buffer being
pushed downstream.
https://bugzilla.gnome.org/show_bug.cgi?id=732756
OpenNI2 makes no guarantees of timestamp starting from zero, just that
it will be a millisecond timestamp. Make timestamps start from zero
manually so things work correctly.
https://bugzilla.gnome.org/show_bug.cgi?id=732535
Allows automatic negotiation of the size in the following case:
gst-launch-1.0 glvideomixer name=m sink_0::xpos=0 sink_1::xpos=320 ! glimagesink \
videotestsrc ! m. \
videotestsrc pattern=1 ! m.
https://bugzilla.gnome.org/show_bug.cgi?id=731878
This is too allow gst-launch debugging with multiple GL contexts as
well as avoiding segfaulting innocent gtk+ apps that have not called
XInitThreads.
https://bugzilla.gnome.org/show_bug.cgi?id=731525
Only reset the decryption engine on the first buffer of a fragment,
not again for the second buffer. This fixes corrupting the second
buffer of a fragment.
https://bugzilla.gnome.org/show_bug.cgi?id=731968
gstwaylandsink.c:480:14: error: comparison of constant -1 with expression of
type 'enum wl_shm_format' is always false
[-Werror,-Wtautological-constant-out-of-range-compare]
if (format == -1)
~~~~~~ ^ ~~
This allows waylandsink to fail gracefully before going to READY
in case one of the required interfaces does not exist. Not all
interfaces are necessary for all modes of operation, but it is
better imho to fail before going to READY if at least one feature
is not supported, than to fail and/or crash at some later point.
In the future we may want to relax this restriction and allow certain
interfaces not to be present under certain circumstances, for example
if there is an alternative similar interface available (for instance,
xdg_shell instead of wl_shell), but for now let's require them all.
Weston supports them all, which is enough for us now. Other compositors
should really implement them if they don't already. I don't like the
idea of supporting many different compositors with different sets of
interfaces implemented. wl_subcompositor, wl_shm and wl_scaler are
really essential for having a nice video sink. Enough said.
This essentially hides the video and allows the application to
potentially draw a black background or whatever else it wants.
This allows to differentiate the "paused" and "stopped" modes
from the user's point of view.
Also reworded a comment there to make my thinking more clear,
since the "reason for keeping the display around" is not really
the exposed() calls, as there is no buffer shown in READY/NULL
anymore.
1) We know that gst_wayland_sink_render() will commit the surface
in the same thread a little later, as gst_wl_window_set_video_info()
is always called from there, so we can save the compositor from
some extra calculations.
2) We should not commit a resize with the new video info while we are still
showing the buffer of the previous video, with the old caps, as that
would probably be a visible resize glitch.
Previously, in order to change the surface size we had to let the pipeline
redraw it, which at first also involved re-negotiating caps, etc, so a
synchronization with the pipeline was absolutely necessary.
At the moment, we are using wl_viewport, which separates the surface size
from the buffer size and it also allows us to commit a surface resize without
attaching a new buffer, so it is enough to just do:
gst_wayland_video_pause_rendering():
wl_subsurface_set_sync()
gst_video_overlay_set_render_rectangle():
wl_subsurface_set_position()
wl_viewport_set_destination()
wl_surface_damage()
wl_surface_commit()
... commit the parent surface ...
gst_wayland_video_resume_rendering():
wl_subsurface_set_desync()
This is enough to synchronize a surface resize and the pipeline can continue
drawing independently. Now of course, the names pause/resume_rendering are
bad. I will rename them in another commit.
Access is protected only for setting/creating/destroying the display
handle. set_caps() for example is not protected because it cannot be
called before changing state to READY, at which point there will be
a display handle available and which cannot change by any thread at
that point
This is because:
* GST_ELEMENT_WARNING/ERROR do lock the OBJECT_LOCK and we deadlock instantly
* In future commits I want to make use of GstBaseSink functions that also
lock the OBJECT_LOCK inside this code
* own_surface is not needed anymore
* gst_wl_window_from_surface is not used externally anymore
* many initializations to 0 are not needed (GObject does them)
This means that the given surface in set_window_handle can now be
the window's top-level surface on top of which waylandsink creates
its own subsurface for rendering the video.
This has many advantages:
* We can maintain aspect ratio by overlaying the subsurface in
the center of the given area and fill the parent surface's area
black in case we need to draw borders (instead of adding another
subsurface inside the subsurface given from the application,
so, less subsurfaces)
* We can more easily support toolkits without subsurfaces (see gtk)
* We can get properly use gst_video_overlay_set_render_rectangle
as our api to set the video area size from the application and
therefore remove gst_wayland_video_set_surface_size.
This drops the ugly GstWaylandWindowHandle structure and is much
more elegant because we can now request the display separately
from the window handle. Therefore the window handle can be requested
in render(), i.e. when it is really needed and we can still open
the correct display for getting caps and creating the pool earlier.
This change also separates setting the wl_surface from setting its size.
Applications should do that by calling two functions in sequence:
gst_video_overlay_set_window_handle (overlay, surface);
gst_wayland_video_set_surface_size (overlay, w, h);
This is the only way to get the negotiation working with the dynamic
detection of formats from the display, because the pipeline needs
to know the supported formats in the READY state and the supported
formats can only be known if we open the display.
Unfortunately,in wayland we cannot have a separate connection to
the display from the rest of the application, so we need to ask for a
window handle when going to READY in order to get the display from it.
And since it's too early to create a top level window from the state
change to READY, create it in render() when there is no other window.
This also changes set_window_handle() to not support window handle
changes in PAUSED/PLAYING (because it's complex to handle and useless
in practice) and make sure that there is always a valid display pointer
around in the READY state.
This fixes weird freezes because of frame_redraw_callback() not being
called from the main thread when it should with weston's toy toolkit.
It's also safer to know that frame_redraw_callback() will always be
called from our display thread... Otherwise it could be called after
the sink has been destroyed for example.
We are not supposed to redraw until we receive a frame callback and this
is especially useful to avoid allocating too many buffers while the
window is not visible, because the compositor may not call wl_buffer.release
until the window becomes visible (ok, this is a wayland bug, but...).
This is achieved by adding an extra reference on the buffers, which does
not allow them to return to the pool. When they are released, this reference
is dropped.
The rest complexity of this patch (hash table, mutex, flag, explicit release calls)
merely exists to allow a safe, guaranteed and deadlock-free destruction sequence.
See the added comment on gstwaylandsink.c for details.
start() makes sure that the minimum ammount of buffers requested is allocated.
stop() makes sure that buffers are actually destroyed and prevents
filling the file system when resizing the surface a lot, because the
wayland-shm-* files will stay on the file system as long as the wl_buffers
created out of them are alive.
This is the initial implementation, without the GstVideoOverlay.expose()
method. It only implements using an external (sub)surface and resizing
it with GstWaylandVideo.
The reference to the sink is not really needed anyway in waylandpool,
what matters basically is that the display is active as long as the
pool is active, so we really want to reference the display object
instead of the sink.
* make use of GstBufferPool::start/stop functions to allocate/deallocate memory
* get rid of struct shm_pool and do all operations cleanly inside WaylandBufferPool
* store a GstVideoInfo during configuration instead of the width & height
and use the stride from the video info instead of hardcoding its value
The reshape property was never used.
Replace the draw property with a signal.
Based on patch by Mathieu Duponchelle <mathieu.duponchelle@epitech.eu>
https://bugzilla.gnome.org/show_bug.cgi?id=704507
This can happen if the playlists have moved due to the variant playlist
now being redirected to another target. This currently only works as long
as the referenced playlists don't change in relation to the variant
playlist, and the new location is purely due to a new path triggered by a
new redirection target of the variant playlist, or a new redirection
target of the playlist itself.
https://bugzilla.gnome.org/show_bug.cgi?id=731164
We add a new signal, get-rollover-counter, to the SRTP encoder. Given a
ssrc the signal will return the currently internal SRTP rollover counter
for the given stream.
For the SRTP decoder we have a new SRTP caps parameter "roc" that needs
to be set when a new SRTP stream is created for a given SSRC.
https://bugzilla.gnome.org/show_bug.cgi?id=726861
Expose one more libcurl option: CURLOPT_SSH_HOST_PUBLIC_KEY_MD5.
This allows authenticating the server by the MD5 fingerprint of
the server's public key.
https://bugzilla.gnome.org/show_bug.cgi?id=723167
The parsing function already frees the old value (if any), avoid a double
free by not freeing it before calling the function without setting the
pointer to NULL
Coverity ID: 1212178
The _parse_url function already frees the previous pointer, avoid
freeing it before without setting to null or we have a double free.
Coverity ID: 1212181
Coverity ID: 1212180
Coverity ID: 1212179
Refactor mssdemux to remove uridownloader to use an internal
source element which reduces startup latency and provides smaller
buffers for better buffering management downstream
data does not have to be freed at all here, it's a pointer to
an arbitrary position inside the current line. Also don't reuse
the data variable for anything else, that will cause crashes
in playlists that have the I-frame playlist URI followed by
other attributes.
CID 1212127
Set up a message handling function to be able to catch errors
from the source element and signal the cond to allow the download
loop to retry the download.
Instead, use a source element linked to a ghostpad to provide
smaller buffers and more granular control for downstream
buffering elements while also reducing startup latency
Only the first buffer of a fragment has its timestamp set, so only
update the segment.position when pushing those buffers to avoid
having GST_CLOCK_TIME_NONE set to the position
https://bugzilla.gnome.org/show_bug.cgi?id=729364
Otherwise we will never recover from previous errors, and especially
will never start again after a flushing seek if downstream returned
GST_FLOW_FLUSHING to us.
hlsdemux can't rely on the source to push flushes on a seek on ready
as that might not make sense. So always resort to flushing the
internal proxy pads by pushing flush events from the source's src pad.
Also as the seeking is not required anymore, only seek if there is
really a byte range to be used. And store a ref to the source's
src pad to avoid doing get_static_pad for every fragment.
In decryption scenario, a buffer is always stored to be sent later
to wait for more data or EOS to be able to strip the final bytes
if requested. In case an error hapenned this buffer can be ignored
and not pushed downstream.
Handle some more error cases:
1) When the source element fails to go to ready
2) When decryption fails
3) When there is no source to handle a specific URI
4) When the URI is invalid
Set up a message handling function to catch errors from the internal
source and store the last return code to identify error situations
when returning from a fragment download.
Also moves the duration increase to after the download when we
know if it was successful or not
When using the internal source, hlsdemux doesn't know the caps of
the input before adding the pad, so remove the arguments that would
use that as it is always NULL.
And use an specific flag to signal when a pad switch is required.
Using the discont flag is a bad idea now because when a fragment
download fails it will lead to exposing a pad group without any
data, causing decodebin to abort.
When receving EOS from the internal src, increase the current positon
by the fragment duration to allow correct restoring of download position
if the bitrate changes
Use the same properties as uridownloader to keep connections alive
between consecutive fragments downloads.
1) set keep-alive property to true
2) keep the element in READY instead of in NULL
Measure the download bitrate to be able to select
the best playlist.
As the buffers are directly pushed downstream and it
might block. The time is only measured from the download
until the pad push and it is started again after the push
returns.
Now the decryption is done buffer by buffer instead of on the
whole fragment at once. As it expects multiples of 16 bytes a
GstAdapter was added to properly chunk the buffers.
Also the last buffer must be resized depending on the value of the
last byte of the fragment, so hlsdemux always keeps a pending buffer
as it doesn't know if it is the last one yet
The GstElement is directly linked into a ghost pad and
its buffers are pushed as received downstream. This way the
buffers are small enough and not a whole fragment that usually
causes extra latency and makes buffering harder
They are very confusing for people, and more often than not
also just not very accurate. Seeing 'last reviewed: 2005' in
your docs is not very confidence-inspiring. Let's just remove
those comments.
Previously if the proxy server hostname was the empty string
curlhttpsink would never even set the libcurl option. For libcurl
however, having a proxy server hostname be the empty string means that
proxying should be disabled even if environment variables might be set.
Now with the restriction lifted, doing this is allowed.
https://bugzilla.gnome.org/show_bug.cgi?id=728960
rtcp_buffer_get_ssrc is called even with RTP buffers. this means we
might end up with an exception and not find any valid RTCP packet type
and thus hit GST_RTCP_TYPE_INVALID. we now take care of this.
https://bugzilla.gnome.org/show_bug.cgi?id=727512
This patch provides the basic infrastructure required for this.
Upload and Download has been ported to this.
Has the nice effect of allowing GstGLMemory to be our
refcounted texture object for any texture type (not just RGBA).
Should not lose any features/video formats.
But only add this for non-live playlists. For live playlists we already
have another thread that is periodically updating playlists.
Reason for this is that sometimes downloading a fragment can fail because
the URIs have changed or expired since last time.
Sequence numbers in different playlists are not guaranteed to be the same for the
same position, e.g. fragments could have different durations in different playlists.
In theory we should do exactly the same for live playlists, but unfortunately we can't
because doing this kind of seeking requires the complete playlist since we started
playback. For live playlists the server is however dropping fragments in the beginning
over time and we have no absolute time references.
