Some buffers and the associated FrameState state may still be pending at
that point. If the wayland connection is shared, then messages for the
buffer may still arrive. However, the associated event queue is already
deleted. So the result is a crash.
With a private connection the associated memory is leaked instead.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
gst_vaapi_window_wayland_set_render_rect() may be called from an arbitrary
thread. That thread may be responsible for making the window visible.
At that point another thread will block in gst_vaapi_window_wayland_sync()
because the frame callback will not be called until the window is visible.
If that happens, then acquiring the display lock in
gst_vaapi_window_wayland_set_render_rect() would result in a deadlock.
Cache the size of the opaque rectangle separately and create the opaque
region right before applying it to the surface.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
Implements new vmethod gst_vaapi_window_set_render_rectangle,
which is doing set the information of the rendered rectangle set by
user.
This is necessary on wayland at least to get exact information of
external surface.
And vaapisink calls this when gst_video_overlay_set_render_rectangle is
called.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
The Wayland sub-surfaces API is used to embed the video into an application
window.
See Appendix A. Wayland Protocol Specification as the following.
"""
The aim of sub-surfaces is to offload some of the compositing work
within a window from clients to the compositor. A prime example is
a video player with decorations and video in separate wl_surface
objects.
This should allow the compositor to pass YUV video buffer processing to
dedicated overlay hardware when possible.
"""
Added new method gst_vaapi_window_wayland_new_with_surface()
Original-Patch-By: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
Zhao Halley <halley.zhao@intel.com>
changzhix.wei@intel.com
Hyunjun Ko <zzoon@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
Currently vaGetSurfaceBufferWl() is used to create wayland buffers.
Unfortunately this is not implemented by the 'media-driver' and Mesa VA-API
drivers. And the implementation provided by 'intel-vaapi-driver' is not
compatible with a Wayland server that uses the iris Mesa driver.
So create the Wayland buffers manually with the zwp_linux_dmabuf_v1 wayland
protocol. Formats and modifiers supported by the Wayland server are taken
into account. If necessary, VPP is enabled to convert the buffer into a
supported format.
Fall back to vaGetSurfaceBufferWl() if creating buffers via dambuf protocol
fails.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/346>
Committing the first buffer for a surface must not be done before
ack_configure() has been sent for the xdg_surface.
With weston, the commit will fail with "error 3: xdg_surface has never been
configured".
Wait in gst_vaapi_window_wayland_show() until configure is done to avoid
this.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/346>
GstVaapiMiniObject and GstVaapiObject are deprecated.
This is the first step to remove them by porting GstVaapiSurface as
a GstMiniBuffer descendant.
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
[wl_shell] is officially [deprecated], so provide support for the
XDG-shell protocol should be provided by all desktop-like compositors.
(In case they don't, we can of course fall back to wl_shell).
Note that the XML file is directly provided by the `wayland-protocols`
dependency and generates the protocol marshalling code.
[wl_shell]: https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Client/group__iface__wl__shell.html
[deprecated]: 698dde1958
This is another step in the gobjectification of the internal library
of gstreamer-vaapi. Now it is the turn of GstVaapiWindow and its
derivates.
The idea is to minimize the changeset keeping the same design as
much as possible.
GstVaapiWindow is defined as an abstract class with two properties:
the GstVaapiDisplay and the native ID. Thus, many of the
GstVaapiObject macros were copied as GstVaapiWindow macros.
The function gst_vaapi_window_new_internal() is kept as a decorator
of for calling gst_vaapi_window_create() and the possibility of
failure.
The descendant classes, such as glx, still use the private
structures, but through the gobject mechanism.
Otherwise the following poll may not return for an arbitrary amount of
time. This can happen if another wayland event queue has flushed and read
our events.
https://bugzilla.gnome.org/show_bug.cgi?id=795224
Converity scan bug:
If the function returns an error value, the error value may be
mistaken for a normal value.
If g_atomic_pointer_compare_and_exchange() fails because the frame is
not the last one, the function fails. Thus, logging an info message.
Sometimes gst_vaapi_window_wayland_sync returns FALSE when poll returns EBUSY
during destruction.
In this case, if GstVaapiWindow is using vpp, leak of vpp surface happens.
This surface is not attached to anything at this moment, so we should release
it manually.
https://bugzilla.gnome.org/show_bug.cgi?id=781695
When the frame listener callbacks 'done', the number of pending
frames are decreased. Nonetheless, there might be occasions where
the buffer listener callbacks 'release', without calling previously
frame's 'done'. This leads to problem with
gst_vaapi_window_wayland_sync() operation.
This patch marks as done those frames which were callbacked, but if
the buffer callbacks 'release' and associated frame is not marked
as 'done' it is so, thus the number of pending frames keeps correct.
https://bugzilla.gnome.org/show_bug.cgi?id=780442
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
Don't call gst_vaapi_window_wayland_sync() when destroying the
wayland window instance, since it might lead to a lock at
gst_poll_wait() when more than one instances of vaapisink are
rendering in the same pipeline, this is because they share the
same window.
Since now all the frames are freed we don't need to freed the
private last_frame, since its address is invalid now.
https://bugzilla.gnome.org/show_bug.cgi?id=780442
Signed-off-by: Hyunjun Ko <zzoon@igalia.com>
Fix leakage of the last wl buffer.
VAAPI wayland sink needs to send a null buffer while destruction,
it assures that all the wl buffers are released. Otherwise, the last
buffer's callback might be not called, which leads to leak of
GstVaapiDisplay.
This was inspired by gstwaylandsink.
https://bugzilla.gnome.org/show_bug.cgi?id=774029
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
The proxy object of wl_buffer for the last frame remains in the
wl_map. Even though we call wl_buffer_destroy() in
frame_release_callback(), the proxy object remains without being
removed, since proxy object is deleted when wayland server sees the
delete request and sends 'delete_id' event.
We need to call roundtrip before destroying event_queue so that the
proxy object is removed. Otherwise, it would be mess up as receiving
'delete_id' event from previous play, when playing in the next
va/wayland window with the same wl_display connection.
https://bugzilla.gnome.org/show_bug.cgi?id=773689
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
Since gst_vaapi_window_vpp_convert_internal is created,
GstVaapiWindowX11/Wayland can use it for conversion.
Note that once it chooses to use vpp, it's going to use vpp
until the session is finished.
https://bugzilla.gnome.org/show_bug.cgi?id=759533
Currently, GstVaapiWindowX11/Wayland are not descendants of GstVaapiWindow.
This patch chains them up to GstVaapiWindow to handle common members in GstVaapiWindow.
https://bugzilla.gnome.org/show_bug.cgi?id=759533
This basically reverts 62c3888b76 (wayland:
decouple wl_buffer from frame).
Otherwise the frame may be overwritten while it is still used by the
compositer:
The frame done callback (frame_done_callback()) is called, when the
compositor is done processing the frame and hands it to the hardware.
The buffer release callback (frame_release_callback()) is called when the
buffer memory is no longer used.
This can be quite some time later: E.g. if weston (with the DRM backend)
puts the buffer on a hardware plane, then then buffer release callback is
called when the kernel is done with the buffer. This is usually when the
next frame is shown, so most likely after the frame done callback for the
next frame!
Since 70eff01d36 "wayland: sync() when
destroy()" the mentioned possible leak should no longer be a problem, so
reverting this change should cause no leaking buffers.
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=758848
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
Setting the sink to flushing causes gst_vaapi_window_wayland_sync() to
return FALSE which makes gst_vaapi_window_wayland_render() return
FALSE which ends up posting an ERROR message in
gst_vaapisink_show_frame_unlocked(). Solution is to just return TRUE
in the EBUSY case.
https://bugzilla.gnome.org/show_bug.cgi?id=753598
Before pushing a the new frame, the render() method calls sync() to flush the
pending frames. Nonetheless, the last pushed frame never gets rendered, leading
to a memory leak too.
This patch calls sync() in the destroy() to flush the pending frames before
destroying the window.
Also a is_cancelled flag is added. This flag tells to not flush the event
queue again since the method failed previously or were cancelled by the user.
https://bugzilla.gnome.org/show_bug.cgi?id=749078
Otherwise wl_display_dispatch_queue() might prevent the pipeline from
shutting down. This can happen e.g. if the wayland compositor exits while
the pipeline is running.
Changes:
* renamed unlock()/unlock_stop() to unblock()/unblock_cancel() in gstvaapiwindow
* splitted the patch removing wl_display_dispatch_queue()
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=747492https://bugzilla.gnome.org/show_bug.cgi?id=749078
wl_display_dispatch_queue() might prevent the pipeline from shutting
down. This can happen e.g. if the wayland compositor exits while the
pipeline is running.
This patch replaces it with these steps:
- With wl_display_prepare_read() all threads announce their intention
to read.
- wl_display_read_events() is thread save. On threads reads, the other
wait for it to finish.
- With wl_display_dispatch_queue_pending() each thread dispatches its
own events.
wl_display_dispatch_queue_pending() was defined since wayland 1.0.2
Original-patch-by: Michael Olbrich <m.olbrich@pengutronix.de>
* stripped out the unlock() unlock_stop() logic
* stripped out the poll handling
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=749078https://bugzilla.gnome.org/show_bug.cgi?id=747492
Since frame in the private data means the last frame sent, it would
semantically better use last_frame.
Also, this patch makes use of g_atomic_pointer_{compare_and_exchange, set}()
functions.
https://bugzilla.gnome.org/show_bug.cgi?id=749078
Wayland window has a pointer to the last pushed frame and use it to set the
flag for stopping the queue dispatch loop. This may lead to memory leaks,
since we are not keeping track of all the queued frames structures.
This patch removes the last pushed frame pointer and change the binary flag
for an atomic counter, keeping track of number of queued frames and use it for
the queue dispatch loop.
https://bugzilla.gnome.org/show_bug.cgi?id=749078
This patch takes out the wayland's buffer from the the frame structure. The
buffer is queued to wayland and destroyed in the "release" callback. The
frame is freed in the surface's "done" callback.
In this way a buffer may be leaked but not the whole frame structure.
- surface 'done' callback is used to throttle the rendering operation and to
unallocate the frame, but not the buffer.
- buffer 'release' callback is used to destroy wl_buffer.
Original-patch-by: Zhao Halley <halley.zhao@intel.com>
* code rebase
* kept the the event_queue for buffer's proxy
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=749078
This patch only intends to improve readability: in the method
gst_vaapi_window_wayland_sync() the if/do instructions are squashed into a
single while loop.
Also renames the frame_redraw_callback() callback into frame_done_callback(),
which is a bit more aligned to Wayland API.
The Wayland compositor may still use the buffer when the frame done
callback is called.
This patch destroys the frame (which contains the buffer) until the
release callback is called. The draw termination callback only controls
the display queue dispatching.
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
https://bugzilla.gnome.org/show_bug.cgi?id=747492
Reword surface pool allocation helpers so that to allow for a simple
form, e.g. gst_vaapi_surface_pool_new(format, width, height); and a
somewhat more elaborated/flexible form with optional allocation flags
and precise GstVideoInfo specification.
This is an API/ABI change, and SONAME version needs to be bumped.
Add a new generic helper function gst_vaapi_window_new() to create
a window without having the caller to check for the display type
himself. i.e. internally, there is now a GstVaapiDisplayClass hook
to create windows, and the actual backend implementation fills it in.
Add new generic helper functions gst_vaapi_texture_new_wrapped()
This is a simplification in view to supporting EGL.
Record the underlying native display instance into the toplevel
GstVaapiDisplay object. This is useful for fast lookups to the
underlying native display, e.g. for creating an EGL display.
As a last resort, if video processing capabilities (VPP) are not available,
or they did not produce anything conclusive enough, then try to fallback to
the original rendering code path whereby the whole VA surface is rendered
as is, no matter of video cropping or deinterlacing requests.
Note: under those conditions, the visual outcome won't be correct but at
least, something gets displayed instead of bailing out.
Try to use VA/VPP processing capabilities to handle video cropping and
additional rendering flags that may not be directly supported by the
underlying hardware when exposing a suitable Wayland buffer for the
supplied VA surface. e.g. deinterlacing, different color primaries than
BT.601, etc.
Update the frame redraw infrastructure with a new FrameState stucture
holds all the necessary information used to display the next pending
surface.
While we are at it, delay the sync operation down to when it is actually
needed. That way, we keep performing additional tasks meanwhile.
