Commit graph

65 commits

Author SHA1 Message Date
Michael Olbrich
887ba40e65 libs: window: wayland: destroy all wayland buffers during finalize
Some buffers and the associated FrameState state may still be pending at
that point. If the wayland connection is shared, then messages for the
buffer may still arrive. However, the associated event queue is already
deleted. So the result is a crash.

With a private connection the associated memory is leaked instead.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
2020-07-31 14:02:34 +02:00
Michael Olbrich
9eb25ab849 libs: wayland: update the opaque region in set_render_rect
gst_vaapi_window_wayland_set_render_rect() may be called from an arbitrary
thread. That thread may be responsible for making the window visible.

At that point another thread will block in gst_vaapi_window_wayland_sync()
because the frame callback will not be called until the window is visible.

If that happens, then acquiring the display lock in
gst_vaapi_window_wayland_set_render_rect() would result in a deadlock.

Cache the size of the opaque rectangle separately and create the opaque
region right before applying it to the surface.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
2020-07-31 14:02:33 +02:00
Hyunjun Ko
a362d99e9e libs: window: implements gst_vaapi_window_set_render_rectangle
Implements new vmethod gst_vaapi_window_set_render_rectangle,
which is doing set the information of the rendered rectangle set by
user.
This is necessary on wayland at least to get exact information of
external surface.

And vaapisink calls this when gst_video_overlay_set_render_rectangle is
called.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
2020-07-31 14:02:33 +02:00
Michael Olbrich
40364c2c09 libs: wayland: implement video overlay API
The Wayland sub-surfaces API is used to embed the video into an application
window.

See Appendix A. Wayland Protocol Specification as the following.

"""
The aim of sub-surfaces is to offload some of the compositing work
within a window from clients to the compositor. A prime example is
a video player with decorations and video in separate wl_surface
objects.

This should allow the compositor to pass YUV video buffer processing to
dedicated overlay hardware when possible.
"""

Added new method gst_vaapi_window_wayland_new_with_surface()

Original-Patch-By: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
                   Zhao Halley <halley.zhao@intel.com>
                   changzhix.wei@intel.com
                   Hyunjun Ko <zzoon@igalia.com>

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
2020-07-31 14:02:33 +02:00
Michael Olbrich
3d1cf3a6da doc: libs: wayland: add 'transfer full' to the returnvalye of gst_vaapi_window_wayland_new
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
2020-07-31 11:14:44 +02:00
Michael Olbrich
afe49e9fdb libs: window: wayland: use dmabuf protocol if available
Currently vaGetSurfaceBufferWl() is used to create wayland buffers.
Unfortunately this is not implemented by the 'media-driver' and Mesa VA-API
drivers. And the implementation provided by 'intel-vaapi-driver' is not
compatible with a Wayland server that uses the iris Mesa driver.

So create the Wayland buffers manually with the zwp_linux_dmabuf_v1 wayland
protocol. Formats and modifiers supported by the Wayland server are taken
into account. If necessary, VPP is enabled to convert the buffer into a
supported format.

Fall back to vaGetSurfaceBufferWl() if creating buffers via dambuf protocol
fails.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/346>
2020-07-27 15:02:19 +00:00
Michael Olbrich
9ccb0c57fa libs: window: wayland: wait for configure before committing the first buffer
Committing the first buffer for a surface must not be done before
ack_configure() has been sent for the xdg_surface.

With weston, the commit will fail with "error 3: xdg_surface has never been
configured".

Wait in gst_vaapi_window_wayland_show() until configure is done to avoid
this.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/346>
2020-07-27 15:02:19 +00:00
He Junyan
608ce681e5 libs: surface: port to GstMiniObject
GstVaapiMiniObject and GstVaapiObject are deprecated.

This is the first step to remove them by porting GstVaapiSurface as
a GstMiniBuffer descendant.

Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
2019-12-20 05:01:08 +00:00
Mathieu Duponchelle
38d25c65c3 doc: fix some incorrect gtk-doc links 2019-05-29 01:41:37 +02:00
Niels De Graef
f84394fa49 libs: wayland: add support for XDG-shell protocol
[wl_shell] is officially [deprecated], so provide support for the
XDG-shell protocol should be provided by all desktop-like compositors.
(In case they don't, we can of course fall back to wl_shell).

Note that the XML file is directly provided by the `wayland-protocols`
dependency and generates the protocol marshalling code.

[wl_shell]: https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Client/group__iface__wl__shell.html
[deprecated]: 698dde1958
2019-02-16 23:55:42 +01:00
Niels De Graef
8e695c8fdb libs: window: wayland: Prefix wl_shell_surface field with wl_
It will help us to distinguish from other Wayland shell surface
(such as XDG-shell) later on.
2019-02-16 19:14:26 +01:00
Niels De Graef
e980fbf83d libs: wayland: Prefix wl_shell field with wl_
It will help us to distinguish from other Wayland shells (such as
XDG-shell) later on.
2019-02-16 18:45:03 +01:00
Víctor Manuel Jáquez Leal
8ef95a7dc2 libs: window: use its own debug category 2018-12-24 12:20:00 +00:00
Víctor Manuel Jáquez Leal
6c364cb9a7 libs: window: refactor as gobject
This is another step in the gobjectification of the internal library
of gstreamer-vaapi. Now it is the turn of GstVaapiWindow and its
derivates.

The idea is to minimize the changeset keeping the same design as
much as possible.

GstVaapiWindow is defined as an abstract class with two properties:
the GstVaapiDisplay and the native ID. Thus, many of the
GstVaapiObject macros were copied as GstVaapiWindow macros.

The function gst_vaapi_window_new_internal() is kept as a decorator
of for calling gst_vaapi_window_create() and the possibility of
failure.

The descendant classes, such as glx, still use the private
structures, but through the gobject mechanism.
2018-12-24 12:20:00 +00:00
Víctor Manuel Jáquez Leal
ae2fbf951a libs: remove already include string.h
Since sysdeps.h includes string.h there's no need to include it again.
2018-09-25 18:22:35 +02:00
Jimmy Ohn
7b077cdcb9 libs: use g_clear_pointer() when possible
https://bugzilla.gnome.org/show_bug.cgi?id=797131
2018-09-13 11:35:27 +02:00
Michael Olbrich
1f71b5a57d wayland: don't poll if there are no pending frames
Otherwise the following poll may not return for an arbitrary amount of
time. This can happen if another wayland event queue has flushed and read
our events.

https://bugzilla.gnome.org/show_bug.cgi?id=795224
2018-04-18 13:57:53 +02:00
Víctor Manuel Jáquez Leal
775f912247 libs: windows: wayland: fail if cannot remove last frame
Converity scan bug:

If the function returns an error value, the error value may be
mistaken for a normal value.

If g_atomic_pointer_compare_and_exchange() fails because the frame is
not the last one, the function fails. Thus, logging an info message.
2017-08-08 17:29:54 +02:00
Hyunjun Ko
2d7c7630c9 libs: windows: wayland: fix leak if failure of sync
Sometimes gst_vaapi_window_wayland_sync returns FALSE when poll returns EBUSY
during destruction.
In this case, if GstVaapiWindow is using vpp, leak of vpp surface happens.
This surface is not attached to anything at this moment, so we should release
it manually.

https://bugzilla.gnome.org/show_bug.cgi?id=781695
2017-04-25 11:24:38 +02:00
Hyunjun Ko
824974e657 libs: window: wayland: mark frames as done
When the frame listener callbacks 'done', the number of pending
frames are decreased. Nonetheless, there might be occasions where
the buffer listener callbacks 'release', without calling previously
frame's 'done'. This leads to problem with
gst_vaapi_window_wayland_sync() operation.

This patch marks as done those frames which were callbacked, but if
the buffer callbacks 'release' and associated frame is not marked
as 'done' it is so, thus the number of pending frames keeps correct.

https://bugzilla.gnome.org/show_bug.cgi?id=780442

Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
2017-04-24 13:16:03 +02:00
Víctor Manuel Jáquez Leal
3b314ba93e libs: window: wayland: don't sync at destroy()
Don't call gst_vaapi_window_wayland_sync() when destroying the
wayland window instance, since it might lead to a lock at
gst_poll_wait() when more than one instances of vaapisink are
rendering in the same pipeline, this is because they share the
same window.

Since now all the frames are freed we don't need to freed the
private last_frame, since its address is invalid now.

https://bugzilla.gnome.org/show_bug.cgi?id=780442

Signed-off-by: Hyunjun Ko <zzoon@igalia.com>
2017-04-24 13:14:05 +02:00
Hyunjun Ko
ca314a25cc libs: window: wayland: null buffer at destroy()
Fix leakage of the last wl buffer.

VAAPI wayland sink needs to send a null buffer while destruction,
it assures that all the wl buffers are released. Otherwise, the last
buffer's callback might be not called, which leads to leak of
GstVaapiDisplay.

This was inspired by gstwaylandsink.

https://bugzilla.gnome.org/show_bug.cgi?id=774029

Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
2017-04-21 16:01:47 +02:00
Jagyum Koo
896a836f1d libs: window: wayland: rt event queue at destroy()
The proxy object of wl_buffer for the last frame remains in the
wl_map. Even though we call wl_buffer_destroy() in
frame_release_callback(), the proxy object remains without being
removed, since proxy object is deleted when wayland server sees the
delete request and sends 'delete_id' event.

We need to call roundtrip before destroying event_queue so that the
proxy object is removed. Otherwise, it would be mess up as receiving
'delete_id' event from previous play, when playing in the next
va/wayland window with the same wl_display connection.

https://bugzilla.gnome.org/show_bug.cgi?id=773689

Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
2017-04-21 16:01:46 +02:00
Víctor Manuel Jáquez Leal
9c3a4edf05 libs: window: wayland: cancel read at poll message
Always call wl_display_cancel_read() when an errno is set, but
different to EAGAIN or EINTR.

https://bugzilla.gnome.org/show_bug.cgi?id=780442
2017-04-21 16:01:34 +02:00
Hyunjun Ko
c5b3577e88 libs: window: x11/wayland: use new api for conversion
Since gst_vaapi_window_vpp_convert_internal is created,
GstVaapiWindowX11/Wayland can use it for conversion.

Note that once it chooses to use vpp, it's going to use vpp
until the session is finished.

https://bugzilla.gnome.org/show_bug.cgi?id=759533
2017-04-11 19:20:38 +02:00
Hyunjun Ko
4752f68a37 libs: window: x11/wayland: chaining up to GstVaapiWindow
Currently, GstVaapiWindowX11/Wayland are not descendants of GstVaapiWindow.
This patch chains them up to GstVaapiWindow to handle common members in GstVaapiWindow.

https://bugzilla.gnome.org/show_bug.cgi?id=759533
2017-04-11 16:41:41 +02:00
Hyunjun Ko
c4ef779931 libs: window: wayland: handle more VAStatus to use vpp
Since the commit landed https://github.com/01org/intel-vaapi-driver/pull/55,
we should consider more returned VAStatus to use vpp.

https://bugzilla.gnome.org/show_bug.cgi?id=779400
2017-03-02 02:22:26 +01:00
Víctor Manuel Jáquez Leal
b1f6da98de libs: fix code style for errors 2016-11-03 12:51:44 +01:00
Víctor Manuel Jáquez Leal
02c0e6c183 libs: trivial comment style fixes
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
2016-02-03 12:20:37 +01:00
Michael Olbrich
e4ed64b731 wayland: free the frame in frame_release_callback()
This basically reverts 62c3888b76 (wayland:
decouple wl_buffer from frame).

Otherwise the frame may be overwritten while it is still used by the
compositer:

The frame done callback (frame_done_callback()) is called, when the
compositor is done processing the frame and hands it to the hardware.
The buffer release callback (frame_release_callback()) is called when the
buffer memory is no longer used.

This can be quite some time later: E.g. if weston (with the DRM backend)
puts the buffer on a hardware plane, then then buffer release callback is
called when the kernel is done with the buffer. This is usually when the
next frame is shown, so most likely after the frame done callback for the
next frame!

Since 70eff01d36 "wayland: sync() when
destroy()" the mentioned possible leak should no longer be a problem, so
reverting this change should cause no leaking buffers.

Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>

https://bugzilla.gnome.org/show_bug.cgi?id=758848
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
2016-01-19 12:43:45 +01:00
Víctor Manuel Jáquez Leal
5fd4c8dffe wayland: rename is_cancelled to sync_failed
Since commit 065a18a3, the semantics of the variable is_cancelled did not make
sense. This commit renames this variable to sync_failed.
2015-08-24 19:24:07 +02:00
Olivier Crete
a83057cdb1 wayland: Don't return GST_FLOW_ERROR on flushing
Setting the sink to flushing causes gst_vaapi_window_wayland_sync() to
return FALSE which makes gst_vaapi_window_wayland_render() return
FALSE which ends up posting an ERROR message in
gst_vaapisink_show_frame_unlocked(). Solution is to just return TRUE
in the EBUSY case.

https://bugzilla.gnome.org/show_bug.cgi?id=753598
2015-08-24 19:23:54 +02:00
Víctor Manuel Jáquez Leal
70eff01d36 wayland: sync() when destroy()
Before pushing a the new frame, the render() method calls sync() to flush the
pending frames. Nonetheless, the last pushed frame never gets rendered, leading
to a memory leak too.

This patch calls sync() in the destroy() to flush the pending frames before
destroying the window.

Also a is_cancelled flag is added. This flag tells to not flush the event
queue again since the method failed previously or were cancelled by the user.

https://bugzilla.gnome.org/show_bug.cgi?id=749078
2015-05-18 15:25:52 +02:00
Michael Olbrich
11b9260b6c vaapisink: implement unlock/unlock_stop for wayland
Otherwise wl_display_dispatch_queue() might prevent the pipeline from
shutting down. This can happen e.g. if the wayland compositor exits while
the pipeline is running.

Changes:
* renamed unlock()/unlock_stop() to unblock()/unblock_cancel() in gstvaapiwindow
* splitted the patch removing wl_display_dispatch_queue()

Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>

https://bugzilla.gnome.org/show_bug.cgi?id=747492

https://bugzilla.gnome.org/show_bug.cgi?id=749078
2015-05-15 17:10:37 +02:00
Víctor Manuel Jáquez Leal
522ec79d2e wayland: wl_display_dispatch_queue() can block forever.
wl_display_dispatch_queue() might prevent the pipeline from shutting
down. This can happen e.g. if the wayland compositor exits while the
pipeline is running.

This patch replaces it with these steps:

- With wl_display_prepare_read() all threads announce their intention
  to read.
- wl_display_read_events() is thread save. On threads reads, the other
  wait for it to finish.
- With wl_display_dispatch_queue_pending() each thread dispatches its
  own events.

wl_display_dispatch_queue_pending() was defined since wayland 1.0.2

Original-patch-by: Michael Olbrich <m.olbrich@pengutronix.de>
* stripped out the unlock() unlock_stop() logic
* stripped out the poll handling

Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>

https://bugzilla.gnome.org/show_bug.cgi?id=749078

https://bugzilla.gnome.org/show_bug.cgi?id=747492
2015-05-15 17:10:37 +02:00
Víctor Manuel Jáquez Leal
882387de30 wayland: rename frame for last_frame
Since frame in the private data means the last frame sent, it would
semantically better use last_frame.

Also, this patch makes use of g_atomic_pointer_{compare_and_exchange, set}()
functions.

https://bugzilla.gnome.org/show_bug.cgi?id=749078
2015-05-14 16:48:43 +02:00
Víctor Manuel Jáquez Leal
c80951ada1 wayland: use a counter as sync flag
Wayland window has a pointer to the last pushed frame and use it to set the
flag for stopping the queue dispatch loop. This may lead to memory leaks,
since we are not keeping track of all the queued frames structures.

This patch removes the last pushed frame pointer and change the binary flag
for an atomic counter, keeping track of number of queued frames and use it for
the queue dispatch loop.

https://bugzilla.gnome.org/show_bug.cgi?id=749078
2015-05-14 16:48:43 +02:00
Víctor Manuel Jáquez Leal
62c3888b76 wayland: decouple wl_buffer from frame
This patch takes out the wayland's buffer from the the frame structure. The
buffer is queued to wayland and destroyed in the "release" callback. The
frame is freed in the surface's "done" callback.

In this way a buffer may be leaked but not the whole frame structure.

- surface 'done' callback is used to throttle the rendering operation and to
  unallocate the frame, but not the buffer.
- buffer 'release' callback is used to destroy wl_buffer.

Original-patch-by: Zhao Halley <halley.zhao@intel.com>
* code rebase
* kept the the event_queue for buffer's proxy

Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>

https://bugzilla.gnome.org/show_bug.cgi?id=749078
2015-05-14 16:48:43 +02:00
Víctor Manuel Jáquez Leal
af8ea3701e wayland: refactor _sync() method and rename callback
This patch only intends to improve readability: in the method
gst_vaapi_window_wayland_sync() the if/do instructions are squashed into a
single while loop.

Also renames the frame_redraw_callback() callback into frame_done_callback(),
which is a bit more aligned to Wayland API.
2015-04-21 17:25:47 +02:00
Michael Olbrich
7548c72a7d wayland: free frame in buffer release callback
The Wayland compositor may still use the buffer when the frame done
callback is called.

This patch destroys the frame (which contains the buffer) until the
release callback is called. The draw termination callback only controls
the display queue dispatching.

Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>

https://bugzilla.gnome.org/show_bug.cgi?id=747492
2015-04-21 17:25:25 +02:00
Michael Olbrich
b8f8d50f41 wayland: destroy vpp buffer pool on resize
Otherwise the old buffers with the old size are used.

https://bugzilla.gnome.org/show_bug.cgi?id=747491
2015-04-14 11:39:58 +02:00
Gwenole Beauchesne
96ce1bc761 videopool: add optional flags for surface pool allocation.
Reword surface pool allocation helpers so that to allow for a simple
form, e.g. gst_vaapi_surface_pool_new(format, width, height); and a
somewhat more elaborated/flexible form with optional allocation flags
and precise GstVideoInfo specification.

This is an API/ABI change, and SONAME version needs to be bumped.
2015-01-27 18:11:45 +01:00
Gwenole Beauchesne
ff0a237458 window: add generic helper to create windows.
Add a new generic helper function gst_vaapi_window_new() to create
a window without having the caller to check for the display type
himself. i.e. internally, there is now a GstVaapiDisplayClass hook
to create windows, and the actual backend implementation fills it in.
Add new generic helper functions gst_vaapi_texture_new_wrapped()

This is a simplification in view to supporting EGL.
2015-01-27 18:11:45 +01:00
Gwenole Beauchesne
1843774c0b display: record native display object.
Record the underlying native display instance into the toplevel
GstVaapiDisplay object. This is useful for fast lookups to the
underlying native display, e.g. for creating an EGL display.
2015-01-27 18:11:44 +01:00
Gwenole Beauchesne
3490212988 window: re-indent all GstVaapiWindow related source code. 2014-07-25 17:36:12 +02:00
Gwenole Beauchesne
15519ebe3d legal: add per-file authorship information.
Credit original authors on a per-file basis as we cannot expect people
to know all country-specific rules, or bother browsing through the git
history.
2013-11-22 06:39:15 +01:00
Gwenole Beauchesne
1362352825 wayland: render the raw surface if VPP failed.
As a last resort, if video processing capabilities (VPP) are not available,
or they did not produce anything conclusive enough, then try to fallback to
the original rendering code path whereby the whole VA surface is rendered
as is, no matter of video cropping or deinterlacing requests.

Note: under those conditions, the visual outcome won't be correct but at
least, something gets displayed instead of bailing out.
2013-08-27 18:29:45 +02:00
Gwenole Beauchesne
7759acd2f0 wayland: add supporting for video processing.
Try to use VA/VPP processing capabilities to handle video cropping and
additional rendering flags that may not be directly supported by the
underlying hardware when exposing a suitable Wayland buffer for the
supplied VA surface. e.g. deinterlacing, different color primaries than
BT.601, etc.
2013-08-27 18:29:45 +02:00
Gwenole Beauchesne
5fdc9a0e95 wayland: add new frame redraw infrastructure.
Update the frame redraw infrastructure with a new FrameState stucture
holds all the necessary information used to display the next pending
surface.

While we are at it, delay the sync operation down to when it is actually
needed. That way, we keep performing additional tasks meanwhile.
2013-08-27 18:28:28 +02:00
Gwenole Beauchesne
b1e5dfab96 display: validate display types. 2013-05-27 17:20:35 +02:00