Commit graph

1947 commits

Author SHA1 Message Date
He Junyan
973b879f95 libs: encoder: vp9: Add allowed_profiles.
We need the allowed_profiles to store the allowed profiles in down
stream's caps.
Command line like:
  vaapivp9enc ! capsfilter caps=video/x-vp9,profile="{ (string)1, \
    (string)3 }"
We need to store GST_VAAPI_PROFILE_VP9_1 and GST_VAAPI_PROFILE_VP9_3
in this list.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/380>
2020-11-30 16:45:54 +00:00
He Junyan
ba58557c14 libs: decoder: H265: Fix a typo in scc reference setting.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/402>
2020-11-30 18:00:38 +08:00
He Junyan
d94b864428 libs: decoder: H265: Add MAIN_422_12 profile supporting.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/390>
2020-11-27 14:29:01 +00:00
He Junyan
ec68ec518b video-format: Add Y212_LE format.
It can be used as HEVC YUV_4:2:2 12bits stream's decoder output, and
also can be used as the input format for encoding HEVC YUV_4:2:2 12bits
stream.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/390>
2020-11-27 14:29:01 +00:00
He Junyan
8af9b46748 libs: decoder: H265: Add MAIN_444_12 profile supporting.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/390>
2020-11-27 14:29:01 +00:00
He Junyan
92b30ffa1a video-format: Add Y412_LE format.
It can be used as HEVC YUV_4:4:4 12bits stream's decoder output, and
also can be used as the input format for encoding HEVC YUV_4:4:4 12bits
stream.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/390>
2020-11-27 14:29:01 +00:00
He Junyan
ed9a708397 libs: decoder: h265: fill missing predictor_palette_size field.
The predictor_palette_size of VAPictureParameterBufferHEVCScc is
forgotten and need to be filled when streams have palettes.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/396>
2020-11-15 12:39:22 +00:00
He Junyan
d72f818029 libs: utils: h265: Use get_profile_from_sps to get profile.
We now use gst_h265_get_profile_from_sps() to replace the old way
of gst_h265_profile_tier_level_get_profile() to get more precise
profile. The new function consider the unstandard cases and give
a more suitable profile decision.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/395>
2020-10-30 07:24:44 +00:00
Víctor Manuel Jáquez Leal
865a46b193 libs: decoder: vp9: 0xff segment pred probs if no temporal update
According to the spec (6.2.11 Segmentation params syntax)
segmentation_pred_prob[i] ast to be 0xff if not temporal_update.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/400>
2020-10-19 16:45:35 +02:00
Víctor Manuel Jáquez Leal
08b4aaa7fa libs: decoder: vp9: avoid reference rewriting
The removed code set all the reference frames to the current frame it is a key
one, but later, all the reference frames were rewritten with the decoded picture
buffers or VA_INVALID_SURFACE if they were not available.

Basically, all this time the first reference frame assignment has been ignored,
and it's not described by the spec, and this patch removes that code.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/400>
2020-10-19 13:42:53 +02:00
Sebastian Dröge
69fbe8c84c Update for gst_video_transfer_function_*() function renaming
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/391>
2020-09-07 12:50:11 +03:00
Víctor Manuel Jáquez Leal
f822366f92 Revert "libs: decoder: h264, h265: in context at least 16 reference surfaces"
This reverts commit b387081a4d as discussed in
b387081a4d
2020-08-22 12:57:54 +02:00
Víctor Manuel Jáquez Leal
c080ee6dc9 libs: surface: egl: guard memory type
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/384>
2020-08-18 08:11:27 +00:00
Víctor Manuel Jáquez Leal
7a3b25884c libs: egl: surface: export EGLImage as DMABuf if GEM not supported
This code path is used when frames are rendered as textures through
GstVideoGLTextureUploadMeta with EGL, mainly under Wayland.

Originally the EGLImage was exported as GEM, which was handled by
Intel drivers, but Gallium ones cannot create VA surfaces from
GEM buffers, only DMABuf.

This patch checks the memory types supported by VA driver to choose
the render the EGLImages from GEM or DMABuf, because GEM is still
better where supported.

DMABuf is well handled either by intel-vaapi-driver and gallium.

Fixes: #137
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/122>
2020-08-14 12:43:20 +02:00
Víctor Manuel Jáquez Leal
992d9e3a20 libs: filter: gst_vaapi_filter_get_memory_types()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/122>
2020-08-14 12:31:37 +02:00
He Junyan
30290115af libs: util: h265: use common parser API to get vaapi profiles.
We can reuse H265 parser's API to recognize the correct profile and
then just need to convert them to VAAPI profiles.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/356>
2020-08-08 19:28:15 +08:00
Víctor Manuel Jáquez Leal
3e9a6f80aa build: update for gl pkg-config file split
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/378>
2020-08-07 17:30:21 +02:00
Xu Guangxin
34b1d195b7 h264dec: mark remaining frames as unreference before exec_picture_refs_modification
8.2.4.2 required this. Some clips will crash if we do not fill the reference list like this.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/376>
2020-08-06 14:13:56 +08:00
He Junyan
55769a16c7 libs: encoder: H265: Enable Main 12 profile support.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/375>
2020-08-06 00:21:50 +08:00
He Junyan
6e97062d35 video format: Fix P012_LE's chrome type typo.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/375>
2020-08-06 00:21:36 +08:00
Jordan Petridis
89580cc230 gstvaapiminiobject.c: fix clang 10 warnings
the typesystem checks in g_atomic_pointer_compare_and_exchange
seem to trigger some false positives with clang 10

similar to gstreamer!584

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/377>
2020-08-04 21:15:01 +03:00
Michael Olbrich
887ba40e65 libs: window: wayland: destroy all wayland buffers during finalize
Some buffers and the associated FrameState state may still be pending at
that point. If the wayland connection is shared, then messages for the
buffer may still arrive. However, the associated event queue is already
deleted. So the result is a crash.

With a private connection the associated memory is leaked instead.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
2020-07-31 14:02:34 +02:00
Michael Olbrich
9eb25ab849 libs: wayland: update the opaque region in set_render_rect
gst_vaapi_window_wayland_set_render_rect() may be called from an arbitrary
thread. That thread may be responsible for making the window visible.

At that point another thread will block in gst_vaapi_window_wayland_sync()
because the frame callback will not be called until the window is visible.

If that happens, then acquiring the display lock in
gst_vaapi_window_wayland_set_render_rect() would result in a deadlock.

Cache the size of the opaque rectangle separately and create the opaque
region right before applying it to the surface.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
2020-07-31 14:02:33 +02:00
Hyunjun Ko
a362d99e9e libs: window: implements gst_vaapi_window_set_render_rectangle
Implements new vmethod gst_vaapi_window_set_render_rectangle,
which is doing set the information of the rendered rectangle set by
user.
This is necessary on wayland at least to get exact information of
external surface.

And vaapisink calls this when gst_video_overlay_set_render_rectangle is
called.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
2020-07-31 14:02:33 +02:00
Michael Olbrich
40364c2c09 libs: wayland: implement video overlay API
The Wayland sub-surfaces API is used to embed the video into an application
window.

See Appendix A. Wayland Protocol Specification as the following.

"""
The aim of sub-surfaces is to offload some of the compositing work
within a window from clients to the compositor. A prime example is
a video player with decorations and video in separate wl_surface
objects.

This should allow the compositor to pass YUV video buffer processing to
dedicated overlay hardware when possible.
"""

Added new method gst_vaapi_window_wayland_new_with_surface()

Original-Patch-By: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
                   Zhao Halley <halley.zhao@intel.com>
                   changzhix.wei@intel.com
                   Hyunjun Ko <zzoon@igalia.com>

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
2020-07-31 14:02:33 +02:00
Michael Olbrich
3d1cf3a6da doc: libs: wayland: add 'transfer full' to the returnvalye of gst_vaapi_window_wayland_new
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
2020-07-31 11:14:44 +02:00
Michael Olbrich
07aadb82fb libs: display: always call close_display()
All close_display() have their own checks for use_foreign_display and only
destroy locally created objects in that case.
Without this objects other than the actuall foreign display itself are
leaked.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/342>
2020-07-31 11:14:43 +02:00
He Junyan
5902e5bfdb video-format: Add the missing P012_LE into GST_VAAPI_FORMATS_ALL.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/373>
2020-07-31 08:38:10 +00:00
He Junyan
6ab488475a libs: decoder: fix a crash issue when get_surface_formats.
Some context does not report any valid format that we can support.
For example, the HEVC 444 12 bits decoder context, all the formats
it reports is not supported now, which make the formats list a NULL
array. We should check that pointer before we use it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/372>
2020-07-31 15:27:38 +08:00
He Junyan
7a9a99d0c4 libs: encoder: h265: choose the profile based on allowed list.
We can decide the profile in ensure_profile(), based on allowed list
passed by the encode. We also need to check whether the entrypoint is
available. Once it is decided, no need to check the hw entrypoint
them again.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/349>
2020-07-30 18:51:51 +08:00
He Junyan
65c8718f5e libs: display: Add a helper function to get profiles by codec.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/349>
2020-07-30 17:46:24 +08:00
He Junyan
3554f06235 libs: encoder: h265: modify set_max_profile to set_allowed_profiles.
In h265, bigger profile idc may not be compatible with the small profile
idc. And more important, there are multi profiles with the same profile
idc. Such as main-422-10, main-444 and main-444-10, they all have profile
idc 4.
So recording the max profile idc is not enough, the encoder needs to know
all allowed profiles when deciding the real profile.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/349>
2020-07-30 17:42:49 +08:00
He Junyan
579aa4cab1 libs: encoder: h265: No need to check hw_max_profile.
In h265, higher profile idc number does not mean better compression
performance and may be not compatible with the lower profile idc.
So, it is not suitable to find the heighest idc for hw to ensure the
compatibility.

On the other side, when the entrypoint of the selected profile is valid,
it means the hw really support this profile, no need to check it again.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/349>
2020-07-30 17:41:53 +08:00
He Junyan
cf3b0120cb libs: profile: Use get_codec_from_caps to get codec type.
There is no need to get a profile from the caps and then convert
that profile into codec type. We can get the codec type by caps's
name easily.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/358>
2020-07-29 16:17:35 +00:00
He Junyan
7acc3c4db8 libs: profile: h265: Fix return value of from_codec_data_h265.
profile_from_codec_data_h265() returns wrong GstVaapiProfile for h265.
The codec data of caps contain the profile IDC, but the mapping between
profile IDC and GstVaapiProfile is wrong.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/358>
2020-07-29 16:17:35 +00:00
He Junyan
ed55dd8df1 libs: utils: vpx: Add utils vpx to handle VP8/9 misc things.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/188>
2020-07-29 15:41:40 +00:00
Víctor Manuel Jáquez Leal
6022d97edf libs: display, context: handle broken jpeg decoder for i965 driver
JPEG decoding in i965 driver is pretty much broken, and the driver is
deprecated which mean authors only accept trivial fixes.

Surfaces for JPEG decoder context in i965 only handle IMC3[1] color
format which is not a common format in GStreamer. It can export it to
I420 at mapping raw bytes, but DMABuf exporting is problematic.

This patch artificially adds NV12 to the context format list when it's
JPEG decoder for i965 and force the usage of old VA-API for surface
creation without specifying color format. Also it artificially
disables the DMABuf announcement.

1. https://docs.microsoft.com/en-us/windows/win32/medfound/recommended-8-bit-yuv-formats-for-video-rendering#420-formats-16-bits-per-pixel

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/369>
2020-07-29 12:06:32 +02:00
Víctor Manuel Jáquez Leal
83d48837d2 libs: context: change function to internal code style
Instead of a getter the function `get_preferred_format()` to
`ensure_preferred_format()` which aligns to the code style.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/369>
2020-07-29 12:02:50 +02:00
Víctor Manuel Jáquez Leal
634fefab5d vaapidecode: build allowed srcpad caps from va context
Instead of generating allowed srcpad caps with generic information,
now it takes the size an formats limits from the decoder's context.

This is possible since srcpad caps are generated after the internal
decoder is created.

The patch replaces gst_vaapi_decoder_get_surface_formats() with
gst_vaapi_decoder_get_suface_attributes().

From these attributes, formats are only used for VASurface memory
caps feature. For system memory caps feature, the old
gst_vaapi_plugin_get_allowed_srcpad_caps() is still used, since
i965 jpeg decoder cannot deliver mappable format for gstreamer.
And for the other caps features (dmabuf and texture upload) the
same static list are used.

This patch also adds DMABuf caps feature only if the context
supports that memory type. Nonetheless, we keep the pre-defined
formats since they are the subset of common derive formats formats
supported either by amd/gallium and both intel drivers, since,
when exporting the fd through vaAcquireBufferHandle()/
vaReleaseBufferHandle(), the formats of the derivable image cannot
be retriebable from the driver. Later we'll use the attribute
formats for the DMABuf feature too, when the code be ported to
vaExportSurfaceHandle().

Finally, the allowed srcpad caps are removed if the internal decoder
is destroyed, since context attribues will change.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/366>
2020-07-27 19:32:00 +02:00
Michael Olbrich
afe49e9fdb libs: window: wayland: use dmabuf protocol if available
Currently vaGetSurfaceBufferWl() is used to create wayland buffers.
Unfortunately this is not implemented by the 'media-driver' and Mesa VA-API
drivers. And the implementation provided by 'intel-vaapi-driver' is not
compatible with a Wayland server that uses the iris Mesa driver.

So create the Wayland buffers manually with the zwp_linux_dmabuf_v1 wayland
protocol. Formats and modifiers supported by the Wayland server are taken
into account. If necessary, VPP is enabled to convert the buffer into a
supported format.

Fall back to vaGetSurfaceBufferWl() if creating buffers via dambuf protocol
fails.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/346>
2020-07-27 15:02:19 +00:00
Michael Olbrich
72d32a91d2 libs: window: allow choosing the format for the vpp pool
Signed-off-by: Michael Olbrich <m.olbrich@pengutronix.de>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/346>
2020-07-27 15:02:19 +00:00
Michael Olbrich
856b9d5980 video-format: add DRM formats to the mapping table
This will be needed for the DMABuf protocol support to map DRM formats to
vaapi and gstreamer formats.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/346>
2020-07-27 15:02:19 +00:00
Michael Olbrich
bef64a6420 libs: display: wayland: add basic dmabuf protocol support
This is just the basic infrastructure. Hook up the interface and collect
all supported formats.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/346>
2020-07-27 15:02:19 +00:00
Michael Olbrich
9ccb0c57fa libs: window: wayland: wait for configure before committing the first buffer
Committing the first buffer for a surface must not be done before
ack_configure() has been sent for the xdg_surface.

With weston, the commit will fail with "error 3: xdg_surface has never been
configured".

Wait in gst_vaapi_window_wayland_show() until configure is done to avoid
this.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/346>
2020-07-27 15:02:19 +00:00
He Junyan
a73e6b7dac libs: encoder: h265: set no P frame automatically.
The double reference lists may be required by drivers and there should
be no P frames in the of stream. The old way of converting P frames to
B frames is by setting `low-delay-b` property, which is unconvenient
and has bad user experience, since most of the users do not know when
to set this property, and if it is not set correctly, the encoding
pipeline fails or even hangs on some platforms. VA driver now provides
a attribute to query whether both reference lists must be un-NULL for
a profile/entrypoint pair.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/284>
2020-07-27 16:38:38 +02:00
He Junyan
ee7677211a libs: encoder: h265: Deprecate the low-delay-b property.
In HEVC, P and B definitions are different from AVC: P frames have
just one reference list and so 1 MV, while B frames have two reference
lists and so 2 MVs. No matter B or P, ist reference lists can contain
forward/backward reference. So P and B can both have bi-directions
dependency, the difference is just their reference list
number (i.e. MV number). This is different from the AVC.

The *low delay b mode* refers to a special HEVC mode, in which the
stream just contain I and B frames, without P frames, and all B frames
only have forward direction dependencies (i.e. all inter frames have 2
reference lists but no backward reference in both lists).  This is
similar to AVC I/P mode, but changing the P to the forward dependent
B.

The `low-delay-b` property is now just used to simply convert all P
frames to B frames when driver does not support P frames (so both
reference lists have the same references frames). This is a little
different from the meaning of low delay b mode (the two ref lists may
have the different reference frames). And the driver now can report
whether it supports P frames correctly, so there is no need to use
this property and deprecate it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/284>
2020-07-27 16:32:49 +02:00
He Junyan
f82850c6d3 libs: decoder: H265: Add MAIN_12 profile supporting.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/359>
2020-07-20 16:26:32 +00:00
He Junyan
7d3a19bace video-format: Add P012_LE format.
It can be used as HEVC YUV_4:2:0 12bits stream's decoder output, and
also can be used as the input format for encoding HEVC YUV_4:2:0 12bits
stream.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/359>
2020-07-20 16:26:32 +00:00
He Junyan
b1832223ff libs: profile: The VP9 profiles' name should be just "0,1,2,3"
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/357>
2020-07-12 20:34:31 +08:00
Víctor Manuel Jáquez Leal
2a93455137 vaapidecode: Remove NO_SURFACE error handling
Since surfaces are not bounded to decoding context it makes no sense
to keep the surface semaphore. This patch removes the handling of
this error.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/merge_requests/353>
2020-07-09 18:13:48 +02:00