The YUV formats have no ambiguity for drivers, so we can add them all.
Some old driver(i965) does not implement full get/put image functions
but can use derive image funtions for the YUV format. It does not
report that kind of formats correctly in image query, but will derive
that YUV format image from surface. The dynamic mapping of YUV format
will block that manner.
Adding more YUV format mapping has no side effect. So considering the
legacy driver conformance, we add all YUV formats mapping statically
and dynamic mapping RBG formats
Fix: #189Fix: #190
Multiple different scenarios could break the display thread creation and
end up blocking waiting for thread o be created. Fix them all by
correctly waiting for a new boolean to become valid.
Some streams have error data introducing unknown NAL type. There are
also kinds of NAL types we do not want to handle. The old manner will
set a decoder error when encounter this, which cause a latent crash bug.
The decoder may successfully decode the picture and insert it into DPB.
But there are error NAL units after the AU which cause the post unit error
and make that frame dropped. The later output of the picture still want
to ref that frame and crash.
No need to set decoder error when can not recognize or handle the NAL
unit, just skip it and continue.
Fix: #191
This patch makes use of GST_PARAM_USER_SHIFT to define the internal
param in encoders to decide which parameters to expose. Thus
gstreamer-vaapi will not interfere with any change in GStreamer in the
future.
Also, the internal symbol was change to
GST_VAAPI_PARAM_ENCODER_EXPOSURE to keep the namespacing.
The command line:
gst-launch-1.0 filesrc location=some_name.mjpeg ! jpegparse !
vaapijpegdec ! videoconvert ! video/x-raw,format=I420 ! vaapisink
will crash on i965 driver because of no pointer check.
We now generate the video format map between GST format and VA format
dynamically based on the image format returned by vaQueryImageFormats.
i965 driver does to report image format of 444P and Y800 forcc, while
the jpeg decoder context VASurfaceAttribPixelFormat use them. We can
not recognize these format and pass a NULL pointer to
gst_vaapi_surface_new_from_formats.
We need to add a pointer check here and let the fallback logic handle
this case correctly.
Other drivers work well.
Improve the mapping between va format and gst format. The new map
will be generated dynamically, based on the query result of image
format in VA driver. Also consider the ambiguity of RGB color
format in LSB mode.
We no longer need this obsolete set_property function now after
switch to standard gobject's property manner.
Also delete the old encoder's property enum in the header file.
G_PARAM_CONSTRUCT make all properties init correctly, we do not
need to init the properties manually.
G_PARAM_FLAG_VAAPI_ENCODER_EXPOSURE is a vaapi encoder specific
flag, means need to expose the property to according encode class.
Install properties for h264 encoder fei class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for h264 fei encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for vp9 encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for vp8 encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for mpeg2 encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for jpeg encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for h265 encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for h264 encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Replace all gstvaapiobject in vaapi encoders with standard gstobject.
Let the gstobject common logic to handle all the init and finalize
works. But the property install/set/get still use the old way, need
to be improved later.
The context overlay was an optimization to apply a video composition
to all the surfaces bound to a context.
But since commit 18031dc6 this optimization was disabled, so it is
better just get rid of it.
In commit 18031dc6 surface's parent context is not assigned because of
circular references. Since then (2013), there's has no issue with
subpictures attached to a context, the current only users of this API.
This patch cleans up all of related code with the unused surface's
parent context.
This code doesn't define the profile used by the internal encoder, but
it used to "predict" which is going to be used and to get the caps
restrictions.
Before the profile was predicted by checking the donwstream caps, but
sometimes they are not defined, setting an unknown profile. In order
to enhances this situation, the encoder asks to internal encoder if it
has one. If so, it is used.
To ask the internal encoder's profile a new accessor function was
added: gst_vaapi_encoder_get_profile()
The return value of vaGetConfigAttributes() of VAConfigAttribRTFormat
is a bitwise flag with *all* the supported chroma types.
Previously it was assumed that the return value was a single value,
thus when returning the GST_VAAPI_CHROMA_TYPE_XXX the code was a
simple case. But it is wrong.
This patch changes the case block with a sequence of ifs testing the
bitwise. For now we assume a "priority" list in the testing sequence.
gst_vaapi_profile_get_name() returns a proper name for
GstCaps. Nonetheless, there are many profiles which don't have a name
representation for that realm.
gst_vaapi_profile_get_va_name() returns the name of the profile
according to its VAProfile name.
This new funtion is used in the encoder error message.
Now that vaapipostproc can possible handle video-direction, it
should also handle the image-orientation event from upstream if
video-direction property is set to auto.
If the video direction is unsupported by the driver, an element
warning is posted in the bus to notify the application.
gst_vaapi_enum_type_get_nick() was added in the library thus it can
be used elsewhere. It retrives the nick from an enum gtype.
This patch locks the display before querying the pipeline caps and
stores the mirror and rotation capabilities, thus they are not queried
every time the video direction is set.
Commit 0afc8131 introduced a regression and only NV12 format were
admitted, failing in any other valid color format.
This patch sets the profile to GST_VAAPI_PROFILE_VP9_0 by default.
When set multiple settings of color balance like hue, saturation,
brightness and contrast for vaapipostproc, they should be set as
parameters of color balance filter, at the same color balance
filter calling.
Otherwise, multiple color balance filter calling will cause
previous setting get reset by the last calling with default value.
Fixes#182.
Signed-off-by: Yan Wang <yan.wang@linux.intel.com>
gst_vaapi_video_pool_reserve_unlocked() hit an undocumented behavoir
because it locks twice the same mutex.
Also, n had different meanings in the current code: as an increase
value and as a new total of allocated surfaces.
This patche removes the undocumented behavoir (usually a deadlock) and
fixes the meaning of n as the new total of allocated surfaces.
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
ICQ is Intelligent Constant Quality. It will use the initial QP
vaule of icq-quality-factor to adjust QP at MB level intelligently
to improve subjective quality.
QVBR is Quality defined VBR. It will use qvbr-quality-factor to
adjust QP for each MB to get enough quality picture without waste
of bits.
The value return from GST_VAAPI_RATECONTROL_MASK will be used by
GST_VAAPI_POPCOUNT32 as its inpput. GST_VAAPI_POPCOUNT32 can only
deal with unsigned int. Otherwise there may be an error of out of
range of integer if we define few more rate-control mode.
If display size is smaller than current frame size, then the crop size
will be set as display size, which either crashes the pipeline or the
output MD5 does not match. Rather it should use the actual decoded size.
This patch removes the cropping set. For rendering we can use aspect
ratio to set display size.
Fixes#175
Signed-off-by: Wang Zhanjun <zhanjunx.wang@intel.com>
Signed-off-by: Xu Guangxin <guangxin.xu@intel.com>
For some main-10 stream, sometime the luma is 8 bits while chrome is more
than 8 bits, which cause using the wrong NV12 surface as the render target
and decoding error.
Fix#176
When create vaapi surface, it is better to use the chroma type get
from jpeg file instead of using fixed 420 format. And the correct
chroma type can be determined by horizontal_factor/vertical_factor
flags that get from jpegparse.
The old mapping values return by gst_vaapi_utils_h265_get_profile_idc is
wrong, though GST_H265_PROFILE_IDC_MAIN and GST_H265_PROFILE_IDC_MAIN_10
happened to be the correct value.
We only support Annex A profile_idc (1-4).
When input frame's formate changes, vp{8,9} encoders don't reset their frame
counter, hence the newly created frame could become a P-frame, leading to some
major troubles (sigabrt in libdrm in case of vp9). This patch adds some frame
prediction-related reset logic to the `flush' methods of GstVaapiEncoderVP8 and
GstVaapiEncoderVP9 implementations.
There are many profile levels that can support
more than 102400 kbps. Thus, increase the max
allowed bitrate property value from 102400 kbps
to 2048000 kbps (same as msdk encoder plugins).
Adds vpp mirroring support to vaapipostproc. Use
property video-direction. Valid values are identity,
horiz or vert. Default is identity (no mirror).
Closes#89
v2: Use GstVideoOrientationMethod enum
v3: Don't warn for VA_MIRROR_NONE.
Use GST_TYPE_VIDEO_ORIENTATION_METHOD type.
v4: Query VAAPI caps when setting mirror value
instead of during per-frame processing.
v5: Return TRUE in warning cases when setting mirror value.
When enable low delay B, the reference list 1 will be same with
reference list 0, so need reset the num_ref_idx_l1_active_minus1
to num_ref_idx_l0_active_minus1.
Fixes: #160
g_assert will take no effect when glib's G_DISABLE_ASSERT macro is
defined. The function inside the g_assert will take no effect and
we will fail to set the correct slice number.
Use VAPictureParameterBufferHEVCExtension&
VASliceParameterBufferHEVCExtension to pass extension setting from
some extension profile clips which may include these information.
The hevc extension setting only supported after libva release 2.2.0
(API 1.2.0).
Allow users to set the target-percentage for
variable rate controls. The default value is
70 (as hard-coded prior).
v2: minimum allowed value changed from 0 to 1
v3: target-percentage unchanged if CBR used
Resolves#129
The advanced trellis algorithm is supported in VA driver. We add
its support as a property named "trellis" of encoder.
It only works for H264 now, should be more in future.
Can not find a suitable chrome_type for this GST_VIDEO_FORMAT_RGB
packed 24 format. Just use GST_VAAPI_CHROMA_TYPE_RGB32 as its chrome
type. This kind of surface will just be created by new API with fourcc
and no old style chrome based creation is available.
fixes: #151
Coverity scan bug:
If the function actually returns a null value, a null pointer
dereference will occur.
In gst_vaapi_encoder_ensure_param_roi_regions(): Return value of
function which returns null is dereferenced without checking
The decoder's surface chroma type should depend on the bit depth
of VP9's parser. For 10bits VP9 stream, we need to use P10LE kind
10 bits surface as the decoder result.
Fixes#155
This patch fixes a regression from commit 5b1fe9c6.
max_qp, in rate control configuration, appeared in libva release
2.1 (API 1.1), thus it is required to guard the VA API version.
Fixes: #150
string_of_va_chroma_format() gets a wrong string format description.
For example, the YUV420_10BPP get a string of 0x00000100 as output.
It's because VA_RT_FORMAT_xxx values are macro definitions. And
STRINGIFY(VA_RT_FORMAT_xxx) will expand to its real value
0x00000XXX.
To avoid the macro evaluation, it is changed to show only the color
format without VA_RT_FORMAT_ prefix.
The current manner can not recognize the correct level and always
set the tier to main. Need to add frame rate check to recognize
levels such as 4.1, 6.2, etc. We also add a logic to check main
and high tier based on bitrate.
Fixes: #145
media-driver currently fails to set a correct value of max_qp when
min_qp is different to zero, in CBR and VBR mode, generating full
quality frames, thus unexpected huge output.
This patch sets max_qp to an arbitrary value to avoid this output
temporary.
Fixes: #144
Currently the minimal value for either min_qp and init_qp are 1,
but VA documentation specifiy that zero is also valid and means
to ignore the quantiser.
The default value is not changed though to avoid behaivor changes
to users.
GValueArray is deprecated. Use GstValueArray instead.
gst_param_spec_array can be deserialized from command line using:
vaapih264enc view-ids="<(uint)40,(uint)100>" num-views=2
While the g_param_spec_value_array() can not, and always get
error: "gst_value_deserialize_g_value_array: unimplemented"
Also fixed an out-of-range bug.
The order in gst_vaapi_enc_picture_encode when encoding one
picture is not very correct. The misc parameters are set before
the picture parameters. Some of the misc parameters such as
ROI may change the current picture parameters. But the later
setting of picture parameter will re-init all picture related
parameters and clear the previous setting. The right order
should be picture parameter first and then misc parameters.
Signed-off-by: He Junyan <junyan.he@hotmail.com>
This code is just confused. It's asking for at least as many bits of
(z-axis) depth as the root window has bits of (color) depth. For rgb565
or rgb888 this is harmless, but at 10 bits per channel this demands a
30-bit or deeper Z buffer. While some hardware could in principle do a
32-bit Z buffer, Mesa does not expose such fbconfigs (at least on Intel
and AMD).
We're not actually using the Z buffer, so just stop asking for one.
[wl_shell] is officially [deprecated], so provide support for the
XDG-shell protocol should be provided by all desktop-like compositors.
(In case they don't, we can of course fall back to wl_shell).
Note that the XML file is directly provided by the `wayland-protocols`
dependency and generates the protocol marshalling code.
[wl_shell]: https://people.freedesktop.org/~whot/wayland-doxygen/wayland/Client/group__iface__wl__shell.html
[deprecated]: 698dde1958
native-id property is problematic since the variable that stores it is
gsize, which is platform specific, and in some is bigger than unsigned
long, and there are not way to handle gsize properties.
Also, GST_VAAPI_ID_INVALID is defined in gsize terms, and we would
like to keep using it for this scope.
This patch removes the native-id property and set it manually in
gst_vaapi_window_new_internal().
gst_vaapi_encoder_put_frame() and gst_vaapi_encoder_flush() duplicates
the same code segment where the coded buffer is created, the picture
encoded on it and pushed to the async queue.
The function gst_vaapi_encoder_encode_and_queue() refactor this.
In order to flush the pending pictures, a new internal encoder vmethod
is used: get_pending_reordered()
This method follows an iterator pattern which will return the next
picture to encode and push.
The base encoder will call this function in a loop when flush() is called.
For now, only H.264 and H.265 encoders implement this flushing mechanism.
This is another step in the gobjectification of the internal library
of gstreamer-vaapi. Now it is the turn of GstVaapiWindow and its
derivates.
The idea is to minimize the changeset keeping the same design as
much as possible.
GstVaapiWindow is defined as an abstract class with two properties:
the GstVaapiDisplay and the native ID. Thus, many of the
GstVaapiObject macros were copied as GstVaapiWindow macros.
The function gst_vaapi_window_new_internal() is kept as a decorator
of for calling gst_vaapi_window_create() and the possibility of
failure.
The descendant classes, such as glx, still use the private
structures, but through the gobject mechanism.
Add 444 10bit yuv format Y410, which can be used to decode
main-444 10bit streams. Currently, this feature is only
supported by media-driver in Icelake.
Add more kinds of chrometype which will be used to describe
new video formats. Sync it with 1.4.0 version header file.
Alse delete useless GST_VAAPI_CHROMA_TYPE_YUV410 chrome type.
Signed-off-by: He Junyan <junyan.he@hotmail.com>
We lack some video format because endianness declare.
The video format should not directly relate to endianness. For example,
ARGB on big endian should not be simplely seen as BGRA on little endian
machine. We should provide endianess convert or format convert help
functions if endianness does not match.
https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/112
Signed-off-by: Junyan He <junyan.he@intel.com>
Add 422 10bit yuv format Y210, which can be used to decode
main-10-422 10bit streams. Currently, this feature is only
supported by media-driver in Icelake.
https://bugzilla.gnome.org/show_bug.cgi?id=797264
VA_ROI_RC_QP_DELTA_SUPPORT return value will be ignored when the
rate control mode is set as CQP. In CQP mode, it shouldn't check
roi_rc_qp_delta_support return value from driver backend.
https://bugzilla.gnome.org/show_bug.cgi?id=797087
In gst_vaapi_context_reset(), if the context has to be destroyed, make
sure to create it first before allocating its associated surfaces.
This patch fixes a regression introduced in commit 82872f4 because
the formats available in the current context now are ensured before
creating the context's surfaces.
https://bugzilla.gnome.org/show_bug.cgi?id=797277
Removed exposed macros GST_VAAPI_OBJECT_DISPLAY() and
GST_VAAPI_OBJECT_ID() to plugins, keeping them only for internal
library usage.
The purpose is readability.
https://bugzilla.gnome.org/show_bug.cgi?id=797139
When running several vaapi elements at the concurrently, at
initialization, there is a race condition when extractin the avaible
formats for images and subpictures.
This patch add a lock when the those arrays are filled.
https://bugzilla.gnome.org/show_bug.cgi?id=797039
Low delay B frame provide the function of transforming
P frame into low delay B frame which frame type is B, but
only reference predictive frames. This can be used when P
frame unsupported. Especially for P and B both unsupported,
in this case, I and low delay B frame can be encoded in a
stream.
https://bugzilla.gnome.org/show_bug.cgi?id=796984
In some cases, the found_picture ended up being evicted and freed, which
would lead to a use after free when accessing picture->base.poc. In this
fix, we take a ref on the picture before calling dpb_evict.
https://bugzilla.gnome.org/show_bug.cgi?id=787124
Similarly to previous patch, we have no error concealment. As a side
effect, it's better to skip slices with missing references then passing
NULL pointers to the accelerator. Passing NULL pointer would lead to
major visual artifact, a behaviour that is likely undefined.
https://bugzilla.gnome.org/show_bug.cgi?id=787124
Usually in case of MVC decoding, dpb size is increasedi if subset sps.
That's why it resets context without this patch.
But for some media it doesn't increase dpb size. Even in this case we
should reset context to deal with MVC decoding.
Otherwise, it leads to assert.
https://bugzilla.gnome.org/show_bug.cgi?id=787124
If the unit could not be parsed, just skip this nal and keep parsing
what is left in the adapter. We need to flush the broken unit in the
decoder specific parser because the generic code does not know about
units boundary. This increases error resilliance.
Before this, the broken unit would stay in the adapter and EOS would be
returned. Which stopped the streaming. Just removing the EOS would have
lead to the adapter size growing indefinitely.
https://bugzilla.gnome.org/show_bug.cgi?id=796863
h264 log messages were logged in default category because a regression
in code. This patch renable the usage of vaapi logging category.
This regression was introduced in commit 7c365bdd.
This patch chains up failure to executing the modification process. The
end result is that we now fail decoding the slice if this process fails.
This avoid sending a corrupted state to the accelerator. In some special
cases, this could lead to unrecoverable errors.
https://bugzilla.gnome.org/show_bug.cgi?id=796832
gst_adapter_masked_scan_uint32_peek() asserts if size is 0. Don't
try and scan in that case. This fixes assertion that would some times
happen when the stream is corrupted.
https://bugzilla.gnome.org/show_bug.cgi?id=796832
This change is due a problem encoding JPEGs with Intel's
media-driver: green/black image when playback jpeg
This patch sets component identifier and quantization table
destination selector in frame header to support packing headers
by Intel's media-driver that does not accept packed header
in AP level.
https://bugzilla.gnome.org/show_bug.cgi?id=796705
The decoder was trying to detect earlier that a field was lost base
on guessing the parity order. This breaks in streams were the parity
order changes.
This patch reverts the field order prediction code added by commit
8dd93e9c8.
https://bugzilla.gnome.org/show_bug.cgi?id=796169
This ensure that we always have sentinels set in the reference
pictures arrays. The code wasn't unsafe, this simply improve the
tracing, so instead of printing 32 lines of zeros, va tracer
prints proper empty lists.
https://bugzilla.gnome.org/show_bug.cgi?id=796169
Gobjectification for GstVaapiDisplay was almost done by the commit 185da3d1.
But still something breaking GObject code convention remains, which is
calling gst_vaapi_display_new() in each decendants.
This patch replaces it with gst_vaapi_display_config(), defined in private
header.
https://bugzilla.gnome.org/show_bug.cgi?id=796470
