This commit tries to centralize the selection of vaCreateSurfaces
version, instead of having fallbacks everywhere.
These fallbacks are hacks, added because new drivers use the latest
version of vaCreateSurfaces (with surface attributes) [1], meanwhile
old drivers (or profiles as JPEG decoder in i965) might rather use the
old version.
In order to select which method, there's detected hack: each config
context has a list of valid formats, in the case of JPEG decoder the
list only contains "rare" 4:2:2 formats (ICM3, GRAY8) which aren't
handled correctly by the current gstreamer-vaapi code [2].
The hack consist in identify if the format list contains an arbitrary
preferred format (which is suposedly well supported by
gstreamer-vaapi, mostly NV12). If no prefered colour format is found,
the the old version of vaCreateSurfaces is used, and the surfaces wil
be mapped into a image with their own color format.
1. https://bugzilla.gnome.org/show_bug.cgi?id=797143
2. https://bugzilla.gnome.org/show_bug.cgi?id=797222
When baseline-as-constrained is set, the decoder will expose support
for baseline decoding and assume that the baseline content is
constrained-baseline. This can be handy to decode streams in hardware
that would otherwise not be possible to decode. A lot of baseline
content is in fact constrained.
The formats array is always created, in order to keep the logic and
to avoid broken caps, if this formats array doesn't contain any
elements, it has to be unref and the function should return NULL.
When the tune is NONE, we now can choose entrypoint freely. So the
GST_VAAPI_ENCODER_TUNE macro may not return the correct current
entrypoint.
We also delay CTU size calculation after entrypoint has been decided.
FEI encoders are not actively mantained neither tested, and it is
using infrastructure that is changing and FEI is stopping this
effort.
Also it is required to rethink how FEI can be used in GStreamer.
Instead of using a parent structure that has to be derived by API
consumers, this change propse a simplification by using the common
pattern of GTK of passing a function pointer and user data which will
be passed as its parameter. That user data contains the state and the
function will be called to update that state.
This new API allows the user to call a single method (process)
which handles the [display] lock/unlock logic internally for
them.
This API supersedes the risky begin, render, end API.
It eliminates the need for the user to call a lock method
(process_begin) before processing the input buffers
(process_render) and calling an unlock method (process_end)
afterwards.
See #219
The current get_profile just return one possible profile for the encode,
which is not enough. For example, if we want to support HEVC 4:4:4
profile, the input of encode should be VYUA rather than NV12 in HEVC
main profile. So the command line:
gst-launch-1.0 videotestsrc num-buffers=200 ! capsfilter \
caps=video/x-raw,format=VUYA,width=800,height=600 ! vaapih265enc \
tune=low-power init-qp=30 ! fakesink
can not work because vaapih265enc just report NV12 in sink caps, we need
to specify the profile obviously like:
gst-launch-1.0 videotestsrc num-buffers=200 ! capsfilter \
caps=video/x-raw,format=VUYA,width=800,height=600 ! vaapih265enc \
tune=low-power init-qp=30 ! capsfilter caps=video/x-h265, \
profile=main-444 ! fakesink
The encode should have the ability to choose the profile based on input
format automatically. If the input video format is VUYA, the main-444
profile should be auto choosed.
We modify to let get_allowed_profiles of each encode sub class to return
an array of all supported profiles based on downstream's allowed caps, or
return NULL if no valid profiles specified by downstream.
If no allowed profiles found, all profiles which belong to the current
encoder's codec will be the candidates.
The function gst_vaapi_encoder_get_surface_attributes collects the surface's
attributes for that profile list we just get.
So for this case, both NV12 and VUYA should be returned.
TODO: some codec like VP9, need to implement the get_profile() function.
We can get all the information about the video format at one shot
when we create the test context for getting the supported formats.
The current way to get the width and height ranges are inefficient,
since it calls the function gst_vaapi_profile_caps_append_encoder()
and it creates another temporal context to detect the resolution
information.
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
Some profile, such as H265_MAIN_444 on new Intel platform, may only
support ENTRYPOINT_SLICE_ENCODE_LP entrypoint. This leads two
problems:
1. We need to specify the tune mode like `vaapih265enc tune=low-power`
every time when we need to use this kind of profile. Or we can not
create the encoder context successfully.
2. More seriously, we set the entrypoint to a fixed value in
init_context_info() and so the create_test_context_config() can not
create the test context for these profile and can not get the
supported video formats, either.
We now change the entrypoint setting based on the tune option of the
encoder. If no tune property provided, we just choose the first
available entrypoint.
Instead of init_context_info() setting the passed profile, it is
assumed that it has to be set by each encoder.
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
The symbol GST_VAAPI_ENTRYPOINT_INVALID is just a representation of
zero, which was already used as an invalid value tacitly. This patch
only makes it explicit.
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
We use GST_VAAPI_OBJECT_NATIVE_DISPLAY with wrong parameter for x11
pixmap creation, which causes crash if we run the internal test case
of:
test-decode --pixmap
The old way make the one config for each profile/entrypoint pair,
which is not very convenient for description the relationship
between them. One profile may contain more than one entrypoints
to within it, so a set like data structure should be more suitable.
GstVaapiMiniObject and GstVaapiObject are deprecated.
This is the first step to remove them by porting GstVaapiSurface as
a GstMiniBuffer descendant.
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
There are several internal functions with 'create' name, but they
don't create any new structure, but rather it initializes that
structure. Renaming those function to reflect better their purpose.
GstVaapiMiniObject and GstVaapiObject are deprecated.
This is the first step to remove them by porting GstVaapiCodedBuffer
as a GstMiniBuffer descendant.
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
GstVaapiMiniObject and GstVaapiObject are deprecrated. This is the
first step to remove them, by porting GstVaapiImage as a
GstMiniObject.
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
The GstVaapiMiniObject is obsolete and we need to replace it. This
patch turns GstVaapiContext into a plain C structure with its own
reference counting mechanism.
Also this patch removes unused overlays attributes.
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
Issue detected by Coverity
`info_to_pack.h264_slice_header` is always allocated by
gst_vaapi_feipak_h264_encode(), thus checking it to free it afterwards
in doesn't make much sense. But it requires to be free on the error
path.
There may be a null pointer dereference, or else the comparison
against null is unnecessary.
In gst_vaapi_encoder_h264_fei_encode: All paths that lead to this null
pointer comparison already dereference the pointer earlier
Issue detected by Coverity
An unsigned value can never be negative, so this test will always
evaluate the same way.
In add_slice_headers: An unsigned value can never be less than 0
Issue detected by Coverity
There may be a null pointer dereference, or else the comparison
against null is unnecessary.
In gst_vaapi_encoder_h264_fei_encode: All paths that lead to this null
pointer comparison already dereference the pointer earlier
Issue detected by Coverity
`info_to_pak` variable in gst_vaapi_encoder_h264_fei_encode() is
declared in the stack, but it is free in
gst_vaapi_feienc_h264_encode() as if declared on the heap.
This patch initializes the structure and removes the free.
A non-heap pointer is placed on the free list, likely causing a crash
later.
In gst_vaapi_encoder_h264_fei_encode: Free of an address-of
expression, which can never be heap allocated.
Issue detected by Coverity
If the FEI mode is not handled the created resources should be
released and return and error code.
The system resource will not be reclaimed and reused, reducing the
future availability of the resource.
In gst_vaapi_encoder_h264_fei_encode: Leak of memory or pointers to
system resources
Don't try to decode until the first I-frame is received within the
currently active sequence. i965 H265 decoder don't show any artifact
but it crashes.
Fixes: #98
GST_VAAPI_FORMATS_ALL collects all declared formats in video-format
as a caps template string, and make them available in caps with
memory:VASurface feature.
Fixes: #199
The code is essentially the same for getting all op default
values. Thus, use a macro to help minimize code duplication
and [hopefully] encourage using the same mechanism for all
default getters.
Currently the parameter of skin-tone-enhancement filter is forced
to zero. In fact it could be set different value by the user.
So create a new property named as "skin-tone-enhancement-level"
for accepting the used defined parameter value.
At the same time, skin-tone-enhancement is marked as deprecated.
When skin-tone-enhancement-level is set, skin-tone-enhancement
will be ignored.
g_return_val_fail() documentations says:
If expr evaluates to FALSE, the current function should be
considered to have undefined behaviour (a programmer error).
The only correct solution to such an error is to change the
module that is calling the current function, so that it avoids
this incorrect call.
So it was missused in a couple parts of the H264 and H265 internal
decoders. This patch changes that to plain conditionals.
Also, it was included a couple code-style fixes.
Found by static analysis. encoder->mb_width * encoder->mb_height
is evaluated using 32-bit arithmetic before widen. Thus, cast
at least one of these to guint64 to avoid overflow.
The YUV formats have no ambiguity for drivers, so we can add them all.
Some old driver(i965) does not implement full get/put image functions
but can use derive image funtions for the YUV format. It does not
report that kind of formats correctly in image query, but will derive
that YUV format image from surface. The dynamic mapping of YUV format
will block that manner.
Adding more YUV format mapping has no side effect. So considering the
legacy driver conformance, we add all YUV formats mapping statically
and dynamic mapping RBG formats
Fix: #189Fix: #190
Multiple different scenarios could break the display thread creation and
end up blocking waiting for thread o be created. Fix them all by
correctly waiting for a new boolean to become valid.
Some streams have error data introducing unknown NAL type. There are
also kinds of NAL types we do not want to handle. The old manner will
set a decoder error when encounter this, which cause a latent crash bug.
The decoder may successfully decode the picture and insert it into DPB.
But there are error NAL units after the AU which cause the post unit error
and make that frame dropped. The later output of the picture still want
to ref that frame and crash.
No need to set decoder error when can not recognize or handle the NAL
unit, just skip it and continue.
Fix: #191
This patch makes use of GST_PARAM_USER_SHIFT to define the internal
param in encoders to decide which parameters to expose. Thus
gstreamer-vaapi will not interfere with any change in GStreamer in the
future.
Also, the internal symbol was change to
GST_VAAPI_PARAM_ENCODER_EXPOSURE to keep the namespacing.
The command line:
gst-launch-1.0 filesrc location=some_name.mjpeg ! jpegparse !
vaapijpegdec ! videoconvert ! video/x-raw,format=I420 ! vaapisink
will crash on i965 driver because of no pointer check.
We now generate the video format map between GST format and VA format
dynamically based on the image format returned by vaQueryImageFormats.
i965 driver does to report image format of 444P and Y800 forcc, while
the jpeg decoder context VASurfaceAttribPixelFormat use them. We can
not recognize these format and pass a NULL pointer to
gst_vaapi_surface_new_from_formats.
We need to add a pointer check here and let the fallback logic handle
this case correctly.
Other drivers work well.
Improve the mapping between va format and gst format. The new map
will be generated dynamically, based on the query result of image
format in VA driver. Also consider the ambiguity of RGB color
format in LSB mode.
We no longer need this obsolete set_property function now after
switch to standard gobject's property manner.
Also delete the old encoder's property enum in the header file.
G_PARAM_CONSTRUCT make all properties init correctly, we do not
need to init the properties manually.
G_PARAM_FLAG_VAAPI_ENCODER_EXPOSURE is a vaapi encoder specific
flag, means need to expose the property to according encode class.
Install properties for h264 encoder fei class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for h264 fei encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for vp9 encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for vp8 encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for mpeg2 encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for jpeg encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for h265 encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Install properties for h264 encoder class. Also set the new get/set
property functions for gobject class. Still use the old properties
way now and this new feature will be enabled later.
Replace all gstvaapiobject in vaapi encoders with standard gstobject.
Let the gstobject common logic to handle all the init and finalize
works. But the property install/set/get still use the old way, need
to be improved later.
The context overlay was an optimization to apply a video composition
to all the surfaces bound to a context.
But since commit 18031dc6 this optimization was disabled, so it is
better just get rid of it.
In commit 18031dc6 surface's parent context is not assigned because of
circular references. Since then (2013), there's has no issue with
subpictures attached to a context, the current only users of this API.
This patch cleans up all of related code with the unused surface's
parent context.
This code doesn't define the profile used by the internal encoder, but
it used to "predict" which is going to be used and to get the caps
restrictions.
Before the profile was predicted by checking the donwstream caps, but
sometimes they are not defined, setting an unknown profile. In order
to enhances this situation, the encoder asks to internal encoder if it
has one. If so, it is used.
To ask the internal encoder's profile a new accessor function was
added: gst_vaapi_encoder_get_profile()
The return value of vaGetConfigAttributes() of VAConfigAttribRTFormat
is a bitwise flag with *all* the supported chroma types.
Previously it was assumed that the return value was a single value,
thus when returning the GST_VAAPI_CHROMA_TYPE_XXX the code was a
simple case. But it is wrong.
This patch changes the case block with a sequence of ifs testing the
bitwise. For now we assume a "priority" list in the testing sequence.
gst_vaapi_profile_get_name() returns a proper name for
GstCaps. Nonetheless, there are many profiles which don't have a name
representation for that realm.
gst_vaapi_profile_get_va_name() returns the name of the profile
according to its VAProfile name.
This new funtion is used in the encoder error message.
Now that vaapipostproc can possible handle video-direction, it
should also handle the image-orientation event from upstream if
video-direction property is set to auto.
If the video direction is unsupported by the driver, an element
warning is posted in the bus to notify the application.
gst_vaapi_enum_type_get_nick() was added in the library thus it can
be used elsewhere. It retrives the nick from an enum gtype.
This patch locks the display before querying the pipeline caps and
stores the mirror and rotation capabilities, thus they are not queried
every time the video direction is set.
Commit 0afc8131 introduced a regression and only NV12 format were
admitted, failing in any other valid color format.
This patch sets the profile to GST_VAAPI_PROFILE_VP9_0 by default.
When set multiple settings of color balance like hue, saturation,
brightness and contrast for vaapipostproc, they should be set as
parameters of color balance filter, at the same color balance
filter calling.
Otherwise, multiple color balance filter calling will cause
previous setting get reset by the last calling with default value.
Fixes#182.
Signed-off-by: Yan Wang <yan.wang@linux.intel.com>
gst_vaapi_video_pool_reserve_unlocked() hit an undocumented behavoir
because it locks twice the same mutex.
Also, n had different meanings in the current code: as an increase
value and as a new total of allocated surfaces.
This patche removes the undocumented behavoir (usually a deadlock) and
fixes the meaning of n as the new total of allocated surfaces.
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
ICQ is Intelligent Constant Quality. It will use the initial QP
vaule of icq-quality-factor to adjust QP at MB level intelligently
to improve subjective quality.
QVBR is Quality defined VBR. It will use qvbr-quality-factor to
adjust QP for each MB to get enough quality picture without waste
of bits.
The value return from GST_VAAPI_RATECONTROL_MASK will be used by
GST_VAAPI_POPCOUNT32 as its inpput. GST_VAAPI_POPCOUNT32 can only
deal with unsigned int. Otherwise there may be an error of out of
range of integer if we define few more rate-control mode.
If display size is smaller than current frame size, then the crop size
will be set as display size, which either crashes the pipeline or the
output MD5 does not match. Rather it should use the actual decoded size.
This patch removes the cropping set. For rendering we can use aspect
ratio to set display size.
Fixes#175
Signed-off-by: Wang Zhanjun <zhanjunx.wang@intel.com>
Signed-off-by: Xu Guangxin <guangxin.xu@intel.com>
For some main-10 stream, sometime the luma is 8 bits while chrome is more
than 8 bits, which cause using the wrong NV12 surface as the render target
and decoding error.
Fix#176
