This is another step in the gobjectification of the internal library
of gstreamer-vaapi. Now it is the turn of GstVaapiWindow and its
derivates.
The idea is to minimize the changeset keeping the same design as
much as possible.
GstVaapiWindow is defined as an abstract class with two properties:
the GstVaapiDisplay and the native ID. Thus, many of the
GstVaapiObject macros were copied as GstVaapiWindow macros.
The function gst_vaapi_window_new_internal() is kept as a decorator
of for calling gst_vaapi_window_create() and the possibility of
failure.
The descendant classes, such as glx, still use the private
structures, but through the gobject mechanism.
Add 444 10bit yuv format Y410, which can be used to decode
main-444 10bit streams. Currently, this feature is only
supported by media-driver in Icelake.
Add more kinds of chrometype which will be used to describe
new video formats. Sync it with 1.4.0 version header file.
Alse delete useless GST_VAAPI_CHROMA_TYPE_YUV410 chrome type.
Signed-off-by: He Junyan <junyan.he@hotmail.com>
We lack some video format because endianness declare.
The video format should not directly relate to endianness. For example,
ARGB on big endian should not be simplely seen as BGRA on little endian
machine. We should provide endianess convert or format convert help
functions if endianness does not match.
https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/issues/112
Signed-off-by: Junyan He <junyan.he@intel.com>
Add 422 10bit yuv format Y210, which can be used to decode
main-10-422 10bit streams. Currently, this feature is only
supported by media-driver in Icelake.
https://bugzilla.gnome.org/show_bug.cgi?id=797264
VA_ROI_RC_QP_DELTA_SUPPORT return value will be ignored when the
rate control mode is set as CQP. In CQP mode, it shouldn't check
roi_rc_qp_delta_support return value from driver backend.
https://bugzilla.gnome.org/show_bug.cgi?id=797087
In gst_vaapi_context_reset(), if the context has to be destroyed, make
sure to create it first before allocating its associated surfaces.
This patch fixes a regression introduced in commit 82872f4 because
the formats available in the current context now are ensured before
creating the context's surfaces.
https://bugzilla.gnome.org/show_bug.cgi?id=797277
Removed exposed macros GST_VAAPI_OBJECT_DISPLAY() and
GST_VAAPI_OBJECT_ID() to plugins, keeping them only for internal
library usage.
The purpose is readability.
https://bugzilla.gnome.org/show_bug.cgi?id=797139
When running several vaapi elements at the concurrently, at
initialization, there is a race condition when extractin the avaible
formats for images and subpictures.
This patch add a lock when the those arrays are filled.
https://bugzilla.gnome.org/show_bug.cgi?id=797039
Low delay B frame provide the function of transforming
P frame into low delay B frame which frame type is B, but
only reference predictive frames. This can be used when P
frame unsupported. Especially for P and B both unsupported,
in this case, I and low delay B frame can be encoded in a
stream.
https://bugzilla.gnome.org/show_bug.cgi?id=796984
In some cases, the found_picture ended up being evicted and freed, which
would lead to a use after free when accessing picture->base.poc. In this
fix, we take a ref on the picture before calling dpb_evict.
https://bugzilla.gnome.org/show_bug.cgi?id=787124
Similarly to previous patch, we have no error concealment. As a side
effect, it's better to skip slices with missing references then passing
NULL pointers to the accelerator. Passing NULL pointer would lead to
major visual artifact, a behaviour that is likely undefined.
https://bugzilla.gnome.org/show_bug.cgi?id=787124
Usually in case of MVC decoding, dpb size is increasedi if subset sps.
That's why it resets context without this patch.
But for some media it doesn't increase dpb size. Even in this case we
should reset context to deal with MVC decoding.
Otherwise, it leads to assert.
https://bugzilla.gnome.org/show_bug.cgi?id=787124
If the unit could not be parsed, just skip this nal and keep parsing
what is left in the adapter. We need to flush the broken unit in the
decoder specific parser because the generic code does not know about
units boundary. This increases error resilliance.
Before this, the broken unit would stay in the adapter and EOS would be
returned. Which stopped the streaming. Just removing the EOS would have
lead to the adapter size growing indefinitely.
https://bugzilla.gnome.org/show_bug.cgi?id=796863
h264 log messages were logged in default category because a regression
in code. This patch renable the usage of vaapi logging category.
This regression was introduced in commit 7c365bdd.
This patch chains up failure to executing the modification process. The
end result is that we now fail decoding the slice if this process fails.
This avoid sending a corrupted state to the accelerator. In some special
cases, this could lead to unrecoverable errors.
https://bugzilla.gnome.org/show_bug.cgi?id=796832
gst_adapter_masked_scan_uint32_peek() asserts if size is 0. Don't
try and scan in that case. This fixes assertion that would some times
happen when the stream is corrupted.
https://bugzilla.gnome.org/show_bug.cgi?id=796832
This change is due a problem encoding JPEGs with Intel's
media-driver: green/black image when playback jpeg
This patch sets component identifier and quantization table
destination selector in frame header to support packing headers
by Intel's media-driver that does not accept packed header
in AP level.
https://bugzilla.gnome.org/show_bug.cgi?id=796705
The decoder was trying to detect earlier that a field was lost base
on guessing the parity order. This breaks in streams were the parity
order changes.
This patch reverts the field order prediction code added by commit
8dd93e9c8.
https://bugzilla.gnome.org/show_bug.cgi?id=796169
This ensure that we always have sentinels set in the reference
pictures arrays. The code wasn't unsafe, this simply improve the
tracing, so instead of printing 32 lines of zeros, va tracer
prints proper empty lists.
https://bugzilla.gnome.org/show_bug.cgi?id=796169
Gobjectification for GstVaapiDisplay was almost done by the commit 185da3d1.
But still something breaking GObject code convention remains, which is
calling gst_vaapi_display_new() in each decendants.
This patch replaces it with gst_vaapi_display_config(), defined in private
header.
https://bugzilla.gnome.org/show_bug.cgi?id=796470
The function name was gst_vaapi_display_create_unlocked(), nonetheless
it wasn't called unlocked. In order to keep the semantics this patch
renames the gst_vaapi_display_create_unlocked() as
gst_vaapi_display_create(), removing the previous function
gst_vaapi_display_create().
https://bugzilla.gnome.org/show_bug.cgi?id=796470
This is, practically, a revert of commit dcf135e2.
The parent logic is useful for the EGL display, which is a decorator
of the real windowing subsystem (X11 or Wayland). Thus it is avoided
calling vaInitialize() and vaTerminate() twice.
https://bugzilla.gnome.org/show_bug.cgi?id=795391
gst_vaapi_display_egl_new_with_native_display() has been broken since
it wasn't used.
Currently it's needed to call this API to create a display providing
the EGL display, so it could avoid duplicated calls to the native
display (eg. eglTerminate).
Signed-off-by: Victor Jaquez <vjaquez@igalia.com>
https://bugzilla.gnome.org/show_bug.cgi?id=795391
This change is due a problem decoding JPEGs with Intel's media-driver:
no image was generated.
This patch relases the VA buffers after vaEndPicture() is called,
and not before (after vaRenderPicture()).
https://bugzilla.gnome.org/show_bug.cgi?id=796505
The specification says,
"log2_max_pic_order_cnt_lsb_minus4 shall be in the range of 0 to 12, inclusive."
This patch changes the upper limit from 6 to 12.
https://bugzilla.gnome.org/show_bug.cgi?id=796179
This removes hard-coded checks on the parent subsystem of potential DRM
devices. These checks were set to exlude devices that do not originate
from the PCI bus, which is only a valid approach on x86 devices.
Other devices may have a DRM device originating from the platform
subsystem, so the checks that were previously restricted to PCI are
extended to cover platform devices as well.
https://bugzilla.gnome.org/show_bug.cgi?id=794840
Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com>
Otherwise the following poll may not return for an arbitrary amount of
time. This can happen if another wayland event queue has flushed and read
our events.
https://bugzilla.gnome.org/show_bug.cgi?id=795224
pic_width_in_luma_samples/pic_height_in_luma_samples can be 16-bit rounded
instead of 32-bit.
In addition, codedbuf_size must be calculated according to this change.
https://bugzilla.gnome.org/show_bug.cgi?id=753229
Even if it is the h264parse fault or bad video file, vaapih264dec
should set a proper value for VAPictureParameterBufferH264.num_ref_frames
as the driver might use it.
Also see "info.ref_frames = dpb_size;" in
gstvaapidecoder_h264.c::ensure_context
https://bugzilla.gnome.org/show_bug.cgi?id=793836
Check input buffers for ROI metas and pass them to VA. Also added a
new "default-roi-delta-qp" property in order to tell the encoder what
delta QP should be applied to ROI by default.
Enabled it for H264 and H265 encoders.
https://bugzilla.gnome.org/show_bug.cgi?id=768248
According to Debian package auto-building, uintptr_t is not an
unsigned long in i386 arch, raising an "incompatible pointer type"
error.
This patch adds a casting for compiler's satisfaction in i386.
Increased max values of periodic key frame for h26x codecs.
This allow more fine tunning of encoder that in certian scenario
want higher periodic key frame.
For example: it doesn't want a key frame each 10 seconds but
each 120 seconds.
https://bugzilla.gnome.org/show_bug.cgi?id=786320
eglGetDisplay() is currently broken in Mesa for Wayland. Also using
eglGetDisplay() is rather fragile, and it is recommended to use
eglGetPlatformDisplay() when possible.
In order to do that, this patch uses the helper in GstGL. If
gstreamer-vaapi is not compiled with GstGL support, eglGetDisplay()
will be used.
https://bugzilla.gnome.org/show_bug.cgi?id=790493
Frames are encoded as different layers. Frame in a particular
layer will use pictures in lower or same layer as references.
Which means decoder can drop the frames in upper layer but still
decode lower layer frames.
B-frames, except the one in top most layer, are reference frames.
All the base layer frames are I or P.
eg: with 3 temporal layers
T3: B1 B3 B5 B7
T2: B2 B6
T1: I0 P4 P8
T1, T2, T3: Temporal Layers
P1...Pn: P-Frames:
B1...Bn: B-frames:
T1: I0->P4 , P4->P8 etc..
T2: I0--> B2 <-- P4
T3: I0--> B1 <-- B2, B2 --> B3 <-- P4
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=788918
Frames are encoded as different layers. A frame in a particular
layer will use pictures in lower or same layer as references.
Which means decoder can drop the frames in upper layer but still
decode lower layer frames.
eg: with 3 temporal layers
T3: P1 P3 P5 P7
T2: P2 P6
T1: P0 P4 P8
T1, T2, T3: Temporal Layers
P1...pn: P-Frames:
P0->P1 , P0->P2, P2->P3, P0->P4......repeat
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=788918
The frame_num generation was not correctly implemented.
According to h264 spec, frame_num should get incremented
for each frame if previous frame is a referece frame.
For eg: IPBPB sequece should have the frame numbers 0,1,2,2,3
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=788918
Adds new property "prediction-type" to select different reference
picture selection modes like hierarchical-p, hierarchical-b etc.
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=788918
Adds some basic building blocks to ease the implementation
of hierarchical prediction modes.
-- add an utility method to find temporal level of each frame
-- define max_ref_frame count based on temporal level count
-- add temporal_level_div[] for finding temporal level each frame
to be encoded.
-- find ip_period based on temporal level count
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=788918
Adds new property "temporal-levels" to select the number of
temporal levels to be included in the encoded stream.
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=788918
We can have p-frame as non-ref and also b-frame as ref
which are not supported yet. Reference flag
is the first machinery needed for more advanced
reference picture selection modes.
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=788918
Adds a new function that changes the internal EGL display to the
current one (eglGetCurrentDisplay()) and sets the current context
too (eglGetCurrentContext()).
This new function is called by gst_vaapi_texture_egl_create() updating
the GstVaapiDisplayEGL with the current EGL display.
https://bugzilla.gnome.org/show_bug.cgi?id=773453
It is required to use the context of the calling thread when wrapping
a foreign texture. According the documentation of
GstVideoGLTextureUploadMeta:
"The caller of gst_video_gl_texture_upload_meta_upload() must
have OpenGL set up and call this from a thread where it is valid
to upload something to an OpenGL texture."
This patch updates the EGL display and context in GstVaapiDisplay
instance to the one used by te renderer that uploads the texture.
Original-patch-by: Daniel van Vugt <daniel.van.vugt@canonical.com>
https://bugzilla.gnome.org/show_bug.cgi?id=773453
Instead of silently fail to export the image if there is not available
the EGL_MESA_drm_image, log an error message. Also a code refactoring
was done.
https://bugzilla.gnome.org/show_bug.cgi?id=773453
In case of streaming, controlling bitrate dynamically for encoder might be
important to manage quality of the streaming.
This patch is to support such a scenario.
https://bugzilla.gnome.org/show_bug.cgi?id=786321
gstreamer-vaapi initializes the display by trial-and-error, thus
logging an error message if the display initialisation fails the user
may be weary of the error message in the screen, if using VA-API 1.0
This commit set the VA error log handler to GStreamer warning level
while calling vaInitialize() and set it to error after that.
https://bugzilla.gnome.org/show_bug.cgi?id=783169
GstVaapiDisplayEGL is a wrapper of another GstVaapiDisplay, either X11
or Wayland. Nonetheless it created another vaDisplay for it, instead
of using the wrapped one.
This patch enables the reuse of the wrapped vaDisplay avoiding
instantiating two.
https://bugzilla.gnome.org/show_bug.cgi?id=782212
This is a difference between the GObject API and the GstVaapi one: the
GValue passed to get a property value, in GObject has to be
initialized with g_value_init(), but in GstVaapi is has not.
In order to overcome this mismatch, this patch call g_value_init()
internally only in the passed one is not already initialized.
https://bugzilla.gnome.org/show_bug.cgi?id=788058
Shuffled some code to avoid to find the properties descriptor in the
array twice, adding the internal functions _set_property() and
_get_property().
https://bugzilla.gnome.org/show_bug.cgi?id=788058
Fixes regression introduced by commit 2eb2b26a.
There is a use case when the decoder set the src caps and immediatly
tries to process the media codec_data, this happens before decoder is
even opened, thus priv->parser is not instantiated yet.
https://bugzilla.gnome.org/show_bug.cgi?id=787818
Instead of handling the macroblock bitrate control as a integer, this
patch changes it as a enum, which is more self documented in the
GStreamer elements.
https://bugzilla.gnome.org/show_bug.cgi?id=787855
This patch fixes a regression introduced in commit 148f867c, since the
props variable is set to object's member variable
encoder->properties. And it is set in the instance initialization,
thus it will not be leaked.
https://bugzilla.gnome.org/show_bug.cgi?id=787733
Creates 2 properties, qp-ip and qp-ib for setting different QP for P/B
frames
and set slice_qp_delta for each frame according to the value provided.
https://bugzilla.gnome.org/show_bug.cgi?id=785923
Creates 2 properties, qp-ip and qp-ib for setting different QP for P/B
frames
and set slice_qp_delta for each frame according to the value provided.
In addition, remove the limitation of (<= 4) when setting
slice_qp_delta.
https://bugzilla.gnome.org/show_bug.cgi?id=785923
Creates new variable for QP for I frame and keep it at configuration and
use this for pic_init_qp and slice_qp_delta setting.
Since changing min qp doesn't make sense, keep min qp as is.
https://bugzilla.gnome.org/show_bug.cgi?id=785923
This is doing the same as h264 encoder as the following:
Using num_ref_frames provided and the result of the Query
VAConfigAttribEncMaxRefFrames, it determines the size of reference list
and perform encoding with multi reference frames as the following:
1\ The num_ref_frames is being considered as the number of
reference picture list0
2\ Encoder adds 1 reference frame more to the reference picture list1
internally if b-frame encoding.
3\ If num_ref_frames is bigger than the number of refrence frames
supported in the driver, it will be lowered.
Also this patch includes:
- Set num_negative_pics and num_positive_pics according to the number of
refs.
- Set delta_poc according to the number of refs.
- Increase max_dec_pic_buffering according to the number of refs
- Change max_num_reorder_pics according to num of bframes
https://bugzilla.gnome.org/show_bug.cgi?id=783804
Users can provide the number of reference frame by this property,
which is exaclty same as h264.
The value of the property will be considered as the number of
reference picture list0 and will add 1 reference frame more to the
reference picture list1 internally if b-frame encoding.
If the value provided is bigger than the number of refrence frames
supported in the driver, it will be lowered.
The maximum value is aligned to the value of the driver supported now.
https://bugzilla.gnome.org/show_bug.cgi?id=783804
Follows the specification as below:
7.4.7.1 in Rec. ITU-T H.265 v4 (12/2016)
num_ref_idx_active_override_flag equal to 1 specifies that the syntax
element num_ref_idx_l0_active_minus1 is present for P and B slices and
that the syntax element num_ref_idx_l1_active_minus1 is present for B
slices.
num_ref_idx_active_override_flag equal to 0 specifies that the syntax
elements num_ref_idx_l0_active_minus1 and num_ref_idx_l1_active_minus1
are not present.
https://bugzilla.gnome.org/show_bug.cgi?id=783804
Remove FIXME code, which makes previous assignation spurious.
This also means to make idr_period equal to keyframe period,
which is same as h264 encoder.
https://bugzilla.gnome.org/show_bug.cgi?id=783804
Adding FEI encoder to core lib.
The code is splitted into three session:
1: gstvaapiencoder_h264_fei.{h,c}
This is the replica of gstvaapiencoder_h264.{c,h} but with FEI.
All the modes ENC, PAK and ENC_PAK are running based
the code in these files.
2: gstvaapifeienc_h264.{h,c}
Abstract implementation intended for ENC (only VME) operation.
3: gstvaapifeipak_h264.{h,c}
Abstrct implementation intended for PAK (only the PAK module)
Right now ENC_PAK, ENC and PAK are running based on code
in gstvaapiencoder_h264_fei.{h,c}. The abstract implementations
in gstvaapifeienc_h264.{h,c} and gstvaapifeipak_h264.{h,c} are
needed if user request for ENC+PAK mode operation.
ENC+PAK: Here we need to invoke two sequence of
vaBeginPicture/vaRenderPicutre/vaEndPicture for each frame,
first for the ENC only and the second for PAK only.
Each mode associated with separate context ,but same pool of surfaces are
shared between the modes.
This is more useful once we have custom BRC algorithms.
Other Contributors:
Wang, Yi <yi.a.wang@intel.com>
Leilei <leilei.shang@intel.com>
Zhong, Xiaoxia <xiaoxia.zhong@intel.com>
xiaominc <xiaomin.chen@intel.com>
Li, Jing B <jing.b.li@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=785712https://bugzilla.gnome.org/show_bug.cgi?id=784667
MbCode, MV and Distortion buffers (fei codec objects)
can be treated as output of different fei modes based user request.
For eg: MbCode and MV are the output of ENC only. MbCode, MV and Dist
can be dumped as output in ENC_PAK mode for analysis purpose.
So treating them as a part of CodedBufferProxy too.
Here we avoided Qp, MbControl and MvPredictor codec objects since
there is no practical use case of treating them as "output buffers".
Other contributors:
Zhong, Xiaoxia <xiaoxia.zhong@intel.com>
xiaominc <xiaomin.chen@intel.com>
Leilei <leilei.shang@intel.com>
Li, Jing B <jing.b.li@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=785712https://bugzilla.gnome.org/show_bug.cgi?id=784667
Add fei codec objects to surface proxy since handling the
fei buffers(codec objects here) external to gstvaapisurfaceproxy
will make the code complicated. Especially considering the behavior
of encoder where the input frame order from upstream and output
frame order to the downstream are not sequential.
Other contributors:
Zhong, Xiaoxia <xiaoxia.zhong@intel.com>
xiaominc <xiaomin.chen@intel.com>
Leilei <leilei.shang@intel.com>
Li, Jing B <jing.b.li@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=785712https://bugzilla.gnome.org/show_bug.cgi?id=784667
There are 6 new va buffer types, each defined as a specific codec object.
Borrowed the code from gstvaapicodecobject , but made a clear separation
to avoid any possible mess-up. Because unlike the other gstvaaicodecobjects,
feicodecobjects can be shared between elements and also can be accessed
from different thread.
Unlike the other fei codecs object, VAEncMiscParameterTypeFEIFrameControl
object is not shared between elements.So we utilize the already
existing gst_vaapi_enc_misc_param_new(), but still keeping the code
in gstvaapfei_objects_priv.h in order to have a better
code readability.
Fixme:
-- Probably we need _locked_map() and _unlocked_map()
-- Context can be associated with PreEnc(not just Enoder)
once we have the proper support inplace, but for now we don't have
PreEnc support, so should be safe enough to use GstVaapiEncoder.
https://bugzilla.gnome.org/show_bug.cgi?id=785712https://bugzilla.gnome.org/show_bug.cgi?id=784667
Add a new vitrual method ensure_secondary_context to the
base encoder which is only required for the FEI entrypoint, that too
only when user configures the ENC+PAK mode. ENC+PAK mode is not something
supported directly by libva or driver, but this can be enabled
from the middleware.
Original Author of this idea: Leilei Shang <leilei.shang@intel.com>
Signed-off-by: Leilei Shang <leilei.shang@intel.com>
Signed-off-by: xiaominc <xiaomin.chen@intel.com>
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=785712https://bugzilla.gnome.org/show_bug.cgi?id=784667
Drops non-base NALs when the base-only property is set to TRUE.
This modifies the behavior for MVC streams with base-only too: All the
non-base units are dropped before they are decoded instead of dropping
the non-base frames.
The relevant part from the H264 spec is:
> Decoders that conform to one or more of the profiles specified in
Annex A rather than the profiles specified in Annexes G or H shall
ignore (remove from the bitstream and discard) the contents of all NAL
units with nal_unit_type equal to 14, 15, or 20.
To eliminate side effects from the offending units:
- PPS's with a broken seq_parameter_set_id (referring to dropped subset
SPS's) are ignored.
- The NAL parsing is skipped and their flags are set to
GST_VAAPI_DECODER_UNIT_FLAG_SKIP.
- Prefix units are not stored in prev_pi. Otherwise, parse_slice() would
use them even if they are flagged to be skipped. Subset SPS's and slice
extension units are not stored there either.
https://bugzilla.gnome.org/show_bug.cgi?id=732266
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
In VA-API 1.0 the enum VAEncPackedHeaderH264_SEI is deprecated, and
instead VAEncPackedHeaderRawData should be used.
This patch creates a compatibility symbol,
VA_ENC_PACKED_HEADER_H264_SEI, to expose the used enum according the
VA-API version.
https://bugzilla.gnome.org/show_bug.cgi?id=784398
In VA-API 1.0 the union bits in VAEncMiscParameterBufferROI has
renamed one member from roi_value_is_qp_delat to
roi_value_is_qp_delta, which is the correct name.
In order to keep back compatibility a macro has added to access this
union member.
https://bugzilla.gnome.org/show_bug.cgi?id=784398
Coverity scan bug:
An assigned value that is never used may represent unnecessary
computation, an incorrect algorithm, or possibly the need for cleanup
or refactoring.
ip_period is assigned first to be rewritter inmediatly after. The
first assignation is spurious.
Coverity scan:
Logically dead code: The indicated dead code may have performed some
action; that action will never occur.
By using pointer arithmetic is impossible to get NULL.
Coverity scan bug:
Out-of-bounds write. This could cause an immediate crash or incorrect
computations.
Coverity basically found that it is possible to assign more than 4
attribs in the array.
In my opinion this was produced because code pattern used pointer
arithmetic, which is not readable nor maintainable.
This patch refactors config_create() to use an array index rather than
pointer arithmetic. Also a run-time check for index size was added.
Converity scan bug:
If the function returns an error value, the error value may be
mistaken for a normal value.
If g_atomic_pointer_compare_and_exchange() fails because the frame is
not the last one, the function fails. Thus, logging an info message.
Coverity scan bug:
If the function returns an error value, the error value may be
mistaken for a normal value.
Function sscanf returns the number of assignations done. Validate this
return value with the number of expected variables to match.
Coverity scan bug:
Dereference after null check: Either the check against null is
unnecessary, or there may be a null pointer dereference.
Variable klass has been validated as non-NULL several time before in
gst_vaapi_object_new() function, so there is no need to check it
again.
Coverity scan bug:
An assigned value that is never used may represent unnecessary
computation, an incorrect algorithm, or possibly the need for cleanup
or refactoring.
ip_period is assigned first to be rewritter inmediatly after. The
first assignation is spurious.
Coverity scan bug:
The copied code will not have its intended effect.
This is a bug from commit cdaf15b2, where the intention is to
initialize RefPicList1 while setting RefPicList0.
Coverity scan bug:
Unintentional integer overflow. The expression's value may not be what
the programmer intended, because the expression is evaluated using a
narrow (i.e. few bits) integer type.
Cast operator to guint64 before computation to avoid narrowing.
merge with 3c5a6add
Coverity scan bug:
An assigned value that is never used may represent unnecessary
computation, an incorrect algorithm, or possibly the need for cleanup
or refactoring.
In the return value of decode_slice() or
gst_mpeg4_parse_video_packet_header() are not success, thus fail
decode_packet() function.
Coverity scan bug:
Dereference after null check: Either the check against null is
unnecessary, or there may be a null pointer dereference.
While looking for hte lowest poc, according to rest of the code, the
picture in the dbp (decoded picture buffer) might be NULL, thus we
could check for a NULL picture before assigned as found.
Also, split a comma operator because it is considered as a bad
practice because it possible side effects.
Coverity scan bug:
Scalars (for example, integers) are not properly
bounds-checked (sanitized) before being used as array or pointer
indexes, loop boundaries, or function arguments are considered as
tainted.
In this case, num_nals were not checked before used as loop control.
Coverity scan bug:
Dereference after null check: Either the check against null is
unnecessary, or there may be a null pointer dereference.
In the original commit for fill_picture_gaps() (commit 5abd2b90) the
prev_picture could be NULL, that's why the code did a null check. But,
since commit 52adebe7, the previous reference frames are tracked, thus
there is no need to check null anymore.
Using num_ref_frames provided and the result of the Query
VAConfigAttribEncMaxRefFrames, it determines the size of reference list
and perform encoding with multi reference frames as the following:
1\ The num_ref_frames is being considered as the number of
reference picture list0
2\ Encoder adds 1 reference frame more to the reference picture list1
internally if b-frame encoding.
3\ If num_ref_frames is bigger than the number of refrence frames
supported in the driver, it will be lowered.
https://bugzilla.gnome.org/show_bug.cgi?id=783803
Users can provide the number of reference frame by this property.
The value of the property will be considered as the number of
reference picture list0 and will add 1 reference frame more to the
reference picture list1 internally if b-frame encoding.
If the value provided is bigger than the number of refrence frames
supported in the driver, it will be lowered.
https://bugzilla.gnome.org/show_bug.cgi?id=783803
This function will query VAConfigAttribEncMaxRefFrames to get the
maximum number of reference frames supported in the driver.
This will be used for h264/h265 encoding.
https://bugzilla.gnome.org/show_bug.cgi?id=783803
Added a new property "compliance-mode", which default is the normal
strict compliant mode.
The second mode, "restrict-buf-alloc", is to limit the coded buffer
allocation size to improve performance in some specific Intel
platforms (there is asignificant performance improvement in parallel
encodings). Under this new mode, we use the MinCR field in A.3.1 for
pre-calculating the coded-buffer size.
https://bugzilla.gnome.org/show_bug.cgi?id=784590
Push frames downstream as soon as possible instead of waiting until
they are ejected from the DPB.
This patch makes the decoder not comply with the H.264 specification,
but it is required for some video cameras.
https://bugzilla.gnome.org/show_bug.cgi?id=762509
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
If the picture is IDR, also submit a SPS header.
This means when frame number reaches to keyframe-period or an force
key unit event arrives, we insert SPS/PPS again.
https://bugzilla.gnome.org/show_bug.cgi?id=776712
GST_VIDEO_CODEC_FRAME_IS_FORCE_KEYFRAME() is a flag usually used to manage
the `frame-lost` event in the case of streaming, such as RTP.
In case of this event, it is needed to start new GOP rather than just
produce an I-frame.
https://bugzilla.gnome.org/show_bug.cgi?id=776712
Insert an AUD as the first NAL of each encoded frame.
Some applications require Access Unit Delimiter for decoding the
stream.
The AU delimeter insertion is done only when the aud parameter is
TRUE (by default is disabled). The reason of this it is because this
header is only available from Intel Gen9 and the VA intel driver
should be 1.8 or superior. Otherwise, the output will be corrupted.
https://bugzilla.gnome.org/show_bug.cgi?id=776712
Signed-off-by: Victor Jaquez <vjaquez@igalia.com>
Currently when num_views is changed by multiview-mode on sink caps, it produces
wrong MVC encoded stream since the array view_ids is not set properly according
to changed num_views.
So this patch initializes all of the array sequentially to handle this case.
Side effect is not going to happen by this patch since this array is being
handled by num_views.
https://bugzilla.gnome.org/show_bug.cgi?id=784321
Until now, the encoder ignored the profile in src caps and chose one
according with the given parameters. But the encoder must honor the
profile specifed in src caps.
This patch do that, and if the encoder needs to choose the profile,
it will do it by following these rules:
1\ If given parameters are not compatible with given profile, the
encoder will bail out with an error.
2\ The encoder will choose the higher profile indicated in the
src caps.
https://bugzilla.gnome.org/show_bug.cgi?id=757941
Since commits in https://bugzilla.gnome.org/show_bug.cgi?id=781142 landed,
they introduced regression in seek.
Formerly, once seek is done, decoder drops P-frames until I-frame arrives.
But since the commits landed, it doesn't drop P-frame and does try to
decode it continuously because active_sps is still alive. See ensure_sps function.
But there are prev_frames and prev_ref_frames reset already, then it
causes assertion.
So it's necessary to reset active_sps/pps also in reset method.
https://bugzilla.gnome.org/show_bug.cgi?id=783726
There are some symbols that are not used when compiling with old
version of libva and those generates a compilation error.
Original-patch-by: Matt Staples <staples255@gmail.com>
Change the hard-coded range of quality-level from {1-8} to {1-7},
since it is the range Intel Open source driver supports.
Also perform the range clamping only if the user provided
quality-level is greater than the max-range suppored by the driver,
because there could be non-intel drivers giving lower value than
the hard-coded max value 7.
https://bugzilla.gnome.org/show_bug.cgi?id=783567
Just set the framerate parameter if the framerate numerator and
denominator are bigger than zero.
Otherwise, in Intel Gen6 driver, a warning is raised disabling the
bitrate control.
Original-patch-by: Hyunjun Ko <zzoon@igalia.com>
https://bugzilla.gnome.org/show_bug.cgi?id=783532
Instead of recalculating the miscellaneous buffer parameters for
every buffer, it is only done once, when the encoder is configured.
And for every buffer, the same structures are just copied.
https://bugzilla.gnome.org/show_bug.cgi?id=783449
This is patch pretends to decouple the assignation of the values
in the parameter structures and the VA buffer's parameters setting.
It may lead to some issues since HRD, framerate or controlrate may
not be handled by the specific encoder, but they are set in
the VA buffer's parameters.
I leave as it because this patch is just a transitional patch.
https://bugzilla.gnome.org/show_bug.cgi?id=783449
According to the VA documentation:
The framerate is specified as a number of frames per second,
as a fraction. The denominator of the fraction is given in
the top half (the high two bytes) of the framerate field, and
the numerator is given in the bottom half (the low two bytes).
For example, if framerate is set to (100 << 16 | 750), this is
750 / 100, hence 7.5fps.
If the denominator is zero (the high two bytes are both zero)
then it takes the value one instead, so the framerate is just
the integer in the low 2 bytes.
This patch fixes the the framerate calculation in vp8 encoder
according to this.
https://bugzilla.gnome.org/show_bug.cgi?id=783449
Move frame-rate parameter from ensure_misc_params() to
ensure_contro_rate_param() since it only has meaning when the
control rate is either VBR or CBR.
https://bugzilla.gnome.org/show_bug.cgi?id=783449
Move the Hypothetical Reference Decoder (HRD) parameter, from
ensure_misc_params() to ensure_control_rate_params(), since it
only shall be defined when the control rate is either VBR or CBR.
https://bugzilla.gnome.org/show_bug.cgi?id=783449
Instead of filling the control rate param in ensure_misc_params(),
this patch refactor it out, as a first step to merge the same code
for all the encoders.
https://bugzilla.gnome.org/show_bug.cgi?id=783449
Instead of using a proxy to story the buffer quality level, the
encoder now uses the native VA structure, which is copied to the
dynamically allocated VAEncMiscParameterBuffer.
This approach is computationally less expensive.
Right now, H264 and HEVC can set as a property the number of slices to
process. But each driver can set a maximum number of slices, depending
on the supported profile & entry point.
This patch verifies the current num_slices to process against the maximum
permitted by the driver and the media size.
https://bugzilla.gnome.org/show_bug.cgi?id=780955
Since we started using VPP in VaapiWindowX11, we need to care about
the case that src rect and window's size are different.
So, once VPP has converted to other format, we should honor the
size of the VPP's surface as source rect. Otherwise, it is cropped
according the previous size of the source rect.
https://bugzilla.gnome.org/show_bug.cgi?id=782542
Set ROI params during encoding each frame, which are set via
gst_vaapi_encoder_add_roi ()
https://bugzilla.gnome.org/show_bug.cgi?id=768248
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
Queries if the driver supports "Region of Interest" (ROI) during the config
creation.
This attribute conveys whether the driver supports region-of-interest (ROI)
encoding, based on user provided ROI rectangles. The attribute value is
partitioned into fields as defined in the VAConfigAttribValEncROI union.
If ROI encoding is supported, the ROI information is passed to the driver
using VAEncMiscParameterTypeROI.
https://bugzilla.gnome.org/show_bug.cgi?id=768248
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
This patch adds the handling of VAEncMiscParameterTypeQualityLevel,
in gstreamer-vaapi encoders:
The encoding quality could be set through this structure, if the
implementation supports multiple quality levels. The quality level set
through this structure is persistent over the entire coded sequence, or
until a new structure is being sent. The quality level range can be queried
through the VAConfigAttribEncQualityRange attribute. A lower value means
higher quality, and a value of 1 represents the highest quality. The quality
level setting is used as a trade-off between quality and speed/power
consumption, with higher quality corresponds to lower speed and higher power
consumption.
The quality level is set by the element's parameter "quality-level" with a
hard-coded range of 1 to 8.
Later, when the encoder is configured in run time, just before start
processing, the quality level is scaled to the codec range. If
VAConfigAttribEncQualityRange is not available in the used VA backend, then
the quality level is set to zero, which means "disabled".
All the available codecs now process this parameter if it is available.
https://bugzilla.gnome.org/show_bug.cgi?id=778733
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
Sometimes gst_vaapi_window_wayland_sync returns FALSE when poll returns EBUSY
during destruction.
In this case, if GstVaapiWindow is using vpp, leak of vpp surface happens.
This surface is not attached to anything at this moment, so we should release
it manually.
https://bugzilla.gnome.org/show_bug.cgi?id=781695
When the frame listener callbacks 'done', the number of pending
frames are decreased. Nonetheless, there might be occasions where
the buffer listener callbacks 'release', without calling previously
frame's 'done'. This leads to problem with
gst_vaapi_window_wayland_sync() operation.
This patch marks as done those frames which were callbacked, but if
the buffer callbacks 'release' and associated frame is not marked
as 'done' it is so, thus the number of pending frames keeps correct.
https://bugzilla.gnome.org/show_bug.cgi?id=780442
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
Don't call gst_vaapi_window_wayland_sync() when destroying the
wayland window instance, since it might lead to a lock at
gst_poll_wait() when more than one instances of vaapisink are
rendering in the same pipeline, this is because they share the
same window.
Since now all the frames are freed we don't need to freed the
private last_frame, since its address is invalid now.
https://bugzilla.gnome.org/show_bug.cgi?id=780442
Signed-off-by: Hyunjun Ko <zzoon@igalia.com>
Fix leakage of the last wl buffer.
VAAPI wayland sink needs to send a null buffer while destruction,
it assures that all the wl buffers are released. Otherwise, the last
buffer's callback might be not called, which leads to leak of
GstVaapiDisplay.
This was inspired by gstwaylandsink.
https://bugzilla.gnome.org/show_bug.cgi?id=774029
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
The proxy object of wl_buffer for the last frame remains in the
wl_map. Even though we call wl_buffer_destroy() in
frame_release_callback(), the proxy object remains without being
removed, since proxy object is deleted when wayland server sees the
delete request and sends 'delete_id' event.
We need to call roundtrip before destroying event_queue so that the
proxy object is removed. Otherwise, it would be mess up as receiving
'delete_id' event from previous play, when playing in the next
va/wayland window with the same wl_display connection.
https://bugzilla.gnome.org/show_bug.cgi?id=773689
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
Clear decoders out on a flush but keep the same instance,
rather than completely recreating them. That avoids
unecessarily freeing and recreating surface pools
and contexts, which can be quite expensive
https://bugzilla.gnome.org/show_bug.cgi?id=781142
The macro GST_VAAPI_OBJECT_DEFINE_CLASS_WITH_CODE only defines
a function that is never used, thus when compiling we might see
this warning (clang):
gstvaapiwindow.c:147:1: warning: unused function 'gst_vaapi_window_class' [-Wunused-function]
GST_VAAPI_OBJECT_DEFINE_CLASS_WITH_CODE (GstVaapiWindow,
^
https://bugzilla.gnome.org/show_bug.cgi?id=759533
Since gst_vaapi_window_vpp_convert_internal is created,
GstVaapiWindowX11/Wayland can use it for conversion.
Note that once it chooses to use vpp, it's going to use vpp
until the session is finished.
https://bugzilla.gnome.org/show_bug.cgi?id=759533
If a backend doesn't support specific format, we can use vpp for conversion
and make it playing.
This api is originated from GstVaapiWindowWayland and moved to GstVaapiWindow,
so that GstVaapiWindowX11 could use it.
https://bugzilla.gnome.org/show_bug.cgi?id=759533
Currently, GstVaapiWindowX11/Wayland are not descendants of GstVaapiWindow.
This patch chains them up to GstVaapiWindow to handle common members in GstVaapiWindow.
https://bugzilla.gnome.org/show_bug.cgi?id=759533
If the profile is main-10 the bit_depth_luma_minus8, in the sequence
parameter buffer, shall be the color format bit depth minus 8, 10-8
which is 2. Also for bit_depth_chroma_minus8.
This patch gets the negotiated sink caps format and queries its
luma's depth and uses that value to fill the mentioned parameters.
https://bugzilla.gnome.org/show_bug.cgi?id=771291
When the function gst_vaapi_encoder_get_surface_formats() was added
it was under the assumption that any VA profile of the specific codec
supported the same format colors. But it is not, for example the
profiles that support 10bit formats.
In other words, different VA profiles of a same codec may support
different color formats in their upload surfaces.
In order to expose all the possible color formats, if no profile is
specified via source caps, or if the encoder doesn't have yet a
context, all the possible VA profiles for the specific codec are
iterated and their color formats are merged.
https://bugzilla.gnome.org/show_bug.cgi?id=771291
In order to get the supported surface formats within a specific
profile this patch adds the GstVaapiProfile as property to
gst_vaapi_encoder_get_surface_formats().
Currently the extracted formats are only those related with the
default profile of the element's codec.
https://bugzilla.gnome.org/show_bug.cgi?id=771291
