Commit graph

4548 commits

Author SHA1 Message Date
Víctor Manuel Jáquez Leal
de19e8a84f va: vp9dec: Minor cleanups.
Added a comment with a future to-do, enhanced another comment and
fixed a typo in an error log message.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2417>
2021-07-17 20:53:32 +02:00
Víctor Manuel Jáquez Leal
b45754b478 va: decoder: Group decoder methods.
Move up gst_va_decoder_get_config() to group decoders function in the
same file area.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2417>
2021-07-17 20:53:29 +02:00
Víctor Manuel Jáquez Leal
ea8b372ceb va: Refactor _format_changed() to _config_is_equal().
Change gst_va_decoder_format_changed() to
gst_va_decoder_config_is_equal(), which is more similar with other
GStreamer API.

The function call is replaced but it has to be negated because the
return value is the opposite.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2417>
2021-07-17 20:51:29 +02:00
Víctor Manuel Jáquez Leal
d2d7ce7157 va: Refactor _change_resolution() to _update_frame_size().
Rename gst_va_decoder_change_resolution() to
gst_va_decoder_update_frame_size() which resembles
gst_va_decoder_set_frame_size().

Also added a comment to clarify the function use and makes more
specific the error message.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2417>
2021-07-17 20:51:29 +02:00
Víctor Manuel Jáquez Leal
1bf2805560 va: Refactor _set_format() to _set_frame_size().
Renamed gst_va_decoder_set_format() to
gst_va_decoder_set_frame_size_with_surfaces() which resembles better
the passed parameters. Internally it creates the vaContext.

Added gst_va_decoder_set_frame_size() which is an alias of
gst_va_decoder_set_frame_size_with_surfaces() without surfaces. This
is the function which replaces gst_va_decoder_set_format() where
used.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2417>
2021-07-17 20:51:27 +02:00
He Junyan
4299596d16 codecs: h265decoder: Fix a typo of NumPocTotalCurr when process ref pic list.
We should use the NumPocTotalCurr value stored in decoder, which is a calculated
valid value, rather than use the invalid value in the slice header. Most of the
time, the NumPocTotalCurr is 0 and make the tmp_refs a very short length, and
causes the decoder's wrong result.
By the way, the NumPocTotalCurr is not the correct name specified in H265 spec,
its name should be NumPicTotalCurr. We change it to the correct name.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2414>
2021-07-16 23:05:18 +08:00
He Junyan
6a9f84a2de va: h265dec: Do not add non reference frames into ref list.
The VA's ReferenceFrames should only contain the reference frame, we
should not add the non reference frames into this list.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2414>
2021-07-16 15:50:47 +08:00
Seungha Yang
9becc1fe7e mfvideoenc: Disable RGB format support
Some GPUs support BGRA format and it will be converted to subsampled
YUV format by GPU internally. Disable this implicit conversion
since the conversion parameters such as input/output colorimetry
are not exposed nor it's written in bitstream (e.g., VUI).
We prefer explicit conversion via our conversion elements.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2410>
2021-07-15 16:50:26 +00:00
He Junyan
67d482f8fb va: h265dec: Fix a temp var overflow bug when write pred weight table.
The temp guint8 var of delta_chroma_offset_l0 and delta_chroma_offset_l1
can not cover the full range of delta_chroma_weight_l0/1 in the slice
header. When overflow happens, the decoder result is wrong.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2412>
2021-07-15 14:25:09 +00:00
He Junyan
ff0e42eb30 va: vp9dec: We need to check the resolution changes for every frame.
The VP9 streams have the ability to change the resolution dynamically
at any time point. It does not send ad KEY frame before change the
resolution, even the INTER frame can change the resolution immediately.
So we need to check the resolution change for each frame and do the
re-negiotiation if needed.

Some insaned stream may play in resolution A first and then dynamically
changes to B, and after 1 or 2 frames, it use a show_existing_frame to
repeat the old frame of resolution A before. So, not only new_picture(),
but also duplicate_picture() need to check this.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2407>
2021-07-15 09:42:51 +00:00
He Junyan
458ac4a693 va: vp9dec: Do not re-create context for dynamical resolution change.
The driver for VP9 should have the ability to handle the dynamical resolution
changes. So if only the resolution changes, we should not re-create the config
and context in negotiation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2407>
2021-07-15 09:42:51 +00:00
He Junyan
6887e77201 va: decoder: Add helper functions to get and change the resolution.
Some codecs such as VP9, its config and context have the ability to
dynamically. When we only change the width and height, no need to
re-create the config and context. The helper function can just change
the resolution without re-creating config and context.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2407>
2021-07-15 09:42:51 +00:00
Seungha Yang
5932a853f8 wasapi2ringbuffer: Close IAudioClient on GstAudioRingBuffer::release
IAudioClient interface is not reusable once it's initialized.
So we should close the handle and reopen it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2402>
2021-07-13 03:36:37 +09:00
Seungha Yang
fa4fb4c47f wasapi2ringbuffer: Run gst-indent
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2402>
2021-07-13 03:36:37 +09:00
Jakub Janků
211f968981 wasapi: fix reinit of audioclient in prepare()
When the sink goes from PLAYING to READY and then back to PLAYING,
the initialization of the audioclient in prepare() fails with the
error AUDCLNT_E_ALREADY_INITIALIZED. As a result, the playback
stops.

To fix this, we need to drop the AudioClient in unprepare() and
grab a new one in prepare() to be able to initialize it again
with the new buffer spec.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2096>
2021-07-12 13:47:07 +02:00
Jakub Janků
766a126380 wasapi: split gst_wasapi_util_get_device_client()
The functionality now resides in
gst_wasapi_util_get_device() and
gst_wasapi_util_get_audio_client().

This is a preparatory patch. It will be used in the following
patch to init/deinit the AudioClient separately from the device.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2096>
2021-07-12 13:47:06 +02:00
Yinhang Liu
2c70641b42 msdkvpp: use NV12 as default format on srcpad
By default, sinkpad is NV12 format and srcpad is BGRA format, the
different format will trigger an implicit format conversion in
msdkvpp, which will cause performance drop.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2394>
2021-07-09 17:31:10 +00:00
He Junyan
cbb1096c52 va: vpp: Improve the color properties setting.
The current setting of color properties are not very correct and
we will get some kind of "unknown Color Standard for YUV format"
warnings printed out by drivers. The video-color already provides
some standard APIs for us, and we can use them directly.
We also change the logic to: Finding the exactly match or explicit
standard first. If not found, we continue to find the most similar
one.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2385>
2021-07-08 13:36:05 +00:00
Seungha Yang
f57370c4db mfvideosrc: Fix negotiation when interlace-mode is specified
Given caps does not need to be strictly subset of device caps.
Allow accept it if device caps and requested caps can intersect

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1619
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2384>
2021-07-08 19:06:28 +09:00
Seungha Yang
2cd83fb4ef mfvideobuffer: Don't error for unexpected Unlock/Unlock2D call
Some GPU vendor's MFT implementation calls IMFMediaBuffer::Unlock()
without previous IMFMediaBuffer::Lock() call. Which is obviously
driver bug but we can ignore the Unlock call.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2381>
2021-07-07 16:07:28 +00:00
He Junyan
77a2ad5b1a va: basedec: Fix some artifact when do the crop copy.
The default video converter setting will add some artifact into
the picture for 10/12 bits conversion. This make the MD5 checksum
change from the original picture.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2379>
2021-07-05 14:36:36 +00:00
Seungha Yang
0d2664c668 d3d11decoder: Enable zero-copy for Qualcomm
Qualcomm GPU works fine with current implementation now.
Noticeable difference between when it was disabled and current
d3d11 implementation is that we now support GstD3D11Memory
pool, so there will be no more frequent re-binding decoder surface anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2377>
2021-07-05 12:17:55 +00:00
Seungha Yang
65a78effdb d3d11vp9dec: Fix for incorrect use_prev_in_find_mv_refs setting
Set use_prev_in_find_mv_refs depending on context. The value seems
to be used by AMD and Qualcomm (Intel and NVIDIA doesn't make difference
as per test)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2376>
2021-07-05 15:02:53 +09:00
Haihao Xiang
04de66ee7b gstmsdkvpp: add RGBP and BGRP in src pad
It requires MFX version 2.4+

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2234>
2021-07-05 03:47:05 +00:00
Seungha Yang
d4f8e3cf53 mediafoundation: Port to IID_PPV_ARGS
Make code short where possible

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2375>
2021-07-04 00:38:44 +09:00
Seungha Yang
f9699b2444 mfaudioenc: Remove pointless enumerating for hardware audio encoder
Hardware audio encoder can exist in theory, but it's untested
and we are not sure whether it can be preferred over software
implementation which is implemented by MS

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2375>
2021-07-04 00:38:33 +09:00
Seungha Yang
812b9eae1d mediafoundation: Fix typos
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2375>
2021-07-04 00:38:00 +09:00
Seungha Yang
3f8632dc96 mediafoundation: Run gst-indent
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2375>
2021-07-04 00:37:57 +09:00
He Junyan
b269cd5319 va: Consider the compatibility when we get_profile() for H265 decoder.
Adding the compatile profiles when we decide the final profile used for decoding.
The final profile candidates include:
1. The profile directly specified by SPS, which is the exact one.
2. The compatile profiles decided by the upstream element such as the h265parse.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2322>
2021-07-02 10:27:41 +00:00
He Junyan
ac51e41928 va: vp9dec: update segmentation and store the result.
The segmentation is stateful, its information may depend on the previous
segmentation setting. For example, if loop_filter_delta_enabled is TRUE,
the filter_level[GST_VP9_REF_FRAME_INTRA][1] should inherit the previous
frame's value and can not be calculated by the current frame's segmentation
data only. So we need to maintain the segmentation state inside the vp9
decoder and update it when the new frame header comes.

We also fix the CLAMP issue of lvl_seg and intra_lvl because of their wrong
uint type here.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2369>
2021-07-02 08:44:15 +00:00
He Junyan
fb4aeb6636 va: allocator: dma: Fail when mapping the non-linear buffer.
The current way of DMA buffer mapping is simply forwarding the job
to parent's map function, which is a mmap(). That can not handle the
non-linear buffers, such as tiling, compressed, etc. The incorrect
mapping of such buffers causes broken images, which are recognized
as bugs. We should directly block this kind of mapping to avoid the
misunderstanding.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2353>
2021-07-02 08:15:49 +00:00
Seungha Yang
d9c486352c mfvideoenc: Don't ignore previous flow return value
In case of ASYNC MFT (hardware encoder), we were ignoring previous
finish_frame or pad_push return value. so, error wasn't propagated.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2370>
2021-06-30 18:35:35 +09:00
Seungha Yang
ee543bdacf d3d11compositor: Add scaling policy to support PAR-aware scaling
Identical to https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1156
but for D3D11.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2263>
2021-06-30 06:10:29 +00:00
Ezequiel Garcia
3b42c1878d v4l2codecs: vp8: Check kernel version
Print a warning if the kernel version is too old.

Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2075>
2021-06-29 21:01:29 +00:00
Ezequiel Garcia
d036b4aad4 v4l2codecs: vp8: Update to the new uAPI
Starting from Linux v5.13, the V4L2 stateless VP8 uAPI
is updated and stable.

Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2075>
2021-06-29 21:01:29 +00:00
Seungha Yang
c8ffd39dfe nvcodec: Enhance CUDA runtime compiler library loading on Windows
The name of installed CUDA runtime compiler library is formed like
nvrtc64_{major-version}{minor-version}_0.dll on Windows
(which is differnt from documented in https://docs.nvidia.com/cuda/nvrtc/index.html)

And minor version might not be exactly same as that of CUDA.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2362>
2021-06-29 17:40:37 +00:00
Seungha Yang
44e3399bf8 d3d11: Add AV1 decoder
Introduce Direct3D11/DXVA AV1 decoder element

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2365>
2021-06-29 07:12:27 +00:00
He Junyan
a2d5223473 va: change AV1 GstVideoAlignment setting to left-top corner.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
abf6c51e83 va: h264dec: Set the GstVideoAlignment correctly.
We should set GstVideoAlignment based on the sequence's crop information.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
027726d6c8 va: h265dec: Set the GstVideoAlignment correctly.
We should set GstVideoAlignment based on the conformance window info.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
49cd009778 va: pool: Add VideoCropMeta to the buffer if crop_top/left > 0.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
85c56c1f07 va: basedec: Copy the frames into other_pool if needed.
If decoder's crop_top/left value > 0 and the downstream does not
support the VideoCropMeta, we need to manually copy the frames
into the other_pool and output it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
55302c9705 va: basedec: Setup the other_pool to copy output if crop_left/top.
If the decoder has crop_top/left value > 0(e.g. the conformance
window in the H265). Which means that the real output picture
locates in the middle of the decoded buffer. If the downstream can
support VideoCropMeta, a VideoCropMeta is added to notify the
real picture's coordinate and size. But if not, we need to copy
it manually and the other_pool is needed. We always assume that
decoded picture starts from top-left corner, and so there is no
need to do this if crop_bottom/right value > 0.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
c03350e234 va: No need to set the alignment for VideoMeta
The base va decoder's video_align is just used for calculation the
real decoded buffer's width and height. It does not have meaning
for the VideoMeta, because it does not align to the real picture
in the output buffer. We will use VideoCropMeta to replace it later.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
98cf9ce6f5 va: Delete the useless align expand in va_pool_set_config().
The base va decoder's video_align is just used for calculation the
real decoded buffer's width and height. While the gst_video_info_align
just calculate the offset and stride based on the video_align. But
all the offsets and strides are overwritten in gst_va_dmabuf_allocator_try
or gst_va_allocator_try, which make that calculation useless.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
Mengkejiergeli Ba
85c17f6c23 msdk: fix qp range for vp9enc
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2349>
2021-06-24 01:30:18 +00:00
Seungha Yang
7c94b9c4b0 d3d11: Add support for GRAY and more YUV formats
By this commit, following formats will be newly supported by d3d11 elements

* Y444_{8, 12, 16}LE formats:
  Similar to other planar formats. Such Y444 variants are not supported
  by Direct3D11 natively, but we can simply map each plane by
  using R8 and/or R16 texture.
* P012_LE:
  It is not different from P016_LE, but defining P012 and P016 separately
  for more explicit signalling. Note that DXVA uses P016 texture
  for 12bits encoded bitstreams.
* GRAY:
  This format is required for some codecs (e.g., AV1) if monochrome
  is supported
* 4:2:0 planar 12bits (I420_12LE) and 4:2:2 planar 8, 10, 12bits
  formats (Y42B, I422_10LE, and I422_12LE)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2346>
2021-06-23 15:35:36 +00:00
Seungha Yang
ba26a5aea8 mfvideoenc: Enhance B-frame timestamp handling
When B-frame is enabled, encoder seems to adjust PTS of encoded sample
by using frame duration.

For instance, one observed timestamp pattern by using B-frame enabled
and 30fps stream is:
* Frame-1: MF pts 0:00.033333300 MF dts 0:00.000000000
* Frame-2: MF pts 0:00.133333300 MF dts 0:00.033333300
* Frame-3: MF pts 0:00.066666600 MF dts 0:00.066666600
* Frame-4: MF pts 0:00.099999900 MF dts 0:00.100000000

We can notice that the amount of PTS shift is frame duration and
Frame-4 exhibits PTS < DTS.

To compensate shifted timestamp, we should
calculate the timestamp offset and re-calculate DTS correspondingly.
Otherwise, total timeline of output stream will be shifted, and that
can cause time sync issue.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2354>
2021-06-23 14:12:22 +00:00
Seungha Yang
569910a5ac mfh264enc, mfh265enc: Set profile string to src caps
Set configured profile to src caps so that downstream can figure
out selected profile.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2347>
2021-06-22 15:20:05 +00:00
Seungha Yang
ba83f29cec decklinkaudiosrc: Don't assume that stream time is always valid
As per SDK doc, IDeckLinkInputCallback::VideoInputFrameArrived
method might not provide video frame and it can be null.
In that case, given stream_time can be invalid.
So, we should not try to convert GST_CLOCK_TIME_NONE
by using gst_clock_adjust_with_calibration()

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2337>
2021-06-21 09:58:46 +00:00