Commit graph

4536 commits

Author SHA1 Message Date
Seungha Yang
5932a853f8 wasapi2ringbuffer: Close IAudioClient on GstAudioRingBuffer::release
IAudioClient interface is not reusable once it's initialized.
So we should close the handle and reopen it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2402>
2021-07-13 03:36:37 +09:00
Seungha Yang
fa4fb4c47f wasapi2ringbuffer: Run gst-indent
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2402>
2021-07-13 03:36:37 +09:00
Jakub Janků
211f968981 wasapi: fix reinit of audioclient in prepare()
When the sink goes from PLAYING to READY and then back to PLAYING,
the initialization of the audioclient in prepare() fails with the
error AUDCLNT_E_ALREADY_INITIALIZED. As a result, the playback
stops.

To fix this, we need to drop the AudioClient in unprepare() and
grab a new one in prepare() to be able to initialize it again
with the new buffer spec.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2096>
2021-07-12 13:47:07 +02:00
Jakub Janků
766a126380 wasapi: split gst_wasapi_util_get_device_client()
The functionality now resides in
gst_wasapi_util_get_device() and
gst_wasapi_util_get_audio_client().

This is a preparatory patch. It will be used in the following
patch to init/deinit the AudioClient separately from the device.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2096>
2021-07-12 13:47:06 +02:00
Yinhang Liu
2c70641b42 msdkvpp: use NV12 as default format on srcpad
By default, sinkpad is NV12 format and srcpad is BGRA format, the
different format will trigger an implicit format conversion in
msdkvpp, which will cause performance drop.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2394>
2021-07-09 17:31:10 +00:00
He Junyan
cbb1096c52 va: vpp: Improve the color properties setting.
The current setting of color properties are not very correct and
we will get some kind of "unknown Color Standard for YUV format"
warnings printed out by drivers. The video-color already provides
some standard APIs for us, and we can use them directly.
We also change the logic to: Finding the exactly match or explicit
standard first. If not found, we continue to find the most similar
one.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2385>
2021-07-08 13:36:05 +00:00
Seungha Yang
f57370c4db mfvideosrc: Fix negotiation when interlace-mode is specified
Given caps does not need to be strictly subset of device caps.
Allow accept it if device caps and requested caps can intersect

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1619
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2384>
2021-07-08 19:06:28 +09:00
Seungha Yang
2cd83fb4ef mfvideobuffer: Don't error for unexpected Unlock/Unlock2D call
Some GPU vendor's MFT implementation calls IMFMediaBuffer::Unlock()
without previous IMFMediaBuffer::Lock() call. Which is obviously
driver bug but we can ignore the Unlock call.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2381>
2021-07-07 16:07:28 +00:00
He Junyan
77a2ad5b1a va: basedec: Fix some artifact when do the crop copy.
The default video converter setting will add some artifact into
the picture for 10/12 bits conversion. This make the MD5 checksum
change from the original picture.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2379>
2021-07-05 14:36:36 +00:00
Seungha Yang
0d2664c668 d3d11decoder: Enable zero-copy for Qualcomm
Qualcomm GPU works fine with current implementation now.
Noticeable difference between when it was disabled and current
d3d11 implementation is that we now support GstD3D11Memory
pool, so there will be no more frequent re-binding decoder surface anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2377>
2021-07-05 12:17:55 +00:00
Seungha Yang
65a78effdb d3d11vp9dec: Fix for incorrect use_prev_in_find_mv_refs setting
Set use_prev_in_find_mv_refs depending on context. The value seems
to be used by AMD and Qualcomm (Intel and NVIDIA doesn't make difference
as per test)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2376>
2021-07-05 15:02:53 +09:00
Haihao Xiang
04de66ee7b gstmsdkvpp: add RGBP and BGRP in src pad
It requires MFX version 2.4+

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2234>
2021-07-05 03:47:05 +00:00
Seungha Yang
d4f8e3cf53 mediafoundation: Port to IID_PPV_ARGS
Make code short where possible

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2375>
2021-07-04 00:38:44 +09:00
Seungha Yang
f9699b2444 mfaudioenc: Remove pointless enumerating for hardware audio encoder
Hardware audio encoder can exist in theory, but it's untested
and we are not sure whether it can be preferred over software
implementation which is implemented by MS

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2375>
2021-07-04 00:38:33 +09:00
Seungha Yang
812b9eae1d mediafoundation: Fix typos
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2375>
2021-07-04 00:38:00 +09:00
Seungha Yang
3f8632dc96 mediafoundation: Run gst-indent
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2375>
2021-07-04 00:37:57 +09:00
He Junyan
b269cd5319 va: Consider the compatibility when we get_profile() for H265 decoder.
Adding the compatile profiles when we decide the final profile used for decoding.
The final profile candidates include:
1. The profile directly specified by SPS, which is the exact one.
2. The compatile profiles decided by the upstream element such as the h265parse.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2322>
2021-07-02 10:27:41 +00:00
He Junyan
ac51e41928 va: vp9dec: update segmentation and store the result.
The segmentation is stateful, its information may depend on the previous
segmentation setting. For example, if loop_filter_delta_enabled is TRUE,
the filter_level[GST_VP9_REF_FRAME_INTRA][1] should inherit the previous
frame's value and can not be calculated by the current frame's segmentation
data only. So we need to maintain the segmentation state inside the vp9
decoder and update it when the new frame header comes.

We also fix the CLAMP issue of lvl_seg and intra_lvl because of their wrong
uint type here.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2369>
2021-07-02 08:44:15 +00:00
He Junyan
fb4aeb6636 va: allocator: dma: Fail when mapping the non-linear buffer.
The current way of DMA buffer mapping is simply forwarding the job
to parent's map function, which is a mmap(). That can not handle the
non-linear buffers, such as tiling, compressed, etc. The incorrect
mapping of such buffers causes broken images, which are recognized
as bugs. We should directly block this kind of mapping to avoid the
misunderstanding.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2353>
2021-07-02 08:15:49 +00:00
Seungha Yang
d9c486352c mfvideoenc: Don't ignore previous flow return value
In case of ASYNC MFT (hardware encoder), we were ignoring previous
finish_frame or pad_push return value. so, error wasn't propagated.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2370>
2021-06-30 18:35:35 +09:00
Seungha Yang
ee543bdacf d3d11compositor: Add scaling policy to support PAR-aware scaling
Identical to https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1156
but for D3D11.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2263>
2021-06-30 06:10:29 +00:00
Ezequiel Garcia
3b42c1878d v4l2codecs: vp8: Check kernel version
Print a warning if the kernel version is too old.

Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2075>
2021-06-29 21:01:29 +00:00
Ezequiel Garcia
d036b4aad4 v4l2codecs: vp8: Update to the new uAPI
Starting from Linux v5.13, the V4L2 stateless VP8 uAPI
is updated and stable.

Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2075>
2021-06-29 21:01:29 +00:00
Seungha Yang
c8ffd39dfe nvcodec: Enhance CUDA runtime compiler library loading on Windows
The name of installed CUDA runtime compiler library is formed like
nvrtc64_{major-version}{minor-version}_0.dll on Windows
(which is differnt from documented in https://docs.nvidia.com/cuda/nvrtc/index.html)

And minor version might not be exactly same as that of CUDA.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2362>
2021-06-29 17:40:37 +00:00
Seungha Yang
44e3399bf8 d3d11: Add AV1 decoder
Introduce Direct3D11/DXVA AV1 decoder element

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2365>
2021-06-29 07:12:27 +00:00
He Junyan
a2d5223473 va: change AV1 GstVideoAlignment setting to left-top corner.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
abf6c51e83 va: h264dec: Set the GstVideoAlignment correctly.
We should set GstVideoAlignment based on the sequence's crop information.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
027726d6c8 va: h265dec: Set the GstVideoAlignment correctly.
We should set GstVideoAlignment based on the conformance window info.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
49cd009778 va: pool: Add VideoCropMeta to the buffer if crop_top/left > 0.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
85c56c1f07 va: basedec: Copy the frames into other_pool if needed.
If decoder's crop_top/left value > 0 and the downstream does not
support the VideoCropMeta, we need to manually copy the frames
into the other_pool and output it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
55302c9705 va: basedec: Setup the other_pool to copy output if crop_left/top.
If the decoder has crop_top/left value > 0(e.g. the conformance
window in the H265). Which means that the real output picture
locates in the middle of the decoded buffer. If the downstream can
support VideoCropMeta, a VideoCropMeta is added to notify the
real picture's coordinate and size. But if not, we need to copy
it manually and the other_pool is needed. We always assume that
decoded picture starts from top-left corner, and so there is no
need to do this if crop_bottom/right value > 0.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
c03350e234 va: No need to set the alignment for VideoMeta
The base va decoder's video_align is just used for calculation the
real decoded buffer's width and height. It does not have meaning
for the VideoMeta, because it does not align to the real picture
in the output buffer. We will use VideoCropMeta to replace it later.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
98cf9ce6f5 va: Delete the useless align expand in va_pool_set_config().
The base va decoder's video_align is just used for calculation the
real decoded buffer's width and height. While the gst_video_info_align
just calculate the offset and stride based on the video_align. But
all the offsets and strides are overwritten in gst_va_dmabuf_allocator_try
or gst_va_allocator_try, which make that calculation useless.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
Mengkejiergeli Ba
85c17f6c23 msdk: fix qp range for vp9enc
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2349>
2021-06-24 01:30:18 +00:00
Seungha Yang
7c94b9c4b0 d3d11: Add support for GRAY and more YUV formats
By this commit, following formats will be newly supported by d3d11 elements

* Y444_{8, 12, 16}LE formats:
  Similar to other planar formats. Such Y444 variants are not supported
  by Direct3D11 natively, but we can simply map each plane by
  using R8 and/or R16 texture.
* P012_LE:
  It is not different from P016_LE, but defining P012 and P016 separately
  for more explicit signalling. Note that DXVA uses P016 texture
  for 12bits encoded bitstreams.
* GRAY:
  This format is required for some codecs (e.g., AV1) if monochrome
  is supported
* 4:2:0 planar 12bits (I420_12LE) and 4:2:2 planar 8, 10, 12bits
  formats (Y42B, I422_10LE, and I422_12LE)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2346>
2021-06-23 15:35:36 +00:00
Seungha Yang
ba26a5aea8 mfvideoenc: Enhance B-frame timestamp handling
When B-frame is enabled, encoder seems to adjust PTS of encoded sample
by using frame duration.

For instance, one observed timestamp pattern by using B-frame enabled
and 30fps stream is:
* Frame-1: MF pts 0:00.033333300 MF dts 0:00.000000000
* Frame-2: MF pts 0:00.133333300 MF dts 0:00.033333300
* Frame-3: MF pts 0:00.066666600 MF dts 0:00.066666600
* Frame-4: MF pts 0:00.099999900 MF dts 0:00.100000000

We can notice that the amount of PTS shift is frame duration and
Frame-4 exhibits PTS < DTS.

To compensate shifted timestamp, we should
calculate the timestamp offset and re-calculate DTS correspondingly.
Otherwise, total timeline of output stream will be shifted, and that
can cause time sync issue.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2354>
2021-06-23 14:12:22 +00:00
Seungha Yang
569910a5ac mfh264enc, mfh265enc: Set profile string to src caps
Set configured profile to src caps so that downstream can figure
out selected profile.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2347>
2021-06-22 15:20:05 +00:00
Seungha Yang
ba83f29cec decklinkaudiosrc: Don't assume that stream time is always valid
As per SDK doc, IDeckLinkInputCallback::VideoInputFrameArrived
method might not provide video frame and it can be null.
In that case, given stream_time can be invalid.
So, we should not try to convert GST_CLOCK_TIME_NONE
by using gst_clock_adjust_with_calibration()

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2337>
2021-06-21 09:58:46 +00:00
Víctor Manuel Jáquez Leal
78f4777a82 msdk: Demote error log message to warning.
It is not an error that the available hardware doesn't support VA-API/MSDK. Just
none plugin features will be registered.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2336>
2021-06-21 06:30:07 +00:00
U. Artie Eoff
300f25ae36 msdk: declare external dependencies
Track kernel and VA driver dependencies so gstreamer
will re-inspect the plugin if any of them change.

Also, do not blacklist the plugin if !msdk_is_available
since it could be a transient issue caused by one or
more external dependency issues (e.g. wrong/missing
driver specified, but corrected by user later on).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2335>
2021-06-17 02:56:45 +00:00
He Junyan
8d7e6bfb86 va: Improve the default mapping between rt_format and video format.
We add 12 bits entries into this default mapping. And the old mapping
is not precise. For example, the NV12 should not be used as the default
mapping for VA_RT_FORMAT_YUV422 and VA_RT_FORMAT_YUV444, it is even not
a 422 or 444 format.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2332>
2021-06-16 14:12:59 +00:00
He Junyan
0849583210 va: Add 12 bits rt_format setting in H265.
In order to support 12 bits format decoding, we need to add the
support for 12 bits rt_format in H265.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2332>
2021-06-16 14:12:59 +00:00
He Junyan
1b4c9eaebb va: Fix a typo in video format mapping.
GST_VIDEO_FORMAT_Y412_LE is a 4:4:4 format and so should be mapped
to VA_RT_YUV444_12 rt format.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2332>
2021-06-16 14:12:59 +00:00
Nicolas Dufresne
4ac9f91921 alphadecodebin: Fix stall due to QoS
alphacombine element is a simple element that assumes buffers are always
paired, or at least that missing buffers are signalled with a GAP. The QoS
implementation in the GstVideoDecoder base class allow decoders dropping
frames independently and that could lead to stall in alphacombine.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2326>
2021-06-14 16:33:15 -04:00
Seungha Yang
a63539b213 d3d11decoder: Don't print error log when no DPB texture is available
... but we are flushing. The condition is quite expected situation
when pipeline is in the middle of seeking operation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2320>
2021-06-14 07:02:20 +00:00
Yinhang Liu
8530ac286a msdkenc: add extbrc support in ext-coding-props property
The SDK can support external bitrate control [1], so add extbrc
to enable this feature.

[1] https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#mfxextcodingoption2

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2139>
2021-06-11 02:00:51 +00:00
Yinhang Liu
ea5636af2c msdkenc: add ext-coding-props for external coding options
This property supports passing multiple parameters using GstStructure.

Example usage:
ext-coding-props="props,key0=value0,key1=value1,..."

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2139>
2021-06-11 02:00:51 +00:00
Seungha Yang
a1350852e7 wasapi2sink: Fix ringbuffer object leak
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2315>
2021-06-10 07:48:38 +00:00
Seungha Yang
cd9b96e143 wasapi2ringbuffer: Implement GstAudioRingBuffer::pause()
WASAPI doesn't support PAUSE so it's not different from Stop().
When pipeline is in paused state, we don't need to waste CPU resource
for feeding silent buffers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2315>
2021-06-10 07:48:38 +00:00
Seungha Yang
b7abd34285 wasapi2src: Add support for loopback recording
... and add various device error handling.

This loopback implementation is functionally identical to that of wasapisrc.
When it's enabled, wasapi2src will read data from render device instead of
capture device.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2311>
2021-06-09 22:15:06 +09:00