Commit graph

4518 commits

Author SHA1 Message Date
He Junyan
fb4aeb6636 va: allocator: dma: Fail when mapping the non-linear buffer.
The current way of DMA buffer mapping is simply forwarding the job
to parent's map function, which is a mmap(). That can not handle the
non-linear buffers, such as tiling, compressed, etc. The incorrect
mapping of such buffers causes broken images, which are recognized
as bugs. We should directly block this kind of mapping to avoid the
misunderstanding.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2353>
2021-07-02 08:15:49 +00:00
Seungha Yang
d9c486352c mfvideoenc: Don't ignore previous flow return value
In case of ASYNC MFT (hardware encoder), we were ignoring previous
finish_frame or pad_push return value. so, error wasn't propagated.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2370>
2021-06-30 18:35:35 +09:00
Seungha Yang
ee543bdacf d3d11compositor: Add scaling policy to support PAR-aware scaling
Identical to https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1156
but for D3D11.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2263>
2021-06-30 06:10:29 +00:00
Ezequiel Garcia
3b42c1878d v4l2codecs: vp8: Check kernel version
Print a warning if the kernel version is too old.

Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2075>
2021-06-29 21:01:29 +00:00
Ezequiel Garcia
d036b4aad4 v4l2codecs: vp8: Update to the new uAPI
Starting from Linux v5.13, the V4L2 stateless VP8 uAPI
is updated and stable.

Signed-off-by: Ezequiel Garcia <ezequiel@collabora.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2075>
2021-06-29 21:01:29 +00:00
Seungha Yang
c8ffd39dfe nvcodec: Enhance CUDA runtime compiler library loading on Windows
The name of installed CUDA runtime compiler library is formed like
nvrtc64_{major-version}{minor-version}_0.dll on Windows
(which is differnt from documented in https://docs.nvidia.com/cuda/nvrtc/index.html)

And minor version might not be exactly same as that of CUDA.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2362>
2021-06-29 17:40:37 +00:00
Seungha Yang
44e3399bf8 d3d11: Add AV1 decoder
Introduce Direct3D11/DXVA AV1 decoder element

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2365>
2021-06-29 07:12:27 +00:00
He Junyan
a2d5223473 va: change AV1 GstVideoAlignment setting to left-top corner.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
abf6c51e83 va: h264dec: Set the GstVideoAlignment correctly.
We should set GstVideoAlignment based on the sequence's crop information.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
027726d6c8 va: h265dec: Set the GstVideoAlignment correctly.
We should set GstVideoAlignment based on the conformance window info.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
49cd009778 va: pool: Add VideoCropMeta to the buffer if crop_top/left > 0.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
85c56c1f07 va: basedec: Copy the frames into other_pool if needed.
If decoder's crop_top/left value > 0 and the downstream does not
support the VideoCropMeta, we need to manually copy the frames
into the other_pool and output it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
55302c9705 va: basedec: Setup the other_pool to copy output if crop_left/top.
If the decoder has crop_top/left value > 0(e.g. the conformance
window in the H265). Which means that the real output picture
locates in the middle of the decoded buffer. If the downstream can
support VideoCropMeta, a VideoCropMeta is added to notify the
real picture's coordinate and size. But if not, we need to copy
it manually and the other_pool is needed. We always assume that
decoded picture starts from top-left corner, and so there is no
need to do this if crop_bottom/right value > 0.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
c03350e234 va: No need to set the alignment for VideoMeta
The base va decoder's video_align is just used for calculation the
real decoded buffer's width and height. It does not have meaning
for the VideoMeta, because it does not align to the real picture
in the output buffer. We will use VideoCropMeta to replace it later.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
He Junyan
98cf9ce6f5 va: Delete the useless align expand in va_pool_set_config().
The base va decoder's video_align is just used for calculation the
real decoded buffer's width and height. While the gst_video_info_align
just calculate the offset and stride based on the video_align. But
all the offsets and strides are overwritten in gst_va_dmabuf_allocator_try
or gst_va_allocator_try, which make that calculation useless.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2298>
2021-06-28 15:16:39 +00:00
Mengkejiergeli Ba
85c17f6c23 msdk: fix qp range for vp9enc
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2349>
2021-06-24 01:30:18 +00:00
Seungha Yang
7c94b9c4b0 d3d11: Add support for GRAY and more YUV formats
By this commit, following formats will be newly supported by d3d11 elements

* Y444_{8, 12, 16}LE formats:
  Similar to other planar formats. Such Y444 variants are not supported
  by Direct3D11 natively, but we can simply map each plane by
  using R8 and/or R16 texture.
* P012_LE:
  It is not different from P016_LE, but defining P012 and P016 separately
  for more explicit signalling. Note that DXVA uses P016 texture
  for 12bits encoded bitstreams.
* GRAY:
  This format is required for some codecs (e.g., AV1) if monochrome
  is supported
* 4:2:0 planar 12bits (I420_12LE) and 4:2:2 planar 8, 10, 12bits
  formats (Y42B, I422_10LE, and I422_12LE)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2346>
2021-06-23 15:35:36 +00:00
Seungha Yang
ba26a5aea8 mfvideoenc: Enhance B-frame timestamp handling
When B-frame is enabled, encoder seems to adjust PTS of encoded sample
by using frame duration.

For instance, one observed timestamp pattern by using B-frame enabled
and 30fps stream is:
* Frame-1: MF pts 0:00.033333300 MF dts 0:00.000000000
* Frame-2: MF pts 0:00.133333300 MF dts 0:00.033333300
* Frame-3: MF pts 0:00.066666600 MF dts 0:00.066666600
* Frame-4: MF pts 0:00.099999900 MF dts 0:00.100000000

We can notice that the amount of PTS shift is frame duration and
Frame-4 exhibits PTS < DTS.

To compensate shifted timestamp, we should
calculate the timestamp offset and re-calculate DTS correspondingly.
Otherwise, total timeline of output stream will be shifted, and that
can cause time sync issue.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2354>
2021-06-23 14:12:22 +00:00
Seungha Yang
569910a5ac mfh264enc, mfh265enc: Set profile string to src caps
Set configured profile to src caps so that downstream can figure
out selected profile.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2347>
2021-06-22 15:20:05 +00:00
Seungha Yang
ba83f29cec decklinkaudiosrc: Don't assume that stream time is always valid
As per SDK doc, IDeckLinkInputCallback::VideoInputFrameArrived
method might not provide video frame and it can be null.
In that case, given stream_time can be invalid.
So, we should not try to convert GST_CLOCK_TIME_NONE
by using gst_clock_adjust_with_calibration()

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2337>
2021-06-21 09:58:46 +00:00
Víctor Manuel Jáquez Leal
78f4777a82 msdk: Demote error log message to warning.
It is not an error that the available hardware doesn't support VA-API/MSDK. Just
none plugin features will be registered.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2336>
2021-06-21 06:30:07 +00:00
U. Artie Eoff
300f25ae36 msdk: declare external dependencies
Track kernel and VA driver dependencies so gstreamer
will re-inspect the plugin if any of them change.

Also, do not blacklist the plugin if !msdk_is_available
since it could be a transient issue caused by one or
more external dependency issues (e.g. wrong/missing
driver specified, but corrected by user later on).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2335>
2021-06-17 02:56:45 +00:00
He Junyan
8d7e6bfb86 va: Improve the default mapping between rt_format and video format.
We add 12 bits entries into this default mapping. And the old mapping
is not precise. For example, the NV12 should not be used as the default
mapping for VA_RT_FORMAT_YUV422 and VA_RT_FORMAT_YUV444, it is even not
a 422 or 444 format.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2332>
2021-06-16 14:12:59 +00:00
He Junyan
0849583210 va: Add 12 bits rt_format setting in H265.
In order to support 12 bits format decoding, we need to add the
support for 12 bits rt_format in H265.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2332>
2021-06-16 14:12:59 +00:00
He Junyan
1b4c9eaebb va: Fix a typo in video format mapping.
GST_VIDEO_FORMAT_Y412_LE is a 4:4:4 format and so should be mapped
to VA_RT_YUV444_12 rt format.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2332>
2021-06-16 14:12:59 +00:00
Nicolas Dufresne
4ac9f91921 alphadecodebin: Fix stall due to QoS
alphacombine element is a simple element that assumes buffers are always
paired, or at least that missing buffers are signalled with a GAP. The QoS
implementation in the GstVideoDecoder base class allow decoders dropping
frames independently and that could lead to stall in alphacombine.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2326>
2021-06-14 16:33:15 -04:00
Seungha Yang
a63539b213 d3d11decoder: Don't print error log when no DPB texture is available
... but we are flushing. The condition is quite expected situation
when pipeline is in the middle of seeking operation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2320>
2021-06-14 07:02:20 +00:00
Yinhang Liu
8530ac286a msdkenc: add extbrc support in ext-coding-props property
The SDK can support external bitrate control [1], so add extbrc
to enable this feature.

[1] https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#mfxextcodingoption2

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2139>
2021-06-11 02:00:51 +00:00
Yinhang Liu
ea5636af2c msdkenc: add ext-coding-props for external coding options
This property supports passing multiple parameters using GstStructure.

Example usage:
ext-coding-props="props,key0=value0,key1=value1,..."

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2139>
2021-06-11 02:00:51 +00:00
Seungha Yang
a1350852e7 wasapi2sink: Fix ringbuffer object leak
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2315>
2021-06-10 07:48:38 +00:00
Seungha Yang
cd9b96e143 wasapi2ringbuffer: Implement GstAudioRingBuffer::pause()
WASAPI doesn't support PAUSE so it's not different from Stop().
When pipeline is in paused state, we don't need to waste CPU resource
for feeding silent buffers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2315>
2021-06-10 07:48:38 +00:00
Seungha Yang
b7abd34285 wasapi2src: Add support for loopback recording
... and add various device error handling.

This loopback implementation is functionally identical to that of wasapisrc.
When it's enabled, wasapi2src will read data from render device instead of
capture device.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2311>
2021-06-09 22:15:06 +09:00
Seungha Yang
a8ec40c850 wasapi2: Rewrite plugin and implement audioringbuffer subclass
... based on MediaFoundation work queue API.

By this commit, wasapi2 plugin will make use of pull mode scheduling
with audioringbuffer subclass.
There are several drawbacks of audiosrc/audiosink subclassing
(not audiobasesrc/audiobasesink) for WASAPI API, which are:
* audiosrc/audiosink classes try to set high priority to
  read/write thread via MMCSS (Multimedia Class Scheduler Service)
  but it's not allowed in case of UWP application.
  In order to use MMCSS in UWP, application should use MediaFoundation
  work queue indirectly.
  Since audiosrc/audiosink scheduling model is not compatible with
  MediaFoundation's work queue model, audioringbuffer subclassing
  is required.
* WASAPI capture device might report larger packet size than expected
  (i.e., larger frames we can read than expected frame size per period).
  Meanwhile, in any case, application should drain all packets at that moment.
  In order to handle the case, wasapi/wasapi2 plugins were making use of
  GstAdapter which is obviously sub-optimal because it requires additional
  memory allocation and copy.
  By implementing audioringbuffer subclassing, we can avoid such inefficiency.

In this commit, all the device read/write operations will be moved
to newly implemented wasapi2ringbuffer class and
existing wasapi2client class will take care of device enumeration
and activation parts only.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2306>
2021-06-08 19:39:27 +09:00
Seungha Yang
4b42671c99 wasapi2: Use AUDCLNT_STREAMFLAGS_NOPERSIST flag
... so that we can disable persistence of our mute/volume status

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2306>
2021-06-08 19:39:26 +09:00
Seungha Yang
d5384ffbd0 wasapi2src: Fix doc typo
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2306>
2021-06-08 19:39:26 +09:00
Seungha Yang
74f81a1a13 d3d11: Add support for YV12 and NV21 formats
Handle UV swapped 4:2:0 8bits formats

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2290>
2021-06-03 17:00:49 +00:00
Seungha Yang
73067bfe0c d3d11window_win32: Ensure closing internal HWND from window thread
Window handle must be closed from its own message thread

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2302>
2021-06-03 15:20:27 +00:00
Yinhang Liu
a997524e3f doc: add the msdk elements
Supported elements:
msdkav1dec, msdkh264dec, msdkh264enc, msdkh265dec, msdkh265enc,
msdkmjpegdec, msdkmjpegenc, msdkmpeg2dec, msdkmpeg2enc, msdkvc1dec,
msdkvp8dec, msdkvp9dec, msdkvp9enc, msdkvpp.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2296>
2021-06-03 10:30:18 +08:00
Daniel Almeida
89bc464360 v4l2codecs: gstv4l2codecsvp8dec: implement a render delay
The v4l2 backend support delayed output for performance reasons.
It is then possible to use render delays to queue multiple requests
simultaneously, thus increasing performance.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2150>
2021-05-31 15:48:24 -04:00
Seungha Yang
b1541a7470 d3d11: Suppress some warning logs
We uses gst_d3d11_device_new() for enumerating device which can
fail for some reason. Don't print warning log for the case.
And decoding capability check is the same case as well.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2286>
2021-05-29 10:30:28 +00:00
Seungha Yang
fed06df5f6 d3d11compositor: Reuse converter on alpha update
... instead of creating converter object

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2276>
2021-05-28 21:44:12 +09:00
Seungha Yang
0d34a0233b d3d11converter: Introduce config to be extensible
Add a config argument like that of GstVideoConverter so that
we can add more options without modifying existing methods

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2276>
2021-05-28 21:44:10 +09:00
Seungha Yang
f3331652f2 d3d11shader: Don't hold state object in GstD3D11Quad
We might want to update state object

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2276>
2021-05-28 21:44:07 +09:00
Nicolas Dufresne
49992be643 v4lcodecs: Validate src formats
This add src format validation, this avoid registering element for
drivers we don't support any of their src formats. This also special
case the AlphaDecodeBin wrapper, as we know that alphacombine element
only support I420 and NV12 for now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2272>
2021-05-27 17:01:11 -04:00
Daniel Almeida
9e86ac4a22 v4l2codecs: add wrappers for alpha decode
codecalpha is a new plugin introduced to support VP8/VP9 alpha as
defined in the WebM and Matroska specifications. It splits the stream
into two streams, one for the alpha and one for the actual content,
then it decodes them separately with vpxdec and finally combine the
results as A420 or AV12 (i.e. YUV + an extra alpha plane).

The workflow above is setup by means of a bin, gstcodecalphabin.

This patch simulates the same workflow into the v4l2codecs namespace,
thus using the new v4l2 stateless decoders for hardware acceleration.

This is so we can register the new alpha decode elements only if the
hardware produces formats we support, i.e. I420 or NV12 for now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2272>
2021-05-27 17:00:06 -04:00
Seungha Yang
da33921f8d d3d11compositor: Fix caps update handling
New caps is applied only when previous buffer is consumed if any.
So, the lastest given caps might not be corresponding to the current buffer
to be handled.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2284>
2021-05-26 16:42:39 +09:00
He Junyan
c4bb5f06c9 va: h265dec: Add current picture into reference list for SCC.
The current picture is not in the DPB, so we need to add it manually
to the reference list when SCC is enabled.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2255>
2021-05-25 12:11:05 +02:00
He Junyan
ceb2df1751 va: h265dec: Set Screen Content extension (SCC) for picture parameters.
We already declare the support of HEVC screen content extension profiles
in the profile mapping list, but we fail to generate the correct VA picture
parameters buffers. This may cause the GPU hang.

We need to fill the buffer of VAPictureParameterBufferHEVCExtension correctly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2255>
2021-05-25 12:11:05 +02:00
He Junyan
bfbb49144c va: h265dec: Use get_profile_from_sps() to recognize the profile.
The function of gst_h265_get_profile_from_sps() is better than the
function gst_h265_profile_tier_level_get_profile() when we recognize
the profile of the stream, becaue it considers the compatibility.

It is also used by h265parse to recognize the profile. So it is
better to keep the same behaviour with the parse and other decoders.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2255>
2021-05-25 12:11:05 +02:00
He Junyan
08dc70b6f4 va: h265dec: Set range extension for picture and slice parameters.
We already declare the support of HEVC range extension profiles in
the profile mapping list, but we fail to generate the correct VA
picture and slice parameters buffers. This may cause the GPU hang.

We need to fill the buffer of VAPictureParameterBufferHEVCExtension
and VASliceParameterBufferHEVCExtension correctly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2255>
2021-05-25 12:11:05 +02:00