Commit graph

4202 commits

Author SHA1 Message Date
Seungha Yang
7c94b9c4b0 d3d11: Add support for GRAY and more YUV formats
By this commit, following formats will be newly supported by d3d11 elements

* Y444_{8, 12, 16}LE formats:
  Similar to other planar formats. Such Y444 variants are not supported
  by Direct3D11 natively, but we can simply map each plane by
  using R8 and/or R16 texture.
* P012_LE:
  It is not different from P016_LE, but defining P012 and P016 separately
  for more explicit signalling. Note that DXVA uses P016 texture
  for 12bits encoded bitstreams.
* GRAY:
  This format is required for some codecs (e.g., AV1) if monochrome
  is supported
* 4:2:0 planar 12bits (I420_12LE) and 4:2:2 planar 8, 10, 12bits
  formats (Y42B, I422_10LE, and I422_12LE)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2346>
2021-06-23 15:35:36 +00:00
Olivier Crête
a931e31141 webrtc lib: Make the datachannel struct private
This will prevent any unsafe access.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2241>
2021-06-21 20:53:09 +00:00
Olivier Crête
dd2da6f2b4 webrtc lib: Make the DTLSTransport struct private
This will prevent any unsafe access.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2241>
2021-06-21 20:53:09 +00:00
Olivier Crête
a0813c5bd2 webrtc lib: Make the icetransport struct private
This will prevent any unsafe access.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2241>
2021-06-21 20:53:09 +00:00
Olivier Crête
5233c349e7 webrtc lib: Make the rtpreceiver struct private
This will prevent any unsafe access.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2241>
2021-06-21 20:53:09 +00:00
Olivier Crête
a6593753a5 webrtc lib: Make the rtpsender struct private
This will prevent any unsafe access.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2241>
2021-06-21 20:53:09 +00:00
Olivier Crête
b5f2de3124 webrtc lib: Make the transceiver struct private
This will prevent any unsafe access.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2241>
2021-06-21 20:53:09 +00:00
Thibault Saunier
d7c716d562 transcoder: Fix usage of g_error_propagate
In the error callback we were propagating an error we were not owning
which is incorrect use of the API.

Also we were clearing a GError we already propagated which is wrong
as propagating gives ownership away.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2325>
2021-06-21 09:18:32 +00:00
Thibault Saunier
d85368eada transcoder: Add a missing object unlocking
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2325>
2021-06-21 09:18:32 +00:00
Seungha Yang
2c14a7253b libs: d3d11: Port to C++
In general, C++ COM APIs are slightly less verbose and more readable
than C APIs. And C++ supports some helper methods
(smart pointer and C++ only macros for example) which are not allowed for C.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2343>
2021-06-20 20:10:24 +09:00
Nicolas Dufresne
2696bcd9e2 vp8decoder: Drain the output queue on EOS/finish
The finish() virtual method was flushing the queue, instead push the
remaining buffers. It is not required to reset in finish() unlike
drain(). This a regression causing last frame to always be lost.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2333>
2021-06-16 17:34:54 +00:00
Nicolas Dufresne
e5e2b6a652 v4l2slvp8dec: Only ask for output delay once per negotiation
While it's technically possible to change it per frame, asking for
that every frame is not very useful. This mimic H264 decoder better.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2333>
2021-06-16 17:34:54 +00:00
Stéphane Cerveau
a71ec17cf0 jpeg2000parse, openjpeg: add support for YCrCb 4:1:1 sampling
Add YCrCb 4:1:1 support in openjpeg elements
and fix in jpeg2000parse the YCrCb 4:1:0 support

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2321>
2021-06-14 11:05:45 +02:00
He Junyan
2db3ce32ef codecs: Fix the H265 poc out of order warning.
We always get a warning such as:
h265decoder gsth265decoder.c:1432:gst_h265_decoder_do_output_picture: \
<vah265dec0> Outputting out of order 255 -> 0, likely a broken stream
in H265 decoder.

The problem is caused because we fail to reset the last_output_poc when
we get IDR and BLA. The incoming IDR and BLA frame already bump all the
frames in the DPB, but we forget to reset the last_output_poc, which
make the POC out of order and generate the warning all the time.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2294>
2021-06-10 12:36:34 +00:00
Seungha Yang
f90506e33b d3d11memory: Implement GstAllocator::mem_copy method
There are a few places which require deep copy
(basesink on drain for example). Also this implementation can be
useful for future use case.
One probable future use case is that copying DPB texture to
another texture for in-place transform since our DPB texture is never
writable, and therefore copying is unavoidable.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2308>
2021-06-10 01:20:32 +09:00
Seungha Yang
4d0f136297 vkinstance: Don't abort in case that system has no available vulkan device
Specification doesn't have restriction that returned
pPhysicalDeviceCount value must be non-zero

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2304>
2021-06-04 07:44:36 +00:00
Seungha Yang
74f81a1a13 d3d11: Add support for YV12 and NV21 formats
Handle UV swapped 4:2:0 8bits formats

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2290>
2021-06-03 17:00:49 +00:00
He Junyan
c442c9bd5e codecs: Integrate H265 DPB full check into need_bump().
The current DPB check of H265 is not very correct. The current frame
is already in the DPB when we check whether the DPB is full.
For example, the DPB max size is 16 and we have 15 ref frames in the
DPB, so the gst_h265_dpb_delete_unused() cleans no one, and then plus
the current frame, the DPB is 16. This causes an error return, but in
fact, the stream is correct.
We now integrate the DPB full check into the need_bump() function.
We add the correct frame into to DPB and then check whether the picture
num is bigger than max_num_pics of DPB(which means there is no room for
the current picture). If true, we bump the DPB immediately.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2291>
2021-06-02 13:10:35 +00:00
Daniel Almeida
82c0005793 codecs: gstvp8decoder: add support for render delay
Some decoding APIs support delayed output for performance reasons.
One example would be to request decoding for multiple frames and
then query for the oldest frame in the output queue.

This also increases throughput for transcoding and improves seek
performance when supported by the underlying backend.

Introduce support in the vp8 base class, so that backends that
support render delays can actually implement it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2150>
2021-05-31 15:48:24 -04:00
Seungha Yang
b1541a7470 d3d11: Suppress some warning logs
We uses gst_d3d11_device_new() for enumerating device which can
fail for some reason. Don't print warning log for the case.
And decoding capability check is the same case as well.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2286>
2021-05-29 10:30:28 +00:00
Víctor Manuel Jáquez Leal
9514340d2d libs: va: display: Handle auto clean up macros.
Add G_DEFINE_AUTOPTR_CLEANUP_FUNC macro for display classes, so auto
pointers are possible to users.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2269>
2021-05-26 05:42:33 +00:00
Seungha Yang
360a195158 d3d11memory: Protect map and unmap with device lock
We should lock memory object with gst_d3d11_device_lock() first
then GST_D3D11_MEMORY_LOCK() need to be used.

One observed deadlock case is that:
- Thread A takes d3d11 device lock
- At the same time, Thread B tries CPU map to d3d11memory which requires
  d3d11 device lock as well, but it's already taken by Thread A.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2267>
2021-05-20 11:19:44 +00:00
Seungha Yang
7a0bc2a91d d3d11memory: Add trace log for debugging locking thread
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2267>
2021-05-20 11:19:44 +00:00
Víctor Manuel Jáquez Leal
d09aae68a5 libs: va: Documentation and annotations.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2196>
2021-05-18 12:15:36 +02:00
He Junyan
c335f00d62 examples: va: Update the VA examples because of the new va lib.
Because we introduce the new va lib, the va examples need to include
new header files and add more library linkage.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2196>
2021-05-18 12:15:36 +02:00
Víctor Manuel Jáquez Leal
031b77ce97 libs: va: display_wrapper: Use gpointer for VADisplay.
In order to be coherent along all the implementation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2196>
2021-05-18 12:15:36 +02:00
He Junyan
e0915ce982 libs: va: Move the VA common logic as a lib.
The VA acceleration now has more usages in linux-like platforms,
such as the MSDK. The different plugins based on the VA acceleration
need to share some common logic and types. We now move the display
related functions and types into a common va lib.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2196>
2021-05-18 12:15:30 +02:00
Olivier Crête
e9f14ed117 webrtcbin: Hold lock while accessing the codec preferences
They could be changed at runtime by the application, so take the lock
when modifying them.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2183>
2021-05-13 15:05:00 -04:00
Olivier Crête
4c3270409d webrtc rtptransceiver: Implement "codec-preferences" property
This allows safer access to the internals of the codec-preferences

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2183>
2021-05-13 15:05:00 -04:00
Olivier Crête
97a78a903a webrtc rtptransceiver: Implement "kind" property
Implement the property as read-only to follow the WebRTC spec

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2183>
2021-05-13 15:05:00 -04:00
Olivier Crête
5fd0ee3227 webrtc rtptransceiver: Implement "current-direction" property
Implement the property as read-only to follow the WebRTC spec

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2183>
2021-05-13 15:05:00 -04:00
Olivier Crête
7e7678f4cb webrtc rtptransceiver: Implement "mid" property
Implement the property as read-only to follow the WebRTC spec

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2183>
2021-05-13 15:05:00 -04:00
Víctor Manuel Jáquez Leal
da27722351 libs: codecs: h264decoder: Assert output_picture virtual method.
For new code it's nice to assert if the derived class implemented the
output_picture virtual method. Otherwise a segmentation fault
occurs. All other decoders assert this method.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2228>
2021-05-07 20:20:59 +00:00
Daniel Almeida
4900e3583a codecs: gstvp9statefulparser: do not carry over segmentation flags
Do not carry over segmentation flags from previous frames. The spec
says in 7.2.10 that the feature data carry over from previous frames
if not updated, but the flags do not.

Consider what would happen if a flag B is to depend on a flag A, and
B carries over as set from another frame. Further consider that A is
now not set in this particular frame. This leads to the invalid state
in which flag B is set but flag A isn't.

This might cause the bitstream to be rejected by accelerators down
the line.

Fix it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2203>
2021-04-30 16:49:28 +00:00
Seungha Yang
817544860d d3d11: Add support for BGRx and RGBx formats
For such formats, we can re-use existing BGRA/RGBA implementations
but ignoring alpha channel

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2174>
2021-04-21 05:45:59 +00:00
Seungha Yang
fee725f3cc codecs: vp9decoder: Update docs
* Remove "FIXME 1.20": All the bits are addressed already by using
  vp9parse element
* Fix copy & paste errors: Some comments were copied from h264decoder
  blindly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2151>
2021-04-09 12:13:24 +00:00
Seungha Yang
f7a341a1f0 codecs: vp9decoder: Make duplicate_picture() vfunc optional
The default implementation was required when superframe parsing
was handled by vp9decoder. For instance, if a superframe consists
of multiple frames with show_existing_frame header, it was vague
that which GstVp9Picture should consume GstVideoCodecFrame.

After 1.18 release, we introduced vp9parse element and
superframe should be handled by upstream vp9parse elemenet now.
So, we don't need to care about the superframe at vp9decoder class
level anymore. Simply, a frame corresponding to show_existing_frame
can be dropped if subclass doesn't implement duplicate_picture().

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2151>
2021-04-09 12:13:24 +00:00
Matthieu De Beule
f449a553a7 Tell programmers that set_volume uses linear scale (fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1439)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1722>
2021-04-07 22:54:37 +00:00
Seungha Yang
50e116c4a7 codecs: vp9decoder: Allow decoding start with intra-only frame
As per spec "7.2 Uncompressed header semantics" and
"8.2 Frame order constraints", decoding can start with intra-only
frame. This commit is for fixing vp90-2-16-intra-only.webm
bitstream test failure.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
2021-04-07 19:32:29 +00:00
Seungha Yang
6eb9856f59 codecs: vp9decoder: Pass GstVideoCodecFrame to duplicate_picture()
... and fix picture duplication logic for vavp9dec

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
2021-04-07 19:32:29 +00:00
Seungha Yang
1f769839c0 codecs: vp9decoder: Port to GstVp9StatefulParser
Use newly implemented VP9 parser. Since new GstVp9FrameHeader
struct holds all the information of the stream, baseclass will not
pass parser object to new_sequence() method anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
2021-04-07 19:32:29 +00:00
Seungha Yang
c36190cbda codecparsers: Reimplement VP9 parser
Existing VP9 parser implementation doesn't provide information
required by other stateless decoding APIs (i.e., DXVA and NVDEC),
specifically loop filter and segmentation parameters might not exist
current frame. So parser needs to fill the information by using previously
parsed information.
We can update the gstvp9parser implementation so that it can provide
all information required by stateless decoding APIs with a huge API break,
or adding more ugly struct in it.
Instead doing as such, this commit introduce a new VP9 parser implementation.

What is different from existing one?
* All variables will follow the specification as much as possible:
  VP9 Bitstream & Decoding Process Specification - v0.6 31st March 2016
* Parser will fill all the required information for decoding frame
  to GstVp9FrameHeader struct. In case of old VP9 parser,
  user needs to read additional data from parser's member variables.
* GstVp9StatefulParser object struct is completely completely opaque

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
2021-04-07 19:32:29 +00:00
Seungha Yang
771326a0fb codecs: vp9decoder: Don't check codec change with show_existing_frame
Show existing frame will zero frame_type value but it doesn't mean
it's keyframe.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
2021-04-07 19:32:29 +00:00
Wim Taymans
cefdd0d0f1 vulkan: provide a custom VK_DEFINE_NON_DISPATCHABLE_HANDLE
If the application did not define one yet, define our own
VK_DEFINE_NON_DISPATCHABLE_HANDLE that is independent of the
architecture.

Vulkan, by default, provides a define that depends on the architecture,
which causes the symbol type to be different. This causes an
architecture dependent .gir file, which then causes multilib
installation problems because the .gir files can't be shared.

Make it possible to override the format specifier and provide
a default one that is compatible with the default non dispatchable
handle.

Return VK_NULL_HANDLE from functions that return a non-dispatchable
handle.

Fixes #1566

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2130>
2021-04-06 12:32:57 +02:00
Marijn Suijten
061e32b197 Add @ prefix to enum-variant references in documentation
Found while working on GStreamer-rs documentation, some enums had this
bit of text pasted verbatim in the enum documentation rather than
attached to the enum-variant.  Fortunately it seems these in WebRTC and
D3D11 are the only ones matching the non-@-prefixed pattern:

    ^ \* GST_\w+:\s*\w+

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2118>
2021-03-28 13:08:24 +00:00
Seungha Yang
76e5e479be codecs: vp9decoder: Fix to output frame when show_existing_frame flag is set
When show_existing_frame flag is set, show_frame flag is zero
but we should output previously decoded frame as specified in frame header.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2114>
2021-03-26 14:50:18 +00:00
Seungha Yang
90181642a3 d3d11decoder: Resurrect zero-copy for fixed-size DPB pool
Enable zero-copy if downstream proposed pool and therefore decoder
can know the amount of buffer required by downstream.
Otherwise decoder will copy when our DPB pool has no sufficient
buffers for later decoding operation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Seungha Yang
39b9f79e11 d3d11: Implement memory pool
Major changes:
* GstD3D11Allocator: This allocator is now device-independent object
  which can allocate GstD3D11Memory object for any GstD3D11Device.
  User can get this object via gst_allocator_find(GST_D3D11_MEMORY_NAME)
* GstD3D11PoolAllocator: A new allocator implementation for texture pool.
  From now on GstD3D11BufferPool will make use of this memory pool allocator
  to avoid frequent texture reallocation. That usually happens because
  of buffer copy (gst_buffer_make_writable for example)

In addition to that, GstD3D11BufferPool will provide GstBuffer with
GstVideoMeta, because CPU access to a GstD3D11Memory without GstVideoMeta
is almost impossible since GPU drivers needs padding for stride alignment.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Seungha Yang
4e7e390cab d3d11decoder: Temporarily remove zero-copy related code
We will re-implement it based on memory pool

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Sebastian Dröge
ffa4d84e54 h2645parser: Catch overflows in AVC/HEVC NAL unit length calculations
Offset and size are stored as 32 bit guint and might overflow when
adding the nal_length_size, so let's avoid that.

For the size this would happen if the AVC/HEVC NAL unit size happens to
be stored in 4 bytes and is 4294967292 or higher, which is likely
corrupted data anyway.

For the offset this is something for the caller of these functions to
take care of but is unlikely to happen as it would require parsing on a
>4GB buffer.

Allowing these overflows causes all kinds of follow-up bugs in the
h2645parse elements, ranging from infinite loops and memory leaks to
potential memory corruptions.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2103>
2021-03-24 09:22:48 +00:00