Commit graph

4167 commits

Author SHA1 Message Date
Seungha Yang
fee725f3cc codecs: vp9decoder: Update docs
* Remove "FIXME 1.20": All the bits are addressed already by using
  vp9parse element
* Fix copy & paste errors: Some comments were copied from h264decoder
  blindly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2151>
2021-04-09 12:13:24 +00:00
Seungha Yang
f7a341a1f0 codecs: vp9decoder: Make duplicate_picture() vfunc optional
The default implementation was required when superframe parsing
was handled by vp9decoder. For instance, if a superframe consists
of multiple frames with show_existing_frame header, it was vague
that which GstVp9Picture should consume GstVideoCodecFrame.

After 1.18 release, we introduced vp9parse element and
superframe should be handled by upstream vp9parse elemenet now.
So, we don't need to care about the superframe at vp9decoder class
level anymore. Simply, a frame corresponding to show_existing_frame
can be dropped if subclass doesn't implement duplicate_picture().

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2151>
2021-04-09 12:13:24 +00:00
Matthieu De Beule
f449a553a7 Tell programmers that set_volume uses linear scale (fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1439)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1722>
2021-04-07 22:54:37 +00:00
Seungha Yang
50e116c4a7 codecs: vp9decoder: Allow decoding start with intra-only frame
As per spec "7.2 Uncompressed header semantics" and
"8.2 Frame order constraints", decoding can start with intra-only
frame. This commit is for fixing vp90-2-16-intra-only.webm
bitstream test failure.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
2021-04-07 19:32:29 +00:00
Seungha Yang
6eb9856f59 codecs: vp9decoder: Pass GstVideoCodecFrame to duplicate_picture()
... and fix picture duplication logic for vavp9dec

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
2021-04-07 19:32:29 +00:00
Seungha Yang
1f769839c0 codecs: vp9decoder: Port to GstVp9StatefulParser
Use newly implemented VP9 parser. Since new GstVp9FrameHeader
struct holds all the information of the stream, baseclass will not
pass parser object to new_sequence() method anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
2021-04-07 19:32:29 +00:00
Seungha Yang
c36190cbda codecparsers: Reimplement VP9 parser
Existing VP9 parser implementation doesn't provide information
required by other stateless decoding APIs (i.e., DXVA and NVDEC),
specifically loop filter and segmentation parameters might not exist
current frame. So parser needs to fill the information by using previously
parsed information.
We can update the gstvp9parser implementation so that it can provide
all information required by stateless decoding APIs with a huge API break,
or adding more ugly struct in it.
Instead doing as such, this commit introduce a new VP9 parser implementation.

What is different from existing one?
* All variables will follow the specification as much as possible:
  VP9 Bitstream & Decoding Process Specification - v0.6 31st March 2016
* Parser will fill all the required information for decoding frame
  to GstVp9FrameHeader struct. In case of old VP9 parser,
  user needs to read additional data from parser's member variables.
* GstVp9StatefulParser object struct is completely completely opaque

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
2021-04-07 19:32:29 +00:00
Seungha Yang
771326a0fb codecs: vp9decoder: Don't check codec change with show_existing_frame
Show existing frame will zero frame_type value but it doesn't mean
it's keyframe.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
2021-04-07 19:32:29 +00:00
Wim Taymans
cefdd0d0f1 vulkan: provide a custom VK_DEFINE_NON_DISPATCHABLE_HANDLE
If the application did not define one yet, define our own
VK_DEFINE_NON_DISPATCHABLE_HANDLE that is independent of the
architecture.

Vulkan, by default, provides a define that depends on the architecture,
which causes the symbol type to be different. This causes an
architecture dependent .gir file, which then causes multilib
installation problems because the .gir files can't be shared.

Make it possible to override the format specifier and provide
a default one that is compatible with the default non dispatchable
handle.

Return VK_NULL_HANDLE from functions that return a non-dispatchable
handle.

Fixes #1566

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2130>
2021-04-06 12:32:57 +02:00
Marijn Suijten
061e32b197 Add @ prefix to enum-variant references in documentation
Found while working on GStreamer-rs documentation, some enums had this
bit of text pasted verbatim in the enum documentation rather than
attached to the enum-variant.  Fortunately it seems these in WebRTC and
D3D11 are the only ones matching the non-@-prefixed pattern:

    ^ \* GST_\w+:\s*\w+

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2118>
2021-03-28 13:08:24 +00:00
Seungha Yang
76e5e479be codecs: vp9decoder: Fix to output frame when show_existing_frame flag is set
When show_existing_frame flag is set, show_frame flag is zero
but we should output previously decoded frame as specified in frame header.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2114>
2021-03-26 14:50:18 +00:00
Seungha Yang
90181642a3 d3d11decoder: Resurrect zero-copy for fixed-size DPB pool
Enable zero-copy if downstream proposed pool and therefore decoder
can know the amount of buffer required by downstream.
Otherwise decoder will copy when our DPB pool has no sufficient
buffers for later decoding operation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Seungha Yang
39b9f79e11 d3d11: Implement memory pool
Major changes:
* GstD3D11Allocator: This allocator is now device-independent object
  which can allocate GstD3D11Memory object for any GstD3D11Device.
  User can get this object via gst_allocator_find(GST_D3D11_MEMORY_NAME)
* GstD3D11PoolAllocator: A new allocator implementation for texture pool.
  From now on GstD3D11BufferPool will make use of this memory pool allocator
  to avoid frequent texture reallocation. That usually happens because
  of buffer copy (gst_buffer_make_writable for example)

In addition to that, GstD3D11BufferPool will provide GstBuffer with
GstVideoMeta, because CPU access to a GstD3D11Memory without GstVideoMeta
is almost impossible since GPU drivers needs padding for stride alignment.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Seungha Yang
4e7e390cab d3d11decoder: Temporarily remove zero-copy related code
We will re-implement it based on memory pool

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
2021-03-24 15:44:42 +00:00
Sebastian Dröge
ffa4d84e54 h2645parser: Catch overflows in AVC/HEVC NAL unit length calculations
Offset and size are stored as 32 bit guint and might overflow when
adding the nal_length_size, so let's avoid that.

For the size this would happen if the AVC/HEVC NAL unit size happens to
be stored in 4 bytes and is 4294967292 or higher, which is likely
corrupted data anyway.

For the offset this is something for the caller of these functions to
take care of but is unlikely to happen as it would require parsing on a
>4GB buffer.

Allowing these overflows causes all kinds of follow-up bugs in the
h2645parse elements, ranging from infinite loops and memory leaks to
potential memory corruptions.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2103>
2021-03-24 09:22:48 +00:00
Matthew Waters
640a65bf96 gst: don't use volatile to mean atomic
volatile is not sufficient to provide atomic guarantees and real atomics
should be used instead.  GCC 11 has started warning about using volatile
with atomic operations.

https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1719

Discovered in https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/issues/868

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2098>
2021-03-22 14:34:36 +11:00
Seungha Yang
824b0ce0f4 Revert "d3d11: Enable native multi-thread protection layer and make use of it"
This reverts commit 872b7f503c.

Native multi-thread protection layer seems to be consuming more CPU
resource than application side protection approach in some cases

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2095>
2021-03-20 17:53:58 +09:00
Seungha Yang
872b7f503c d3d11: Enable native multi-thread protection layer and make use of it
... instead of our own GRecMutex locking. In this way, any other
Direct3D11 client (MediaFoundation for example) can safely call
any Direct3D11 API even when we are sharing our Direct3D11 device
with others.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2092>
2021-03-18 16:37:37 +00:00
Seungha Yang
615f52f2f7 d3d11device: Hold ID3D11VideoDevice and ID3D11VideoContext object
... instead of QueryInterface-ing per elements. Note that
ID3D11VideoDevice and ID3D11VideoContext objects might not be available
if device doesn't support video interface.
So GstD3D11Device object will create those objects only when requested.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 14:44:55 +09:00
Seungha Yang
cdce27d975 d3d11device: Fix wrong printf formatting
Add missing '%' there

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2069>
2021-03-11 13:31:18 +09:00
Philippe Normand
a6b5e3c7d5 transcoder: Add state-changed signal
Similar to GstPlayer, a new signal for state tracking is now emitted at runtime,
as a commodity for applications which then don't need to monitor the pipeline
GstBus for state changes anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2028>
2021-03-10 17:13:57 +00:00
Philippe Normand
12a7bf9f7c player: Rewrite as GstPlay wrapper
For the time being the GstPlayer library remains as a wrapper for GstPlay, in
order to keep existing applications working and give them time to port to
GstPlay. GstPlayer will remain in -bad for a couple cycles and the plan for
GstPlay is to move it to -base before 1.20.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
2021-03-09 18:03:48 +00:00
Philippe Normand
f009802771 play: Flush API bus before exiting main loop
Otherwise the bus might attempt to dispatch queued messages after the thread
ended, causing runtime warnings.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
2021-03-09 18:03:48 +00:00
Philippe Normand
19d57dcfa5 play: Plug media_info leak
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
2021-03-09 18:03:48 +00:00
Stephan Hesse
e007c47c3f play: Rename internal buffering field to buffering_percent
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
2021-03-09 18:03:48 +00:00
Stephan Hesse
a74651ddc1 play: Introducing the new playback library
This aims to be a replacement for the GstPlayer library. In GstPlay, notifications are
sent as application messages through a dedicated GstBus. The GMainContext-based
signal dispatcher was replaced by a GObject signal adapter, now relying on the
bus to emit its signals. The signal dispatcher is now optional and fully
decoupled from the GstPlay object.

Co-authored with: Philippe Normand <philn@igalia.com>

Fixes #394

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
2021-03-09 18:03:48 +00:00
Seungha Yang
646c34794f d3d11memory: Fix for wrong texture_array_size returns
Fix mismatched return values

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2051>
2021-03-02 23:10:46 +09:00
Seungha Yang
a4c6130477 d3d11: Fix wrong preprocessing blocks
Missed in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2051>
2021-03-02 23:10:46 +09:00
Thibault Saunier
1d03fdcc09 transcoder: Add some missing API guards
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2044>
2021-03-01 12:37:44 +00:00
Thibault Saunier
d252ab2e06 transcoder: Fix potential use of uninitialized variables
gst_structure_get won't touch variables if the field is not present
leading to potential use of initialized vars

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2044>
2021-03-01 12:37:44 +00:00
Thibault Saunier
8c2fd4fff7 transcoder: Rework the API to create/get SignalAdapter
We can only have 1 single GstTranscoderSignalAdapter object for a
given GstTranscoder object, this enforces that by avoiding to expose
a constructor and instead add a method to GstTranscoder to get the
signal adapter (internally creating it when needed). We can still
cleanly ensure that the signal adapter is running for the requested
GMainContext and return NULL if it is not the case.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2044>
2021-03-01 12:37:44 +00:00
Philippe Normand
552007f839 transcoder: Remove un-needed gst_init call
We can safely assume GStreamer is already initialized from here.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1840>
2021-02-26 15:37:23 +00:00
Thibault Saunier
984f0c2d2f transcoder: Port to a GstBus API instead
Following the move made by GstPlayer in:

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/35

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1840>
2021-02-26 15:37:23 +00:00
Thibault Saunier
7cf6e4d8f2 transcoder: Automatically generate enums GTypes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1840>
2021-02-26 15:37:23 +00:00
Thibault Saunier
0a1dcc9546 transcoder: Port to G_DECLARE
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1840>
2021-02-26 15:37:23 +00:00
He Junyan
315b101394 codecs: AV1decoder: Add the AV1 decoder base class.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1636>
2021-02-23 13:50:51 +08:00
Seungha Yang
0ee13755a8 d3d11: Add a method for conversion from DXGI format to GstVideoFormat
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2016>
2021-02-17 03:48:10 +09:00
He Junyan
636a77a83f codecs: mpeg2decoder: Move frame_unref to handle_frame.
In the current code, we call frame_unref only when the frame is
outputted. This is OK for normal playback, but when seek happens,
the frames stored in DPB is not outputted and causes some memory
leak.
The correct way is that we should call frame_unref every time we
finish the handle_frame(), which is also the behaviour of H264/H265
decoder.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2014>
2021-02-11 16:18:29 +08:00
Seungha Yang
f49f039ff2 codecs: h265decoder: Add support for interlaced stream
* Invoke GstH265DecoderClass::new_sequence() method per interlaced
  stream status update so that subclass can update caps.
* Parse picture timing SEI and set buffer flags on GstH265Picture
  object. Subclass can refer to it like that of our h264decoder
  implementation.
* Remove pointless GstH265PictureField enum

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2008>
2021-02-09 16:19:41 +00:00
Seungha Yang
12baab442c d3d11memory: Add a method for querying texture array size
... and the number of textures in use.

Direct3D11 texture array is usually used for decoder DPB pool,
and d3d11 decoder elements might want to know
whether there's available texture resource or not.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2003>
2021-02-03 20:46:22 +00:00
Seungha Yang
86e312c1b1 codecs: h264decoder: Add support for output delay
Some decoding APIs support delayed output or a command for decoding
a frame doesn't need to be sequential to corresponding command for
getting decoded frame. For instance, subclass might be able to
request decoding for multiple frames and then get for one (oldest)
decoded frame or so.
If aforementioned case is supported by specific decoding API,
delayed output might show better throughput performance.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1925>
2021-01-29 16:22:28 +00:00
Seungha Yang
0b0bf1b0bf d3d11: Suppress some warning debug messages
* Don't warn for live object, since ID3D11Debug itself seems to be
  holding refcount of ID3D11Device at the moment we called
  ID3D11Debug::ReportLiveDeviceObjects(). It would report live object
  always
* Device might not be able to support some formats (e.g., P010)
  especially in case of WARP device. We don't need to warn about that.
* gst_d3d11_device_new() can be used for device enumeration. Don't warn
  even if we cannot create D3D11 device with given adapter index therefore.
* Don't warn for HLSL compiler warning. It's just noise and
  should not be critical thing at all

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1986>
2021-01-27 04:46:42 +09:00
He Junyan
e8ec8f7dd7 codecparsers: av1: Fix a typo in frame_restoration_type setting.
Fixes: #1500
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1974>
2021-01-23 10:53:44 +00:00
Matthew Waters
82a9e4630c vulkan: remove duplicated check
Checking the same value twice is pointless

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1504

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1977>
2021-01-22 12:38:09 +00:00
He Junyan
b93315fa6c codecparsers: Exclude the size of obu_size when identify OBU.
obu->obu_size does not contain the bytes of obu_size itself, we need
to exclude it when doing the saint check.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1614>
2021-01-19 18:38:03 +00:00
He Junyan
057fedd178 codecs: mpeg2decoder: Fix a typo in header file's comment.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1963>
2021-01-20 00:57:34 +08:00
Seungha Yang
d16b5b7f7f d3d11: Allow building UWP features with Desktop features if possible
WINAPI_PARTITION_DESKTOP and WINAPI_PARTITION_APP can coexist.
Although UWP only binaries should be used for production stage,
this change will be useful for development stage

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1962>
2021-01-19 11:23:56 +00:00
Mathieu Duponchelle
86c009e7aa webrtc: expose transport property on sender and receiver
As advised by !1366#note_629558 , the nice transport should be
accessed through:

> transceiver->sender/receiver->transport/rtcp_transport->icetransport

All the objects on the path can be accessed through properties
except sender/receiver->transport. This patch addresses that.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1952>
2021-01-13 19:22:42 +00:00
Seungha Yang
0f7af4b143 d3d11: Move core methods to gst-libs
Move d3d11 device, memory, buffer pool and minimal method
to gst-libs so that other plugins can access d3d11 resource.
Since Direct3D is primary graphics API on Windows, we need
this infrastructure for various plugins can share GPU resource
without downloading GPU memory.
Note that this implementation is public only for -bad scope
for now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
2021-01-13 15:01:20 +00:00
Seungha Yang
ae7c20a6ed codecs: h264picture: Count only complete complementary field pair for dpb fullness decision
Our DPB implementation was designed as such that allowing
temporary DPB overflow in the middle of field picture decoding
and incomplete field pair should not trigger DPB bumping.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1947>
2021-01-10 23:27:27 +09:00