Existing VP9 parser implementation doesn't provide information
required by other stateless decoding APIs (i.e., DXVA and NVDEC),
specifically loop filter and segmentation parameters might not exist
current frame. So parser needs to fill the information by using previously
parsed information.
We can update the gstvp9parser implementation so that it can provide
all information required by stateless decoding APIs with a huge API break,
or adding more ugly struct in it.
Instead doing as such, this commit introduce a new VP9 parser implementation.
What is different from existing one?
* All variables will follow the specification as much as possible:
VP9 Bitstream & Decoding Process Specification - v0.6 31st March 2016
* Parser will fill all the required information for decoding frame
to GstVp9FrameHeader struct. In case of old VP9 parser,
user needs to read additional data from parser's member variables.
* GstVp9StatefulParser object struct is completely completely opaque
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
If the application did not define one yet, define our own
VK_DEFINE_NON_DISPATCHABLE_HANDLE that is independent of the
architecture.
Vulkan, by default, provides a define that depends on the architecture,
which causes the symbol type to be different. This causes an
architecture dependent .gir file, which then causes multilib
installation problems because the .gir files can't be shared.
Make it possible to override the format specifier and provide
a default one that is compatible with the default non dispatchable
handle.
Return VK_NULL_HANDLE from functions that return a non-dispatchable
handle.
Fixes#1566
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2130>
Found while working on GStreamer-rs documentation, some enums had this
bit of text pasted verbatim in the enum documentation rather than
attached to the enum-variant. Fortunately it seems these in WebRTC and
D3D11 are the only ones matching the non-@-prefixed pattern:
^ \* GST_\w+:\s*\w+
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2118>
Enable zero-copy if downstream proposed pool and therefore decoder
can know the amount of buffer required by downstream.
Otherwise decoder will copy when our DPB pool has no sufficient
buffers for later decoding operation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
Major changes:
* GstD3D11Allocator: This allocator is now device-independent object
which can allocate GstD3D11Memory object for any GstD3D11Device.
User can get this object via gst_allocator_find(GST_D3D11_MEMORY_NAME)
* GstD3D11PoolAllocator: A new allocator implementation for texture pool.
From now on GstD3D11BufferPool will make use of this memory pool allocator
to avoid frequent texture reallocation. That usually happens because
of buffer copy (gst_buffer_make_writable for example)
In addition to that, GstD3D11BufferPool will provide GstBuffer with
GstVideoMeta, because CPU access to a GstD3D11Memory without GstVideoMeta
is almost impossible since GPU drivers needs padding for stride alignment.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
Offset and size are stored as 32 bit guint and might overflow when
adding the nal_length_size, so let's avoid that.
For the size this would happen if the AVC/HEVC NAL unit size happens to
be stored in 4 bytes and is 4294967292 or higher, which is likely
corrupted data anyway.
For the offset this is something for the caller of these functions to
take care of but is unlikely to happen as it would require parsing on a
>4GB buffer.
Allowing these overflows causes all kinds of follow-up bugs in the
h2645parse elements, ranging from infinite loops and memory leaks to
potential memory corruptions.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2103>
... instead of QueryInterface-ing per elements. Note that
ID3D11VideoDevice and ID3D11VideoContext objects might not be available
if device doesn't support video interface.
So GstD3D11Device object will create those objects only when requested.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
For the time being the GstPlayer library remains as a wrapper for GstPlay, in
order to keep existing applications working and give them time to port to
GstPlay. GstPlayer will remain in -bad for a couple cycles and the plan for
GstPlay is to move it to -base before 1.20.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
This aims to be a replacement for the GstPlayer library. In GstPlay, notifications are
sent as application messages through a dedicated GstBus. The GMainContext-based
signal dispatcher was replaced by a GObject signal adapter, now relying on the
bus to emit its signals. The signal dispatcher is now optional and fully
decoupled from the GstPlay object.
Co-authored with: Philippe Normand <philn@igalia.com>
Fixes#394
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
We can only have 1 single GstTranscoderSignalAdapter object for a
given GstTranscoder object, this enforces that by avoiding to expose
a constructor and instead add a method to GstTranscoder to get the
signal adapter (internally creating it when needed). We can still
cleanly ensure that the signal adapter is running for the requested
GMainContext and return NULL if it is not the case.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2044>
In the current code, we call frame_unref only when the frame is
outputted. This is OK for normal playback, but when seek happens,
the frames stored in DPB is not outputted and causes some memory
leak.
The correct way is that we should call frame_unref every time we
finish the handle_frame(), which is also the behaviour of H264/H265
decoder.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2014>
* Invoke GstH265DecoderClass::new_sequence() method per interlaced
stream status update so that subclass can update caps.
* Parse picture timing SEI and set buffer flags on GstH265Picture
object. Subclass can refer to it like that of our h264decoder
implementation.
* Remove pointless GstH265PictureField enum
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2008>
Some decoding APIs support delayed output or a command for decoding
a frame doesn't need to be sequential to corresponding command for
getting decoded frame. For instance, subclass might be able to
request decoding for multiple frames and then get for one (oldest)
decoded frame or so.
If aforementioned case is supported by specific decoding API,
delayed output might show better throughput performance.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1925>
* Don't warn for live object, since ID3D11Debug itself seems to be
holding refcount of ID3D11Device at the moment we called
ID3D11Debug::ReportLiveDeviceObjects(). It would report live object
always
* Device might not be able to support some formats (e.g., P010)
especially in case of WARP device. We don't need to warn about that.
* gst_d3d11_device_new() can be used for device enumeration. Don't warn
even if we cannot create D3D11 device with given adapter index therefore.
* Don't warn for HLSL compiler warning. It's just noise and
should not be critical thing at all
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1986>
As advised by !1366#note_629558 , the nice transport should be
accessed through:
> transceiver->sender/receiver->transport/rtcp_transport->icetransport
All the objects on the path can be accessed through properties
except sender/receiver->transport. This patch addresses that.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1952>
Move d3d11 device, memory, buffer pool and minimal method
to gst-libs so that other plugins can access d3d11 resource.
Since Direct3D is primary graphics API on Windows, we need
this infrastructure for various plugins can share GPU resource
without downloading GPU memory.
Note that this implementation is public only for -bad scope
for now.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
Spec says:
In a frame picture top_field_first being set to ‘1’ indicates that the
top field of the reconstructed frame is the first field output by the
decoding process. top_field_first being set to ‘0’ indicates that the
bottom field of the reconstructed frame is the first field output by
decoding process.
Here, the "output" should be interpreted just as the output order, not
including the decoding order. The field should be decoded as the order
they comes in the stream. Namely, no matter top_field_first is 0 or 1,
the first coming field is the first one to be decoded.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1929>
Instead of implementing exactly the same thing ourself but making
`GstBus` not know that it is the case.
Since we are *sure* that the bus can't have been access at the point
where we add the watch we are guaranteed that the current thread
maincontext is going to be used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1870>
Context creation and retrieval is required, the symbols are exported
with the header missing. Users most likely define GST_USE_UNSTABLE_API
so they're aware of the implications of using a header that might change
between releases.
Signed-off-by: Marius Vlad <marius.vlad@collabora.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1688>
In practice, we encounter streams that have one or more temporal units
error. When that kind of error temporal units is in annex b format, the
whole temporal unit should be discarded.
But the temporal units before it are correct and can be used. More
important, because of the error temporal unit, the parser is in a wrong
state and all later temporal unit are also parsed uncorrectly.
We need to add this API to reset the annex_b state only when we meet
some temporal unit error.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1464>
1. store more tile info when parse tile group.
The column, row, tile offset and tile data size are all useful for
decoder process, especially for HW kind decoder such as VAAPI dec.
Also fix the tile group skip size for each tile data.
2. No min_inner_tile_width requirement in newest spec.
3. Calculate the sbs of each tile for both uniform tile and non-uniformi
tile.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1464>
The metadata OBUs, for example, ITUT_T35 has an undefined payload such
as itu_t_t35_payload_bytes field in AV1 spec, which may cause the failure
of parsing the trailings bits. We can give a warning and ignore this kind
of errors.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1464>
In case that "pic_order_cnt_type" is equal to zero, ref picture
list for B slice should not include non-existing picture
as per spec 8.2.4.2.3. And, the second field is not needed
for the process of frame picture reference list construction
since it needs to be frame unit, not field picture in that case.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1812>
We were trying to construct reference picture list even for
I slice before this commit. Reference list is required only for
P, SP and B slices. Also, if all existing reference pictures
are gap pictures, we don't need to construct lists.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1812>
Initial support for interlaced stream. Subclass should implement
new_field_picture() vfunc. Otherwise, baseclass will assume that
subclass doesn't support interlaced stream.
Restrictions:
* Reference picture modification process for interlaced stream
is not implemented yet
* PAFF (Picture Adaptive Frame Field) is not properly implemented.
* Field display ordering (e.g., top-field-first) decision should
be enhanced via picture timing SEI parsing
* Gap in field picture should be handled
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1534>
Managing reference picture type by using two variables
(ref and long_term) seems to be redundant and that can be
represented by using a single enum value.
This is to sync this implementation with gstreamer-vaapi so that
make comparison between this and gstreamer-vaapi easier and also
in order to minimize the change required for subclass to be able
to support interlaced.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1534>
Storing it per-stream requires taking the manifest lock which can apparenly be
hold for aeons. And since the QoS event comes from the video rendering thread
we *really* do not want to do that.
Storing it as-is in the element is fine, the important part is knowing the
earliest time downstream.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1021>
This will save the last independent slice and fill in the missing
information for dependent slices. This was left over during the porting
from gstreamer-vaapi. The private variable prev_independent_slice was
already there.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1750>
* Move all DPB bumping process into GstH265Dpb internal
* Handle DPB add process in GstH265Dpb struct
* Make implementation to be 1:1 mappable with hevc specification
* Fix wrong DPB bumping implementation especially when no_output_of_prior_pics_flag
was specified.
With fixes from Nicolas Dufresne <nicolas.dufresne@collabora.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1748>
VA-API needs AC and DC quant scales for both luma and chroma, and the loop
filter level for current frame, but these values are not available outside
the private GstVp9Parser structure. And these values may change from frame
to frame, so they are picture specific.
This patch add GstVp9Segmentation structure array to GstVp9Picture to expose
it to derived classes. This approach is safer than passing the parser at
picture handling flow.
Also, this patch, in order to solve Documentation CI, mark as private the
GstVp9Picture structure.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1700>
In order to know the chroma format, besides profile, subsampling_x and
subsampling_y are needed (Spec 7.2.2 Color config semantics). These values are
in GstVp9Parser but not in GstVp9Framehdr.
Also, bit_depth is available in parser but not frame header. Evenmore, those
values are copied to picture structure later.
In case of VA-API, to configure the pipeline, it is require to know the chroma
format and depth.
It is possible to know chroma and depth through caps coming from vp9parser, but
it requires string parsing. It would be less error prone to get these values
through the parser structure at new_sequence() virtual method.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1700>
Move the show_frame check from sub class to vp8 decoder's base class.
Calling the sub class' output_picture() function only when the frame
is displayed and marking the other automatically as decode only.
This is done to avoid logic and code repetition in subclasses.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1670>
