A previous fix, a275e1e029, is correct but was too
permissive since it treats all un-matched NAL units the same as AU delimiters
even though some other NAL unit types can be encountered in the processing loop.
The problem this can cause is that some hardware decoders experience bad
performance when handling FD units that precede the SPS.
This change restores the original behavior for FDs so that they're ignored until
the SPS is received and it preserves the codec conformance test gains that the
fix has achieved.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7166>
glCheckFrameStatus() can fail by returning 0, and otherwise return a
status. Fix the trace to make it clear when we get an unkown status
compare to having an error, in which case we also trace the error code.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7291>
Parts may emit bus messages that want to take the splitmuxsrc
lock and prevent the downward state change. Avoid a deadlock
after a part sends an error message by taking a ref and
dropping the lock around the unprepare call
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7053>
Publish fragment-id in the messages that splitmuxsink and splitmuxsrc
send, so when they are received out of order (due to async finalization,
for example), they can still be identified / ordered correctly.
Fix a race in the splitmuxsink unit test where messages might be
received out of order
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7053>
Add a `num-lookahead` property that will 'prepare' a number of
fragments in advance of the playhead if they have been deactivated
or closed by a limited number of `num-open-fragments`. It can help
to avoid any play stalls reading the indexes or headers of the next
file from high-latency media or on resource limited machines.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7053>
Publish the playback offset for and duration into the
splitmuxsink-fragment-closed bus message as each fragment
finishes.
These can be passed to splitmuxsrc via the 'add-fragment'
signal to avoid splitmuxsrc measuring all files on startup
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7053>
Add a reasonably large default for the number of simulataneous
files to open, that won't affect users that split recordings into
a few large files, but will help prevent fd exhaustion for users
that make recordings with lots of small fragments
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7053>
When calculating the timestamp offset to apply to
media streams in a fragment, ensure that all fragments
are offset "together" to preserve alignment in cases
where there might gaps in a recording at a fragment boundary.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7053>
Add a signal that allows adding fragments with a specific offset
and duration directly to splitmuxsrc's list. By providing the
fragment's offset on the playback timeline and duration directly,
splitmuxsrc doesn't need to measure the fragment making for faster
startup times.
Add a bus message that's published when fragments are measured,
reporting the offset and duration, so they can be cached by an
application and used on future invocations.
Add examples for handling the bus message and using the 'add-fragment'
signal.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7053>
Add a property to limit the number of parts splitmux will open
simultaneously. Modify the part handling to support deactivating
and reactivating the demuxing for each part.
The default is '0', to preserve the existing behaviour of opening
all parts at the beginning.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7053>
According to https://w3c.github.io/webrtc-pc/#set-the-session-description
(steps in 4.6.10.), we should be creating and associating transceivers when
setting session descriptions.
Before this commit, webrtcbin deviated from the spec:
1. Transceivers from sink pads where created when the sink pad was
requested, but not associated after setting local description, only
when signaling is STABLE.
2. Transceivers from remote offers were not created after applying the
the remote description, only when the answer is created, and were then
only associated once signaling is STABLE.
This commit makes webrtcbin follow the spec more closely with regards to
timing of transceivers creation and association.
A unit test is added, checking that the transceivers are created and
associated after every session description is set.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7156>
If a downstream buffer pool is offered, vulkanupload checks its allocation
parameters to honor them. Only adds to usage the TRANSFER bits, which are
required to upload buffers.
Also, fail if the buffer pool cannot be configured with the current parameters.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7219>
If the stream has a special colorimetry that is not in the colorimetry
list, it will cause negotiation to fail. We should allow passing any
colorimetry, so add an extra structure without the colorimetry field.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7029>
video-info supports encoded format to have RGB color-matrix, while
v4l2object just leave the v4l2 matrix to default when mapping
GST_VIDEO_COLOR_MATRIX_RGB. It causes gst matrix changed to be
GST_VIDEO_COLOR_MATRIX_BT601 when mapping v4l2 colorimetry.
So add support for encoded format with RGB color-matrix in v4l2object.
Note that for M2M encoders, we should in theory assume that that we can
transfer this value from OUTPUT to CAPTURE queues, though its only true
if the drivers does not do CSC. For now, we don't support any RGB
codecs, but leaving a note for the future.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3952>
The V4L2_MAP_QUANTIZATION macro has been fixed to something a lot saner,
fix our replica accordingly. The new macro now simply set the quantization
to full range is the pixel formats is RGB based, or if the JPEG
colorspace is used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3952>
Now that driver version is expected to be equal or superior to 1.3.275 the bug
in NVIDIA and RADV regarding usage is solved, we can revert commit b7ded81f7b.
Also this patch sets the internal usage variable after all the validation are
run, thus the state don't keep an invalid usage.
Finally, the now unused supported_usage variable is dropped.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7247>
Virtual method set_config() can be called several times, and if the number of
profiles counter isn't reset the pool will reach an error state.
The purpose of number of profiles is to check the number of valid vulkan video
profiles (two in the case of transcoding use-case, for example) so it's local to
set_config() virtual method.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7247>
Fixing warnings
GStreamer-CRITICAL **: 01:21:25.862: gst_value_set_int_range_step:
assertion 'start < end' failed
Although when QSV runtime reports a codec is supported, resolution query
fails sometimes, espeically VP9 encoder case on Windows.
Don't try to register an element if resolution query returned an error
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7250>
A fence configured in GstD3D12Memory should be used only for
write access to be completed. And because d3d12 -> d3d11 copy path
is read access to d3d12 resource, we should not set fence to
memory. Otherwise another read access to the d3d12 resource
will wait for d3d11 device context's copy operation although
simultaneous read access is allowed.
Use background thread to keep d3d12 resource and wait for d3d11 device's
copy operation instead.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7243>
When configured in constant bitrate mode, the muxer computes timing information
using the configured bitrate and the byte counter (now = bytes sent / byterate).
When an application changes the bitrate in CBR mode during playback, the
relationship between bytes sent and bitrate is no longer valid so new timing
values will be off by the ratio of the old bitrate to the new bitrate.
Furthermore, it will upset the way that padding is generated.
pad_stream() works by trying to fit the byte counter to now * byterate.
The result is that when decreasing bitrate, the muxer stalls, waiting until the
byte counter is in agreement with now * byterate. Also, when increasing
bitrate, the padding will spike in volume until the byte counter fits with
now * byterate.
If the byte counter is scaled by the ratio of new bitrate / old bitrate when
adjusting bitrate, then padding is generated in a way that applications would
more likely expect.
One detail this change doesn't yet address is whether the next PCR will match up
optimally with the previous PCR right after the byte counter is scaled. In that
case, some correction may be necessary. Also, perhaps the user should be
prevented from changing from bitrate=0 to bitrate=nonzero during playback since
it's not straightforward how to scale the byte counter in that case.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7158>
We would previously register a whole bunch of encoder/decoder for which the caps
were ... "unknown/unknown".
Add a function to quickly check (without generating caps) whether a given
AVCodecID has a known mapping (which can include the {video|audio}/x-gst-av-*
ones) without generating the caps.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6237>
videoscale does not have convert function, so remove the convert
description in it's classification. Otherwise, if we want use
autovideoconvert to convert colorsapce, autovideoconvert will select
videoscale to do convert and this will cause to fail.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7215>
VLC counts METADATA as 1 even if the specification states you must not.
This leads to asfdemux failing since there are no bytes left when asfdemux
tries to extract the "last" header.
Do not fail hard in this case and try to proceed when everything else went
fine.
So at least gst-discoverer will see what's in the file.
Closes#3684
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7209>
This is to avoid a regression in validation layer (introduced by commit
916c4e70cd) when using vulkandownload
VUID-VkImageMemoryBarrier2-srcAccessMask-03914 .. vkCmdPipelineBarrier2():
pDependencyInfo->pImageMemoryBarriers[1].srcAccessMask (VK_ACCESS_TRANSFER_READ_BIT)
is not supported by stage mask (VK_PIPELINE_STAGE_2_VIDEO_DECODE_BIT_KHR)
since vulkandownload set DPB memories' access mask to
VK_ACCESS_TRANSFER_READ_BIT, while they are retain by the DPB queue, so when
they are used as DPB after been shown, this validation error is raised.
Must of the barrier values are set ignoring the previous state of the vulkan
images.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7211>
None of the symbols in webrtc-audio-coding-1 are marked with
`__declspec(dllexport)`, rendering the library usable only if
it was built with GCC/Clang.
The only fix available (as the pulseaudio copy has not been updated
with Google's upstream) is to ensure the fallback builds statically.
Although this change will also affect webrtcdsp's dependency on
webrtc-audio-processing-1, it does not break its compilation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6407>
The access flags are kept around the operations, but when the buffer is
released, the access flag should be reset to its original value, since queue
transfers can be done along the pipeline and, when reusing the buffer, the new
queue might not support the latest access flag.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7165>
Instead of dragging the last destination pipeline stage as current barrier
source pipeline stage (which isn't a valid semantic) this patch adds a parameter
to gst_vulkan_operation_add_frame_barrier() to set the source pipeline stage to
define the barrier.
The previous logic brought problems particularly with queue transfers, when the
new queue doesn't support the stage set during a previous operation in a
different queue.
Now the operation API is closer to Vulkan semantics.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7165>
Doing so resets the stride from the VideoMeta and it wasn't done before
the commit below. While on it, drop the plane size check as we can't
reliably predict the correct size when using DRM modifiers.
Fixes: 89b0a6fa23 ("va: refactor buffer import")
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7187>
null event NT handle to ID3D12Fence::SetEventOnCompletion()
will block the calling CPU thread already, thus it has no point that
creating an event NT handle in order to immediate wait for fence at CPU-side.
Note that passing a valid event NT handle to the fence API might be useful
when we need to wait for the fence value later (or timeout is required),
or want to wait for multiple fences at once via WaitForMultipleObjects().
But it's not a considered use case for now.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7176>
GstD3D12Window.priv.input_info is referenced by mouse event handler
in order to calculate corresponding original position
if scene is rotated/flipped by the videosink.
Fixing regression introduced by recent d3d12videosink refactoring
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7177>
The driver - AKA intel-vaapi-driver - has been unmaintained for four years
now and encoding appears to be broken in various cases. As it's unlikely
that the situation will improve, blocklist the driver for encoding.
Decoding appears to be stable enough to keep it enabled.
The driver can still be used by setting the `GST_VA_ALL_DRIVERS` env
variable.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7170>
GstFFMpegAudDec.context can be nullptr if decoder got closed
without opening new context. Note that we don't need to clear
AVCodecContext.extradata there since avcodec_free_context()
will do clear the data if needed.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7180>
The driver frame counters (processed, dropped, buffer level) are not
always correct apparently, and don't allow reliably assigning a frame
number to captured frames.
Instead of relying on them, count the number of frames directly here and
detect dropped frames based on the capture times of the frames: if more
than 1.75 frame durations are between two frames, then there must've
been a dropped frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7163>
Define a default usage and use it instead of repeating the same bitwise
addition.
Therefore, when usage is defined as zero, the usage is defined with the
format's supported usage and the default usage, now without the storage
bit, but with color and input attachment bits.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6798>
Not doing so would mean that tags would be overidden by any tag events sent by
upstream. Also only send a tag event directly if upstream never sent one.
By default use GST_TAG_MERGE_REPLACE to override tags that exist in both the
upstream event and this element with the ones from this element, but provide a
new "merge-mode" property to adjust the behaviour.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7145>
avcodec_close() is deprecated and it's not supported anymore to re-open
a codec, so we only ever allocate the codec in set_format() now and
always free it after usage.
As part of this, also fix various memory leaks in related code paths.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6505>
Direct access to AVInputFormat::read_probe() is not possible anymore
with ffmpeg 7.0, and the usefulness of this typefinder seems limited
anyway. An alternative implementation around av_probe_input_format3() or
similar would be possible but it would be going over all possible ffmpeg
probes at once.
Having a typefinder here means that basically every application will
load the gst-libav plugin when typefinding is necessary, which has
unnecessary performance impacts. If a typefinder from here was indeed
missing from typefindfunctions in gst-plugins-base then it would be
better to add it there directly.
Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3378
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6505>
If a d3d12 memory holds non-direct-queue fence but the fence was
created with D3D12_FENCE_FLAG_SHARED flag, use the fence instead of
waiting for fence at CPU side. Note that d3d12ipcsrc or
d3d12screencapture elements will hold such sharable fence.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7139>
This needs significant work to use the new Metal→Vulkan integration
extension `VK_EXT_metal_objects`
```
MoltenVK/mvk_deprecated_api.h:132:1: note: 'vkGetMTLDeviceMVK' has been explicitly marked deprecated here
MVK_DEPRECATED_USE_MTL_OBJS
^
MoltenVK/mvk_deprecated_api.h:74:52: note: expanded from macro 'MVK_DEPRECATED_USE_MTL_OBJS'
#define MVK_DEPRECATED_USE_MTL_OBJS VKAPI_ATTR [[deprecated("Use the VK_EXT_metal_objects extension instead.")]]
^
../sys/applemedia/videotexturecache-vulkan.mm:303:20: error: 'vkSetMTLTextureMVK' is deprecated:
Use the VK_EXT_metal_objects extension instead.
VkResult err = vkSetMTLTextureMVK (memory->vulkan_mem.image, texture);
^
MoltenVK/mvk_deprecated_api.h:151:1: note: 'vkSetMTLTextureMVK' has been explicitly marked deprecated here
MVK_DEPRECATED_USE_MTL_OBJS
^
MoltenVK/mvk_deprecated_api.h:74:52: note: expanded from macro 'MVK_DEPRECATED_USE_MTL_OBJS'
#define MVK_DEPRECATED_USE_MTL_OBJS VKAPI_ATTR [[deprecated("Use the VK_EXT_metal_objects extension instead.")]]
^
2 errors generated.
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
With clang on macOS:
```
error: format specifies type 'long' but the argument has type 'uint64_t' (aka 'unsigned long long')
...
error: format specifies type 'unsigned long' but the argument has type 'VkImageView' (aka 'unsigned long long')
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
When building for iOS in Cerbero, as of MoltenVK SDK 1.3.283, we have
to statically link to libMoltenVK since it no longer ships a dylib.
This requires linking to libc++, so we find the dep with the objc++
compiler to ensure that meson uses the right linker.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
${libdir}/gstreamer-1.0/include is only valid after installation, but
extra_cflags are added unconditionally, so we can't use that for
include flags.
Instead, let's add the include flag via variables, which are different
for installed and uninstalled pc files.
This is particularly bad for consuming GStreamer via CMake which barfs
on non-existent include paths.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7130>
since the encoded output is changing based on version
it does not make sense to check the output bitstream with a fixed
bytearray since the version in the target might vary. So sticking
to checking the number of output buffers and encoded frame size
similar to the other tests
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7141>
The code is simplified by using GQuarks for looking for caps features, and
removing inner loops.
Also, it's used the pad template caps to compare with the incoming caps because
is cheaper at the beginning of negotiation, where the pad template caps is used.
And, since the ANY caps where removed, there's no need to check for an initial
intersection.
Finally, the completion of caps features is done through a loop.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6698>
The video meta API now is a mandatory request for DMA kind negotiation. In
current code, the raw method always adds that video meta API in the query
of propose_allocation(), so we do not need to do the duplicated task here.
Just adding a comment to declare that.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6698>
The ANY caps in pad template caps seems to mess up the DMA negotiation.
The command of:
GST_GL_API=opengl gst-launch-1.0 -vf videotestsrc ! video/x-raw,format=NV12 !
vapostproc ! "video/x-raw(memory:DMABuf)" ! glimagesink
fails to negotiate, but in fact, the vapostproc can convert the input NV12
formant into the RGBA format to render.
The ANY may help the passthough mode, but we should make the negotiate correct
first.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6698>
gst_element_send_event(FLUSH_START / FLUSH_STOP) returns FALSE in cases
where any of the most downstream elements have unlinked pads, even if
the pipeline is successfully flushed.
Currently this is considered expected behavior in GStreamer. This patch
updates gst-validate to treat it as such and therefore not fail the test
for a "failing" flush.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7064>
After signal recovery the capture times for the next frames are simply
wrong. Experimentally this affected 2-3 frames and seemed to be related
to the buffer fill level after signal recovery, so drop at least 5
frames and up to fill level + 1 frames in this situation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7106>
Since a buffer resource will occupy at least 64KB,
allocating upload resource per decoding command might not be
an optimal approach. Instead, use sub-region of a upload resource
for multiple decoding command if sub-regions are not overlapped
each other.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7108>
- Align `glib_debug`, `glib_assert` and `glib_checks` options with GLib,
otherwise glib subproject won't inherit their value. Previous names
and values are preserved using Meson's deprecation mechanism.
- Add `extra-checks` and `benchmarks` options in the main project so it
can be inherited in GStreamer subprojects.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1165>
When enable parallel encoding, it is possible that the unshown frame
is not output but it is already be marked as a repeated frame header.
So we need to use a dedicated buffer to hold the repeat frame header,
don't mix it with the orignal frame data.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6867>
Schedule (semi-)static resource upload at converter creation time.
And use single resource for all vertex, index, and constant
buffers, since separate resources will waste GPU memory.
Note that size and address of a committed resource are 64K aligned
even if requested buffer size is small.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7081>
If the first buffers have no timestamp then the sink position would be
initialized to 0. The source pad might output this buffer, which would then
initialize the source position to 0 too.
Afterwards two buffers with a valid but huge timestamp might arrive before any
of them are output on the source pad. The first one would set the sink position
to a huge value, the second one would notice that the difference between the
huge value and 0 is certainly larger than max-size-time and consider the queue
as full.
Instead, simply don't update the times from buffers without timestamps and
assume whatever was set before is still valid, i.e. the buffer has the same
timestamp as the previous one.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7071>
We were storing the probe id in a different structure (DecodebinOutputStream)
than the pad it is targetting (which is in MultiQueueSlot).
The problem is that when re-targetting outputs (to a different slot)... we would
end up having an invalid probe id, or not have a reference to an existing one.
Instead, store the probe id in the same structure as the pad it's targetting
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7069>
As NVIDIA Amperium. In this case the each output buffer is also a DPB,
but using a different view layer.
Still pending a validation layer issue:
VUID-VkVideoBeginCodingInfoKHR-flags-07244
Co-authored-by: Victor Jaquez <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6954>
