Add macro which converts picture frame number to suitable timestamp in
nanoseconds for use in V4L2 VB2 buffer lookup. Since multiple codecs do
the same operation and almost all got it wrong, do it in one place so it
can be fixed in one place again, if needed.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
The GstMpeg2Picture system_frame_number is guint32, constant 1000 is guint32,
GstV4l2CodecMpeg2Dec *_ref_ts multiplication result is u64 .
```
u64 result = (u32)((u32)system_frame_number * (u32)1000);
```
behaves the same as
```
u64 result = (u32)(((u32)system_frame_number * (u32)1000) & 0xffffffff);
```
so in case `system_frame_number > 4294967295 / 1000`, the `result` will
wrap around. Since the `result` is really used as a cookie used to look
up V4L2 buffers related to the currently decoded frame, this wraparound
leads to visible corruption during MPEG2 decoding. At 30 FPS this occurs
after cca. 40 hours of playback .
Fix this by changing the 1000 from u32 to u64, i.e.:
```
u64 result = (u64)((u32)system_frame_number * (u64)1000ULL);
```
this way, the wraparound is prevented and the correct cookie is used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
The GstAV1Picture system_frame_number is guint32, constant 1000 is guint32,
GstV4l2CodecAV1Dec v4l2_av1_frame.*_frame_ts multiplication result is u64 .
```
u64 result = (u32)((u32)system_frame_number * (u32)1000);
```
behaves the same as
```
u64 result = (u32)(((u32)system_frame_number * (u32)1000) & 0xffffffff);
```
so in case `system_frame_number > 4294967295 / 1000`, the `result` will
wrap around. Since the `result` is really used as a cookie used to look
up V4L2 buffers related to the currently decoded frame, this wraparound
leads to visible corruption during AV1 decoding. At 30 FPS this occurs
after cca. 40 hours of playback .
Fix this by changing the 1000 from u32 to u64, i.e.:
```
u64 result = (u64)((u32)system_frame_number * (u64)1000ULL);
```
this way, the wraparound is prevented and the correct cookie is used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
The GstVp9Picture system_frame_number is guint32, constant 1000 is guint32,
GstV4l2CodecVp9Dec v4l2_vp9_frame.*_frame_ts multiplication result is u64 .
```
u64 result = (u32)((u32)system_frame_number * (u32)1000);
```
behaves the same as
```
u64 result = (u32)(((u32)system_frame_number * (u32)1000) & 0xffffffff);
```
so in case `system_frame_number > 4294967295 / 1000`, the `result` will
wrap around. Since the `result` is really used as a cookie used to look
up V4L2 buffers related to the currently decoded frame, this wraparound
leads to visible corruption during VP9 decoding. At 30 FPS this occurs
after cca. 40 hours of playback .
Fix this by changing the 1000 from u32 to u64, i.e.:
```
u64 result = (u64)((u32)system_frame_number * (u64)1000ULL);
```
this way, the wraparound is prevented and the correct cookie is used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
The GstVp8Picture system_frame_number is guint32, constant 1000 is guint32,
GstV4l2CodecVp8Dec v4l2_vp8_frame.*_frame_ts multiplication result is u64 .
```
u64 result = (u32)((u32)system_frame_number * (u32)1000);
```
behaves the same as
```
u64 result = (u32)(((u32)system_frame_number * (u32)1000) & 0xffffffff);
```
so in case `system_frame_number > 4294967295 / 1000`, the `result` will
wrap around. Since the `result` is really used as a cookie used to look
up V4L2 buffers related to the currently decoded frame, this wraparound
leads to visible corruption during VP8 decoding. At 30 FPS this occurs
after cca. 40 hours of playback .
Fix this by changing the 1000 from u32 to u64, i.e.:
```
u64 result = (u64)((u32)system_frame_number * (u64)1000ULL);
```
this way, the wraparound is prevented and the correct cookie is used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
We were previously always querying index 0, and while the number of planes per
buffer will never change, it seems more proper to query the right buffer rather
than always the first one.
This was found while reading strace logs, and wondering why the
V4L2_BUF_FLAG_MAPPED flag was present on all ¬0 indices even though that
happened before VIDIOC_EXPBUF.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5647>
When decoding stream using hardware V4L2 decoder element, in any of the
currently supported formats, the decoding will fail once frame number
1000000 is reached. The reported error clearly indicates a wrap-around
occured, instead of receiving decoded frame 1000000, frame 0 is received
from the hardware V4L2 decoder driver.
The problem is actually not in the driver itself, but rather in gstreamer,
which uses `struct v4l2_buffer` member `.timestamp` in a special way. The
timestamp of buffers with encoded data added to the SINK (input) queue of
the driver is copied by the driver into matching buffers with decoded data
added to the SOURCE (output) queue of the driver. In fact, the timestamp
is not a timestamp at all, but rather in this special case, only part of
it is used as an incrementing frame counter.
The `.timestamp` is of type `struct timeval`, which is defined in
`sys/time.h` [1]. Only the `tv_usec` member of this structure is used
for the incrementing frame counter. However, suseconds_t tv_usec [2]
may be limited to range [-1, 1000000]:
"
[XSI] The type suseconds_t shall be a signed integer type capable of
storing values at least in the range [-1, 1000000].
"
Therefore, once frame 1000000 is reached, a rollover occurs and decoding
fails.
Fix this by using both `struct timeval` members, `.tv_sec` and `.tv_usec`
with matching modular arithmetic, this way the failure would occur again
just short of 2^84 frames, which should be plenty.
[1] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_time.h.html
[2] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_types.h.html
A test case using stateless hardware h264 decoder, the WARN/ERROR output
in gstreamer log indicates a failure occurred. With this change, that
error no longer occurs and the WARN/ERROR are not present:
```
pc$ gst-launch-1.0 videotestsrc num-buffers=1001001 pattern=6 ! \
video/x-raw,width=16,height=16,format=I420 ! \
x264enc ! filesink location=/tmp/test.h264
dut$ GST_DEBUG="*:3" gst-launch-1.0 filesrc location=/tmp/test.h264 ! \
h264parse ! v4l2slh264dec ! fakesink
...
0:03:51.393677606 12111 0x370df400 WARN \
v4l2codecs-decoder gstv4l2decoder.c:1157:gst_v4l2_request_set_done:<v4l2decoder2> \
Requested frame 1000000, but driver returned frame 0.
0:03:51.394140597 12111 0x370df400 WARN \
v4l2codecs-decoder gstv4l2decoder.c:1157:gst_v4l2_request_set_done:<v4l2decoder2> \
Requested frame 1000001, but driver returned frame 1.
0:03:51.394425216 12111 0x370df400 WARN \
v4l2codecs-decoder gstv4l2decoder.c:1157:gst_v4l2_request_set_done:<v4l2decoder2> \
Requested frame 1000002, but driver returned frame 2.
0:03:51.394665211 12111 0x370df400 WARN \
v4l2codecs-decoder gstv4l2decoder.c:1157:gst_v4l2_request_set_done:<v4l2decoder2> \
Requested frame 1000003, but driver returned frame 3.
0:03:51.394785833 12111 0x370df400 WARN \
v4l2codecs-h264dec gstv4l2codech264dec.c:1059:gst_v4l2_codec_h264_dec_output_picture:<v4l2slh264dec0> \
error: Failed to decode frame 1000000
ERROR: from element /GstPipeline:pipeline0/v4l2slh264dec:v4l2slh264dec0: Failed to decode frame 1000000
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5598>
While adding arbitrary tile support, a round up operation was badly
converter. This caused the Y component of the stride to be 0. This
eventually lead to a crash in glupoad preceded by the following
assertion.
gst_gl_buffer_allocation_params_new: assertion 'alloc_size > 0' failed
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5458>
The `gst_video_decoder_negotiate_pool` function expects the
`decide_allocation` function to always provide a pool and will fail to
negotiate if the pool is missing. If we return immediately (even if we
don't need to do anything special) negotiation will fail if the
downstream element does not propose a pool.
Fix by chaining up to the default `decide_allocation` function which
adds a fallback pool if one was not already proposed.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4630>
This patch adds an element to stream video data to an uvc video gadget.
The element handles the uvc events STREAMON, STREAMOFF, SETUP and DATA.
to start, stop and configure the video buffer flow by the use of pad
probes. It works with linux kernels of versions higher than v6.1.
The element makes use of the v4l2sink proxy property v4l2sink::device
to locate the corresponding device to parse the configfs for additional
data.
The code in uvc.c is basically derived from /lib/uvc.c in
https://git.ideasonboard.org/uvc-gadget.git.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1304>
In current tile representation, only tiles with power of two
width and height in bytes are supported. This limitation
prevents adding more complex tiles formats.
In this patch, we deprecate tile_ws and tile_hs from GstVideoFormatInfo and
replace if with an array of GstVideoTileInfo. Each plane tiles are then
described with their pixels width/height, line stride and total size.
The helper gst_video_format_info_get_tile_sizes() that depends on the
deprecated API is also being removed. This can simply be removed as it wasn't
in any stable release yet.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3424>
This change allow output caps to be updated even though we stay in
streaming state. This is needed so that any upstream updated to fields
like framerate, hdr data, etc. can result in a downstream caps event
being pushed.
Previously, any of these changes was being ignored and the downstream
caps would not reflect it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3328>
Currently if the user is not able to access the devices under /dev/media*,
either due to no media devices present on the system or simply no permission
to access the device, v4l2codecs initialises with no features or debug messages.
Since calling `GST_DEBUG="v4l2*:7" gst-inspect-1.0 v4l2codecs` is a typical way
to diagnose why element(s) failed to enumerate, we should be more verbose here
when the user is not able to access any /dev/media* device. So print a simple
debug message in this case to aid debugging.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3088>
The function g_array_sized_new() leaves the len to 0, but the slice
implementation assumes it would be set to 4. Sending multiple slices is
not yet support for H.264 as no driver needed it yet, but if that code
was to be used it would have overflowed as the array would never grow as
multiple 0 by 2 always results in 0.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1079>
Copy V4L2_PIX_FMT_P010 define from linux header.
V4L2_PIX_FMT_P010 is the little endian definition of P010 so map
it GST_VIDEO_FORMAT_P010_10LE.
Add it v4l2 default video formats to allows v4l2 decoders to
enumerate and use it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2590>
V4L spec now requires decode_params flags to be set in accordance to the
frame's type. In particular this is required by H.264 decoder of NVIDIA
Tegra SoC to operate properly. Set the flags based on type of parsed
slices.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1757>
Make all codecs consistent so that subclass can know additional DPB
size requirement depending on render-delay configuration regardless
of codec. Note that render-delay feature is not implemented for AV1
yet but it's planned.
Also, consider new_sequence() is mandatory requirement, not optional
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2343>
Produce an error if we try to use the feature of holding capture buffer
but it is not supported by the driver. Ingoring this can lead to stalls
as the driver will run-out of capture buffer to decode into. This
affects slice decoders but also split-field interlaced decoding.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2009>
