We had a problem with negotiation of the framerate.
Gstreamer was querying the FRAMEINTERVALS based on the max frame size
instead of the desired frame size.
This was resulting in non-negotiated errors when trying to run with a
smaller frame size and fps higher than the max for the max image size.
Fx the max framerate for 1024x1024 RGB on CMOSIS4000 is 28.292
While for 1024x100 RGB it is 280.867
But Gstreamer would allow any framerates bigger than 28.292 no matter
the frame size used...
I have fixed it by 1st changing the CAPS query to use the minimum frame
size instead of maximum.
This however has the downside of allowing gstreamer to negotiate
framerates that are too high if the image size is bigger than the
minimum.
This is not a huge problem since our driver just CLAMPS the fps value to
the max then.
However gstreamer was not being properly notified of this change, and
would therefore report a wrong fps in the CAPS structure.
Note that the fps would be correct inside the buffer info.
Since gstreamer was reading the fps back after setting it.
It was just not being "propagated" to the CAPS structure.
I have also added a WARNING to this point so we can see if the fps that
gstreamer tries to apply was accepted or not.
And the next part of the fix was to add a framerate check after the
frame size has been established.
I did this inside the fixate_caps function of the v4l2src, which was
calling the TRY_FMT in order to check if the format was correct.
So I just added a check for the ENUM_FRAMEINTERVALS in there.
And now we get the non-negotiated again if the fps is too high for the
selected frame size.
Also added a couple of warnings so it is easy to see that this was the
cause.
See:
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3037
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7850>
There is no point in having an endian marker on 8 bit bayer format names since
it is just one byte. Thus remove it.
This also fixes an incompatibility with plugins bad where there is no endian
marker on 8 bit bayer format names as well.
Fixes: #3729
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7826>
This notably follow the way we order the template and keeps the
format:Interlaced caps at the end. This change also fixes
an early skip check, that would skip if a driver only supports
alternate interlacing for a specific format. It also fixes
a bug where only the last resolution of a discrete frame size
was allowed to use format:Interlaced. Finally, similar to template
caps code, simplify the caps for earch featurs, making the debug output
manageable and (marginally) improve negotiation speed.
This change will make it easier to introduce memory:DMABuf.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7540>
Valgrind complains about uninitialized memory used in an ioctl
Syscall param ioctl(VKI_V4L2_G_TUNER).reserved points to uninitialised byte(s)
at 0x719294F: ioctl (ioctl.c:36)
by 0x3126A817: gst_v4l2_fill_lists (v4l2_calls.c:185)
by 0x3126A817: gst_v4l2_open (v4l2_calls.c:589)
by 0x3123F1C2: gst_v4l2_device_provider_probe_device (gstv4l2deviceprovider.c:122)
by 0x3123F648: gst_v4l2_device_provider_device_from_udev (gstv4l2deviceprovider.c:301)
by 0x3123F998: provider_thread (gstv4l2deviceprovider.c:395)
by 0x796FA50: ??? (in /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.7200.4)
by 0x710CAC2: start_thread (pthread_create.c:442)
by 0x719DA03: clone (clone.S:100)
Address 0x44008a34 is on thread 11's stack
in frame #1, created by gst_v4l2_open (v4l2_calls.c:524)
Uninitialised value was created by a stack allocation
at 0x3126A024: gst_v4l2_open (v4l2_calls.c:524)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6144>
This is not trully supported in V4L2, but we can emulate this similar to
what other elements do. In this patch we ensure that 0/1 is supported by
encoders (caps query),and uses a default of 30fps whenever we need to
set a framerate into the driver.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7352>
This patch addresses the issue where GStreamer would throw an error when
attempting to use bt2100-hlg colorimetry with V4L2, which is not
supported by the current V4L2 kernel. When bt2100-hlg colorimetry is set
from caps, the check for transfer (GST_VIDEO_TRANSFER_ARIB_STD_B67) is
bypassed.
The main improvement is to avoid checking the transfer value in
gst_v4l2_video_colorimetry_matches when it is
GST_VIDEO_TRANSFER_ARIB_STD_B67. This is because the transfer value in
the cinfo parameter comes from gst_v4l2_object_get_colorspace, which
converts the transfer to another value, causing a mismatch.
Since the kernel does not support GST_VIDEO_TRANSFER_ARIB_STD_B67,
gst_v4l2_object_get_colorspace cannot map it correctly from V4L2 to
GStreamer. Therefore, we ignore this check to prevent errors.
changes:
- Added a condition in gst_v4l2_video_colorimetry_matches to bypass the
transfer check when the transfer is GST_VIDEO_TRANSFER_ARIB_STD_B67.
- Ensured that the pipeline does not throw errors due to unsupported
bt2100-hlg colorimetry in V4L2.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7212>
If the stream has a special colorimetry that is not in the colorimetry
list, it will cause negotiation to fail. We should allow passing any
colorimetry, so add an extra structure without the colorimetry field.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7029>
video-info supports encoded format to have RGB color-matrix, while
v4l2object just leave the v4l2 matrix to default when mapping
GST_VIDEO_COLOR_MATRIX_RGB. It causes gst matrix changed to be
GST_VIDEO_COLOR_MATRIX_BT601 when mapping v4l2 colorimetry.
So add support for encoded format with RGB color-matrix in v4l2object.
Note that for M2M encoders, we should in theory assume that that we can
transfer this value from OUTPUT to CAPTURE queues, though its only true
if the drivers does not do CSC. For now, we don't support any RGB
codecs, but leaving a note for the future.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3952>
The V4L2_MAP_QUANTIZATION macro has been fixed to something a lot saner,
fix our replica accordingly. The new macro now simply set the quantization
to full range is the pixel formats is RGB based, or if the JPEG
colorspace is used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3952>
Certain V4L2 drivers can report that a video receiver is seeing
some signal, but that it is unable to synchronize to it. IOW: the driver
can sometimes report V4L2_IN_ST_NO_SYNC and not report V4L2_IN_ST_NO_SIGNAL.
In particular, I've seen the tc358743 (HDMI-to-CSI2 converter) driver
sometimes report this when deployed to a fleet of embedded Raspberry Pis.
The relevant kernel code is in [1]. The video output is not practically
usable when V4L2_IN_ST_NO_SYNC is reported (only visually corrupted frames,
sometimes with random "snow", are received). I assume that this happens when
either the HDMI cable is poorly plugged in or damaged or when a CSI2 FFC
cable is used and is damaged.
The change in this commit is useful for detecting this working-but-not-really
condition in application code. Applications already listening for the "Signal lost"
message will gain the ability to handle this condition.
There seem to be more V4L2 error flags like this, see [2]. However, I do not
have practical experience with them and adding only V4L2_IN_ST_NO_SYNC seems
like a safer option.
[1]: https://github.com/raspberrypi/linux/blob/be8498ee21aa/drivers/media/i2c/tc358743.c#L1534
[2]: https://www.kernel.org/doc/html/v6.6/userspace-api/media/v4l/vidioc-enuminput.html
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7021>
Otherwise, if we run in to the copy case, this can cause these
groups to stay around with queued flag set, but never actually
queued, until gst_v4l2_allocator_flush() is called, which then
erroneously frees the associated memories, causing the release
function to decrement the allocator refcount where it was never
incremented, resulting in early allocator disposal, and either
deadlock or use after free.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6552>
Some decoder drivers need to wait enough capture buffers before
starting to decode. But the dequeued buffer flag LAST but empty
has no chance to queue back to driver, which makes decode hang
after seek. So need to queue back such kind of buffer to driver.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6579>
Output buffers don't have to be writable. Accepting read-only buffers
from the V4L2 buffer pool allows upstream elements to write directly
into the V4L2 buffers without triggering a CPU copy into a new buffer
from the same V4L2 buffer pool every time.
Tested with the vivid output device:
GST_DEBUG=GST_PERFORMANCE:7 gst-launch-1.0 videotestsrc ! v4l2sink device=/dev/video5
With this change, gst_v4l2_buffer_pool_dqbuf() must be allowed to not
resize read-only memories of output buffers.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6572>
Some driver doesn't implement enum_framesize. The maximum supported
size can be got by trying format with a very large size. Also need
to set max_width/max_height for this case, otherwise default encoded
buffer size 256kB is too small.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6416>
When reopening a v4l2 device, the v4l2object->poll will include some old fds,
which was assigned to this device before. If the pipeline opens multiple v4l2
devices, the old fd may been assigned to other v4l2 devices when reopening
devices.
This will cause the timing of the pipeline become confusing when polling devices,
leading functional abnormalities.
Therefore, when closing v4l2object, remove the old fds in poll to ensure that the
pipeline timing is normal.
Signed-off-by: Chao Guo <chao.guo@nxp.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5820>
This simplifies the way it picks the closest caps to preference and take into
consideration the framerate to avoid picking high resolution at 5fps or so.
Simply calculate a "distance" of caps A and B from the preference and put
closest first, sorting by framerate first.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5777>
The default 2MB ENCODED_BUFFER_SIZE can't support some 4K video playback. We now
detect the driver reported maximum resolution and choose an appropriate
default bitstream size accordingly. For 4K video these results in around 4MB
buffer instead of 2MB.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4549>
If the v4l2videoenc receives an QUERY_ALLOCATION, it must not propose a
currently used pool, because it cannot be sure that the allocation query came
from exactly the same upstream element. The QUERY_ALLOCATION will not contain
the internal OUTPUT pool.
The upstream element (the basesrc) detects that the newly proposed pool differs
from the old pool. It deactivates the old pool and switches to the new pool.
If there was a format change, a new OUTPUT buffer pool will be allocated in
gst_v4l2_object_set_format_full() and the CAPTURE task will be stopped to switch
the format. If there hasn't been a format change,
gst_v4l2_object_set_format_full() will not be called. The old pool will be kept
and reused.
Without a format change, the processing task continues running.
This leads to the situation that the processing task is running, but the OUTPUT
buffer pool (the old pool) is deactivated. Therefore, the encoder is not able to
get buffers from the OUTPUT pool and encoding cannot continue.
This situation can be triggered by sending a RECONFIGURE event without a format
change.
Resolve this situation by ensuring that the OUTPUT buffer pool is always
activated when frames arrive at the encoder.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4235>
There is a CAPTURE pool in the same function. While the CAPTURE pool is called
cpool, using pool for the OUTPUT pool is confusing.
Using opool for the OUTPUT pool makes it more obvious, which pool is used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4235>
While adding arbitrary tile support, a round up operation was badly
converter. This caused the Y component of the stride to be 0. This
eventually lead to a crash in glupoad preceded by the following
assertion.
gst_gl_buffer_allocation_params_new: assertion 'alloc_size > 0' failed
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5458>
gst_video_info_set_interlaced_format() can return an error if the
width/height causes integer overflow. Handle this case, so that we can
fail cleanly. This has been experienced while testing an in-progress
driver.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5286>
Some drivers will push an buffer flagged LAST but empty. In decoder
case, this results in an "producing too many buffer" warning, even
though the result is entirely correct. Detect this case in order to
signal EOS earlier and avoid this warning.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5286>
