Commit graph

165 commits

Author SHA1 Message Date
Shengqi Yu
7576d14762 v4l2object: append non colorimetry structure to probed caps
If the stream has a special colorimetry that is not in the colorimetry
list, it will cause negotiation to fail. We should allow passing any
colorimetry, so add an extra structure without the colorimetry field.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7029>
2024-07-31 09:28:18 +00:00
Hou Qi
5dffbd492c v4l2: Fix colorimetry mismatch for encoded format with RGB color-matrix
video-info supports encoded format to have RGB color-matrix, while
v4l2object just leave the v4l2 matrix to default when mapping
GST_VIDEO_COLOR_MATRIX_RGB. It causes gst matrix changed to be
GST_VIDEO_COLOR_MATRIX_BT601 when mapping v4l2 colorimetry.

So add support for encoded format with RGB color-matrix in v4l2object.
Note that for M2M encoders, we should in theory assume that that we can
transfer this value from OUTPUT to CAPTURE queues, though its only true
if the drivers does not do CSC. For now, we don't support any RGB
codecs, but leaving a note for the future.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3952>
2024-07-30 20:26:06 +00:00
Nicolas Dufresne
1ddb8797b5 v4l2object: SRGB colorspace is documented limited-range
Split JPEG and SRGB so that we can follow the specified difference. The
SRGB definition in V4L2 does not follow the standard, and is document
so. This is also why JPEG colorspace exists.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3952>
2024-07-30 20:26:06 +00:00
Nicolas Dufresne
20eb14b85b v4l2object: Fix size of plane_size array calculation
Due to missing parenthesys, only the first element of the array was
being cleared. As it is a staticly sized array in the object, this
code could also be simplified.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3952>
2024-07-30 20:26:06 +00:00
Nicolas Dufresne
152df21644 v4l2object: Fix translation of quantization
The V4L2_MAP_QUANTIZATION macro has been fixed to something a lot saner,
fix our replica accordingly. The new macro now simply set the quantization
to full range is the pixel formats is RGB based, or if the JPEG
colorspace is used.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3952>
2024-07-30 20:26:06 +00:00
Shengqi Yu
1875c178cd v4l2object: use v4l2 reported width for padded_width when complex video formats
Stride means bytes per line, and padded_width means pixels. Here,
padded_width shoule be pix width reported by v4l2 instead of stride.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7076>
2024-07-01 18:53:04 +00:00
Jakub Vaněk
0b65f667af v4l2src: Interpret V4L2 report of sync loss as video signal loss
Certain V4L2 drivers can report that a video receiver is seeing
some signal, but that it is unable to synchronize to it. IOW: the driver
can sometimes report V4L2_IN_ST_NO_SYNC and not report V4L2_IN_ST_NO_SIGNAL.

In particular, I've seen the tc358743 (HDMI-to-CSI2 converter) driver
sometimes report this when deployed to a fleet of embedded Raspberry Pis.
The relevant kernel code is in [1]. The video output is not practically
usable when V4L2_IN_ST_NO_SYNC is reported (only visually corrupted frames,
sometimes with random "snow", are received). I assume that this happens when
either the HDMI cable is poorly plugged in or damaged or when a CSI2 FFC
cable is used and is damaged.

The change in this commit is useful for detecting this working-but-not-really
condition in application code. Applications already listening for the "Signal lost"
message will gain the ability to handle this condition.

There seem to be more V4L2 error flags like this, see [2]. However, I do not
have practical experience with them and adding only V4L2_IN_ST_NO_SYNC seems
like a safer option.

[1]: https://github.com/raspberrypi/linux/blob/be8498ee21aa/drivers/media/i2c/tc358743.c#L1534
[2]: https://www.kernel.org/doc/html/v6.6/userspace-api/media/v4l/vidioc-enuminput.html

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7021>
2024-06-12 17:26:48 +00:00
Piotr Brzeziński
9ca8f16a3b macos: Listen for audio devices being added/removed
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6981>
2024-06-01 13:21:59 +00:00
Piotr Brzeziński
477beab403 osxaudio: Avoid using private APIs on iOS
Turns out AudioConvertHostTimeToNanos and AudioGetCurrentHostTime are macOS-only APIs, which prevents apps using
GStreamer on iOS from being accepted into App Store.

This commit replaces those functions with a manual version of what they do - mach_absolute_time() for the current time,
and data from mach_timebase_info() at the beginning to convert host timestamps to nanoseconds.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6789>
2024-05-22 08:58:24 +00:00
Sebastian Dröge
0ef396359c gst: Move GstQueueArray as GstVecDeque to core
And change lengths and indices from guint to gsize for a more correct type.

Also deprecate GstQueueArray and implement it in terms of GstVecDeque.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6779>
2024-05-06 18:25:42 +00:00
Philipp Zabel
46a41667a3 v4l2bufferpool: Ensure freshly created buffers are not marked as queued
Otherwise, if we run in to the copy case, this can cause these
groups to stay around with queued flag set, but never actually
queued, until gst_v4l2_allocator_flush() is called, which then
erroneously frees the associated memories, causing the release
function to decrement the allocator refcount where it was never
incremented, resulting in early allocator disposal, and either
deadlock or use after free.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6552>
2024-04-18 16:42:43 +00:00
Qian Hu (胡骞)
8d003f00e9 v4l2: add multiplane y42b(yuv422m)
for some jpg file, mediatek v4l2 jpeg decoder
hardware produce multi plane YUV 4:2:2 data

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6617>
2024-04-16 09:03:47 +00:00
Hou Qi
105d232fde v4l2bufferpool: queue back the buffer flagged LAST but empty
Some decoder drivers need to wait enough capture buffers before
starting to decode. But the dequeued buffer flag LAST but empty
has no chance to queue back to driver, which makes decode hang
after seek. So need to queue back such kind of buffer to driver.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6579>
2024-04-15 18:07:17 +00:00
Philipp Zabel
e1f5bacf8d v4l2: bufferpool: Drop writable check on output pool process
Output buffers don't have to be writable. Accepting read-only buffers
from the V4L2 buffer pool allows upstream elements to write directly
into the V4L2 buffers without triggering a CPU copy into a new buffer
from the same V4L2 buffer pool every time.

Tested with the vivid output device:

  GST_DEBUG=GST_PERFORMANCE:7 gst-launch-1.0 videotestsrc ! v4l2sink device=/dev/video5

With this change, gst_v4l2_buffer_pool_dqbuf() must be allowed to not
resize read-only memories of output buffers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6572>
2024-04-15 17:11:00 +00:00
Tim Blechmann
1c9fe19b23 v4l2: enforce a pixel aspect ratio of 1/1 if no data are available
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6242>
2024-04-07 10:14:18 +00:00
Philipp Zabel
6f9872cb56 v4l2: allocator: Fix unref log/trace on memory release
Use gst_object_unref() instead of g_object_unref() in
gst_v4l2_allocator_release(), so refcounting log and
tracer get to know about this unref.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6551>
2024-04-06 11:44:27 +00:00
Elliot Chen
e4ee4ca716 v4l2: fix error in calculating padding bottom for tile format
This is a regression while porting to arbitrary tile dimensions
introduced in !3424.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6480>
2024-04-05 13:28:47 +00:00
Elizabeth Figura
c308f013a7 atdec: Handle channel counts greater than 2
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6157>
2024-04-05 06:54:24 +00:00
Elizabeth Figura
277d6ddf22 atdec: Use gst_audio_decoder_set_output_caps() directly
The code currently sets the same caps in two different ways, and neither of them correctly handle the channel mask.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6157>
2024-04-05 06:54:24 +00:00
Haihua Hu
37e3a38ba9 v4l2src: need maintain the caps order in caps compare when fixate
if the calculated "distance" of caps A and B from the preference are
equal, need to keep the original order instead of swap them

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6451>
2024-03-28 12:53:01 +00:00
Hou Qi
024d3ab051 v4l2: Also set max_width/max_height if enum framesize fail
Some driver doesn't implement enum_framesize. The maximum supported
size can be got by trying format with a very large size. Also need
to set max_width/max_height for this case, otherwise default encoded
buffer size 256kB is too small.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6416>
2024-03-22 16:02:51 +00:00
Piotr Brzeziński
e9802f5f41 macos: Add Apple AAC encoder (atenc)
Adds the `atenc` element capable of encoding AAC-LC audio, using the AudioToolbox framework.
It's able to encode up to 7.1 channel configurations.
Comes with basic knobs for rate control (bitrate for CBR, quality for VBR).

Support for more profiles (LD, HE-AAC) should be simple, but is not included here because of bugs
with parsing of the AudioSpecificConfig.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6254>
2024-03-12 19:50:06 +00:00
Piotr Brzeziński
d3fba31da0 macos: Move atdec from applemedia (-bad) to osxaudio (-good)
osxaudio has a few helper methods potentially useful in atdec (or future atenc), like GStreamer -> CoreAudio
channel mapping. Doesn't make sense to duplicate them in applemedia, and atdec is the only audio-oriented
element there anyway.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6223>
2024-03-12 09:55:10 +00:00
Maksym Khomenko
ccf544a50e osxaudio: add mapping for top/left/right surround channels
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5731>
2024-02-20 08:03:15 +00:00
Maksym Khomenko
f1e02ebb92 osxaudio: correct mapping for left/right surround
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5731>
2024-02-20 08:03:15 +00:00
Chao Guo
2e75b8c8e9 v4l2object: clear old fds in poll when closing v4l2object
When reopening a v4l2 device, the v4l2object->poll will include some old fds,
which was assigned to this device before. If the pipeline opens multiple v4l2
devices, the old fd may been assigned to other v4l2 devices when reopening
devices.

This will cause the timing of the pipeline become confusing when polling devices,
leading functional abnormalities.

Therefore, when closing v4l2object, remove the old fds in poll to ensure that the
pipeline timing is normal.

Signed-off-by: Chao Guo <chao.guo@nxp.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5820>
2023-12-19 15:23:23 +00:00
Xavier Claessens
b80f4a1fa4 v4l2src: Consider framerate during caps selection
This simplifies the way it picks the closest caps to preference and take into
consideration the framerate to avoid picking high resolution at 5fps or so.
Simply calculate a "distance" of caps A and B from the preference and put
closest first, sorting by framerate first.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5777>
2023-12-08 21:05:46 +00:00
robert
e3e8147a74 ximagesrc: fix xnavigation linking issue
Fixes #3083

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5589>
2023-11-03 17:36:58 +00:00
robert
737c32b9b6 ximagesrc: fix compile-time warning and XInitThreads()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5493>
2023-11-01 09:17:24 +00:00
Shengqi Yu
25c00b5ba2 v4l2object: scale the encoded sizeimage based on maximum resolution
The default 2MB ENCODED_BUFFER_SIZE can't support some 4K video playback. We now
detect the driver reported maximum resolution and choose an appropriate
default bitstream size accordingly. For 4K video these results in around 4MB
buffer instead of 2MB.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4549>
2023-10-23 14:10:56 +00:00
Robert Ayrapetyan
3d807d4f6d ximagesrc: add navigation support
Add a basic navigation support:
- mouse events (buttons/move)
- keyboard events (keys)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5273>
2023-10-13 23:34:54 +00:00
Michael Tretter
0563a25494 v4l2videoenc: unconditionally activate the OUTPUT pool
If the v4l2videoenc receives an QUERY_ALLOCATION, it must not propose a
currently used pool, because it cannot be sure that the allocation query came
from exactly the same upstream element. The QUERY_ALLOCATION will not contain
the internal OUTPUT pool.

The upstream element (the basesrc) detects that the newly proposed pool differs
from the old pool. It deactivates the old pool and switches to the new pool.

If there was a format change, a new OUTPUT buffer pool will be allocated in
gst_v4l2_object_set_format_full() and the CAPTURE task will be stopped to switch
the format. If there hasn't been a format change,
gst_v4l2_object_set_format_full() will not be called. The old pool will be kept
and reused.

Without a format change, the processing task continues running.

This leads to the situation that the processing task is running, but the OUTPUT
buffer pool (the old pool) is deactivated. Therefore, the encoder is not able to
get buffers from the OUTPUT pool and encoding cannot continue.

This situation can be triggered by sending a RECONFIGURE event without a format
change.

Resolve this situation by ensuring that the OUTPUT buffer pool is always
activated when frames arrive at the encoder.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4235>
2023-10-11 19:35:54 +00:00
Michael Tretter
41ce99ebab v4l2videoenc: fix activation of internal pool
Fix the buffer pool activation if the driver does not support VIDIOC_CREATE_BUFS
the same way as it was fixed for the v4l2videodec.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4235>
2023-10-11 19:35:54 +00:00
Michael Tretter
5e72e1985a v4l2videoenc: rename OUTPUT pool to opool
There is a CAPTURE pool in the same function. While the CAPTURE pool is called
cpool, using pool for the OUTPUT pool is confusing.

Using opool for the OUTPUT pool makes it more obvious, which pool is used.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4235>
2023-10-11 19:35:54 +00:00
Nicolas Dufresne
bcfbdfbbca v4l2: Fix tiled formats stride conversion
While adding arbitrary tile support, a round up operation was badly
converter. This caused the Y component of the stride to be 0. This
eventually lead to a crash in glupoad preceded by the following
assertion.

  gst_gl_buffer_allocation_params_new: assertion 'alloc_size > 0' failed

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5458>
2023-10-11 14:13:53 +00:00
Dominique Leroux
7affa01e05 osxaudio: add individual elements registration for gst-full compatibility
Found that osxaudiosink could not be added standalone in gst-full build
using
-Dgst-full-elements=osxaudio:osxaudiosink because element registration
was
done at the plugin level. Now src/sink elements and deviceprovider have
their
individual registration.

Copied/adapted from the alsa plugin.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5419>
2023-09-28 21:44:48 +00:00
Hou Qi
be9d9371b7 v4l2videodec: Correctly free caps to avoid memory leak
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5379>
2023-09-24 12:50:01 +00:00
Nicolas Dufresne
c1e03081c0 v4l2: object: Handle video helper return value
gst_video_info_set_interlaced_format() can return an error if the
width/height causes integer overflow. Handle this case, so that we can
fail cleanly. This has been experienced while testing an in-progress
driver.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5286>
2023-09-11 15:05:34 -04:00
Nicolas Dufresne
353cb2da92 v4l2: bufferpool: Avoid warnings on empty last buffer
Some drivers will push an buffer flagged LAST but empty. In decoder
case, this results in an "producing too many buffer" warning, even
though the result is entirely correct. Detect this case in order to
signal EOS earlier and avoid this warning.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5286>
2023-09-11 18:08:21 +00:00
Nicolas Dufresne
65350b601e v4l2: bufferpool: Do not resize compressed buffer
Avoid resizing compressed buffer to their maximum size. This fixes a
regression that caused valid but very large streams to be generated.

Fixes #2953

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5286>
2023-09-11 18:08:21 +00:00
Nicolas Dufresne
1e7ff1ac45 gstv4l2object: fix TODO comment about HDR configure
add following todo list
- Missing capture (v4l2src) HDR10 configuration and/or reporting
- The API is not capable of HDR to HDR conversion as controls are
      not specific to queues

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4403>
2023-08-22 21:20:11 +00:00
HuQian
fc7b776387 gstv4l2object: passing HDR10 information
when playing some codec such as matroska with vp9 codec,
demuxer will save information like video_mastering_display_info
and video_content_light_level in caps that decoder need,
v4l2videodecoder can use it by calling V4L2_CTRL_CLASS_COLORIMETRY
ioctl.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4403>
2023-08-22 21:20:11 +00:00
Ming Qian
fd720fbf64 v4l2object: clear format lists if source change event is received
If decoder notify a source change event when the capture format is
changed, not the resolution changed.

then gst_v4l2_object_acquire_format will retuen false due to
unsupported format.

we need to clear the format lists in the source change flow,
and reenumerate format list

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5218>
2023-08-22 19:26:22 +00:00
Nicolas Dufresne
54ae2fcf77 v4l2: allocator: Don't close foreign dmabuf
Imported dmabuf are not being duped, so they should never be closed. Instead,
we ensure their live time by having strong reference on their original
buffer. This should fix potential flickering due to dmabuf being closed
too early.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5101>
2023-08-21 20:45:14 +00:00
Nicolas Dufresne
8974318003 v4l2: bufferpool: Fix hang when splitting buffer
Now that we can split GStreamer buffers over multiple v4l2 buffer, we may
endup waiting for these buffers to be processed. Avoid waiting for any of
the parts being processed. As a side effect, the pool will now try to
grow if the number of buffers is not sufficient, and will fail
otherwise.

This fixes a hang if the very first frame did not fit. In this case, the
driver will retrain that buffer until the capture is setup, but
GStreamer won't setup the capture until process() function have
returned.

Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5100

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5143>
2023-08-21 20:01:39 +00:00
Jan Schmidt
461f943b52 osxaudio: Interpolate clock by counting elapsed time since render calls
When advancing the ringbuffer, store the processed CoreAudio sample
time, then interpolate the clock in the _get_delay() calls to smooth
the clock. CoreAudio's "latency" report is always a constant and
otherwise leads to the clock generating a latency-time staircase.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5140>
2023-08-07 21:33:45 +00:00
Jan Schmidt
e22c7fb3e4 osxaudio: Share debug category in the internal coreaudio object
Make the internal coreaudio object output debug to the same
debug category by making it shared between code units.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5140>
2023-08-07 21:33:45 +00:00
Jan Schmidt
f5d2ea76b4 osxaudio: Attempt to configure the segment size in CoreAudio
Set the BufferFrame size in CoreAudio so it will deliver data
in ringbuffer segment units when recording. Otherwise, CoreAudio
will provide data in whatever granularity it wants, with no
relationship to the requested latency-time.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5140>
2023-08-07 21:33:45 +00:00
Jan Schmidt
2df9283d3f osxaudiosrc: Set sample timestamps
Set the timestamp on output buffers based on the incoming sample
times from Core Audio

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5140>
2023-08-07 21:33:45 +00:00
Nicolas Dufresne
76fbc79494 v4l2: bufferpool: Keep processing bitstream buffer
Bitstream buffers may no fit a single v4l2 buffer, following spec
recommendation, keep processing the buffer until all the data has been
queued.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5100>
2023-08-01 11:02:12 +00:00