This patch will now set the maximum of buffers to 32, allowing to grow the
pool for drivers that supports that and will respect the minimum buffers
reported by the driver. This was made to fix a stall with the virtio CODEC
driver.
Fixes#672
There are often only two buffers queued in the kernel so no new buffers are
requested.
With every qbuf, the kernel receives a new DMABUF for the specified index.
This most likely differs from the last DMABUF and the old cached entry is
released. This results in a lot of map/unmap overhead if the kernel driver
needs a mapping for the buffer.
With a larger queue, it's quite likely, that both old and new DMABUFs are
also mapped for another index. So the map/unmap is skipped, because the
mapping is reference counted.
The corresponding allocated buffers don't contain any actual memory, so
allocating them is quite cheep. So the log message is updated to clarify
this.
When using this mode each frame is split in two fields, each one being
transferred using its own buffer.
This is implemented with the V4L2_FIELD_ALTERNATE field format in v4l2.
This mode is enabled using a caps filter such as
"v4l2src ! video/x-raw\(format:Interlaced\)"
Here are the main changes related to this feature:
- use the INTERLACED caps feature with this mode.
- in this mode both fields of a given frame have the same sequence/offset
so adjust the algorithm checking for lost field/frame accordingly.
- double pool's min number of buffers as each frame requires 2 buffers.
Fix#504
Co-authored-by: Zeeshan Ali <zeenix@collabora.co.uk>
Use GST_VIDEO_INFO_FIELD_HEIGHT() instead of GST_VIDEO_INFO_HEIGHT()
when we actually want the field height rather than the frame height.
So far both are equals but that won't longer be the case when
implementing alternate interlace mode.
Lower the verbosity of the 'sync' log message emitted
each buffer from gst_v4l2src_create down to LOG(6)
from INFO(4). This brings the logging behavior of
v4l2src closer to the GStreamer guidelines, which
recommend the INFO level be reserved for rare or
one-off messages.
gst_v4l2_object_set_format_full() was returning FALSE without setting
an error. Caller code (gst_v4l2src_fixate()) was then derefing a
NULL pointer when trying to handle the error.
Before we do streamon, we queue all capture buffers by calling
resurrect. When the driver supports CREATE_BUFS, this would lead
to buffers being allocated till the maximum of 32 is reached.
Instead, we now save the number of allocated buffers and queue this
amount.
libv4l2 reset the colorpace to 0 and does not do any request to the
driver. This yields an invalid colorspace which currently cause a
negotiation failure. This workaround by ignoring bad values during the
TRY_FMT step.
By passing `NULL` to `g_signal_new` instead of a marshaller, GLib will
actually internally optimize the signal (if the marshaller is available
in GLib itself) by also setting the valist marshaller. This makes the
signal emission a bit more performant than the regular marshalling,
which still needs to box into `GValue` and call libffi in case of a
generic marshaller.
Note that for custom marshallers, one would use
`g_signal_set_va_marshaller()` with the valist marshaller instead.
Try configuring the v4l2 output with the alignments from upstream when
importing its buffers. This allows us to support importing with
non-standard strides and/or heights if supported by the driver.
We were already supporting horizontal padding by setting bytesperline to
the buffer stride but not vertical one.
We are now updating the format height with the padded height and crop to
the actual video resolution if needed.
In commit e2ff87732d ("v4l2videodec: support orphaning") support for
orphaning the capture buffer pool was added when the format is
renegotiated. However, the commit forgot to check that a pool existed
before doing this. This is needed because it's possible for the format
to be renegotiated before a capture pool is allocated, which would
result in trying to orphan a NULL pool and lead to a NULL pointer
dereference.
Fix this by checking a pool exists first. If the pool doesn't exist,
there are no buffers to be reclaimed, so skip the allocation query in
that case.
When building osxaudio, the required 'AudioToolbox' dependency is
misspelled as 'AudioToolBox', which crashes the build with error:
ld: framework not found AudioToolBox
Backward playback will drain and flush every frame. Stop playback
when backward playback have race condition between exit thread and
streaming thread flush. Add one check to avoid it.
Fixes#639
Update to the latest installed headers (output of make headers_install)
from the media tree, keeping the slight modifications to the includes.
This includes typo fixes in enum v4l2_mpeg_video_multi_slice_mode,
MPEG-2 level and profile enums, new FWHT and H.264 Qp controls, new
RGB(A) formats, and new continuous bytestream and dynamic resolution
format flags.
There used to be some profile/level support in encoders. This code was moved to
GstV4l2Codecs and is now also used for decoders. The caps templates for the
H.264, H.265, MPEG4, VP8 and VP9 encoders and decoders should now reflect the
profiles and levels advertised by the kernel.
Allocator resources cannot be freed when a buffer pool is orphaned
while its buffers are in use. They should, however, be freed once those
buffers are no longer needed. This patch disposes of any buffers
belonging to an orphaned pool as they are released, and makes sure
that the allocator is cleaned up when the last buffer is returned.
When trying to orphan a buffer pool, successfully return and unref
the pool when the pool is either successfully stopped or orphaned.
Indicate failure and leave the pool untouched otherwise.
lockFocusIfCanDraw is deprecated in mac os 10.14. Apple suggests a
different way to do what that does, but for now, just suppress the deprecation.
There's no way to disable just that deprecation, so shut them all down.
OpenGL is also deprecated in mac os 10.14. There is a gentle way to
turn off just those deprecations (GL_SILENCE_DEPRECATION), but since
this commit turns them all off, that's moot.
https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/577
Recent kernels allow REQBUFS(0) on a queue that still has buffers in
use (mmapped or exported via dmabuf), orphaning all buffers on the queue.
If this is supported, the v4l2videodec element does not have to send a
drain request downstream.
Now that the v4l2allocator allows orphaning the V4L2 buffer queue, add
support for orphaning in the v4l2bufferpool. gst_v4l2_buffer_pool_orphan
can be used as a replacement for gst_v4l2_buffer_pool_stop, without
having to wait for buffers to be returned to the pool.
Recent kernels allow REQBUFS(0) on a queue that still has buffers in
use (mmapped or exported via dmabuf), orphaning all buffers on the queue.
Orphaning the allocator causes it to release all buffers with
REQBUFS(0), even if they are still in use. An orphaned allocator can
only be stopped. It can not be restarted or create new buffers.
Update to the latest installed headers (output of make headers_install)
from the media tree, keeping the slight modifications to the includes.
This includes new HEVC controls, the AdobeRGB -> opRGB rename, a new
capabilities field for v4l2_requestbuffers and v4l2_create_buffers, new
32-bit YUV formats, and request_fd changes.
The caps features were lost when sorting caps structures in
gst_v4l2src_fixate(). This was breaking alternate as
GST_CAPS_FEATURE_FORMAT_INTERLACED was removed from the caps.
The V4L2 elements already set the delta unit buffer flag when dequeueing
the buffer, but gst_video_encoder_finish_frame overwrites it from the
passed codec frame's sync point flag. Set the flag correctly.
This fixes a critical warning if the last-sample property is enabled:
(gst-launch-1.0:391): GStreamer-CRITICAL **: 01:12:57.428: gst_object_unref: assertion 'object != NULL' failed
If the allocation query does not contain any allocation pools,
gst_query_parse_nth_allocation_pool will leave the local pool,
min, and max variables undefined, so check the array length first.
If pool is NULL, do not call gst_object_unref.
The function gst_v4l2_object_add_interlace_mode() has repeating code so
it's best use a loop instead. That will make it easy and simple to add
additional interlace modes in a following patch.
The previous patch did not even compile on any possible platform or C
standard. That commit also didn't have a proper commit message.
Android ships Linux with a different signature for ioctl. They first
released an ioctl with int as request type, and later "fixed" it by
adding an override with unsign, which is still not matching Linux and
BSD implementation which uses unsigned long int.
If there was no interlace-mode field in the caps. Read back the value
selected by the driver. This way, if the driver does not support
progressive, then it will automatically negotiate the returned mode
unless this mode is not supported by GStreamer.
This method was already used for colorimetry. Just like colorimetry, the
interlace mode is not longer probed by v4l2src dues to performance
issues.
Fixes#511
There is no specific needs to duplicate the FD. Unlike the exportation,
we don't depend on code that will call close. This will make debugging
easyer since the traced FD will match the exporter.
... and fallback to gst_audio_info_set_format for not yet supported layouts.
Fix audio playback on iOS 12.
Based on patch from Byron Schiel <byron@canary.is>
https://bugzilla.gnome.org/show_bug.cgi?id=796919
This is to support Amlogic CODEC driver which does not provide a full
list of formats when the driver is initially opened. GStreamer does
not strictly need this full list initially, but only later, in order
to negotiate with downstream if multiple format can be selected.
With this change, we will no longer probe twice the device, since the
probed list can be directly used for negotation.
Add a debug message right before waiting for the driver.
This is useful in order to debug drivers without a properly
implemented decoder or encoder stop command.
This allow forcing going through the transform driver even if there
isn't an conversion happening. This is usedful when the m2m driver can
be used to adapt the type of memory between two drivers.
This method will check if a buffer, base on it's video meta,
can be imported. It will also try and adapt the request stride
in case this is the only that miss-match.
https://bugzilla.gnome.org/show_bug.cgi?id=583890
The preparation code imports the buffer, doing bunch of
validation. Only queue the buffer in the driver if the
importation worked. This way we don't rely on the driver
to validate.
https://bugzilla.gnome.org/show_bug.cgi?id=583890
This change has no effect. We will need to acquire a buffer from the
pool later in order to validate / adapt with the video alignment for
the downstream buffers.
https://bugzilla.gnome.org/show_bug.cgi?id=583890
Always initially use try_format(), delaying set_format() to when the
allocation is being negotiated. This avoid having two code paths, and
will be help adding support for properly importing buffers of specific
strides and offsets.
https://bugzilla.gnome.org/show_bug.cgi?id=583890
The desired colorimetry is logged with all parameters (colorpsace,
range, matrix, and transfer function), but of the values actually
set by the driver, only colorspace is logged. Complete the debug
log message to display all colorimetry parameters:
Desired colorspace is 8:1:1:1
Got format of 640x480, format YU12, nb planes 1, colorspace 8
->
Desired colorspace is 8:1:1:1
Got format of 640x480, format YU12, nb planes 1, colorspace 8:0:0:0
https://bugzilla.gnome.org/show_bug.cgi?id=796940
gstjpegdec sets 1:4:0:0 colorimetry (full range BT.601 YCbCr encoding
with unknown primaries and unknown transfer function). This currently
gets translated to bt601 or bt709 depending on resolution.
Both cases result in a negotiation failure:
ERROR: from element /GstPipeline:pipeline0/v4l2video0convert:v4l2video0convert0: Device '/dev/video0' does not support 1:4:0:0 colorimetry
Improve the guessing game by selecting JPEG colorimetry (JPEG colorspace
with sRGB transfer function) under these specific conditions, and loosen
the matching so that 1:4:0:0 input gets accepted if the device is
actually configured to 1:4:7:1 (V4L2_PIX_FMT_JPEG default).
https://bugzilla.gnome.org/show_bug.cgi?id=796940
Setting the priv field to a magic value stops V4L2 core from zeroing
the extended colorimetry fields quantization, ycbcr_enc, and xfer_func
for non-mplane queues.
https://bugzilla.gnome.org/show_bug.cgi?id=796940
The input from an v4l2 device that was used the first time was
remembered for next times, and set again always the pipeline is
set to READY state. This was making that users wasn't able to
select a different input without having to create a new pipeline.
This patch makes that v4l2src element forget previous used input
when going to NULL state, so it will check again for the current
selected input when going again to READY state. Users can change
to NULL state, select a new input with a VIDIOC_S_INPUT ioctl
and change to PLAYING again.
https://bugzilla.gnome.org/show_bug.cgi?id=796908
When the pool is started, we allocate and release buffer, expecting
the pool release-buffer handler to queue them. Though, as we rely
on release function, there is no direct way to detect that this
process didn't work.
To check this, validate that the number of queued buffer is the same
as the number of allocated buffers. This allow returning an error
when buffer importation was refused by the driver.
https://bugzilla.gnome.org/show_bug.cgi?id=583890
This will avoid sending EOS on v4l2src when a driver sends an empty
buffers. This case would be a bug in the driver, but yet the camera
should keep running.
This also removes the check for corrupted buffers, as this check is
already done later.
https://bugzilla.gnome.org/show_bug.cgi?id=794842
We were calling gst_v4l2_is_buffer_valid() before and inside
gst_v4l2_buffer_pool_qbuf() as we needed to access the group. The second
check failed since the writability of the buffer get inherited from the
GstMemory, which lead to pipeline failure. As we cannot avoid the extra
ref, it would be racy otherwise, just pass the group to _dbuf() so it
does not have to call gst_v4l2_is_buffer_valid() again.
https://bugzilla.gnome.org/show_bug.cgi?id=796692
Some camera firmware crash is TRY_FMT is called during streaming. As a
side effect. This try and detect that the same format as currently
running is about to be tried, and skip renegotiation.
https://bugzilla.gnome.org/show_bug.cgi?id=796789
In some cases, set_format() may get called twice before the output
format is set. Running an allocation query in this case is both not
needed and will cause assertion due tot he NULL caps.
The first converter to be found will now gain the name v4l2convert.
Other converters will be named after the m2m dev node end point they are
attached to.
https://bugzilla.gnome.org/show_bug.cgi?id=784958
When the decoder get linked further, it will receive a renegotiation
event from downstream. This case is not supported and should be ignored.
This fixes issues when this encoder is used inside an GstRtspServer
pipeline.
https://bugzilla.gnome.org/show_bug.cgi?id=796525
This fixes patch dd1c5aed65 which
pretended to always set colorimetry but the patch was incomplete.
This is again best effort considering the spec says that for CAPTURE
you may only read this value.
gst_v4l2_buffer_pool_flush() executes streamoff for the output, but
streamoff->streamon for the capture of the decoder.
gst_v4l2_buffer_pool_streamon() on capture assumes that is able to
resurrect the buffers from the pool, but acquiring buffers fails if the
buffer pool is still flushing.
The decoder needs to stop flushing the pools before calling
gst_v4l2_buffer_pool_flush() to restart the v4l2 device. Otherwise
starting the decoding thread might fail, because there are no buffers in
the capture pool.
This fixes a regression that was introduced in 97985a335c
("v4l2videodec: Add dynamic resolution change support").
https://bugzilla.gnome.org/show_bug.cgi?id=796681
The S_CROP call doesn't work on mem2mem output queues. Use the
S_SELECTION call to set the crop rectangle and only fall back to
S_CROP for ancient kernels.
This will allow v4l2videoenc to set the coded size on the output
queue via S_FMT and then set the visible size via the crop rectangle,
as required by the V4L2 codec API.
https://bugzilla.gnome.org/show_bug.cgi?id=796672
Some drivers need output buffers set before capture buffers.
CODA cannot set output format if capture is streaming.
Exynos MFC fails on output STREAMON if capture is already streaming.
This patch delays capture activation until output is configured and
streaming
https://bugzilla.gnome.org/show_bug.cgi?id=796693
Drop truncated frames regardless if they have the ERROR flag or not.
Truncated frame causes video frame map failure in many elements
including cluttersink, glupload etc.
In this patch we use a non-blocking poll in order to return all input
buffers (buffers from v4l2-output queue). This prevent holding too long
on upstreaming buffer in importing.
https://bugzilla.gnome.org/show_bug.cgi?id=794904
gst_v4l2_dup() will now take care of setting
v4l2capture->no_initial_format and keep_aspect instead of doing it
manually.
Fix a typo as keep_aspect was set twice on v4l2output but never on
v4l2capture.
https://bugzilla.gnome.org/show_bug.cgi?id=795028
libv4l2 does not prevent exporting DMABuf even when emulated formats are
in use. As a side effect, userspace ends up with buffers of the original
formats which will cause issues.
https://bugzilla.gnome.org/show_bug.cgi?id=795097
BT2020_12 is not represented in V4L2, so drivers providing full colority
for BT2020 will set V4L2_XFER_FUNC_709 transfer function. To fix the
issue, we bump this to BT2020_12 if the resoltion is 4K, but we should
only do that if the colorspace is BT2020 to start with, otherwise it's
not possible to use normal BT709 for 4K 8bit formats.
So far we were only setting colorimetry for OUTPUT devices (v4l2sink or
m2m sink pad). This prevented selecting through caps negotiation the
colorimetry for CAPTURE devices (v4l2src or m2m src pad). This is rarely
selectable, but trying is harmless.
As stated in commit c2956036b8 in -bad,
the wasapi elements are now better than directsound, and should be
preferred if they are available.
For a later release, once the elements have more testing, we can
consider moving them to -good.
Use value instead of version macro when testing for mac OS version,
since the define for the newer version may not be defined when
compiling against older versions.
https://bugzilla.gnome.org/show_bug.cgi?id=788402
The g_list_insert_sorted() will behave like prepend when the compare
function returns 0. In our case, we want to maintain the order hence
append. This fixes this issue and improve the sorting algorithm to make
a 10x10 prefered over 10x200 with a preference of 10x8 (and similar
cases which was badly handled). This fixes generally fixes issue were a
sub-optimal format / size is picked.
https://bugzilla.gnome.org/show_bug.cgi?id=792435
This implements a "big hammer" reallocation method. We effectively
drain and stop both side of the decoder and restart. This though is
the most generic method. This change should enable on most drivers
adaptive streaming.
https://bugzilla.gnome.org/show_bug.cgi?id=752962
This is problematic in the current design at it seriously slow down
startup of applications. As of now, no known application uses the
colorimetry and the interlace-modes for anything (the two fields that
won't be probed). So let's disable it, in the long term we'll try and
find a way to interact with the provider so applicaiton could opt-in
these slow probing methods for more advance configuration.
The difference between mmap and mmap64 is the type of 'offset' argument.
mmap64 always uses a 64-bit interger as offset, while mmap uses off_t,
whose size can vary on different operating systems or architectures.
However, not all operating systems support mmap64. Fortunately, although
FreeBSD only has mmap, its off_t is always 64-bit regardless of
architectures, so we can simply use mmap when sizeof(off_t) == 8.
https://bugzilla.gnome.org/show_bug.cgi?id=791779
This way we can pass the pad name instead of the element for tracing
which helps identifying which v4l2object is used withing M2M element
like decoder, encoder and transform. For the reference, pads are name
<parent-name>:<pad-name>.
Commit 1f31715c98 ("v4l2videodec: use visible size, not coded size,
for downstream negotiation filter") added support for removing the
padding obtained as the difference between width/height from G_FMT and
visible width/height from G_SELECTION from the probed caps obtained
via TRY_FMT.
This patch fixes the padding removal for drivers that only round up
height, but not width, to the padded frame size. This might happen
because horizontal padding can be handled by line stride (bytesperline),
but there is no such thing as plane stride in the V4L2 API for
single-buffer planar formats.
https://bugzilla.gnome.org/show_bug.cgi?id=791271
In MPLANE mode, the driver may set data_offset, which represent some
padding at the start of the buffer used internally. This portion of the
data need to be skipped, though it is included in bytesused.
This patch removes frame size sanity check as the method used will no
longer work. This check was simply there to help detect broken kernel
drivers. It would be re-implement by estimating the plane size, which is
not totally trivial and may be too much work for a simple debug check.
https://bugzilla.gnome.org/show_bug.cgi?id=733501
Issue an invalid VIDIOC_EXPBUF ioctl to the driver to check if the
driver supports dmabuf export. If the driver does not implement the
IOCTL, the error is ENOTTY. Any other error codes mean that the driver
implements VIDIOC_EXPBUF.
https://bugzilla.gnome.org/show_bug.cgi?id=779466
The purpose of being able to flush the buffer pool is only to
unlock any blocked operation. Doing streamoff/streamon had the
side effect of turning off and on the camera. As we do a flush_start
/ flush_stop sequence when shutting down, that would cause a really
quick sequence of streamoff/streamon/streamoff/close which was
causing some cameras to stop working.
https://bugzilla.gnome.org/show_bug.cgi?id=783945
The code relied on the list compare function to fixate the caps
but if the caps only has one structure, the compare function will
never get called. Capture device for which there is only one
structure in the caps would then get some assertion and later
fail badly.
Instead, fixate before inserting into the list and split the reading
and the fixation of the structures.
try_fmt will update the caps colorimetry and interlace-mode. Before this
call, those field are missing. The caps equality check was always
failing when a spurious reconfigure event was received.
These have been copy pasted all over the place and are not used anymore.
All object have it's own category now. This fixes build warning since
the VP9 decoder had vp8 category declared.
Whenever we import from downstream pool (userptr or dmabuf-import), we
should copy over the flags and timestamp, otherwise downstream will not
get proper synchronization or will not be able to notice frames that has
corruption in it.
https://bugzilla.gnome.org/show_bug.cgi?id=785680
