Commit graph

423 commits

Author SHA1 Message Date
Nicolas Dufresne e00682a4c0 egl: Fix direct dmabuf upload without DMABuf caps feature
The DMAbuf accept function was ensuring the in_dma_info values was valid if
the in_caps have change. But the check was bogus since the in_caps was being
modified without a pointer change. As a side effect, on the second accept
call, the drm_fourcc was reset to 0, which cause the uploader to fallback.

Fix this by ensuring we always have a valid dma_frm info directly in the
set_caps() function. Also remove the bogus caps changed check and remove any
modification to the info structure and always do that inner checks.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5461>
2023-10-16 17:30:32 +00:00
Nicolas Dufresne a869760199 egl: Fix indirect dmabuf importation of none DRM formats
DRM Modifiers are not generically transferrable from a format like NV12 to
their indirect shading format (R8 / RG88). So the helper to this do needs
to be removed from our API.

To make things worse, we support indirect formats that aren't DRM format in
the first place. Notably NV12_16L32 (aka MM21) is not (yet) a DRM format. Yet,
each plane can be indirectly imported using R8/RG88 and a detiling shader.
This patch also removes this constraint restoring zero-copy playback on
Mediatek SoC.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5461>
2023-10-16 17:30:32 +00:00
Matthew Waters 714c78ac51 glcolorconvert: add A422/A444 to output switch statement
Fixes glcolorconvert writing to A422 or A444 formats.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5455>
2023-10-10 22:47:59 +11:00
Piotr Brzeziński 64a58c37ac glfilter: Only add parent meta if inbuf != outbuf
This was causing a memory leak in cases like `gltestsrc ! gltransformation scale-x=0.5 ! glimagesink`.
Parent meta was being added in assumption that those buffers are different, which was not the case here,
creating a reference loop and never freeing the buffer.

Co-authored-by: Matthew Waters <matthew@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5445>
2023-10-10 03:17:07 +00:00
Nicolas Dufresne 0eee03d688 videoaggregator: Avoid creating caps that are not in the template
By using the gst_caps_set_simple() to set the format on all structures, the
compositor may create invalid combinations as the caps may contain passthrough
caps. Avoid this issue by intersecting the resul with its original.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5386>
2023-10-03 21:13:00 +00:00
Nicolas Dufresne 0fc63b58d0 video: Add support for a list of any video formats
Adds list of formats that should be used by element in needs to passthrough
video. It contains the full list of video format plus DMA_DRM format
and will be extended in the future as needed. This patches includes 3 new
symbols:

  - GST_VIDEO_FORMATS_ANY_STR
  - GST_VIDEO_FORMATS_ANY
  - gst_video_formats_any()

The last one can be used by bindings or for code that prefers having
GstVideoFormat values instead of strings.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5386>
2023-10-03 21:13:00 +00:00
Sebastian Dröge 510f71fbe4 video-format: Fix up video formats ordering by quality
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5412>
2023-10-02 09:39:21 +03:00
Nicolas Dufresne 63da298b84 glupload: Fix always true compiler warning
There was an if case that was always true. After re-reading the Khronos
spec, this case is actually undefined. It is better to just always ignore
the format.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5410>
2023-09-28 20:56:35 +00:00
Jochen Henneberg d9cd8b375e rtpbasedepayload: Fixes for cleanup of header ext. aggregation
* Free delayed buffer on _finalize()
* Reset header aggregation cache and delayed buffer on state change
  from READY_TO_PAUSE

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5278>
2023-09-27 11:49:11 +00:00
Jochen Henneberg 372709400f rtpbasedepayload: Only aggregate RTP header extensions if in use
Right now we split the RTP header from the current buffer into a new
buffer and aggregate those buffers for later processing if the
depayloader creates an output buffer.

This is cumbersome as it happens even if none of the incoming RTP
buffers carries RTP header extensions at all just because header
aggregation has been enabled in the depayloader class.

This commit will start aggregation only in case that there really are
RTP header extensions available on an incoming RTP buffer. The check
is trivial and cheap. Once activated we keep aggregation active for
all buffers. The active state is reset on state change READY_TO_PAUSE.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5278>
2023-09-27 11:49:10 +00:00
Matthew Waters 237d195209 gl/format: fix swizzle indices for RGB10A2/BGR10A2
Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3003
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5400>
2023-09-27 19:45:57 +10:00
Daniel Moberg 7446839e0d rtspconnection: Add API for adding extra http request headers
This commit adds capability to add custom headers to any http requests
during http tunnel mode. If header exist new header will replace old.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5268>
2023-09-26 06:35:43 +00:00
Seungha Yang 69d1679914 video: Add GBR 16bits formats
Adding 16bits planar RGB formats

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5375>
2023-09-23 13:12:55 +00:00
Eric f7cb79874a rtspconnection: Ignore trailing whitespace in headers
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5358>
2023-09-21 22:06:22 +00:00
Michiel Westerbeek 03bf8e9386 video-scaler, audio-resampler: downgrade 'can't find exact taps' to debug
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5369>
2023-09-20 16:31:47 +00:00
Maksym Khomenko 5b252a1511 appsrc: use macros for getters/setters
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5321>
2023-09-20 08:54:01 +00:00
Maksym Khomenko 60591960c3 appsink: add max-time and max-buffers properties
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5321>
2023-09-20 08:54:01 +00:00
Maksym Khomenko 4c13ccec16 appsrc: extract buffering level calculations
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5321>
2023-09-20 08:54:01 +00:00
He Junyan 0ace130107 glcolorconvert: Calculate correct swizzles for Y410, Y412 formats
The GST_VIDEO_FORMAT_Y410, GST_VIDEO_FORMAT_Y412_LE and GST_VIDEO_FORMAT_Y412_BE
formats in fact are packed formats, which have just 1 plane. But we have special
setting for them rather than using get_single_planar_format_gl_swizzle_order().

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5314>
2023-09-11 22:01:20 +00:00
Nicolas Dufresne 1eaa671feb glupload: Only allow non-linear formats with direct dmabuf uploaders
As we don't have any mapping from YUV formats + modifiers to an equivalent
emulated format (e.g. NV12 + modifier -> R8+modifier/RG88+modifier), do no
allow these formats to be used with the indirect DMABuf uploader.

Fixes #2942

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5270>
2023-09-11 19:55:19 +00:00
Nicolas Dufresne 5af78522b7 video: Use generated format order
Update the order according to what the generator produces.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5304>
2023-09-10 19:06:07 -04:00
Nicolas Dufresne d1043226d7 video: Fix unknown big endian format list
That list contained multiple formats that did not exists.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5304>
2023-09-10 19:06:07 -04:00
Matthew Waters fac5a83fd9 gl: add support for I420 10/12 bit formats
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5292>
2023-09-07 17:03:19 +00:00
Matthew Waters 95db85e664 gl/mixer: use the pad's video info instead of creating our own
The pad's video info will match the actual video format in use.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5290>
2023-09-07 02:12:29 +00:00
Philippe Normand bb5d5e0066 glcontext_egl: Remove spurious mutex unlock
The context is not locked in any call site of the "failed" goto.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5291>
2023-09-06 12:03:42 +01:00
Matthew Waters 9d867356df gl/bufferpool: add configuration to extend buffer lifetime before reuse
Fixes a potential GPU stall if an immediately freed texture/buffer is
attempted to be reused immediately by the CPU, e.g. when uploading.

Problematic scenario is this:
1. element does GPU processing reading from texture
2. frees the buffer back to the pool
3. pool acquire returns the just released buffer
4. GPU processing then has to wait for the previous GPU operation to
   complete causing a stall

If there was a reliable way to know whether a buffer had been finished
with across all GPU drivers, we would use it.  However as that does not
exist, this workaround is to keep the released buffer unusable until the
next released buffer.

This is the same approach as is used in the qml (Qt5) elements.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5144>
2023-08-31 01:24:38 +00:00
Stephan Seitz 9d3b57f362 sdp: fix wrong error message for missing clock-rate in caps
When using `gst_sdp_media_set_media_from_caps` on `application/x-rtp` caps
without `clock-rate` it wrongly reports missing payload type even if `payload`
is present in the caps.

This seems to be a copy&paste error from the error message for missing payload
type.

When using payload=10, both `clock-rate` and some other media properties are
defined by the RTP standard so I was wondering whether I could omit `clock-rate`
and was confused about the error message.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5250>
2023-08-26 18:01:38 +00:00
Sebastian Dröge d50c842d87 video: Fix ordering of video formats in GST_VIDEO_FORMATS_ALL_STR
This now follows the algorithm again that is described in the
documentation and implemented in gstreamer-rs.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5243>
2023-08-25 15:27:02 +00:00
Matthew Waters a49cd5f94d gl: add support for 16-bit A420/A422/A444
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5233>
2023-08-24 12:13:19 +10:00
Matthew Waters faf404a938 video: add support for A420/A422/A444 16-bit formats
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5233>
2023-08-24 12:03:39 +10:00
Matthew Waters e67bdaf4d5 glcolorconvert: fix writing to subsampled yuv formats >8-bit
It's not quite enough to have the GstVideoInfo use a RGBA64 format, the
GstGLFormat (texture format) also needs to match as that's what actually
is used for allocation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5226>
2023-08-24 00:56:44 +00:00
Matthew Waters 4c627ea2bb gl: add support for 12-bit A444/A422/A420
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5226>
2023-08-24 00:56:44 +00:00
Matthew Waters 202309fa2c video: add support for 12-bit A420/A422/A444
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5226>
2023-08-24 00:56:43 +00:00
Matthew Waters 746d99b932 gl: add support A422/A444
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5224>
2023-08-23 15:00:05 +10:00
Matthew Waters 9a56945173 video: add support for 8-bit A422/A444
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5213>
2023-08-23 01:00:24 +00:00
Hou Qi 93e1bc94ae gl: wayland: fix memory leak when do egl_close
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5219>
2023-08-22 20:39:06 +00:00
Matthew Waters 3981257bc5 glcolorconvert: add support for 10-bit A420/A422/A444
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5209>
2023-08-22 12:18:42 +00:00
Matthew Waters cc59841a45 gl/format: add support for RGBA64_LE/BE
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5209>
2023-08-22 12:18:42 +00:00
Jan Schmidt 6053650b85 audio: Make sure to stop ringbuffer on error
Add gst_audio_ring_buffer_set_errored() that will mark the
ringbuffer as errored only if it is currently started or paused,
so gst_audio_ringbuffer_stop() can be sure that the error
state means that the ringbuffer was started and needs stop called.

Fixes a crash with osxaudiosrc if the source element posts
an error, because the ringbuffer would not get stopped and CoreAudio
would continue trying to do callbacks.

Also, anywhere that modifies the ringbuffer state, make sure to
use atomic operations, to guarantee their visibility

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5205>
2023-08-21 08:50:45 +00:00
Matthew Waters 1f9a13da55 gl: support uploading/downloading 10-bit A420/A422/A444
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5193>
2023-08-17 01:55:12 +00:00
Jochen Henneberg d086491909 rtpbasedepayload: Fixed HdrExt handling for aggregated output buffer
If a depayloader aggregates multiple RTP buffers into one buffer only
the last RTP buffer was checked for header extensions. Now the
depayloader remembers all RTP packets pushed before a output buffer is
pushed and checks all RTP buffers for header extensions.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4979>
2023-08-15 07:12:03 +02:00
Jan Schmidt 62f09513e5 audiobasesink: Don't wait on gap events
Don't call wait_event() at all for gap events, as basesink will
end up waiting for the time that the gap event would be rendered
out at the audio device. There's no need to render it at all,
just treat it as a handy point to resync the audio if needed,
let the ringbuffer render silence, and place the next buffer
into the ringbuffer where it belongs.

The only thing we really need to do is make sure the ringbuffer
and clock are running, and wait for preroll.

Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2749

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5178>
2023-08-14 14:57:16 +00:00
Olivier Crête f3cd913b3c sdpmessage: Parse zero clock-rate as default
It seems there is at least one broken RTSP server out there that returns a clock-rate of 0.
Let's just ignore it and use the default in that case.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5171>
2023-08-14 08:26:59 +00:00
Nicolas Dufresne d604b3655e video: Add Mediatek 10bit formats
These 10bit formats are identical to NV12_16L32S, but 64bytes of data is being
prefixed with 16bytes data with four pixels of lower 2bits per byte. For
MT2110T, the lower two bits set so each bytes contains a column of 4 pixels,
also describe as tiled lower 2 bits. MT2110T has been chosen as a name to match
the vendor chosen name. This format is unlikely to exist for other vendors.

For MT2110R, the 2 low bits are in raster order.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3444>
2023-08-08 16:08:16 +00:00
He Junyan a9e9001778 video DMA: Add several video formats support
Add Y410, Y412, Y210, Y212, BGR10A2, P010 and P012 format support.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5159>
2023-08-08 12:29:20 +00:00
He Junyan 3cebfc7e84 glupload: Delete the flag of METHOD_FLAG_CAN_ACCEPT_RAW for DMABuf
When this flag is enabled, the transform_caps() simply set passthrough
to generate the raw caps. This is not correct, because the sink and
src have different format/drm-format fields.
We already add system memory conversion for DMABuf manner, so no more
need for this flag.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan 094270322b glupload: make nvmm upload manner only consider system memory
The current _nvmm_upload_transform_caps() only simply apply
"memory:NVMM" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:

  video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

it will be changed as

  video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

For GLMemory kind caps, no drm-format should appear.

So we should let it only transforms which it can recognize.

And in _nvmm_upload_accept(), we should only accept the "memory:NVMM"
feature in input caps.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan 122ffa56ed glupload: make directviv upload manner only consider system memory
The current _directviv_upload_transform_caps() only simply apply
"memory:GLMemory" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:

  video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

it will be changed as

  video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

For GLMemory kind caps, no drm-format should appear.

So we should let it only transforms which it can recognize.

And in _directviv_upload_accept(), we should only accept the system
memory as input caps.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan b9c97ba76d glupload: make gl memory upload manner only consider memory:GLMemory feature
The current _gl_memory_upload_transform_caps() only simply apply
"memory:GLMemory" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:

  video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

it will be changed as

  video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

For GLMemory kind caps, no drm-format should appear.

So we should let it only transforms which it can recognize.

And in _gl_memory_upload_propose_allocation(), we should only allocate
the allocator and buffer pool for the caps with "memory:GLMemory"
feature.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan 414f91d0fa glupload: make meta upload manner only consider TextureUploadMeta feature
The current _upload_meta_upload_transform_caps() only simply apply
"memory:GLMemory" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:

  video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

it will be changed as

  video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

For GLMemory kind caps, no drm-format should appear.

So we should let it only transforms which it can recognize.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00