Commit graph

405 commits

Author SHA1 Message Date
He Junyan
0ace130107 glcolorconvert: Calculate correct swizzles for Y410, Y412 formats
The GST_VIDEO_FORMAT_Y410, GST_VIDEO_FORMAT_Y412_LE and GST_VIDEO_FORMAT_Y412_BE
formats in fact are packed formats, which have just 1 plane. But we have special
setting for them rather than using get_single_planar_format_gl_swizzle_order().

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5314>
2023-09-11 22:01:20 +00:00
Nicolas Dufresne
1eaa671feb glupload: Only allow non-linear formats with direct dmabuf uploaders
As we don't have any mapping from YUV formats + modifiers to an equivalent
emulated format (e.g. NV12 + modifier -> R8+modifier/RG88+modifier), do no
allow these formats to be used with the indirect DMABuf uploader.

Fixes #2942

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5270>
2023-09-11 19:55:19 +00:00
Nicolas Dufresne
5af78522b7 video: Use generated format order
Update the order according to what the generator produces.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5304>
2023-09-10 19:06:07 -04:00
Nicolas Dufresne
d1043226d7 video: Fix unknown big endian format list
That list contained multiple formats that did not exists.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5304>
2023-09-10 19:06:07 -04:00
Matthew Waters
fac5a83fd9 gl: add support for I420 10/12 bit formats
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5292>
2023-09-07 17:03:19 +00:00
Matthew Waters
95db85e664 gl/mixer: use the pad's video info instead of creating our own
The pad's video info will match the actual video format in use.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5290>
2023-09-07 02:12:29 +00:00
Philippe Normand
bb5d5e0066 glcontext_egl: Remove spurious mutex unlock
The context is not locked in any call site of the "failed" goto.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5291>
2023-09-06 12:03:42 +01:00
Matthew Waters
9d867356df gl/bufferpool: add configuration to extend buffer lifetime before reuse
Fixes a potential GPU stall if an immediately freed texture/buffer is
attempted to be reused immediately by the CPU, e.g. when uploading.

Problematic scenario is this:
1. element does GPU processing reading from texture
2. frees the buffer back to the pool
3. pool acquire returns the just released buffer
4. GPU processing then has to wait for the previous GPU operation to
   complete causing a stall

If there was a reliable way to know whether a buffer had been finished
with across all GPU drivers, we would use it.  However as that does not
exist, this workaround is to keep the released buffer unusable until the
next released buffer.

This is the same approach as is used in the qml (Qt5) elements.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5144>
2023-08-31 01:24:38 +00:00
Stephan Seitz
9d3b57f362 sdp: fix wrong error message for missing clock-rate in caps
When using `gst_sdp_media_set_media_from_caps` on `application/x-rtp` caps
without `clock-rate` it wrongly reports missing payload type even if `payload`
is present in the caps.

This seems to be a copy&paste error from the error message for missing payload
type.

When using payload=10, both `clock-rate` and some other media properties are
defined by the RTP standard so I was wondering whether I could omit `clock-rate`
and was confused about the error message.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5250>
2023-08-26 18:01:38 +00:00
Sebastian Dröge
d50c842d87 video: Fix ordering of video formats in GST_VIDEO_FORMATS_ALL_STR
This now follows the algorithm again that is described in the
documentation and implemented in gstreamer-rs.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5243>
2023-08-25 15:27:02 +00:00
Matthew Waters
a49cd5f94d gl: add support for 16-bit A420/A422/A444
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5233>
2023-08-24 12:13:19 +10:00
Matthew Waters
faf404a938 video: add support for A420/A422/A444 16-bit formats
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5233>
2023-08-24 12:03:39 +10:00
Matthew Waters
e67bdaf4d5 glcolorconvert: fix writing to subsampled yuv formats >8-bit
It's not quite enough to have the GstVideoInfo use a RGBA64 format, the
GstGLFormat (texture format) also needs to match as that's what actually
is used for allocation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5226>
2023-08-24 00:56:44 +00:00
Matthew Waters
4c627ea2bb gl: add support for 12-bit A444/A422/A420
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5226>
2023-08-24 00:56:44 +00:00
Matthew Waters
202309fa2c video: add support for 12-bit A420/A422/A444
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5226>
2023-08-24 00:56:43 +00:00
Matthew Waters
746d99b932 gl: add support A422/A444
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5224>
2023-08-23 15:00:05 +10:00
Matthew Waters
9a56945173 video: add support for 8-bit A422/A444
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5213>
2023-08-23 01:00:24 +00:00
Hou Qi
93e1bc94ae gl: wayland: fix memory leak when do egl_close
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5219>
2023-08-22 20:39:06 +00:00
Matthew Waters
3981257bc5 glcolorconvert: add support for 10-bit A420/A422/A444
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5209>
2023-08-22 12:18:42 +00:00
Matthew Waters
cc59841a45 gl/format: add support for RGBA64_LE/BE
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5209>
2023-08-22 12:18:42 +00:00
Jan Schmidt
6053650b85 audio: Make sure to stop ringbuffer on error
Add gst_audio_ring_buffer_set_errored() that will mark the
ringbuffer as errored only if it is currently started or paused,
so gst_audio_ringbuffer_stop() can be sure that the error
state means that the ringbuffer was started and needs stop called.

Fixes a crash with osxaudiosrc if the source element posts
an error, because the ringbuffer would not get stopped and CoreAudio
would continue trying to do callbacks.

Also, anywhere that modifies the ringbuffer state, make sure to
use atomic operations, to guarantee their visibility

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5205>
2023-08-21 08:50:45 +00:00
Matthew Waters
1f9a13da55 gl: support uploading/downloading 10-bit A420/A422/A444
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5193>
2023-08-17 01:55:12 +00:00
Jochen Henneberg
d086491909 rtpbasedepayload: Fixed HdrExt handling for aggregated output buffer
If a depayloader aggregates multiple RTP buffers into one buffer only
the last RTP buffer was checked for header extensions. Now the
depayloader remembers all RTP packets pushed before a output buffer is
pushed and checks all RTP buffers for header extensions.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4979>
2023-08-15 07:12:03 +02:00
Jan Schmidt
62f09513e5 audiobasesink: Don't wait on gap events
Don't call wait_event() at all for gap events, as basesink will
end up waiting for the time that the gap event would be rendered
out at the audio device. There's no need to render it at all,
just treat it as a handy point to resync the audio if needed,
let the ringbuffer render silence, and place the next buffer
into the ringbuffer where it belongs.

The only thing we really need to do is make sure the ringbuffer
and clock are running, and wait for preroll.

Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2749

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5178>
2023-08-14 14:57:16 +00:00
Olivier Crête
f3cd913b3c sdpmessage: Parse zero clock-rate as default
It seems there is at least one broken RTSP server out there that returns a clock-rate of 0.
Let's just ignore it and use the default in that case.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5171>
2023-08-14 08:26:59 +00:00
Nicolas Dufresne
d604b3655e video: Add Mediatek 10bit formats
These 10bit formats are identical to NV12_16L32S, but 64bytes of data is being
prefixed with 16bytes data with four pixels of lower 2bits per byte. For
MT2110T, the lower two bits set so each bytes contains a column of 4 pixels,
also describe as tiled lower 2 bits. MT2110T has been chosen as a name to match
the vendor chosen name. This format is unlikely to exist for other vendors.

For MT2110R, the 2 low bits are in raster order.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3444>
2023-08-08 16:08:16 +00:00
He Junyan
a9e9001778 video DMA: Add several video formats support
Add Y410, Y412, Y210, Y212, BGR10A2, P010 and P012 format support.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5159>
2023-08-08 12:29:20 +00:00
He Junyan
3cebfc7e84 glupload: Delete the flag of METHOD_FLAG_CAN_ACCEPT_RAW for DMABuf
When this flag is enabled, the transform_caps() simply set passthrough
to generate the raw caps. This is not correct, because the sink and
src have different format/drm-format fields.
We already add system memory conversion for DMABuf manner, so no more
need for this flag.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan
094270322b glupload: make nvmm upload manner only consider system memory
The current _nvmm_upload_transform_caps() only simply apply
"memory:NVMM" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:

  video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

it will be changed as

  video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

For GLMemory kind caps, no drm-format should appear.

So we should let it only transforms which it can recognize.

And in _nvmm_upload_accept(), we should only accept the "memory:NVMM"
feature in input caps.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan
122ffa56ed glupload: make directviv upload manner only consider system memory
The current _directviv_upload_transform_caps() only simply apply
"memory:GLMemory" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:

  video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

it will be changed as

  video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

For GLMemory kind caps, no drm-format should appear.

So we should let it only transforms which it can recognize.

And in _directviv_upload_accept(), we should only accept the system
memory as input caps.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan
b9c97ba76d glupload: make gl memory upload manner only consider memory:GLMemory feature
The current _gl_memory_upload_transform_caps() only simply apply
"memory:GLMemory" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:

  video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

it will be changed as

  video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

For GLMemory kind caps, no drm-format should appear.

So we should let it only transforms which it can recognize.

And in _gl_memory_upload_propose_allocation(), we should only allocate
the allocator and buffer pool for the caps with "memory:GLMemory"
feature.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan
414f91d0fa glupload: make meta upload manner only consider TextureUploadMeta feature
The current _upload_meta_upload_transform_caps() only simply apply
"memory:GLMemory" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:

  video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

it will be changed as

  video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

For GLMemory kind caps, no drm-format should appear.

So we should let it only transforms which it can recognize.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan
fe07763fd8 glupload: make raw manner only consider system memory
The current _raw_data_upload_transform_caps() only simply apply
"memory:GLMemory" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:

  video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

it will be changed as

  video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
  interlace-mode=(string)progressive, multiview-mode=(string)mono, \
  framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002

For GLMemory kind caps, no drm-format should appear.

So we should let it only transforms which it can recognize.
We also should recognize the system memory caps in _accept() early, if
the input is not system memory, we just return early.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan
cae46d36b5 glupload: change the accept function to import dmabuf with modifier
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan
21faad5eca glupload: enable drm kind caps in glupload plugin for DMA buf
Most of the time, the RGB kind formats are OpenGL native supported
format which has only one plane. They can be imported at one shot
using no matter DIRECT or INDIRECT mode.
While YUV kind formats which have multi planes have two ways to import.
They can be DIRECT imported, which requires GL_OES_EGL_image_external
extension. The output format should be RGBA and TARGET should be set
as OES after imported. The other way, they can be INDIRECT imported,
which makes each plane as a texture. In this mode, the imported textures
have different fourcc from the original format. For example, the NV12
format can be imported as a R8 texture for the first plane and RG88
texture for the second plane. The output TARGET should be sets as 2D
in this mode.

When converting sink caps to src caps, we first filter the feature of
"video/x-raw(memory:DMABuf)" and system memory. Then Based on the
external_only flag (INDIRECT mode does not care while DIRECT mode cares),
we transform the drm-format into the gst video format.

When converting src caps into sink caps, we first filter the correct
TARGET(INDIRECT mode contains 2D only while DIRECT mode contains 2D,
OES or both of them) gstructure. Then Based on the include_external flag
(INDIRECT mode always true while DIRECT mode depends on TARGET), we
transform the gst video format into drm-format.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan
96ea1b8da9 glupload: Add a helper function of _filter_caps_with_features()
We sometimes need to filter some special feature for the next step.
This helper function can help us to filter out un-needed features
in a caps.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan
95707bb80d glupload: add a helper function to convert drm formats into gst formats
Co-authored-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan
6dd3d6f1c1 glupload: add a helper function to convert gst formats into drm formats
Co-authored-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
He Junyan
22bb3107d7 gl: add a method to check whether DMA modifier is supported in EGL
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
2023-08-03 20:58:23 +00:00
Nirbheek Chauhan
eb9be6de78 glcolorconvert: Fix syntax for GLSL shaders
Shader compilation was failing on macOS:

gstglslstage.c:519:_compile_shader:<glslstage1> fragment shader compilation failed:
ERROR: 0:10: 'input_swizzle' : syntax error: Array size must appear after variable name

Co-authored-by: Matthew Waters <matthew@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5123>
2023-07-31 17:33:20 +05:30
Matthew Waters
7fcef3818e glcolorconvert: expose the swizzle glsl functions
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5109>
2023-07-28 03:29:09 +00:00
Matthew Waters
bff5d6d725 glcolorconvert: expose the YUV->RGB glsl function
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5109>
2023-07-28 03:29:09 +00:00
Matthew Waters
b32e2cd9dd gl: expose calculating swizzle indices to/from RGBA/YUVA
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5109>
2023-07-28 03:29:09 +00:00
Matthew Waters
a90f6d5d67 gl/format: add helper for returning the number of components in a GL format
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5109>
2023-07-28 03:29:09 +00:00
Matthew Waters
5bf85e7588 gl/colorconvert: use swizzle indices instead of chars
Allows using a uniform to control which value goes where and not rewrite
the shader for every format combination.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5109>
2023-07-28 03:29:08 +00:00
Matthew Waters
00316db5e5 gl/eagl: ignore OpenGLES deprecation on ios
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5111>
2023-07-27 13:05:37 +00:00
He Junyan
9e863976c9 video: dma: let gst_video_info_dma_drm_to_caps() return caps with DMABuf feature
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5107>
2023-07-26 20:49:02 +00:00
He Junyan
5299c34452 video: Add GST_VIDEO_FORMATS_STR to declare all video formats as a string
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5094>
2023-07-25 19:34:58 +00:00
Víctor Manuel Jáquez Leal
8b38614173 video: add GST_VIDEO_DMA_DRM_CAPS_MAKE
It is requried this macro because it doesn't define format field, neither
drm-format which is suppossed to be generated in run-time.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4981>
2023-07-19 09:57:50 +00:00
He Junyan
bc48588818 video: dma: Set DMA caps' format field to DMA_DRM
The current way of dma caps uses the drm-format to replace the orginal
format field. The absence of format field means it can accept all formats.
It causes problems when clipping with other old DMA or video/x-raw(ANY)
caps, the result will contain both format field and drm-format field,
which is not valid DMA caps.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4981>
2023-07-19 09:57:50 +00:00