Commit graph

24 commits

Author SHA1 Message Date
Piotr Brzeziński
9cbe9a52fe vtenc: Fix deadlock after GST_FLOW_ERROR is received on frame push
This was easy to trigger when testing with e.g. vtenc ! vtdec ! glimagesink and closing the sink via window button,
causing GST_FLOW_ERROR to be received by the output loop, stopping it with the queue still full. This made the
enqueue_buffer() callback to lock waiting for space in our queue, while handle_frame() was waiting for the internal
VideoToolbox queue to free up, so that VTCompressionSessionEncodeFrame could finish. As the output loop was not
running, both functions waited forever.

Fixed by 1) immediately emptying our queue when GST_FLOW_ERROR is received (like we already did with _FLUSHING)
and 2) unconditionally setting the flushing flag in finish_encoding() when it sees the output loop stopped because
of GST_FLOW_ERROR, so that enqueue_buffer() will immediately discard any new frames coming out of VideoToolbox.
Both of those make sure we never run into the both-queues-full scenario.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5303>
2023-09-20 13:10:02 +00:00
Piotr Brzeziński
510e227996 vtenc: Fix cosmetic issues
Removes accidental GST_VIDEO_DECODER macro usage.
Also sets the initial size of the output queue to avoid having to expand when first buffers come through.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5164>
2023-08-09 07:35:32 +00:00
Piotr Brzeziński
dc494d9edb vtenc: Move output loop to a separate thread
vtenc has an async output queue, which we only iterate over after another frame is enqueued.
At the very least it means we're always a frame behind the fastest possible output.
In edge cases it's also bug-prone - for example if we only have 1 frame, the downstream caps negotiation
will never happen.

This commit adds a separate task running on the source pad, which only iterates over the output queue
and pushes frames out as soon as they're put there. The queue length is limited to ensure we don't encode
too far ahead compared to what downstream can consume. Any failures that occur when pushing data downstream
will be signalled in self->downstream_ret so that other parts of code can act accordingly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4967>
2023-07-21 13:05:26 +00:00
Andoni Morales Alastruey
fc6e6fb547 vtenc: remove duplicated framerate and size variables
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4282>
2023-06-19 08:37:55 +00:00
Andoni Morales Alastruey
23812bbc92 vtenc: apply DTS offset to ensure DTS <= PTS
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4282>
2023-06-19 08:37:54 +00:00
Tim-Philipp Müller
81716a1097 gst-plugins-bad: re-indent with GNU indent 2.2.12
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4182>
2023-03-17 03:18:54 +00:00
Nirbheek Chauhan
2d6da02653 applemedia: Consolidate GstVideoFormat <-> CVPixelFormat conversion
In the process we have changed the color value range from video-range
to full-range, which is probably what people want in the first place.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4135>
2023-03-14 23:35:16 +00:00
Piotr Brzeziński
b3830b08fd vtenc: Fix checking for certain CPU variants when running in VMs
These checks were introduced to prevent exposing ARGB64/RGBA64 in the caps
when running on M1 Pro/Max with macOS <13 because of a bug in VideoToolbox.
Unfortunately, the initial buffer size of 15 is too short when running
in a VM - the CPU brand string there looks like "Apple M1 Pro (Virtual)",
which due to its length causes sysctlbyname to return -1, resulting in
broken formats still showing up in the caps.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4080>
2023-02-28 16:00:07 +00:00
Piotr Brzeziński
04615ca55a vtenc/vtdec: Fix typo in RGBA64_LE availability define
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3912>
2023-02-13 17:48:37 +00:00
Piotr Brzeziński
8d81c4827c vtenc: Disable ARGB/RGBA64 caps on M1 Pro/Max with macOS <13
Fixes #1358.
Passing ARGB64/RGBA64 to vtenc caused the encoding to fail
when running on M1 Pro/Max variants with macOS 12.x, so let's
remove these formats from caps when such scenario is detected.
This issue appears to have been fixed OS-side in macOS 13.0.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3912>
2023-02-13 17:48:37 +00:00
Piotr Brzeziński
ae200a4d62 vtenc: Disable HW acceleration for interlaced ProRes
Due to a bug in the VT API, attempting to encode interlaced content
with ProRes results in an error, halting the pipeline instead of
gracefully falling back to software encoding.
Should be removed in the future if Apple ever fixes this issue.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3222>
2023-02-02 06:18:02 +00:00
Piotr Brzeziński
6bf15124e7 applemedia: Add HEVC support to vtenc and vtdec
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2754>
2022-07-16 05:20:56 +00:00
Nirbheek Chauhan
bb32532f60 applemedia: Disable 64RGBALE support on older macOS
The kCVPixelFormatType_64RGBALE enum is only available on macOS Big
Sur (11.3) and newer. We also cannot use that while configuring the
encoder or decoder on older macOS.

Define the symbol unconditionally, but only use it when we're running
on Big Sur with __builtin_available().

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1613>
2022-02-01 16:57:04 +00:00
Nirbheek Chauhan
6ae4497c07 applemedia: Remove some unnecessary variables
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1613>
2022-02-01 16:57:04 +00:00
Nirbheek Chauhan
f1449ac116 docs: Rename "OS X" to "macOS" in some documentation
No one uses the term "Mac OS X" anymore, it's "macOS". "OS X" is even
worse, because people will usually start the search with "mac".

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1607>
2022-01-30 19:17:18 +05:30
Nirbheek Chauhan
d05fcadf4e applemedia: Document vtenc / vtdec elements
Also preserve-alpha property should only be exposed on the
vtenc_prores element since h264 does not support transparency.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-docs/-/issues/94

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1601>
2022-01-30 02:17:24 +05:30
Nirbheek Chauhan
6b7d819c25 vtenc: Signal ignored alpha component with ProRes
When the image is opaque but the output ProRes format has an alpha
component (4 component, 32 bits per pixel), Apple requires that we
signal that it should be ignored by setting the depth to 24 bits per
pixel. Not doing so causes the encoded files to fail validation.

So we set that in the caps and qtmux sets the depth value in the
container, which will be read by demuxers so that decoders can skip
those bytes entirely. qtdemux does this, but vtdec does not use this
information at present.

The sister change was made in qtmux and qtdemux in:
https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/merge_requests/1061

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1489>
2022-01-04 18:01:54 +00:00
Nirbheek Chauhan
b8d6fd905e applemedia: Add ARGB64_BE, RGBA64_LE support to vtenc/vtdec
We can add this now that ARGB64_BE videoconvert support was added in:
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1247

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1214>
2021-11-01 09:24:52 +00:00
Nirbheek Chauhan
fed69c48b6 vtenc: Add FieldDetail properties for interlaced input
Standard interlace handling:
* If we have interlace-mode=interleaved and the field order, we just
  set it when creating the session
* If we have interlace-mode=(interleaved|mixed) and no field order, we
  set the field order on the first buffer

The encoder session does not support changing the FieldDetail after it
has started encoding frames, so we cannot support mixed streams
correctly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1214>
2021-11-01 09:24:52 +00:00
Nirbheek Chauhan
29a32703a5 vtenc: Add a property to forcibly ignore alpha values
This PropertyKey is not documented in any headers anywhere, so we need
to define it ourselves.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1214>
2021-11-01 09:24:52 +00:00
Nirbheek Chauhan
4e37f99975 vtenc: Set colorimetry information
It looks like VideoToolbox doesn't support all our colorimetries.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1214>
2021-11-01 09:24:52 +00:00
Nirbheek Chauhan
0f0e68080c applemedia: Add ProRes support to vtenc and vtdec
For vtdec, we continue to prefer NV12; else we pick whatever
downstream wants. In the special case where we're decoding 10-bit or
12-bit ProRes formats, we will prefer AYUV64.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1214>
2021-11-01 09:24:52 +00:00
Nirbheek Chauhan
db638134e7 vtenc: Improve error reporting in chain function
Otherwise it is quite difficult to figure out why the chain function
failed. Also assert not reached for case statements that should not be
hit.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1214>
2021-11-01 09:24:52 +00:00
Thibault Saunier
019971a3c7 Move files from gst-plugins-bad into the "subprojects/gst-plugins-bad/" subdir 2021-09-24 16:14:36 -03:00
Renamed from sys/applemedia/vtenc.c (Browse further)