Commit graph

169 commits

Author SHA1 Message Date
Nicolas Dufresne
98b44fdb46 video: Add support for linear 32x32 NV12 tiles
This adds linear 32x32 NV12 based tiles. This format is notably used by
Allwinner VCU and exposed in V4L2 as being "SUNXI Tiled" format. In this
patch we generalize the plane info calculation so we can share this part
with the 4L4 variant.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/754>
2020-07-14 21:43:56 -04:00
Nicolas Dufresne
7d1028424c video: Add NV12_4L4 tile format
This format is produced by Verisillicon VC8000D VPU decoder, it is a simple 4x4
tiling layout in a linear way.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/753>
2020-07-14 17:33:31 +00:00
Sebastian Dröge
81e7cd1fca video-converter: Add scaling fast-path for GBRA format
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/684

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/749>
2020-07-10 12:25:56 +00:00
Nicolas Dufresne
dcc7242ba5 videoconverter: Relax frame size checks
Since we are using VideoMeta, the converter (similarly to the video_frame_copy
utility) should have no issue dealing with frames that are slightly larger.
This situation occure as some element will use padded width/height for
allocation, which results in a VideoMeta width/height being larger then the
display width/height found in the negotiated caps.

Fixes #790

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/747>
2020-07-09 20:35:05 +00:00
Vivia Nikolaidou
ad55d3ce9d video-converter: Make fast path work for equivalent transfer functions
For example, BT709, BT601, and BT2020_10 all have theoretically
different transfer functions, but the same function in practice. In
these cases, we should use the fast path for negotiating. Also,
BT2020_12 is essentially the same as the other three, just with one more
decimal point, so it gives the same result for fewer bits. This is now
also aliased to the former three.

Also make videoconvert do passthrough if the caps have equivalent
transfer functions but are otherwise matching.

As of the previous commit, we write the correct transfer function for
BT601, instead of the (functionally identical but different ISO code)
transfer function for BT709. Files created using GStreamer prior to that
commit write the wrong transfer function for BT601 and are, strictly
speaking, 2:4:5:4 instead. However, this commit takes care of
negotiation, so that conversions from/to the same transfer function are
done using the fast path.

Fixes #783

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/724>
2020-07-03 11:57:49 +03:00
Jan Schmidt
205bb066ed video-converter: Add checks for configuration sanity.
If the cropping or scaling input or output rects put us completely
outside the input/output frame respectively, we can't draw anything
except black safely. Check for those conditions and don't set up a
configuration that attempts to access out of bounds memory outside
the input/output framebuffers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/696>
2020-06-12 06:49:56 +00:00
Jan Schmidt
bf5d51c5da video-converter: Guard against invalid frame input
If the frames passed in to gst_video_converter_frame()
have a different layout than was configured for, the
conversion code might go out of bounds and crash.

Do a sanity check on each frame passed in, and in the
absence of a return value in the API, just
refuse the conversion in invalid cases and leave the
destination frame untouched so it's obvious to
users that it was broken.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/696>
2020-06-12 06:49:56 +00:00
Haihao Xiang
7816cbf9a4 video: add support for Y412 format
Y412 is a packed 12 bits 4:4:4:4 format in the order U, Y, V, A, 2 bytes
per component with the color value stored in the 12 most significant
bits

Refer to https://github.com/torvalds/linux/blob/master/include/uapi/drm/drm_fourcc.h#L182
for the LE variant
2020-03-27 13:19:34 +00:00
Haihao Xiang
ea063721d2 video: add support for Y212 format
Y212 is a packed 12 bits 4:2:2 format in the order Y, U, Y, V, 2 bytes
per component with the color value stored in the 12 most significant
bits

Refer to https://github.com/torvalds/linux/blob/master/include/uapi/drm/drm_fourcc.h#L174
for the LE variant
2020-03-27 13:19:34 +00:00
Haihao Xiang
e4e3fc5666 video: add support for P012 format
P012 is a semi-planar 4:2:0 format, 2bytes per component with the value
stored in the 12 most significant bits
2020-03-16 21:01:27 +00:00
Edward Hervey
df35256dad video-converter: Fix RGB-XYZ-RGB conversion
The matrices were in the wrong order.
Instead of the conversion matrix being
  _ XYZ_TO_RGB_output * RGB_TO_XYZ_input * input_RGB
It was
  _ RGB_TO_XYZ_input * XYZ_TO_RGB_output * input_RGB
2019-11-21 13:46:38 +00:00
Sebastian Dröge
2bc1b38825 video-converter: Use G_N_ELEMENTS instead of writing it out manually 2019-10-22 09:50:27 +03:00
Sebastian Dröge
bcc1d666ec video-converter: Scaling-only fast-paths for planar formats are keeping interlacing
So we can also use them for interlaced content and speed up scaling a
bit.
2019-10-22 09:50:27 +03:00
Sebastian Dröge
255beae3ec video-converter: Don't use fast-path halfing/doubling rescalers for interlaced planar formats
They will mix lines from the top and bottom field and by that
effectively deinterlace the video, badly.
2019-10-22 09:49:43 +03:00
Sebastian Dröge
565b797f57 video-converter: Use interlaced scalers for vertical scaling if needed
Without this, scaling e.g. interlaced UYVY causes corrupted output with
lines as follows: f1 f1 f2 f2, i.e. two lines of each field and only
then the other field.
2019-10-22 09:49:18 +03:00
Thibault Saunier
909baa2360 Pass the code through codespell 2019-08-30 13:05:36 +00:00
Seungha Yang
c28721d89f video: Add P016 LE/BE formats
Add semi-planar 4:2:0 16 bits format.
2019-08-07 21:54:07 +00:00
Seungha Yang
90cf991337 video: Add Y444_16LE and Y444_16BE formats
Add 16 bits planar 4:4:4 YUV formats.
2019-08-07 12:10:47 +00:00
Thibault Saunier
287897e465 doc: Fix some gtk-doc comments 2019-05-13 11:34:08 -04:00
Seungha Yang
f4b054d349 video: Add RGB10A2_LE pixel format
Packed 10 bits per each R, G and B channel with MSB 2bits alpha channel.
This format is mapped to Windows' DXGI_FORMAT_R10G10B10A2_UNORM format which is
required for 10bits HDR rendering.

Note that this RGB10A2_LE format is R - B channel swapped version of BGR10A2_LE
2019-05-05 08:12:29 +00:00
Haihao Xiang
3b6415b421 video: add BGR10A2_LE pixel format
This pixel format is a packed 4:4:4 RGB with alpha channel (B-G-R-A),
R/G/B channel has 10 bits and alpha channel is the MSB 2 bits.
2019-04-04 01:59:18 +00:00
Haihao Xiang
90f8cca04e video: add VUYA pixel format
AYUV in gstreamer was defined in A-Y-U-V order in memory[1], however
Microsoft defined another AYUV format in V-U-Y-A order in memory[2]. Add
VUYA format for the latter in order to distinguish the two formats

[1] https://gstreamer.freedesktop.org/documentation/design/mediatype-video-raw.html#formats
[2] https://docs.microsoft.com/en-us/windows/desktop/medfound/recommended-8-bit-yuv-formats-for-video-rendering#ayuv)
2019-03-21 23:05:09 +00:00
Wim Taymans
a6552ee02e video-converter: fix number of allocated lines
We make an allocator for temporary lines and then use this for all
the steps in the conversion that can do in-place processing.

Keep track of the number of lines each step needs and use this to
allocate the right number of lines.

Previously we would not always allocate enough lines and we would
end up with conversion errors as lines would be reused prematurely.

Fixes #350
2019-01-11 11:47:51 -05:00
Edward Hervey
bb6b518506 video-converter: Remove unused variable/calculation
Since the refactoring in cdd86d025a
calculating the stride was no longer needed in setup_scale.
2018-12-17 12:21:01 +01:00
Wangfei
0ac7d1187b video: add Y410 pixel format.
This pixel format is packed format with 4:4:4 sample. And 10
available bits of Y/U/V and 2 available bits of alpha stored
in 4 Bytes.

Format defined in:
https://docs.microsoft.com/en-us/windows/desktop/medfound/10-bit-and-16-bit-yuv-video-formats
2018-12-01 21:05:36 -05:00
Wangfei
cc0bcd5856 video: add Y210 pixel format.
This pixel format is packed format with 4:2:2 sample and 10
available bits of each channel.

https://bugzilla.gnome.org/show_bug.cgi?id=797267
2018-10-31 15:01:34 +00:00
ayaka
388b48511e video: Add NV12_10LE40 pixel format
This pixel format is a fully packed variant of NV12_10LE32,
a luma pixel would take 10bits in memory, without any
filled bits between pixels in a stride. The color range
follows the BT.2020 standard.

In order to get a better performance in hardware memory
operation, it may expend the stride, append zero data at the
end of echo lines.

Pack function by Nicolas Dufresne.

https://bugzilla.gnome.org/show_bug.cgi?id=795462

Signed-off-by: Nicolas Dufresne <nicolas@ndufresne.ca>
Signed-off-by: ayaka <ayaka@soulik.info>
2018-05-19 13:09:08 -04:00
Edward Hervey
1a1570bb1e video: Silence "restrict" issues with ORC code
The problem is that even though the functions we are calling are
in-place transformation, orc automatically puts the restrict keyword
on all arguments. To silence that warning just create yet-another
variable containing the same value.

https://bugzilla.gnome.org/show_bug.cgi?id=795765
2018-05-05 10:27:12 +02:00
Nicolas Dufresne
9a3ee4838f Revert "video: Add NV12_10LE40 pixel format"
This reverts commit 35d0783fca.
2018-04-24 17:05:17 -04:00
ayaka
35d0783fca video: Add NV12_10LE40 pixel format
This pixel format is a fully packed variant of NV12, a luma
pixel would take 10bits in memory, without any filled bits
between pixels in a stride. The color range follows
the BT.2020 standard.

In order to get a performance in hardware memory
operation, it may expend the stride, append zero data at the
end of echo lines.

Signed-off-by: ayaka <ayaka@soulik.info>

https://bugzilla.gnome.org/show_bug.cgi?id=795462
2018-04-24 16:51:27 -04:00
Nicolas Dufresne
c256e96047 video: Add NV16_10LE32 support
This adds a 10 bit variant for NV16 packed into 32 bits little endian
words. The MSB 2 bits are padding. This format is used on Xilinx SoC and
identified with the FOURCC XV20.

https://bugzilla.gnome.org/show_bug.cgi?id=789876
2018-01-29 16:01:34 +00:00
Nicolas Dufresne
f7a27d7403 video: Add GRAY10_LE32 support
This add a 10bit variant of gray scale packed into 32bits little endian
words. The MSB 2 bits are padding and should be ignored. This format is
used on Xilinx SoC and is identified with the FOURCC XV10.

https://bugzilla.gnome.org/show_bug.cgi?id=789876
2018-01-29 16:01:34 +00:00
Nicolas Dufresne
2b9725d0df video: Add NV12_10LE32 support
This adds a 10bit variant for NV12 which packs 3 10bit components
into little endian 32bit words. The MSB 2 bits are padding and should be
ignored. This format is used on Xilinx SoC and is identified with there
with the FOURCC XV15

https://bugzilla.gnome.org/show_bug.cgi?id=789876
2018-01-29 16:01:34 +00:00
Edward Hervey
31d5ac15b2 video-converter: Fix undefined left shift
Cast value to target type
2017-11-04 12:18:39 +01:00
Vincent Penquerc'h
32ef8f54d4 video-converter: fix scaler leak
https://bugzilla.gnome.org/show_bug.cgi?id=780764
2017-03-31 16:14:50 +01:00
Sebastian Dröge
77f802f8e6 video: Add I420/I422/Y444_12LE/BE and GBRA video formats
https://bugzilla.gnome.org/show_bug.cgi?id=780100
2017-03-16 17:32:27 +02:00
Emeric Grange
2fcab9e7ef video: Add GBRA_10LE/BE, GBR_12LE/BE, GBRA_12LE/BE pixel formats
With additional bugfixes from Sebastian Dröge <sebastian@centricular.com>

https://bugzilla.gnome.org/show_bug.cgi?id=780100
2017-03-16 16:44:12 +02:00
Thibault Saunier
099ac9faf2 docs: Convert gtkdoc comments to markdown
Modernizing the documentation, making it simpler to read an
modify and allowing us to possibly switch to hotdoc in the
future.
2017-03-10 18:19:17 -03:00
Sebastian Dröge
0f7b9dbe75 video-converter: Give a name to the thread-pool threads
This way they can be distinguished from any other threads in the same
process.
2017-02-24 10:04:21 +02:00
Sebastian Dröge
6d20fcc9df video-converter: Only lock the thread pool mutex when running with more than 1 thread
There's no reason to lock anything if only the current thread is ever
going to do any work.
2017-02-24 10:02:28 +02:00
Sebastian Dröge
49ac382b47 video-converter: Implement multi-threaded scaling/conversion
This adds a property to select the maximum number of threads to use for
conversion and scaling. During processing, each plane is split into
an equal number of consecutive lines that are then processed by each
thread.

During tests, this gave up to 1.8x speedup with 2 threads and up to 3.2x
speedup with 4 threads when converting e.g. 1080p to 4k in v210.

https://bugzilla.gnome.org/show_bug.cgi?id=778974
2017-02-23 21:55:29 +02:00
Sebastian Dröge
d0c1b34288 video-converter: Fix crashes in fast-paths when converting interlaced formats with different vertical subsampling
E.g. the following pipelines fail because chroma values after the last
line are read (note: 486 % 4 == 2):

gst-launch-1.0 videotestsrc ! "video/x-raw,interlace-mode=interleaved,width=720,height=486,format=UYVY" ! videoconvert ! "video/x-raw,format=I420" ! fakesink
gst-launch-1.0 videotestsrc ! "video/x-raw,interlace-mode=interleaved,width=720,height=486,format=I420" ! videoconvert ! "video/x-raw,format=UYVY" ! fakesink
gst-launch-1.0 videotestsrc ! "video/x-raw,interlace-mode=interleaved,width=720,height=486,format=I420" ! videoconvert ! "video/x-raw,format=AYUV" ! fakesink
2017-01-11 18:41:14 +02:00
Nicolas Dufresne
7a40442ad5 video: Add VYUY pixel format
This format is sometimes the output of JPEG decoders. It is the same as
YUY2 and UYVY but with a different component order.

https://bugzilla.gnome.org/show_bug.cgi?id=767450
2016-11-01 19:55:20 +02:00
Wim Taymans
9144a787df video-converter: fix compilation on big-endian 2016-07-07 17:29:34 +02:00
Wim Taymans
5e752f4eda video-converter: fix interlaced scaling some more
Fix problem with the line cache where it would forget the first line in
the cache in some cases.
Keep as much backlog as we have taps. This generally works better and we
could do even better by calculating the overlap in all taps.
Allocated enough lines for the line cache.
Use only half the number of taps for the interlaced lines because we
only have half the number of lines.
The pixel shift should be relative to the new output pixel size so scale
it.
Fixes: https://bugzilla.gnome.org/show_bug.cgi?id=767921
2016-06-22 16:21:13 +02:00
Joan Pau Beltran
abb88801e7 video: add IYU2 format
This existed in 0.10 and is needed by dc1394src.

IYU2 format is a YUV fully-sampled packed format similar to v308
but with different component order (U-Y-V instead of Y-U-V).

http://www.fourcc.org/yuv.php#IYU2

https://bugzilla.gnome.org/show_bug.cgi?id=763026#c5
2016-06-01 12:07:05 +01:00
Wim Taymans
a12f51c3de video-converter: add more fastpaths for I420 -> RGB
Use the I420->BGRA and a new I420->ARGB to speed up any I420 to RGB
operation.
2016-04-22 15:30:19 +02:00
Scott D Phillips
079ceb894c video: add P010 format support
P010 is a YUV420 format with an interleaved U-V plane and 2-bytes per
component with the the color value stored in the 10 most significant
bits.

https://bugzilla.gnome.org/show_bug.cgi?id=761607
---
Changes since v2:
- Set bits=16 in DPTH10_10_10_HI
Changes since v1:
- Fixed x-offset calculation in uv.
- Added 6-bit shifts to FormatInfo.
2016-03-29 11:16:42 +03:00
Göran Jönsson
babcf4d30c video-converter: add direct UYVY to GRAY8 conversion function
https://bugzilla.gnome.org/show_bug.cgi?id=761851
2016-02-26 23:56:51 +00:00
Wim Taymans
83fe1c7705 video-converter: ignore matrix for RGB formats
For RGB formats, the matrix in the colorimetry (conversion from YUV to
RGB) is irrelevant and we should ignore it and assume the identity
transform for everything we do.

Fixes https://bugzilla.gnome.org/show_bug.cgi?id=759624
2016-01-20 10:02:20 +01:00