Commit graph

2763 commits

Author SHA1 Message Date
Marek Vasut
552a171671 vp9decoder: Simplify gst_v4l2_codecs_vp9_dec_fill_refs()
In case reference_frames is NULL, return outright. Remove the
duplicate check from subsequent conditionals. No functional change.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
2023-12-20 18:47:39 +00:00
Marek Vasut
4560cdff5c h265decoder: Align wraparound fix
Instead of casting GST_CODEC_PICTURE_FRAME_NUMBER (ref_pic) to u64,
use 1000ULL which is also u64 . This only aligns the behavior here
with '*decoder: Fix multiplication wraparound' commits.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
2023-12-20 18:47:39 +00:00
Marek Vasut
7725aa7d77 h264decoder: Align wraparound fix
Instead of casting GST_CODEC_PICTURE_FRAME_NUMBER (ref_pic) to u64,
use 1000ULL which is also u64 . This only aligns the behavior here
with '*decoder: Fix multiplication wraparound' commits.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
2023-12-20 18:47:39 +00:00
Marek Vasut
3cbf09d0c9 mpeg2decoder: Fix multiplication wraparound
The GstMpeg2Picture system_frame_number is guint32, constant 1000 is guint32,
GstV4l2CodecMpeg2Dec *_ref_ts multiplication result is u64 .

```
u64 result = (u32)((u32)system_frame_number * (u32)1000);
```
behaves the same as
```
u64 result = (u32)(((u32)system_frame_number * (u32)1000) & 0xffffffff);
```

so in case `system_frame_number > 4294967295 / 1000`, the `result` will
wrap around. Since the `result` is really used as a cookie used to look
up V4L2 buffers related to the currently decoded frame, this wraparound
leads to visible corruption during MPEG2 decoding. At 30 FPS this occurs
after cca. 40 hours of playback .

Fix this by changing the 1000 from u32 to u64, i.e.:
```
u64 result = (u64)((u32)system_frame_number * (u64)1000ULL);
```
this way, the wraparound is prevented and the correct cookie is used.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
2023-12-20 18:47:39 +00:00
Marek Vasut
50fb6f8c02 av1decoder: Fix multiplication wraparound
The GstAV1Picture system_frame_number is guint32, constant 1000 is guint32,
GstV4l2CodecAV1Dec v4l2_av1_frame.*_frame_ts multiplication result is u64 .

```
u64 result = (u32)((u32)system_frame_number * (u32)1000);
```
behaves the same as
```
u64 result = (u32)(((u32)system_frame_number * (u32)1000) & 0xffffffff);
```

so in case `system_frame_number > 4294967295 / 1000`, the `result` will
wrap around. Since the `result` is really used as a cookie used to look
up V4L2 buffers related to the currently decoded frame, this wraparound
leads to visible corruption during AV1 decoding. At 30 FPS this occurs
after cca. 40 hours of playback .

Fix this by changing the 1000 from u32 to u64, i.e.:
```
u64 result = (u64)((u32)system_frame_number * (u64)1000ULL);
```
this way, the wraparound is prevented and the correct cookie is used.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
2023-12-20 18:47:39 +00:00
Marek Vasut
6f74818a07 vp9decoder: Fix multiplication wraparound
The GstVp9Picture system_frame_number is guint32, constant 1000 is guint32,
GstV4l2CodecVp9Dec v4l2_vp9_frame.*_frame_ts multiplication result is u64 .

```
u64 result = (u32)((u32)system_frame_number * (u32)1000);
```
behaves the same as
```
u64 result = (u32)(((u32)system_frame_number * (u32)1000) & 0xffffffff);
```

so in case `system_frame_number > 4294967295 / 1000`, the `result` will
wrap around. Since the `result` is really used as a cookie used to look
up V4L2 buffers related to the currently decoded frame, this wraparound
leads to visible corruption during VP9 decoding. At 30 FPS this occurs
after cca. 40 hours of playback .

Fix this by changing the 1000 from u32 to u64, i.e.:
```
u64 result = (u64)((u32)system_frame_number * (u64)1000ULL);
```
this way, the wraparound is prevented and the correct cookie is used.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
2023-12-20 18:47:39 +00:00
Marek Vasut
15c9a6caa9 vp8decoder: Fix multiplication wraparound
The GstVp8Picture system_frame_number is guint32, constant 1000 is guint32,
GstV4l2CodecVp8Dec v4l2_vp8_frame.*_frame_ts multiplication result is u64 .

```
u64 result = (u32)((u32)system_frame_number * (u32)1000);
```
behaves the same as
```
u64 result = (u32)(((u32)system_frame_number * (u32)1000) & 0xffffffff);
```

so in case `system_frame_number > 4294967295 / 1000`, the `result` will
wrap around. Since the `result` is really used as a cookie used to look
up V4L2 buffers related to the currently decoded frame, this wraparound
leads to visible corruption during VP8 decoding. At 30 FPS this occurs
after cca. 40 hours of playback .

Fix this by changing the 1000 from u32 to u64, i.e.:
```
u64 result = (u64)((u32)system_frame_number * (u64)1000ULL);
```
this way, the wraparound is prevented and the correct cookie is used.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
2023-12-20 18:47:39 +00:00
Mengkejiergeli Ba
141cd38715 h264parse: Remove dead code
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5833>
2023-12-20 13:03:25 +00:00
Sebastian Dröge
2e86fb691a video-format: Fix format order once again
RGBA should be before RBGA. Both the Python script and the gstreamer-rs
tests agree on that, but somehow this is not caught by the CI.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5837>
2023-12-20 05:33:43 +00:00
Olivier Crête
f5d5f2cf1a meta: Add API to register metas in two steps
And also remove the specific registration APIs for
serializable meta.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5830>
2023-12-19 22:41:31 +00:00
Olivier Crête
2a9c4e3270 meta: Move the clear operation to its own vfunc
Some transforms always assumed that the transformation was some kind
of copy. So adding a "clear" operation didn't work out in practice.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5830>
2023-12-19 22:41:31 +00:00
Philippe Normand
61d3f5c8e0 play: Include pipeline dump in error details structure
This can be useful for debugging purposes. It can't be done on application side
because the on_error callback tears down the pipeline.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5828>
2023-12-18 20:13:38 +00:00
Philippe Normand
c6c567e3e4 play: Fix error details parsing
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5828>
2023-12-18 20:13:38 +00:00
Robert Mader
c481fe7059 camerabin: Correctly relink viewfinderbin_queue
This reverts a part of de92a6c7f2. Unlike `image_filter` and
`video_filter`, `viewfinder_filter` does not get linked to `src` but
`viewfinderbin_queue`. Thus the fix in the mentioned commit does not
apply for it and should be reverted.

This was not spotted earlier as only the other filters are used in
the project that uncovered the issue.

Fixes: de92a6c7f2 ("camerabin: Fix source updates with user filters")
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5689>
2023-12-18 17:07:31 +00:00
Seungha Yang
890d59e97e av1parser: Fix array sizes in scalability structure
Since the AV1 specification is not explicitly mentioning about
the array size bounds, array sizes in scalability structure
should be defined as possible maximum sizes that can have.

Also, this commit removes GST_AV1_MAX_SPATIAL_LAYERS define from
public header which is API break but the define is misleading
and this patch is introducing ABI break already

ZDI-CAN-22300

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5823>
2023-12-18 10:58:24 +00:00
Xavier Claessens
533f814fd9 h264parse: test - AU align with SEI between frame slices
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5741>
2023-12-17 21:42:47 +00:00
Daniel Morin
49f200cb54 h264parse: Improved AU boundary detection
- AU boundary detection reviewed to follow more closely H.264 spec. and more
  specifically clauses 7.4.1.2.3 and 7.4.1.2.4.
- The gist of the changes is a look-a-head in then next AU required identify the
  last vcl-nal of current AU and firt vcl-nal of next AU (according to
  7.4.1.2.4) followed by the identification of the first nal of next AU
  (according to 7.4.1.2.3).
- A backlog of all nals of current AU and next AU up to the point where current
  AU can identified completed is kept.
- In NAL alignement mode vcl-nal are sent immediatly but the history is kept to
  allow AU boundary detection. Non-vcl-nal can be delayed up to the reception of
  the next vcl-nal to allow a correct AUD insertion.
- Based on this improved AU boudary detection we can avoid erronous AUD
  insertion, like the one highlighted by test
  test_parse_sliced_with_prefix_and_sei_nal_au.
- Add support for MVC AU boundary detection. (H.7.4.1.2.4)
- Explicitly report SVC not supported. We don't have the SVC NAL parsing
  required to identify boundary. (missing dependency_id and quality_id fields
  from SVC, see G.7.4.1.2.4)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5741>
2023-12-17 21:42:47 +00:00
Xavier Claessens
f076a9bf2f unixfd: Serialize buffer metas
Serialize every GstMeta that supports serialization into the NEW_BUFFER
payload. This is especially important for GstVideoMeta in the case of
multiplanar buffers, or if stride!=width.

Sponsored-by: Netflix Inc.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5355>
2023-12-17 16:13:26 +00:00
He Junyan
35390de50d vacompositor: consider the DMA kind input for sink pad
Co-authored-by: Víctor Jáquez <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5160>
2023-12-16 10:56:09 +00:00
He Junyan
0bbaa00c3f vacompositor: Override the update_caps() of video aggregator class
The input of the vacompositor may be DMA buffers. And in this case, the input
caps has the format=DMA_DRM, which can not be recognized by base video
aggregator class' find_best_format() function. So we need to override the
update_caps() virtual function.

Also we consider the DMA kind caps in negotiated_src_caps() for output.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5160>
2023-12-16 10:56:09 +00:00
He Junyan
e68724c8e0 vacompositor: add helper function to get formats from caps
Co-authored-by: Víctor Jáquez <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5160>
2023-12-16 10:56:09 +00:00
He Junyan
c76999fe7b vacompositor: add helper function to choose format
The function is based on the most supported formats by Intel/Mesa VA drivers.

Co-authored-by: Víctor Jáquez <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5160>
2023-12-16 10:56:09 +00:00
He Junyan
eb4d2dd204 vacompositor: record the input caps for each input pad
The caps of each input sink pad wil decide the final output format and
caps of the src pad.

Co-authored-by: Víctor Jáquez <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5160>
2023-12-16 10:56:09 +00:00
Seungha Yang
800a83b435 d3d12decoder: Implement threaded decoding
To achieve maximum throughput, waiting on command commit thread
is not ideal. And render-delay will introduce unwanted latency.
Best is to split thread and wait finished decoding job in a dedicated
output thread

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5812>
2023-12-15 22:56:33 +09:00
Seungha Yang
7471db9a30 d3d12decoder: Disable d3d11 interop
It does not seem to work with some AMD iGPU

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5812>
2023-12-15 20:40:21 +09:00
Seungha Yang
7afa914054 d3d12fence: Reset fence after waiting done
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5812>
2023-12-15 19:48:40 +09:00
Seungha Yang
a24a155279 d3d12: Enable debug layer
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5812>
2023-12-15 19:48:40 +09:00
Seungha Yang
7443cc00b5 d3d12: Requires ID3D12Device4 interface
ID3D12Device4::CreateCommandList1() method is required

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5812>
2023-12-15 19:48:39 +09:00
Seungha Yang
818c95e8c3 d3d12: Update allocation params signalling
Sync up with d3d11 implementation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5812>
2023-12-15 19:48:36 +09:00
Víctor Manuel Jáquez Leal
8210784dab vabasedec: clean up decide_allocation() vmethod
When creating a new VA pool set config size to zero because it's not used.

Also, given the potential different sizes from software buffer pools and VA
buffer pools, this patch handle that potential different values.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5805>
2023-12-14 12:50:32 +00:00
Víctor Manuel Jáquez Leal
4fab6bb45a vabasetransform: clean up decide_allocation() vmethod
When creating a new VA pool set config size to zero because it's not used.

Also, given the potential different sizes from software buffer pools and VA
buffer pools, this patch handle that potential different values.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5805>
2023-12-14 12:50:32 +00:00
Víctor Manuel Jáquez Leal
33e49023ff vacompositor: clean up decide_allocation() vmethod
When creating a new VA pool set config size to zero because it's not used.

Also, given the potential different sizes from software buffer pools and VA
buffer pools, this patch handle that potential different values.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5805>
2023-12-14 12:50:32 +00:00
Víctor Manuel Jáquez Leal
1c7ce208bd vapluginsutils: add helper gst_va_create_other_pool()
This helper function creates a software-based buffer pool, where if size is
zero, its value is retrieved from the canonical size.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5805>
2023-12-14 12:50:32 +00:00
Víctor Manuel Jáquez Leal
4a22cc8fb3 va: no need to provide a buffer size for VA pool
VA drivers allocate surfaces given their properties, so there's no need to
provide a buffer size to the VA pool.

Though, the buffer size is provided by the driver, or the canonical size
is used for single planed surfaces.

This patch removes the need to provide a size for the function
gst_va_pool_new_with_config() and adds a helper method to retrieve the surface
size, gst_va_pool_get_buffer_size(). Also change the callers accordingly.

Changes for custom VA pool creation will be addressed in the following commits.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5805>
2023-12-14 12:50:32 +00:00
Seungha Yang
fab1c5f953 d3d12: Use d3dx12.h helper library
... and remove manually implemented helper methods

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5807>
2023-12-14 18:08:00 +09:00
Seungha Yang
38c04a5f34 d3d12: Use DirectX-Headers and drop MinGW build support
There are some conflicts between MinGW toolchan and DirectX-Headers.
And due to ABI differences, more fixes are required.
We will support only MSVC build for now

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5807>
2023-12-14 18:08:00 +09:00
Mengkejiergeli Ba
41f13f4373 msdk: Remove dead code
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5796>
2023-12-14 00:47:28 +00:00
Víctor Manuel Jáquez Leal
f413714b37 vaallocator: use gst_va_display_check_version()
Instead of the local method to parse the vendor string.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5800>
2023-12-13 09:12:28 +00:00
Víctor Manuel Jáquez Leal
8574bb8914 vadisplay: add gst_va_display_check_version()
This function compares the driver version with the user provided one to check if
driver's is equal or bigger.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5800>
2023-12-13 09:12:28 +00:00
Víctor Manuel Jáquez Leal
94e6a6e3de vaallocator: clean up use derived feature
Remove allocator member variable for use derived feature which doesn't need to be kept,
it's just for configuration purposes.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5799>
2023-12-12 18:39:49 +01:00
He Junyan
5ae7bbf5fb va: h265dec: Add support for more -still and -intra profiles
They are actually compitable with the main and main-10 profiles.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5797>
2023-12-12 21:30:16 +08:00
He Junyan
5345652896 va: Add decoder compatible caps string field into profiles' map
Some profiles such as main or main-10 in HEVC can support more
compatible profiles such as main-still-picture or main-10-intra.
We should add them into decoder's sink caps.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5797>
2023-12-12 21:30:08 +08:00
He Junyan
354aceb9cd va: Add the entrypoint parameter to gst_va_profile_caps()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5797>
2023-12-12 20:41:54 +08:00
Mengkejiergeli Ba
484a636841 msdkvpp: Add fix_format for src caps
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5208>
2023-12-12 14:56:12 +08:00
Mengkejiergeli Ba
5536f742e9 msdk: Add a help func to fix the map
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5208>
2023-12-12 14:56:12 +08:00
Mengkejiergeli Ba
8081a665f8 msdk: Remove func to export dmabuf to va surface
Since we use va allocator and va dma allocator to create surface, the
surface is already exported in the case of dma allocator. So remove
redundant export function in msdk allocator.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5208>
2023-12-12 14:56:12 +08:00
Yinhang Liu
61282ebd34 msdkenc: Add modifier support
Co-authored-by: Mengkejiergeli Ba <mengkejiergeli.ba@intel.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5208>
2023-12-12 14:56:12 +08:00
Yinhang Liu
9ccdfae545 msdkvpp: Add modifier support
Co-authored-by: Mengkejiergeli Ba <mengkejiergeli.ba@intel.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5208>
2023-12-12 14:56:12 +08:00
Yinhang Liu
63c0fdd29a msdkdec: Add modifier support
Co-authored-by: Mengkejiergeli Ba <mengkejiergeli.ba@intel.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5208>
2023-12-12 14:56:12 +08:00
Mengkejiergeli Ba
e65e954e2b msdk: Config pool with max/min number of buffers
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5208>
2023-12-12 14:56:12 +08:00