Commit graph

192 commits

Author SHA1 Message Date
Thibault Saunier
932f0a6175 validate: flow: Fix logging upstream events
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:46 +00:00
Thibault Saunier
56000b91b7 validate: Add a way to force monitoring all pipelines in a .validatetest file
See documentation for more details

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
302797a965 validate: Add a deep-property-path parameter to the wait signal
Allowing wait actions to wait on any property of any element in the pipeline,
even for elements that might be added later in the pipeline. This also
works for pads which can be pretty useful

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
2c88bbf07f validate: Allow overriding features rank early in testfiles meta
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
f5b0f91d33 validate: launcher: Remove log files for passing tests by default
Adding an option to keep them no matter what.
Log files are often pretty large and keeping them around can be annoying,
usually people won't look at logs files for passing tests, and we do not
even print them out.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
0cdd22f176 validate: Remove some regex related python 3.12 warnings
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
dfd5357214 validate: scenario: Add a way to make the select-streams run several times
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Thibault Saunier
19060b2ce0 validate: Add a way to retrieve HTTP server port in .validatetest files
By setting a `$(http_server_port)s` variable in a dedicated config file
and making sure that file can always be imported in `.validatetest`
files.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700>
2024-11-07 20:52:45 +00:00
Edward Hervey
36b32de831 validate: Fix leaks in ssim components
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7782>
2024-10-31 01:39:19 +00:00
Tim-Philipp Müller
b3245eb7bd validate: skip curlhttpsrc test_get_range when running in valgrind
Seems to reliably fail (timeout) when running in valgrind
on the post-F40 CI.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7753>
2024-10-26 02:10:37 +01:00
Tim-Philipp Müller
bf5b0283a2 validate: add srtp test_roc to valgrind exclude list
Consistently fails on CI after F40 upgrade.

See #3939

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7753>
2024-10-26 02:09:44 +01:00
Tim-Philipp Müller
f64cf9998f validate: launcher: add --fair-sched=try to valgrind args
Enables fairer scheduling of threads in valgrind, possibly
at cost of performance though.

Hopefully helps with valgrind core/base jobs deadlocking or
timing out when runners are not under load.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7753>
2024-10-26 00:23:01 +01:00
Edward Hervey
5ac43063b6 tests: Blacklist more netsim test
They are know to be racy/failing

See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/792

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7736>
2024-10-25 10:44:24 +00:00
Corentin Damman
339e6e0069 validate: use real_main instead of main in run_test_from_file
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7704>
2024-10-23 13:43:03 +00:00
Thibault Saunier
14a9138a88 validate: flow: Display diffs properly
When called after writing the report we can't see them

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5035>
2024-10-19 10:30:05 +00:00
Sebastian Dröge
6233eb0ff3 common: Stop using GQuark-based GstStructure field name API
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7432>
2024-09-26 19:21:29 +03:00
Thibault Saunier
9f898f839c validate: pad-monitor: Fix remaining pad functions data handling
That case was missed in a571753089

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7477>
2024-09-10 15:50:52 +00:00
Thibault Saunier
0d24821167 validate: launcher: Add support for lldb
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7452>
2024-09-10 11:23:02 +00:00
Thibault Saunier
a571753089 validate: pad-monitor: Fix pad function data properly
Until now we were overriding pad functions forgetting about the function
data (that are set using the _full variant of the functions setters), meaning
that the data was lost and any user of that feature would get empty data when
the wrapped function were called.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7466>
2024-09-08 12:02:25 +00:00
Thibault Saunier
9f410a8714 validate: scenario: Handle the fact that structs field names don't have a static lifetime anymore
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7453>
2024-09-05 18:58:24 +00:00
Sebastian Dröge
ec11c337c3 validate: Properly use errors = 'replace' for handling invalid utf8 characters
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7453>
2024-09-05 18:58:24 +00:00
Thibault Saunier
87c69e5174 ci: Fail tests if we forget to checkout expectation files
And add missing expectation files

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7400>
2024-08-21 17:53:38 +00:00
Thibault Saunier
8fdd59f9d5 validate: flow: Allow logging upstream events
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7382>
2024-08-21 13:55:15 +00:00
Sebastian Dröge
417c5e19b7 validate: Copy action structure before retrieving strings from it
The returned strings are only valid for as long as the structure is valid.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7333>
2024-08-09 10:26:30 +00:00
Sebastian Dröge
604cc0901c validate: Fix copying of action name
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7333>
2024-08-09 10:26:30 +00:00
Edward Hervey
c623f2db21 validate: reporter: Don't pollute logs
There was a stray g_printerr introduced by 2a4b9c8dc1

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7275>
2024-07-30 11:39:02 +02:00
Alicia Boya García
506af5b8e4 gst-validate: Don't treat FALSE when sending flushes as an error
gst_element_send_event(FLUSH_START / FLUSH_STOP) returns FALSE in cases
where any of the most downstream elements have unlinked pads, even if
the pipeline is successfully flushed.

Currently this is considered expected behavior in GStreamer. This patch
updates gst-validate to treat it as such and therefore not fail the test
for a "failing" flush.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7064>
2024-07-02 19:06:02 +00:00
Jordan Petridis
60454336c8 validate: Remove G_REGEX_OPTIMIZE usage
It's not needed and causes issues with valgrind (which doesn't support jit)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7112>
2024-06-28 14:13:31 +00:00
Thibault Saunier
c8dfb7c8f0 validate: tool: Use gst_macos_main
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7056>
2024-06-18 22:31:40 +00:00
Loïc Le Page
cbc58c0483 gst-validate: add formatting for GstAudioMeta
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6461>
2024-04-16 15:33:42 +00:00
Thibault Saunier
2a4b9c8dc1 validate: reporter: Use an MT safe weak ref to reporter for GLog handling
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6412>
2024-03-20 16:45:34 +00:00
Jonas K Danielsson
f0b8a33f10 check: Add valgrind leak exception for srt test
The srt unittest test_src_listener_sink_call will sometimes fail under
valgrind with the following splat:

 Memcheck, a memory error detector
 Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
 Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
 Parent PID: 14579

 HEAP SUMMARY:
     in use at exit: 799,848 bytes in 2,182 blocks
   total heap usage: 64,090 allocs, 61,908 frees, 37,891,032 bytes allocated

 120 bytes in 1 blocks are definitely lost in loss record 1,563 of 1,681
    at 0x4842FF5: operator new(unsigned long) (vg_replace_malloc.c:422)
    by 0x6031E29: srt::sync::SetThreadLocalError(CUDTException const&) (sync_posix.cpp:461)
    by 0x5FCD77E: CUDT::epoll_wait(int, std::set<int, std::less<int>,
    std::allocator<int> >*, std::set<int, std::less<int>,
    std::allocator<int> >*, long, std::set<int, std::less<int>,
    std::allocator<int> >*, std::set<int, std::less<int>, std::allocator<int> >*) [clone .cold] (api.cpp:3796)
    by 0x5FE2F79: UDT::epoll_wait2(int, int*, int*, int*, int*, long, int*, int*, int*, int*) (api.cpp:4277)
    by 0x5F0C626: gst_srt_object_read (gstsrtobject.c:1569)
    by 0x5F0F978: gst_srt_src_fill (gstsrtsrc.c:180)
    by 0x5F5A2A1: gst_base_src_default_create (gstbasesrc.c:1620)
    by 0x5F5C9AE: gst_base_src_get_range (gstbasesrc.c:2630)
    by 0x5F5EF5A: gst_base_src_loop (gstbasesrc.c:2959)
    by 0x4918B13: gst_task_func (gsttask.c:399)
    by 0x4A60B33: g_thread_pool_thread_proxy.lto_priv.0 (gthreadpool.c:354)
    by 0x4A5DC41: g_thread_proxy (gthread.c:826)
    by 0x4F532A4: start_thread (pthread_create.c:481)
    by 0x4C71322: clone (clone.S:95)

An issue has been started against libsrt here:

https://github.com/Haivision/srt/issues/2867

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6098>
2024-02-12 13:24:21 +00:00
Xavier Claessens
561f8b71ee structure: Allow STRICT flag only in _serialize_full()
The STRICT flag makes _serialize() nullable which is an API break for
bindings. Forbid it and add _serialize_full() that accepts it and is
properly annotated.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5829>
2024-02-09 15:45:33 +00:00
Thibault Saunier
1891fff7a9 validate: flow: Plug leak about ignored fields
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/899>
2024-01-31 11:00:24 +00:00
Tim-Philipp Müller
bd77d81267 validate: filter check.gst-editing-services.complex_effect_bin_desc
Add this check to the known issues list, fails way too often.

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2973

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6003>
2024-01-29 00:08:55 +00:00
Robert Mader
46259c634b check: Add valgrind leak exception for glcolorconvert
The following commit will add a test triggering a leak in GLX / the
driver.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5948>
2024-01-25 20:39:37 +00:00
ekwange
100497df20 validate: check: Fix known-not-leaky to work
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3940>
2024-01-15 19:56:06 +00:00
ekwange
264a57dabe validate: launcher: Fix typo
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3940>
2024-01-15 19:56:06 +00:00
ekwange
248044329c validate: launcher: Add debug message and fix typo
Add messages to clearly see why test was excluded.
And fix typo prerrol

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3940>
2024-01-15 19:56:06 +00:00
Thibault Saunier
7fe786135f validate: scenario: Do not use a NULL scenario description structure
The scenario meta is optional and we have to take that into account.

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3199
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5892>
2024-01-06 09:47:16 -03:00
Thibault Saunier
04ae39f25b validate: scenario: Add a 'forward-eos' parameter to forward-appsink-to-appsrc
There might be cases where the user doesn't want to forward it

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5803>
2023-12-27 02:14:19 +00:00
Thibault Saunier
80e44ac344 validate: scenario: Handle the case where execute_next_action is called with an action that is OK
It is now a valid case when running subaction in foreign scenarios

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5803>
2023-12-27 02:14:19 +00:00
Thibault Saunier
93f1ffe715 validate: doc: Enhance the way we render action type parameters
And update gst-validate-action-types.md

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5803>
2023-12-27 02:14:19 +00:00
Thibault Saunier
f1e34de800 validate: scenario: Do not print position on sub pipelines
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5803>
2023-12-27 02:14:19 +00:00
Thibault Saunier
a4a97c7568 validate: doc: Enhance documentation about testfile and meta fields
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5803>
2023-12-27 02:14:19 +00:00
Thibault Saunier
a5a65b74ff validate: scenario: Ensure all pipelines with scenarios are stopped when exiting runner
Otherwise we end up with "dangling" sub pipelines on SIGINT

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5803>
2023-12-27 02:14:18 +00:00
Thibault Saunier
6abc36ad34 validate: scenario: Cleanup the 'switch-track' action implementation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5803>
2023-12-27 02:14:18 +00:00
Thibault Saunier
fbf478b3f5 validate: scenario: Add a way to set pipeline base-time, start-time and force using the system clock
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5803>
2023-12-27 02:14:18 +00:00
Thibault Saunier
aad06b091e validate: scenario: Add an action type to forward stream from an appsink to an appsrc
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5803>
2023-12-27 02:14:18 +00:00
Thibault Saunier
fd4cd3d85f validate:scenario: Add a 'create-sub-pipeline' action type
And allow running parallel pipelines, scenarios, and pass data from
one to the other using appsrc/appsink

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5803>
2023-12-27 02:14:18 +00:00