Thibault Saunier
f5b0f91d33
validate: launcher: Remove log files for passing tests by default
...
Adding an option to keep them no matter what.
Log files are often pretty large and keeping them around can be annoying,
usually people won't look at logs files for passing tests, and we do not
even print them out.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700 >
2024-11-07 20:52:45 +00:00
Thibault Saunier
19060b2ce0
validate: Add a way to retrieve HTTP server port in .validatetest files
...
By setting a `$(http_server_port)s` variable in a dedicated config file
and making sure that file can always be imported in `.validatetest`
files.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7700 >
2024-11-07 20:52:45 +00:00
Tim-Philipp Müller
b3245eb7bd
validate: skip curlhttpsrc test_get_range when running in valgrind
...
Seems to reliably fail (timeout) when running in valgrind
on the post-F40 CI.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7753 >
2024-10-26 02:10:37 +01:00
Tim-Philipp Müller
bf5b0283a2
validate: add srtp test_roc to valgrind exclude list
...
Consistently fails on CI after F40 upgrade.
See #3939
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7753 >
2024-10-26 02:09:44 +01:00
Tim-Philipp Müller
f64cf9998f
validate: launcher: add --fair-sched=try to valgrind args
...
Enables fairer scheduling of threads in valgrind, possibly
at cost of performance though.
Hopefully helps with valgrind core/base jobs deadlocking or
timing out when runners are not under load.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7753 >
2024-10-26 00:23:01 +01:00
Edward Hervey
5ac43063b6
tests: Blacklist more netsim test
...
They are know to be racy/failing
See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/792
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7736 >
2024-10-25 10:44:24 +00:00
Thibault Saunier
0d24821167
validate: launcher: Add support for lldb
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7452 >
2024-09-10 11:23:02 +00:00
Sebastian Dröge
ec11c337c3
validate: Properly use errors = 'replace'
for handling invalid utf8 characters
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7453 >
2024-09-05 18:58:24 +00:00
Thibault Saunier
87c69e5174
ci: Fail tests if we forget to checkout expectation files
...
And add missing expectation files
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7400 >
2024-08-21 17:53:38 +00:00
Jonas K Danielsson
f0b8a33f10
check: Add valgrind leak exception for srt test
...
The srt unittest test_src_listener_sink_call will sometimes fail under
valgrind with the following splat:
Memcheck, a memory error detector
Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
Parent PID: 14579
HEAP SUMMARY:
in use at exit: 799,848 bytes in 2,182 blocks
total heap usage: 64,090 allocs, 61,908 frees, 37,891,032 bytes allocated
120 bytes in 1 blocks are definitely lost in loss record 1,563 of 1,681
at 0x4842FF5: operator new(unsigned long) (vg_replace_malloc.c:422)
by 0x6031E29: srt::sync::SetThreadLocalError(CUDTException const&) (sync_posix.cpp:461)
by 0x5FCD77E: CUDT::epoll_wait(int, std::set<int, std::less<int>,
std::allocator<int> >*, std::set<int, std::less<int>,
std::allocator<int> >*, long, std::set<int, std::less<int>,
std::allocator<int> >*, std::set<int, std::less<int>, std::allocator<int> >*) [clone .cold] (api.cpp:3796)
by 0x5FE2F79: UDT::epoll_wait2(int, int*, int*, int*, int*, long, int*, int*, int*, int*) (api.cpp:4277)
by 0x5F0C626: gst_srt_object_read (gstsrtobject.c:1569)
by 0x5F0F978: gst_srt_src_fill (gstsrtsrc.c:180)
by 0x5F5A2A1: gst_base_src_default_create (gstbasesrc.c:1620)
by 0x5F5C9AE: gst_base_src_get_range (gstbasesrc.c:2630)
by 0x5F5EF5A: gst_base_src_loop (gstbasesrc.c:2959)
by 0x4918B13: gst_task_func (gsttask.c:399)
by 0x4A60B33: g_thread_pool_thread_proxy.lto_priv.0 (gthreadpool.c:354)
by 0x4A5DC41: g_thread_proxy (gthread.c:826)
by 0x4F532A4: start_thread (pthread_create.c:481)
by 0x4C71322: clone (clone.S:95)
An issue has been started against libsrt here:
https://github.com/Haivision/srt/issues/2867
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6098 >
2024-02-12 13:24:21 +00:00
Tim-Philipp Müller
bd77d81267
validate: filter check.gst-editing-services.complex_effect_bin_desc
...
Add this check to the known issues list, fails way too often.
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2973
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6003 >
2024-01-29 00:08:55 +00:00
Robert Mader
46259c634b
check: Add valgrind leak exception for glcolorconvert
...
The following commit will add a test triggering a leak in GLX / the
driver.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5948 >
2024-01-25 20:39:37 +00:00
ekwange
100497df20
validate: check: Fix known-not-leaky to work
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3940 >
2024-01-15 19:56:06 +00:00
ekwange
264a57dabe
validate: launcher: Fix typo
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3940 >
2024-01-15 19:56:06 +00:00
ekwange
248044329c
validate: launcher: Add debug message and fix typo
...
Add messages to clearly see why test was excluded.
And fix typo prerrol
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3940 >
2024-01-15 19:56:06 +00:00
ekwang.lee
6b0f7729e1
validate:launcher: Using surrogateescape error handler
...
Use surrogateescape error handler to convert undecodable bytes without error.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5008 >
2023-11-28 13:00:15 +00:00
Thibault Saunier
a56abe2c9d
validate: launcher: Lower some 'coredumpctl' info messages
...
Those are verbose and are not really useful in our context
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5666 >
2023-11-15 19:07:36 +00:00
Thibault Saunier
ad68ce7f64
validate: launcher: Make the output markdown file more readable
...
Without ansi color codes and marking logs as such
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5666 >
2023-11-15 19:07:36 +00:00
Thibault Saunier
c530c94239
validate: launcher: Keep gst debug logs in main log fails in debug mode
...
It makes reading the logs much simpler
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5666 >
2023-11-15 19:07:36 +00:00
Thibault Saunier
89434a077c
validate: launcher: Add a way to specify the source to use for precise seeking tests
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5666 >
2023-11-15 19:07:36 +00:00
Jordan Petridis
e436f87ea2
launcher: Fix ambigious python strings
...
```
gst-devtools/validate/launcher/baseclasses.py:2399: SyntaxWarning: invalid escape sequence '\.'
if re.findall("%s\..*\.%s$" % (re.escape(mfile_bname), self.FILE_EXTENSION), f):
gst-devtools/validate/launcher/apps/gstvalidate.py:1354: SyntaxWarning: invalid escape sequence '\.'
("file\.transcode.*mxf",
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5541 >
2023-11-11 12:47:12 +00:00
Jordan Yelloz
b1f3722530
gst-validate: Fixed compatibility with Python 3.12
...
config.readfp() was removed in python 3.12 and config.read_file() does the same
thing and has been available since Python 3.2
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5624 >
2023-11-08 15:50:52 -07:00
Detlev Casanova
78e40a018f
dev-tools: Add glib valgrind suppressions
...
The local glib subproject doesn't exist so the glib/glib.supp file
cannot be included.
As it is needed for the do_lookup_by_name() function call, let's add the
system wide suppression file so that its version matches the installed glib
version.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5434 >
2023-10-31 12:06:30 -04:00
Jordan Petridis
18578643ca
check: Move CI specific ignored tests into their own list
...
Instead of ignoring the tests always, check if a CI env var is
present and only ignore the tests then.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1060 >
2023-10-22 05:23:30 +03:00
Jordan Petridis
88e6dd0555
ci: switch the Fedora base image to f34
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1060 >
2023-10-22 05:23:28 +03:00
Alicia Boya García
135f859629
validate: Fix error on GstTest.copy()
...
Fixes regression introduced in ba61160d6c
,
where running check tests with gst-validate-launcher -f would trigger
this exception:
AttributeError: 'GstCheckTest' object has no attribute 'reports'.
Did you mean: 'reporter'?
The member `reports` is meant to be just part of GstValidateTest, but
not other subclasses, even though a usage is still found in the base
class GstTest in the method test_end().
This patch introduces an override of the methods copy() and test_end()
in GstValidateTest so that `reports` is copied and cleared respectively,
but only for validate tests.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5281 >
2023-09-05 14:16:57 +00:00
Alicia Boya García
ba61160d6c
validate-launcher: Fix race condition on repeated tests
...
Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2900
The `reports` list was being copied as a reference, therefore, copies of
a test ended up inadvertedly sharing the same list of reports. Reports
added by one instance of the test would be reflected in all instances.
This caused a race condition where, if a test was run on repeat with
gst-validate-launcher -f, very often wrong log file was shown to the
user. For instance, gst-validate-launcher would say "test failed, see
log for iteration7", but iteration7 would contain "TEST PASSED".
Worse, the runner would add the report to that incorrect log file,
mixing problems between different executions of the tests.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5177 >
2023-08-13 03:18:50 +00:00
Thibault Saunier
2837f14b6b
tests: Move srtp test known issue def to the 'check' testuite
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3979 >
2023-02-16 02:39:05 +00:00
Thibault Saunier
1146548c3d
validate:launcher: try to bne more explicit about where to add known failure
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3979 >
2023-02-16 02:39:04 +00:00
Thibault Saunier
c637d90be9
validate:launcher: Handle known return code issue in none validate based tests
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3979 >
2023-02-16 02:39:04 +00:00
Jan Schmidt
794ffe6ec6
validate: Place regex flags at the start of the regex
...
In Python 3.11 it is an error to have regex flags in the middle
of an expression, so make sure they appear at the start.
Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1630
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3643 >
2022-12-28 10:04:41 +00:00
Tim-Philipp Müller
6c621bba02
validate-launcher: update testsuites checks for fixed bug
...
Should make CI functional again.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3646 >
2022-12-27 17:14:06 +01:00
Thibault Saunier
9abceda343
validate:launcher: Cleanup test uuid when copying it
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3122 >
2022-10-05 20:29:22 +00:00
Thibault Saunier
f3c162cc85
validate: launcher: Add a argument to avoid rereuning flaky tests
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3122 >
2022-10-05 20:29:22 +00:00
Thibault Saunier
1577911d75
validate: launcher: Keep variable framerate from input when possible
...
But disable it if forcing a framerate for some reason
Fixing our support for variable framerate in the encoding profile
serialization format.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3122 >
2022-10-05 20:29:22 +00:00
Thibault Saunier
fbddaffc62
validate: launcher: Fix some pep8 issues
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3122 >
2022-10-05 20:29:22 +00:00
Thibault Saunier
f9e6044124
validate: Handle testfiles that need an HTTP server
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3122 >
2022-10-05 20:29:22 +00:00
Jordan Petridis
272560463c
validate: don't look for the common supression file
...
We no longer have a common submodule
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2841 >
2022-08-31 14:51:20 +00:00
Seungha Yang
0d5c937642
Revert "videoconvertscale: Add properties to disable scaling/converting in videoconvert/videoscale"
...
This reverts commit cd7a91cef1
.
Reverting properties, scaling in videoconvert and converting in
videoscale will be disabled by the other commit
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2778 >
2022-08-16 12:08:36 +00:00
Thibault Saunier
4de4aa1010
validate: Do not print Known error tests
...
Those are passing tests in practice and it doesn't give any useful information
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2264 >
2022-04-21 12:50:43 -04:00
Thibault Saunier
cd7a91cef1
videoconvertscale: Add properties to disable scaling/converting in videoconvert/videoscale
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/898 >
2022-04-20 17:09:20 -04:00
Haihua Hu
df0958e855
ximagesink/xvimagesink: use GST_XINITTHREADS to ensure call to XInitThreads
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2098 >
2022-04-04 15:49:02 +08:00
Mathieu Duponchelle
61eeb866b0
validate: loggable: fix Callable import
...
Since 3.3 importing Callable from collections is deprecated,
it should be imported from collections.abc .
Since 3.10 the alias has been removed altogether.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1504 >
2022-01-11 16:39:56 +00:00
Tim-Philipp Müller
8dfab0b08c
meson: update for meson.build_root() and .build_source() deprecation
...
-> use meson.project_build_root() or .global_build_root() instead.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1183 >
2021-10-20 11:20:44 +00:00
Thibault Saunier
9a183e45d4
validate:launcher: Reindent utils.py
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1121 >
2021-10-14 12:59:29 +00:00
Thibault Saunier
004a83f82a
validate:launcher: Check if the running MR will close a referenced issue
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1121 >
2021-10-14 12:59:29 +00:00
Thibault Saunier
42501adc4c
ci: Check for closed bugs for unit tests too
...
And add FIXMEs for test that should have been reenabled but haven't,
we will reenable them when CI is stable enough!
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1121 >
2021-10-14 12:59:29 +00:00
Thibault Saunier
968ecbba54
tests: Move all unit tests issues to the GStreamer monorepo
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1121 >
2021-10-14 12:59:29 +00:00
Thibault Saunier
287814a83d
Revert "tests: Blacklist some tests that recently failed"
...
This reverts commit 9f13ee5b0e
as
it was merge as part of the bigger CI gardening MR
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1139 >
2021-10-13 20:34:42 +00:00
Thibault Saunier
7a25a4ce02
ges:test: Document some GES failures
...
Namely races in:
* check.gst-editing-services.edit_while_seeked_with_stop_it
* check.gst-editing-services.check_layer_activness_gaps_it
See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/803
and https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/804
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1130 >
2021-10-13 18:01:38 +00:00