Alicia Boya García
ba61160d6c
validate-launcher: Fix race condition on repeated tests
...
Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2900
The `reports` list was being copied as a reference, therefore, copies of
a test ended up inadvertedly sharing the same list of reports. Reports
added by one instance of the test would be reflected in all instances.
This caused a race condition where, if a test was run on repeat with
gst-validate-launcher -f, very often wrong log file was shown to the
user. For instance, gst-validate-launcher would say "test failed, see
log for iteration7", but iteration7 would contain "TEST PASSED".
Worse, the runner would add the report to that incorrect log file,
mixing problems between different executions of the tests.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5177 >
2023-08-13 03:18:50 +00:00
Thibault Saunier
2837f14b6b
tests: Move srtp test known issue def to the 'check' testuite
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3979 >
2023-02-16 02:39:05 +00:00
Thibault Saunier
1146548c3d
validate:launcher: try to bne more explicit about where to add known failure
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3979 >
2023-02-16 02:39:04 +00:00
Thibault Saunier
c637d90be9
validate:launcher: Handle known return code issue in none validate based tests
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3979 >
2023-02-16 02:39:04 +00:00
Jan Schmidt
794ffe6ec6
validate: Place regex flags at the start of the regex
...
In Python 3.11 it is an error to have regex flags in the middle
of an expression, so make sure they appear at the start.
Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1630
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3643 >
2022-12-28 10:04:41 +00:00
Tim-Philipp Müller
6c621bba02
validate-launcher: update testsuites checks for fixed bug
...
Should make CI functional again.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3646 >
2022-12-27 17:14:06 +01:00
Thibault Saunier
9abceda343
validate:launcher: Cleanup test uuid when copying it
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3122 >
2022-10-05 20:29:22 +00:00
Thibault Saunier
f3c162cc85
validate: launcher: Add a argument to avoid rereuning flaky tests
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3122 >
2022-10-05 20:29:22 +00:00
Thibault Saunier
1577911d75
validate: launcher: Keep variable framerate from input when possible
...
But disable it if forcing a framerate for some reason
Fixing our support for variable framerate in the encoding profile
serialization format.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3122 >
2022-10-05 20:29:22 +00:00
Thibault Saunier
fbddaffc62
validate: launcher: Fix some pep8 issues
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3122 >
2022-10-05 20:29:22 +00:00
Thibault Saunier
f9e6044124
validate: Handle testfiles that need an HTTP server
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3122 >
2022-10-05 20:29:22 +00:00
Jordan Petridis
272560463c
validate: don't look for the common supression file
...
We no longer have a common submodule
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2841 >
2022-08-31 14:51:20 +00:00
Seungha Yang
0d5c937642
Revert "videoconvertscale: Add properties to disable scaling/converting in videoconvert/videoscale"
...
This reverts commit cd7a91cef1
.
Reverting properties, scaling in videoconvert and converting in
videoscale will be disabled by the other commit
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2778 >
2022-08-16 12:08:36 +00:00
Thibault Saunier
4de4aa1010
validate: Do not print Known error tests
...
Those are passing tests in practice and it doesn't give any useful information
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2264 >
2022-04-21 12:50:43 -04:00
Thibault Saunier
cd7a91cef1
videoconvertscale: Add properties to disable scaling/converting in videoconvert/videoscale
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/898 >
2022-04-20 17:09:20 -04:00
Haihua Hu
df0958e855
ximagesink/xvimagesink: use GST_XINITTHREADS to ensure call to XInitThreads
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2098 >
2022-04-04 15:49:02 +08:00
Mathieu Duponchelle
61eeb866b0
validate: loggable: fix Callable import
...
Since 3.3 importing Callable from collections is deprecated,
it should be imported from collections.abc .
Since 3.10 the alias has been removed altogether.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1504 >
2022-01-11 16:39:56 +00:00
Tim-Philipp Müller
8dfab0b08c
meson: update for meson.build_root() and .build_source() deprecation
...
-> use meson.project_build_root() or .global_build_root() instead.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1183 >
2021-10-20 11:20:44 +00:00
Thibault Saunier
9a183e45d4
validate:launcher: Reindent utils.py
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1121 >
2021-10-14 12:59:29 +00:00
Thibault Saunier
004a83f82a
validate:launcher: Check if the running MR will close a referenced issue
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1121 >
2021-10-14 12:59:29 +00:00
Thibault Saunier
42501adc4c
ci: Check for closed bugs for unit tests too
...
And add FIXMEs for test that should have been reenabled but haven't,
we will reenable them when CI is stable enough!
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1121 >
2021-10-14 12:59:29 +00:00
Thibault Saunier
968ecbba54
tests: Move all unit tests issues to the GStreamer monorepo
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1121 >
2021-10-14 12:59:29 +00:00
Thibault Saunier
287814a83d
Revert "tests: Blacklist some tests that recently failed"
...
This reverts commit 9f13ee5b0e
as
it was merge as part of the bigger CI gardening MR
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1139 >
2021-10-13 20:34:42 +00:00
Thibault Saunier
7a25a4ce02
ges:test: Document some GES failures
...
Namely races in:
* check.gst-editing-services.edit_while_seeked_with_stop_it
* check.gst-editing-services.check_layer_activness_gaps_it
See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/803
and https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/804
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1130 >
2021-10-13 18:01:38 +00:00
Thibault Saunier
fbee6c8b23
validate:launcher: Avoid spamming terminal when inspecting unit tests
...
And redirect GStreamer logs when necessary
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1130 >
2021-10-13 18:01:38 +00:00
Thibault Saunier
34d05cd876
validate:launcher: Run validate unit tests as GstValidate tests
...
So we have all the features and we can properly document known issue
for them
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1130 >
2021-10-13 18:01:38 +00:00
Thibault Saunier
9f13ee5b0e
tests: Blacklist some tests that recently failed
...
See:
- https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/739
- https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/740
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/927 >
2021-10-13 12:37:08 +00:00
Thibault Saunier
4eb355ea13
tests: Mark check.gstreamer.gst_gstbin.test_watch_for_state_change flaky
...
See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/778
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113 >
2021-10-12 16:59:16 +00:00
Thibault Saunier
673c6c59d7
ges:tests: Mark check.gst-plugins-base.validate.giosrc.read-growing-file as flaky
...
See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/776
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113 >
2021-10-12 16:59:16 +00:00
Thibault Saunier
c10aaa07b0
ges:tests: Mark ges 'check_edit_in_frames_with_framerate_mistmatch as flaky
...
See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/775
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113 >
2021-10-12 16:59:16 +00:00
Thibault Saunier
2d4ba75732
tests:webrtcbin: Make known flaky webrtcbin as such
...
See https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/773
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113 >
2021-10-12 16:59:16 +00:00
Thibault Saunier
7dfaac92fa
validate:launcher: Do not dump on failure test we will retry
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113 >
2021-10-12 16:59:16 +00:00
Thibault Saunier
47afe6db28
tests: Blacklist tests that are currently producing many CI failures
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113 >
2021-10-12 16:59:16 +00:00
Thibault Saunier
23dbfdf70d
validate:launcher: Allow more flexible handling of flaky tests
...
And retry twice flaky tests
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113 >
2021-10-12 16:59:16 +00:00
Thibault Saunier
ff8d8bbc97
validate:launcher: Fix progress reporting with --n-runs
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113 >
2021-10-12 16:59:16 +00:00
Thibault Saunier
4762b138b6
validate:launcher: Support retry on failure when using --n-runs
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113 >
2021-10-12 16:59:16 +00:00
Thibault Saunier
f2efd3b133
validate:launcher: Fix potential invalid index access
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113 >
2021-10-12 16:59:16 +00:00
Thibault Saunier
b084abad79
validate:launcher: Fix adding logs to xunit
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1113 >
2021-10-12 16:59:16 +00:00
Thibault Saunier
ce76a286ed
validate:launcher: Sync asset only when passing --sync
...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1020 >
2021-10-06 17:00:54 +00:00
Thibault Saunier
776d8a6617
Add gst-integration-testsuites files as a submodule
2021-09-24 16:36:50 -03:00
Thibault Saunier
2701634c29
Move files from gst-devtools into the "subprojects/gst-devtools/" subdir
2021-09-24 16:15:38 -03:00