Thibault Saunier
766f339fb8
launcher: Add support for running tests inside rr
...
Allowing us to easily run the tests forever and then replay the
failures!
https://rr-project.org/
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197 >
2020-05-26 17:19:36 -04:00
Thibault Saunier
0b41b8a8c0
validate:launcher: Add a generator to generate test for frame accurate seeking
2020-03-12 12:35:45 -03:00
Thibault Saunier
2e45db5ba4
validate:launcher: Enhance support for running ssim tests
...
Using a special 'ssim' variable in pipeline dicts to activate it
Similare to what we do for validateflow
2020-03-11 16:36:20 -03:00
Thibault Saunier
080fdf8e2f
validate:launcher: Add support for the imagesequence protocol
2020-03-06 16:05:38 -03:00
Thibault Saunier
3343f166da
validate:launcher: Move get_fakesink_for_media_type to utils
...
So it can be reused in other apps like GES
2019-06-26 12:09:03 -04:00
Thibault Saunier
0a56447bcc
validate:launcher: Add python suppression files
2019-04-11 13:17:14 +00:00
Thibault Saunier
f7517e503c
validate:launcher: Do not user python 3.5 features
2019-04-11 13:17:14 +00:00
Thibault Saunier
787939f750
validate:launcher: Add a list of well known subpression files from gst-build subprojects
...
Fixes https://gitlab.freedesktop.org/gstreamer/gst-devtools/issues/38
2019-04-03 13:38:42 +00:00
Thibault Saunier
01aa026e5a
validate:launcher: Try to send SIGINT before killing processes
...
Giving a chance for validate to print reports
2019-04-03 13:38:42 +00:00
Thibault Saunier
f2e180e51f
validate:launcher: Add a way to force coloration
2019-04-03 13:38:42 +00:00
Thibault Saunier
ddb81f29e8
validate:launcher: factor out TTY check and enhance iteration output
2019-04-03 13:38:42 +00:00
Thibault Saunier
c8a3b34a3a
validate:launcher: Add a way to force stdout coloring
2019-04-03 13:38:42 +00:00
Thibault Saunier
3a826e1e3d
validate:launcher: Implement bug checks for gitlab
...
And use new gitlab urls for all the bugs
2019-04-03 13:38:42 +00:00
Thibault Saunier
3d5099ee41
validate:launcher: Use gst-integration-testsuites subproject as default testsuite repo if avalaible
2019-04-03 13:38:42 +00:00
Thibault Saunier
773f2e456c
validate:launcher: Move '_format_config_template' to the utilities
...
So it can be reused by GES tests
2019-03-11 18:14:23 -03:00
Thibault Saunier
1c51d3499e
validate:launcher: Add a GST_VALIDATE_LAUNCHER_MAIN_DIR env variable
...
This is generally usefull so we do not have to pass -M every time we launch the launcher
And it adds support for nesting launcher calls always respecting the provided main directory
+ Fix some new pep8 errors
2018-11-30 11:00:10 -03:00
Thibault Saunier
69ec48feef
validate:launcher: Allow retrieving coredumps from within flatpak
2018-07-01 12:55:06 -04:00
Thibault Saunier
58c90448ca
validate: launcher: Add support for running tests with a pushfile source
...
Introducing the `.media_info.push` media info extension, which is meant
to let the launcher know that those file should run with the "pushfile://"
protocol.
And allow symlinking "normal" `.media_info` to their `.pushfile` variant
so that both can share the exact same content.
2018-06-15 12:05:14 -04:00
Thibault Saunier
c1f89b4acb
validate: launcher: Add a way to simply run SSIM checks on rendered files
...
We will run a simple pipeline with the IQA element to run ssim (dssim)
tests on the rendered files, comparing it with a reference file.
For now we use the very empiric 1.0 value as a ssim error threshold and
the goal is basically to detect completely broken renderings.
2018-05-13 09:44:17 -04:00
Thibault Saunier
25e3b90225
validate:launcher: printc is accepting object as arguments
...
Make sure in all code paths those are converted to strings
2018-04-15 20:07:19 -03:00
Thibault Saunier
a70658a464
launcher: Print test number in the order they finish
...
Instead of the test index in the list of tests as it is
meaningless to the user and feels weird.
Also minor fix in the test name display when running with --forever.
2018-03-23 21:04:46 -03:00
Thibault Saunier
2b40c6c306
launcher: Clean up outpout
...
Make our stdout output simpler to follow by:
- Not printing the tests we launch (it is not really useful in the end)
- Using `\r` when printing the passed tests
- Not reprinting all the test in a now useless summary
2018-03-23 18:12:16 -03:00
Edward Hervey
1a95559045
validate: Reduce time waiting for subprocess to stop
...
stopping the subprocess is done from the main thread, this would
throttle starting/stopping any tests by one second.
Start with 50ms, and gradually increase the wait between iterations
2017-12-03 11:38:08 +01:00
Thibault Saunier
4dc95f0fc2
launcher: Automatically disable output coloration if not supported
2017-08-26 10:52:25 -03:00
Thibault Saunier
3c62c315a9
validate: Implement RTSP support
2017-06-07 11:17:53 -04:00
Thibault Saunier
addd76fc85
validate: Mark tests as SKIPPED when installation is missing a GStreamer plugin
2017-02-07 13:12:09 -03:00
Thibault Saunier
f24e22446b
validate: Properly kill subprocesses on windows
2017-01-03 16:24:19 -03:00
Thibault Saunier
acfdd04e01
validate:launcher: Allow specifying timeout as a known issue
...
And minor cleanups
2016-11-28 19:37:08 -03:00
Thibault Saunier
e132c11a95
validate:launcher: Handle checking bug status for expected failures
2016-11-28 18:40:00 -03:00
Thibault Saunier
2857eaf2ad
validate: launcher: Allow checking if bugs linked to blacklist is fixed
2016-11-16 17:39:45 -03:00
Thibault Saunier
8bd0cb13e8
validate:launcher: Timeout if running gdb takes too much time
2016-11-14 18:44:27 -03:00
Thibault Saunier
20879e9c69
validate:launcher: Fix usage in an uninstalled environment
2016-11-09 17:38:04 -03:00
Thibault Saunier
5452672e89
validate:launcher: Try to generate a backtrace on segfaults
2016-11-09 10:19:45 -03:00
Thibault Saunier
1e51aeb942
validate:launcher: Port to Python3
...
And sync logging.py with Pitivi version
2016-11-09 10:13:42 -03:00
Thibault Saunier
9c9fe14293
meson:validate:test: Properly set paths to run launcher based tests
...
Adding a --validate-tools-path option to the launcher, allowing
to pass it from meson.
2016-09-09 12:24:54 -03:00
Vineeth T M
65560d8b03
Validate: Fix consider, launch spelling mistakes
...
https://bugzilla.gnome.org/show_bug.cgi?id=763289
2016-03-08 09:28:11 +02:00
Sebastian Dröge
233c99fbb7
validate/launcher: Treat DASH like HLS in another place
2015-08-15 16:19:46 +02:00
Vineeth T M
074406bc2c
validate:launcher: unquote the path to remove special characters
...
When getting path from url using, url2path, it is returning
special characters (%20 for space etc..), instead of returning
plain path. path.unquote replaces the same..
https://bugzilla.gnome.org/show_bug.cgi?id=752611
2015-07-22 16:39:43 +01:00
Guillaume Desmottes
f6681ff003
validate: rename get_valgrind_suppression_file()
...
Summary:
This function is actually not specific to valgrind so we can make it more
generic.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D162
2015-05-11 21:55:24 +02:00
Guillaume Desmottes
c5c39d88b1
validate: move look_for_file_in_source_dir and get_valgrind_suppression_file to utils
...
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D130
2015-05-01 15:15:26 +02:00
Thibault Saunier
22fba81138
validate: Rename gst-qa-assets to gst-integration-testsuites
2014-12-05 20:51:43 +01:00
Thibault Saunier
71566688bf
validate: Remove the default testsuite implementation
...
The default testsuite implementation should belong to the default
asset repo where we have the corresponding knowledge.
We should style manage a sensible list of known blacklisted tests,
encoding profiles, and generators in GstValidate itself and allow testsuite
actual implementations to easily use them though the register_default_*
methods.
This allow us to be able to remove the ugly execfile() call.
2014-12-05 12:17:40 +01:00
Thibault Saunier
36a2d61068
validate:launcher: Force clock sync for some protocols
...
In HLS for example, not having clock sync might lead to races and failures
do not test that for now
2014-11-20 12:09:29 +01:00
Mathieu Duponchelle
9bdc6e9b6d
validate-launcher: utils: fix various pyflakes / uncaught pep8 issues.
...
https://bugzilla.gnome.org/show_bug.cgi?id=739208
2014-10-30 15:40:17 +01:00
Mathieu Duponchelle
8c1e84b5f4
validate-launcher: pep8ify sources.
...
https://bugzilla.gnome.org/show_bug.cgi?id=739208
2014-10-30 15:40:17 +01:00
Mathieu Duponchelle
b0beefacfa
validate-launcher: restructure filesystem
...
https://bugzilla.gnome.org/show_bug.cgi?id=739091
2014-10-30 15:31:29 +01:00