Thibault Saunier
8639dc528c
validate:launcher: Add an option to output HTML if commonmark is installed
2019-04-11 13:17:14 +00:00
Thibault Saunier
92f1979ec9
validate: Add a mecanism to rerun failling tests
...
And add a way to mark some 'flakes' as tolerated
2019-04-03 13:38:42 +00:00
Thibault Saunier
8da6ecef13
validate:launcher: By default use cpu_count / 2
2019-04-03 13:38:42 +00:00
Thibault Saunier
19dfd63408
validate:launcher: Review default for --mute
...
`--mute` is now True by default and added `--unmute` if the user wants
to have visual/audio feedback. It was really annoying to have thousand
of window pop up by default
2019-04-03 13:38:42 +00:00
Thibault Saunier
3a826e1e3d
validate:launcher: Implement bug checks for gitlab
...
And use new gitlab urls for all the bugs
2019-04-03 13:38:42 +00:00
Thibault Saunier
3d5099ee41
validate:launcher: Use gst-integration-testsuites subproject as default testsuite repo if avalaible
2019-04-03 13:38:42 +00:00
Thibault Saunier
9b69bcad08
validate:launcher: Handle launching launching a sub launcher
...
If you use validate-launcher in a meson testsuite, those test now
gets integrated as one unique testsuite (with a pretty long namespace).
2019-01-26 15:54:00 +00:00
Thibault Saunier
6665652cff
validate:launcher: Refactor the "main" function
...
- Move the parser code into a `LauncherConfig.create_parser()` method
- Remove the need to pass libsdir to the _TestsLauncher object
- Extract out a `setup_launcher_from_args` function
2019-01-26 15:54:00 +00:00
Thibault Saunier
1b3867b82d
launcher: Move http serveur and xvfb server to the main test runner object
...
No good reason for it to be in the main function
2019-01-26 15:54:00 +00:00
Thibault Saunier
1c51d3499e
validate:launcher: Add a GST_VALIDATE_LAUNCHER_MAIN_DIR env variable
...
This is generally usefull so we do not have to pass -M every time we launch the launcher
And it adds support for nesting launcher calls always respecting the provided main directory
+ Fix some new pep8 errors
2018-11-30 11:00:10 -03:00
Thibault Saunier
a28ce1b0c9
validate: Update default testsuite git repository
2018-11-22 21:06:36 -03:00
Alicia Boya García
2c32ccd7a7
gst-validate-launcher: Stop in --gdb by default, add --gdb-non-stop
...
This patch modifies the default behavior of --gdb to not run and quit
automatically the test, but rather wait for user input. This is
usually much more convenient to debug all kinds of bugs.
The automatic run behavior has been moved to a new command switch:
--gdb-non-stop
https://bugzilla.gnome.org/show_bug.cgi?id=796389
2018-05-25 12:59:38 +02:00
Edward Hervey
58e62f651c
validate-launcher: Allow running tests out-of-order
...
When the --shuffle option is used, the tests will be run out of order.
This optimizes CPU utilization since it allows running synchronized
and unsynchronized tests at the same.
2017-12-03 11:42:18 +01:00
Thibault Saunier
e9862b9fda
validate:launcher: Allow disabling using the number of failed tests as exitcode
...
This is usefull on CI servers where the test results will be inspected
and the status of the build built from it.
2017-09-06 16:38:39 -03:00
Jimmy Ohn
bd0e8e410e
validate: launcher: Modify the order of the parser argument
...
Modify the order of the parser argument before setting dir_group
https://bugzilla.gnome.org/show_bug.cgi?id=786715
2017-08-28 17:01:39 -03:00
Thibault Saunier
2c6c25d4bf
validate:launcher: Use the number of failed test as exit code
...
We used to always return 0, which was not right!
2017-08-18 11:37:28 -03:00
Thibault Saunier
2cf93f491b
validate: launcher: Namespace test name with the testsuite name
...
Also allowing users to pass test names directly
2017-07-18 10:54:53 -04:00
Thibault Saunier
823ce95d70
validate:launcher: Add an option for user to modify timeout values
...
Allowing to expand the test timeout when running on slow platforms
2017-02-06 12:16:41 -03:00
Thibault Saunier
0ee2147137
Revert "Revert "validate:launcher: Add an app handler for unit tests described in meson""
...
This reverts commit 5656e2a1b2
.
2017-01-03 13:01:31 -03:00
Edward Hervey
167bfca852
validate: Ensure non-standard testsuite location is taken into account
...
This is a regression that was introduced by 6504b9152c
If we have non-standard main_dir or qa_assets, make sure we prepend the
checked-out testsuites directory to the list of expected ones
2017-01-03 07:05:12 +01:00
Edward Hervey
5656e2a1b2
Revert "validate:launcher: Add an app handler for unit tests described in meson"
...
This reverts commit a4aa5c60bb
.
Does not work outside of gst-build (i.e. validate won't work when used either
installed, in a prefix, or in gst-uninstalled).
2017-01-02 17:21:14 +01:00
Thibault Saunier
5fc11cf389
launcher: Make --update-media-info check if generating frames info or not
2016-12-22 10:29:18 -03:00
Thibault Saunier
a4aa5c60bb
validate:launcher: Add an app handler for unit tests described in meson
...
This way we can run all tests with the launcher which brings in many
features.
And add a testsuite for GStreamer unit tests.
2016-12-22 10:29:18 -03:00
Thibault Saunier
58711edce8
validate:launcher: Add a -v option to print subprocesses to stdout
2016-12-22 10:08:25 -03:00
Thibault Saunier
076c21fbe5
validate:launcher: Run cpu_count test in parallel by default
2016-12-22 10:08:24 -03:00
Thibault Saunier
d5de0b702c
validate:launcher: Do not list tests on unneeded testers
2016-12-22 10:08:23 -03:00
Thibault Saunier
6504b9152c
validate:launcher: Allow specifying several testsuite dirs
2016-12-22 10:08:21 -03:00
Thibault Saunier
e3731ba839
validate: Minor documentation fixes.
2016-11-17 15:43:15 -03:00
Thibault Saunier
2857eaf2ad
validate: launcher: Allow checking if bugs linked to blacklist is fixed
2016-11-16 17:39:45 -03:00
Thibault Saunier
f5828b2b5c
validate:launcher: Allow running the testsuite N number of times
2016-11-14 13:32:07 -03:00
Thibault Saunier
1e51aeb942
validate:launcher: Port to Python3
...
And sync logging.py with Pitivi version
2016-11-09 10:13:42 -03:00
Thibault Saunier
3ece6a065e
validate: launcher: Always clean all tests at the end
...
Making sure that if an exception of anything happens we will
properly clean all the tests, or at least try to.
2016-09-08 13:10:29 -03:00
Thibault Saunier
23203ca0bd
validate:launcher: Use a xunit reporter only when explicitely specified
2016-09-08 10:39:56 -03:00
Thibault Saunier
8973fa04eb
validate:launcher: Report error and exit when a testsuite could not load
...
Otherwise the user might end up seeing a lot of meaningless logs about
'removed' tests.
2016-09-08 10:39:56 -03:00
Mathieu Duponchelle
ec1d8d7d8c
launcher: Add --dump-on-failure switch
...
When the test fails, it can be useful to have the log files dumped
to stdout.
https://bugzilla.gnome.org/show_bug.cgi?id=741092
2016-09-01 03:39:18 +02:00
Edward Hervey
1e1475d352
validate: Fix usage for non-standard http server port
...
Translate the various stream_info URI from the standard port to the
specified port
Make the port option an integer option
2016-07-04 16:19:09 +02:00
Reynaldo H. Verdejo Pinochet
07fa1b3ca5
validate: fix some recurring typos
2016-06-09 14:31:48 -07:00
Thiago Santos
f4cdbd006b
launcher: set gsettings-backend to prevent weird deadlocks
...
It seems like some sort of forking/dconf/gtype combination can
deadlock occasionally. Setting the gsettings backend to memory
makes it go away.
Same issue: https://cgit.freedesktop.org/gstreamer/gst-plugins-good/commit/tests/check/Makefile.am?id=8e2c1d1de56bddbff22170f8b17473882e0e63f9
2016-05-15 06:34:21 -03:00
Thibault Saunier
83c652ba76
validate: launcher: Make sure to properly setup all testsuites
...
When a first testsuite will set paths, it does not mean that we should
just register following testsuite test manager default tests.
So we need to make a difference between the media paths the user passed
with --media-path and the ones defined by the testsuite.
2016-03-23 20:10:51 +01:00
Thibault Saunier
39a0be8e0e
validate:launcher: Add a way to fail if test have been removed/added
2016-02-23 11:47:15 +01:00
Thibault Saunier
ade0fb3fed
validate: launcher: Set scenario manager config before discovering testsuites
2015-11-16 16:45:13 +01:00
Thibault Saunier
8899fd8dd1
validate: Implement support to run tests inside gdb
...
Making debugging races leading to crashes easier to debug
2015-11-10 18:17:54 +01:00
Vineeth TM
e4d490fd45
validate:launcher: Fix documentation
...
Fix some trivial spelling mistakes in documentation
and document about --update-media-info.
https://bugzilla.gnome.org/show_bug.cgi?id=752748
2015-08-05 17:17:41 -04:00
Thibault Saunier
1fd14635ed
validate: launcher: Add a --force-sync option
...
Which should put the testsuite in a clean state (basically using git
reset --hard for git based testsuite for example)
2015-08-04 15:23:54 +02:00
Vineeth T M
6c48e89aab
validate:launcher: skip default media path for --media-paths option
...
when --media-paths is specified, then no need to check the default media.
And add Force argument to let testsuite force the inclusion of
default media directory.
https://bugzilla.gnome.org/show_bug.cgi?id=752461
2015-07-16 16:31:09 +02:00
Vineeth T M
1ef0fdd66c
validate:main.py: trivial document fixes
...
gst-validate-launch is being used instead of gst-validate-launcher
in a couple of places.
https://bugzilla.gnome.org/show_bug.cgi?id=752455
2015-07-16 09:35:17 +02:00
Thibault Saunier
a0b3287d40
validate:launcher: Add a way to run tests without displaying the output
...
Summary:
Adding a --no-display option and running Xvfb virtual frame buffer X
server.
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D105
2015-04-17 21:51:29 +02:00
Thibault Saunier
76eedece5c
validate:launcher: Error out if valgrind is not available on the system
...
Summary:
When the user wants to use valgrind, make sure it is present on the
system before doing anything
Reviewers: gdesmott
Differential Revision: http://phabricator.freedesktop.org/D104
2015-04-17 19:58:18 +02:00
Thibault Saunier
0fcee6ed84
validate:launcher: Make validate the only default testsuite
...
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D93
2015-04-13 11:27:08 +02:00
Thibault Saunier
5a6d6b9475
validate:launcher: Let the responsibility to update asset to the testsuite
...
Summary:
It makes it easier to make sure that the assets needed for a specific
testsuite are available when needed
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D92
2015-04-13 11:27:08 +02:00