Thibault Saunier
54a1fc8b30
validate:launcher: Take ValidateAction execution as a marker of test update
2017-02-17 15:29:41 -03:00
Thibault Saunier
40016b9956
validate:scenario: Report action done with the execution duration
2017-02-17 15:29:41 -03:00
Thibault Saunier
7773ca7765
validate:launcher: Add a way to specify a set of tests to run under the leak tracer
...
https://bugzilla.gnome.org/show_bug.cgi?id=767856
2017-02-10 15:56:11 -03:00
Thibault Saunier
addd76fc85
validate: Mark tests as SKIPPED when installation is missing a GStreamer plugin
2017-02-07 13:12:09 -03:00
Thibault Saunier
823ce95d70
validate:launcher: Add an option for user to modify timeout values
...
Allowing to expand the test timeout when running on slow platforms
2017-02-06 12:16:41 -03:00
Thibault Saunier
5d1d8e6971
validate:launcher: Verify that Gst supression file could be found
2017-02-03 11:02:49 -03:00
Thibault Saunier
de2cbda854
validate: Make sure to update children environment from current env
...
And minor fix in function call
2017-01-06 12:12:21 -03:00
Thibault Saunier
ca1f8a03e0
validate:launcher: Make the IPC server socket blocking
...
0.0 does not mean blocking in python3 and makes everything failling
on windows.
2017-01-03 16:26:20 -03:00
Thibault Saunier
f24e22446b
validate: Properly kill subprocesses on windows
2017-01-03 16:24:19 -03:00
Thibault Saunier
f6d0636466
validate:launcher: Fix running on windows
2017-01-03 15:34:39 -03:00
Thibault Saunier
2863986ea3
validate: Respect active testers when listing tests
2017-01-03 14:52:38 -03:00
Thibault Saunier
0ee2147137
Revert "Revert "validate:launcher: Add an app handler for unit tests described in meson""
...
This reverts commit 5656e2a1b2
.
2017-01-03 13:01:31 -03:00
Edward Hervey
5656e2a1b2
Revert "validate:launcher: Add an app handler for unit tests described in meson"
...
This reverts commit a4aa5c60bb
.
Does not work outside of gst-build (i.e. validate won't work when used either
installed, in a prefix, or in gst-uninstalled).
2017-01-02 17:21:14 +01:00
Thibault Saunier
1bc6cb3615
validate:launcher: Take into account test duration when filtering them
...
Otherwise running -t 'some.*test' will run long tests (longer than hard
timeout) which is not what the user expect.
2016-12-23 14:59:03 -03:00
Thibault Saunier
d013e26cbc
validate: Whitlist blacklisted test only when explicitely
...
Otherwise running -t '.*reverse.*' will also run blacklisted
tests which is probably not what use wants.
2016-12-22 10:29:18 -03:00
Thibault Saunier
5fc11cf389
launcher: Make --update-media-info check if generating frames info or not
2016-12-22 10:29:18 -03:00
Thibault Saunier
a4aa5c60bb
validate:launcher: Add an app handler for unit tests described in meson
...
This way we can run all tests with the launcher which brings in many
features.
And add a testsuite for GStreamer unit tests.
2016-12-22 10:29:18 -03:00
Thibault Saunier
1f9c67e6ff
validate:launcher: Allow specifying a timeout factor
2016-12-22 10:08:27 -03:00
Thibault Saunier
085ebf03ba
validate:launcher: Print the iteration number when running forever
2016-12-22 10:08:26 -03:00
Thibault Saunier
d5de0b702c
validate:launcher: Do not list tests on unneeded testers
2016-12-22 10:08:23 -03:00
Thibault Saunier
6504b9152c
validate:launcher: Allow specifying several testsuite dirs
2016-12-22 10:08:21 -03:00
Thibault Saunier
26692e749c
validate:launcher: Stop running test subprocesses in a shell
...
And instead properly use a list of argument for the subprocesses.
2016-12-22 10:07:58 -03:00
Thibault Saunier
a842f9d511
validate:launcher: Handle missing media info file
2016-11-30 14:07:04 -03:00
Thibault Saunier
b0784ffcb3
validate:launcher: Make sure that the IPC server is shutdown before closing the socket
2016-11-30 07:40:05 -03:00
Thibault Saunier
2959af0761
validate:launcher: Make sure to check string when verifying expected failures
...
The value can potentially be None and we should handle that
2016-11-29 14:47:35 -03:00
Thibault Saunier
acfdd04e01
validate:launcher: Allow specifying timeout as a known issue
...
And minor cleanups
2016-11-28 19:37:08 -03:00
Thibault Saunier
03453961e8
validate:launcher: Properly report stack trace as such in the xunit file
2016-11-28 18:40:00 -03:00
Thibault Saunier
aec8693258
validate:launcher: Remove now useless validatelog
...
We are now doing IPC to communicate with the launcher
so let it simply go to stdout.
2016-11-28 18:40:00 -03:00
Thibault Saunier
e132c11a95
validate:launcher: Handle checking bug status for expected failures
2016-11-28 18:40:00 -03:00
Thibault Saunier
2857eaf2ad
validate: launcher: Allow checking if bugs linked to blacklist is fixed
2016-11-16 17:39:45 -03:00
Thibault Saunier
615fab620a
validate:launcher: Consider wanted tests as whitelisted
2016-11-16 12:02:32 -03:00
Thibault Saunier
f5828b2b5c
validate:launcher: Allow running the testsuite N number of times
2016-11-14 13:32:07 -03:00
Thibault Saunier
5452672e89
validate:launcher: Try to generate a backtrace on segfaults
2016-11-09 10:19:45 -03:00
Thibault Saunier
1e51aeb942
validate:launcher: Port to Python3
...
And sync logging.py with Pitivi version
2016-11-09 10:13:42 -03:00
Edward Hervey
a29f4c390c
baseclasses: Also check for minimum number of video-tracks
...
Some scenarios might only be for video files and are meaningless for
audio-only files
2016-10-26 17:34:49 +02:00
Thibault Saunier
41be7bb2a9
validate:launcher: Minor fix in returncode check
2016-09-26 13:36:48 -03:00
Thibault Saunier
e376da4c17
validate:launcher: Do not use unset sent_eos variable
...
And rename class member to sent_eos_time as it is more accurate
2016-09-12 14:21:30 -03:00
Thibault Saunier
9c9fe14293
meson:validate:test: Properly set paths to run launcher based tests
...
Adding a --validate-tools-path option to the launcher, allowing
to pass it from meson.
2016-09-09 12:24:54 -03:00
Thibault Saunier
3ece6a065e
validate: launcher: Always clean all tests at the end
...
Making sure that if an exception of anything happens we will
properly clean all the tests, or at least try to.
2016-09-08 13:10:29 -03:00
Thibault Saunier
03548cd63d
validate:launcher: Allow specifying expected tests errors
...
In the future instead of blacklisting tests we should define
what error is expected, and this way when the bug is closed,
we will notice, also, it will allow us to check GstValidate
error reporting itself.
2016-09-08 13:10:29 -03:00
Thibault Saunier
2fff14e469
validate: Pass information about GstValidate execution over a socket
...
Instead of trying to parsing stdout, generate json messages and
send them over a socket so that gst-validate-launcher can properly
have informations about gst-validate subprocess execution.
2016-09-08 13:10:29 -03:00
Thibault Saunier
23203ca0bd
validate:launcher: Use a xunit reporter only when explicitely specified
2016-09-08 10:39:56 -03:00
Thibault Saunier
8973fa04eb
validate:launcher: Report error and exit when a testsuite could not load
...
Otherwise the user might end up seeing a lot of meaningless logs about
'removed' tests.
2016-09-08 10:39:56 -03:00
Mathieu Duponchelle
ec1d8d7d8c
launcher: Add --dump-on-failure switch
...
When the test fails, it can be useful to have the log files dumped
to stdout.
https://bugzilla.gnome.org/show_bug.cgi?id=741092
2016-09-01 03:39:18 +02:00
Thibault Saunier
5a745829dc
validate: Add jpeg as known format
2016-07-29 15:54:02 -04:00
Thibault Saunier
bcc7a1fcca
validate:launcher: Add h265, opus and vp9 as known formats
2016-07-15 08:56:02 -04:00
Guillaume Desmottes
2ddbcb666f
validate: generate valgrind suppression traces
...
Makes fixing easier as then we can just re-use the generated trace.
Reviewed-by: Thibault Saunier <tsaunier@gnome.org>
Differential Revision: https://phabricator.freedesktop.org/D953
2016-05-24 10:28:26 -04:00
Thibault Saunier
83c652ba76
validate: launcher: Make sure to properly setup all testsuites
...
When a first testsuite will set paths, it does not mean that we should
just register following testsuite test manager default tests.
So we need to make a difference between the media paths the user passed
with --media-path and the ones defined by the testsuite.
2016-03-23 20:10:51 +01:00
Thibault Saunier
9190bcf62a
validate:launcher: Handle testslist files even running several testsuites
...
Only if those testsuites do not use the same Tester as we
currently can't know to what testsuite a test belongs.
2016-03-23 20:10:01 +01:00
Thibault Saunier
621bda9b62
validate: Use GstValidate logging system to print buffering avancement
...
So it can be used in the launcher.
And make sure to properly parse the info in the launcher.
2016-03-21 14:34:27 +01:00