Thibault Saunier
0ee2147137
Revert "Revert "validate:launcher: Add an app handler for unit tests described in meson""
...
This reverts commit 5656e2a1b2
.
2017-01-03 13:01:31 -03:00
Edward Hervey
167bfca852
validate: Ensure non-standard testsuite location is taken into account
...
This is a regression that was introduced by 6504b9152c
If we have non-standard main_dir or qa_assets, make sure we prepend the
checked-out testsuites directory to the list of expected ones
2017-01-03 07:05:12 +01:00
Edward Hervey
5656e2a1b2
Revert "validate:launcher: Add an app handler for unit tests described in meson"
...
This reverts commit a4aa5c60bb
.
Does not work outside of gst-build (i.e. validate won't work when used either
installed, in a prefix, or in gst-uninstalled).
2017-01-02 17:21:14 +01:00
Thibault Saunier
1bc6cb3615
validate:launcher: Take into account test duration when filtering them
...
Otherwise running -t 'some.*test' will run long tests (longer than hard
timeout) which is not what the user expect.
2016-12-23 14:59:03 -03:00
Thibault Saunier
d013e26cbc
validate: Whitlist blacklisted test only when explicitely
...
Otherwise running -t '.*reverse.*' will also run blacklisted
tests which is probably not what use wants.
2016-12-22 10:29:18 -03:00
Thibault Saunier
5fc11cf389
launcher: Make --update-media-info check if generating frames info or not
2016-12-22 10:29:18 -03:00
Thibault Saunier
a4aa5c60bb
validate:launcher: Add an app handler for unit tests described in meson
...
This way we can run all tests with the launcher which brings in many
features.
And add a testsuite for GStreamer unit tests.
2016-12-22 10:29:18 -03:00
Thibault Saunier
1f9c67e6ff
validate:launcher: Allow specifying a timeout factor
2016-12-22 10:08:27 -03:00
Thibault Saunier
085ebf03ba
validate:launcher: Print the iteration number when running forever
2016-12-22 10:08:26 -03:00
Thibault Saunier
58711edce8
validate:launcher: Add a -v option to print subprocesses to stdout
2016-12-22 10:08:25 -03:00
Thibault Saunier
076c21fbe5
validate:launcher: Run cpu_count test in parallel by default
2016-12-22 10:08:24 -03:00
Thibault Saunier
d5de0b702c
validate:launcher: Do not list tests on unneeded testers
2016-12-22 10:08:23 -03:00
Thibault Saunier
6504b9152c
validate:launcher: Allow specifying several testsuite dirs
2016-12-22 10:08:21 -03:00
Thibault Saunier
26692e749c
validate:launcher: Stop running test subprocesses in a shell
...
And instead properly use a list of argument for the subprocesses.
2016-12-22 10:07:58 -03:00
Thibault Saunier
e552c18c80
launcher: Handle stack trace information as jenkins expect it in the xunit file
2016-12-01 10:52:32 -03:00
Thibault Saunier
a842f9d511
validate:launcher: Handle missing media info file
2016-11-30 14:07:04 -03:00
Thibault Saunier
b0784ffcb3
validate:launcher: Make sure that the IPC server is shutdown before closing the socket
2016-11-30 07:40:05 -03:00
Thibault Saunier
2959af0761
validate:launcher: Make sure to check string when verifying expected failures
...
The value can potentially be None and we should handle that
2016-11-29 14:47:35 -03:00
Thibault Saunier
acfdd04e01
validate:launcher: Allow specifying timeout as a known issue
...
And minor cleanups
2016-11-28 19:37:08 -03:00
Thibault Saunier
03453961e8
validate:launcher: Properly report stack trace as such in the xunit file
2016-11-28 18:40:00 -03:00
Thibault Saunier
aec8693258
validate:launcher: Remove now useless validatelog
...
We are now doing IPC to communicate with the launcher
so let it simply go to stdout.
2016-11-28 18:40:00 -03:00
Thibault Saunier
e132c11a95
validate:launcher: Handle checking bug status for expected failures
2016-11-28 18:40:00 -03:00
Thibault Saunier
e3731ba839
validate: Minor documentation fixes.
2016-11-17 15:43:15 -03:00
Thibault Saunier
2857eaf2ad
validate: launcher: Allow checking if bugs linked to blacklist is fixed
2016-11-16 17:39:45 -03:00
Thibault Saunier
615fab620a
validate:launcher: Consider wanted tests as whitelisted
2016-11-16 12:02:32 -03:00
Thibault Saunier
8bd0cb13e8
validate:launcher: Timeout if running gdb takes too much time
2016-11-14 18:44:27 -03:00
Thibault Saunier
f5828b2b5c
validate:launcher: Allow running the testsuite N number of times
2016-11-14 13:32:07 -03:00
Thibault Saunier
20879e9c69
validate:launcher: Fix usage in an uninstalled environment
2016-11-09 17:38:04 -03:00
Thibault Saunier
5452672e89
validate:launcher: Try to generate a backtrace on segfaults
2016-11-09 10:19:45 -03:00
Thibault Saunier
1e51aeb942
validate:launcher: Port to Python3
...
And sync logging.py with Pitivi version
2016-11-09 10:13:42 -03:00
Thibault Saunier
d797f94692
validate:launcher: Do not try to set DISPLAY envvar to None
2016-11-02 12:48:49 -03:00
Edward Hervey
a29f4c390c
baseclasses: Also check for minimum number of video-tracks
...
Some scenarios might only be for video files and are meaningless for
audio-only files
2016-10-26 17:34:49 +02:00
Edward Hervey
8c20943a9d
validate: Blacklist failing hls tests
...
See https://bugzilla.gnome.org/show_bug.cgi?id=773159
2016-10-18 15:41:11 +02:00
Thibault Saunier
032fccd7af
meson: Fix installing configured files
2016-09-26 13:36:49 -03:00
Thibault Saunier
41be7bb2a9
validate:launcher: Minor fix in returncode check
2016-09-26 13:36:48 -03:00
Thibault Saunier
e376da4c17
validate:launcher: Do not use unset sent_eos variable
...
And rename class member to sent_eos_time as it is more accurate
2016-09-12 14:21:30 -03:00
Thibault Saunier
9c9fe14293
meson:validate:test: Properly set paths to run launcher based tests
...
Adding a --validate-tools-path option to the launcher, allowing
to pass it from meson.
2016-09-09 12:24:54 -03:00
Thibault Saunier
3ece6a065e
validate: launcher: Always clean all tests at the end
...
Making sure that if an exception of anything happens we will
properly clean all the tests, or at least try to.
2016-09-08 13:10:29 -03:00
Thibault Saunier
03548cd63d
validate:launcher: Allow specifying expected tests errors
...
In the future instead of blacklisting tests we should define
what error is expected, and this way when the bug is closed,
we will notice, also, it will allow us to check GstValidate
error reporting itself.
2016-09-08 13:10:29 -03:00
Thibault Saunier
2fff14e469
validate: Pass information about GstValidate execution over a socket
...
Instead of trying to parsing stdout, generate json messages and
send them over a socket so that gst-validate-launcher can properly
have informations about gst-validate subprocess execution.
2016-09-08 13:10:29 -03:00
Thibault Saunier
23203ca0bd
validate:launcher: Use a xunit reporter only when explicitely specified
2016-09-08 10:39:56 -03:00
Thibault Saunier
8973fa04eb
validate:launcher: Report error and exit when a testsuite could not load
...
Otherwise the user might end up seeing a lot of meaningless logs about
'removed' tests.
2016-09-08 10:39:56 -03:00
Thibault Saunier
f7f600e730
validate: launcher: Fix the condition to check if we need an http server
...
We could be checking if a string was in None
And use gs_string_assign when assigning the first string
without using printf like format.
2016-09-08 10:39:56 -03:00
Mathieu Duponchelle
ec1d8d7d8c
launcher: Add --dump-on-failure switch
...
When the test fails, it can be useful to have the log files dumped
to stdout.
https://bugzilla.gnome.org/show_bug.cgi?id=741092
2016-09-01 03:39:18 +02:00
Thibault Saunier
da327d8625
validate: Add support for Meson as alternative/parallel build system
...
https://github.com/mesonbuild/meson
2016-08-25 15:12:31 -03:00
Edward Hervey
fcff932e09
validate: Un-blacklist tests that are fixed
...
the bug reports to which they report have been closed and I can't make
them fail locally.
2016-08-13 16:56:18 +02:00
Edward Hervey
7d99e780a7
validate: Blacklist more ogg files
...
https://bugzilla.gnome.org/show_bug.cgi?id=769545
2016-08-13 15:39:18 +02:00
Edward Hervey
37771192a6
validate: Blacklist scrub_forward_seeking.op2b-mpeg2-wave_hd_mxf
...
See https://bugzilla.gnome.org/show_bug.cgi?id=764025
2016-08-12 12:33:30 +02:00
Thibault Saunier
5a745829dc
validate: Add jpeg as known format
2016-07-29 15:54:02 -04:00
Thibault Saunier
bcc7a1fcca
validate:launcher: Add h265, opus and vp9 as known formats
2016-07-15 08:56:02 -04:00