Commit graph

149 commits

Author SHA1 Message Date
Thibault Saunier
2f35acf558 validate:launcher: Enhance command printing when using a server
So it can be copy pasted and work
2017-06-20 15:46:15 -04:00
Thibault Saunier
335c14bf5b validate:launcher: Fix launching testsuite with relative paths 2017-06-16 17:36:34 -04:00
Thibault Saunier
9a45cd41a9 validate:launcher: Handle test that can't be run in parralel 2017-06-07 15:23:26 -04:00
Thibault Saunier
0e163ffae8 validate:launcher: Handle optional tests
For example RTSP tests might not be avalaible if gst-rtsp-server-example-uri is not avalaible
2017-06-07 11:17:53 -04:00
Thibault Saunier
0beb8647ce validate:launcher: Print logfiles when printing test result
It was often annoying to check wrong logs because the result are
not printed near the reference to logs.
2017-06-07 11:17:53 -04:00
Thibault Saunier
37de89e6db validate:launcher: Do not run scenarios that need prerolling on RTSP stream 2017-06-07 11:17:53 -04:00
Thibault Saunier
3c62c315a9 validate: Implement RTSP support 2017-06-07 11:17:53 -04:00
Thibault Saunier
02d716d6d8 validate:launcher: Handle not redirecting valgrind output 2017-06-01 16:57:50 -04:00
Edward Hervey
400d1f9326 validate: Implement Scenario.__repr__
Allows better debugging when looking at logs
2017-05-25 15:50:23 +02:00
Edward Hervey
9188968f5c validate: Add live-related features to scenarios and medias
Note: The notion of "live" here is in the *content* sense and not in the
GStreamer sense.
Ex:
  * A rtsp stream is always "live" in the GStreamer sense but might not always
    provide live content.
  * HLS/DASH streams are not "live" in the GStreamer sense but might
    provide "live" content.

Some scenarios might:
* require live content
* not be compatible with live content

This patch adds two new properties for scenarios:
* live_content_required (default False) for scenarios that can only work with
  live content.
* live_content_compatible (default False) for scenarios that can work with
  both live and non-live content.
This patch adds support for reading a "live" property from stream_info
2017-05-25 14:04:00 +02:00
Thibault Saunier
40a31b05b6 launcher: Allow using the base launcher as a test manager
No reason to force people to subclass it in simple cases.
2017-04-30 14:36:16 -03:00
Edward Hervey
325c1b37dd baseclasses: Add method to set a specific list of scenarios
https://bugzilla.gnome.org/show_bug.cgi?id=781314
2017-04-14 16:55:06 +02:00
Thibault Saunier
49271bc721 validate:launcher: Add information on media info files parsing failures 2017-02-27 12:12:38 -03:00
Thibault Saunier
fbde653a0c validate: launcher: Fix the way we retrieve command name
We are now using a list of args for subprocess so just using it is simple now
2017-02-27 12:12:38 -03:00
Thibault Saunier
9caee62bc6 validate: Fix call to decode() on a string 2017-02-27 12:12:38 -03:00
Thibault Saunier
f2b42cbaf8 validate:launcher: Various fixes to make the Test class directly usable
And the launcher installed with meson usable
2017-02-21 13:40:28 -03:00
Thibault Saunier
54a1fc8b30 validate:launcher: Take ValidateAction execution as a marker of test update 2017-02-17 15:29:41 -03:00
Thibault Saunier
40016b9956 validate:scenario: Report action done with the execution duration 2017-02-17 15:29:41 -03:00
Thibault Saunier
7773ca7765 validate:launcher: Add a way to specify a set of tests to run under the leak tracer
https://bugzilla.gnome.org/show_bug.cgi?id=767856
2017-02-10 15:56:11 -03:00
Thibault Saunier
addd76fc85 validate: Mark tests as SKIPPED when installation is missing a GStreamer plugin 2017-02-07 13:12:09 -03:00
Thibault Saunier
823ce95d70 validate:launcher: Add an option for user to modify timeout values
Allowing to expand the test timeout when running on slow platforms
2017-02-06 12:16:41 -03:00
Thibault Saunier
5d1d8e6971 validate:launcher: Verify that Gst supression file could be found 2017-02-03 11:02:49 -03:00
Thibault Saunier
de2cbda854 validate: Make sure to update children environment from current env
And minor fix in function call
2017-01-06 12:12:21 -03:00
Thibault Saunier
ca1f8a03e0 validate:launcher: Make the IPC server socket blocking
0.0 does not mean blocking in python3 and makes everything failling
on windows.
2017-01-03 16:26:20 -03:00
Thibault Saunier
f24e22446b validate: Properly kill subprocesses on windows 2017-01-03 16:24:19 -03:00
Thibault Saunier
f6d0636466 validate:launcher: Fix running on windows 2017-01-03 15:34:39 -03:00
Thibault Saunier
2863986ea3 validate: Respect active testers when listing tests 2017-01-03 14:52:38 -03:00
Thibault Saunier
0ee2147137 Revert "Revert "validate:launcher: Add an app handler for unit tests described in meson""
This reverts commit 5656e2a1b2.
2017-01-03 13:01:31 -03:00
Edward Hervey
5656e2a1b2 Revert "validate:launcher: Add an app handler for unit tests described in meson"
This reverts commit a4aa5c60bb.

Does not work outside of gst-build (i.e. validate won't work when used either
installed, in a prefix, or in gst-uninstalled).
2017-01-02 17:21:14 +01:00
Thibault Saunier
1bc6cb3615 validate:launcher: Take into account test duration when filtering them
Otherwise running -t 'some.*test' will run long tests (longer than hard
timeout) which is not what the user expect.
2016-12-23 14:59:03 -03:00
Thibault Saunier
d013e26cbc validate: Whitlist blacklisted test only when explicitely
Otherwise running -t '.*reverse.*' will also run blacklisted
tests which is probably not what use wants.
2016-12-22 10:29:18 -03:00
Thibault Saunier
5fc11cf389 launcher: Make --update-media-info check if generating frames info or not 2016-12-22 10:29:18 -03:00
Thibault Saunier
a4aa5c60bb validate:launcher: Add an app handler for unit tests described in meson
This way we can run all tests with the launcher which brings in many
features.

And add a testsuite for GStreamer unit tests.
2016-12-22 10:29:18 -03:00
Thibault Saunier
1f9c67e6ff validate:launcher: Allow specifying a timeout factor 2016-12-22 10:08:27 -03:00
Thibault Saunier
085ebf03ba validate:launcher: Print the iteration number when running forever 2016-12-22 10:08:26 -03:00
Thibault Saunier
d5de0b702c validate:launcher: Do not list tests on unneeded testers 2016-12-22 10:08:23 -03:00
Thibault Saunier
6504b9152c validate:launcher: Allow specifying several testsuite dirs 2016-12-22 10:08:21 -03:00
Thibault Saunier
26692e749c validate:launcher: Stop running test subprocesses in a shell
And instead properly use a list of argument for the subprocesses.
2016-12-22 10:07:58 -03:00
Thibault Saunier
a842f9d511 validate:launcher: Handle missing media info file 2016-11-30 14:07:04 -03:00
Thibault Saunier
b0784ffcb3 validate:launcher: Make sure that the IPC server is shutdown before closing the socket 2016-11-30 07:40:05 -03:00
Thibault Saunier
2959af0761 validate:launcher: Make sure to check string when verifying expected failures
The value can potentially be None and we should handle that
2016-11-29 14:47:35 -03:00
Thibault Saunier
acfdd04e01 validate:launcher: Allow specifying timeout as a known issue
And minor cleanups
2016-11-28 19:37:08 -03:00
Thibault Saunier
03453961e8 validate:launcher: Properly report stack trace as such in the xunit file 2016-11-28 18:40:00 -03:00
Thibault Saunier
aec8693258 validate:launcher: Remove now useless validatelog
We are now doing IPC to communicate with the launcher
so let it simply go to stdout.
2016-11-28 18:40:00 -03:00
Thibault Saunier
e132c11a95 validate:launcher: Handle checking bug status for expected failures 2016-11-28 18:40:00 -03:00
Thibault Saunier
2857eaf2ad validate: launcher: Allow checking if bugs linked to blacklist is fixed 2016-11-16 17:39:45 -03:00
Thibault Saunier
615fab620a validate:launcher: Consider wanted tests as whitelisted 2016-11-16 12:02:32 -03:00
Thibault Saunier
f5828b2b5c validate:launcher: Allow running the testsuite N number of times 2016-11-14 13:32:07 -03:00
Thibault Saunier
5452672e89 validate:launcher: Try to generate a backtrace on segfaults 2016-11-09 10:19:45 -03:00
Thibault Saunier
1e51aeb942 validate:launcher: Port to Python3
And sync logging.py with Pitivi version
2016-11-09 10:13:42 -03:00