Thibault Saunier
4fac7bf9fd
validate: launcher: Run rtsp tests against both V1 and V2
...
https://bugzilla.gnome.org/show_bug.cgi?id=781446
2017-09-22 16:31:56 -03:00
Thibault Saunier
e9862b9fda
validate:launcher: Allow disabling using the number of failed tests as exitcode
...
This is usefull on CI servers where the test results will be inspected
and the status of the build built from it.
2017-09-06 16:38:39 -03:00
Jimmy Ohn
bd0e8e410e
validate: launcher: Modify the order of the parser argument
...
Modify the order of the parser argument before setting dir_group
https://bugzilla.gnome.org/show_bug.cgi?id=786715
2017-08-28 17:01:39 -03:00
Thibault Saunier
4dc95f0fc2
launcher: Automatically disable output coloration if not supported
2017-08-26 10:52:25 -03:00
Thibault Saunier
598128fc0c
meson: Fix the way we set the testsuite version
2017-08-26 09:28:51 -03:00
Thibault Saunier
2c6c25d4bf
validate:launcher: Use the number of failed test as exit code
...
We used to always return 0, which was not right!
2017-08-18 11:37:28 -03:00
Thibault Saunier
cb99482b9a
launcher: Avoid exceptions when inspecting renders files
...
We were a bit to strict on the Exception types which lead to
the launcher failling itself when it shouldn't
2017-08-14 16:57:50 -03:00
Tim-Philipp Müller
6b661f394f
meson: fix a few meson warnings
...
WARNING: The variable(s) 'DATADIR', 'LIBDIR' in the input file
'subprojects/gst-devtools/validate/launcher/config.py.in' are not
present in the given configuration data
WARNING: Passed invalid keyword argument "scanobj_args". This will
become a hard error in the future.
WARNING: Keyword argument "install" defined multiple times. This
will be a an error in future Meson releases.
2017-08-10 14:35:09 +01:00
Edward Hervey
9ae20ee3e0
validate: Re-enable mxf op2b tests
...
https://bugzilla.gnome.org/show_bug.cgi?id=785119
2017-07-25 09:55:02 +02:00
Thibault Saunier
8ab723b153
validate:launcher:check: Make sure to register tests from the testsuite
...
Instead of having them listed from the app manager. This is needed
to avoid backtrace as tests now have to be register when setting up
the testsuite.
2017-07-19 11:49:54 -04:00
Thibault Saunier
1a28e7b043
validate: Factor out a method to set properties on elements in utils
...
Make sure to use it where appropriate and add some logging when
setting an object property from an action.
And use the valgrind.conf to set all the properties instead of having
a mixture of a config scenario and the config file (making sure the
max-lateness is set on any sink)
2017-07-19 11:34:19 -04:00
Edward Hervey
1cfe980044
validate: Blacklist op2b mxf files
...
See https://bugzilla.gnome.org/show_bug.cgi?id=785119
2017-07-19 15:47:28 +02:00
Thibault Saunier
2177d8589c
validate:launcher: Error out if no testsuite could be loaded
2017-07-18 12:09:13 -04:00
Thibault Saunier
2cf93f491b
validate: launcher: Namespace test name with the testsuite name
...
Also allowing users to pass test names directly
2017-07-18 10:54:53 -04:00
Thibault Saunier
f9ef2bc56a
validate: launcher: Properly use TestsLauncher.list_test to load tests
...
Otherwise we might skip check_defined_tests.
2017-07-18 10:54:21 -04:00
Sebastian Dröge
c8cd695bb9
validate: Un-blacklist scrub_forward_seeking.op2b-mpeg2-wave_hd_mxf
...
It works now after various mxfdemux changes.
https://bugzilla.gnome.org/show_bug.cgi?id=764025
2017-07-14 10:37:31 +03:00
Edward Hervey
ee72ae8913
gstvalidate: Re-allow tests that should be fixed
2017-07-13 08:46:05 +02:00
Thibault Saunier
40a08e6080
validate: launcher: Add some missing env variables in command to launch test
2017-07-11 10:11:49 -04:00
Thibault Saunier
c540601ed0
validate:launcher: Do not check ModuleNotFound exception
...
It is a subclass of ImportError and is avalaible only since 3.6
https://ci.gstreamer.net/job/pitivi-flatpak/626/console
2017-06-28 15:54:13 -04:00
Thibault Saunier
3b20a1ccde
validate:launcher: Disable seek with stop on RTSP streams
...
It is actually not supported
2017-06-28 13:02:33 -04:00
Thibault Saunier
831464d96f
validate:launcher: Avoid useless and expensive deep copies
2017-06-27 16:50:35 -04:00
Thibault Saunier
8ea68e9e06
validate:launcher: Speed up xml parsing using lxml if avalaible
2017-06-27 16:50:35 -04:00
Thibault Saunier
2f35acf558
validate:launcher: Enhance command printing when using a server
...
So it can be copy pasted and work
2017-06-20 15:46:15 -04:00
Thibault Saunier
82dd230ef1
validate:launcher: No need to use -validate as a tracer for RTSP server
...
It is now linked into the server app
2017-06-20 10:43:09 -04:00
Thibault Saunier
335c14bf5b
validate:launcher: Fix launching testsuite with relative paths
2017-06-16 17:36:34 -04:00
Thibault Saunier
543e7a983a
validate:launcher: Do not which(None), the rtsp-server command is unset if not avalaible
...
https://bugzilla.gnome.org/show_bug.cgi?id=783551
2017-06-08 12:18:19 -04:00
Thibault Saunier
9a45cd41a9
validate:launcher: Handle test that can't be run in parralel
2017-06-07 15:23:26 -04:00
Thibault Saunier
6772fa7d27
validate: Add 'our own' RTSP server implementation
2017-06-07 11:17:53 -04:00
Thibault Saunier
0e163ffae8
validate:launcher: Handle optional tests
...
For example RTSP tests might not be avalaible if gst-rtsp-server-example-uri is not avalaible
2017-06-07 11:17:53 -04:00
Thibault Saunier
0beb8647ce
validate:launcher: Print logfiles when printing test result
...
It was often annoying to check wrong logs because the result are
not printed near the reference to logs.
2017-06-07 11:17:53 -04:00
Thibault Saunier
37de89e6db
validate:launcher: Do not run scenarios that need prerolling on RTSP stream
2017-06-07 11:17:53 -04:00
Thibault Saunier
3c62c315a9
validate: Implement RTSP support
2017-06-07 11:17:53 -04:00
Thibault Saunier
4a38efb6a0
validate:launcher: Add missing is_live implementation for FakeMediaDescriptor
2017-06-01 16:57:50 -04:00
Thibault Saunier
02d716d6d8
validate:launcher: Handle not redirecting valgrind output
2017-06-01 16:57:50 -04:00
Thibault Saunier
dba5675d81
validate:launcher: Add a way to pass arguments to the leak tracer
2017-06-01 16:57:50 -04:00
Edward Hervey
400d1f9326
validate: Implement Scenario.__repr__
...
Allows better debugging when looking at logs
2017-05-25 15:50:23 +02:00
Edward Hervey
9188968f5c
validate: Add live-related features to scenarios and medias
...
Note: The notion of "live" here is in the *content* sense and not in the
GStreamer sense.
Ex:
* A rtsp stream is always "live" in the GStreamer sense but might not always
provide live content.
* HLS/DASH streams are not "live" in the GStreamer sense but might
provide "live" content.
Some scenarios might:
* require live content
* not be compatible with live content
This patch adds two new properties for scenarios:
* live_content_required (default False) for scenarios that can only work with
live content.
* live_content_compatible (default False) for scenarios that can work with
both live and non-live content.
This patch adds support for reading a "live" property from stream_info
2017-05-25 14:04:00 +02:00
Thibault Saunier
40a31b05b6
launcher: Allow using the base launcher as a test manager
...
No reason to force people to subclass it in simple cases.
2017-04-30 14:36:16 -03:00
Thibault Saunier
e9db1c19d0
launcher:check: Ensure to set build dir before rebuilding
2017-04-20 10:13:00 -03:00
Edward Hervey
325c1b37dd
baseclasses: Add method to set a specific list of scenarios
...
https://bugzilla.gnome.org/show_bug.cgi?id=781314
2017-04-14 16:55:06 +02:00
Edward Hervey
330ae663dc
validate: Improve video fakesink properties
...
Try to emulate a bit better a real video sink by making the video
fakesink handle/calculate/report QoS.
Also use the same lateness value as default videosink
2017-04-04 14:53:30 +02:00
Edward Hervey
99cade5ba4
validate: Make the HTTP server multi-threaded
...
Avoids having one test blocking all other tests
2017-03-02 17:40:18 +01:00
Thibault Saunier
49271bc721
validate:launcher: Add information on media info files parsing failures
2017-02-27 12:12:38 -03:00
Thibault Saunier
fbde653a0c
validate: launcher: Fix the way we retrieve command name
...
We are now using a list of args for subprocess so just using it is simple now
2017-02-27 12:12:38 -03:00
Thibault Saunier
bf21c2f64e
validate:launcher: Fix typo in meson build definitions
2017-02-27 12:12:38 -03:00
Thibault Saunier
9caee62bc6
validate: Fix call to decode() on a string
2017-02-27 12:12:38 -03:00
Thibault Saunier
b97c13d0a7
validate:launcher: Fix setting meson build dir
2017-02-24 17:33:08 -03:00
Thibault Saunier
f2b42cbaf8
validate:launcher: Various fixes to make the Test class directly usable
...
And the launcher installed with meson usable
2017-02-21 13:40:28 -03:00
Thibault Saunier
a39ef816a6
validate: launcher: Do not force using current module dir to run check tests
2017-02-21 13:40:28 -03:00
Thibault Saunier
54a1fc8b30
validate:launcher: Take ValidateAction execution as a marker of test update
2017-02-17 15:29:41 -03:00