mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2024-12-22 08:17:01 +00:00
Code Issues 1 Projects Releases Wiki Activity
2416 commits 44 branches 1510 tags 509 MiB
Commit graph

247 commits

Author SHA1 Message Date
Thibault Saunier
35003bc25b validate:launcher: Do not print passing tests if not running in a tty 2019-01-31 23:47:11 -03:00
Mathieu Duponchelle
d5e3f4d31c TestsManager: stop displaying blacklisted tests on stdout 
It's basically spam, better suited to the debug logs
 2019-01-30 00:27:32 +00:00
Thibault Saunier
440055214a validate:launcher: Never print lines larger than the terminal 2019-01-29 16:09:30 -03:00
Thibault Saunier
2551e8750e validate:launcher: Make baseclasses.py pep8 compliant 2019-01-26 15:54:00 +00:00
Thibault Saunier
9b69bcad08 validate:launcher: Handle launching launching a sub launcher 
If you use validate-launcher in a meson testsuite, those test now
gets integrated as one unique testsuite (with a pretty long namespace).
 2019-01-26 15:54:00 +00:00
Thibault Saunier
6665652cff validate:launcher: Refactor the "main" function 
- Move the parser code into a `LauncherConfig.create_parser()` method
- Remove the need to pass libsdir to the _TestsLauncher object
- Extract out a `setup_launcher_from_args` function
 2019-01-26 15:54:00 +00:00
Thibault Saunier
1b3867b82d launcher: Move http serveur and xvfb server to the main test runner object 
No good reason for it to be in the main function
 2019-01-26 15:54:00 +00:00
Thibault Saunier
7befe3d033 validate:launcher: Display unix nickname of signals leading to test failure 2019-01-15 16:55:32 -03:00
Thibault Saunier
5692f8d74d validate:launcher: Fix error message about 'crashed' test 2019-01-15 16:05:41 -03:00
Guillaume Desmottes
b1f22e7130 validate: fix crash if timeout when media_descriptor is None 
Some tests may not have any media_descriptor. If those were failing to
shutdown after EOS we were calling get_protocol() on None.
 2018-12-17 10:34:43 +01:00
Thibault Saunier
72995d5bbe validate: launcher: Add a way to retrieve trace without coredumpctl 
Simply spnning on segfaults (like gst-launch) and catch that in
the launcher to transform the timeout into a segfault and grab a gdb
backtrace
 2018-12-07 09:06:50 -03:00
Thibault Saunier
64afff0c3e launcher:scenario: Fix the way we compute scenario path/name when paths supplied 
We were just iterating over the list without any check and could end
up with a patch that was not corresponding to the actual scenario.
 2018-09-08 11:13:19 -03:00
Thibault Saunier
f0cfdf9d14 validate:launcher: Use fakevideosink everywhere it makes sense. 2018-09-07 16:19:40 -03:00
Alicia Boya García
5ff299f4e7 gst-validate-launcher: Print copypaste-friendlier commands 
This patch removes the quotes surrounding the command shown by
gst-validate to reproduce the issues -- which were troublesome when
copying and pasting.

It also introduces escaping for the arguments, so that the command line
can be copied and pasted in the terminal without further changes.

https://bugzilla.gnome.org/show_bug.cgi?id=796897
 2018-07-30 16:13:25 -04:00
Thibault Saunier
615a372274 validate: launcher: Print some ERROR log when inspecting scenario fails 2018-07-21 11:43:40 -04:00
Thibault Saunier
4f009f4098 validate:launcher: Minor output string message 2018-07-12 19:13:21 -04:00
Thibault Saunier
69ec48feef validate:launcher: Allow retrieving coredumps from within flatpak 2018-07-01 12:55:06 -04:00
Thibault Saunier
acb3a276c8 validate:launcher: Fix the --forever switch 
It was not stopping on error.

https://bugzilla.gnome.org/show_bug.cgi?id=796608
 2018-06-17 08:45:23 -04:00
Thibault Saunier
58c90448ca validate: launcher: Add support for running tests with a pushfile source 
Introducing the `.media_info.push` media info extension, which is meant
to let the launcher know that those file should run with the "pushfile://"
protocol.

And allow symlinking "normal" `.media_info` to their `.pushfile` variant
so that both can share the exact same content.
 2018-06-15 12:05:14 -04:00
Thibault Saunier
b2e71e1404 validate: media-check: Add a way to skip pluggin parsers 
This is useful when you want to check only the demuxer output.

- Keep the information in the media file so that we can launch media-check
  with the proper arguments in the launcher. Update it accordingly.
- Refactor compare_streams to simplify it, which in the end leads to
  reporting all the issues instead of exiting on the first one.
 2018-06-15 12:05:14 -04:00
Alicia Boya García
40dfb7174e gst-validate-launcher: let gdb handle SIGINT itself 
Otherwise both gdb and gst-validate-launcher will react to ^C at the
same time, gdb will be killed by SIGHUP (because gst-validate-launcher
quitted in consequence of the ^C) and the terminal state will be left
garbled because readline inside gdb had disabled echo.

https://bugzilla.gnome.org/show_bug.cgi?id=796396
 2018-05-25 12:59:41 +02:00
Alicia Boya García
bfa143caa9 gst-validate-launcher: disable timeouts when debugging in gdb interactively 
An interactive debugging session can be going for a long time, we don't
want any timeouts in that case.

https://bugzilla.gnome.org/show_bug.cgi?id=796397
 2018-05-25 12:59:40 +02:00
Alicia Boya García
2c32ccd7a7 gst-validate-launcher: Stop in --gdb by default, add --gdb-non-stop 
This patch modifies the default behavior of --gdb to not run and quit
automatically the test, but rather wait for user input. This is
usually much more convenient to debug all kinds of bugs.

The automatic run behavior has been moved to a new command switch:
--gdb-non-stop

https://bugzilla.gnome.org/show_bug.cgi?id=796389
 2018-05-25 12:59:38 +02:00
Thibault Saunier
bb71fd9944 validate:launcher: Error out loudly if the testlist changes 
When --fail-on-testlist-change is set.
 2018-05-15 14:45:54 -04:00
Thibault Saunier
2dc7dd1257 validate: launcher: Make sure testsuites are used/configured once only 2018-05-13 19:10:59 -04:00
Thibault Saunier
ef9ff93405 validate:launcher: Add a way to check if a gst feature is present 
And make sure iqa is present to run IQA tests.
 2018-05-13 16:35:41 -04:00
Thibault Saunier
b00ab02254 validate:launcher: Cleanup the way we find where -validate tools are 2018-05-13 09:44:17 -04:00
Thibault Saunier
c1f89b4acb validate: launcher: Add a way to simply run SSIM checks on rendered files 
We will run a simple pipeline with the IQA element to run ssim (dssim)
tests on the rendered files, comparing it with a reference file.

For now we use the very empiric 1.0 value as a ssim error threshold and
the goal is basically to detect completely broken renderings.
 2018-05-13 09:44:17 -04:00
Thibault Saunier
77ee198b1b validate:launcher: Add support for specifying a workdir in tests 2018-04-15 20:50:22 -03:00
Thibault Saunier
8382fddbe9 validate:launcher: Add a method to find tests in a TestManager 2018-04-12 23:30:19 -03:00
Thibault Saunier
55dee0b8a6 validate:launcher: Extract method to set a validate configuration on tests 2018-04-12 23:07:16 -03:00
Thibault Saunier
a70658a464 launcher: Print test number in the order they finish 
Instead of the test index in the list of tests as it is
meaningless to the user and feels weird.

Also minor fix in the test name display when running with --forever.
 2018-03-23 21:04:46 -03:00
Thibault Saunier
cd1c4eb44d launcher: Minor indentation issue fixes 2018-03-23 18:12:16 -03:00
Thibault Saunier
2b40c6c306 launcher: Clean up outpout 
Make our stdout output simpler to follow by:
  - Not printing the tests we launch (it is not really useful in the end)
  - Using `\r` when printing the passed tests
  - Not reprinting all the test in a now useless summary
 2018-03-23 18:12:16 -03:00
Edward Hervey
d71d28523d validate/baseclasses: Release un-needed data when test ends 
This was keeping around 500-700kB of data for each test, which was
gradually raising memory usage of a full run by 100MB+

The reports are definitely not needed, and we only need to keep
information from the subprocess env variable that we might need
later on for final reporting
 2017-12-18 10:51:05 +01:00
Edward Hervey
cb04515cbd validate/baseclasses: Don't leak several hundred MB of XML 
The xml-based MediaDescriptor were keeping open the XML file and the
associated ElementTree structures, resulting in memory usage of several
hundred megabytes.

Instead cache the information we need immediately and release the
XML structure
 2017-12-18 09:48:21 +01:00
Edward Hervey
92285ef261 validate: Use a single TCPServer for subprocess communication 
Instead of creating a separate TCPServer for each test, just create
one which handles all connections in a threaded fashion.

Shaves off ~500ms per test

https://bugzilla.gnome.org/show_bug.cgi?id=791159
 2017-12-03 12:23:29 +01:00
Edward Hervey
58e62f651c validate-launcher: Allow running tests out-of-order 
When the --shuffle option is used, the tests will be run out of order.
This optimizes CPU utilization since it allows running synchronized
and unsynchronized tests at the same.
 2017-12-03 11:42:18 +01:00
Thibault Saunier
f16b900643 validate:launcher: Launch tests in _TestsLauncher not in TestsManagaer 
So that Test from several TestManager can run in parallel and thus avoid
waiting for tests from one TestManager to run the following one.,

Also by design TestsLauncher should always have been the responsible for
... launching tests.
 2017-12-02 09:42:19 -03:00
Thibault Saunier
2c6c25d4bf validate:launcher: Use the number of failed test as exit code 
We used to always return 0, which was not right!
 2017-08-18 11:37:28 -03:00
Thibault Saunier
cb99482b9a launcher: Avoid exceptions when inspecting renders files 
We were a bit to strict on the Exception types which lead to
the launcher failling itself when it shouldn't
 2017-08-14 16:57:50 -03:00
Thibault Saunier
1a28e7b043 validate: Factor out a method to set properties on elements in utils 
Make sure to use it where appropriate and add some logging when
setting an object property from an action.

And use the valgrind.conf to set all the properties instead of having
a mixture of a config scenario and the config file (making sure the
max-lateness is set on any sink)
 2017-07-19 11:34:19 -04:00
Thibault Saunier
2177d8589c validate:launcher: Error out if no testsuite could be loaded 2017-07-18 12:09:13 -04:00
Thibault Saunier
2cf93f491b validate: launcher: Namespace test name with the testsuite name 
Also allowing users to pass test names directly
 2017-07-18 10:54:53 -04:00
Thibault Saunier
f9ef2bc56a validate: launcher: Properly use TestsLauncher.list_test to load tests 
Otherwise we might skip check_defined_tests.
 2017-07-18 10:54:21 -04:00
Thibault Saunier
40a08e6080 validate: launcher: Add some missing env variables in command to launch test 2017-07-11 10:11:49 -04:00
Thibault Saunier
c540601ed0 validate:launcher: Do not check ModuleNotFound exception 
It is a subclass of ImportError and is avalaible only since 3.6
https://ci.gstreamer.net/job/pitivi-flatpak/626/console
 2017-06-28 15:54:13 -04:00
Thibault Saunier
8ea68e9e06 validate:launcher: Speed up xml parsing using lxml if avalaible 2017-06-27 16:50:35 -04:00
Thibault Saunier
2f35acf558 validate:launcher: Enhance command printing when using a server 
So it can be copy pasted and work
 2017-06-20 15:46:15 -04:00
Thibault Saunier
335c14bf5b validate:launcher: Fix launching testsuite with relative paths 2017-06-16 17:36:34 -04:00
Thibault Saunier
9a45cd41a9 validate:launcher: Handle test that can't be run in parralel 2017-06-07 15:23:26 -04:00
Thibault Saunier
0e163ffae8 validate:launcher: Handle optional tests 
For example RTSP tests might not be avalaible if gst-rtsp-server-example-uri is not avalaible
 2017-06-07 11:17:53 -04:00
Thibault Saunier
0beb8647ce validate:launcher: Print logfiles when printing test result 
It was often annoying to check wrong logs because the result are
not printed near the reference to logs.
 2017-06-07 11:17:53 -04:00
Thibault Saunier
37de89e6db validate:launcher: Do not run scenarios that need prerolling on RTSP stream 2017-06-07 11:17:53 -04:00
Thibault Saunier
3c62c315a9 validate: Implement RTSP support 2017-06-07 11:17:53 -04:00
Thibault Saunier
02d716d6d8 validate:launcher: Handle not redirecting valgrind output 2017-06-01 16:57:50 -04:00
Edward Hervey
400d1f9326 validate: Implement Scenario.__repr__ 
Allows better debugging when looking at logs
 2017-05-25 15:50:23 +02:00
Edward Hervey
9188968f5c validate: Add live-related features to scenarios and medias 
Note: The notion of "live" here is in the *content* sense and not in the
GStreamer sense.
Ex:
  * A rtsp stream is always "live" in the GStreamer sense but might not always
    provide live content.
  * HLS/DASH streams are not "live" in the GStreamer sense but might
    provide "live" content.

Some scenarios might:
* require live content
* not be compatible with live content

This patch adds two new properties for scenarios:
* live_content_required (default False) for scenarios that can only work with
  live content.
* live_content_compatible (default False) for scenarios that can work with
  both live and non-live content.
This patch adds support for reading a "live" property from stream_info
 2017-05-25 14:04:00 +02:00
Thibault Saunier
40a31b05b6 launcher: Allow using the base launcher as a test manager 
No reason to force people to subclass it in simple cases.
 2017-04-30 14:36:16 -03:00
Edward Hervey
325c1b37dd baseclasses: Add method to set a specific list of scenarios 
https://bugzilla.gnome.org/show_bug.cgi?id=781314
 2017-04-14 16:55:06 +02:00
Thibault Saunier
49271bc721 validate:launcher: Add information on media info files parsing failures 2017-02-27 12:12:38 -03:00
Thibault Saunier
fbde653a0c validate: launcher: Fix the way we retrieve command name 
We are now using a list of args for subprocess so just using it is simple now
 2017-02-27 12:12:38 -03:00
Thibault Saunier
9caee62bc6 validate: Fix call to decode() on a string 2017-02-27 12:12:38 -03:00
Thibault Saunier
f2b42cbaf8 validate:launcher: Various fixes to make the Test class directly usable 
And the launcher installed with meson usable
 2017-02-21 13:40:28 -03:00
Thibault Saunier
54a1fc8b30 validate:launcher: Take ValidateAction execution as a marker of test update 2017-02-17 15:29:41 -03:00
Thibault Saunier
40016b9956 validate:scenario: Report action done with the execution duration 2017-02-17 15:29:41 -03:00
Thibault Saunier
7773ca7765 validate:launcher: Add a way to specify a set of tests to run under the leak tracer 
https://bugzilla.gnome.org/show_bug.cgi?id=767856
 2017-02-10 15:56:11 -03:00
Thibault Saunier
addd76fc85 validate: Mark tests as SKIPPED when installation is missing a GStreamer plugin 2017-02-07 13:12:09 -03:00
Thibault Saunier
823ce95d70 validate:launcher: Add an option for user to modify timeout values 
Allowing to expand the test timeout when running on slow platforms
 2017-02-06 12:16:41 -03:00
Thibault Saunier
5d1d8e6971 validate:launcher: Verify that Gst supression file could be found 2017-02-03 11:02:49 -03:00
Thibault Saunier
de2cbda854 validate: Make sure to update children environment from current env 
And minor fix in function call
 2017-01-06 12:12:21 -03:00
Thibault Saunier
ca1f8a03e0 validate:launcher: Make the IPC server socket blocking 
0.0 does not mean blocking in python3 and makes everything failling
on windows.
 2017-01-03 16:26:20 -03:00
Thibault Saunier
f24e22446b validate: Properly kill subprocesses on windows 2017-01-03 16:24:19 -03:00
Thibault Saunier
f6d0636466 validate:launcher: Fix running on windows 2017-01-03 15:34:39 -03:00
Thibault Saunier
2863986ea3 validate: Respect active testers when listing tests 2017-01-03 14:52:38 -03:00
Thibault Saunier
0ee2147137 Revert "Revert "validate:launcher: Add an app handler for unit tests described in meson"" 
This reverts commit 5656e2a1b2.
 2017-01-03 13:01:31 -03:00
Edward Hervey
5656e2a1b2 Revert "validate:launcher: Add an app handler for unit tests described in meson" 
This reverts commit a4aa5c60bb.

Does not work outside of gst-build (i.e. validate won't work when used either
installed, in a prefix, or in gst-uninstalled).
 2017-01-02 17:21:14 +01:00
Thibault Saunier
1bc6cb3615 validate:launcher: Take into account test duration when filtering them 
Otherwise running -t 'some.*test' will run long tests (longer than hard
timeout) which is not what the user expect.
 2016-12-23 14:59:03 -03:00
Thibault Saunier
d013e26cbc validate: Whitlist blacklisted test only when explicitely 
Otherwise running -t '.*reverse.*' will also run blacklisted
tests which is probably not what use wants.
 2016-12-22 10:29:18 -03:00
Thibault Saunier
5fc11cf389 launcher: Make --update-media-info check if generating frames info or not 2016-12-22 10:29:18 -03:00
Thibault Saunier
a4aa5c60bb validate:launcher: Add an app handler for unit tests described in meson 
This way we can run all tests with the launcher which brings in many
features.

And add a testsuite for GStreamer unit tests.
 2016-12-22 10:29:18 -03:00
Thibault Saunier
1f9c67e6ff validate:launcher: Allow specifying a timeout factor 2016-12-22 10:08:27 -03:00
Thibault Saunier
085ebf03ba validate:launcher: Print the iteration number when running forever 2016-12-22 10:08:26 -03:00
Thibault Saunier
d5de0b702c validate:launcher: Do not list tests on unneeded testers 2016-12-22 10:08:23 -03:00
Thibault Saunier
6504b9152c validate:launcher: Allow specifying several testsuite dirs 2016-12-22 10:08:21 -03:00
Thibault Saunier
26692e749c validate:launcher: Stop running test subprocesses in a shell 
And instead properly use a list of argument for the subprocesses.
 2016-12-22 10:07:58 -03:00
Thibault Saunier
a842f9d511 validate:launcher: Handle missing media info file 2016-11-30 14:07:04 -03:00
Thibault Saunier
b0784ffcb3 validate:launcher: Make sure that the IPC server is shutdown before closing the socket 2016-11-30 07:40:05 -03:00
Thibault Saunier
2959af0761 validate:launcher: Make sure to check string when verifying expected failures 
The value can potentially be None and we should handle that
 2016-11-29 14:47:35 -03:00
Thibault Saunier
acfdd04e01 validate:launcher: Allow specifying timeout as a known issue 
And minor cleanups
 2016-11-28 19:37:08 -03:00
Thibault Saunier
03453961e8 validate:launcher: Properly report stack trace as such in the xunit file 2016-11-28 18:40:00 -03:00
Thibault Saunier
aec8693258 validate:launcher: Remove now useless validatelog 
We are now doing IPC to communicate with the launcher
so let it simply go to stdout.
 2016-11-28 18:40:00 -03:00
Thibault Saunier
e132c11a95 validate:launcher: Handle checking bug status for expected failures 2016-11-28 18:40:00 -03:00
Thibault Saunier
2857eaf2ad validate: launcher: Allow checking if bugs linked to blacklist is fixed 2016-11-16 17:39:45 -03:00
Thibault Saunier
615fab620a validate:launcher: Consider wanted tests as whitelisted 2016-11-16 12:02:32 -03:00
Thibault Saunier
f5828b2b5c validate:launcher: Allow running the testsuite N number of times 2016-11-14 13:32:07 -03:00
Thibault Saunier
5452672e89 validate:launcher: Try to generate a backtrace on segfaults 2016-11-09 10:19:45 -03:00
Thibault Saunier
1e51aeb942 validate:launcher: Port to Python3 
And sync logging.py with Pitivi version
 2016-11-09 10:13:42 -03:00
Edward Hervey
a29f4c390c baseclasses: Also check for minimum number of video-tracks 
Some scenarios might only be for video files and are meaningless for
audio-only files
 2016-10-26 17:34:49 +02:00
Thibault Saunier
41be7bb2a9 validate:launcher: Minor fix in returncode check 2016-09-26 13:36:48 -03:00