mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2024-10-01 16:22:25 +00:00
Code Issues 1 Projects Releases Wiki Activity
2350 commits 44 branches 1508 tags 318 MiB
Commit graph

402 commits

Author SHA1 Message Date
Thibault Saunier
d013e26cbc validate: Whitlist blacklisted test only when explicitely 
Otherwise running -t '.*reverse.*' will also run blacklisted
tests which is probably not what use wants.
 2016-12-22 10:29:18 -03:00
Thibault Saunier
5fc11cf389 launcher: Make --update-media-info check if generating frames info or not 2016-12-22 10:29:18 -03:00
Thibault Saunier
a4aa5c60bb validate:launcher: Add an app handler for unit tests described in meson 
This way we can run all tests with the launcher which brings in many
features.

And add a testsuite for GStreamer unit tests.
 2016-12-22 10:29:18 -03:00
Thibault Saunier
1f9c67e6ff validate:launcher: Allow specifying a timeout factor 2016-12-22 10:08:27 -03:00
Thibault Saunier
085ebf03ba validate:launcher: Print the iteration number when running forever 2016-12-22 10:08:26 -03:00
Thibault Saunier
58711edce8 validate:launcher: Add a -v option to print subprocesses to stdout 2016-12-22 10:08:25 -03:00
Thibault Saunier
076c21fbe5 validate:launcher: Run cpu_count test in parallel by default 2016-12-22 10:08:24 -03:00
Thibault Saunier
d5de0b702c validate:launcher: Do not list tests on unneeded testers 2016-12-22 10:08:23 -03:00
Thibault Saunier
6504b9152c validate:launcher: Allow specifying several testsuite dirs 2016-12-22 10:08:21 -03:00
Thibault Saunier
26692e749c validate:launcher: Stop running test subprocesses in a shell 
And instead properly use a list of argument for the subprocesses.
 2016-12-22 10:07:58 -03:00
Thibault Saunier
e552c18c80 launcher: Handle stack trace information as jenkins expect it in the xunit file 2016-12-01 10:52:32 -03:00
Thibault Saunier
a842f9d511 validate:launcher: Handle missing media info file 2016-11-30 14:07:04 -03:00
Thibault Saunier
b0784ffcb3 validate:launcher: Make sure that the IPC server is shutdown before closing the socket 2016-11-30 07:40:05 -03:00
Thibault Saunier
2959af0761 validate:launcher: Make sure to check string when verifying expected failures 
The value can potentially be None and we should handle that
 2016-11-29 14:47:35 -03:00
Thibault Saunier
acfdd04e01 validate:launcher: Allow specifying timeout as a known issue 
And minor cleanups
 2016-11-28 19:37:08 -03:00
Thibault Saunier
03453961e8 validate:launcher: Properly report stack trace as such in the xunit file 2016-11-28 18:40:00 -03:00
Thibault Saunier
aec8693258 validate:launcher: Remove now useless validatelog 
We are now doing IPC to communicate with the launcher
so let it simply go to stdout.
 2016-11-28 18:40:00 -03:00
Thibault Saunier
e132c11a95 validate:launcher: Handle checking bug status for expected failures 2016-11-28 18:40:00 -03:00
Thibault Saunier
e3731ba839 validate: Minor documentation fixes. 2016-11-17 15:43:15 -03:00
Thibault Saunier
2857eaf2ad validate: launcher: Allow checking if bugs linked to blacklist is fixed 2016-11-16 17:39:45 -03:00
Thibault Saunier
615fab620a validate:launcher: Consider wanted tests as whitelisted 2016-11-16 12:02:32 -03:00
Thibault Saunier
8bd0cb13e8 validate:launcher: Timeout if running gdb takes too much time 2016-11-14 18:44:27 -03:00
Thibault Saunier
f5828b2b5c validate:launcher: Allow running the testsuite N number of times 2016-11-14 13:32:07 -03:00
Thibault Saunier
20879e9c69 validate:launcher: Fix usage in an uninstalled environment 2016-11-09 17:38:04 -03:00
Thibault Saunier
5452672e89 validate:launcher: Try to generate a backtrace on segfaults 2016-11-09 10:19:45 -03:00
Thibault Saunier
1e51aeb942 validate:launcher: Port to Python3 
And sync logging.py with Pitivi version
 2016-11-09 10:13:42 -03:00
Thibault Saunier
d797f94692 validate:launcher: Do not try to set DISPLAY envvar to None 2016-11-02 12:48:49 -03:00
Edward Hervey
a29f4c390c baseclasses: Also check for minimum number of video-tracks 
Some scenarios might only be for video files and are meaningless for
audio-only files
 2016-10-26 17:34:49 +02:00
Edward Hervey
8c20943a9d validate: Blacklist failing hls tests 
See https://bugzilla.gnome.org/show_bug.cgi?id=773159
 2016-10-18 15:41:11 +02:00
Thibault Saunier
032fccd7af meson: Fix installing configured files 2016-09-26 13:36:49 -03:00
Thibault Saunier
41be7bb2a9 validate:launcher: Minor fix in returncode check 2016-09-26 13:36:48 -03:00
Thibault Saunier
e376da4c17 validate:launcher: Do not use unset sent_eos variable 
And rename class member to sent_eos_time as it is more accurate
 2016-09-12 14:21:30 -03:00
Thibault Saunier
9c9fe14293 meson:validate:test: Properly set paths to run launcher based tests 
Adding a --validate-tools-path option to the launcher, allowing
to pass it from meson.
 2016-09-09 12:24:54 -03:00
Thibault Saunier
3ece6a065e validate: launcher: Always clean all tests at the end 
Making sure that if an exception of anything happens we will
properly clean all the tests, or at least try to.
 2016-09-08 13:10:29 -03:00
Thibault Saunier
03548cd63d validate:launcher: Allow specifying expected tests errors 
In the future instead of blacklisting tests we should define
what error is expected, and this way when the bug is closed,
we will notice, also, it will allow us to check GstValidate
error reporting itself.
 2016-09-08 13:10:29 -03:00
Thibault Saunier
2fff14e469 validate: Pass information about GstValidate execution over a socket 
Instead of trying to parsing stdout, generate json messages and
send them over a socket so that gst-validate-launcher can properly
have informations about gst-validate subprocess execution.
 2016-09-08 13:10:29 -03:00
Thibault Saunier
23203ca0bd validate:launcher: Use a xunit reporter only when explicitely specified 2016-09-08 10:39:56 -03:00
Thibault Saunier
8973fa04eb validate:launcher: Report error and exit when a testsuite could not load 
Otherwise the user might end up seeing a lot of meaningless logs about
'removed' tests.
 2016-09-08 10:39:56 -03:00
Thibault Saunier
f7f600e730 validate: launcher: Fix the condition to check if we need an http server 
We could be checking if a string was in None

And use gs_string_assign when assigning the first string
without using printf like format.
 2016-09-08 10:39:56 -03:00
Mathieu Duponchelle
ec1d8d7d8c launcher: Add --dump-on-failure switch 
When the test fails, it can be useful to have the log files dumped
to stdout.

https://bugzilla.gnome.org/show_bug.cgi?id=741092
 2016-09-01 03:39:18 +02:00
Thibault Saunier
da327d8625 validate: Add support for Meson as alternative/parallel build system 
https://github.com/mesonbuild/meson
 2016-08-25 15:12:31 -03:00
Edward Hervey
fcff932e09 validate: Un-blacklist tests that are fixed 
the bug reports to which they report have been closed and I can't make
them fail locally.
 2016-08-13 16:56:18 +02:00
Edward Hervey
7d99e780a7 validate: Blacklist more ogg files 
https://bugzilla.gnome.org/show_bug.cgi?id=769545
 2016-08-13 15:39:18 +02:00
Edward Hervey
37771192a6 validate: Blacklist scrub_forward_seeking.op2b-mpeg2-wave_hd_mxf 
See https://bugzilla.gnome.org/show_bug.cgi?id=764025
 2016-08-12 12:33:30 +02:00
Thibault Saunier
5a745829dc validate: Add jpeg as known format 2016-07-29 15:54:02 -04:00
Thibault Saunier
bcc7a1fcca validate:launcher: Add h265, opus and vp9 as known formats 2016-07-15 08:56:02 -04:00
Edward Hervey
1e1475d352 validate: Fix usage for non-standard http server port 
Translate the various stream_info URI from the standard port to the
specified port
Make the port option an integer option
 2016-07-04 16:19:09 +02:00
Edward Hervey
b77797da4b validate: Use decodebin3/uridecodebin3 when specified 
Set the USE_PLAYBIN3 environment variable to use those elements instead
of legacy ones.
 2016-06-30 14:58:03 +02:00
Reynaldo H. Verdejo Pinochet
07fa1b3ca5 validate: fix some recurring typos 2016-06-09 14:31:48 -07:00
Vineeth TM
99f9f3f408 validate:launcher: Add support for relative path while providing file path 
Instead of providing full absolute path while validating the file, should be
able to provide the relative path with respect to the present directory.

https://bugzilla.gnome.org/show_bug.cgi?id=753494
 2016-06-04 12:51:35 +01:00
Guillaume Desmottes
2ddbcb666f validate: generate valgrind suppression traces 
Makes fixing easier as then we can just re-use the generated trace.

Reviewed-by: Thibault Saunier <tsaunier@gnome.org>
Differential Revision: https://phabricator.freedesktop.org/D953
 2016-05-24 10:28:26 -04:00
Guillaume Desmottes
4322138c8f validate: fix typo if Xvfb is not installed 
Reviewed-by: Thibault Saunier <tsaunier@gnome.org>
Differential Revision: https://phabricator.freedesktop.org/D375
 2016-05-24 10:28:23 -04:00
Thiago Santos
f4cdbd006b launcher: set gsettings-backend to prevent weird deadlocks 
It seems like some sort of forking/dconf/gtype combination can
deadlock occasionally. Setting the gsettings backend to memory
makes it go away.

Same issue: https://cgit.freedesktop.org/gstreamer/gst-plugins-good/commit/tests/check/Makefile.am?id=8e2c1d1de56bddbff22170f8b17473882e0e63f9
 2016-05-15 06:34:21 -03:00
Thiago Santos
d0a7d9b094 validate: add non-seekable fragment file seek tests to the blacklist 
The file has no index and is not easily seekable, seeking in these
kind of files isn't implemented.
 2016-05-06 21:27:53 -03:00
Thibault Saunier
83c652ba76 validate: launcher: Make sure to properly setup all testsuites 
When a first testsuite will set paths, it does not mean that we should
just register following testsuite test manager default tests.

So we need to make a difference between the media paths the user passed
with --media-path and the ones defined by the testsuite.
 2016-03-23 20:10:51 +01:00
Thibault Saunier
9190bcf62a validate:launcher: Handle testslist files even running several testsuites 
Only if those testsuites do not use the same Tester as we
currently can't know to what testsuite a test belongs.
 2016-03-23 20:10:01 +01:00
Thibault Saunier
187df093eb launcher: Avoid caching all the debug logs in memory 
And just write the temporary XML file on disc
 2016-03-23 12:41:49 +01:00
Thibault Saunier
23ad66c552 validate: launcher: Blacklist dash and HLS failling seeking tests 
As described in https://bugzilla.gnome.org/show_bug.cgi?id=764020
 2016-03-22 19:02:27 +01:00
Thibault Saunier
a566b2c4b9 validate:testsuite: Blacklist validate.hls.playback.scrub_forward_seeking.hls_bibbop as it is racy 2016-03-22 17:46:50 +01:00
Thibault Saunier
621bda9b62 validate: Use GstValidate logging system to print buffering avancement 
So it can be used in the launcher.
And make sure to properly parse the info in the launcher.
 2016-03-21 14:34:27 +01:00
Thibault Saunier
9b93cc884f validate:launcher: Remove dependency on wget 2016-03-21 14:09:26 +01:00
Vineeth T M
65560d8b03 Validate: Fix consider, launch spelling mistakes 
https://bugzilla.gnome.org/show_bug.cgi?id=763289
 2016-03-08 09:28:11 +02:00
Vineeth T M
7d9e9b3b79 validate: Fix wrong condition check when adding tests 
When listing tests, checking whether uri is present or not and displaying error.
But uri does notneed to be present in case of pipeline generator. So the condition check is wrong.
This results in validateelements testsuite not working. Hence modifying the condition to
not error out on valid cases.

https://bugzilla.gnome.org/show_bug.cgi?id=762422
 2016-03-05 12:56:14 +01:00
Thibault Saunier
9708c7eb19 validate:launcher: Strip os.pathsep from extra env variables 
We might be working with something that is not a path
 2016-03-01 15:10:15 +01:00
Thibault Saunier
39a0be8e0e validate:launcher: Add a way to fail if test have been removed/added 2016-02-23 11:47:15 +01:00
Vineeth T M
bc66079d2a validate:launcher: handle spaces in transcode output path 
When there are spaces in transcoding output path, then it fails. Hence adding the
path in double quotes

https://bugzilla.gnome.org/show_bug.cgi?id=756217
 2016-02-17 22:33:32 +01:00
Vineeth T M
e2d2a6865f validate:launcher: Add proper check for is_seekable 
The seekable variable in media_info file is of type string. When checking if the file
is seekable using is_seekable, it just returns the string, resulting in it always being true.
It should actually be comparing the string and returning true or false based on comparison

https://bugzilla.gnome.org/show_bug.cgi?id=755854
 2016-02-17 22:33:16 +01:00
Thibault Saunier
ade0fb3fed validate: launcher: Set scenario manager config before discovering testsuites 2015-11-16 16:45:13 +01:00
Vineeth T M
0f7105d317 validate: launcher: Print error when media-info files not present 
When there are no media-info files present and --generate-media-info
option is not given, then it just fails without printing error.
Printing an error stating, use --generate-media-info if there are no
media info files.
When there are neither media files and media info files, print error
stating the same

https://bugzilla.gnome.org/show_bug.cgi?id=755087
 2015-11-11 10:12:58 +01:00
Thibault Saunier
8899fd8dd1 validate: Implement support to run tests inside gdb 
Making debugging races leading to crashes easier to debug
 2015-11-10 18:17:54 +01:00
Sebastian Dröge
3bd31854a8 validate: Un-blacklist seeking HLS tests 
They are reliable enough after 36b80edb72
in gst-plugins-base now.
 2015-10-27 15:51:44 +02:00
Thibault Saunier
aef41ba72a validate: Add support for prores 2015-10-14 12:01:15 +01:00
Vineeth TM
aa2c93c3d4 validate:launcher: throw valgrind error only for definite loss 
errors-for-leak-kinds should be set to definite, because almost every test case
, will have possibly lost memory, which may or may not be a leak.
And throwing error for all these cases doesn't seem to be correct.

https://bugzilla.gnome.org/show_bug.cgi?id=752754
 2015-10-02 17:46:51 +02:00
Vineeth T M
e7b65fe5df validate: launcher: Fix media_check class name and add double quotes for valgrind logs 
When creating the class names for media check, uri is being used,
instead of the path. Hence converting the uri using uri2path and creating
class name.
Add double quotes for valgrind logs, to support special characters like space

https://bugzilla.gnome.org/show_bug.cgi?id=752808
 2015-10-02 17:43:50 +02:00
Vineeth T M
1b700f13b4 validate: launcher: Support relative path for folder names 
Even though relative paths are supported, right now it does not
work when we give the path as 'media/' present in the current directory.
Adding support for the same.

https://bugzilla.gnome.org/show_bug.cgi?id=754100
 2015-10-02 17:20:18 +02:00
Thibault Saunier
91b3001f77 validate:launcher: Treat min-media-duration as a float 2015-09-30 18:13:28 +02:00
Thibault Saunier
2b5db0c092 validate: Put the GstValidate testsuite version in the launcher config 
This way testsuite implementation can have the information
 2015-08-21 10:36:56 +02:00
Sebastian Dröge
5b81bb4bc9 validate/launcher: Blacklist some HLS seeking tests again 2015-08-16 17:53:28 +02:00
Thiago Santos
b77f56a234 validate/launcher: Un-blacklist validate.dash.playback.reverse_playback.* 
Fixed now
 2015-08-16 08:50:36 -03:00
Sebastian Dröge
f898f3ec8b validate/launcher: Un-blacklist validate.hls.playback.reverse_playback.* 
It apparently succeeds now.
 2015-08-16 12:26:16 +02:00
Sebastian Dröge
fea86d3556 validate/launcher: De-blacklist some HLS tests 2015-08-15 19:07:02 +02:00
Sebastian Dröge
1f8a5483d3 validate/launcher: Blacklist some DASH tests 2015-08-15 19:04:14 +02:00
Thibault Saunier
bb2532aa54 launcher: Handle override files for media files 
And make sure to create a new dict for extra_env_vars when instanciating
GstValidateTest
 2015-08-15 18:30:53 +02:00
Thibault Saunier
367e6cc4f4 launcher: Make sure MediaDescriptor is set in the GstValidateTest class itself 
It is used there but was set in each and every subclasses
 2015-08-15 18:30:53 +02:00
Sebastian Dröge
233c99fbb7 validate/launcher: Treat DASH like HLS in another place 2015-08-15 16:19:46 +02:00
Vineeth TM
e4d490fd45 validate:launcher: Fix documentation 
Fix some trivial spelling mistakes in documentation
and document about --update-media-info.

https://bugzilla.gnome.org/show_bug.cgi?id=752748
 2015-08-05 17:17:41 -04:00
Vineeth TM
94c684d401 validate:launcher: escape the characters to remove bad range in regex 
When media file name consists of some special characters of the format
[b-a].mp3, then it fails with 'bad character range' error and exits.
call re.escape to escape the characters before using it in findall

https://bugzilla.gnome.org/show_bug.cgi?id=752650
 2015-08-05 16:56:18 -04:00
Vineeth TM
ac172a9ac6 validate:launcher: improve uri generation for --medias-path 
When --medias-paths option is being used, right now we have to
specify the full path, like /home/user/gst/master/media/
But when inside master directory, would like to specify only
media/ and expect it to work. Using os.path.abspath and create uri based on that.
This way we can either just pass media/ or pass the full path as parameters.

https://bugzilla.gnome.org/show_bug.cgi?id=752518
 2015-08-05 16:12:47 -04:00
Vineeth TM
b37c550b2b validate:launcher: skip setting up test suite for --medias-paths 
in validate.py, some mixer test generators are being added by default.
When passing --media-paths, i would not want to test these.
So instead of setting up the validate test suite, just call tester.register_defaults().

https://bugzilla.gnome.org/show_bug.cgi?id=752518
 2015-08-05 16:12:47 -04:00
Thibault Saunier
1fd14635ed validate: launcher: Add a --force-sync option 
Which should put the testsuite in a clean state (basically using git
reset --hard for git based testsuite for example)
 2015-08-04 15:23:54 +02:00
Nicolas Dufresne
bc297a8295 validate: Enable flac reverse playback tests 
This is now supported and works as expected.
 2015-08-04 08:53:17 -04:00
Vineeth T M
aa6aebe41e validate:launcher: add quotes to the file path 
When folder name contains space or other special characters,
it fails to recognise the same and error is thrown. Adding the path
inside  to recognise the same

https://bugzilla.gnome.org/show_bug.cgi?id=752611
 2015-07-22 16:40:07 +01:00
Vineeth T M
074406bc2c validate:launcher: unquote the path to remove special characters 
When getting path from url using, url2path, it is returning
special characters (%20 for space etc..), instead of returning
plain path. path.unquote replaces the same..

https://bugzilla.gnome.org/show_bug.cgi?id=752611
 2015-07-22 16:39:43 +01:00
Vineeth TM
6bf6fe7dec validate:launcher: handle file path in --medias-paths 
right now --medias-paths accepts only directories. Added support to
accept file path as well.

https://bugzilla.gnome.org/show_bug.cgi?id=752692
 2015-07-22 16:35:49 +01:00
Vineeth T M
6c48e89aab validate:launcher: skip default media path for --media-paths option 
when --media-paths is specified, then no need to check the default media.
And add Force argument to let testsuite force the inclusion of
default media directory.

https://bugzilla.gnome.org/show_bug.cgi?id=752461
 2015-07-16 16:31:09 +02:00
Vineeth T M
1ef0fdd66c validate:main.py: trivial document fixes 
gst-validate-launch is being used instead of gst-validate-launcher
in a couple of places.

https://bugzilla.gnome.org/show_bug.cgi?id=752455
 2015-07-16 09:35:17 +02:00
Thibault Saunier
3e978a46b4 validate:launcher: Avoid printing twice env variables 
When printing test command.
 2015-06-12 11:31:13 +02:00
Thibault Saunier
d89b4884c6 validate:launcher: extra_env_variables is a dictionnary 2015-05-14 17:44:17 +02:00
Thibault Saunier
dc2f4f11c6 validate:launcher: Always set the protocol when creating a GstValidateMediaDescriptor 
Summary: Depends on D174

Reviewers: Mathieu_Du, gdesmott

Differential Revision: http://phabricator.freedesktop.org/D175
 2015-05-14 15:56:57 +02:00
Thibault Saunier
4ada36dbfc validate:launcher: Allow passing extra env variables to the tests 
Summary: Depends on D173

Reviewers: Mathieu_Du, gdesmott

Differential Revision: http://phabricator.freedesktop.org/D174
 2015-05-14 15:56:57 +02:00
Guillaume Desmottes
4927c65710 validate: disable QOS features when running with valgrind 
Reviewers: thiblahute

Differential Revision: http://phabricator.freedesktop.org/D156
 2015-05-11 21:55:24 +02:00
Guillaume Desmottes
f6681ff003 validate: rename get_valgrind_suppression_file() 
Summary:
This function is actually not specific to valgrind so we can make it more
generic.

Reviewers: thiblahute

Differential Revision: http://phabricator.freedesktop.org/D162
 2015-05-11 21:55:24 +02:00
Guillaume Desmottes
b947e1c2ef use the setup_sink_props_max_lateness config scenario with valgrind 
Reviewers: thiblahute

Differential Revision: http://phabricator.freedesktop.org/D141
 2015-05-04 14:13:46 +02:00
Guillaume Desmottes
db366e0a37 Fix 'stoped' typo 
Reviewers: thiblahute

Differential Revision: http://phabricator.freedesktop.org/D137
 2015-05-01 18:19:39 +02:00
Guillaume Desmottes
133c415ade validate: display the URL of ignored Valgrind bugs 
Summary:
We don't want to forget about those so best to remind it when starting tests
as we do with blacklisted tests.

Reviewers: thiblahute

Differential Revision: http://phabricator.freedesktop.org/D131
 2015-05-01 15:15:26 +02:00
Guillaume Desmottes
c5c39d88b1 validate: move look_for_file_in_source_dir and get_valgrind_suppression_file to utils 
Reviewers: thiblahute

Differential Revision: http://phabricator.freedesktop.org/D130
 2015-05-01 15:15:26 +02:00
Thibault Saunier
753ba408dd validate:launcher: Concider unset MediaDescriptor duration has 'infinite' 2015-04-28 16:44:42 +02:00
Guillaume Desmottes
044ab241ee validate: use -debug versions of bins when running from source 
Summary:
Those versions are using rpath instead of libtool's wrappers and so will be
faster to start and won't confuse valgrind.

Reviewers: thiblahute

Differential Revision: http://phabricator.freedesktop.org/D116
 2015-04-21 11:19:17 +02:00
Guillaume Desmottes
8c2684c9a7 validate: add gstvalidate.supp valgrind suppression file 
Differential Revision: http://phabricator.freedesktop.org/D115
 2015-04-20 15:29:28 +02:00
Thibault Saunier
31d11ff8c3 validate:launcher: Use full HD screen as default screen size in xvfb 2015-04-19 11:56:29 +02:00
Thibault Saunier
a0b3287d40 validate:launcher: Add a way to run tests without displaying the output 
Summary:
Adding a --no-display option and running Xvfb virtual frame buffer X
server.

Reviewers: Mathieu_Du

Differential Revision: http://phabricator.freedesktop.org/D105
 2015-04-17 21:51:29 +02:00
Thibault Saunier
76eedece5c validate:launcher: Error out if valgrind is not available on the system 
Summary:
When the user wants to use valgrind, make sure it is present on the
system before doing anything

Reviewers: gdesmott

Differential Revision: http://phabricator.freedesktop.org/D104
 2015-04-17 19:58:18 +02:00
Thibault Saunier
adbe811175 validate:launcher: Pass GST_VALIDATE_SCENARIO to the subprocess env only 
Summary:
And make sure to remove it from the env if the user has it in its main
environment.

Without that commit we ended up passing scenarios from previous tests
to the following ones where None were specified.

Reviewers: Mathieu_Du

Differential Revision: http://phabricator.freedesktop.org/D98
 2015-04-16 12:41:25 +02:00
Guillaume Desmottes
8a6494ddf8 validate: set GST_GL_XINITTHREADS 
This ensure that XInitThreads is called and so gl contexts are properly
initialized.

https://bugzilla.gnome.org/show_bug.cgi?id=747840

Signed-off-by: Guillaume Desmottes <guillaume.desmottes@collabora.co.uk>
 2015-04-14 12:56:32 +02:00
Thibault Saunier
0fcee6ed84 validate:launcher: Make validate the only default testsuite 
Reviewers: Mathieu_Du

Differential Revision: http://phabricator.freedesktop.org/D93
 2015-04-13 11:27:08 +02:00
Thibault Saunier
5a6d6b9475 validate:launcher: Let the responsibility to update asset to the testsuite 
Summary:
It makes it easier to make sure that the assets needed for a specific
testsuite are available when needed

Reviewers: Mathieu_Du

Differential Revision: http://phabricator.freedesktop.org/D92
 2015-04-13 11:27:08 +02:00
Thibault Saunier
2cf082c4be validate:launcher: Set more env variable in the launcher command desc 
Summary:
Adding if present:
  * LD_PRELOAD
  * DISPLAY
  * GST_VALIDATE_CONFIG
  * GST_VALIDATE_OVERRIDE

+  enhance the add_env_variable method to more easily set envvar from
current value

Reviewers: Mathieu_Du

Differential Revision: http://phabricator.freedesktop.org/D78
 2015-04-10 16:53:41 +02:00
Thibault Saunier
bbd04eef4c validate:launcher: Use GST_VALIDATE_SCENARIO envvar to set scenarios 
Summary:
Instead of concidering all apps will have a --set-scenario argument
which is not going to be the case as soon as we run the tests through
LD_PRELOAD

Reviewers: Mathieu_Du

Differential Revision: http://phabricator.freedesktop.org/D70
 2015-04-10 16:53:40 +02:00
Thibault Saunier
2fe03f1993 validate:launcher: Move get_current_position from GstValidatePipelineTest to GstValidateTest 
This is where it belongs

Reviewers: Mathieu_Du

Differential Revision: http://phabricator.freedesktop.org/D69
 2015-04-10 16:53:40 +02:00
Guillaume Desmottes
1bb98b49ca validate: GstValidateMediaCheckTest should inherit from GstValidateTest 2015-03-31 09:45:06 +02:00
Young Han Lee
75205828c7 validate:launcher: Install config.py for non-development mode 
Running installed gst-validate-launcher aborted with the following error.

File "lib/gst-validate-launcher/python/launcher/baseclasses.py", line 28, in <module>
  import config
ImportError: No module named config

This is because config.py is added but not installed
in ba6d209b3f.

https://bugzilla.gnome.org/show_bug.cgi?id=747087
 2015-03-31 09:29:16 +02:00
Young Han Lee
4725f4ec09 validate:launcher: Show timeout seconds for timeout result message 
Current timeout message doesn't show how many seconds a test took and
it is timeouted by normal timeout or hard timeout.

This patch changes the message like following.

1. normal timeout
old : validate.http.playback.reverse_playback.raw_video_mov: Timeout (Application timed out)
new : validate.http.playback.reverse_playback.raw_video_mov: Timeout (Application timed out: 120 secs)

2. hard timeout
old : validate.http.playback.reverse_playback.raw_video_mov: Timeout (Application timed out)
new : validate.http.playback.reverse_playback.raw_video_mov: Timeout (Hard timeout reached: 600 secs)

https://bugzilla.gnome.org/show_bug.cgi?id=746957
 2015-03-30 13:32:57 +02:00
Young Han Lee
4c7ec6b8b1 validate:launcher: Fix wrong test number with -j option 
When '-j n' option is given, first n tests print test number 0.
This is caused by test_num part of 919db98605.

https://bugzilla.gnome.org/show_bug.cgi?id=747006
 2015-03-30 13:02:39 +02:00
Thibault Saunier
ba6d209b3f validate:launcher: Avoid depending on PyGObject 
Summary:
And rely on our knowledge of the configuration to figure out where the
suppression file has been installed

Reviewers: gdesmott

Differential Revision: http://phabricator.freedesktop.org/D61
 2015-03-28 23:35:07 +01:00
Guillaume Desmottes
4f805422a7 validate: check VALGRIND_ERROR_CODE in Test as well 
We were doing it only in GstValidateTest which was overriding the default
implementation.
 2015-03-26 13:59:30 +01:00
Guillaume Desmottes
9ad79ffcbc validate: don't increase hard_timeout is if it's None 
Some tests, like the media check ones, have None as hard_timeout.
 2015-03-26 13:57:34 +01:00
Guillaume Desmottes
daaa32676c validate: increase VALGRIND_TIMEOUT_FACTOR 
5 wasn't enough for my poor laptop.
 2015-03-26 11:29:26 +01:00
Guillaume Desmottes
66a675cde8 validate: increase the normal timeout as well when using valgrind 2015-03-26 11:29:06 +01:00
Guillaume Desmottes
f1985bc738 validate: raise an error if valgrind detected issues 
Differential Revision: http://phabricator.freedesktop.org/D53
 2015-03-26 09:18:27 +01:00
Guillaume Desmottes
c23bac2912 validate: store valgrind logs to its own file 2015-03-23 13:41:03 +01:00
Guillaume Desmottes
cb8348c7b1 launcher: try using gst.supp as valgrind suppressions file 
https://bugzilla.gnome.org/show_bug.cgi?id=746465
 2015-03-20 10:34:49 +01:00
Guillaume Desmottes
271f9f3c8e launcher: add valgrind support 
Add a --valgrind option to gst-validate-launcher to run the tests inside
Valgrind and tune GLib's memory allocator accordingly.

Fix https://bugzilla.gnome.org/show_bug.cgi?id=746465
 2015-03-20 10:34:49 +01:00
Guillaume Desmottes
e7762d13cc validate:launcher: Fix small typo 2015-03-19 12:22:39 +01:00
Thibault Saunier
ad5b5f5369 validate:launcher: Make sure to show apps specific options in the help 2015-03-18 17:06:14 +01:00
Thibault Saunier
e9da098e33 validate:launcher: Fix test number printing 2015-03-18 17:05:57 +01:00
Thibault Saunier
7079d1910d validate:launcher Rename _other_testsuite_for_tester 
To _check_tester_has_other_testsuite
 2015-03-17 12:41:26 +01:00
Thibault Saunier
919db98605 validate:launcher: Cache all the tests in the runner 
This way we do not have to re ask all the test managers
what tests should be run.
 2015-03-17 12:41:26 +01:00
Thibault Saunier
d1411cfc6a validate:launcher: Add a way to simply run validate default tests on uris 
Summary:
This allows us to easily run all the scenarios on a particular file doing:

  $ gst-validate-launcher validate --validate-check-uri file:///some/media/file.webm

Reviewers: Mathieu_Du

Differential Revision: http://phabricator.freedesktop.org/D36
 2015-03-13 18:04:42 +00:00
Thibault Saunier
08d8e01f21 validate:launcher: Make sure TIMEOUTs do not get converted to ERROR 
This was a regression introduced in c0e3d2e4f1
 2015-03-09 18:41:54 +01:00
Thibault Saunier
d08a6b8e0c validate:launcher: Do not forget to install apps/__init__.py 
It is a python module that should be usable by external apps/testsuites
 2015-03-06 09:39:36 +01:00
Thibault Saunier
a17c214dba validate:launcher: Better handle GST debug log outputs redirection 2015-03-04 17:57:21 +01:00
Thibault Saunier
c0e3d2e4f1 validate:launcher: First rely on the presence of criticals to set tests result 
In the case of external applications they might not set their exist
code bases on the result of validate so we should rely on what
validates as to say first.
 2015-03-04 17:30:41 +01:00
Thibault Saunier
4c13ec7121 validate:launcher: Print the long help in less when possible 2015-02-24 20:39:22 +01:00
Thibault Saunier
6309f8168a validate:launcher: Mention testsuite implementation in the help 2015-02-24 20:34:43 +01:00
Thibault Saunier
c5393f79f3 validate:launcher: Fix typo s/FILE_EXTENDION/FILE_EXTENSION/g 2015-02-19 13:57:30 +01:00
Ramiro Polla
2d5ad1ac62 validate: launcher: Use cElementTree for XML parsing 
Using cElementTree instead of ElementTree speeds up parsing of media
descriptor files.

The total time spent parsing XML files drops from ~0.64 s to ~0.24 s,
leading to faster initialisation times for gst-validate-launcher.

https://bugzilla.gnome.org/show_bug.cgi?id=743293
 2015-02-17 17:27:40 +01:00
Thibault Saunier
be20eb015c validate:launcher:baseclasses: Avoid raising axception when all getting scenarios 
https://bugzilla.gnome.org/show_bug.cgi?id=743994
 2015-02-14 16:32:10 +01:00
Thibault Saunier
00b222f38f launcher: Use gst-integration-testsuites FDO git repo 
And make sure that people that were using the old repo get the origin
repo properly updated.

https://bugzilla.gnome.org/show_bug.cgi?id=743994
 2015-02-14 16:32:10 +01:00
Thibault Saunier
a85ace1783 validate: launcher: Allow discovering scenario from full path 
https://bugzilla.gnome.org/show_bug.cgi?id=743994
 2015-02-14 16:32:10 +01:00
Thibault Saunier
11f923fa52 validate: Fix the check of action that can be *not* executed 
The check was wrong and we ended up allowing seek actions to no be
executed.

API:
    GST_VALIDATE_ACTION_TYPE_NO_EXECUTION_NOT_FATAL

https://bugzilla.gnome.org/show_bug.cgi?id=743994
 2015-02-14 16:32:09 +01:00
Thibault Saunier
eac25a3ad6 validate:launcher: Implement a FakeMediaDescriptor 
This allows us to more cleanly implement Simple pipeline test
generation

https://bugzilla.gnome.org/show_bug.cgi?id=743994
 2015-02-14 16:32:08 +01:00
Thibault Saunier
da31e3914f validate: Enhance support for simple pipeline test generation 
The GstValidatePipelineGenerator was quite limited in term
of configuration for user who just want to specify pipelines
to run with/without scenario.

Enhance the API so that we can properly configure that.

https://bugzilla.gnome.org/show_bug.cgi?id=743994
 2015-02-14 16:32:08 +01:00
Ramiro Polla
d833a51134 validate: launcher: Print test name in Result 
https://bugzilla.gnome.org/show_bug.cgi?id=743063
 2015-02-05 15:18:39 +01:00
Ramiro Polla
c0cefecd23 validate: launcher: Add option to run tests in parallel 
Patch 4/4 to implement parallel test execution.

https://bugzilla.gnome.org/show_bug.cgi?id=743063
 2015-02-05 15:18:39 +01:00
Ramiro Polla
1f981762c2 validate: launcher: Print test number on result 
With parallel test execution, it will be hard to track which result
relates to which test. Therefore, the test number should be printed
along with the results as well.

Patch 3/4 to implement parallel test execution.

https://bugzilla.gnome.org/show_bug.cgi?id=743063
 2015-02-05 15:18:39 +01:00
Ramiro Polla
a9a366427e validate: launcher: Use jobs list to take track of tests running 
Currently the tests are still run serially.

Patch 2/4 to implement parallel test execution.

https://bugzilla.gnome.org/show_bug.cgi?id=743063
 2015-02-05 15:18:38 +01:00
Ramiro Polla
d9097f8bf7 validate: launcher: Use test index instead of counting test numbers 
Patch 1/4 to implement parallel test execution.

https://bugzilla.gnome.org/show_bug.cgi?id=743063
 2015-02-05 15:18:38 +01:00
Ramiro Polla
9c46f5f889 validate: launcher: Support simultaneous requests in RangeHTTPServer 
https://bugzilla.gnome.org/show_bug.cgi?id=743063
 2015-02-05 15:18:38 +01:00
Ramiro Polla
ef246a497c validate: launcher: Make TestManager handle waiting for processes 
Patch 4/4 to make TestManager handle waiting for processes instead of
expecting each Test to do it.

https://bugzilla.gnome.org/show_bug.cgi?id=743063
 2015-02-05 15:18:38 +01:00
Ramiro Polla
0026c2804f validate: launcher: Use a Queue to test for test completion 
TestManager will use a Queue to track progress for all tests. This
commit implements a queue inside Test to simplify the transition.

Patch 3/4 to make TestManager handle waiting for processes instead of
expecting each Test to do it.

https://bugzilla.gnome.org/show_bug.cgi?id=743063
 2015-02-05 15:18:38 +01:00
Ramiro Polla
bd4c221141 validate: launcher: Initialize Test start time outside of wait_process 
wait_process will be moved to TestManager, so the values used to track
process update must remain inside Test.

Patch 2/4 to make TestManager handle waiting for processes instead of
expecting each Test to do it.
 2015-02-05 15:18:38 +01:00
Ramiro Polla
498f7002f3 validate: launcher: Split process_update() out of wait_process() 
Patch 1/4 to make TestManager handle waiting for processes instead of
expecting each Test to do it.
 2015-02-05 15:18:37 +01:00
Ramiro Polla
d12f55daf4 validate: launcher: Move logfile handling out of Reporter and into Test 
This makes each Test handle its own logfile, allowing the Reporter to
work on multiple tests at the same time.

Patch 5/5 to move logfile handling out of Reporter and into Test.
 2015-02-05 15:18:37 +01:00
Ramiro Polla
500206d3ad validate: launcher: Remove redundant check 
self.out is always available when _get_captured() is called.

Patch 4/5 to move logfile handling out of Reporter and into Test.
 2015-02-05 15:18:37 +01:00
Ramiro Polla
177eee728f validate: launcher: Split test log file handling in Reporter 
Patch 3/5 to move logfile handling out of Reporter and into Test.
 2015-02-05 15:18:37 +01:00
Ramiro Polla
b9357e3b17 validate: launcher: Separate Reporter from current Test 
Instead of saving the current Test in Reporter for every test, use
function parameters to achieve the same goal.

Patch 2/5 to move logfile handling out of Reporter and into Test.
 2015-02-05 15:18:37 +01:00
Ramiro Polla
39a4092434 validate: launcher: Initialize reporter timer before starting all tests 
Patch 1/5 to move logfile handling out of Reporter and into Test.
 2015-02-05 15:18:37 +01:00
Ramiro Polla
bdedd7abb9 validate: launcher: Don't wait for processes longer than necessary 2015-02-05 15:18:36 +01:00
Ramiro Polla
b29f378296 validate: launcher: Fix test log header output 
Write log file header before running tests, instead of overwriting the
file afterwards.

https://bugzilla.gnome.org/show_bug.cgi?id=742966
 2015-01-16 16:24:16 +01:00
Ramiro Polla
9c3606a867 validate: launcher: Always create log files 
Create log files even when stdout redirection is enabled.
This commit partially reverts 20c28de.

https://bugzilla.gnome.org/show_bug.cgi?id=742973
 2015-01-16 16:24:13 +01:00
Ramiro Polla
4e0388c631 validate: launcher: Introduce new parameter for log file redirecting 
Allow log file redirection through the new --redirect-logs parameter.
Keep the old --logs-dir stdout/stderr parameter, but reset to the
default logs directory in that case, and set redirect_logs internally.
This also prevents the creation of an stdout/stderr directory for
writing xunit.xml.

https://bugzilla.gnome.org/show_bug.cgi?id=742973
 2015-01-16 16:24:13 +01:00
Thibault Saunier
1e39db18ad validate: launcher: Take the timeout as ref timeout to compute hard_timeout 
when it is provided.
 2014-12-08 18:38:46 +01:00
Thibault Saunier
5beaf5dfa8 validate: launcher: Set a hard timeout on GstValidate tests if we know the duration 2014-12-08 16:34:48 +01:00
Thibault Saunier
a2abf628dc validate: launcher: Force kill subprocess when done with them 
Making sure that we do not end up having spurious subprocess around
 2014-12-08 16:19:20 +01:00
Thiago Santos
d25fb034fa launcher: baseclass: add missing parameter 
Fixes "NameError: global name 'options' is not defined"
 2014-12-08 08:43:57 -03:00
Thibault Saunier
8450dff17d validate: launcher: Properly handle non default main dir 
for the case of the new testsuite files
 2014-12-08 10:10:04 +01:00
Thibault Saunier
5bfd579bf4 validate: launcher: Make the gstvalidate application a python module 2014-12-07 18:32:37 +01:00
Thibault Saunier
272032c0b0 validate: Handle setting the HTTP server local path from testsuites 2014-12-05 20:51:44 +01:00
Thibault Saunier
22fba81138 validate: Rename gst-qa-assets to gst-integration-testsuites 2014-12-05 20:51:43 +01:00
Thibault Saunier
3dff417fdf validate: Factor out an LauncherConfig class type to handle configurations 
Allowing us to more simply define default value and expose an API on
top of it
 2014-12-05 19:23:43 +01:00
Thibault Saunier
515c8405f0 validate: Remove file specific blacklisted tests 2014-12-05 16:16:53 +01:00
Thibault Saunier
988e9a370e validate: Let the user know when new tests are added, or tests are REMOVED 2014-12-05 16:16:53 +01:00
Thibault Saunier
d38307a073 validate: Add a cleaner API to setup tests in testsuite files 
With the testsuite format you will get a setup_tests(tests_manager,
options) function called for each TestManager.

The function will have the exact same role as with old config
file but with a clean API and not magic global variables.

This implies that we need default blacklist to be directly set
on the TestManager and not on options.blacklisted_test
 2014-12-05 16:16:53 +01:00
Thibault Saunier
b852fbcbfb validate: Add a way to sync all assets, including big ones 2014-12-05 12:17:40 +01:00
Thibault Saunier
71566688bf validate: Remove the default testsuite implementation 
The default testsuite implementation should belong to the default
asset repo where we have the corresponding knowledge.

We should style manage a sensible list of known blacklisted tests,
encoding profiles, and generators in GstValidate itself and allow testsuite
actual implementations to easily use them though the register_default_*
methods.

This allow us to be able to remove the ugly execfile() call.
 2014-12-05 12:17:40 +01:00
Thibault Saunier
ccbcc04498 validate: Disable coloration of GST_DEBUG logs when we have no-color 
Do that only when those logs are not saved to a file
 2014-12-05 12:16:54 +01:00
Vineeth T M
163d286f5b validate: fix typo in documentation 
There are some typing mistakes in gst-validate-launcher --help
Hence fixing the same.

https://bugzilla.gnome.org/show_bug.cgi?id=740833
 2014-11-28 11:16:09 +01:00
Thibault Saunier
b5219eaceb validate: Handle unlimited tests duration 
Running full length scenario when the user asks
 2014-11-25 19:41:01 +01:00
Thibault Saunier
65eb14de36 validate: Add an option to update all .media_info files 2014-11-25 19:41:00 +01:00
Thibault Saunier
36a2d61068 validate:launcher: Force clock sync for some protocols 
In HLS for example, not having clock sync might lead to races and failures
do not test that for now
 2014-11-20 12:09:29 +01:00
Thibault Saunier
a15b2544ce validate: Handle wrong paths when listing avalaible apps 
User can make mistake or we can have an empty path.
 2014-11-19 17:59:26 +01:00
Thibault Saunier
5d8f6083b4 validate: Add audiomixer test to the default testsuite 2014-11-16 18:27:32 +01:00
Thibault Saunier
e2d8096cb7 validate: Do not exit when we can not discover a result file 
Loggable.error actually exit the process, it is not what we want!

+ Avoid a backtrace
 2014-11-04 17:33:08 +01:00
Mathieu Duponchelle
7bbd3ed288 apps: gstvalidate.py: fix various pyflakes / uncaught pep8 issues. 
https://bugzilla.gnome.org/show_bug.cgi?id=739208
 2014-10-30 15:40:17 +01:00
Mathieu Duponchelle
9bdc6e9b6d validate-launcher: utils: fix various pyflakes / uncaught pep8 issues. 
https://bugzilla.gnome.org/show_bug.cgi?id=739208
 2014-10-30 15:40:17 +01:00
Mathieu Duponchelle
b30d27e35e validate-launcher: main: fix various pyflakes / uncaught pep8 issues. 
https://bugzilla.gnome.org/show_bug.cgi?id=739208
 2014-10-30 15:40:17 +01:00
Mathieu Duponchelle
95065edebd validate-launcher: loggable: fix various pyflakes / uncaught pep8 issues. 
https://bugzilla.gnome.org/show_bug.cgi?id=739208
 2014-10-30 15:40:17 +01:00
Mathieu Duponchelle
a0a88e1708 validate-launcher: baseclasses: fix various pyflakes / uncaught pep8 issues. 
https://bugzilla.gnome.org/show_bug.cgi?id=739208
 2014-10-30 15:40:17 +01:00
Mathieu Duponchelle
8c1e84b5f4 validate-launcher: pep8ify sources. 
https://bugzilla.gnome.org/show_bug.cgi?id=739208
 2014-10-30 15:40:17 +01:00
Mathieu Duponchelle
4d569b51ed launcher: add a way to specify an application directory. 
https://bugzilla.gnome.org/show_bug.cgi?id=739091
 2014-10-30 15:31:29 +01:00
Mathieu Duponchelle
74ed40c904 launcher: Don't implement product-specific TestManagers. 
This manager will be moved in GES.

https://bugzilla.gnome.org/show_bug.cgi?id=739091
 2014-10-30 15:31:29 +01:00
Mathieu Duponchelle
b0beefacfa validate-launcher: restructure filesystem 
https://bugzilla.gnome.org/show_bug.cgi?id=739091
 2014-10-30 15:31:29 +01:00