In the future instead of blacklisting tests we should define
what error is expected, and this way when the bug is closed,
we will notice, also, it will allow us to check GstValidate
error reporting itself.
Instead of trying to parsing stdout, generate json messages and
send them over a socket so that gst-validate-launcher can properly
have informations about gst-validate subprocess execution.
Instead of providing full absolute path while validating the file, should be
able to provide the relative path with respect to the present directory.
https://bugzilla.gnome.org/show_bug.cgi?id=753494
Makes fixing easier as then we can just re-use the generated trace.
Reviewed-by: Thibault Saunier <tsaunier@gnome.org>
Differential Revision: https://phabricator.freedesktop.org/D953
When a first testsuite will set paths, it does not mean that we should
just register following testsuite test manager default tests.
So we need to make a difference between the media paths the user passed
with --media-path and the ones defined by the testsuite.
When listing tests, checking whether uri is present or not and displaying error.
But uri does notneed to be present in case of pipeline generator. So the condition check is wrong.
This results in validateelements testsuite not working. Hence modifying the condition to
not error out on valid cases.
https://bugzilla.gnome.org/show_bug.cgi?id=762422
The seekable variable in media_info file is of type string. When checking if the file
is seekable using is_seekable, it just returns the string, resulting in it always being true.
It should actually be comparing the string and returning true or false based on comparison
https://bugzilla.gnome.org/show_bug.cgi?id=755854
When there are no media-info files present and --generate-media-info
option is not given, then it just fails without printing error.
Printing an error stating, use --generate-media-info if there are no
media info files.
When there are neither media files and media info files, print error
stating the same
https://bugzilla.gnome.org/show_bug.cgi?id=755087
errors-for-leak-kinds should be set to definite, because almost every test case
, will have possibly lost memory, which may or may not be a leak.
And throwing error for all these cases doesn't seem to be correct.
https://bugzilla.gnome.org/show_bug.cgi?id=752754
When creating the class names for media check, uri is being used,
instead of the path. Hence converting the uri using uri2path and creating
class name.
Add double quotes for valgrind logs, to support special characters like space
https://bugzilla.gnome.org/show_bug.cgi?id=752808
Even though relative paths are supported, right now it does not
work when we give the path as 'media/' present in the current directory.
Adding support for the same.
https://bugzilla.gnome.org/show_bug.cgi?id=754100
When media file name consists of some special characters of the format
[b-a].mp3, then it fails with 'bad character range' error and exits.
call re.escape to escape the characters before using it in findall
https://bugzilla.gnome.org/show_bug.cgi?id=752650
When --medias-paths option is being used, right now we have to
specify the full path, like /home/user/gst/master/media/
But when inside master directory, would like to specify only
media/ and expect it to work. Using os.path.abspath and create uri based on that.
This way we can either just pass media/ or pass the full path as parameters.
https://bugzilla.gnome.org/show_bug.cgi?id=752518
in validate.py, some mixer test generators are being added by default.
When passing --media-paths, i would not want to test these.
So instead of setting up the validate test suite, just call tester.register_defaults().
https://bugzilla.gnome.org/show_bug.cgi?id=752518
When folder name contains space or other special characters,
it fails to recognise the same and error is thrown. Adding the path
inside to recognise the same
https://bugzilla.gnome.org/show_bug.cgi?id=752611
When getting path from url using, url2path, it is returning
special characters (%20 for space etc..), instead of returning
plain path. path.unquote replaces the same..
https://bugzilla.gnome.org/show_bug.cgi?id=752611
when --media-paths is specified, then no need to check the default media.
And add Force argument to let testsuite force the inclusion of
default media directory.
https://bugzilla.gnome.org/show_bug.cgi?id=752461
Summary:
This function is actually not specific to valgrind so we can make it more
generic.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D162
Summary:
We don't want to forget about those so best to remind it when starting tests
as we do with blacklisted tests.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D131
Summary:
Those versions are using rpath instead of libtool's wrappers and so will be
faster to start and won't confuse valgrind.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D116
Summary:
When the user wants to use valgrind, make sure it is present on the
system before doing anything
Reviewers: gdesmott
Differential Revision: http://phabricator.freedesktop.org/D104
Summary:
And make sure to remove it from the env if the user has it in its main
environment.
Without that commit we ended up passing scenarios from previous tests
to the following ones where None were specified.
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D98
This ensure that XInitThreads is called and so gl contexts are properly
initialized.
https://bugzilla.gnome.org/show_bug.cgi?id=747840
Signed-off-by: Guillaume Desmottes <guillaume.desmottes@collabora.co.uk>
Summary:
It makes it easier to make sure that the assets needed for a specific
testsuite are available when needed
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D92
Summary:
Adding if present:
* LD_PRELOAD
* DISPLAY
* GST_VALIDATE_CONFIG
* GST_VALIDATE_OVERRIDE
+ enhance the add_env_variable method to more easily set envvar from
current value
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D78
Summary:
Instead of concidering all apps will have a --set-scenario argument
which is not going to be the case as soon as we run the tests through
LD_PRELOAD
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D70
Running installed gst-validate-launcher aborted with the following error.
File "lib/gst-validate-launcher/python/launcher/baseclasses.py", line 28, in <module>
import config
ImportError: No module named config
This is because config.py is added but not installed
in ba6d209b3f.
https://bugzilla.gnome.org/show_bug.cgi?id=747087
Current timeout message doesn't show how many seconds a test took and
it is timeouted by normal timeout or hard timeout.
This patch changes the message like following.
1. normal timeout
old : validate.http.playback.reverse_playback.raw_video_mov: Timeout (Application timed out)
new : validate.http.playback.reverse_playback.raw_video_mov: Timeout (Application timed out: 120 secs)
2. hard timeout
old : validate.http.playback.reverse_playback.raw_video_mov: Timeout (Application timed out)
new : validate.http.playback.reverse_playback.raw_video_mov: Timeout (Hard timeout reached: 600 secs)
https://bugzilla.gnome.org/show_bug.cgi?id=746957
Summary:
And rely on our knowledge of the configuration to figure out where the
suppression file has been installed
Reviewers: gdesmott
Differential Revision: http://phabricator.freedesktop.org/D61
Summary:
This allows us to easily run all the scenarios on a particular file doing:
$ gst-validate-launcher validate --validate-check-uri file:///some/media/file.webm
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D36
In the case of external applications they might not set their exist
code bases on the result of validate so we should rely on what
validates as to say first.
Using cElementTree instead of ElementTree speeds up parsing of media
descriptor files.
The total time spent parsing XML files drops from ~0.64 s to ~0.24 s,
leading to faster initialisation times for gst-validate-launcher.
https://bugzilla.gnome.org/show_bug.cgi?id=743293
The GstValidatePipelineGenerator was quite limited in term
of configuration for user who just want to specify pipelines
to run with/without scenario.
Enhance the API so that we can properly configure that.
https://bugzilla.gnome.org/show_bug.cgi?id=743994
With parallel test execution, it will be hard to track which result
relates to which test. Therefore, the test number should be printed
along with the results as well.
Patch 3/4 to implement parallel test execution.
https://bugzilla.gnome.org/show_bug.cgi?id=743063
TestManager will use a Queue to track progress for all tests. This
commit implements a queue inside Test to simplify the transition.
Patch 3/4 to make TestManager handle waiting for processes instead of
expecting each Test to do it.
https://bugzilla.gnome.org/show_bug.cgi?id=743063
wait_process will be moved to TestManager, so the values used to track
process update must remain inside Test.
Patch 2/4 to make TestManager handle waiting for processes instead of
expecting each Test to do it.
This makes each Test handle its own logfile, allowing the Reporter to
work on multiple tests at the same time.
Patch 5/5 to move logfile handling out of Reporter and into Test.
Instead of saving the current Test in Reporter for every test, use
function parameters to achieve the same goal.
Patch 2/5 to move logfile handling out of Reporter and into Test.
Allow log file redirection through the new --redirect-logs parameter.
Keep the old --logs-dir stdout/stderr parameter, but reset to the
default logs directory in that case, and set redirect_logs internally.
This also prevents the creation of an stdout/stderr directory for
writing xunit.xml.
https://bugzilla.gnome.org/show_bug.cgi?id=742973
With the testsuite format you will get a setup_tests(tests_manager,
options) function called for each TestManager.
The function will have the exact same role as with old config
file but with a clean API and not magic global variables.
This implies that we need default blacklist to be directly set
on the TestManager and not on options.blacklisted_test
The default testsuite implementation should belong to the default
asset repo where we have the corresponding knowledge.
We should style manage a sensible list of known blacklisted tests,
encoding profiles, and generators in GstValidate itself and allow testsuite
actual implementations to easily use them though the register_default_*
methods.
This allow us to be able to remove the ugly execfile() call.
