The user might have scenarios specific to a particular pipeline, and the
application might have several pipelines running and scenarios that
apply on specific pipeline. We have to handle that valid use case.
Summary:
Move variable declarations in the for block so we won't try re-free
tldir in case of early short circuiting of the 'for' code.
Depends on D348
Reviewers: thiblahute
Reviewed By: thiblahute
Differential Revision: https://phabricator.freedesktop.org/D349
Summary:
We were checking if the path was a full one but was using the
scenario_name instead of this path when trying to load the scenario.
Depends on D346
Reviewers: thiblahute
Reviewed By: thiblahute
Differential Revision: https://phabricator.freedesktop.org/D348
errors-for-leak-kinds should be set to definite, because almost every test case
, will have possibly lost memory, which may or may not be a leak.
And throwing error for all these cases doesn't seem to be correct.
https://bugzilla.gnome.org/show_bug.cgi?id=752754
When creating the class names for media check, uri is being used,
instead of the path. Hence converting the uri using uri2path and creating
class name.
Add double quotes for valgrind logs, to support special characters like space
https://bugzilla.gnome.org/show_bug.cgi?id=752808
writer_new_discover() API should be able to accept NULL GError and in case of
error, if GError is passed on as parameter, it should be propagated, else it
should be free'd.
https://bugzilla.gnome.org/show_bug.cgi?id=753340
When file name consists of characters from other languages, say korean,
then it throws an error
Error initializing: Invalid byte sequence in conversion input
Hence setting locale to all to fix this.
And changing the media-info argument to type G_OPTION_ARG_FILENAME
https://bugzilla.gnome.org/show_bug.cgi?id=753486
Even though relative paths are supported, right now it does not
work when we give the path as 'media/' present in the current directory.
Adding support for the same.
https://bugzilla.gnome.org/show_bug.cgi?id=754100
As soon as the track is changed, the pipeline state is set to NULL
by execution 'stop' action even if there is a 'playback-time' with 5sec.
If the AV sink is not synchronized,
audio fakesink and video fakesink has different position value.
When the validate request the position information of pipeline
to do 'stop' action, the audio fakesink response of the position query
with the bigger value than 5sec.
https://bugzilla.gnome.org/show_bug.cgi?id=755101
If the scenario handles the states and wants to stay in PAUSED, it's not a
good idea to change the state to PLAYING when receiving BUFFERING=100%. This
caused a race condition in varios seeking tests, most often in the dash scrub
seeking test.
If the scenario handles the states and wants to stay in PAUSED, it's not a
good idea to change the state to PLAYING when receiving BUFFERING=100%. This
caused a race condition in varios seeking tests, most often in the dash scrub
seeking test.
When message_async is not called during error cases, needs_parsing GList is
not being freed resulting in leak. Hence free'ing the same in finalize.
https://bugzilla.gnome.org/show_bug.cgi?id=753339
There is no check to see if stream info is available. This leads to
assertion error. Adding proper error messages for the same and reported
the same as a validate warning message.
https://bugzilla.gnome.org/show_bug.cgi?id=752758
When discovering the files, there will be different kind of errors. If we print
the exact message, then it will be more helpful for user. Especially in the case
of missing plugins, displaying which plugin is missing as error message
https://bugzilla.gnome.org/show_bug.cgi?id=752758
When media file name consists of some special characters of the format
[b-a].mp3, then it fails with 'bad character range' error and exits.
call re.escape to escape the characters before using it in findall
https://bugzilla.gnome.org/show_bug.cgi?id=752650
When --medias-paths option is being used, right now we have to
specify the full path, like /home/user/gst/master/media/
But when inside master directory, would like to specify only
media/ and expect it to work. Using os.path.abspath and create uri based on that.
This way we can either just pass media/ or pass the full path as parameters.
https://bugzilla.gnome.org/show_bug.cgi?id=752518
in validate.py, some mixer test generators are being added by default.
When passing --media-paths, i would not want to test these.
So instead of setting up the validate test suite, just call tester.register_defaults().
https://bugzilla.gnome.org/show_bug.cgi?id=752518
Right now reverse playback happens till the beginning of the media file.
But for files which are longer than 150 seconds,
Timeout 'Hard timeout reached: 150 secs' error happens. So we should set the
start time within 150 seconds.
https://bugzilla.gnome.org/show_bug.cgi?id=753216
file:// base stream-id will vary depending on the file path. As we
don't expect everyone to use the same absolute path to place the
validate testsuite, the resulting stream-id changes. Because of that,
we can't match the stream-id in the recorded file, hence cannot do
further check. We work around this by doing what filesink would do,
which is compute a SHA256 of the URI which we can use to first
validate the ID is prefixed like expected, and decide if we should
consider the stream IDs the same or not.
https://bugzilla.gnome.org/show_bug.cgi?id=753079
while comparing the media descriptor with --expected-results, the return
values are not being handled properly, which results in wrong comparision
https://bugzilla.gnome.org/show_bug.cgi?id=748390
As stated in the bug, this comparison failing is not a critical
error, warning is enough. Add a comment so nobody thinks it's a
coding error.
https://bugzilla.gnome.org/review?bug=748390
when comparing tags, two conditions in if an else if are same
the correct way is to first check if both are NULL and return.
changed the condition accordingly.
https://bugzilla.gnome.org/show_bug.cgi?id=748390
When file name consists of characters from other languages, say korean,
then it throws an error
Error initializing: Invalid byte sequence in conversion input
Hence setting locale to all to fix this.
And changing the media-info argument to type G_OPTION_ARG_FILENAME
https://bugzilla.gnome.org/show_bug.cgi?id=752945
Since _set_done() is meant to be thread safe,
it can not be used with g_object_add_weak_pointer(),
instead, one must use GWeakRef. But since it is in the API,
document that fact and add a couple assertions to make sure
it doesn't get broken in the future.
In case of files, which don't have duration in header, baseparse
estimates the duration only after 1.5 seconds. But Async_done event
is sent before the duration is estimated, which results in error.
If duration query fails, getting the duration from the media-info being
passed through --set-media-info. If media-info is also not set,
printing an error message and throwing error.
https://bugzilla.gnome.org/show_bug.cgi?id=752521
When folder name contains space or other special characters,
it fails to recognise the same and error is thrown. Adding the path
inside to recognise the same
https://bugzilla.gnome.org/show_bug.cgi?id=752611
When getting path from url using, url2path, it is returning
special characters (%20 for space etc..), instead of returning
plain path. path.unquote replaces the same..
https://bugzilla.gnome.org/show_bug.cgi?id=752611
When folder name contains spaces during --medias-paths, it does not
create the media info, but still it shows as passed.
Returing failed during this case
https://bugzilla.gnome.org/show_bug.cgi?id=752611
when --media-paths is specified, then no need to check the default media.
And add Force argument to let testsuite force the inclusion of
default media directory.
https://bugzilla.gnome.org/show_bug.cgi?id=752461
Summary:
When running valgrind we'll have 2 scenarios loaded (the normal one and
"setup_sink_props_max_lateness.scenario"). The loading code shouldn't assume
which one will contain the description it actually care about and so just look
for the fields it actually needs.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D199
Allowing users to decide the time between which the action should be
executed. In some cases executing on idle might lead to action not
being executed fast enough so the user might want to force an interval
in that case.
/usr/bin/ld: gst-validate-images-check.o: undefined reference to symbol 'gst_init'
/home/tpm/gst/glib-master/gstreamer/gst/.libs/libgstreamer-1.0.so.0: error adding symbols: DSO missing from command line
Summary:
+ Bump gst-video dependency to 1.4 as we need GstVideoConvert
Depends on D213: validate: Mark gst_validate_report a G_GNUC_PRINTF
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D215
Summary:
That is a new tool that uses ssim algorithm to compare images
+ Add a GstValidateVideo internal library adding an helper Gssim class
Depends on D210
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D211
Summary:
Properly handling the different types that can represent ClockTime
Make use of it in gst_validate_action_get_clocktime
API: gst_validate_utils_get_clocktime
Depends on D209
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D210
Summary:
+ Fix a minor mixup bug between klass_overrides and name_overrides
Depends on D205
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D206
Summary:
This way we can subclass them getting a proper
context in the various override methods.
Depends on D204
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D205
Summary:
Otherwise we end up with circular / complicated dependencies between
Validate, its libraries, and the plugins
Depends on D203
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D204
Otherwise we end up with rounding error and instead of
seeking to 0.1 we seek to 0.09999999999 for example
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D203
This method is similar to runner_printf() but can be used
only once. The user needs to make sure all the pipeline
are in NULL state when this is called.
The method emits a "STOPPING" signal and at that point
overrides or monitors should do extra processing/checks if
needed.
+ Make use of it everywhere where it makes sense.
API:
gst_validate_runner_exit
GstValidateRunner::stopping signal
Summary:
Before returning GST_FLOW_ERROR, an element must post an ERROR GstMessage,
enforce that.
Reviewers: thiblahute, Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D201
Using sqrt of -1 is not valid and leads to undefined results.
When comparing the return value of the fucntion in validate-scenario,
it is being checked with ret == -1, so it makes sense to just return -1 in error case.
https://bugzilla.gnome.org/show_bug.cgi?id=748389
Summary:
It is possible to keep executing actions after the pipeline
has been destroyed.
API:
GST_VALIDATE_ACTION_TYPE_DOESNT_NEED_PIPELINE
Depends on D171
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D172
Summary:
Making simpler to follow the execute_next_action function.
Depends on D169
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D170
Summary:
If from anything >= PAUSED to anything <= READY we can not query
pipeline position, so do not try to.
Depends on D168
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D169
Summary:
Currently the only supported action is gtk-put-event allowing press and
release keyboard keys.
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D117
Summary: The x264 code is pretty hardcore so I just opened a bug for now.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D164
Summary:
This function is actually not specific to valgrind so we can make it more
generic.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D162
Summary:
Useful to know if we are executing the 'stop' command provided by the scenario
or not.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D155
Summary: Fix invalid read when executing without having the actual position.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D147
Summary:
This allows us to set a property on all the elements of the pipeline matching
a specific klass name.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D140
Summary:
We want to have a chance to set property on all the elements of the pipelines,
including the existing children when the element is added.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D138
Summary:
We don't want to forget about those so best to remind it when starting tests
as we do with blacklisted tests.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D131
Summary:
Those versions are using rpath instead of libtool's wrappers and so will be
faster to start and won't confuse valgrind.
Reviewers: thiblahute
Differential Revision: http://phabricator.freedesktop.org/D116
Summary:
When the user wants to use valgrind, make sure it is present on the
system before doing anything
Reviewers: gdesmott
Differential Revision: http://phabricator.freedesktop.org/D104
And document it properly.
Summary:
The stop action was defined as "setting state to NULL" but
its actual goal is to stop the execution of the scenario. Make sure
that the scenario will not try to execute other actions when that
one has been executed.
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D103
Summary:
And make sure to remove it from the env if the user has it in its main
environment.
Without that commit we ended up passing scenarios from previous tests
to the following ones where None were specified.
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D98
This ensure that XInitThreads is called and so gl contexts are properly
initialized.
https://bugzilla.gnome.org/show_bug.cgi?id=747840
Signed-off-by: Guillaume Desmottes <guillaume.desmottes@collabora.co.uk>
Summary:
It makes it easier to make sure that the assets needed for a specific
testsuite are available when needed
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D92
Summary:
+ typedef GstValidateActionReturn so it can be used in the introspection
+ Add GST_VALIDATE_EXECUTE_ACTION_ERROR_REPORTED which should be used
to tell Validate that something wrong happened so the sub action
won't be executed, but that it should not report an error itself
as it has already been handled in the action function.
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D81
Summary:
And fix a bug where config actions were added to the list of action even
if they had already been executed
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D80
Summary:
Adding if present:
* LD_PRELOAD
* DISPLAY
* GST_VALIDATE_CONFIG
* GST_VALIDATE_OVERRIDE
+ enhance the add_env_variable method to more easily set envvar from
current value
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D78
Summary:
Add a very simple plugin that will allow any GApplication to easily be
used with GstValidate using the LD_PRELOAD feature
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D75
Summary: And print the current repeat value of the action that have such a field
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D73
Summary:
Instead of concidering all apps will have a --set-scenario argument
which is not going to be the case as soon as we run the tests through
LD_PRELOAD
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D70
Running installed gst-validate-launcher aborted with the following error.
File "lib/gst-validate-launcher/python/launcher/baseclasses.py", line 28, in <module>
import config
ImportError: No module named config
This is because config.py is added but not installed
in ba6d209b3f.
https://bugzilla.gnome.org/show_bug.cgi?id=747087
Current timeout message doesn't show how many seconds a test took and
it is timeouted by normal timeout or hard timeout.
This patch changes the message like following.
1. normal timeout
old : validate.http.playback.reverse_playback.raw_video_mov: Timeout (Application timed out)
new : validate.http.playback.reverse_playback.raw_video_mov: Timeout (Application timed out: 120 secs)
2. hard timeout
old : validate.http.playback.reverse_playback.raw_video_mov: Timeout (Application timed out)
new : validate.http.playback.reverse_playback.raw_video_mov: Timeout (Hard timeout reached: 600 secs)
https://bugzilla.gnome.org/show_bug.cgi?id=746957
Summary:
And rely on our knowledge of the configuration to figure out where the
suppression file has been installed
Reviewers: gdesmott
Differential Revision: http://phabricator.freedesktop.org/D61
Summary:
@report was invalid when we were trying to clear the mutex.
validate: scenario: remove weak pointer when destroying action
Free an invalid read when the scenario is destroyed after the action.
Differential Revision: http://phabricator.freedesktop.org/D44
Summary:
This allows us to easily run all the scenarios on a particular file doing:
$ gst-validate-launcher validate --validate-check-uri file:///some/media/file.webm
Reviewers: Mathieu_Du
Differential Revision: http://phabricator.freedesktop.org/D36
When linking actions execution without waiting on execution context, then
idle callback should keep being called so following action keep being
executed.
Summary:
There is currently no way to handle the fact that action types
might be handled only by a specific application but not handling
this action types would not cause any difference for the good execution
of the scenario as a whole
Differential Revision: http://phabricator.freedesktop.org/D33
In the case of external applications they might not set their exist
code bases on the result of validate so we should rely on what
validates as to say first.
Fixing annotations and make GstValidateIssue refcounted
We break the ABI in that commit but I do not expect anyone to register
issue type outside GstValidate yet.
Add padding in the structures so we can avoid breaking the ABI again later.
Development mode has been determined by whether the launcher is in git
repo
or not. This could be wrong when the launcher is installed to
subdirectory of other project's git repo, such as jhbuild. It is normal
to install compiled output to subdirectory of your jhbuild.
Changed logic gets the first commit hash of current git repo and
compares it with gst-devtools' the first commit hash.
https://bugzilla.gnome.org/show_bug.cgi?id=744781
Summary: As part of the preparation for a port to tracer.
Test Plan: This is a test, we won't test tests
Reviewers: tsaunier
Differential Revision: http://internal.opencreed.com:8888/D19
Using cElementTree instead of ElementTree speeds up parsing of media
descriptor files.
The total time spent parsing XML files drops from ~0.64 s to ~0.24 s,
leading to faster initialisation times for gst-validate-launcher.
https://bugzilla.gnome.org/show_bug.cgi?id=743293
Avoiding user to have to print them in each and every action type
implementation.
This requires adding some API to prepare actions before printing them.
Preparing action in that case mean parsing the values contained in the
GstStructure parsing equations and setting back the actual value
afterward
API:
* GstValidatePrepateAction
* gst_validate_action_type_set_prepare_function
Summary:
- Add a way to force action to be executed in their own GSource dispatch, disabling chain action execution
API:
GstValidateScenario::execute-on-idle property
This reverts commit b976319ef7f977b8ce910c4b8aa1a843da3b264f.
Now that the exact same structure can be used to represent different
action types, we can not rely on the structure size to stuff
informations into the action. Users should just make use of
GstMiniObject.qdata.
Sub action will allow user to executed action *right* after the
previous action has been completed, meaning in the end that both
action can be considered as one single action.
+ Factor out a function to fill an GstValidateAction structure from a
GstStructure
+ Factor out a function to set action playback time
The new action might change the position on purpose and we should not
fail in that case.
Also at that point we know the test of position after the seek has
been executed
+ Minor cosmetic fixes
https://bugzilla.gnome.org/show_bug.cgi?id=743994
An interlaced action is an action that will be executed ASYNC but
without that will not block following actions during its execution.
The action should be set to done later on at any point during the
execution of the scenario.
API:
+ GST_VALIDATE_EXECUTE_ACTION_INTERLACED
+ GST_VALIDATE_ACTION_TYPE_INTERLACED
https://bugzilla.gnome.org/show_bug.cgi?id=743994
The GstValidatePipelineGenerator was quite limited in term
of configuration for user who just want to specify pipelines
to run with/without scenario.
Enhance the API so that we can properly configure that.
https://bugzilla.gnome.org/show_bug.cgi?id=743994
This action type can take some time, we need to make sure that the
combiner/input-selector element properly pushed a buffer marked
as DISCONT to concider the action is done.
https://bugzilla.gnome.org/show_bug.cgi?id=743994
We should be able to execute the next action as soon as the previous
one is fully completed, make sure the code tries to do that and does
not artificially add some waiting time.
And make sure if the gst_validate_action_set_done is called from outside
our execution thread, we do not try to execute anything
https://bugzilla.gnome.org/show_bug.cgi?id=743994
With parallel test execution, it will be hard to track which result
relates to which test. Therefore, the test number should be printed
along with the results as well.
Patch 3/4 to implement parallel test execution.
https://bugzilla.gnome.org/show_bug.cgi?id=743063
TestManager will use a Queue to track progress for all tests. This
commit implements a queue inside Test to simplify the transition.
Patch 3/4 to make TestManager handle waiting for processes instead of
expecting each Test to do it.
https://bugzilla.gnome.org/show_bug.cgi?id=743063
wait_process will be moved to TestManager, so the values used to track
process update must remain inside Test.
Patch 2/4 to make TestManager handle waiting for processes instead of
expecting each Test to do it.
This makes each Test handle its own logfile, allowing the Reporter to
work on multiple tests at the same time.
Patch 5/5 to move logfile handling out of Reporter and into Test.
Instead of saving the current Test in Reporter for every test, use
function parameters to achieve the same goal.
Patch 2/5 to move logfile handling out of Reporter and into Test.
+ And implement a corrupt-socket-recv action
+ Only compile this on Linux, LD_PRELOAD won't work on Windows.
For now the registering of the action is done through
a call to socket_interposer_init, this will get better
when we refactor the action logic.
https://bugzilla.gnome.org/show_bug.cgi?id=743871
Allow log file redirection through the new --redirect-logs parameter.
Keep the old --logs-dir stdout/stderr parameter, but reset to the
default logs directory in that case, and set redirect_logs internally.
This also prevents the creation of an stdout/stderr directory for
writing xunit.xml.
https://bugzilla.gnome.org/show_bug.cgi?id=742973
As caps events are downstream, caps set travels from sinks to
sources. Adding pending setcaps values to sink pads makes no sense
as when a new caps is set on the sink it would compare with values
currently set on the source pad, causing a critical failure when
renegotiation happens.
With the testsuite format you will get a setup_tests(tests_manager,
options) function called for each TestManager.
The function will have the exact same role as with old config
file but with a clean API and not magic global variables.
This implies that we need default blacklist to be directly set
on the TestManager and not on options.blacklisted_test
The default testsuite implementation should belong to the default
asset repo where we have the corresponding knowledge.
We should style manage a sensible list of known blacklisted tests,
encoding profiles, and generators in GstValidate itself and allow testsuite
actual implementations to easily use them though the register_default_*
methods.
This allow us to be able to remove the ugly execfile() call.
If the user did not specify any playback time we should be able to
execute actions even if the pipeline can't answer the position query
+ Make simpler to read the conditions of an action execution
The ->execute function now return a GstValidateExecuteActionReturn
which can be set as ASYNC in order to tell the scenario that the action
will be executed asynchronously, when the action is done, the caller is
responsible for calling gst_validate_action_set_done(); so that the
scenario keeps going on.
In this commit we make sure that the old API keeps working as
GST_VALIDATE_EXECUTE_ACTION_ERROR == FALSE and
GST_VALIDATE_EXECUTE_ACTION_OK == TRUE
Morevover GstValidateExecuteActionReturn is just a define
API:
+ gst_validate_action_set_done
+ GstValidateExecuteActionReturn
https://bugzilla.gnome.org/show_bug.cgi?id=739854
We had quite a bit of code dedicated to handled GstPipeline monitoring
inside GstValidateBinMonitor, cleanly split that code into a new object
type
https://bugzilla.gnome.org/show_bug.cgi?id=740704
Depending on the type of event where the bug occurs,
it is not the same issue type. That allows us to have
much precise reports, and better explain the user
where the issue stands.
By default an action has no playback-time, this makes it actionable
immediatly.
When no playback-time is set on a set-property action, it will
be activated the moment the element is added in the pipeline.
