Commit graph

742 commits

Author SHA1 Message Date
Mathieu Duponchelle
e5976cad43 validate: do not compile for android. 2015-02-03 15:51:34 +01:00
Mathieu Duponchelle
4ce91b98f0 validate: Implement a fault injection library.
+ And implement a corrupt-socket-recv action
+ Only compile this on Linux, LD_PRELOAD won't work on Windows.

For now the registering of the action is done through
a call to socket_interposer_init, this will get better
when we refactor the action logic.

https://bugzilla.gnome.org/show_bug.cgi?id=743871
2015-02-03 15:30:54 +01:00
Mathieu Duponchelle
89048ad862 validate: tests more issues with caps.
https://bugzilla.gnome.org/show_bug.cgi?id=743387
2015-01-26 18:08:47 +01:00
Mathieu Duponchelle
bf2c949aee validate: Add a test case for caps missing field.
+ Make the fake decoder have video/x-raw caps.

https://bugzilla.gnome.org/show_bug.cgi?id=743387
2015-01-26 18:08:47 +01:00
Mathieu Duponchelle
887349167b validate: prepare tests for port to tracers backend.
https://bugzilla.gnome.org/show_bug.cgi?id=743387
2015-01-26 18:08:47 +01:00
Mathieu Duponchelle
165e35b3f0 bin-monitor: add itself as gobject data. 2015-01-22 22:07:37 +01:00
Ramiro Polla
b29f378296 validate: launcher: Fix test log header output
Write log file header before running tests, instead of overwriting the
file afterwards.

https://bugzilla.gnome.org/show_bug.cgi?id=742966
2015-01-16 16:24:16 +01:00
Ramiro Polla
9c3606a867 validate: launcher: Always create log files
Create log files even when stdout redirection is enabled.
This commit partially reverts 20c28de.

https://bugzilla.gnome.org/show_bug.cgi?id=742973
2015-01-16 16:24:13 +01:00
Ramiro Polla
4e0388c631 validate: launcher: Introduce new parameter for log file redirecting
Allow log file redirection through the new --redirect-logs parameter.
Keep the old --logs-dir stdout/stderr parameter, but reset to the
default logs directory in that case, and set redirect_logs internally.
This also prevents the creation of an stdout/stderr directory for
writing xunit.xml.

https://bugzilla.gnome.org/show_bug.cgi?id=742973
2015-01-16 16:24:13 +01:00
Thiago Santos
10fe72aa52 pad-monitor: plug caps leak on iterator resync 2015-01-09 14:04:16 -03:00
Thiago Santos
389f6d96b4 pad-monitor: use the same filter caps when querying downstream caps
To avoid comparing the real result that has been filtered against
a much larger caps that contains all possibilities.
2015-01-09 14:04:13 -03:00
Thibault Saunier
0c48665458 validate: tests: disable g_log handler
It messes up our own failures counter

And pass test-utils into gst-indent
2014-12-09 10:11:02 +01:00
Thiago Santos
6df63915c2 tests: padmonitor: disable glog handling
It messes up our own failures counter
2014-12-08 18:53:55 -03:00
Thiago Santos
7b4df44f61 pad-monitor: get correct caps to check for proxied fields in caps queries
Elements should proxy the peer element's caps fields and not what they
have currently set on their pads when replying to a caps query
2014-12-08 17:28:47 -03:00
Thiago Santos
af2707d3a3 pad-monitor: Only add pending caps fields for source pads
As caps events are downstream, caps set travels from sinks to
sources. Adding pending setcaps values to sink pads makes no sense
as when a new caps is set on the sink it would compare with values
currently set on the source pad, causing a critical failure when
renegotiation happens.
2014-12-08 17:28:47 -03:00
Thibault Saunier
1e39db18ad validate: launcher: Take the timeout as ref timeout to compute hard_timeout
when it is provided.
2014-12-08 18:38:46 +01:00
Thibault Saunier
5beaf5dfa8 validate: launcher: Set a hard timeout on GstValidate tests if we know the duration 2014-12-08 16:34:48 +01:00
Thibault Saunier
a2abf628dc validate: launcher: Force kill subprocess when done with them
Making sure that we do not end up having spurious subprocess around
2014-12-08 16:19:20 +01:00
Thiago Santos
d25fb034fa launcher: baseclass: add missing parameter
Fixes "NameError: global name 'options' is not defined"
2014-12-08 08:43:57 -03:00
Thibault Saunier
8450dff17d validate: launcher: Properly handle non default main dir
for the case of the new testsuite files
2014-12-08 10:10:04 +01:00
Thibault Saunier
5bfd579bf4 validate: launcher: Make the gstvalidate application a python module 2014-12-07 18:32:37 +01:00
Thibault Saunier
ce02887627 validate: Remove remaining reference to launcher/apps/validate
It has been removed now.
2014-12-06 10:53:37 +01:00
Thibault Saunier
1bc32a888f validate: Avoid assert removing an already removed signal handler
And, make sure that we set the return value != 0 when we receive
SIGINT
2014-12-05 20:51:44 +01:00
Thibault Saunier
272032c0b0 validate: Handle setting the HTTP server local path from testsuites 2014-12-05 20:51:44 +01:00
Thibault Saunier
22fba81138 validate: Rename gst-qa-assets to gst-integration-testsuites 2014-12-05 20:51:43 +01:00
Thibault Saunier
130a2892b1 validate: print execution of set_subtitles actions 2014-12-05 19:23:43 +01:00
Thibault Saunier
3dff417fdf validate: Factor out an LauncherConfig class type to handle configurations
Allowing us to more simply define default value and expose an API on
top of it
2014-12-05 19:23:43 +01:00
Thibault Saunier
515c8405f0 validate: Remove file specific blacklisted tests 2014-12-05 16:16:53 +01:00
Thibault Saunier
988e9a370e validate: Let the user know when new tests are added, or tests are REMOVED 2014-12-05 16:16:53 +01:00
Thibault Saunier
d38307a073 validate: Add a cleaner API to setup tests in testsuite files
With the testsuite format you will get a setup_tests(tests_manager,
options) function called for each TestManager.

The function will have the exact same role as with old config
file but with a clean API and not magic global variables.

This implies that we need default blacklist to be directly set
on the TestManager and not on options.blacklisted_test
2014-12-05 16:16:53 +01:00
Thibault Saunier
b852fbcbfb validate: Add a way to sync all assets, including big ones 2014-12-05 12:17:40 +01:00
Thibault Saunier
71566688bf validate: Remove the default testsuite implementation
The default testsuite implementation should belong to the default
asset repo where we have the corresponding knowledge.

We should style manage a sensible list of known blacklisted tests,
encoding profiles, and generators in GstValidate itself and allow testsuite
actual implementations to easily use them though the register_default_*
methods.

This allow us to be able to remove the ugly execfile() call.
2014-12-05 12:17:40 +01:00
Thibault Saunier
36aed39259 validate: Make sure to at least listen to GStreamer and GLib g_logs
If somewhere else someone is overriding the g_log default handler,
we would not get notified of anything.
2014-12-05 12:16:55 +01:00
Thibault Saunier
ccbcc04498 validate: Disable coloration of GST_DEBUG logs when we have no-color
Do that only when those logs are not saved to a file
2014-12-05 12:16:54 +01:00
Thibault Saunier
582f1cfc06 validate: Factor out a function to print action types parametters
+ Remove playback-type from the list and just print it
2014-12-05 12:16:54 +01:00
Thiago Santos
db67a92b3f pad-monitor: do not enforce caps querying rules for converters
Some encoders/decoders can also be converters, do not enforce
caps proxying rules for them
2014-11-28 17:21:23 -03:00
Vineeth T M
163d286f5b validate: fix typo in documentation
There are some typing mistakes in gst-validate-launcher --help
Hence fixing the same.

https://bugzilla.gnome.org/show_bug.cgi?id=740833
2014-11-28 11:16:09 +01:00
Thibault Saunier
b5219eaceb validate: Handle unlimited tests duration
Running full length scenario when the user asks
2014-11-25 19:41:01 +01:00
Thibault Saunier
c952886813 validate: Already having a monitor is no error 2014-11-25 19:41:01 +01:00
Thibault Saunier
1753afbe8d validate: pre commit hook: Do not try to run pep8 on non python files! 2014-11-25 19:41:01 +01:00
Thibault Saunier
62bf27f5e3 validate: Add scub_*_seeking_full scenarios
Which basically do the same thing as scrub_*_seeking but during
throughout the whole duration of the media
2014-11-25 19:41:00 +01:00
Thibault Saunier
b56ebc8a43 validate: Enhance documentation 2014-11-25 19:41:00 +01:00
Thibault Saunier
65eb14de36 validate: Add an option to update all .media_info files 2014-11-25 19:41:00 +01:00
Thibault Saunier
11c49a7db8 validate:scenario: Execute actions without playback time without a valid position
If the user did not specify any playback time we should be able to
execute actions even if the pipeline can't answer the position query

+ Make simpler to read the conditions of an action execution
2014-11-25 19:41:00 +01:00
Thibault Saunier
cdfa1ee61b validate:scenario: Properly handle ASYNC action execution in the API
The ->execute function now return a GstValidateExecuteActionReturn
which can be set as ASYNC in order to tell the scenario that the action
will be executed asynchronously, when the action is done, the caller is
responsible for calling gst_validate_action_set_done(); so that the
scenario keeps going on.

In this commit we make sure that the old API keeps working as
GST_VALIDATE_EXECUTE_ACTION_ERROR == FALSE and
GST_VALIDATE_EXECUTE_ACTION_OK == TRUE

Morevover GstValidateExecuteActionReturn is just a define

API:
    + gst_validate_action_set_done
    + GstValidateExecuteActionReturn

https://bugzilla.gnome.org/show_bug.cgi?id=739854
2014-11-25 19:41:00 +01:00
Thibault Saunier
4cbcb97258 validate: Add a GstValidateActionTypeFlag flag
Allowing us to define action types more in detail.
Keep backward compatibility, at least with the C API

https://bugzilla.gnome.org/show_bug.cgi?id=739854
2014-11-25 19:29:13 +01:00
Thibault Saunier
fcb0f20828 validate: Add a GstValidatePipelineMonitor subclass
We had quite a bit of code dedicated to handled GstPipeline monitoring
inside GstValidateBinMonitor, cleanly split that code into a new object
type

https://bugzilla.gnome.org/show_bug.cgi?id=740704
2014-11-25 19:29:13 +01:00
Thibault Saunier
36a2d61068 validate:launcher: Force clock sync for some protocols
In HLS for example, not having clock sync might lead to races and failures
do not test that for now
2014-11-20 12:09:29 +01:00
Thibault Saunier
057a864223 validate: Don't fail getting master report from a ghostpad without target 2014-11-20 12:09:26 +01:00
Thibault Saunier
6300fb9602 validate: Print current position even if we do not know the rate
That could cause gst-validate-launcher to wrongly concider tests
as timeout
2014-11-19 20:05:57 +01:00