Commit graph

55 commits

Author SHA1 Message Date
Thibault Saunier 87cc7da213 validate:launcher: Move the MediaDescriptor class to the baseclasses.py file 2014-07-16 15:47:58 +02:00
Thibault Saunier af383ad0c0 validate:launcher: Move the notion of test generator to the baseclasses
This can be very usefull for all the TestManager and thus exposes a
higher level API for test writers.
2014-07-07 11:27:34 +02:00
Thibault Saunier f10ff2b610 validate:launcher: Add a way to create test suite outside the three
+ Make sure to namespace the API
+ Remove cruft about G_V_PROTOCOL_VIDEO_RESTRICTION_CAPS
2014-07-07 11:27:34 +02:00
Lubosz Sarnecki 1043aa3f55 python: change shebangs to python2 2014-05-19 10:21:07 +02:00
Thibault Saunier 8a295da795 validate: Keep scenario discovering logs in a file 2014-05-07 12:44:51 +02:00
Thibault Saunier cdb8822ae3 validate: Add the notion of "long" tests so that we can avoid some test to be run if they are too long 2014-05-07 11:30:09 +02:00
Thibault Saunier 2077e76aa2 validate: Add a gst-validate-launcher documentation 2014-05-02 18:30:23 +02:00
Thibault Saunier 447b299dab validate:launcher: Port OptionParser to ArgParse 2014-05-02 18:30:23 +02:00
Thibault Saunier 804cdb5ad2 validate: Handle per file special scenarios
When a file is int the same folder as a media file and has a name like:
mediafilename.mkv.scenarios_name.scenario  we run that scenario on that
particular file
2014-05-02 18:30:23 +02:00
Thibault Saunier da42500df7 validate: Add a scenarios that switchs subtitle track
+ Make it easier and cleaner to tell that a switch is actually disabling
a track type.

And run the scenario in gst-validate-launcher by default
2014-05-02 18:30:22 +02:00
Thibault Saunier 28bd6ee17a launcher: Now using git annex to handle media files 2014-04-28 13:08:17 +02:00
Thibault Saunier a3a132489a Revert "validate:launcher: Always put gst-validate result as stderr in reports"
This reverts commit 925ff7542b.

Actually jenkins never  truncates on failure stacktrace... we do not
want to set gst-validate as failure stacktrace in our results. That
commit was not usefull.
2014-04-26 09:16:26 +02:00
Thibault Saunier 925ff7542b validate:launcher: Always put gst-validate result as stderr in reports
This way jenkins will always keep the information in its database even
if the test passes
2014-04-26 08:11:20 +02:00
Thibault Saunier bd936dae4c validate:launcher: Properly set error message when sending EOS did not work 2014-04-25 14:51:28 +02:00
Thibault Saunier be72ec5f6a validate:launcher: Add support for audio track switching scenario 2014-04-25 14:51:28 +02:00
Thibault Saunier 8b4542540c validate:launcher: Make it possible to run any scenario test in gst-validate 2014-04-25 14:33:06 +02:00
Thibault Saunier 7133e4b4e0 validate:launcher: Use the new validatelog file
Making the output cleaner and clearer in junit XML file
2014-04-25 14:33:06 +02:00
Thibault Saunier 02abb60a67 validate:launcher: Send SIGINT signal instead of killing the subprocess
This way we get the result from GstValidate even on timeouts
2014-04-22 10:49:13 +02:00
Thibault Saunier f1851235d5 validate:launcher: Print the number of the test being run 2014-04-02 19:17:24 +02:00
Thibault Saunier 3d8201d52b validate:launcher: Properly handle missing scenarios on the system 2014-03-28 15:41:30 +01:00
Thibault Saunier bc40235600 validate:launcher: Handle the fact that win32 apps end with .exe 2014-03-28 15:00:01 +01:00
Thibault Saunier 3408869f49 validate:launcher: Put gst logs in a specific file
+ Make default timeout 30seconds just in case.
2014-03-28 10:40:19 +01:00
Thibault Saunier 8fdf84b084 validate: launcher: Let the use debug on test fail
When a test timeouts, let the user know about the subprocess etc,
and let him possibly connect gdb to it.
2014-03-26 20:08:43 +01:00
Thibault Saunier 07b968a4f6 validate:tools: Clean test between runs when running forever 2014-02-19 14:34:10 +01:00
Thibault Saunier f6bf92cbc9 validate: Do not check result furthers if alredy set as passing 2014-02-19 14:34:10 +01:00
Thibault Saunier 1189c48ff8 validate:launcher: Avoid running useless tests
For example we should not check if duration are equal when transcoding
with scenario set.

Also checking if position is in the seeked segment should be done at
a lower level
2014-02-18 21:07:31 +01:00
Thibault Saunier d377158b42 validate:launcher: Avoid seeking in output files to parse them
Tihs creates issue and missing content.
2014-02-18 21:07:31 +01:00
Thibault Saunier c3adb05b2a validate:tools: Handle cases were EOS does not stop the pipeline in the launcher
+ Fix parsing of GstClockTime
+ Avoid using play_15s scenario when not necessary
2014-02-18 21:07:31 +01:00
Thibault Saunier 90d9a686d7 validate: tools: Use the new scenario discovering fearure in the launcher 2014-02-18 21:07:31 +01:00
Thibault Saunier 904cdd6411 validate:tools: Rework the way we handle options
Make groups so it is easier for users to find what they look for
By default have 1 single directory where everything is oututed
(main-dir)
Add a way to specify how and where to look for remote assets
2014-02-18 21:07:31 +01:00
Thibault Saunier ead3eb98a4 validate:tools:launcher: Take into account the position value when rendering
When rendering a files we try to use the size of the outputed file to
determine wether we are timeout or not, but if that fails
try to check the position
2014-02-18 21:07:31 +01:00
Thibault Saunier ca12b78be3 validate:tools: Use regex for parsing when appropriate 2014-02-18 21:07:31 +01:00
Thibault Saunier 2f689fab73 validate: toold: Properly define scenario properties 2014-02-18 21:07:30 +01:00
Thibault Saunier cd18609763 validate: Rename simple_backward to reverse_playback as this is what it does 2014-02-18 21:07:30 +01:00
Thibault Saunier 5e96c79000 validate:tools: Make use of the new seek_with_stop scenario 2014-02-18 21:07:30 +01:00
Thibault Saunier cd098eb28c validate:tools: Implement the logic of validate ouput parsing in the baseclass
+ Add some logic to check that we are mot playing outside wanted segment
2014-02-18 21:07:30 +01:00
Thibault Saunier 2c52d6374c tools:validate: Make default blacklist handled by managers themselves 2014-02-18 21:07:30 +01:00
Thibault Saunier 1a9119cb33 validate: tools: Add a --fatal-error option to the launcher 2014-02-12 11:23:25 +01:00
Thibault Saunier 2520f4f110 validate: tools: Implement the notion of hard timeout
Allowing to define timeout that is not relative to the last observed number.
2014-02-12 11:23:25 +01:00
Thibault Saunier 2391a56714 validate: tools: Create a class for scenarios 2014-02-12 11:23:25 +01:00
Thibault Saunier b3c71bba02 validate:tools: Blacklist some scenario/protocol combinations
And add the option for user to easilly blacklist tests
2014-02-06 12:02:59 +01:00
Thibault Saunier 3bce8f8966 validate:tools: Print test result in the terminal after the end of each test 2014-02-06 12:02:59 +01:00
Thibault Saunier 299a677f09 validate:tools: Add an option to run testforever 2014-02-06 12:02:59 +01:00
Thibault Saunier d000ca62a4 validate:tools: Use the same semantic for all tests classnames 2014-02-06 12:02:59 +01:00
Thibault Saunier c0673d7eea validate:tools: Add support for testing http streams locally 2014-02-06 12:02:59 +01:00
Thibault Saunier 1c614b9ed9 validate:tools: Add informations about the test in the log files 2014-02-06 12:02:58 +01:00
Thibault Saunier a6f3d5270c validate:tools: Remove reference to get_backtrace which is not implemented
+ Enhance Message about launched apps
2014-02-06 12:02:58 +01:00
Thibault Saunier 26e3a9e3c5 validate:tools: Do not duplicated name in the classname in xunit reports 2014-02-06 12:02:58 +01:00
Thibault Saunier 6bd9177205 validate:tools: Properly inform the user about the log location when test fails 2014-02-06 12:02:58 +01:00
Thibault Saunier 8697dd3da4 validate:tools: Add an option to desativate ANSI colors
And enhance some debugging output
2014-02-06 12:02:58 +01:00