Commit graph

72 commits

Author SHA1 Message Date
Thibault Saunier
f4db183b2b validate:launcher: Properly handle libsdir when gst-validate is installed
+ Fix the _in_devel function
+ Install the validate default testsuite implementation in the right
place
2014-08-12 13:16:43 +02:00
Thibault Saunier
f6c62d071c validate:launcher: Let testsuite know the actual file in which they are 2014-08-12 13:16:43 +02:00
Thibault Saunier
bf30bf4fd7 validate:launcher: Add the logic of needed env variables in tests 2014-08-12 13:16:43 +02:00
Thibault Saunier
20c28def3c validate:launcher: Handle stdout/stderr as possible logfiles
Allowing people to get all the logs in the terminal
2014-08-12 13:16:43 +02:00
Thibault Saunier
702992cf7a validate:launcher: Disable racy HLS tests
+ Add need-clock-synk to switch_audio_track_while_paused as it relies on
the clock sync to pause and then display subtitles
2014-08-06 09:08:36 +02:00
Thibault Saunier
b9712e336e validate:launcher: Fix test number print 2014-08-04 13:02:46 +02:00
Thibault Saunier
1aa0d4eb4a validate:launcher: Allow informing minimum media duration in scenarios
Allowing the launcher to avoid running tests on medias that are not long
enough
2014-07-23 20:43:57 +02:00
Thibault Saunier
386b572066 validate: launcher: Force clock syncronization for some scenarios
In some cases it is necessary that the clock is sync so that all the
actions can be executed.
2014-07-23 17:49:44 +02:00
Thibault Saunier
40003689df validate: Launcher: Make sure tests are always executed in same order 2014-07-23 13:48:44 +02:00
Thibault Saunier
1cdbba4327 validate: Launcher: Fix a backtrace using self in a @staticmethod 2014-07-22 15:50:47 +02:00
Thibault Saunier
cf540d002b validate:launcher: Fix a backtrace using an undefined method 2014-07-18 16:28:49 +02:00
Thibault Saunier
3cfa480eba valdate:launcher: Do not refer to self in @staticmethod
There is no self in there.
2014-07-17 17:04:59 +02:00
Thibault Saunier
f5454f07f0 validate:launcher: Properly check that encoded files have the exact wanted format 2014-07-16 15:47:58 +02:00
Thibault Saunier
fe9ed41d6c validate:launcher: Add a method to create a GstValidateMediaDescriptor from a uri 2014-07-16 15:47:58 +02:00
Thibault Saunier
265688fcd6 validate:launcher: Move MediaFormatCombination to baseclasses.py
+ Add some simple helpers
2014-07-16 15:47:58 +02:00
Thibault Saunier
f1163d61ba validate:launcher: Implement a GstValidateEncodingTestInterface class
Allowing code to be shared between apps that run rendering tests
2014-07-16 15:47:58 +02:00
Thibault Saunier
d61c0d6166 validate:launcher: Make a MediaDescriptor baseclass to be used by any application 2014-07-16 15:47:58 +02:00
Thibault Saunier
87cc7da213 validate:launcher: Move the MediaDescriptor class to the baseclasses.py file 2014-07-16 15:47:58 +02:00
Thibault Saunier
af383ad0c0 validate:launcher: Move the notion of test generator to the baseclasses
This can be very usefull for all the TestManager and thus exposes a
higher level API for test writers.
2014-07-07 11:27:34 +02:00
Thibault Saunier
f10ff2b610 validate:launcher: Add a way to create test suite outside the three
+ Make sure to namespace the API
+ Remove cruft about G_V_PROTOCOL_VIDEO_RESTRICTION_CAPS
2014-07-07 11:27:34 +02:00
Lubosz Sarnecki
1043aa3f55 python: change shebangs to python2 2014-05-19 10:21:07 +02:00
Thibault Saunier
8a295da795 validate: Keep scenario discovering logs in a file 2014-05-07 12:44:51 +02:00
Thibault Saunier
cdb8822ae3 validate: Add the notion of "long" tests so that we can avoid some test to be run if they are too long 2014-05-07 11:30:09 +02:00
Thibault Saunier
2077e76aa2 validate: Add a gst-validate-launcher documentation 2014-05-02 18:30:23 +02:00
Thibault Saunier
447b299dab validate:launcher: Port OptionParser to ArgParse 2014-05-02 18:30:23 +02:00
Thibault Saunier
804cdb5ad2 validate: Handle per file special scenarios
When a file is int the same folder as a media file and has a name like:
mediafilename.mkv.scenarios_name.scenario  we run that scenario on that
particular file
2014-05-02 18:30:23 +02:00
Thibault Saunier
da42500df7 validate: Add a scenarios that switchs subtitle track
+ Make it easier and cleaner to tell that a switch is actually disabling
a track type.

And run the scenario in gst-validate-launcher by default
2014-05-02 18:30:22 +02:00
Thibault Saunier
28bd6ee17a launcher: Now using git annex to handle media files 2014-04-28 13:08:17 +02:00
Thibault Saunier
a3a132489a Revert "validate:launcher: Always put gst-validate result as stderr in reports"
This reverts commit 925ff7542b.

Actually jenkins never  truncates on failure stacktrace... we do not
want to set gst-validate as failure stacktrace in our results. That
commit was not usefull.
2014-04-26 09:16:26 +02:00
Thibault Saunier
925ff7542b validate:launcher: Always put gst-validate result as stderr in reports
This way jenkins will always keep the information in its database even
if the test passes
2014-04-26 08:11:20 +02:00
Thibault Saunier
bd936dae4c validate:launcher: Properly set error message when sending EOS did not work 2014-04-25 14:51:28 +02:00
Thibault Saunier
be72ec5f6a validate:launcher: Add support for audio track switching scenario 2014-04-25 14:51:28 +02:00
Thibault Saunier
8b4542540c validate:launcher: Make it possible to run any scenario test in gst-validate 2014-04-25 14:33:06 +02:00
Thibault Saunier
7133e4b4e0 validate:launcher: Use the new validatelog file
Making the output cleaner and clearer in junit XML file
2014-04-25 14:33:06 +02:00
Thibault Saunier
02abb60a67 validate:launcher: Send SIGINT signal instead of killing the subprocess
This way we get the result from GstValidate even on timeouts
2014-04-22 10:49:13 +02:00
Thibault Saunier
f1851235d5 validate:launcher: Print the number of the test being run 2014-04-02 19:17:24 +02:00
Thibault Saunier
3d8201d52b validate:launcher: Properly handle missing scenarios on the system 2014-03-28 15:41:30 +01:00
Thibault Saunier
bc40235600 validate:launcher: Handle the fact that win32 apps end with .exe 2014-03-28 15:00:01 +01:00
Thibault Saunier
3408869f49 validate:launcher: Put gst logs in a specific file
+ Make default timeout 30seconds just in case.
2014-03-28 10:40:19 +01:00
Thibault Saunier
8fdf84b084 validate: launcher: Let the use debug on test fail
When a test timeouts, let the user know about the subprocess etc,
and let him possibly connect gdb to it.
2014-03-26 20:08:43 +01:00
Thibault Saunier
07b968a4f6 validate:tools: Clean test between runs when running forever 2014-02-19 14:34:10 +01:00
Thibault Saunier
f6bf92cbc9 validate: Do not check result furthers if alredy set as passing 2014-02-19 14:34:10 +01:00
Thibault Saunier
1189c48ff8 validate:launcher: Avoid running useless tests
For example we should not check if duration are equal when transcoding
with scenario set.

Also checking if position is in the seeked segment should be done at
a lower level
2014-02-18 21:07:31 +01:00
Thibault Saunier
d377158b42 validate:launcher: Avoid seeking in output files to parse them
Tihs creates issue and missing content.
2014-02-18 21:07:31 +01:00
Thibault Saunier
c3adb05b2a validate:tools: Handle cases were EOS does not stop the pipeline in the launcher
+ Fix parsing of GstClockTime
+ Avoid using play_15s scenario when not necessary
2014-02-18 21:07:31 +01:00
Thibault Saunier
90d9a686d7 validate: tools: Use the new scenario discovering fearure in the launcher 2014-02-18 21:07:31 +01:00
Thibault Saunier
904cdd6411 validate:tools: Rework the way we handle options
Make groups so it is easier for users to find what they look for
By default have 1 single directory where everything is oututed
(main-dir)
Add a way to specify how and where to look for remote assets
2014-02-18 21:07:31 +01:00
Thibault Saunier
ead3eb98a4 validate:tools:launcher: Take into account the position value when rendering
When rendering a files we try to use the size of the outputed file to
determine wether we are timeout or not, but if that fails
try to check the position
2014-02-18 21:07:31 +01:00
Thibault Saunier
ca12b78be3 validate:tools: Use regex for parsing when appropriate 2014-02-18 21:07:31 +01:00
Thibault Saunier
2f689fab73 validate: toold: Properly define scenario properties 2014-02-18 21:07:30 +01:00