Thibault Saunier
5983df47c5
validate:launcher: Cleanup the way we generate tests adding the notion of TestGenerator
...
Making it easier to extend the testsuite.
2014-07-07 11:27:34 +02:00
Thibault Saunier
5c85e4500d
validate:tools: Dot the pipeline on usefull places
...
Meaning on warning and state changes.
2014-07-07 11:27:34 +02:00
Sebastian Dröge
3ecd8352eb
gst-validate: Add $(GIO_LIBS) and $(GIO_CFLAGS) as required
2014-06-28 11:36:27 +02:00
Thibault Saunier
bbe56a4fd9
validate: Fix launcher when running installed
2014-06-17 15:10:41 +02:00
Thibault Saunier
4f01b2bd8a
validate:launcher:reporter: Sort Final report by results
2014-06-16 16:47:18 +02:00
Thibault Saunier
b8e1c3e64c
validate:launcher:ges: Fix rendered duration checking
2014-06-16 16:46:21 +02:00
Edward Hervey
ca44a11429
validate: Run gst-indent on all code
...
so whitespace. much indent. spacing !
2014-06-03 10:05:25 +02:00
Thibault Saunier
0bde7bcacb
validate: Add a scrub_backward_seeking scenario
...
+ Make use of it in ges-launch and do not try to seek while playing in
GES as it is not supported yet
2014-05-27 12:33:08 +02:00
Lubosz Sarnecki
1043aa3f55
python: change shebangs to python2
2014-05-19 10:21:07 +02:00
Thibault Saunier
dae0c4ef81
validate: Add a scenario that switches state many intensively
...
+ Use it by default in the launcher tests
2014-05-07 16:45:27 +02:00
Thibault Saunier
384686a7f7
validate: Minor fix for blacklisted test output formatting
2014-05-07 12:44:51 +02:00
Thibault Saunier
0b4bd7940a
validate:launcher: Add an option to only launch the http server
2014-05-07 12:44:51 +02:00
Thibault Saunier
8a295da795
validate: Keep scenario discovering logs in a file
2014-05-07 12:44:51 +02:00
Thibault Saunier
720b9eff47
validate: Properly check that outputed videos have a correct duration
2014-05-07 11:30:39 +02:00
Thibault Saunier
cdb8822ae3
validate: Add the notion of "long" tests so that we can avoid some test to be run if they are too long
2014-05-07 11:30:09 +02:00
Thibault Saunier
49ab41743b
validate: The 'buffering' variable needs to be static
...
We need its value between bus_callback calls to be the same
2014-05-04 09:30:16 +02:00
Luis de Bethencourt
eabcf8f106
gst-validate: some static variables can be local
...
buffering is only used inside the bus_callback, so it can have that local
scope. same thing with ret which is only used in the main function.
2014-05-02 17:25:07 -04:00
Luis de Bethencourt
0b01b578ce
gst-validate: small typo in usage summary
2014-05-02 16:53:51 -04:00
Thibault Saunier
bb42d287e7
validate: Fix the name of the sintel blacklisting
2014-05-02 19:00:49 +02:00
Thibault Saunier
fbcee57902
validate: Implement frame by frame writing in the media descriptor writer
...
+ Add an option to fully parse media files in the gst-validate-media-check tool
2014-05-02 18:30:23 +02:00
Thibault Saunier
5d200f8819
validate: Add a scenario that disable subtitles
...
+ Clean the sythax to define switch-track action that actually
desactivate the track
2014-05-02 18:30:23 +02:00
Thibault Saunier
867bfec188
validate: Add a scenario that switches subtitle track while paused
...
+ Integrate it in the launcher
2014-05-02 18:30:23 +02:00
Thibault Saunier
872472feb3
validate:launcher: Sensibly simplify scenario handling
2014-05-02 18:30:23 +02:00
Thibault Saunier
9c5009c724
validate: Use ges-launch recursing path new feature
...
And fix path to URI conversion
2014-05-02 18:30:23 +02:00
Thibault Saunier
2077e76aa2
validate: Add a gst-validate-launcher documentation
2014-05-02 18:30:23 +02:00
Thibault Saunier
447b299dab
validate:launcher: Port OptionParser to ArgParse
2014-05-02 18:30:23 +02:00
Thibault Saunier
5e2b577372
validate: Can not do reverse playback on sintel sample
...
+ Minor improvement in the CLI
2014-05-02 18:30:23 +02:00
Thibault Saunier
97482e2512
validate:launcher: Do not except meaningless argument in ges-launch
2014-05-02 18:30:23 +02:00
Thibault Saunier
804cdb5ad2
validate: Handle per file special scenarios
...
When a file is int the same folder as a media file and has a name like:
mediafilename.mkv.scenarios_name.scenario we run that scenario on that
particular file
2014-05-02 18:30:23 +02:00
Thibault Saunier
1ffb6b4e1c
validate: Allow specifying scenarios to parse when lisiting them
...
It used to only handle the scenario present in proper paths, we
also need to handle special scenarios provided by users on the fly
2014-05-02 18:30:22 +02:00
Thibault Saunier
f165fb41d0
validate: Handle ERROR on the bus when monitoring the pipeline
...
This way the user get a clear information in the report about the issue
+ sensibly cleanup code
2014-05-02 18:30:22 +02:00
Thibault Saunier
da42500df7
validate: Add a scenarios that switchs subtitle track
...
+ Make it easier and cleaner to tell that a switch is actually disabling
a track type.
And run the scenario in gst-validate-launcher by default
2014-05-02 18:30:22 +02:00
Thibault Saunier
3281add6b5
validate: Add a switch_audio_track_while_paused scenario
...
And run it as a default
2014-05-02 18:30:22 +02:00
Thibault Saunier
a62560bc81
validate: Add an action to set an external URI file on playbin at runtime
2014-05-02 18:30:22 +02:00
Thibault Saunier
2a9e0824b2
validate: Override switch_track action when using a playbin
...
And use the playbin feature for that when the pipeline is based on playbin
2014-05-02 18:30:22 +02:00
Thibault Saunier
28bd6ee17a
launcher: Now using git annex to handle media files
2014-04-28 13:08:17 +02:00
Thibault Saunier
a3a132489a
Revert "validate:launcher: Always put gst-validate result as stderr in reports"
...
This reverts commit 925ff7542b
.
Actually jenkins never truncates on failure stacktrace... we do not
want to set gst-validate as failure stacktrace in our results. That
commit was not usefull.
2014-04-26 09:16:26 +02:00
Thibault Saunier
925ff7542b
validate:launcher: Always put gst-validate result as stderr in reports
...
This way jenkins will always keep the information in its database even
if the test passes
2014-04-26 08:11:20 +02:00
Thibault Saunier
bd936dae4c
validate:launcher: Properly set error message when sending EOS did not work
2014-04-25 14:51:28 +02:00
Thibault Saunier
da6dc3882b
validate:launcher: Add a way to specify tests filtering only on defaults
2014-04-25 14:51:28 +02:00
Thibault Saunier
be72ec5f6a
validate:launcher: Add support for audio track switching scenario
2014-04-25 14:51:28 +02:00
Thibault Saunier
8b4542540c
validate:launcher: Make it possible to run any scenario test in gst-validate
2014-04-25 14:33:06 +02:00
Thibault Saunier
8035094128
validate:launcher: Do not run reverse playback on mpegts files
2014-04-25 14:33:06 +02:00
Thibault Saunier
fba15f71c6
validate:launcher: Cleanup media descriptor usage
2014-04-25 14:33:06 +02:00
Thibault Saunier
81b0c74bfa
validate:launch: Port to the new media_info format
2014-04-25 14:33:06 +02:00
Thibault Saunier
d280d0dbc9
validate: Add a media-descriptor parser and writer
2014-04-25 14:33:06 +02:00
Thibault Saunier
2d7d03d2f7
validate: Add a 'stop' action to stop a pipeline
...
It uses the GST_MESSAGE_REQUEST state with the scenario as a source
so that application can stop running when they receive it on the bus.
2014-04-25 14:33:06 +02:00
Thibault Saunier
7133e4b4e0
validate:launcher: Use the new validatelog file
...
Making the output cleaner and clearer in junit XML file
2014-04-25 14:33:06 +02:00
Thibault Saunier
38eda37341
validate: Handle position printing at the monitor level
...
Instead of replicating that code all around
2014-04-23 11:27:44 +02:00
Thibault Saunier
810e432da2
validate: Add printing utilities
...
Allowing the user to print everyting in a file through the
GST_VALIDATE_FILE env variable
2014-04-23 11:16:32 +02:00
Thibault Saunier
02abb60a67
validate:launcher: Send SIGINT signal instead of killing the subprocess
...
This way we get the result from GstValidate even on timeouts
2014-04-22 10:49:13 +02:00
Thibault Saunier
7f54c5dba8
validate:launcher: Always set sync=True on fakesink on playback pipelines
...
This way we are in closer condition of real sink playback.
+ some minor cleanup in gst-validate.c
2014-04-22 09:42:59 +02:00
Thibault Saunier
bbbd5fe2d0
validate:launcher: ring-buffer-max-size is in bytes
2014-04-17 12:58:48 +02:00
Thibault Saunier
ad25b4d160
validate: Handle g_log errors at the gst-validate level
2014-04-17 12:23:04 +02:00
Thibault Saunier
f45524a3cc
validate:launcher: Fix default blacklist management
2014-04-15 15:26:36 +02:00
Thibault Saunier
f1851235d5
validate:launcher: Print the number of the test being run
2014-04-02 19:17:24 +02:00
Thibault Saunier
86df60e16f
validate:launcher: Fix mixup in media_check tests expected file path
2014-04-02 12:12:11 +02:00
Thibault Saunier
78f91ae8da
validate:launcher: Flush stdout each time we print
...
So everything gets printed on time on windows and jenkins
2014-03-31 14:37:51 +02:00
Thibault Saunier
36a0f6a674
validate: launcher: Use the ConfigPraser object everywhere for file_infos
2014-03-31 11:03:48 +02:00
Thibault Saunier
3d8201d52b
validate:launcher: Properly handle missing scenarios on the system
2014-03-28 15:41:30 +01:00
Thibault Saunier
339703d2eb
validate:launcher: Handle windows path to construct arguments
2014-03-28 15:00:45 +01:00
Thibault Saunier
bc40235600
validate:launcher: Handle the fact that win32 apps end with .exe
2014-03-28 15:00:01 +01:00
Thibault Saunier
3408869f49
validate:launcher: Put gst logs in a specific file
...
+ Make default timeout 30seconds just in case.
2014-03-28 10:40:19 +01:00
Thibault Saunier
8fdf84b084
validate: launcher: Let the use debug on test fail
...
When a test timeouts, let the user know about the subprocess etc,
and let him possibly connect gdb to it.
2014-03-26 20:08:43 +01:00
Thibault Saunier
870df6e4e2
validate:launcher: Do not set sample path to letter in ges-launch
2014-03-26 11:46:48 +01:00
Thibault Saunier
5510b96639
validate:launcher: Start the server only when actually needed to run filtered tests
2014-03-26 11:00:32 +01:00
Thibault Saunier
36f9ba8f52
launcher: Allow user to set media-files directory
...
That was broken by 71dee6c384
2014-03-19 18:43:38 +01:00
Thibault Saunier
6d8ff8a92a
tools:validate: Start printing position on ASYNC_DONE
...
As this is what is done in the scenarios.
2014-03-19 17:45:36 +01:00
Edward Hervey
12ccebe0a9
launcher: Don't hardcode option defaults
...
Since they are relative to other options, we need to post-process them
to get the proper value.
Fixes using the launcher with non-default MAIN_DIR
2014-03-19 18:09:09 +01:00
Edward Hervey
c39a44441e
launcher: Warn if MAIN_DIR isn't present
...
And move blacklist file listing to further down
2014-03-19 17:13:14 +01:00
Edward Hervey
e62a2ce4c4
launcher: No need to start a web server when listing tests
...
It's not needed and makes listing faster.
Also sort the list of tests
2014-03-19 17:04:14 +01:00
Edward Hervey
71dee6c384
launcher: --sync: Only update/clone git repo if specified
...
Allows:
* handling non-git-based asset directory
* working offline
* working without forcing updates
2014-03-19 17:03:05 +01:00
Thibault Saunier
e75e7df0ac
validate: tools: Fix path to media folder
2014-02-19 14:39:30 +01:00
Thibault Saunier
07b968a4f6
validate:tools: Clean test between runs when running forever
2014-02-19 14:34:10 +01:00
Thibault Saunier
f6bf92cbc9
validate: Do not check result furthers if alredy set as passing
2014-02-19 14:34:10 +01:00
Thibault Saunier
1189c48ff8
validate:launcher: Avoid running useless tests
...
For example we should not check if duration are equal when transcoding
with scenario set.
Also checking if position is in the seeked segment should be done at
a lower level
2014-02-18 21:07:31 +01:00
Thibault Saunier
d377158b42
validate:launcher: Avoid seeking in output files to parse them
...
Tihs creates issue and missing content.
2014-02-18 21:07:31 +01:00
Thibault Saunier
32d9d5dc44
validate:launcher: Properly classify test for media check
2014-02-18 21:07:31 +01:00
Thibault Saunier
0449817c11
validate:launcher: Handle issue with unknown framerate in HLS while transcoding
2014-02-18 21:07:31 +01:00
Thibault Saunier
c3adb05b2a
validate:tools: Handle cases were EOS does not stop the pipeline in the launcher
...
+ Fix parsing of GstClockTime
+ Avoid using play_15s scenario when not necessary
2014-02-18 21:07:31 +01:00
Thibault Saunier
90d9a686d7
validate: tools: Use the new scenario discovering fearure in the launcher
2014-02-18 21:07:31 +01:00
Thibault Saunier
4467b27b9f
validate: Add a way to save details about avalaible scenarios in a file
2014-02-18 21:07:31 +01:00
Thibault Saunier
9e66ac00f6
validate: Add a way to add a "description" to scenario files
...
Print details about the descriptions when listing scenario in a KeyFile
format
The description can contain any information about the scenario such as its duration before
EOS, how long the pipeline needs to be so the scenario can be applied...etc
2014-02-18 21:07:31 +01:00
Thibault Saunier
efc6938d01
validate: tools: Init gst-validate before listing scenarios
...
And return 0 when only listing scenarios
2014-02-18 21:07:31 +01:00
Thibault Saunier
904cdd6411
validate:tools: Rework the way we handle options
...
Make groups so it is easier for users to find what they look for
By default have 1 single directory where everything is oututed
(main-dir)
Add a way to specify how and where to look for remote assets
2014-02-18 21:07:31 +01:00
Thibault Saunier
ead3eb98a4
validate:tools:launcher: Take into account the position value when rendering
...
When rendering a files we try to use the size of the outputed file to
determine wether we are timeout or not, but if that fails
try to check the position
2014-02-18 21:07:31 +01:00
Thibault Saunier
10c192f693
validate: Better organize rendered files
2014-02-18 21:07:31 +01:00
Thibault Saunier
ca12b78be3
validate:tools: Use regex for parsing when appropriate
2014-02-18 21:07:31 +01:00
Thibault Saunier
df712e9404
validate:toold: Add a --output-dir parametter
2014-02-18 21:07:31 +01:00
Thibault Saunier
27e3758918
validate:tools: Keep file extension in test classnames
...
+ add test "namespace" in transcoded files
2014-02-18 21:07:31 +01:00
Thibault Saunier
2718923ae5
validate:tools: Add a class to back pipeline creation in gst-validate
2014-02-18 21:07:31 +01:00
Thibault Saunier
5927a6d8d8
validate:tools: Define supported protocols in an enum
2014-02-18 21:07:31 +01:00
Thibault Saunier
2f689fab73
validate: toold: Properly define scenario properties
2014-02-18 21:07:30 +01:00
Thibault Saunier
cd18609763
validate: Rename simple_backward to reverse_playback as this is what it does
2014-02-18 21:07:30 +01:00
Thibault Saunier
1ecd26eef6
validate:tools: Add namespace in apps global variables
...
Avoiding conflicts
2014-02-18 21:07:30 +01:00
Thibault Saunier
932fcaf9b8
validate:tools: Allow user to append paths to medias
2014-02-18 21:07:30 +01:00
Thibault Saunier
5e96c79000
validate:tools: Make use of the new seek_with_stop scenario
2014-02-18 21:07:30 +01:00
Thibault Saunier
cd098eb28c
validate:tools: Implement the logic of validate ouput parsing in the baseclass
...
+ Add some logic to check that we are mot playing outside wanted segment
2014-02-18 21:07:30 +01:00
Thibault Saunier
2c52d6374c
tools:validate: Make default blacklist handled by managers themselves
2014-02-18 21:07:30 +01:00
Thibault Saunier
7ab0447227
validate:tools: Only discover files with media-check
2014-02-18 21:07:30 +01:00
Thibault Saunier
a406ebb4c2
validate:tools: Add a 'discover-only' option to media-check
2014-02-18 21:07:30 +01:00
Thibault Saunier
1a9119cb33
validate: tools: Add a --fatal-error option to the launcher
2014-02-12 11:23:25 +01:00
Thibault Saunier
2520f4f110
validate: tools: Implement the notion of hard timeout
...
Allowing to define timeout that is not relative to the last observed number.
2014-02-12 11:23:25 +01:00
Thibault Saunier
2391a56714
validate: tools: Create a class for scenarios
2014-02-12 11:23:25 +01:00
Thibault Saunier
ba1f428351
validate: tools: Change timeouts depending on used protocol
2014-02-12 11:23:25 +01:00
Thibault Saunier
8b8562ea85
validate:tools: Implement Buffering support in the various tools
2014-02-12 11:23:23 +01:00
Thibault Saunier
b3c71bba02
validate:tools: Blacklist some scenario/protocol combinations
...
And add the option for user to easilly blacklist tests
2014-02-06 12:02:59 +01:00
Thibault Saunier
3bce8f8966
validate:tools: Print test result in the terminal after the end of each test
2014-02-06 12:02:59 +01:00
Thibault Saunier
299a677f09
validate:tools: Add an option to run testforever
2014-02-06 12:02:59 +01:00
Thibault Saunier
d9386f7542
validate: tools: Cleanup the way we return code in position query
2014-02-06 12:02:59 +01:00
Thibault Saunier
d000ca62a4
validate:tools: Use the same semantic for all tests classnames
2014-02-06 12:02:59 +01:00
Thibault Saunier
c0673d7eea
validate:tools: Add support for testing http streams locally
2014-02-06 12:02:59 +01:00
Thibault Saunier
c2e5c10767
validate:tools: Some cleanup in gst-validate test launcher
2014-02-06 12:02:59 +01:00
Thibault Saunier
33a744b688
validate:tools: Add actuall tests for media checking
2014-02-06 12:02:58 +01:00
Thibault Saunier
a3e7342eef
validate: tools: media-check: When comparing with a file just compare
...
We do not want to know if the file is seekable etc, but in that case we
want to see that the results are stable throughout the various runs
Also make sure to report an understandable error if the media file info
could not be parsed
2014-02-06 12:02:58 +01:00
Thibault Saunier
677e111f0b
validate: tools: Do not dot the pipeline every 50ms, it is a bit exessive
2014-02-06 12:02:58 +01:00
Thibault Saunier
d485180074
validate:tools: Return an exit code != 0 if pipeline can't go to playing
...
And give some information to the user about why the return code is !=0
everywhere it happens
2014-02-06 12:02:58 +01:00
Thibault Saunier
1c614b9ed9
validate:tools: Add informations about the test in the log files
2014-02-06 12:02:58 +01:00
Thibault Saunier
a6f3d5270c
validate:tools: Remove reference to get_backtrace which is not implemented
...
+ Enhance Message about launched apps
2014-02-06 12:02:58 +01:00
Thibault Saunier
26e3a9e3c5
validate:tools: Do not duplicated name in the classname in xunit reports
2014-02-06 12:02:58 +01:00
Thibault Saunier
654bacebc9
validate:tools: Do not check if position > duration
...
This is actually done by the scenario themselve. Instead if it is the
case, we return 0, this way it will timeout if it happens too many times
concecutively
2014-02-06 12:02:58 +01:00
Thibault Saunier
6bd9177205
validate:tools: Properly inform the user about the log location when test fails
2014-02-06 12:02:58 +01:00
Thibault Saunier
eb23274a5b
validate:tools: Add an option to generate .media_info files
...
So we can properly choose what media should be tested only placing
media_file as needed.
2014-02-06 12:02:58 +01:00
Thibault Saunier
32e1acbf88
validate:tools: use more scenarios in gst-validate launcher
...
And ensure that the list does not get mixed up with as we are sharing
"symboles" between all the files
2014-02-06 12:02:58 +01:00
Thibault Saunier
8e81727b87
validate:tools: Do not try to transcode images
2014-02-06 12:02:58 +01:00
Thibault Saunier
32e7b9a55e
validate:tools: Minor cleanups
2014-02-06 12:02:58 +01:00
Thibault Saunier
38615bcae2
validate:tools: Fix classname in gst-launch transcoding tests
2014-02-06 12:02:58 +01:00
Thibault Saunier
dc29cc2857
validate:tools: Properly name the project launcher in the report
2014-02-06 12:02:58 +01:00
Thibault Saunier
0b3b456f6a
validate:tools: Create the rendering directory if it does not exist
2014-02-06 12:02:58 +01:00
Thibault Saunier
8697dd3da4
validate:tools: Add an option to desativate ANSI colors
...
And enhance some debugging output
2014-02-06 12:02:58 +01:00
Thibault Saunier
eae7e94f44
validate:tools: Remove our dependency to PyGobject
2014-02-06 12:02:58 +01:00
Thibault Saunier
0c535ef9cf
validate:tools: Do not forget to add Makefile.am and main.py
2014-02-06 12:02:58 +01:00
Thibault Saunier
bd3aaded49
validate:tools: Enhance the way we detect if ges-launch can be used
...
We make sure it has been compiled against gst-validate
2014-02-06 12:02:58 +01:00
Thibault Saunier
c7aa259c78
validate:tools: Set video/webm instead of video/x-matroska as caps for webm
...
+ some mirore indentation cleanups
2014-02-06 12:02:58 +01:00
Thibault Saunier
384dd33843
validate:tools: Do not forget to keep our ref to file_info g-v-transcode
2014-02-06 12:02:57 +01:00
Thibault Saunier
446e5c88c2
validate:tools: Cleanup how we check result of rendering test
...
Factor out a method in the utils, and make use of it for both ges-launch and
gst-validate-transcode
2014-02-06 12:02:57 +01:00
Thibault Saunier
ff30c6ba3c
validate:tools: Don't give file duration as timeout for gst-validate
...
We use the other mean letting us actually control the process
advancement.
2014-02-06 12:02:57 +01:00
Thibault Saunier
cdff1c93ca
validate:tools: Veryfy test manager are operationnal before using them
2014-02-06 12:02:57 +01:00
Thibault Saunier
e30f6372f9
validate:tools: Move the main function in a dedictaed file
2014-02-06 12:02:57 +01:00
Thibault Saunier
c33d10470f
validate:tools: Print some statistic at the end of the test run
2014-02-06 12:02:57 +01:00
Thibault Saunier
d8fc68479c
validate:tools: Print position every 50ms in gst-validate
2014-02-06 12:02:57 +01:00
Thibault Saunier
ea7ae57d3b
validate:tools: Rename files around and integrate into autotools
...
File distribution used to be messy, clean it all up. Also make sure the
launcher is integrated into the autotools.
2014-02-06 12:02:57 +01:00
Thibault Saunier
ac9820a435
validate: tools: Enhance error message for GstValidate tests
2014-02-06 12:02:57 +01:00
Thibault Saunier
1f3432a33e
validate: tools: Concider timeouts as errors when printing tests
2014-02-06 12:02:57 +01:00
Thibault Saunier
dac5b38a4d
validate: tools: Refactor and add a GstValidateTranscodeTest class
2014-02-06 12:02:57 +01:00
Thibault Saunier
e591882794
validate: launcher: add the debug logger from pitivi
...
It is way more powerfull, simple to use and usefull
than the stock python one and has been proved to work reliably
2014-02-06 12:02:57 +01:00
Thibault Saunier
b51e143fdf
validate: tools: Unref the pipeline before the runner and monitor
...
Avoids segfault in some cases, and monitors and runners have week ref on
their targets.
2014-02-06 12:02:57 +01:00
Thibault Saunier
0d6d397e30
validate: tools: Add a gst-validate test manager
2014-02-06 12:02:57 +01:00
Thibault Saunier
e9b2967504
validate: tools: Cleanup test launcher tool
...
Previous commit was not meant to be pushed and those two should have
been fixed up together, sorry for the mistake
2014-02-06 12:02:57 +01:00
Thibault Saunier
b4a2ca6286
Add a test launcher tool
2013-12-31 11:45:07 +01:00