Thibault Saunier
b00ab02254
validate:launcher: Cleanup the way we find where -validate tools are
2018-05-13 09:44:17 -04:00
Thibault Saunier
c1f89b4acb
validate: launcher: Add a way to simply run SSIM checks on rendered files
...
We will run a simple pipeline with the IQA element to run ssim (dssim)
tests on the rendered files, comparing it with a reference file.
For now we use the very empiric 1.0 value as a ssim error threshold and
the goal is basically to detect completely broken renderings.
2018-05-13 09:44:17 -04:00
Thibault Saunier
c93f1704f8
validate: Add support for the new testbin protocol
2018-05-13 09:44:17 -04:00
Edward Hervey
58fb4f9354
validate: Remove hls.*seek_with_stop blacklisting
...
The issue is closed upstream (because of concentrating on decodebin3
instead), and initial forever testing seems to show the issue doesn't
happen anymore
2018-05-07 17:30:13 +02:00
Edward Hervey
01a54641f9
validate: Remove protocol-specific timeouts
...
Since we now check position/status of pipeline at regular intevals,
we no longer need to impose a different timeout based on the
protocol used.
Avoids having 4min long timeouts for no reason (30s is enough)
2017-12-03 12:23:51 +01:00
Edward Hervey
65e2c1567a
gstvalidate: Lower timeout to check for rtsp-server to be up
...
Check every 100ms, avoids throttling all rtsp tests by 500ms
2017-12-03 11:38:08 +01:00
Thibault Saunier
4fac7bf9fd
validate: launcher: Run rtsp tests against both V1 and V2
...
https://bugzilla.gnome.org/show_bug.cgi?id=781446
2017-09-22 16:31:56 -03:00
Edward Hervey
9ae20ee3e0
validate: Re-enable mxf op2b tests
...
https://bugzilla.gnome.org/show_bug.cgi?id=785119
2017-07-25 09:55:02 +02:00
Edward Hervey
1cfe980044
validate: Blacklist op2b mxf files
...
See https://bugzilla.gnome.org/show_bug.cgi?id=785119
2017-07-19 15:47:28 +02:00
Thibault Saunier
2cf93f491b
validate: launcher: Namespace test name with the testsuite name
...
Also allowing users to pass test names directly
2017-07-18 10:54:53 -04:00
Sebastian Dröge
c8cd695bb9
validate: Un-blacklist scrub_forward_seeking.op2b-mpeg2-wave_hd_mxf
...
It works now after various mxfdemux changes.
https://bugzilla.gnome.org/show_bug.cgi?id=764025
2017-07-14 10:37:31 +03:00
Edward Hervey
ee72ae8913
gstvalidate: Re-allow tests that should be fixed
2017-07-13 08:46:05 +02:00
Thibault Saunier
3b20a1ccde
validate:launcher: Disable seek with stop on RTSP streams
...
It is actually not supported
2017-06-28 13:02:33 -04:00
Thibault Saunier
831464d96f
validate:launcher: Avoid useless and expensive deep copies
2017-06-27 16:50:35 -04:00
Thibault Saunier
82dd230ef1
validate:launcher: No need to use -validate as a tracer for RTSP server
...
It is now linked into the server app
2017-06-20 10:43:09 -04:00
Thibault Saunier
543e7a983a
validate:launcher: Do not which(None), the rtsp-server command is unset if not avalaible
...
https://bugzilla.gnome.org/show_bug.cgi?id=783551
2017-06-08 12:18:19 -04:00
Thibault Saunier
6772fa7d27
validate: Add 'our own' RTSP server implementation
2017-06-07 11:17:53 -04:00
Thibault Saunier
37de89e6db
validate:launcher: Do not run scenarios that need prerolling on RTSP stream
2017-06-07 11:17:53 -04:00
Thibault Saunier
3c62c315a9
validate: Implement RTSP support
2017-06-07 11:17:53 -04:00
Thibault Saunier
4a38efb6a0
validate:launcher: Add missing is_live implementation for FakeMediaDescriptor
2017-06-01 16:57:50 -04:00
Edward Hervey
330ae663dc
validate: Improve video fakesink properties
...
Try to emulate a bit better a real video sink by making the video
fakesink handle/calculate/report QoS.
Also use the same lateness value as default videosink
2017-04-04 14:53:30 +02:00
Thibault Saunier
bc3e9505ea
validate:launcher: Allow passing extra env var to simple pipeline generator
2017-01-31 08:15:38 -03:00
Thibault Saunier
5fc11cf389
launcher: Make --update-media-info check if generating frames info or not
2016-12-22 10:29:18 -03:00
Thibault Saunier
26692e749c
validate:launcher: Stop running test subprocesses in a shell
...
And instead properly use a list of argument for the subprocesses.
2016-12-22 10:07:58 -03:00
Thibault Saunier
1e51aeb942
validate:launcher: Port to Python3
...
And sync logging.py with Pitivi version
2016-11-09 10:13:42 -03:00
Edward Hervey
8c20943a9d
validate: Blacklist failing hls tests
...
See https://bugzilla.gnome.org/show_bug.cgi?id=773159
2016-10-18 15:41:11 +02:00
Thibault Saunier
9c9fe14293
meson:validate:test: Properly set paths to run launcher based tests
...
Adding a --validate-tools-path option to the launcher, allowing
to pass it from meson.
2016-09-09 12:24:54 -03:00
Thibault Saunier
03548cd63d
validate:launcher: Allow specifying expected tests errors
...
In the future instead of blacklisting tests we should define
what error is expected, and this way when the bug is closed,
we will notice, also, it will allow us to check GstValidate
error reporting itself.
2016-09-08 13:10:29 -03:00
Thibault Saunier
f7f600e730
validate: launcher: Fix the condition to check if we need an http server
...
We could be checking if a string was in None
And use gs_string_assign when assigning the first string
without using printf like format.
2016-09-08 10:39:56 -03:00
Edward Hervey
fcff932e09
validate: Un-blacklist tests that are fixed
...
the bug reports to which they report have been closed and I can't make
them fail locally.
2016-08-13 16:56:18 +02:00
Edward Hervey
7d99e780a7
validate: Blacklist more ogg files
...
https://bugzilla.gnome.org/show_bug.cgi?id=769545
2016-08-13 15:39:18 +02:00
Edward Hervey
37771192a6
validate: Blacklist scrub_forward_seeking.op2b-mpeg2-wave_hd_mxf
...
See https://bugzilla.gnome.org/show_bug.cgi?id=764025
2016-08-12 12:33:30 +02:00
Edward Hervey
1e1475d352
validate: Fix usage for non-standard http server port
...
Translate the various stream_info URI from the standard port to the
specified port
Make the port option an integer option
2016-07-04 16:19:09 +02:00
Edward Hervey
b77797da4b
validate: Use decodebin3/uridecodebin3 when specified
...
Set the USE_PLAYBIN3 environment variable to use those elements instead
of legacy ones.
2016-06-30 14:58:03 +02:00
Vineeth TM
99f9f3f408
validate:launcher: Add support for relative path while providing file path
...
Instead of providing full absolute path while validating the file, should be
able to provide the relative path with respect to the present directory.
https://bugzilla.gnome.org/show_bug.cgi?id=753494
2016-06-04 12:51:35 +01:00
Thiago Santos
d0a7d9b094
validate: add non-seekable fragment file seek tests to the blacklist
...
The file has no index and is not easily seekable, seeking in these
kind of files isn't implemented.
2016-05-06 21:27:53 -03:00
Thibault Saunier
23ad66c552
validate: launcher: Blacklist dash and HLS failling seeking tests
...
As described in https://bugzilla.gnome.org/show_bug.cgi?id=764020
2016-03-22 19:02:27 +01:00
Thibault Saunier
a566b2c4b9
validate:testsuite: Blacklist validate.hls.playback.scrub_forward_seeking.hls_bibbop as it is racy
2016-03-22 17:46:50 +01:00
Vineeth T M
7d9e9b3b79
validate: Fix wrong condition check when adding tests
...
When listing tests, checking whether uri is present or not and displaying error.
But uri does notneed to be present in case of pipeline generator. So the condition check is wrong.
This results in validateelements testsuite not working. Hence modifying the condition to
not error out on valid cases.
https://bugzilla.gnome.org/show_bug.cgi?id=762422
2016-03-05 12:56:14 +01:00
Vineeth T M
bc66079d2a
validate:launcher: handle spaces in transcode output path
...
When there are spaces in transcoding output path, then it fails. Hence adding the
path in double quotes
https://bugzilla.gnome.org/show_bug.cgi?id=756217
2016-02-17 22:33:32 +01:00
Vineeth T M
0f7105d317
validate: launcher: Print error when media-info files not present
...
When there are no media-info files present and --generate-media-info
option is not given, then it just fails without printing error.
Printing an error stating, use --generate-media-info if there are no
media info files.
When there are neither media files and media info files, print error
stating the same
https://bugzilla.gnome.org/show_bug.cgi?id=755087
2015-11-11 10:12:58 +01:00
Sebastian Dröge
3bd31854a8
validate: Un-blacklist seeking HLS tests
...
They are reliable enough after 36b80edb72
in gst-plugins-base now.
2015-10-27 15:51:44 +02:00
Vineeth T M
e7b65fe5df
validate: launcher: Fix media_check class name and add double quotes for valgrind logs
...
When creating the class names for media check, uri is being used,
instead of the path. Hence converting the uri using uri2path and creating
class name.
Add double quotes for valgrind logs, to support special characters like space
https://bugzilla.gnome.org/show_bug.cgi?id=752808
2015-10-02 17:43:50 +02:00
Vineeth T M
1b700f13b4
validate: launcher: Support relative path for folder names
...
Even though relative paths are supported, right now it does not
work when we give the path as 'media/' present in the current directory.
Adding support for the same.
https://bugzilla.gnome.org/show_bug.cgi?id=754100
2015-10-02 17:20:18 +02:00
Sebastian Dröge
5b81bb4bc9
validate/launcher: Blacklist some HLS seeking tests again
2015-08-16 17:53:28 +02:00
Thiago Santos
b77f56a234
validate/launcher: Un-blacklist validate.dash.playback.reverse_playback.*
...
Fixed now
2015-08-16 08:50:36 -03:00
Sebastian Dröge
f898f3ec8b
validate/launcher: Un-blacklist validate.hls.playback.reverse_playback.*
...
It apparently succeeds now.
2015-08-16 12:26:16 +02:00
Sebastian Dröge
fea86d3556
validate/launcher: De-blacklist some HLS tests
2015-08-15 19:07:02 +02:00
Sebastian Dröge
1f8a5483d3
validate/launcher: Blacklist some DASH tests
2015-08-15 19:04:14 +02:00
Thibault Saunier
bb2532aa54
launcher: Handle override files for media files
...
And make sure to create a new dict for extra_env_vars when instanciating
GstValidateTest
2015-08-15 18:30:53 +02:00