Commit graph

99 commits

Author SHA1 Message Date
Thibault Saunier
b00ab02254 validate:launcher: Cleanup the way we find where -validate tools are 2018-05-13 09:44:17 -04:00
Thibault Saunier
c1f89b4acb validate: launcher: Add a way to simply run SSIM checks on rendered files
We will run a simple pipeline with the IQA element to run ssim (dssim)
tests on the rendered files, comparing it with a reference file.

For now we use the very empiric 1.0 value as a ssim error threshold and
the goal is basically to detect completely broken renderings.
2018-05-13 09:44:17 -04:00
Thibault Saunier
c93f1704f8 validate: Add support for the new testbin protocol 2018-05-13 09:44:17 -04:00
Edward Hervey
58fb4f9354 validate: Remove hls.*seek_with_stop blacklisting
The issue is closed upstream (because of concentrating on decodebin3
instead), and initial forever testing seems to show the issue doesn't
happen anymore
2018-05-07 17:30:13 +02:00
Thibault Saunier
77ee198b1b validate:launcher: Add support for specifying a workdir in tests 2018-04-15 20:50:22 -03:00
Thibault Saunier
3f66772fa1 check: Use meson introspect to list meson tests 2018-03-20 08:51:04 -03:00
Edward Hervey
01a54641f9 validate: Remove protocol-specific timeouts
Since we now check position/status of pipeline at regular intevals,
we no longer need to impose a different timeout based on the
protocol used.

Avoids having 4min long timeouts for no reason (30s is enough)
2017-12-03 12:23:51 +01:00
Edward Hervey
65e2c1567a gstvalidate: Lower timeout to check for rtsp-server to be up
Check every 100ms, avoids throttling all rtsp tests by 500ms
2017-12-03 11:38:08 +01:00
Thibault Saunier
4fac7bf9fd validate: launcher: Run rtsp tests against both V1 and V2
https://bugzilla.gnome.org/show_bug.cgi?id=781446
2017-09-22 16:31:56 -03:00
Edward Hervey
9ae20ee3e0 validate: Re-enable mxf op2b tests
https://bugzilla.gnome.org/show_bug.cgi?id=785119
2017-07-25 09:55:02 +02:00
Thibault Saunier
8ab723b153 validate:launcher:check: Make sure to register tests from the testsuite
Instead of having them listed from the app manager. This is needed
to avoid backtrace as tests now have to be register when setting up
the testsuite.
2017-07-19 11:49:54 -04:00
Edward Hervey
1cfe980044 validate: Blacklist op2b mxf files
See https://bugzilla.gnome.org/show_bug.cgi?id=785119
2017-07-19 15:47:28 +02:00
Thibault Saunier
2cf93f491b validate: launcher: Namespace test name with the testsuite name
Also allowing users to pass test names directly
2017-07-18 10:54:53 -04:00
Sebastian Dröge
c8cd695bb9 validate: Un-blacklist scrub_forward_seeking.op2b-mpeg2-wave_hd_mxf
It works now after various mxfdemux changes.

https://bugzilla.gnome.org/show_bug.cgi?id=764025
2017-07-14 10:37:31 +03:00
Edward Hervey
ee72ae8913 gstvalidate: Re-allow tests that should be fixed 2017-07-13 08:46:05 +02:00
Thibault Saunier
40a08e6080 validate: launcher: Add some missing env variables in command to launch test 2017-07-11 10:11:49 -04:00
Thibault Saunier
3b20a1ccde validate:launcher: Disable seek with stop on RTSP streams
It is actually not supported
2017-06-28 13:02:33 -04:00
Thibault Saunier
831464d96f validate:launcher: Avoid useless and expensive deep copies 2017-06-27 16:50:35 -04:00
Thibault Saunier
82dd230ef1 validate:launcher: No need to use -validate as a tracer for RTSP server
It is now linked into the server app
2017-06-20 10:43:09 -04:00
Thibault Saunier
543e7a983a validate:launcher: Do not which(None), the rtsp-server command is unset if not avalaible
https://bugzilla.gnome.org/show_bug.cgi?id=783551
2017-06-08 12:18:19 -04:00
Thibault Saunier
9a45cd41a9 validate:launcher: Handle test that can't be run in parralel 2017-06-07 15:23:26 -04:00
Thibault Saunier
6772fa7d27 validate: Add 'our own' RTSP server implementation 2017-06-07 11:17:53 -04:00
Thibault Saunier
37de89e6db validate:launcher: Do not run scenarios that need prerolling on RTSP stream 2017-06-07 11:17:53 -04:00
Thibault Saunier
3c62c315a9 validate: Implement RTSP support 2017-06-07 11:17:53 -04:00
Thibault Saunier
4a38efb6a0 validate:launcher: Add missing is_live implementation for FakeMediaDescriptor 2017-06-01 16:57:50 -04:00
Thibault Saunier
dba5675d81 validate:launcher: Add a way to pass arguments to the leak tracer 2017-06-01 16:57:50 -04:00
Thibault Saunier
e9db1c19d0 launcher:check: Ensure to set build dir before rebuilding 2017-04-20 10:13:00 -03:00
Edward Hervey
330ae663dc validate: Improve video fakesink properties
Try to emulate a bit better a real video sink by making the video
fakesink handle/calculate/report QoS.

Also use the same lateness value as default videosink
2017-04-04 14:53:30 +02:00
Thibault Saunier
b97c13d0a7 validate:launcher: Fix setting meson build dir 2017-02-24 17:33:08 -03:00
Thibault Saunier
f2b42cbaf8 validate:launcher: Various fixes to make the Test class directly usable
And the launcher installed with meson usable
2017-02-21 13:40:28 -03:00
Thibault Saunier
a39ef816a6 validate: launcher: Do not force using current module dir to run check tests 2017-02-21 13:40:28 -03:00
Thibault Saunier
7773ca7765 validate:launcher: Add a way to specify a set of tests to run under the leak tracer
https://bugzilla.gnome.org/show_bug.cgi?id=767856
2017-02-10 15:56:11 -03:00
Thibault Saunier
bc3e9505ea validate:launcher: Allow passing extra env var to simple pipeline generator 2017-01-31 08:15:38 -03:00
Thibault Saunier
de2cbda854 validate: Make sure to update children environment from current env
And minor fix in function call
2017-01-06 12:12:21 -03:00
Thibault Saunier
30221faa32 validate: check: Enhance test names 2017-01-06 11:48:14 -03:00
Thibault Saunier
ce0d2ee3e1 validate: Fix gstcheck when not running on a meson build dir 2017-01-03 13:11:42 -03:00
Thibault Saunier
0ee2147137 Revert "Revert "validate:launcher: Add an app handler for unit tests described in meson""
This reverts commit 5656e2a1b2.
2017-01-03 13:01:31 -03:00
Edward Hervey
5656e2a1b2 Revert "validate:launcher: Add an app handler for unit tests described in meson"
This reverts commit a4aa5c60bb.

Does not work outside of gst-build (i.e. validate won't work when used either
installed, in a prefix, or in gst-uninstalled).
2017-01-02 17:21:14 +01:00
Thibault Saunier
5fc11cf389 launcher: Make --update-media-info check if generating frames info or not 2016-12-22 10:29:18 -03:00
Thibault Saunier
a4aa5c60bb validate:launcher: Add an app handler for unit tests described in meson
This way we can run all tests with the launcher which brings in many
features.

And add a testsuite for GStreamer unit tests.
2016-12-22 10:29:18 -03:00
Thibault Saunier
26692e749c validate:launcher: Stop running test subprocesses in a shell
And instead properly use a list of argument for the subprocesses.
2016-12-22 10:07:58 -03:00
Thibault Saunier
1e51aeb942 validate:launcher: Port to Python3
And sync logging.py with Pitivi version
2016-11-09 10:13:42 -03:00
Edward Hervey
8c20943a9d validate: Blacklist failing hls tests
See https://bugzilla.gnome.org/show_bug.cgi?id=773159
2016-10-18 15:41:11 +02:00
Thibault Saunier
9c9fe14293 meson:validate:test: Properly set paths to run launcher based tests
Adding a --validate-tools-path option to the launcher, allowing
to pass it from meson.
2016-09-09 12:24:54 -03:00
Thibault Saunier
03548cd63d validate:launcher: Allow specifying expected tests errors
In the future instead of blacklisting tests we should define
what error is expected, and this way when the bug is closed,
we will notice, also, it will allow us to check GstValidate
error reporting itself.
2016-09-08 13:10:29 -03:00
Thibault Saunier
f7f600e730 validate: launcher: Fix the condition to check if we need an http server
We could be checking if a string was in None

And use gs_string_assign when assigning the first string
without using printf like format.
2016-09-08 10:39:56 -03:00
Thibault Saunier
da327d8625 validate: Add support for Meson as alternative/parallel build system
https://github.com/mesonbuild/meson
2016-08-25 15:12:31 -03:00
Edward Hervey
fcff932e09 validate: Un-blacklist tests that are fixed
the bug reports to which they report have been closed and I can't make
them fail locally.
2016-08-13 16:56:18 +02:00
Edward Hervey
7d99e780a7 validate: Blacklist more ogg files
https://bugzilla.gnome.org/show_bug.cgi?id=769545
2016-08-13 15:39:18 +02:00
Edward Hervey
37771192a6 validate: Blacklist scrub_forward_seeking.op2b-mpeg2-wave_hd_mxf
See https://bugzilla.gnome.org/show_bug.cgi?id=764025
2016-08-12 12:33:30 +02:00