Thibault Saunier
e8162cc957
validate:launcher: Error out in the check testsuite if rebuilding failed
2019-02-06 21:14:13 +00:00
Mathieu Duponchelle
90c2d32030
Scenarios: override seek flags for RTSP tests
...
Our RTSP server is not accurate, it makes no sense to perform
accuracy checks on the client-side segments.
2019-02-02 02:08:58 +01:00
Thibault Saunier
78781de913
validate:launcher:punittest: Raise an exception if a testsuite can't be loaded
2019-01-30 15:57:13 -03:00
Thibault Saunier
9b69bcad08
validate:launcher: Handle launching launching a sub launcher
...
If you use validate-launcher in a meson testsuite, those test now
gets integrated as one unique testsuite (with a pretty long namespace).
2019-01-26 15:54:00 +00:00
Thibault Saunier
3f4f815500
launcher: Cleanup the way we find python test command line
...
By setting it before the test base class adds the current testsuite name in the classname
2019-01-26 15:54:00 +00:00
Thibault Saunier
9cf213dee0
launcher: Fix test listing in meson VS gstcheck tests
...
You might select tests that match Meson but not gstcheck in which
case the 'meson only' variant is exposed but those should never exist.
2019-01-26 15:54:00 +00:00
Thibault Saunier
a3bab7c66e
validate:launcher: Add %(config_path)s
in the pipeline desc vars
...
When defining pipelines_descriptions to run test on in a `.json` file, you might
need to point to paths in the testsuite directory (for media files URIs
for example), you can now do
`"pipeline": "filesrc location="$(config_path)s/../medias/some/file.mkv...`
2019-01-26 10:11:24 -03:00
Alexandru Băluț
9a04ba5000
validate:launcher: Fix error message
2018-12-14 12:00:18 +01:00
Alexandru Băluț
b280850718
validate:launcher: Fix discovery of commands
2018-12-14 11:17:41 +01:00
Thibault Saunier
b0c0c2d846
validate:launcher: Do not CK_FORK on our test
...
Otherwise the process can't cleanly quit on assertion because of the way libcheck runner is implemented
2018-12-07 09:06:50 -03:00
Thibault Saunier
21e23c72fc
validate:launcher: Do not take CK_DEFAULT_TIMEOUT into account to set the timeout
...
The timeout is what is set in the meson build definition otherwise we will not behave as 'meson test' which we should avoid
2018-12-06 23:17:29 -03:00
Thibault Saunier
ec9464dd90
validate:launcher: Don't about unexisting tests when filtering tests
...
It was wrong
2018-11-28 10:19:52 -03:00
Thibault Saunier
b971a7cd23
validate:launcher: Fix setting meson tests as "parallel"
2018-11-28 10:11:00 -03:00
Thibault Saunier
9ab1d3b6fb
validate:launcher: Avoid using not yet set variables
...
And make the file pep8 compliant with latest pep8 checker.
2018-11-26 10:16:12 -03:00
Thibault Saunier
4bb2648154
validate: launcher: Add a way to load pipeline tests from a scenario
2018-11-25 12:01:12 -03:00
Thibault Saunier
f0cfdf9d14
validate:launcher: Use fakevideosink everywhere it makes sense.
2018-09-07 16:19:40 -03:00
Thibault Saunier
29ef55c469
validate:launcher: Stop spamming envvars in unit tests command printing.
...
We used to print the whole environment, making it ugly and hard to
read.
2018-07-08 17:04:25 -04:00
Thibault Saunier
d80fb50c4b
validate:launcher: Add a TestManager to run python tests
...
Add a stupid simple testsuite made to be configured from the outside
2018-06-15 17:57:24 -04:00
Thibault Saunier
58c90448ca
validate: launcher: Add support for running tests with a pushfile source
...
Introducing the `.media_info.push` media info extension, which is meant
to let the launcher know that those file should run with the "pushfile://"
protocol.
And allow symlinking "normal" `.media_info` to their `.pushfile` variant
so that both can share the exact same content.
2018-06-15 12:05:14 -04:00
Thibault Saunier
b2e71e1404
validate: media-check: Add a way to skip pluggin parsers
...
This is useful when you want to check only the demuxer output.
- Keep the information in the media file so that we can launch media-check
with the proper arguments in the launcher. Update it accordingly.
- Refactor compare_streams to simplify it, which in the end leads to
reporting all the issues instead of exiting on the first one.
2018-06-15 12:05:14 -04:00
Thibault Saunier
b00ab02254
validate:launcher: Cleanup the way we find where -validate tools are
2018-05-13 09:44:17 -04:00
Thibault Saunier
c1f89b4acb
validate: launcher: Add a way to simply run SSIM checks on rendered files
...
We will run a simple pipeline with the IQA element to run ssim (dssim)
tests on the rendered files, comparing it with a reference file.
For now we use the very empiric 1.0 value as a ssim error threshold and
the goal is basically to detect completely broken renderings.
2018-05-13 09:44:17 -04:00
Thibault Saunier
c93f1704f8
validate: Add support for the new testbin protocol
2018-05-13 09:44:17 -04:00
Edward Hervey
58fb4f9354
validate: Remove hls.*seek_with_stop blacklisting
...
The issue is closed upstream (because of concentrating on decodebin3
instead), and initial forever testing seems to show the issue doesn't
happen anymore
2018-05-07 17:30:13 +02:00
Thibault Saunier
77ee198b1b
validate:launcher: Add support for specifying a workdir in tests
2018-04-15 20:50:22 -03:00
Thibault Saunier
3f66772fa1
check: Use meson introspect to list meson tests
2018-03-20 08:51:04 -03:00
Edward Hervey
01a54641f9
validate: Remove protocol-specific timeouts
...
Since we now check position/status of pipeline at regular intevals,
we no longer need to impose a different timeout based on the
protocol used.
Avoids having 4min long timeouts for no reason (30s is enough)
2017-12-03 12:23:51 +01:00
Edward Hervey
65e2c1567a
gstvalidate: Lower timeout to check for rtsp-server to be up
...
Check every 100ms, avoids throttling all rtsp tests by 500ms
2017-12-03 11:38:08 +01:00
Thibault Saunier
4fac7bf9fd
validate: launcher: Run rtsp tests against both V1 and V2
...
https://bugzilla.gnome.org/show_bug.cgi?id=781446
2017-09-22 16:31:56 -03:00
Edward Hervey
9ae20ee3e0
validate: Re-enable mxf op2b tests
...
https://bugzilla.gnome.org/show_bug.cgi?id=785119
2017-07-25 09:55:02 +02:00
Thibault Saunier
8ab723b153
validate:launcher:check: Make sure to register tests from the testsuite
...
Instead of having them listed from the app manager. This is needed
to avoid backtrace as tests now have to be register when setting up
the testsuite.
2017-07-19 11:49:54 -04:00
Edward Hervey
1cfe980044
validate: Blacklist op2b mxf files
...
See https://bugzilla.gnome.org/show_bug.cgi?id=785119
2017-07-19 15:47:28 +02:00
Thibault Saunier
2cf93f491b
validate: launcher: Namespace test name with the testsuite name
...
Also allowing users to pass test names directly
2017-07-18 10:54:53 -04:00
Sebastian Dröge
c8cd695bb9
validate: Un-blacklist scrub_forward_seeking.op2b-mpeg2-wave_hd_mxf
...
It works now after various mxfdemux changes.
https://bugzilla.gnome.org/show_bug.cgi?id=764025
2017-07-14 10:37:31 +03:00
Edward Hervey
ee72ae8913
gstvalidate: Re-allow tests that should be fixed
2017-07-13 08:46:05 +02:00
Thibault Saunier
40a08e6080
validate: launcher: Add some missing env variables in command to launch test
2017-07-11 10:11:49 -04:00
Thibault Saunier
3b20a1ccde
validate:launcher: Disable seek with stop on RTSP streams
...
It is actually not supported
2017-06-28 13:02:33 -04:00
Thibault Saunier
831464d96f
validate:launcher: Avoid useless and expensive deep copies
2017-06-27 16:50:35 -04:00
Thibault Saunier
82dd230ef1
validate:launcher: No need to use -validate as a tracer for RTSP server
...
It is now linked into the server app
2017-06-20 10:43:09 -04:00
Thibault Saunier
543e7a983a
validate:launcher: Do not which(None), the rtsp-server command is unset if not avalaible
...
https://bugzilla.gnome.org/show_bug.cgi?id=783551
2017-06-08 12:18:19 -04:00
Thibault Saunier
9a45cd41a9
validate:launcher: Handle test that can't be run in parralel
2017-06-07 15:23:26 -04:00
Thibault Saunier
6772fa7d27
validate: Add 'our own' RTSP server implementation
2017-06-07 11:17:53 -04:00
Thibault Saunier
37de89e6db
validate:launcher: Do not run scenarios that need prerolling on RTSP stream
2017-06-07 11:17:53 -04:00
Thibault Saunier
3c62c315a9
validate: Implement RTSP support
2017-06-07 11:17:53 -04:00
Thibault Saunier
4a38efb6a0
validate:launcher: Add missing is_live implementation for FakeMediaDescriptor
2017-06-01 16:57:50 -04:00
Thibault Saunier
dba5675d81
validate:launcher: Add a way to pass arguments to the leak tracer
2017-06-01 16:57:50 -04:00
Thibault Saunier
e9db1c19d0
launcher:check: Ensure to set build dir before rebuilding
2017-04-20 10:13:00 -03:00
Edward Hervey
330ae663dc
validate: Improve video fakesink properties
...
Try to emulate a bit better a real video sink by making the video
fakesink handle/calculate/report QoS.
Also use the same lateness value as default videosink
2017-04-04 14:53:30 +02:00
Thibault Saunier
b97c13d0a7
validate:launcher: Fix setting meson build dir
2017-02-24 17:33:08 -03:00
Thibault Saunier
f2b42cbaf8
validate:launcher: Various fixes to make the Test class directly usable
...
And the launcher installed with meson usable
2017-02-21 13:40:28 -03:00