mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2025-02-03 21:12:26 +00:00
Code Issues 1 Projects Releases Wiki Activity
2535 commits 44 branches 1513 tags 374 MiB
Commit graph

158 commits

Author SHA1 Message Date
Thibault Saunier
91ac75aa4c launcher: check: Properly set PLUGIN_PATH and registry when running in gst-build 
This makes registry

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>
 2020-05-26 19:16:26 -04:00
Thibault Saunier
90cc65f7ee launcher: Ensure that -j tests run in parallel when running forever 
So that you can reproduce the issue you want faster!

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>
 2020-05-26 17:19:36 -04:00
Thibault Saunier
766f339fb8 launcher: Add support for running tests inside rr 
Allowing us to easily run the tests forever and then replay the
failures!

https://rr-project.org/

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/197>
 2020-05-26 17:19:36 -04:00
Thibault Saunier
884c45ff3e validate:launcher:check: Add support for gst-tester-1.0 
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-devtools/-/merge_requests/189>
 2020-05-07 09:39:16 -04:00
Thibault Saunier
bf952d3c8b validate: Introduce the concept of "Test files" 
This way we can have a single file that wraps scenarios,
`gst-validate-1.0` arguments, as well as a configuration.

It changes the name of `description` of scenarios to use `meta`
The goal is to replace tests describes in python with dictionary
to fully self contained `.validatetest` files which look like:

```
meta,
    handles-states=true,
    ignore-eos=true,
    gst-validate-args = {
         "videotestsrc pattern=blue ! video/x-raw,format=I420,framerate=1/1 ! timeoverlay ! $(videosink) name=videosink allocation-meta-flags=0",
    },
    configs = {
         "$(validateflow), pad=videosink:sink, buffers-checksum=true, ignored-fields={\"buffers=meta\", }",
    }

play
seek, start=0.0, stop=5.0, flags=accurate+flush, rate=1.0
crank-clock, expected-elapsed-time=0.0
crank-clock, repeat=4, expected-elapsed-time=1.0
crank-clock, expected-elapsed-time=1.0
stop, on-message=eos
```
 2020-04-27 16:03:33 -04:00
Thibault Saunier
1468b57430 validate:launcher: Fix reporting on failure when running forever/fatal 2020-03-12 12:35:45 -03:00
Thibault Saunier
0b41b8a8c0 validate:launcher: Add a generator to generate test for frame accurate seeking 2020-03-12 12:35:45 -03:00
Thibault Saunier
b4c1424ea2 validate:launcher: Add support for skipped media info files 
Those are skipped to generate tests by default but are updated when
required, this will allow us to generate specific test on demand for
those
 2020-03-11 16:36:16 -03:00
Thibault Saunier
080fdf8e2f validate:launcher: Add support for the imagesequence protocol 2020-03-06 16:05:38 -03:00
Thibault Saunier
bc622ec658 validate:launcher: Batch inspecting scenarios 
Removing almost 1 second to start running tests with the default
testsuite
 2020-03-06 15:36:10 -03:00
Thibault Saunier
d1c3607bc3 validate:launcher: Allow passing configs paths when loading from dictionary 
The same way we allow it for scenarios
 2020-03-06 15:36:10 -03:00
Thibault Saunier
d3c6611b76 validate:launcher: Fix defining several scenario with a same config 
When generating tests from dictionary the dict format allows passing
several scenario for a same config and pipelines, but this was breaking
the case where expected flow is different with each config, instead we
should generate one config per scenario, fixing the expectation files
generated.
 2020-02-06 14:33:56 -03:00
Stéphane Cerveau
2581fef684 gst-validate-launcher: update documentation 
Use the new api to create your custom testsuite.
Fix some broken links and enhance the logging system.
 2020-01-24 14:52:49 +00:00
Thibault Saunier
d543375948 validate:launcher: Take our timeout factor into account for gstcheck 2019-12-30 15:56:02 +00:00
Thibault Saunier
1f67876928 validate:launcher: Use python dict for pipeline description 2019-12-10 13:02:16 +00:00
Thibault Saunier
562750213f validate: launcher: Make encoding extra check use common code path 
Reusing the reporting infrastructure instead of shurtcuting it
 2019-11-20 18:07:48 -03:00
Thibault Saunier
8a6ee4841b validate: Add vp9 transcoding tests 
Making sure to encode small frames as vp9enc is slow.
 2019-11-20 18:07:47 -03:00
Stéphane Cerveau
998185acbc gstvalidate: fix GstValidateRTSPMediaDescriptor typo 2019-11-06 18:21:11 +01:00
Tim-Philipp Müller
280321dee1 validate: remove autotools build 2019-10-15 00:24:00 +01:00
Philippe Normand
3351807107 validate/launcher: Ensure the HTTP server is started when a pipeline needs it 
Pipelines declared in gst-integration-testsuites can rely on the validate HTTP
server, so when an URI pointing to it is detected, advertise the server as
needed before starting the test.

For this to work the test scenario should explicitely declare the pipeline uri,
as shown in this example:

    "some_playbin3":
    {
        "pipeline": "playbin3 uri=%(uri)s video-sink=%(videosink)s",
        "config": [
            "%(validateflow)s, pad=sink:sink"
        ],
        "scenarios": ["play_15s"],
        "uri": "http://127.0.0.1:%(http-server-port)s/defaults/html/foo.html"
    }
 2019-08-21 08:37:38 +01:00
Thibault Saunier
8e01e03364 validate:launcher: Allow passing any extra_data in json test definition 
This means that we can now pass any extra key that `populate_tests`
expects, meaning any key expected by FakeMediaDescriptor and
a few other keys supported by the methods such as
`expected-issues` and `extra_env_vars`
 2019-08-20 13:55:52 +00:00
Thibault Saunier
556bc0bb4f launcher: Raise an exception when provided scenario can't be found 2019-08-19 10:33:20 +00:00
Mathieu Duponchelle
5a068bff72 validate: Update blacklisting reason for fast forward rtsp 
While https://gitlab.freedesktop.org/gstreamer/gst-rtsp-server/issues/14
was merged, the client side (in particular rtpbasedepayload) still
isn't expected to work appropriately
 2019-08-01 21:04:12 +02:00
Mathieu Duponchelle
93114b55ab json tests: assume all user pipelines can seek reverse 
When doing a targeted test, it is up to the user to make sure
their pipeline + scenario behaves correctly.
 2019-07-26 02:26:20 +02:00
Thibault Saunier
3343f166da validate:launcher: Move get_fakesink_for_media_type to utils 
So it can be reused in other apps like GES
 2019-06-26 12:09:03 -04:00
Thibault Saunier
d60ea5da63 validate:launcher: Set ORC_CODE=backup when running gst unit tests under valgrind 2019-04-17 11:04:45 -04:00
Thibault Saunier
ebf0844d2c validate:launcher: Avoid forking when running gstcheck tests in gdb 2019-04-11 13:17:14 +00:00
Thibault Saunier
446f1d8197 validate:launcher: Make extra_logfiles a set and fix their names 2019-04-11 13:17:14 +00:00
Thibault Saunier
2fcdc27c3a validate:launcher: Remove spurious print 2019-04-11 13:17:14 +00:00
Thibault Saunier
787939f750 validate:launcher: Add a list of well known subpression files from gst-build subprojects 
Fixes https://gitlab.freedesktop.org/gstreamer/gst-devtools/issues/38
 2019-04-03 13:38:42 +00:00
Thibault Saunier
ac7efe9500 validate:launcher: Rework expected-issues data format 
Instead of having the issues centered on the test classes, they
are now focusing on the "bug".

And harmise names on `expected_issue` not `expected_failures`
 2019-04-03 13:38:42 +00:00
Thibault Saunier
3a826e1e3d validate:launcher: Implement bug checks for gitlab 
And use new gitlab urls for all the bugs
 2019-04-03 13:38:42 +00:00
Thibault Saunier
8d00a74f1a validate:launcher: Enhance and standardize output 2019-04-03 13:38:42 +00:00
Thibault Saunier
6310145a44 validate:launcher: Set CK_MULTIPLIER=10 in GstCheck tests when using valgrind 
Fixes https://gitlab.freedesktop.org/gstreamer/gst-devtools/issues/39
 2019-03-22 11:23:08 -03:00
Thibault Saunier
773f2e456c validate:launcher: Move '_format_config_template' to the utilities 
So it can be reused by GES tests
 2019-03-11 18:14:23 -03:00
Charlie Turner
a443b921a2 validateflow: interpolate folder directories into pipeline descriptions. 
Allow 'pipeline' fields in validateflow to be interpolated with
directory keys like $(medias)s.
 2019-02-21 13:57:17 +00:00
Thibault Saunier
3d3264b342 Revert "validate:launcher: Cache the result of meson introspect" 
This reverts commit 05ce6d3b92.

We can't do that as it breaks meson logic to set envvars
 2019-02-10 01:25:52 -03:00
Thibault Saunier
05ce6d3b92 validate:launcher: Cache the result of meson introspect 
Running it takes quite some time and we can easily cache it.
 2019-02-09 17:24:13 -03:00
Alicia Boya García
e96f2ca714 New validate plugin: validateflow 
validateflow can be used to check the buffers and events flowing through
a custom pipeline match an expectation file. This can be used to test
non-regular-playback use cases like demuxers handling adaptive streaming
fragment pushing.

This patch includes also new actions used for these cases:
`appsrc-push`, `appsrc-eos` and `flush` (plus `checkpoint`, which is
only available with validateflow).
 2019-02-07 18:19:22 +00:00
Thibault Saunier
e8162cc957 validate:launcher: Error out in the check testsuite if rebuilding failed 2019-02-06 21:14:13 +00:00
Mathieu Duponchelle
90c2d32030 Scenarios: override seek flags for RTSP tests 
Our RTSP server is not accurate, it makes no sense to perform
accuracy checks on the client-side segments.
 2019-02-02 02:08:58 +01:00
Thibault Saunier
78781de913 validate:launcher:punittest: Raise an exception if a testsuite can't be loaded 2019-01-30 15:57:13 -03:00
Thibault Saunier
9b69bcad08 validate:launcher: Handle launching launching a sub launcher 
If you use validate-launcher in a meson testsuite, those test now
gets integrated as one unique testsuite (with a pretty long namespace).
 2019-01-26 15:54:00 +00:00
Thibault Saunier
3f4f815500 launcher: Cleanup the way we find python test command line 
By setting it before the test base class adds the current testsuite name in the classname
 2019-01-26 15:54:00 +00:00
Thibault Saunier
9cf213dee0 launcher: Fix test listing in meson VS gstcheck tests 
You might select tests that match Meson but not gstcheck in which
case the 'meson only' variant is exposed but those should never exist.
 2019-01-26 15:54:00 +00:00
Thibault Saunier
a3bab7c66e validate:launcher: Add %(config_path)s in the pipeline desc vars 
When defining pipelines_descriptions to run test on in a `.json` file, you might
need to point to paths in the testsuite directory (for media files URIs
for example), you can now do

 `"pipeline": "filesrc location="$(config_path)s/../medias/some/file.mkv...`
 2019-01-26 10:11:24 -03:00
Alexandru Băluț
9a04ba5000 validate:launcher: Fix error message 2018-12-14 12:00:18 +01:00
Alexandru Băluț
b280850718 validate:launcher: Fix discovery of commands 2018-12-14 11:17:41 +01:00
Thibault Saunier
b0c0c2d846 validate:launcher: Do not CK_FORK on our test 
Otherwise the process can't cleanly quit on assertion because of the way libcheck runner is implemented
 2018-12-07 09:06:50 -03:00
Thibault Saunier
21e23c72fc validate:launcher: Do not take CK_DEFAULT_TIMEOUT into account to set the timeout 
The timeout is what is set in the meson build definition otherwise we will not behave as 'meson test' which we should avoid
 2018-12-06 23:17:29 -03:00