Charlie Turner
a443b921a2
validateflow: interpolate folder directories into pipeline descriptions.
...
Allow 'pipeline' fields in validateflow to be interpolated with
directory keys like $(medias)s.
2019-02-21 13:57:17 +00:00
Guillaume Desmottes
b20f73099f
validate: allow config to check for minimum buffer frequency on pads
...
This change allow tests to check performance of elements by checking the
frequency at which buffers are pushed on src pads.
I re-used most of the logic from fpsdisplaysink to compute the
frequency.
We can now uses something like:
GST_VALIDATE_CONFIG='core,min-buffer-frequency=60,target-element-factory-name=v4l2src'
The 'buffer-frequency-start' optional field can be used to ignore the
frequency during the start of the pipeline. This is useful when testing live
pipelines where configuring and setting up elements can take some time slowing
down the first buffers.
2019-02-20 12:12:46 +01:00
Guillaume Desmottes
40f263e857
validate: factor out gst_validate_element_matches_target()
2019-02-20 12:12:46 +01:00
Charlie Turner
4602ee61c4
validateflow: Fix double-free on stdout
2019-02-18 11:05:26 +00:00
Thibault Saunier
34b5192b94
validate: report: Fix the way we print 'repeat' values
2019-02-15 14:19:03 +00:00
Thibault Saunier
53b2ca8f62
validate: Force LC_NUMERIC to C as it is required by our expression parser
...
And... add some expression parser unit tests
2019-02-15 14:19:03 +00:00
Thibault Saunier
394242c224
validate:scenario: Enhance variable implementation
...
- Stop arbitrarily consider params as ClockTime based on their names
but add a convetion that the `.type` field of the ActionType should
end by `(GstClockTime)` when it is a clock time.
2019-02-15 14:19:03 +00:00
Thibault Saunier
b7616e5fab
validate:scenario: Make gst_validate_action_new public
...
Mainly so it can be used in unit tests.
2019-02-15 14:19:03 +00:00
Thibault Saunier
be854d4be8
validate:scenario: Allow using set-vars from numeric expressions
...
And require them to follow the `$varname` (can't be $(varname) as
parenthesis have another meaning in those expressions).
Still accept "duration" and "position" as varname for backward compat
but update our scenarios anyway.
2019-02-15 14:19:03 +00:00
Thibault Saunier
28e041b2cb
validate: scenario: Parse playback times as we execute the scenario
...
This way we will be able to use 'set-vars' for it
2019-02-15 14:19:03 +00:00
Thibault Saunier
d24477d9e4
validate:scenario: Enforce a synthax $(varname) to reference variables
...
This way it is clear that you are using a variable reading the scenario
and we can verify that what the scenario writer intents is to use an
already set variable.
2019-02-15 14:19:03 +00:00
Guillaume Desmottes
3ebc26e265
validate: don't override max-latency if config contains multiple structs
...
gst_validate_utils_get_clocktime() is resetting the value if it's not
present in the struct so we were overriding it on the next iterations.
2019-02-12 02:21:14 +00:00
Guillaume Desmottes
da22878a5f
validate: baseclasses: include env variable in logged command
...
We were missing the env variables in the command written to the log
file, making it impossible to re-run the test later from the logs.
2019-02-11 15:17:33 +00:00
Alicia Boya García
1d4801d679
gst-validate-scenario: Add GST_VALIDATE_SCENARIO_EOS_HANDLING_LOCK
...
There was a race in appsrc-push when the pushed buffer caused an EOS.
The EOS event could be handled by the main thread, finishing the test
while the action, executing in the streaming thread, has not finished
yet.
A mutex is now introduced to add mutual exclusion for the two threads so
that an EOS does not cause the termination of the test while the action
is still going.
2019-02-11 14:15:30 +00:00
Thibault Saunier
5a2e2d5001
validateflow: Print some indication that the flow checking is happening
2019-02-11 08:34:36 -03:00
Edward Hervey
4467c9925d
validate: Add autotools support for flow plugin
2019-02-11 11:09:46 +01:00
Thibault Saunier
3d3264b342
Revert "validate:launcher: Cache the result of meson introspect"
...
This reverts commit 05ce6d3b92
.
We can't do that as it breaks meson logic to set envvars
2019-02-10 01:25:52 -03:00
Thibault Saunier
5a6307f913
validate:launcher: Do not sort tests all the time
...
Do it once only once it is fully populated
2019-02-09 17:25:03 -03:00
Thibault Saunier
05ce6d3b92
validate:launcher: Cache the result of meson introspect
...
Running it takes quite some time and we can easily cache it.
2019-02-09 17:24:13 -03:00
Thibault Saunier
a88156a416
validate:launcher: Add a simple way to profile app
2019-02-09 17:23:28 -03:00
Guillaume Desmottes
0e03b38cc2
validate: scenario: not need to use an atomic to handle dropped count
...
It's all handled from the same thread.
2019-02-08 14:39:22 +01:00
Guillaume Desmottes
78a8306955
validate: scenario: fix dropped checking when terminating scenario
...
We want to early return if either no max value has been set for the
scenario or if we didn't receive any QoS information.
2019-02-08 14:25:52 +01:00
Guillaume Desmottes
916bf5cdf1
validate: properly namespace config related checks
2019-02-08 13:32:12 +01:00
Alicia Boya García
e96f2ca714
New validate plugin: validateflow
...
validateflow can be used to check the buffers and events flowing through
a custom pipeline match an expectation file. This can be used to test
non-regular-playback use cases like demuxers handling adaptive streaming
fragment pushing.
This patch includes also new actions used for these cases:
`appsrc-push`, `appsrc-eos` and `flush` (plus `checkpoint`, which is
only available with validateflow).
2019-02-07 18:19:22 +00:00
Guillaume Desmottes
0c83ff56dd
baseclass: add_validate_config: don't use self.proc_env
...
self.proc_env is created when starting the test but this API can be call
by generator when creating the test.
2019-02-07 17:13:02 +01:00
Guillaume Desmottes
921a073bdf
validate: allow scenarios to define a max nb of dropped buffers
...
The 'max-dropped' description field can now be used to specify the max
number of buffers than can be dropped by the QoS system.
2019-02-07 14:12:59 +00:00
Thibault Saunier
e8162cc957
validate:launcher: Error out in the check testsuite if rebuilding failed
2019-02-06 21:14:13 +00:00
Guillaume Desmottes
2d8ef6e81c
validate: allow scenarios to define max pipeline latency
...
The 'max-latency' description field can now be used to specify the max
latency allowed for the running pipeline.
2019-02-06 16:07:02 +01:00
Mathieu Duponchelle
90c2d32030
Scenarios: override seek flags for RTSP tests
...
Our RTSP server is not accurate, it makes no sense to perform
accuracy checks on the client-side segments.
2019-02-02 02:08:58 +01:00
Thibault Saunier
49102316df
validate:scenario: Set variables on all action fields
2019-02-01 23:46:21 +00:00
Thibault Saunier
e50d41420e
validate: Handle G_LOG_ERROR in our glog handler
2019-02-01 23:46:21 +00:00
Thibault Saunier
2f957ab932
validate:scenario: Rename action 'define-consts' to 'set-vars'
...
Those are not consts are they can be modified at runtime
2019-02-01 23:46:21 +00:00
Thibault Saunier
d0ee26e950
validate: Cleanup flags/enum_from_string
2019-02-01 23:46:21 +00:00
Mathieu Duponchelle
f9190236af
scenario: prioritize SCENARIOS_PATH when including
2019-02-01 23:08:16 +01:00
Thibault Saunier
71d2e3f418
validate: Let people know the testsuite starts running
...
Now that we do not print infos about successful tests when redirecting.
2019-02-01 01:24:33 -03:00
Thibault Saunier
ba9b2c5ec5
validate: Keep trying to print stack traces when our log hanlder is removed
...
There are cases where a crash happens after the program ends
2019-02-01 00:08:45 -03:00
Thibault Saunier
68573ae919
validate:scenario: Give pipeline position when failling on EOS
2019-01-31 23:47:11 -03:00
Thibault Saunier
35003bc25b
validate:launcher: Do not print passing tests if not running in a tty
2019-01-31 23:47:11 -03:00
Thibault Saunier
25e0694d1e
validate:scenario: Explicitely mark generated stop() action as such
2019-01-31 22:37:17 -03:00
Thibault Saunier
78781de913
validate:launcher:punittest: Raise an exception if a testsuite can't be loaded
2019-01-30 15:57:13 -03:00
Edward Hervey
d1319a1b6d
pad-monitor: Remove unused pad getrange override
2019-01-30 14:30:52 +01:00
Edward Hervey
de59b3ad00
validate: Use g_object_{get|set}_qdata where applicable
...
This provides a substantial speedup compared to using strings
2019-01-30 14:29:48 +01:00
Mathieu Duponchelle
d5e3f4d31c
TestsManager: stop displaying blacklisted tests on stdout
...
It's basically spam, better suited to the debug logs
2019-01-30 00:27:32 +00:00
Thibault Saunier
440055214a
validate:launcher: Never print lines larger than the terminal
2019-01-29 16:09:30 -03:00
Thibault Saunier
7e70bf8729
validate: scenario: Mark action as being executed right before calling ->execute
...
And make sure that we do not try to execute the following action
In the case the action type leads to the GMainContext to be ieterated
2019-01-29 13:09:36 -03:00
Jordan Petridis
31164cf0a6
validate: Report the full test name in the xunit file
...
Looks like gitlab prefers this way of representing tests as it
displays only the name field in its junit reports.
Close #32
2019-01-26 16:55:23 +00:00
Thibault Saunier
2551e8750e
validate:launcher: Make baseclasses.py pep8 compliant
2019-01-26 15:54:00 +00:00
Thibault Saunier
9b69bcad08
validate:launcher: Handle launching launching a sub launcher
...
If you use validate-launcher in a meson testsuite, those test now
gets integrated as one unique testsuite (with a pretty long namespace).
2019-01-26 15:54:00 +00:00
Thibault Saunier
6665652cff
validate:launcher: Refactor the "main" function
...
- Move the parser code into a `LauncherConfig.create_parser()` method
- Remove the need to pass libsdir to the _TestsLauncher object
- Extract out a `setup_launcher_from_args` function
2019-01-26 15:54:00 +00:00
Thibault Saunier
1b3867b82d
launcher: Move http serveur and xvfb server to the main test runner object
...
No good reason for it to be in the main function
2019-01-26 15:54:00 +00:00