Thibault Saunier
7161b21334
validate:launcher: Do not dump to big log files
...
Avoiding ' The script exceeded the maximum execution time set for the job' in GitLab
2019-05-13 11:37:38 -04:00
Thibault Saunier
b9cb5db869
validate:launcher: Ignore possibly lost for now
2019-04-17 09:06:55 -04:00
Mathieu Duponchelle
96361bd426
TestsManager: _add_blacklist in set_default_blacklist
...
Otherwise test suites that want to set_default_blacklist and
add tests in setup_tests were seeing their blacklist ignored.
Split up and rename set_blacklists() to complete the refactoring
2019-04-17 02:31:30 +02:00
Mathieu Duponchelle
61d0605852
Check indirect leaks
2019-04-17 02:29:36 +02:00
Thibault Saunier
3fa393c498
validate:launcher: Do not copy logs for non flaky tests
...
That looks weird for users and is incorrect
2019-04-15 16:13:52 -04:00
Thibault Saunier
81b1368237
validate:launcher: Fix printing debug logs URIs
2019-04-15 11:50:16 -04:00
Thibault Saunier
e182272262
validate:launcher: Do not stop tcp server when reiterating tests runs
2019-04-12 12:33:25 -04:00
Thibault Saunier
d709cb54a9
validate:launcher: Associate issues with the bug they come from
...
Making it simpler to follow when print the known issues
2019-04-12 10:13:15 -04:00
Thibault Saunier
81f198f812
validate:launcher: Indent known issues printing
2019-04-12 10:12:45 -04:00
Thibault Saunier
e1625da9ed
validate:launcher: Print the number of times the test was run when --forever
2019-04-11 10:38:46 -04:00
Thibault Saunier
d053a07706
validate:launcher: Use md viewer to dump md logs if avalaible
...
And enhance the markdown
See https://github.com/axiros/terminal_markdown_viewer
2019-04-11 13:17:14 +00:00
Thibault Saunier
8639dc528c
validate:launcher: Add an option to output HTML if commonmark is installed
2019-04-11 13:17:14 +00:00
Thibault Saunier
1294cb7b9b
validate:launcher: Move all logs to one single log files to be displayed to end user
2019-04-11 13:17:14 +00:00
Thibault Saunier
446f1d8197
validate:launcher: Make extra_logfiles a set and fix their names
2019-04-11 13:17:14 +00:00
Thibault Saunier
adbbfa1ec1
validate:launcher: Remove noise about empty known issues in logs
2019-04-11 13:17:14 +00:00
Thibault Saunier
148bb081b8
validate:launcher: Fix printing application name in the logs
2019-04-11 13:17:14 +00:00
Thibault Saunier
787939f750
validate:launcher: Add a list of well known subpression files from gst-build subprojects
...
Fixes https://gitlab.freedesktop.org/gstreamer/gst-devtools/issues/38
2019-04-03 13:38:42 +00:00
Thibault Saunier
92f1979ec9
validate: Add a mecanism to rerun failling tests
...
And add a way to mark some 'flakes' as tolerated
2019-04-03 13:38:42 +00:00
Thibault Saunier
520c2102cc
validate:launcher: Reset process on clean
2019-04-03 13:38:42 +00:00
Thibault Saunier
e279b1ff56
validate:launcher: Dump applied known issues in the logs
2019-04-03 13:38:42 +00:00
Thibault Saunier
b0cbae0fa7
validate:launcher: Print name of the test to be debugged
...
When --debug was passed
2019-04-03 13:38:42 +00:00
Thibault Saunier
abaa382847
validate: Generate dot files and link to them on CI
2019-04-03 13:38:42 +00:00
Thibault Saunier
28d413f059
validate:launcher: Add a way to say that a known issues can happen several times
2019-04-03 13:38:42 +00:00
Thibault Saunier
ba69336bf8
validate:launcher: Remove now useless code to check sending EOS brings down the pipeline
...
If we want to make an exception we can use known issues these days
2019-04-03 13:38:42 +00:00
Thibault Saunier
d73e283691
validate:launcher: Be a bit more resilient in GstValidateListener
...
And handle exception decoding received json info
2019-04-03 13:38:42 +00:00
Thibault Saunier
374917b9c4
validate:launcher: Add a way to require a non fatal error when matching a fatal one
2019-04-03 13:38:42 +00:00
Thibault Saunier
ddb81f29e8
validate:launcher: factor out TTY check and enhance iteration output
2019-04-03 13:38:42 +00:00
Thibault Saunier
0a2af1b027
validate:launcher: Add generated known issue list to the logs instead of printing then
2019-04-03 13:38:42 +00:00
Thibault Saunier
ac7efe9500
validate:launcher: Rework expected-issues data format
...
Instead of having the issues centered on the test classes, they
are now focusing on the "bug".
And harmise names on `expected_issue` not `expected_failures`
2019-04-03 13:38:42 +00:00
Thibault Saunier
6a4639352b
validate:launcher: Allow printing log URL on a CI server
2019-04-03 13:38:42 +00:00
Thibault Saunier
06b8f23a34
validate:launcher: Fix getting skip-parser when generating media descriptor
...
On old version it didn't exist
2019-04-03 13:38:42 +00:00
Thibault Saunier
a77825ecbc
validate:launcher: Allow referencing known issue by exiting signal names
2019-04-03 13:38:42 +00:00
Thibault Saunier
7dbdb84ca8
validate:launcher: Output markdown as much as possible
2019-04-03 13:38:42 +00:00
Thibault Saunier
3a826e1e3d
validate:launcher: Implement bug checks for gitlab
...
And use new gitlab urls for all the bugs
2019-04-03 13:38:42 +00:00
Thibault Saunier
ad28bffa20
validate:launcher: Take into account the timeout when checking EOS
2019-04-03 13:38:42 +00:00
Thibault Saunier
6f9e5d4494
Pass python files through autopep8
2019-04-03 13:38:42 +00:00
Thibault Saunier
091ce6bcfe
validate:launcher: Help the user add known issues
...
And make it clear a bug should be opened about it
2019-04-03 13:38:42 +00:00
Thibault Saunier
b4e96b4f4e
validate:launcher: Enhance dumping log files output
2019-04-03 13:38:42 +00:00
Thibault Saunier
3d5099ee41
validate:launcher: Use gst-integration-testsuites subproject as default testsuite repo if avalaible
2019-04-03 13:38:42 +00:00
Guillaume Desmottes
da22878a5f
validate: baseclasses: include env variable in logged command
...
We were missing the env variables in the command written to the log
file, making it impossible to re-run the test later from the logs.
2019-02-11 15:17:33 +00:00
Thibault Saunier
5a6307f913
validate:launcher: Do not sort tests all the time
...
Do it once only once it is fully populated
2019-02-09 17:25:03 -03:00
Alicia Boya García
e96f2ca714
New validate plugin: validateflow
...
validateflow can be used to check the buffers and events flowing through
a custom pipeline match an expectation file. This can be used to test
non-regular-playback use cases like demuxers handling adaptive streaming
fragment pushing.
This patch includes also new actions used for these cases:
`appsrc-push`, `appsrc-eos` and `flush` (plus `checkpoint`, which is
only available with validateflow).
2019-02-07 18:19:22 +00:00
Guillaume Desmottes
0c83ff56dd
baseclass: add_validate_config: don't use self.proc_env
...
self.proc_env is created when starting the test but this API can be call
by generator when creating the test.
2019-02-07 17:13:02 +01:00
Thibault Saunier
71d2e3f418
validate: Let people know the testsuite starts running
...
Now that we do not print infos about successful tests when redirecting.
2019-02-01 01:24:33 -03:00
Thibault Saunier
35003bc25b
validate:launcher: Do not print passing tests if not running in a tty
2019-01-31 23:47:11 -03:00
Mathieu Duponchelle
d5e3f4d31c
TestsManager: stop displaying blacklisted tests on stdout
...
It's basically spam, better suited to the debug logs
2019-01-30 00:27:32 +00:00
Thibault Saunier
440055214a
validate:launcher: Never print lines larger than the terminal
2019-01-29 16:09:30 -03:00
Thibault Saunier
2551e8750e
validate:launcher: Make baseclasses.py pep8 compliant
2019-01-26 15:54:00 +00:00
Thibault Saunier
9b69bcad08
validate:launcher: Handle launching launching a sub launcher
...
If you use validate-launcher in a meson testsuite, those test now
gets integrated as one unique testsuite (with a pretty long namespace).
2019-01-26 15:54:00 +00:00
Thibault Saunier
6665652cff
validate:launcher: Refactor the "main" function
...
- Move the parser code into a `LauncherConfig.create_parser()` method
- Remove the need to pass libsdir to the _TestsLauncher object
- Extract out a `setup_launcher_from_args` function
2019-01-26 15:54:00 +00:00