Thibault Saunier
d053a07706
validate:launcher: Use md viewer to dump md logs if avalaible
...
And enhance the markdown
See https://github.com/axiros/terminal_markdown_viewer
2019-04-11 13:17:14 +00:00
Thibault Saunier
8639dc528c
validate:launcher: Add an option to output HTML if commonmark is installed
2019-04-11 13:17:14 +00:00
Thibault Saunier
1294cb7b9b
validate:launcher: Move all logs to one single log files to be displayed to end user
2019-04-11 13:17:14 +00:00
Thibault Saunier
446f1d8197
validate:launcher: Make extra_logfiles a set and fix their names
2019-04-11 13:17:14 +00:00
Thibault Saunier
adbbfa1ec1
validate:launcher: Remove noise about empty known issues in logs
2019-04-11 13:17:14 +00:00
Thibault Saunier
148bb081b8
validate:launcher: Fix printing application name in the logs
2019-04-11 13:17:14 +00:00
Thibault Saunier
787939f750
validate:launcher: Add a list of well known subpression files from gst-build subprojects
...
Fixes https://gitlab.freedesktop.org/gstreamer/gst-devtools/issues/38
2019-04-03 13:38:42 +00:00
Thibault Saunier
92f1979ec9
validate: Add a mecanism to rerun failling tests
...
And add a way to mark some 'flakes' as tolerated
2019-04-03 13:38:42 +00:00
Thibault Saunier
520c2102cc
validate:launcher: Reset process on clean
2019-04-03 13:38:42 +00:00
Thibault Saunier
e279b1ff56
validate:launcher: Dump applied known issues in the logs
2019-04-03 13:38:42 +00:00
Thibault Saunier
b0cbae0fa7
validate:launcher: Print name of the test to be debugged
...
When --debug was passed
2019-04-03 13:38:42 +00:00
Thibault Saunier
abaa382847
validate: Generate dot files and link to them on CI
2019-04-03 13:38:42 +00:00
Thibault Saunier
28d413f059
validate:launcher: Add a way to say that a known issues can happen several times
2019-04-03 13:38:42 +00:00
Thibault Saunier
ba69336bf8
validate:launcher: Remove now useless code to check sending EOS brings down the pipeline
...
If we want to make an exception we can use known issues these days
2019-04-03 13:38:42 +00:00
Thibault Saunier
d73e283691
validate:launcher: Be a bit more resilient in GstValidateListener
...
And handle exception decoding received json info
2019-04-03 13:38:42 +00:00
Thibault Saunier
374917b9c4
validate:launcher: Add a way to require a non fatal error when matching a fatal one
2019-04-03 13:38:42 +00:00
Thibault Saunier
ddb81f29e8
validate:launcher: factor out TTY check and enhance iteration output
2019-04-03 13:38:42 +00:00
Thibault Saunier
0a2af1b027
validate:launcher: Add generated known issue list to the logs instead of printing then
2019-04-03 13:38:42 +00:00
Thibault Saunier
ac7efe9500
validate:launcher: Rework expected-issues data format
...
Instead of having the issues centered on the test classes, they
are now focusing on the "bug".
And harmise names on `expected_issue` not `expected_failures`
2019-04-03 13:38:42 +00:00
Thibault Saunier
6a4639352b
validate:launcher: Allow printing log URL on a CI server
2019-04-03 13:38:42 +00:00
Thibault Saunier
06b8f23a34
validate:launcher: Fix getting skip-parser when generating media descriptor
...
On old version it didn't exist
2019-04-03 13:38:42 +00:00
Thibault Saunier
a77825ecbc
validate:launcher: Allow referencing known issue by exiting signal names
2019-04-03 13:38:42 +00:00
Thibault Saunier
7dbdb84ca8
validate:launcher: Output markdown as much as possible
2019-04-03 13:38:42 +00:00
Thibault Saunier
3a826e1e3d
validate:launcher: Implement bug checks for gitlab
...
And use new gitlab urls for all the bugs
2019-04-03 13:38:42 +00:00
Thibault Saunier
ad28bffa20
validate:launcher: Take into account the timeout when checking EOS
2019-04-03 13:38:42 +00:00
Thibault Saunier
6f9e5d4494
Pass python files through autopep8
2019-04-03 13:38:42 +00:00
Thibault Saunier
091ce6bcfe
validate:launcher: Help the user add known issues
...
And make it clear a bug should be opened about it
2019-04-03 13:38:42 +00:00
Thibault Saunier
b4e96b4f4e
validate:launcher: Enhance dumping log files output
2019-04-03 13:38:42 +00:00
Thibault Saunier
3d5099ee41
validate:launcher: Use gst-integration-testsuites subproject as default testsuite repo if avalaible
2019-04-03 13:38:42 +00:00
Guillaume Desmottes
da22878a5f
validate: baseclasses: include env variable in logged command
...
We were missing the env variables in the command written to the log
file, making it impossible to re-run the test later from the logs.
2019-02-11 15:17:33 +00:00
Thibault Saunier
5a6307f913
validate:launcher: Do not sort tests all the time
...
Do it once only once it is fully populated
2019-02-09 17:25:03 -03:00
Alicia Boya García
e96f2ca714
New validate plugin: validateflow
...
validateflow can be used to check the buffers and events flowing through
a custom pipeline match an expectation file. This can be used to test
non-regular-playback use cases like demuxers handling adaptive streaming
fragment pushing.
This patch includes also new actions used for these cases:
`appsrc-push`, `appsrc-eos` and `flush` (plus `checkpoint`, which is
only available with validateflow).
2019-02-07 18:19:22 +00:00
Guillaume Desmottes
0c83ff56dd
baseclass: add_validate_config: don't use self.proc_env
...
self.proc_env is created when starting the test but this API can be call
by generator when creating the test.
2019-02-07 17:13:02 +01:00
Thibault Saunier
71d2e3f418
validate: Let people know the testsuite starts running
...
Now that we do not print infos about successful tests when redirecting.
2019-02-01 01:24:33 -03:00
Thibault Saunier
35003bc25b
validate:launcher: Do not print passing tests if not running in a tty
2019-01-31 23:47:11 -03:00
Mathieu Duponchelle
d5e3f4d31c
TestsManager: stop displaying blacklisted tests on stdout
...
It's basically spam, better suited to the debug logs
2019-01-30 00:27:32 +00:00
Thibault Saunier
440055214a
validate:launcher: Never print lines larger than the terminal
2019-01-29 16:09:30 -03:00
Thibault Saunier
2551e8750e
validate:launcher: Make baseclasses.py pep8 compliant
2019-01-26 15:54:00 +00:00
Thibault Saunier
9b69bcad08
validate:launcher: Handle launching launching a sub launcher
...
If you use validate-launcher in a meson testsuite, those test now
gets integrated as one unique testsuite (with a pretty long namespace).
2019-01-26 15:54:00 +00:00
Thibault Saunier
6665652cff
validate:launcher: Refactor the "main" function
...
- Move the parser code into a `LauncherConfig.create_parser()` method
- Remove the need to pass libsdir to the _TestsLauncher object
- Extract out a `setup_launcher_from_args` function
2019-01-26 15:54:00 +00:00
Thibault Saunier
1b3867b82d
launcher: Move http serveur and xvfb server to the main test runner object
...
No good reason for it to be in the main function
2019-01-26 15:54:00 +00:00
Thibault Saunier
7befe3d033
validate:launcher: Display unix nickname of signals leading to test failure
2019-01-15 16:55:32 -03:00
Thibault Saunier
5692f8d74d
validate:launcher: Fix error message about 'crashed' test
2019-01-15 16:05:41 -03:00
Guillaume Desmottes
b1f22e7130
validate: fix crash if timeout when media_descriptor is None
...
Some tests may not have any media_descriptor. If those were failing to
shutdown after EOS we were calling get_protocol() on None.
2018-12-17 10:34:43 +01:00
Thibault Saunier
72995d5bbe
validate: launcher: Add a way to retrieve trace without coredumpctl
...
Simply spnning on segfaults (like gst-launch) and catch that in
the launcher to transform the timeout into a segfault and grab a gdb
backtrace
2018-12-07 09:06:50 -03:00
Thibault Saunier
64afff0c3e
launcher:scenario: Fix the way we compute scenario path/name when paths supplied
...
We were just iterating over the list without any check and could end
up with a patch that was not corresponding to the actual scenario.
2018-09-08 11:13:19 -03:00
Thibault Saunier
f0cfdf9d14
validate:launcher: Use fakevideosink everywhere it makes sense.
2018-09-07 16:19:40 -03:00
Alicia Boya García
5ff299f4e7
gst-validate-launcher: Print copypaste-friendlier commands
...
This patch removes the quotes surrounding the command shown by
gst-validate to reproduce the issues -- which were troublesome when
copying and pasting.
It also introduces escaping for the arguments, so that the command line
can be copied and pasted in the terminal without further changes.
https://bugzilla.gnome.org/show_bug.cgi?id=796897
2018-07-30 16:13:25 -04:00
Thibault Saunier
615a372274
validate: launcher: Print some ERROR log when inspecting scenario fails
2018-07-21 11:43:40 -04:00
Thibault Saunier
4f009f4098
validate:launcher: Minor output string message
2018-07-12 19:13:21 -04:00