validateflow can be used to check the buffers and events flowing through
a custom pipeline match an expectation file. This can be used to test
non-regular-playback use cases like demuxers handling adaptive streaming
fragment pushing.
This patch includes also new actions used for these cases:
`appsrc-push`, `appsrc-eos` and `flush` (plus `checkpoint`, which is
only available with validateflow).
- Move the parser code into a `LauncherConfig.create_parser()` method
- Remove the need to pass libsdir to the _TestsLauncher object
- Extract out a `setup_launcher_from_args` function
When defining pipelines_descriptions to run test on in a `.json` file, you might
need to point to paths in the testsuite directory (for media files URIs
for example), you can now do
`"pipeline": "filesrc location="$(config_path)s/../medias/some/file.mkv...`
I think it was a mistake to call them <error> as the two notions are
different (we marked failed test as "failures" in the <testuite> node).
Should make gitlab happy with our file!
This is generally usefull so we do not have to pass -M every time we launch the launcher
And it adds support for nesting launcher calls always respecting the provided main directory
+ Fix some new pep8 errors
This patch removes the quotes surrounding the command shown by
gst-validate to reproduce the issues -- which were troublesome when
copying and pasting.
It also introduces escaping for the arguments, so that the command line
can be copied and pasted in the terminal without further changes.
https://bugzilla.gnome.org/show_bug.cgi?id=796897
Introducing the `.media_info.push` media info extension, which is meant
to let the launcher know that those file should run with the "pushfile://"
protocol.
And allow symlinking "normal" `.media_info` to their `.pushfile` variant
so that both can share the exact same content.
This is useful when you want to check only the demuxer output.
- Keep the information in the media file so that we can launch media-check
with the proper arguments in the launcher. Update it accordingly.
- Refactor compare_streams to simplify it, which in the end leads to
reporting all the issues instead of exiting on the first one.
Otherwise both gdb and gst-validate-launcher will react to ^C at the
same time, gdb will be killed by SIGHUP (because gst-validate-launcher
quitted in consequence of the ^C) and the terminal state will be left
garbled because readline inside gdb had disabled echo.
https://bugzilla.gnome.org/show_bug.cgi?id=796396
This patch modifies the default behavior of --gdb to not run and quit
automatically the test, but rather wait for user input. This is
usually much more convenient to debug all kinds of bugs.
The automatic run behavior has been moved to a new command switch:
--gdb-non-stop
https://bugzilla.gnome.org/show_bug.cgi?id=796389
We will run a simple pipeline with the IQA element to run ssim (dssim)
tests on the rendered files, comparing it with a reference file.
For now we use the very empiric 1.0 value as a ssim error threshold and
the goal is basically to detect completely broken renderings.
The issue is closed upstream (because of concentrating on decodebin3
instead), and initial forever testing seems to show the issue doesn't
happen anymore
Instead of the test index in the list of tests as it is
meaningless to the user and feels weird.
Also minor fix in the test name display when running with --forever.
Make our stdout output simpler to follow by:
- Not printing the tests we launch (it is not really useful in the end)
- Using `\r` when printing the passed tests
- Not reprinting all the test in a now useless summary
This was keeping around 500-700kB of data for each test, which was
gradually raising memory usage of a full run by 100MB+
The reports are definitely not needed, and we only need to keep
information from the subprocess env variable that we might need
later on for final reporting
The xml-based MediaDescriptor were keeping open the XML file and the
associated ElementTree structures, resulting in memory usage of several
hundred megabytes.
Instead cache the information we need immediately and release the
XML structure
Since we now check position/status of pipeline at regular intevals,
we no longer need to impose a different timeout based on the
protocol used.
Avoids having 4min long timeouts for no reason (30s is enough)
Instead of creating a separate TCPServer for each test, just create
one which handles all connections in a threaded fashion.
Shaves off ~500ms per test
https://bugzilla.gnome.org/show_bug.cgi?id=791159
When the --shuffle option is used, the tests will be run out of order.
This optimizes CPU utilization since it allows running synchronized
and unsynchronized tests at the same.
stopping the subprocess is done from the main thread, this would
throttle starting/stopping any tests by one second.
Start with 50ms, and gradually increase the wait between iterations
So that Test from several TestManager can run in parallel and thus avoid
waiting for tests from one TestManager to run the following one.,
Also by design TestsLauncher should always have been the responsible for
... launching tests.
WARNING: The variable(s) 'DATADIR', 'LIBDIR' in the input file
'subprojects/gst-devtools/validate/launcher/config.py.in' are not
present in the given configuration data
WARNING: Passed invalid keyword argument "scanobj_args". This will
become a hard error in the future.
WARNING: Keyword argument "install" defined multiple times. This
will be a an error in future Meson releases.
Make sure to use it where appropriate and add some logging when
setting an object property from an action.
And use the valgrind.conf to set all the properties instead of having
a mixture of a config scenario and the config file (making sure the
max-lateness is set on any sink)
