Instead of having the issues centered on the test classes, they
are now focusing on the "bug".
And harmise names on `expected_issue` not `expected_failures`
validateflow can be used to check the buffers and events flowing through
a custom pipeline match an expectation file. This can be used to test
non-regular-playback use cases like demuxers handling adaptive streaming
fragment pushing.
This patch includes also new actions used for these cases:
`appsrc-push`, `appsrc-eos` and `flush` (plus `checkpoint`, which is
only available with validateflow).
- Move the parser code into a `LauncherConfig.create_parser()` method
- Remove the need to pass libsdir to the _TestsLauncher object
- Extract out a `setup_launcher_from_args` function
This patch removes the quotes surrounding the command shown by
gst-validate to reproduce the issues -- which were troublesome when
copying and pasting.
It also introduces escaping for the arguments, so that the command line
can be copied and pasted in the terminal without further changes.
https://bugzilla.gnome.org/show_bug.cgi?id=796897
Introducing the `.media_info.push` media info extension, which is meant
to let the launcher know that those file should run with the "pushfile://"
protocol.
And allow symlinking "normal" `.media_info` to their `.pushfile` variant
so that both can share the exact same content.
This is useful when you want to check only the demuxer output.
- Keep the information in the media file so that we can launch media-check
with the proper arguments in the launcher. Update it accordingly.
- Refactor compare_streams to simplify it, which in the end leads to
reporting all the issues instead of exiting on the first one.
Otherwise both gdb and gst-validate-launcher will react to ^C at the
same time, gdb will be killed by SIGHUP (because gst-validate-launcher
quitted in consequence of the ^C) and the terminal state will be left
garbled because readline inside gdb had disabled echo.
https://bugzilla.gnome.org/show_bug.cgi?id=796396
This patch modifies the default behavior of --gdb to not run and quit
automatically the test, but rather wait for user input. This is
usually much more convenient to debug all kinds of bugs.
The automatic run behavior has been moved to a new command switch:
--gdb-non-stop
https://bugzilla.gnome.org/show_bug.cgi?id=796389
We will run a simple pipeline with the IQA element to run ssim (dssim)
tests on the rendered files, comparing it with a reference file.
For now we use the very empiric 1.0 value as a ssim error threshold and
the goal is basically to detect completely broken renderings.
Instead of the test index in the list of tests as it is
meaningless to the user and feels weird.
Also minor fix in the test name display when running with --forever.
Make our stdout output simpler to follow by:
- Not printing the tests we launch (it is not really useful in the end)
- Using `\r` when printing the passed tests
- Not reprinting all the test in a now useless summary
This was keeping around 500-700kB of data for each test, which was
gradually raising memory usage of a full run by 100MB+
The reports are definitely not needed, and we only need to keep
information from the subprocess env variable that we might need
later on for final reporting
The xml-based MediaDescriptor were keeping open the XML file and the
associated ElementTree structures, resulting in memory usage of several
hundred megabytes.
Instead cache the information we need immediately and release the
XML structure
Instead of creating a separate TCPServer for each test, just create
one which handles all connections in a threaded fashion.
Shaves off ~500ms per test
https://bugzilla.gnome.org/show_bug.cgi?id=791159
When the --shuffle option is used, the tests will be run out of order.
This optimizes CPU utilization since it allows running synchronized
and unsynchronized tests at the same.
So that Test from several TestManager can run in parallel and thus avoid
waiting for tests from one TestManager to run the following one.,
Also by design TestsLauncher should always have been the responsible for
... launching tests.
Make sure to use it where appropriate and add some logging when
setting an object property from an action.
And use the valgrind.conf to set all the properties instead of having
a mixture of a config scenario and the config file (making sure the
max-lateness is set on any sink)
Note: The notion of "live" here is in the *content* sense and not in the
GStreamer sense.
Ex:
* A rtsp stream is always "live" in the GStreamer sense but might not always
provide live content.
* HLS/DASH streams are not "live" in the GStreamer sense but might
provide "live" content.
Some scenarios might:
* require live content
* not be compatible with live content
This patch adds two new properties for scenarios:
* live_content_required (default False) for scenarios that can only work with
live content.
* live_content_compatible (default False) for scenarios that can work with
both live and non-live content.
This patch adds support for reading a "live" property from stream_info
