We will run a simple pipeline with the IQA element to run ssim (dssim)
tests on the rendered files, comparing it with a reference file.
For now we use the very empiric 1.0 value as a ssim error threshold and
the goal is basically to detect completely broken renderings.
The issue is closed upstream (because of concentrating on decodebin3
instead), and initial forever testing seems to show the issue doesn't
happen anymore
Instead of the test index in the list of tests as it is
meaningless to the user and feels weird.
Also minor fix in the test name display when running with --forever.
Make our stdout output simpler to follow by:
- Not printing the tests we launch (it is not really useful in the end)
- Using `\r` when printing the passed tests
- Not reprinting all the test in a now useless summary
This was keeping around 500-700kB of data for each test, which was
gradually raising memory usage of a full run by 100MB+
The reports are definitely not needed, and we only need to keep
information from the subprocess env variable that we might need
later on for final reporting
The xml-based MediaDescriptor were keeping open the XML file and the
associated ElementTree structures, resulting in memory usage of several
hundred megabytes.
Instead cache the information we need immediately and release the
XML structure
Since we now check position/status of pipeline at regular intevals,
we no longer need to impose a different timeout based on the
protocol used.
Avoids having 4min long timeouts for no reason (30s is enough)
Instead of creating a separate TCPServer for each test, just create
one which handles all connections in a threaded fashion.
Shaves off ~500ms per test
https://bugzilla.gnome.org/show_bug.cgi?id=791159
When the --shuffle option is used, the tests will be run out of order.
This optimizes CPU utilization since it allows running synchronized
and unsynchronized tests at the same.
stopping the subprocess is done from the main thread, this would
throttle starting/stopping any tests by one second.
Start with 50ms, and gradually increase the wait between iterations
So that Test from several TestManager can run in parallel and thus avoid
waiting for tests from one TestManager to run the following one.,
Also by design TestsLauncher should always have been the responsible for
... launching tests.
WARNING: The variable(s) 'DATADIR', 'LIBDIR' in the input file
'subprojects/gst-devtools/validate/launcher/config.py.in' are not
present in the given configuration data
WARNING: Passed invalid keyword argument "scanobj_args". This will
become a hard error in the future.
WARNING: Keyword argument "install" defined multiple times. This
will be a an error in future Meson releases.
Make sure to use it where appropriate and add some logging when
setting an object property from an action.
And use the valgrind.conf to set all the properties instead of having
a mixture of a config scenario and the config file (making sure the
max-lateness is set on any sink)
