Commit graph

1608 commits

Author SHA1 Message Date
Thibault Saunier
3f668f3e80 validate: media-check: Also check that segments are correct 2018-06-15 12:05:13 -04:00
Edward Hervey
1a9455d2f6 validate/media-descriptor: Fix indentation 2018-06-05 16:38:10 +02:00
Edward Hervey
9af908195c validate: Add a new issue to detect invalid event seqnum
Events should always have a valid seqnum. Add a new issue which
allows detecting such events. And use that check in the
pad monitor
2018-06-05 16:37:35 +02:00
Edward Hervey
18b0d109f2 validate: Update all gitignore 2018-06-05 16:25:46 +02:00
Alicia Boya García
40dfb7174e gst-validate-launcher: let gdb handle SIGINT itself
Otherwise both gdb and gst-validate-launcher will react to ^C at the
same time, gdb will be killed by SIGHUP (because gst-validate-launcher
quitted in consequence of the ^C) and the terminal state will be left
garbled because readline inside gdb had disabled echo.

https://bugzilla.gnome.org/show_bug.cgi?id=796396
2018-05-25 12:59:41 +02:00
Alicia Boya García
bfa143caa9 gst-validate-launcher: disable timeouts when debugging in gdb interactively
An interactive debugging session can be going for a long time, we don't
want any timeouts in that case.

https://bugzilla.gnome.org/show_bug.cgi?id=796397
2018-05-25 12:59:40 +02:00
Alicia Boya García
2c32ccd7a7 gst-validate-launcher: Stop in --gdb by default, add --gdb-non-stop
This patch modifies the default behavior of --gdb to not run and quit
automatically the test, but rather wait for user input. This is
usually much more convenient to debug all kinds of bugs.

The automatic run behavior has been moved to a new command switch:
--gdb-non-stop

https://bugzilla.gnome.org/show_bug.cgi?id=796389
2018-05-25 12:59:38 +02:00
Thibault Saunier
683b19895e validate: Error out if gst_parse_launch sets an error.
https://bugzilla.gnome.org/show_bug.cgi?id=796240
2018-05-18 11:53:40 -04:00
Thibault Saunier
bb71fd9944 validate:launcher: Error out loudly if the testlist changes
When --fail-on-testlist-change is set.
2018-05-15 14:45:54 -04:00
Thibault Saunier
1055540d55 validate:launcher: Do not print time spent if the testsuite never started 2018-05-15 14:45:52 -04:00
Thibault Saunier
2dc7dd1257 validate: launcher: Make sure testsuites are used/configured once only 2018-05-13 19:10:59 -04:00
Thibault Saunier
ef9ff93405 validate:launcher: Add a way to check if a gst feature is present
And make sure iqa is present to run IQA tests.
2018-05-13 16:35:41 -04:00
Thibault Saunier
b00ab02254 validate:launcher: Cleanup the way we find where -validate tools are 2018-05-13 09:44:17 -04:00
Thibault Saunier
c1f89b4acb validate: launcher: Add a way to simply run SSIM checks on rendered files
We will run a simple pipeline with the IQA element to run ssim (dssim)
tests on the rendered files, comparing it with a reference file.

For now we use the very empiric 1.0 value as a ssim error threshold and
the goal is basically to detect completely broken renderings.
2018-05-13 09:44:17 -04:00
Thibault Saunier
c93f1704f8 validate: Add support for the new testbin protocol 2018-05-13 09:44:17 -04:00
Edward Hervey
58fb4f9354 validate: Remove hls.*seek_with_stop blacklisting
The issue is closed upstream (because of concentrating on decodebin3
instead), and initial forever testing seems to show the issue doesn't
happen anymore
2018-05-07 17:30:13 +02:00
Nirbheek Chauhan
21e7ed5025 meson: Update option names to omit disable_ prefixes
Also yield common options to the outer project (gst-build in our case)
so that they don't have to be set manually.
2018-05-05 20:27:50 +05:30
Thibault Saunier
ba3b27fa83 validate: pipeline: Handle the case where a pad has no monitor
We do not monitor ghost pads, only real pads, so this is a totally
legitimate case.

https://bugzilla.gnome.org/show_bug.cgi?id=792536
2018-04-27 17:33:29 +02:00
Kai Kang
d9256865b4 validate: fix out of source tree build error
It fails to generate gst-validate-enum-types.h and gst-validate-enum-types.c
when build out of source tree. Add the path for template files.

https://bugzilla.gnome.org/show_bug.cgi?id=795531

Signed-off-by: Kai Kang <kai.kang@windriver.com>
2018-04-25 08:54:46 -03:00
Thibault Saunier
77ee198b1b validate:launcher: Add support for specifying a workdir in tests 2018-04-15 20:50:22 -03:00
Thibault Saunier
25e3b90225 validate:launcher: printc is accepting object as arguments
Make sure in all code paths those are converted to strings
2018-04-15 20:07:19 -03:00
Thibault Saunier
8382fddbe9 validate:launcher: Add a method to find tests in a TestManager 2018-04-12 23:30:19 -03:00
Thibault Saunier
55dee0b8a6 validate:launcher: Extract method to set a validate configuration on tests 2018-04-12 23:07:16 -03:00
Thibault Saunier
b9ea967275 validate: Handle having a list of structure based/file path configs 2018-04-12 23:03:04 -03:00
Thibault Saunier
a70658a464 launcher: Print test number in the order they finish
Instead of the test index in the list of tests as it is
meaningless to the user and feels weird.

Also minor fix in the test name display when running with --forever.
2018-03-23 21:04:46 -03:00
Thibault Saunier
cd1c4eb44d launcher: Minor indentation issue fixes 2018-03-23 18:12:16 -03:00
Thibault Saunier
2b40c6c306 launcher: Clean up outpout
Make our stdout output simpler to follow by:
  - Not printing the tests we launch (it is not really useful in the end)
  - Using `\r` when printing the passed tests
  - Not reprinting all the test in a now useless summary
2018-03-23 18:12:16 -03:00
Thibault Saunier
3f66772fa1 check: Use meson introspect to list meson tests 2018-03-20 08:51:04 -03:00
Tim-Philipp Müller
dede83a542 Back to development 2018-03-20 10:57:53 +00:00
Tim-Philipp Müller
207c447a40 Release 1.14.0 2018-03-19 20:29:08 +00:00
Tim-Philipp Müller
8e82a9dd43 Release 1.13.91 2018-03-13 19:30:44 +00:00
Tim-Philipp Müller
34d56abedd validate: GST_EXPORT -> GST_VALIDATE_API
We need different export decorators for the different libs.
For now no actual change though, just rename before the release,
and add prelude headers to define the new decorator to GST_EXPORT.
2018-03-13 14:00:01 +00:00
Tim-Philipp Müller
da907da8e4 Release 1.13.90 2018-03-03 22:54:59 +00:00
Tim-Philipp Müller
9fdfb2d7b7 validate: tools: fix build dependencies for validate-rtsp-server
In file included from ../subprojects/gst-devtools/validate/tools/gst-validate-rtsp-server.c:21:0:
.../gst/gst.h:31:10: fatal error: gst/gstenumtypes.h: No such file or directory
2018-02-19 12:02:04 +00:00
Sebastian Dröge
0b9218f23f validategtk: Stop using deprecated keymap API
gstvalidategtk.c:184:7: error: ‘gdk_keymap_get_default’ is deprecated: Use 'gdk_keymap_get_for_display' instead [-Werror=deprecated-declarations]
       gdk_keymap_get_entries_for_keyval (gdk_keymap_get_default (),
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2018-02-18 12:21:34 +02:00
Tim-Philipp Müller
32d146b7de Fix versions 2018-02-15 21:04:21 +00:00
Tim-Philipp Müller
41abaf99a9 Back to development 2018-02-15 19:44:37 +00:00
Tim-Philipp Müller
dab15df5eb Release 1.13.1 2018-02-15 18:28:13 +00:00
Tim-Philipp Müller
4906612269 validate: dist enum types templates 2018-02-15 18:28:13 +00:00
Edward Hervey
55274bc89f validate-pad-monitor: Use GST_SEQNUM_INVALID
Instead of 0 (which is valid)
2018-02-10 14:20:44 +01:00
Wonchul Lee
0235ad6d03 tools: gst-validate-images-check: Fix typo
https://bugzilla.gnome.org/show_bug.cgi?id=792035
2018-01-31 17:40:56 +00:00
Edward Hervey
d71d28523d validate/baseclasses: Release un-needed data when test ends
This was keeping around 500-700kB of data for each test, which was
gradually raising memory usage of a full run by 100MB+

The reports are definitely not needed, and we only need to keep
information from the subprocess env variable that we might need
later on for final reporting
2017-12-18 10:51:05 +01:00
Edward Hervey
cb04515cbd validate/baseclasses: Don't leak several hundred MB of XML
The xml-based MediaDescriptor were keeping open the XML file and the
associated ElementTree structures, resulting in memory usage of several
hundred megabytes.

Instead cache the information we need immediately and release the
XML structure
2017-12-18 09:48:21 +01:00
Nicolas Dufresne
aa8e27f2a3 meson: Add missing force_rtsp2 scenario
Without this file, gst-validate installed using meson will fail all
RTSP2 tests
2017-12-17 16:22:51 -05:00
Edward Hervey
01a54641f9 validate: Remove protocol-specific timeouts
Since we now check position/status of pipeline at regular intevals,
we no longer need to impose a different timeout based on the
protocol used.

Avoids having 4min long timeouts for no reason (30s is enough)
2017-12-03 12:23:51 +01:00
Edward Hervey
92285ef261 validate: Use a single TCPServer for subprocess communication
Instead of creating a separate TCPServer for each test, just create
one which handles all connections in a threaded fashion.

Shaves off ~500ms per test

https://bugzilla.gnome.org/show_bug.cgi?id=791159
2017-12-03 12:23:29 +01:00
Edward Hervey
58e62f651c validate-launcher: Allow running tests out-of-order
When the --shuffle option is used, the tests will be run out of order.
This optimizes CPU utilization since it allows running synchronized
and unsynchronized tests at the same.
2017-12-03 11:42:18 +01:00
Edward Hervey
1a95559045 validate: Reduce time waiting for subprocess to stop
stopping the subprocess is done from the main thread, this would
throttle starting/stopping any tests by one second.
Start with 50ms, and gradually increase the wait between iterations
2017-12-03 11:38:08 +01:00
Edward Hervey
65e2c1567a gstvalidate: Lower timeout to check for rtsp-server to be up
Check every 100ms, avoids throttling all rtsp tests by 500ms
2017-12-03 11:38:08 +01:00
Thibault Saunier
f16b900643 validate:launcher: Launch tests in _TestsLauncher not in TestsManagaer
So that Test from several TestManager can run in parallel and thus avoid
waiting for tests from one TestManager to run the following one.,

Also by design TestsLauncher should always have been the responsible for
... launching tests.
2017-12-02 09:42:19 -03:00
Edward Hervey
9acfa7fe4b validate: Don't leak strings
We only use them in the error/debug case anyway
2017-11-25 13:10:41 +01:00
Edward Hervey
c33b2e240c validate-report: Plug leaks
The trace was never freed, nor were the output of g_str_split
2017-11-25 12:46:05 +01:00
Edward Hervey
b5bb7d7016 validate-scenario: Don't assume element have factories
Some elements might not originate from factories (like custom/internal
elements).

Avoids dereferencing a NULL pointer
2017-11-23 12:27:11 +01:00
Edward Hervey
ff6055b195 validate-scenario: Handle non-relative switch
Make sure we stay within the number of present streams (and avoid
out-of-bound read).

CID #1415470
2017-11-22 16:37:44 +01:00
Edward Hervey
81c3ab0b12 validate-scenario: Handle switching stream of type not present
Unlikely to happen, but at least don't end up doing unsafe calculation
with n == 0 afterwards

CID #1415453
2017-11-22 16:37:44 +01:00
Nirbheek Chauhan
aa7f753f8a meson: Always require the latest gst-rtsp-server
In the worst case, when building with gst-uninstalled, we will try to
link against an older gst-rtsp-server provided by the system. Found by
philn.
2017-11-12 20:09:53 +05:30
Edward Hervey
9658e12086 validate: Call g-ir-scanner with the same toolchain as the rest 2017-11-08 17:22:47 +01:00
Edward Hervey
2eb3df74b5 validate-analyze: Update for xml format changes 2017-10-27 10:00:07 +02:00
Thibault Saunier
4fac7bf9fd validate: launcher: Run rtsp tests against both V1 and V2
https://bugzilla.gnome.org/show_bug.cgi?id=781446
2017-09-22 16:31:56 -03:00
Thibault Saunier
e9862b9fda validate:launcher: Allow disabling using the number of failed tests as exitcode
This is usefull on CI servers where the test results will be inspected
and the status of the build built from it.
2017-09-06 16:38:39 -03:00
Jimmy Ohn
bd0e8e410e validate: launcher: Modify the order of the parser argument
Modify the order of the parser argument before setting dir_group

https://bugzilla.gnome.org/show_bug.cgi?id=786715
2017-08-28 17:01:39 -03:00
Thibault Saunier
4dc95f0fc2 launcher: Automatically disable output coloration if not supported 2017-08-26 10:52:25 -03:00
Thibault Saunier
598128fc0c meson: Fix the way we set the testsuite version 2017-08-26 09:28:51 -03:00
Thibault Saunier
2c6c25d4bf validate:launcher: Use the number of failed test as exit code
We used to always return 0, which was not right!
2017-08-18 11:37:28 -03:00
Thibault Saunier
cb99482b9a launcher: Avoid exceptions when inspecting renders files
We were a bit to strict on the Exception types which lead to
the launcher failling itself when it shouldn't
2017-08-14 16:57:50 -03:00
Tim-Philipp Müller
8b91fef492 validate: hide some private symbols 2017-08-12 12:04:42 +01:00
Thibault Saunier
8393a3d5ca validate: Plug a potential leak when retrieving peer pad 2017-08-10 21:44:14 -04:00
Thibault Saunier
7be8ecd628 validate: Fix going over ghostpads/proxypads 2017-08-10 19:29:07 -04:00
Tim-Philipp Müller
6b661f394f meson: fix a few meson warnings
WARNING: The variable(s) 'DATADIR', 'LIBDIR' in the input file
  'subprojects/gst-devtools/validate/launcher/config.py.in' are not
  present in the given configuration data
WARNING: Passed invalid keyword argument "scanobj_args". This will
  become a hard error in the future.
WARNING: Keyword argument "install" defined multiple times. This
  will be a an error in future Meson releases.
2017-08-10 14:35:09 +01:00
Thibault Saunier
7a83d2aed7 validate:win32: Update .def file. 2017-08-07 16:00:53 -04:00
Thibault Saunier
74a560163a validate: Mark symbols explicitly for export with GST_EXPORT
With an exception:
  * gst_validate_monitor_setup

which was never declared in headers and should always have been static.
2017-08-07 15:56:21 -04:00
Thibault Saunier
823c9ca834 validate: Fix building the tracer
It fails on some platforms, I guess this is the reason
2017-07-26 17:25:25 -04:00
Thibault Saunier
8ef1050d1e validate: Also mkenums with autotools
And fix the build with stricter gcc arguments.
2017-07-26 16:18:02 -04:00
Thibault Saunier
b9d6f9df9e validate: Add a way to print information about pipeline status
Similare to what is done with gst-launch.

And finally generate GTypes for our flags and enums.
2017-07-26 15:45:37 -04:00
Edward Hervey
40eb48d21f validate-scenario: Fix NULL pointer usage
for good this time ...

CID #1415570
2017-07-26 15:22:49 +02:00
Edward Hervey
1b48ffdff5 validate-report: Fix a leak in error cases
CID #1415494
2017-07-26 15:18:57 +02:00
Edward Hervey
9ae20ee3e0 validate: Re-enable mxf op2b tests
https://bugzilla.gnome.org/show_bug.cgi?id=785119
2017-07-25 09:55:02 +02:00
Thibault Saunier
be95f623b7 wind32: Update .def file. 2017-07-21 10:30:37 -04:00
Edward Hervey
6ad8bd3e7c validate-scenario: Protect against priv NULL usage
CID #1415570
2017-07-20 14:21:59 +02:00
Thibault Saunier
2fb3545aa7 validate: Do not install now removed setup_sink_props_max_lateness.scenario file 2017-07-19 12:16:53 -04:00
Thibault Saunier
8ab723b153 validate:launcher:check: Make sure to register tests from the testsuite
Instead of having them listed from the app manager. This is needed
to avoid backtrace as tests now have to be register when setting up
the testsuite.
2017-07-19 11:49:54 -04:00
Thibault Saunier
1a28e7b043 validate: Factor out a method to set properties on elements in utils
Make sure to use it where appropriate and add some logging when
setting an object property from an action.

And use the valgrind.conf to set all the properties instead of having
a mixture of a config scenario and the config file (making sure the
max-lateness is set on any sink)
2017-07-19 11:34:19 -04:00
Thibault Saunier
1d568ff11f validate:scenario: Allow not config action to be executed from config files
When those are special cased to support that, such as the `set-property`
action.

This special handling was added in

  4927c65710
  validate: disable QOS features when running with valgrind

before we started to support executing arbitrary config action from
configuration files.
2017-07-19 11:34:12 -04:00
Thibault Saunier
b3134e89d9 validate: scenario: Fix running config action from the config file 2017-07-19 10:23:34 -04:00
Edward Hervey
1cfe980044 validate: Blacklist op2b mxf files
See https://bugzilla.gnome.org/show_bug.cgi?id=785119
2017-07-19 15:47:28 +02:00
Edward Hervey
8b9b6ead3f validate: Cast GList data content before usage
Apart from code readability, it allows compilers to detect wrong usages,
such as the call to gst_validate_action_new() which was using the wrong
argument
2017-07-19 11:02:44 +02:00
Thibault Saunier
2177d8589c validate:launcher: Error out if no testsuite could be loaded 2017-07-18 12:09:13 -04:00
Thibault Saunier
2cf93f491b validate: launcher: Namespace test name with the testsuite name
Also allowing users to pass test names directly
2017-07-18 10:54:53 -04:00
Thibault Saunier
f9ef2bc56a validate: launcher: Properly use TestsLauncher.list_test to load tests
Otherwise we might skip check_defined_tests.
2017-07-18 10:54:21 -04:00
Thibault Saunier
f7252cbd48 validate: Remove some dead code
CID 1415457
2017-07-18 10:47:00 -04:00
Thibault Saunier
e70557dc3e validate: Plug a minor string leak
CID 1415459
2017-07-18 10:45:29 -04:00
Thibault Saunier
9ee7b4483c validate: Don't create scenario on a monitor which has no target
Not very probable but avoids a potential NULL pointer dereferencing.

CID 1415460
2017-07-18 10:42:00 -04:00
Thibault Saunier
6572af916b validate: Always only use the first description in scenarios
Also pluging a leak of the descrption copied structure

CID 1415463
2017-07-18 10:38:24 -04:00
Thibault Saunier
7463819969 validate: ssim: Do not compare unsigned to < 0
CID 1415473
2017-07-18 10:27:49 -04:00
Thibault Saunier
c7483a90f0 validate: Plug a string leak
CID 1415475
2017-07-18 10:23:31 -04:00
Thibault Saunier
566adba269 validate:ssim: Let user know when no file have been compared
Fixing a possible division by zero issue.

CID 1415482
2017-07-18 10:19:23 -04:00
Thibault Saunier
28f8787c17 validate: Add missing break statement
CID 1415485
2017-07-18 10:16:07 -04:00
Thibault Saunier
7eb9746adc Check g_file_set_contents() return value
CID 1415486
2017-07-18 10:14:59 -04:00
Thibault Saunier
44260d70bd validate: Plug leak of copy of a va_list
CID 1415490
2017-07-18 10:12:07 -04:00
Thibault Saunier
11ef28f3dc validate: Do not check NULL pointer uselessly
CID 141593
2017-07-18 10:10:06 -04:00