This introduce new command line options, --parts and --part-index. When
--parts is set to a value larger then 1, the tests will be split in the
same number of group. The group number identified by --part-index will
be executed.
This is being added in orther to support gliblab CI parallel feature.
It is not safe for `_preformat_levels` to assume that all of the fields in a
`TerminalController` have the same type; at least in my environment, some of
these fields are populated with `bytes` while others remain strings.
This change conditionally applies decoding to each control sequence separately
using a helper function `_as_string`. As a side-effect, it also eliminates some
code repetition in `_preformat_levels`.
Closes#50.
The colon character commonly used to separate the element name and the
pad name is reserved in Windows filesystems, so it's better to use
something safer.
This patch replaces it with '-'. Please update
gst-integration-testsuites too where another commit has renamed all the
files.
`gst_pad_pull_range` should always be called from the streaming thread,
we now check that when pull_range is called, and if the sinkpad calling
the function has a GstTask with a running thread, the function is called
from that thread.
In file included from ../../../../dist/linux_x86_64/include/gstreamer-1.0/gst/gst.h:55,
from ../validate/gst/validate/gst-validate-scenario.c:45:
../validate/gst/validate/gst-validate-scenario.c: In function ‘gst_validate_scenario_load’:
../../../../dist/linux_x86_64/include/gstreamer-1.0/gst/gstinfo.h:645:5: error: ‘%s’ directive argument is null [-Werror=format-overflow=]
645 | gst_debug_log ((cat), (level), __FILE__, GST_FUNCTION, __LINE__, \
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
646 | (GObject *) (object), __VA_ARGS__); \
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../../../dist/linux_x86_64/include/gstreamer-1.0/gst/gstinfo.h:1067:26: note: in expansion of macro ‘GST_CAT_LEVEL_LOG’
1067 | #define GST_ERROR(...) GST_CAT_LEVEL_LOG (GST_CAT_DEFAULT, GST_LEVEL_ERROR, NULL, __VA_ARGS__)
| ^~~~~~~~~~~~~~~~~
../validate/gst/validate/gst-validate-scenario.c:3615:5: note: in expansion of macro ‘GST_ERROR’
3615 | GST_ERROR ("Invalid name for scenario '%s'", scenario_name);
| ^~~~~~~~~
../validate/gst/validate/gst-validate-scenario.c:3615:44: note: format string is defined here
3615 | GST_ERROR ("Invalid name for scenario '%s'", scenario_name);
| ^~
../validate/plugins/flow/gstvalidateflow.c:75:3: error: redefinition of typedef 'ValidateFlowOverride' is a C11 feature [-Werror,-Wtypedef-redefinition]
} ValidateFlowOverride;
^
../validate/plugins/flow/gstvalidateflow.h:31:23: note: previous definition is here
G_DECLARE_FINAL_TYPE (ValidateFlowOverride, validate_flow_override,
^
[3470/4053] Compiling C object 'subprojects/gst-devtools/validate/gst/validate/28db7b6@@gstvalidatetracer@sha/gst-validate-reporter.c.o'.
../subprojects/gst-devtools/validate/gst/validate/gst-validate-reporter.c:186:31: warning: format string is not a string literal [-Wformat-nonliteral]
message = g_strdup_vprintf (format, vacopy);
^~~~~~
[3487/4053] Compiling C object 'subprojects/gst-devtools/validate/gst/validate/28db7b6@@gstvalidatetracer@sha/gst-validate-report.c.o'.
../subprojects/gst-devtools/validate/gst/validate/gst-validate-report.c:1007:34: warning: format string is not a string literal [-Wformat-nonliteral]
tmp = gst_info_strdup_vprintf (format, args);
^~~~~~
[76/151] Compiling C object 'subprojects/gst-devtools/validate/plugins/flow/697521d@@gstvalidateflow@sha/gstvalidateflow.c.o'.
../subprojects/gst-devtools/validate/plugins/flow/gstvalidateflow.c:125:65: warning: format string is not a string literal [-Wformat-nonliteral]
if (!flow->error_writing_file && vfprintf (flow->output_file, format, ap) < 0) {
^~~~~~
Pipelines declared in gst-integration-testsuites can rely on the validate HTTP
server, so when an URI pointing to it is detected, advertise the server as
needed before starting the test.
For this to work the test scenario should explicitely declare the pipeline uri,
as shown in this example:
"some_playbin3":
{
"pipeline": "playbin3 uri=%(uri)s video-sink=%(videosink)s",
"config": [
"%(validateflow)s, pad=sink:sink"
],
"scenarios": ["play_15s"],
"uri": "http://127.0.0.1:%(http-server-port)s/defaults/html/foo.html"
}
This means that we can now pass any extra key that `populate_tests`
expects, meaning any key expected by FakeMediaDescriptor and
a few other keys supported by the methods such as
`expected-issues` and `extra_env_vars`
The way this was implemented was simply wrong, first of all checking
the position against the segment after a seek in PAUSED by a query
of the pipeline position is of limited interest, and can only work
in forward playback.
Furthermore the check was a huge blob of code that didn't even look
like it was pretending to do a good job at checking the position in
reverse playback.
This also adds the notion of global variables which will be useable
in config files too.
And add some documentation about default variables in scenarios
Meaning that instead of getting 1 "Detected on" line per monitor,
there will be one per "branch" like:
Detected on <audioconvert1:sink, audioconvert1:src, audioresample1:sink, audioresample1:src, smart-adder-adder:sink_0, smart-adder-adder:src, smart-adder-capsfilter:sink, smart-adder-capsfilter:src, capsfilter2:sink, capsfilter2:src, tee1:sink, tee1:src_0>
Making it simpler to read and a bit less verbose.
The action is generally useful but was implemented in a way that
was restricting its usage for no good reason. Refactor the
implementation adding more argument so it can be used in a wider
context, such as uvch264src.
Something like:
``` bash
echo "video-request-key-unit, direction=upstream, all-header=true, count=1, target-element-factory-name=h264parse, srcpad=src, playback-time=1.0" > tmp.scenario && \
echo "stop,playback-time=2.0" >> tmp.scenario && \
gst-validate-1.0 --set-scenario=tmp.scenario uvch264src \
device=/dev/video0 name=src iframe-period=33 auto-start=true src.vfsrc ! queue ! fakesink \
src.vidsrc ! queue ! video/x-h264,width=1280,height=720,framerate=30/1 ! h264parse ! fakesink
```
works now.
So it can be used directly in the documentation Also add a special "all"
argument to `gst-validate-1.0 --inspect-action-type` so we can generate
the documentation for all action types easily.
_Q_VALIDATE_MONITOR was defined twice because it wasn't declared
as extern in the header, so it would be defined as variable in all
included files. This doesn't seem to cause problems on Linux, but
seems to cause build failures on macOS.
Fixes#42
Otherwise test suites that want to set_default_blacklist and
add tests in setup_tests were seeing their blacklist ignored.
Split up and rename set_blacklists() to complete the refactoring
Basically nothing guarantees that the set of pads we aggregated the flow
for is the same as the one that was aggregated during the actual data
flow as some pads could have been removed meanwhile.
Instead of having the issues centered on the test classes, they
are now focusing on the "bug".
And harmise names on `expected_issue` not `expected_failures`
`--mute` is now True by default and added `--unmute` if the user wants
to have visual/audio feedback. It was really annoying to have thousand
of window pop up by default
Commit 394242c224 ("validate:scenario: Enhance variable
implementation") caused the duration parameter to be stored
as a double instead of GstClockTime, which the _execute_pause
implementation expects. Fix the parameter type and use
gst_validate_action_get_clocktime to handle duration correctly.
https://gitlab.freedesktop.org/gstreamer/gst-devtools/merge_requests/73
There is a mockdecryptor that has been added into validate-sources and
this element is base on GstBaseTransform. This added a deps against
gstbase which was leading to linking errors when building with meson.
Especially the init section and the --quiet.
Remove the whole manual build/source dir include addition
to the g-ir-scanner args seeing that things worked fine
without the args being passed to the scanner at all.
Previously validateflow tests did not fail when the pad was not
attached.
This was a limitation caused by how the Validate API worked. Before, the
`notify::validate-runner` signal was not emitted until a monitor was
attached to the override. This made impossible to listen for the
runner's `stopping` signal.
This patch fixes the problem by setting `validate-runner` for all
existing overrides when the runner is initialized and adding checks in
validateflow to error in the case no pad was attached.
While in many cases it's desirable to wait for a buffer to be pushed
downstream when using appsrc-push, in some cases this is not possible as
such pushing action is dependent on following actions that would not be
executed if we wait.
An example for this is prerolling:
appsrc ! qtdemux ! video/x-h264 ! decodebin name=dec ! %(videosink)s
description, seek=false, handles-states=true
appsrc-push, target-element-name=appsrc0, file-name="raw_h264.0.mp4"
set-state, state=playing
appsrc-eos, target-element-name=appsrc0
In order for the preroll to occur, both the appsrc needs to push the
buffer and the state needs to reach PLAYING. But `set-state` cannot
finish if the buffer has not been pushed (the state transition does not
finish) and conversely pushing the buffer will not finish until the
state has reached.
Making appsrc-push not wait for the buffer solves this problem. This
patch makes appsrc-push aware of this issue by only waiting for the
buffer to be pushed if the pipeline is in a state that allows buffers to
flow.
Since gst_validate_action_set_done() is asynchronous, the bus EOS
handler may already be running before the action is actually finished.
This patch ensures that is not a problem.
This change allow tests to check performance of elements by checking the
frequency at which buffers are pushed on src pads.
I re-used most of the logic from fpsdisplaysink to compute the
frequency.
We can now uses something like:
GST_VALIDATE_CONFIG='core,min-buffer-frequency=60,target-element-factory-name=v4l2src'
The 'buffer-frequency-start' optional field can be used to ignore the
frequency during the start of the pipeline. This is useful when testing live
pipelines where configuring and setting up elements can take some time slowing
down the first buffers.
- Stop arbitrarily consider params as ClockTime based on their names
but add a convetion that the `.type` field of the ActionType should
end by `(GstClockTime)` when it is a clock time.
And require them to follow the `$varname` (can't be $(varname) as
parenthesis have another meaning in those expressions).
Still accept "duration" and "position" as varname for backward compat
but update our scenarios anyway.
This way it is clear that you are using a variable reading the scenario
and we can verify that what the scenario writer intents is to use an
already set variable.
There was a race in appsrc-push when the pushed buffer caused an EOS.
The EOS event could be handled by the main thread, finishing the test
while the action, executing in the streaming thread, has not finished
yet.
A mutex is now introduced to add mutual exclusion for the two threads so
that an EOS does not cause the termination of the test while the action
is still going.
validateflow can be used to check the buffers and events flowing through
a custom pipeline match an expectation file. This can be used to test
non-regular-playback use cases like demuxers handling adaptive streaming
fragment pushing.
This patch includes also new actions used for these cases:
`appsrc-push`, `appsrc-eos` and `flush` (plus `checkpoint`, which is
only available with validateflow).
- Move the parser code into a `LauncherConfig.create_parser()` method
- Remove the need to pass libsdir to the _TestsLauncher object
- Extract out a `setup_launcher_from_args` function
When defining pipelines_descriptions to run test on in a `.json` file, you might
need to point to paths in the testsuite directory (for media files URIs
for example), you can now do
`"pipeline": "filesrc location="$(config_path)s/../medias/some/file.mkv...`
Work around broken disthook check in release.mak so we don't
have to update the common submodules for that (applies only
to this module because the version number is in the top-level
meson.build but the package/dist directory is a subdir). This
only became a problem now because the common submodule hadn't
been updated for the last few years.
We use visual studio module definitions for the list of symbols to
export when targetting Windows. Fixes CI failure:
../validate/tools/gst-validate.c:460: undefined reference to `gst_validate_spin_on_fault_signals'
I think it was a mistake to call them <error> as the two notions are
different (we marked failed test as "failures" in the <testuite> node).
Should make gitlab happy with our file!
This is generally usefull so we do not have to pass -M every time we launch the launcher
And it adds support for nesting launcher calls always respecting the provided main directory
+ Fix some new pep8 errors
This is particularly useful for scenario that define constants
that are used to check video frame checksum for example, we can
now have one single 'scenario' file that defines consts for the
checksum of the frames, and those can be reused everywhere.
gst_validate_override_register_by_name() was not working when using a
pad name because by the time gst_validate_pad_monitor_do_setup()
was called to set the name of the monitor it was too late for overrides
to have any effect.
Patch written by Thibault.
