Currently implemented tests are:
* Settup and cleanup on monitor is done properly
* Some tests in the PadMonitor are done properly, namely:
- Buffer before segment
- Buffer outside segment
- First buffer running time is always 0
- The Demuxer flow aggregation is properly checked
https://bugzilla.gnome.org/show_bug.cgi?id=736379
Instead just include required (public and local) header
gst-validate-scenario.h:43:44: error: redefinition of typedef 'GstValidateActionParameter' is a C11 feature [-Werror,-Wtypedef-redefinition]
WHen seeking in paused the position right after should be pretty much
the exact one, but sometimes it can be a little different because of
rounding issues and similare.
When checking that all action were executed, we need to make sure that
actions such as EOS or stop are not taken into account as we might have
shorter medias than the duration of the scenario, and that should not be
fatal.
+ Plug a leak on the way
emit-signal action allows to emit signals to elements in scenarios.
The implementation only accepts signals without arguments for now but
it can be extended to use parameters if needed in the future
Get the GValue directly from the structure and do not assume everything
is stored as a string and use the GstStructure's GValue to set the property
to the instances
Allows setting element's properties during a scenario. Very useful
for testing that elements behave correctly when changing properties
during playing state
https://bugzilla.gnome.org/show_bug.cgi?id=733070
Position/duration query may fail, or yield unknown values (eg,
unknown duration for live streams). In these cases, we must ensure
we do not use those invalid values.
https://bugzilla.gnome.org/show_bug.cgi?id=715160
If the initial buffer is before segment.start, we don't want to raise
the "first buffer doesn't have 0 running-time" issue.
Also add debug for tracking issues
gst-validate-utils.c:413:7: error: variable 'v0' is used uninitialized whenever 'if' condition is true [-Werror,-Wsometimes-uninitialized]
if (c == '!') {
^~~~~~~~
gst-validate-utils.c:424:10: note: uninitialized use occurs here
return v0;
^~
gst-validate-utils.c:413:3: note: remove the 'if' if its condition is always false
if (c == '!') {
^~~~~~~~~~~~~~~
gst-validate-utils.c:411:13: note: initialize the variable 'v0' to silence this warning
gdouble v0;
^
= 0.0
1
This property enables the user to have actions executed independently
of the state of the pipeline.
Conflicts:
validate/gst/validate/gst-validate-scenario.c
While buffering we should no try to execute anything as we would not be
controlling properly the execution.
+ Activate scrub forward seeking for HTTP streams
This reverts commit 925ff7542b.
Actually jenkins never truncates on failure stacktrace... we do not
want to set gst-validate as failure stacktrace in our results. That
commit was not usefull.
During that time we will just not execute any new action
+ Lower WARNING to DEBUG when no playbcak_time is provided for an
action, it should just be 0.
For encoded data we might need buffers that have timestamp <
segment.start to make sure that we have the keyframe, etc... but for raw
data, buffer end should strictly be inside the segment, be more strict
about that.
Query caps will actually get the caps from downstream and those caps
might be different in case there is a Filter in between. What we want is
to check that the caps set on the internally linked pads are correct.
Print details about the descriptions when listing scenario in a KeyFile
format
The description can contain any information about the scenario such as its duration before
EOS, how long the pipeline needs to be so the scenario can be applied...etc
GST_VALIDATE_SCENARIOS_PATH was used only for loading scenarios,
so any in that path would not be listed by -l.
Change-Id: If3cb94867ef3876933bda02477675c8ccf67baaf
In case we have serialized events right after a buffer that had no
timestamp set we concider that last timestamp was 0, but we can
actually not concider the timestamp at all in that case as it is
only "meaningless value".
This make it easier for user to understand the difference between
the two concepts and avoids confusion.
Change-Id: Ib42913722c93a1e7e3c8b156173c458230946592
Conflicts:
validate/tools/gst-validate-transcoding.c
validate/tools/gst-validate.c
This action can be used to change the rank of a particular element,
so you can force a particular element to be used when using
autoplugging elements (such as decodebin, encodebin, and friends)
This type of actions is used to change some parametter on GStreamer
core and it plugins, it can be fore example, to change the rank of a
plugin or things like that.
It is a random number, but it will in most cases give people a hint
that gst-validate reported a critical issue, and thus set the return
code, only by looking at it
Also make use of gst_validate_runner_print() in
gst-validate-transcoding.c as we were copy pasting that method there.
Always keep probing the pipeline for the current position and compare
with the latest requested seek segment to detect if the seek boundaries
are being respected
The "switch-track" command can be used to switch tracks. The "type"
argument selects which track type to change (can be "audio", "video",
or "text"). The "index" argument selects which track of this type
to use: it can be either a number, which will be the Nth track of
the given type, or a number with a "+" or "-" prefix, which means
a relative change (eg, "+1" means "next track", "-1" means "previous
track").
Conflicts:
validate/gst/validate/gst-validate-scenario.c
When used with LD_PRELOAD, the application might use various pipelines
for several different thing, we need to make it possible to spcify a
specific pipeline (or set of pipelines) on which to run the scenario.
The format is in the form of:
scenario_name:pipelinename_pattern*
Improve buffer timestamp range check:
* Only do it for encoders or decoders
* Audio has an acceptable tolerance of 100ms
To do this, keep track of the caps on the pad and store
if it is dealing with audio or video
Allows both src and sink pad to keep track of the current caps, but
the duplicated caps check is still only applied to sink pads as
src pads can push the same caps multiple times when it isn't linked
Makes the result a bit more readable than a compact multi-line list.
FIXME: Figure out how to print the description of the issues (which can
spawn multiple lines) in a nice way.
Tag events are hard to track and check if properly serialized because
they mutate too much inside elements. There is no reliable way currently
to match a tag event pushed into an element and another tag event
leaving the element (other than if the pointers are actually the same).
This test will find the first input selector with more than one
sink pad, and cycle through them till it gets back to the original
one. Five seconds between switches. The test checks that some data
was sent from the input selector when each of the sink pads was
selected.
We know are sequential so whenever the wanted position is passed we
should execute the action.
This avoid issue with the tolerance when we have high rate playback
Testing on source pads can lead to false positives when pads are
unlinked. The caps event is sticky and will be pushed again later
when another buffer/event is pushed, leading to an acceptable
situation to push the caps twice.
When seeking we might want to execute seeks at a playback time inferior than previous
seek, so we need to be able to define the order in which actions have to be
executed, the simplest way is to just concider that actions are always
order in the XML files.
+ Add some more debugs
Conflicts:
gst/validate/gst-validate-scenario.c
Update to have the same features as gst-validate.
1) Handle interrupts properly, with the additional of having the
'eos-on-shutdown' argument that sends EOS to the pipeline. This is
very useful for transcoding processes to finish correctly.
2) Print issues on the end of application
Adds a new expected-results argument to receive a file that is used
as a base for comparison with the new results. In case differences are
found, the application will print those issues.
They weren't waiting for the pipeline to properly change state
before sending seek events, that would cause some events to
return TRUE even if they were not handled
The tests are very simple as they only write the first error they
found during playback. If no error is set, an empty string is
printed.
The playback pipeline isn't monitored with validate monitors for now
This struct stores information about a media and tests run on it. It
also has a few helper functions that allows storing the results to a
file and loading it back.
Instead of having the file-checker object that would compare the
extracted values from the file to expected results set to its properties,
the media-info will store the values and it will be possible to compare
old media-info with new media-info from the same file. This allows
tracking improvements and regressions on different gstreamer versions.
Right now, the media-info is very tiny and doesn't store much info, only
the uri and the file size in bytes, but it will receive more additions in
the upcoming commits for storing duration, media topology, seekability and
playback information.
The GstQaRunner is now a simple aggregator of reports that it receives
from monitors and filechecker. This allows it to be used in both
scenarios without APIs that expect GstElement or Monitors, that are
only used on the pipeline monitoring QA tests.
Adds a property that triggers the file playback tests on
GstQaFileCheker. Also enable it in the gst-transcoding post file checks.
The implementation is simple, just create a playbin2 and use fakesinks
as sinks, set it to playing and wait for either EOS or ERROR messages.
Use our own custom comparison to allow to add more fine grained error
reporting. Also the encoding profile is_equal function is too strict as
it also compares profiles names, that doesn't matter to us.
This commit implementation is still initial and needs improvements as it
isn't using the restriction caps, which includes information that might not be
on the profile format caps.
It is an object that is capable to run a few file checks. The
implemented tests are: file size, duration, if the file is seekable and
comparing the file stream types with a encoding profile
The pause action instructs the pipeline to go to paused state and then
return to playing. It has the argument 'duration', that indicates the
duration for which the pipeline will remain in paused
expose gst_qa_issue_register and gst_qa_issue_new to allow applications
to register their own custom issues.
Issues IDs should use Areas higher than GST_QA_AREA_OTHER for custom
areas. And to add more issues to existing areas, the IDs should be
higher than GST_QA_ISSUE_ID_CUSTOM_FIRST.
Custom issues registering should be done at startup and from the same
thread as there is no locking around the issues hashtable
Shared objects listed in GST_QA_OVERRIDE are loaded on startup,
and the symbol gst_qa_create_overrides is run. It should create
any override needed. While it can do anything it wants, this
is discouraged.
GST_QA_OVERRIDE should be a comma separated list of shared objects,
any relative paths should be from the current working directory
at the time they are loaded (ie, if the process to be traced
changes cwd, use absolute paths).
No attempt whatsoever is made at not running what was not meant.
Includes a sample shared object for illustration purposes.
Overrides can now be registerd by gtype, meaning that they will
be attached to monitors that the target is of the requested type.
Also by element klass, that will check that the element has the
selected class in its details
It is used to iterate over overrides and modify the report level if
the overrides wants to do so.
Also adds a new mutex only for the overrides to avoid deadlocks when
reporting if we used the same lock for iterating the overrides
Reports now point to Issues, that are uniquely identified and have
translatable descriptions. This way we are going to be able to uniquely
identify the issues and applications can enable/disable checks for
specific elements.
That was never set, but it is needed for the reporter to properly add
reports to the runner.
We still keep a reference on the monitor to make things simpler
This way we can report issues from a scenario
Also add a Seek aread to the known areas list
We now need to pass the runner to the scenario instead of the
pipeline as the GstQaReporter interface needs it.
Various type of object should be able to do some reporting, so we have
to make sure all the code to do that is in one place. Creating an interface
makes it simple to share information and it avoid to have a baseclass for
something that is not actually important enough to create a baseclass.
Conflicts:
gst/qa/gst-qa-pad-monitor.c
A scenario correspond to a suite of action to execute on a pipeline,
for the time being, we only support seeking the pipeline, but in the
future we can imagine doing some queries, setting pipeline state, etc...
The scenario can be loaded thanks to the GST_QA_SCENARIO environment
variable, making it usable with any existant application, in case, the
application can be used interactively, the user should either, not load
any scenario or let the application run without interacting with it.
When handling elements that spawn multiple threads (hardware
enc/decoders), the pad monitor has to protect its variables specially
because some checks involve iterating over internally linked pads to
add/get some data for comparison (expected events, timestamp ranges,
caps).
Aside from locking its own mutex, the pad monitor can also lock the
parent's mutex when it needs to use data from its internally linked
pads. The locking order should always be parent and then individual
pad-monitor mutexes. This should prevent deadlocks when multiple
pad-monitors from the same element start doing checks at the same time
from different threads.
Only check if fields are proxied for sink getcaps as it is when
downstream restrictions should be proxied. Also improve the
fields comparison to handle single value x multi value
(list/array/range) contain relations.
Replace the macro with a more powerful variadic function that can
check for more acceptable types for the same caps.
This removes a few more false positives
flush events shouldn't fail, so we don't need to rollback when it
returns false from downstream (this is common when downstream is still
not-linked) and it would cause gst-qa to spit false positives.
Also refactor the common event handling for both sink and src event
functions into a common place. Currently we handle flushes the same
for both pad's directions
