Commit graph

723 commits

Author SHA1 Message Date
Thiago Santos
71fd0e57cd qa-monitor: remove reference to the runner
qa-monitor implements qa-reporter, and we already have a runner stored
there.
2013-08-07 16:23:25 -03:00
Thiago Santos
6d4a8fd754 gitignore: ignore more binaries 2013-08-07 16:23:25 -03:00
Thiago Santos
a6b3816c35 qa-file-check: add new binary to run file checks easily
It creates a GstQaFileChecker and runs it on the passed URI with
the tests enabled as arguments
2013-08-07 16:23:25 -03:00
Thiago Santos
08aae8336b qa-runner: simplify runner to not hold refs to monitor/pipeline
The GstQaRunner is now a simple aggregator of reports that it receives
from monitors and filechecker. This allows it to be used in both
scenarios without  APIs that expect GstElement or Monitors, that are
only used on the pipeline monitoring QA tests.
2013-08-07 16:10:57 -03:00
Thiago Santos
61d39d6e74 file-checker: add file playback testing feature
Adds a property that triggers the file playback tests on
GstQaFileCheker. Also enable it in the gst-transcoding post file checks.

The implementation is simple, just create a playbin2 and use fakesinks
as sinks, set it to playing and wait for either EOS or ERROR messages.
2013-08-07 11:31:04 -03:00
Thiago Santos
b959a781a9 qa-reporter: fix typo 2013-08-06 19:42:21 -03:00
Thiago Santos
499f65a041 file-checker: include restriction caps tests when checking for profiles
Also move the caps check earlier on the path, to error out sooner and
avoid iterating the sub streams without needing
2013-08-06 19:42:09 -03:00
Thiago Santos
71351b3e33 file-checker: replace encoding profile comparison
Use our own custom comparison to allow to add more fine grained error
reporting. Also the encoding profile is_equal function is too strict as
it also compares profiles names, that doesn't matter to us.

This commit implementation is still initial and needs improvements as it
isn't using the restriction caps, which includes information that might not be
on the profile format caps.
2013-08-06 18:32:07 -03:00
Thiago Santos
e7853ec69f qa-reporter: fix crash by avoiding unref an integer 2013-08-06 10:37:15 -03:00
Thiago Santos
c81e6198f9 makefile: fix build of gst-qa- tools 2013-08-06 10:36:47 -03:00
Thiago Santos
cde09a7e6c file-checker: add error report and new report types
Add a list of new report types and use them in the file-checker.
The errors are mostly related to testing file attributes against
expected values
2013-08-06 10:36:02 -03:00
Thiago Santos
369e936b8f qa-file-checker: add a file checker object/runner
It is an object that is capable to run a few file checks. The
implemented tests are: file size, duration, if the file is seekable and
comparing the file stream types with a encoding profile
2013-08-05 14:16:06 -03:00
Thiago Santos
e5cafa3375 qa-scenario: adding eos scenario action
Allows sending EOS to the pipeline
2013-08-01 18:08:44 -03:00
Thiago Santos
caba58c029 qa-scenario: add new scenario action - Pause
The pause action instructs the pipeline to go to paused state and then
return to playing. It has the argument 'duration', that indicates the
duration for which the pipeline will remain in paused
2013-08-01 09:35:59 -03:00
Thiago Santos
60b0c7383f qa-scenario: refactor to accomodate more actions
Refactor to be able to reuse to add more actions to scenarios.
Planned are pauses and encoding changes
2013-08-01 01:27:20 -03:00
Thiago Santos
2901e35f5c qa-scenario: avoid assertion on dispose
After an error, the pipeline might still be null, check before unreffing
2013-07-31 15:01:13 -03:00
Thiago Santos
f1368356bc qa-scenario: fix typo on define variable 2013-07-31 15:00:56 -03:00
Thiago Santos
a395133aa2 qa-report: fix typo on assertion comparison 2013-07-31 15:00:33 -03:00
Vincent Penquerc'h
3a419c3fd8 gst-qa-element-monitor: do not bypass monitor factory
A pad monitor was created directly. Prefer going through the
factory.
2013-07-31 08:10:42 -03:00
Vincent Penquerc'h
5356f10644 gst-qa-reporter: fix use of uninitialized repeat field 2013-07-31 08:10:40 -03:00
Vincent Penquerc'h
711fa22a49 gst-qa-reporter: fix report leak when discarding repeated report 2013-07-31 08:10:37 -03:00
Vincent Penquerc'h
dbcbcfa868 gst-qa: fix build in po
Using a lot of grep and some cargo culting.
2013-07-31 08:10:28 -03:00
Vincent Penquerc'h
16bbbb5733 gst-qa: make tools depend on libraries
This fixes parallel build randomly breaking.
2013-07-31 08:10:27 -03:00
Thiago Santos
76679b1c7e qa-report: expose API for adding custom issues
expose gst_qa_issue_register and gst_qa_issue_new to allow applications
to register their own custom issues.

Issues IDs should use Areas higher than GST_QA_AREA_OTHER for custom
areas. And to add more issues to existing areas, the IDs should be
higher than GST_QA_ISSUE_ID_CUSTOM_FIRST.

Custom issues registering should be done at startup and from the same
thread as there is no locking around the issues hashtable
2013-07-30 17:07:13 -03:00
Thiago Santos
2034fb11d9 Fix typos 2013-07-30 16:21:15 -03:00
Thiago Santos
4e9bedcde8 qa-runner: Remove printing API from qa-runner
Replace it with functions to list the reports
2013-07-30 16:20:49 -03:00
Vincent Penquerc'h
4456e24cdf pad-monitor: fix NULL format string
An empty message should be an empty string.
2013-07-30 13:55:14 -03:00
Thiago Santos
2eba57d387 qa-runner: removing _setup call
Do setup on the _new function directly instead of having a separate
call for that
2013-07-30 10:21:32 -03:00
Thiago Santos
044084949a qa-override: add more pad overrides for buffer probe and caps
Add override functions for custom checking of buffer probe and
getcaps/setcaps functions.
2013-07-30 09:57:20 -03:00
Thiago Santos
55109ae7f9 pad-monitor: call the event/query/buffer overrides
Use the new event/buffer/query overrides to allow custom checks
on those scenarios
2013-07-30 09:41:06 -03:00
Thiago Santos
8a66b3d4ed qa-override: add callbacks for query/buffer/event functions
Add callbacks for pad event/buffer/query functions in case the
override wants to do additional checks
2013-07-30 09:41:05 -03:00
Vincent Penquerc'h
7fedf5a111 gst-qa-override-registry: load overrides dynamically
Shared objects listed in GST_QA_OVERRIDE are loaded on startup,
and the symbol gst_qa_create_overrides is run. It should create
any override needed. While it can do anything it wants, this
is discouraged.

GST_QA_OVERRIDE should be a comma separated list of shared objects,
any relative paths should be from the current working directory
at the time they are loaded (ie, if the process to be traced
changes cwd, use absolute paths).

No attempt whatsoever is made at not running what was not meant.

Includes a sample shared object for illustration purposes.
2013-07-30 09:41:00 -03:00
Thiago Santos
09fda8bbd2 qa-override-registry: register overrides by gtype and klass
Overrides can now be registerd by gtype, meaning that they will
be attached to monitors that the target is of the requested type.

Also by element klass, that will check that the element has the
selected class in its details
2013-07-29 13:17:50 -03:00
Thiago Santos
e0d041d67b qa-monitor: implement intercept_report
It is used to iterate over overrides and modify the report level if
the overrides wants to do so.

Also adds a new mutex only for the overrides to avoid deadlocks when
reporting if we used the same lock for iterating the overrides
2013-07-29 12:01:02 -03:00
Thiago Santos
f7d7d093e3 qa-reporter: fix copy n paste left over 2013-07-29 11:35:20 -03:00
Thiago Santos
79e7392a9c qa-reporter: add function for intercepting reports
after report creation, this function is called and implementers can
modify the report to their liking before it is posted to the runner
2013-07-29 11:34:42 -03:00
Vincent Penquerc'h
6e19324584 qa-monitor: chain gst_qa_monitor_finalize to parent's finalize
It was chaining to the parent's dispose.
2013-07-29 10:46:29 -03:00
Thiago Santos
3a13d59a1d gst-qa-report: put the correct format to avoid compiler warnings 2013-07-29 10:06:48 -03:00
Vincent Penquerc'h
2bac08370c i18n: copy necessary files in-tree 2013-07-29 09:57:25 -03:00
Vincent Penquerc'h
1c053ffda7 GstIssueId: make this uintptr_t
As it's used a a placeholder pointer for g_hash_table use,
it needs to be converted back and forth to a pointer.
2013-07-29 09:56:45 -03:00
Thiago Santos
dbc61a88b6 gst-qa-override-registry: adding the override-registry
This registry should contain the list of GstQaOverride to
be used on the pipelines being monitored
2013-07-29 09:53:54 -03:00
Thiago Santos
794ea2652d qa-override: adds qa-override that can change the report level of issues
Useful for customizing the level of issues for particular elements/tests
when they are more relevant or have to be disabled
2013-07-26 12:16:29 -03:00
Thiago Santos
773d8adc5e qa-report: splitting a GstQaReport into a GstQaIssue and GstQaReport
Reports now point to Issues, that are uniquely identified and have
translatable descriptions. This way we are going to be able to uniquely
identify the issues and applications can enable/disable checks for
specific elements.
2013-07-26 12:16:29 -03:00
Thibault Saunier
1e1e0b922e qa: Make it possible to set a scenario from the command line in test apps 2013-07-25 16:58:15 -03:00
Thibault Saunier
382fcc9a01 qa: Properly set reporter's runner reference
That was never set, but it is needed for the reporter to properly add
reports to the runner.
We still keep a reference on the monitor to make things simpler
2013-07-25 16:58:14 -03:00
Thibault Saunier
d804ad475a reporter: Use Gst debugging log in the _report method directly
Using __VALIST__ was not properly working
+ Add a gstqareporter debug category
2013-07-25 16:58:14 -03:00
Thibault Saunier
4fb8f47770 scenario: Implement the GstQaReporter interface and make use of it
This way we can report issues from a scenario
Also add a Seek aread to the known areas list

We now need to pass the runner to the scenario instead of the
pipeline as the GstQaReporter interface needs it.
2013-07-25 16:58:13 -03:00
Thibault Saunier
4992249848 qa: Add a GstQaReporter interface that objects needing reporting can implement
Various type of object should be able to do some reporting, so we have
to make sure all the code to do that is in one place. Creating an interface
makes it simple to share information and it avoid to have a baseclass for
something that is not actually important enough to create a baseclass.

Conflicts:
	gst/qa/gst-qa-pad-monitor.c
2013-07-25 16:57:44 -03:00
Thibault Saunier
7508e35b7e qa: Add a GstQaScenario class making it possible to execute scenarios
A scenario correspond to a suite of action to execute on a pipeline,
for the time being, we only support seeking the pipeline, but in the
future we can imagine doing some queries, setting pipeline state, etc...

The scenario can be loaded thanks to the GST_QA_SCENARIO environment
variable, making it usable with any existant application, in case, the
application can be used interactively, the user should either, not load
any scenario or let the application run without interacting with it.
2013-07-25 16:56:40 -03:00
Thiago Santos
4a3f06885a pad-monitor: add lots of locking
When handling elements that spawn multiple threads (hardware
enc/decoders), the pad monitor has to protect its variables specially
because some checks involve iterating over internally linked pads to
add/get some data for comparison (expected events, timestamp ranges,
caps).

Aside from locking its own mutex, the pad monitor can also lock the
parent's mutex when it needs to use data from its internally linked
pads. The locking order should always be parent and then individual
pad-monitor mutexes. This should prevent deadlocks when multiple
pad-monitors from the same element start doing checks at the same time
from different threads.
2013-07-24 16:16:07 -03:00