gstreamer/docs/gst-validate-launcher.md

---
short-description: Integration testsuite builder and launcher
...

# gst-validate-launcher

`gst-validate-launcher` is an application to create full testsuites on
top of the GstValidate tools, testing behaviour with dynamic pipelines
and user actions (seeking, changing the pipeline state, etc.) as
described by the [scenario](GstValidateScenario) format.

## Run the GstValidate default testsuite

GstValidate comes with a default testsuite to be executed on a default
set of media samples. Those media samples are stored with `git-annex` so
you will need it to be able to launch the default testsuite.

The first time you launch the testsuite, you will need to make sure that
the media samples are downloaded. To do so and launch the testsuite you
can simply do:

    gst-validate-launcher validate --sync

This will only launch the GstValidate tests and not other applications
that might be supported (currently `ges-launch` is also supported and
has its own default testsuite).

Launching the default testsuite will open/close many windows, you might
want to mute it so you can keep using your computer:

    gst-validate-launcher validate --sync --mute

## Example of a testsuite implementation

To implement a testsuite, you will have to write some simple python code
that defines the tests to be launched by `gst-validate-launcher`.

In this example, we will assume that you want to write a whole new
testsuite based on your own media samples and [scenarios](GstValidateScenario). The
set of media files and the testsuite implementation file will be
structured as follow:

    testsuite_folder/
      |-> testsuite.py
      |-> sample_files/
          |-> file.mp4
          |-> file1.mkv
          |-> file2.ogv
      |-> scenarios
          |-> scenario.scenario
          |-> scenario1.scenario

You should generate the `.media_info` files. To generate them for local
files, you can use:

    gst-validate-launcher --medias-paths /path/to/sample_files/ --generate-media-info

For remote streams, you should use
`gst-validate-media-check-GST_API_VERSION`. For an http stream you can
for example do:

    gst-validate-media-check-GST_API_VERSION http://someonlinestream.com/thestream \
                  --output-file /path/to/testsuite_folder/sample_files/thestream.stream_info


The `gst-validate-launcher` will use the generated `.media_info` and
`.stream_info` files to validate the tests as those contain the
necessary information.

Then you will need to write the `testsuite.py` file. You can for example
implement the following testsuite:

``` python
"""
The GstValidate custom testsuite
"""

import os
from launcher.baseclasses import MediaFormatCombination
from launcher.apps.gstvalidate import *
TEST_MANAGER = "validate"

KNOWN_ISSUES = {}

def setup_tests(test_manager, options):
    print("Setting up the custom testsuite")
    assets_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), ".", "samples_files"))
    options.add_paths(assets_dir)

    # This step will register default data for the test manager:
    # - scenarios such as `play_15s`, `reverse_playback` etc.
    # - encoding formats such as "mp4,h264,mp3" etc.
    # - blacklist such as dash.media_check.*
    # - test generators:
    #   - GstValidatePlaybinTestsGenerator
    #   - GstValidateMediaCheckTestsGenerator
    #   - GstValidateTranscodingTestsGenerator
    # This 'defaults' can be found in 'gst-devtools/validate/launcher/apps/gstvalidate.py#register_defaults'
    # test_manager.register_defaults()

    # Add scenarios
    scenarios = []
    scenarios.append("play_5s")
    scenarios.append("seek_backward")
    test_manager.set_scenarios(scenarios)

    # Add encoding formats used by the transcoding generator
    test_manager.add_encoding_formats([
            MediaFormatCombination("mp4", "mp3", "h264"),])

    # Add generators
    # GstValidatePlaybinTestsGenerator needs at least one media file
    test_manager.add_generators([GstValidateMediaCheckTestsGenerator(test_manager)])
    # GstValidatePlaybinTestsGenerator needs at least one scenario
    test_manager.add_generators([GstValidatePlaybinTestsGenerator(test_manager)])
    # GstValidateTranscodingTestsGenerator needs at least one MediaFormatCombination
    test_manager.add_generators([GstValidateTranscodingTestsGenerator(test_manager)])

    # list of combo to blacklist tests. Here it blacklists all tests with playback.seek_backward
    test_manager.set_default_blacklist([
            ("custom_testsuite.file.playback.seek_backward.*",
             "Not supported by this testsuite."),])

    # you can even pass known issues to bypass an existing error in your custom testsuite
    test_manager.add_expected_issues(KNOWN_ISSUES)
    return True
```

Once this is done, you've got a testsuite that will:

-   Run playbin pipelines on `file.mp4`, `file1.mkv` and `file2.ogv`>
    executing `play_5s` and `seek_backward` scenarios

-   Transcode `file.mp4,` `file1.mkv` and `file2.ogv` to h264 and
    mp3 in a MP4 container

The only thing to do to run the testsuite is:


    gst-validate-launcher --testsuites-dir=/path/to/testsuite_folder/ testsuite


# Invocation

You can find detailed information about the launcher by launching it:

    gst-validate-launcher --help

You can list all the tests with:

    gst-validate-launcher --testsuites-dir=/path/to/testsuite_folder/ testsuite -L