gstreamer/docs/design/effects.txt

Effects
-------

Summary
-------
1. Basic ideas
2. Problems
3. Propositions  to solve those problems
  A. The registry
  B. Effects configurability
  C. Keyframes
4. Use-cases
5. API draft

The goal of this proposal is to find a simple way to handle effects through an
API which would allow developers to handle any use-case

1. Basic ideas
----------------

  * Effects are actually descendant of GESTrackOperation

  * You can add effects on any clip or layer

  * You can add effects between several clips and control them as a unique effect.

  * Effects are configurable and those could change during time

  * We should be able to handle the gnome-video-effects standard. We should also offer
    a system to be able to handle any other effect system.

  * We can implement complexe effects. It means effects that are more
    than adding GstElement-s to the timeline. It can also mean effects
    that are both: video and audio at the same time

2. Problems
----------
  * We should be able to handle list of available effects on the system at
    runtime.

  * We should be able to configure effects through an API in GES and
    not directly configuring the corresponding GstElement properties.

  * We should also expose the GstElement-s contained in an effect so
    it is possible for people to control their properties as they wish.

  * We should be able to implement and handle complexe effects directly in GES

  * We should be able to configure effects in the time -> Keyframes without
    duplicating code from GStreamer


3. Propositions to solve those problems
---------------------------------------

A. The registry => Still to design

  We could implement a GESRegistry which would actually
  retrieve elements (effects) from the GSTRegistry and any other mean
  such as gnome-video-effects to let us get all the effects that are present
  on the system....
  This way the developers could have the list of all the effects
  that are installed on the system pretty easily.

B. Effects configurability

  The idea to be able to configure effects through a simple API in GES would
  be to add an API in GESTrackObject to access the gst-object properties that
  user would like to configure.
  We would also have a method to set those properties easily

  We should also find a way to handle that in thecase of systems such as gnome-effects

C. Keyframes
  We may want to handle this use-case directly in GES and for any kind of
  time related configuration? FIXME
    => Special specifications for that?

4. Use-cases
-----------

  UC-1. The user wants to add an effect to an entire clip => GESTimelineObject new API

  UC-2. The developer wants to allow his users to configure effects => New 
  GESTrackOperation API

  UC-3. The user wants to add an effect on a special portion of a clip, we should
  allow him to configure that thanks to a system of keyframes. FIXME

  UC-4. We want to implement an effect which isn't only composed by a bin, but is more
  complexe than that (ex: "effect '24'") => we have the GESTrackOperation
  which is the base class (abstract) for this kind of implementation. This class should 
  implement vmethods to get/set configurable properties.

  UC-5. A developer wants to implement effect which handle music and video at the same
  time, Would the solution be to implement a GESTimelineEffect to handle this
  special usecase? FIXME

  UC-6. The developers wants to configure each elements of an effect the way he wants
  with a full control over it.

  UC-7. Developers want to expose all effects present on the system to the end-user

5. API draft
------------


  A. GESTrackObject new API

      signals:
      -------
        * property-changed: emited when a property of the GESTrackObject gst element
          is changed

      /**
      * ges_track_object_list_gst_properties:
      *
      * @object: The origin #GESTrackObject
      *
      * Get all the usefull configurable properties of the GstElement contained in @object.
      *
      * Returns: an array of GParamSpec of the configurable properties of the
      * GstElement-s contained in @object
      */
      GParamSpec **
      ges_track_object_list_gst_properties (GESTrackObject *object);

      -> Usecases: Let user know all the property he can configure.

      /**
      * ges_track_object_set_gst_property:
      *
      * @object: The origin #GESTrackObject
      * @property_name: The name of the property
      * @value: the value
      *
      * Sets a property of a GstElement contained in @object.
      *
      */
      void ges_track_object_set_gst_property (GESTrackObject *object,
                                              const gchar *property_name,
                                              GValue * value);
        -> Usecases:
          + Let user configure effects easily (UC-3)

      /**
      * ges_track_object_get_gst_property:
      *
      * @object: The origin #GESTrackObject
      * @property_name: The name of the property
      * @value: return location for the property value
      *
      * Gets a property of a GstElement contained in @object.
      */
      void ges_track_object_get_gst_property (GESTrackObject *object,
                                              const gchar *property_name,
                                              GValue * value);

  B. GESTimelineObject new API

      signals:
      -------
        * effect-added: emited when an effect is added
        * effect-removed: emited when an effect is removed
        * effect-moved: emited when an effect is moved

      /**
      * ges_timeline_object_add_effect:
      *
      * @object: The origin #GESTimelineObject
      * @effect_material: The #GESMaterial from which to create the effect
      * @position: The top position you want to give to the effect,
      * -1 if you want it to be added at the end of effects.
      *
      * Adds a new effect corresponding to @material to the #GESTimelineObject
      *
      * Returns: The newly created #GESTrackEffect, or %NULL if there was an error.
      */
      GESTrackEffect *ges_timeline_object_add_effect (GESTimelineObject *object,
                                                      GESMaterial *effect_material,
                                                      gint position);

      /**
      * ges_timeline_object_get_effects:
      *
      * @object: The origin #GESTimelineObject
      *
      * Returns: a #GList of the #GESTrackEffect that are applied on
      * @object order by ascendant priorities.
      * The refcount of the objects will be increased. The user will have to
      * unref each #GESTrackOperation and free the #GList.
      */
      GList *
      ges_timeline_object_get_effects (GESTimelineObject *object);
        -> Usecases:
          + First step to allow the configuration of effects (UC-3)

      /**
      * ges_timeline_object_set_top_effect_position:
      *
      * @object: The origin #GESTimelineObject
      * @effect: The #GESTrackEffect to move
      * @newposition: the new position at which to move the @effect
      *
      * Returns: %TRUE if @effect was successfuly moved, %FALSE otherwize.
      */
      gboolean
      ges_timeline_object_set_top_effect_position (GESTimelineObject *object,
            GESTrackEffect *effect, guint newposition);

  C - The GESTrackEffect API:

      /**
      * ges_track_effect_new_from_material:
      *
      * @material: The #GESMaterial from which to create this GESTrackEffect
      *
      * Creates a new #GESTrackEffect from a #GESMaterial
      *
      * Returns: a newly created #GESTrackEffect, or %NULL if something went wrong.
      */
    GESTrackEffect *ges_track_effect_new_from_material(GESTrackEffect *effect, GESMaterial *material);

      /**
      * ges_track_effect_new_from_bin_desc:
      *
      * @bin_dec: The gst-launch like bin description of the effect
      *
      * Creates a new #GESTrackEffect from the description of the bin. This is a 
      * convenience method for testing puposes.
      *
      * Returns: a newly created #GESTrackEffect, or %NULL if something went wrong.
      */
    GESTrackEffect *ges_track_effect_new_from_bin_desc(GESTrackEffect *effect,
                                                       const gchar *bin_desc);

  D - The GESTimelineEffect API:

  The GESTimelineEffect basically doesn't have anything else but what GESTimelineObject has.

    -> Usecases: The user wants to control multiple effects in sync. The user wants to add
                 an effect to the whole timeline. The user wants to had an effect to a 
                 segment of the timeline without caring about what clip it is applied on.

=================
  TODO GESRegistry API:
      This should be a singleton since we don't want an app to instanciate more
      than one registry. It should be able to get effects from various sources.
      We should also make sure any custom effect is detected.

      /**
      * ges_registry_get_default:
      *
      * Returns a newly created #GESEffectRegistry or the existing one increasing
      * its refcount
      */
      GESEffectRegistry *
      ges_registry_get_default (void);
        -> Usecases:
          + Have a registry of all effects that are on the système (UC-8)

      /**
      * ges_effect_registry_get_effect_list:
      *
      * @self: The origin #GESEffectRegistry
      *
      * Returns a #GList of #GESEffectDescriptors. The
      */
      GList *
      ges_registry_get_effect_list (GESEffectRegistry *self);
        -> Usecases:
          + Get all effects descriptors that are on the système (UC-8)