mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-09-23 04:20:17 +00:00
321 lines
11 KiB
Text
321 lines
11 KiB
Text
Effects
|
||
-------
|
||
|
||
Summary
|
||
-------
|
||
1. Basic ideas
|
||
2. Problems
|
||
3. Propositions to solve those problems
|
||
A. The registry
|
||
B. Effects configurability
|
||
C. Keyframes
|
||
4. Use-cases
|
||
5. API draft
|
||
|
||
The goal of this proposal is to find a simple way to handle effects through an
|
||
API which would allow developers to handle any use-case
|
||
|
||
1. Basic ideas
|
||
----------------
|
||
|
||
* Effects are actually descendant of GESTrackOperation
|
||
|
||
* You can add effects on any clip, layer or on part of them
|
||
|
||
* Effects are configurable and those could change during time
|
||
|
||
* We should be able to handle the gnome-video-effects standard
|
||
|
||
* We can implement complexe effects. It means effects that are more
|
||
than adding GstElement-s to the timeline. It can also mean effects
|
||
that are both: video and audio at the same time
|
||
|
||
2. Problems
|
||
----------
|
||
* We should be able to handle list of available effects on the system at
|
||
runtime.
|
||
|
||
* We should be able to configure effects through an API in GES and
|
||
not directly configuring the corresponding GstElement properties.
|
||
|
||
* We should also expose the GstElement-s contained in an effect so
|
||
it is possible for people to control their properties as they wish.
|
||
|
||
* We should be able to handle complexe effects.
|
||
|
||
* We should be able to implement complexe effects directly in GES
|
||
|
||
* We should be able to configure effects in the time -> Keyframes without
|
||
duplicating code from GStreamer
|
||
|
||
|
||
3. Propositions to solve those problems
|
||
---------------------------------------
|
||
|
||
A. The registry
|
||
|
||
We could implement a GESEffectRegistry which would actually
|
||
retrieve elements (effects) from the GSTRegistry and any other mean
|
||
such as gnome-video-effects to let us get all the effects that are present
|
||
on the system....
|
||
This way the developers could have the list of all the effects
|
||
that are installed on the system pretty easily.
|
||
|
||
B. Effects configurability
|
||
|
||
The idea to be able to configure effects through a simple API in GES would
|
||
be to create a class GESEffect which would actually be a mapping between
|
||
a GESTrackOperation and the properties of the GstElements contained in the
|
||
GESTrackOperation.
|
||
Basicly it would allow the retrieving of all configurable elements properties:
|
||
* names
|
||
* human readable names
|
||
* types
|
||
* possible values
|
||
* ...
|
||
We would also have a method to set those properties easily
|
||
|
||
The problem which also happens is that in the case of gnome-video-effects,
|
||
we should be able to allow access to some properties whiches have been
|
||
specified. We will need to improve the current standard in order to handle
|
||
this use-case.
|
||
|
||
C. Keyframes
|
||
We may want to handle this use-case directly in GES and for any kind of
|
||
time related configuration? FIXME
|
||
=> Special specifications for that?
|
||
|
||
4. Use-cases
|
||
-----------
|
||
|
||
UC-1. The user wants to add an effect to an entire clip => GESTimelineObject new API
|
||
|
||
UC-2. The user wants to add an effect to an entire layer => GESTimelineLayer API.
|
||
|
||
UC-3. The developer wants to allow his users to configure effects => mapping between
|
||
GESTrackEffect and the configurable properties of elements (GESEffect).
|
||
|
||
UC-4. The user wants to add an effect on a special portion of a clip, we should
|
||
allow him to configure that thanks to a system of keyframes. FIXME
|
||
|
||
UC-5. We want to implement an effect which isn't only composed by a bin, but is more
|
||
complexe than that (ex: "effect '24'") => we have the GESTrackOperation
|
||
which is the base class (abstract) for this kind of implementation. We should
|
||
also have a GESPropertiesMapping abstract class which would be the base
|
||
class of GESEffect and could be used in this use-case.
|
||
|
||
UC-6. A developer wants to implement effect which handle music and video at the same
|
||
time, Would the solution be to implement a GESTimelineEffect to handle this
|
||
special usecase? FIXME
|
||
|
||
UC-7. The developers wants to configure each elements of an effect the way he wants
|
||
with a full control over it.
|
||
|
||
UC-8. Developers want to expose all effects present on the system to the end-user
|
||
|
||
5. API draft V1
|
||
------------
|
||
|
||
A. The GESEffectDescriptor
|
||
It is a structure that describes effects so they can easily be instanciate.
|
||
They can be composed of a simple description of a the bin, they can also
|
||
define more complexe effects, the GESEffectContainer will be able to handle
|
||
those cases.
|
||
|
||
/**
|
||
* ges_effect_descriptor_get_effect_name:
|
||
*
|
||
* Returns the name of the effect
|
||
*/
|
||
const gchar *
|
||
ges_effect_descriptor_get_effect_name (GESEffectDescriptor *self);
|
||
|
||
/**
|
||
* ges_effect_descriptor_get_effect_description:
|
||
*
|
||
* Returns the description of the effect
|
||
*/
|
||
const gchar *
|
||
ges_effect_descriptor_get_effect_description (GESEffectDescriptor *self);
|
||
|
||
/**
|
||
* ges_effect_descriptor_get_effect_categories:
|
||
*
|
||
* Returns an array of
|
||
*/
|
||
gchar **
|
||
ges_effect_descriptor_get_effect_categories (GESEffectDescriptor *self);
|
||
|
||
B. GESEffectRegistry API:
|
||
This should be a singleton since we don't want an app to instanciate more
|
||
than one registry. It should be able to get effects from various sources.
|
||
(UC-8)
|
||
|
||
/**
|
||
* ges_effect_registry_new:
|
||
*
|
||
* Returns a newly created #GESEffectRegistry
|
||
*/
|
||
GESEffectRegistry *
|
||
ges_effect_registry_new (void);
|
||
-> Usecases:
|
||
+ Have a registry of all effects that are on the système (UC-8)
|
||
|
||
/**
|
||
* ges_effect_registry_get_all:
|
||
*
|
||
* @self: The origin #GESEffectRegistry
|
||
*
|
||
* Returns a #GList of #GESEffectDescriptors. The
|
||
*/
|
||
GList *
|
||
ges_effect_registry_get_all (GESEffectRegistry *self);
|
||
-> Usecases:
|
||
+ Get all effects descriptors that are on the système (UC-8)
|
||
|
||
/**
|
||
* ges_effect_set_thumbnail:
|
||
*
|
||
* @self: The origin #GESEffect
|
||
*
|
||
* Returns %TRUE if it has been set, %FALSE otherwize
|
||
*/
|
||
gboolean *
|
||
ges_effect_set_thumbnail (GESEffect *self, ?? *thumbnail);
|
||
|
||
/**
|
||
* ges_effect_get_thumbnail:
|
||
*
|
||
* @self: The origin #GESEffect
|
||
*
|
||
* Returns the #GstBuffer or %NULL if the thumbnail hasen't been set yet.
|
||
*/
|
||
?? *
|
||
ges_effect_get_thumbnail (GESEffect *self);
|
||
|
||
=> Can we think about a way to add effects thumbnails directly in Gst or use
|
||
system thumbnails when running on Gnome?
|
||
|
||
C. The GESffectContainer API
|
||
|
||
This is an interface which would be first implemented in GESTimelineObject
|
||
and GESTimelineLayer.
|
||
|
||
/**
|
||
* ges_effect_container_add_effect:
|
||
*
|
||
* @object: The origin #GESEffectContainer
|
||
*
|
||
* Adds a new effect corresponding to the @effect_desc to the #GESEffectContainer
|
||
* If an effect with the exact same bin_desc as already been added to this @object,
|
||
* it will not be added, and the corresponding #GESEffect will be returned.
|
||
*
|
||
* Returns: The newly created #GESEffect, or %NULL if there was an error.
|
||
*/
|
||
GESEffect *ges_effect_container_add_effect (GESEffectContainer *object,
|
||
GESEffectDescriptor *effect_desc, gchar *name);
|
||
=> Usecases:
|
||
+ We can add any kind of effects to any object which comes from a class that
|
||
implements this interface (UC-1 and UC-2)
|
||
|
||
/**
|
||
* ges_effect_container_get_effects:
|
||
*
|
||
* @object: The origin #GESEffectContainer
|
||
*
|
||
*
|
||
* Returns the #GList of #GESEffect that are applied on the
|
||
* @object.
|
||
* The refcount of the objects will be increased. The user will have to
|
||
* unref each #GESEffect and free the #GList.
|
||
*/
|
||
GList *ges_effect_container_get_effects (GESEffectContainer *object);
|
||
-> Usecases:
|
||
+ First step to allow to allow the configuration of effects (UC-3)
|
||
|
||
/**
|
||
* ges_effect_container_get_effect_from_name:
|
||
*
|
||
* @object: The origin #GESEffectContainer
|
||
*
|
||
* Returns the #GESEffect that as @name as a name or NULL if we can't find that
|
||
* effect. The refcount of the objects will be increased. The user will have to
|
||
* unref each #GESEffect and free the #GList.
|
||
*/
|
||
GESEffect *ges_effect_container_get_effect_from_name (GESEffectContainer *object,
|
||
const gchar *name);
|
||
-> Usecases:
|
||
+ First step to allow the configuration of effects (UC-3)
|
||
|
||
/**
|
||
* ges_effect_container_remove:
|
||
*
|
||
* @object: The origin #GESEffectContainer
|
||
*
|
||
* The refcount of the objects will be increased. The user will have to unref each
|
||
* #GESEffect and free the #GList.
|
||
*/
|
||
gboolean *ges_effect_container_get_effect_from_name (GESEffectContainer *object,
|
||
const gchar *name);
|
||
-> Usecases:
|
||
+ First step to allow to allow the configuration of effects (UC-3)
|
||
|
||
D - The GESEffect API:
|
||
|
||
This class is an abstract class which aims at mapping all properties of an effect
|
||
so we can configure it through this API. They shouldn't be instanciate by clients,
|
||
but they are created calling ges_effect_container_add_effect().
|
||
|
||
/**
|
||
* ges_effect_get_all_properties:
|
||
*
|
||
* @self: The origin #GESEffectContainer
|
||
*
|
||
* Returns an array of the GParamSpec** that can be configured. They should be
|
||
* freed after use.
|
||
*/
|
||
GList *ges_effect_get_all_properties(GESEffect *self);
|
||
-> Usecases:
|
||
+ Let user configure effects easily (UC-3)
|
||
|
||
/**
|
||
* ges_effect_set_property:
|
||
*
|
||
* @self: The origin #GESEffectContainer
|
||
* @property_name : the name of the property to set
|
||
* @value : the value
|
||
*
|
||
* Returns an array of the GParamSpec** that can be configured. They should be
|
||
* freed after use.
|
||
*/
|
||
gboolean *ges_effect_set_property (GESEffect *self, gchar *property_name,
|
||
GValue *value);
|
||
-> Usecases:
|
||
+ Let user configure effects easily (UC-3)
|
||
|
||
/**
|
||
* ges_effect_get_gst_elements:
|
||
*
|
||
* @self: The origin #GESEffectContainer
|
||
*
|
||
* Returns a #GList of all the GstElement-s that are usefull in the effect.
|
||
* The user is responsible for unreffing all elements and freeing the
|
||
* #GList
|
||
*/
|
||
GList *ges_effect_get_gst_elements (GESEffect *self);
|
||
-> Usecases:
|
||
+ Let user have the full control of effects configuration (UC-3)
|
||
|
||
/**
|
||
* ges_effect_get_descriptor:
|
||
*
|
||
* @self: The origin #GESEffect
|
||
*
|
||
* Returns the #GESEffectDescriptor that describes the effect.
|
||
*/
|
||
GList *ges_effect_get_descriptor (GESEffect *self);
|
||
-> Usecases:
|
||
+ The user want's to have all the characteristic of an effect
|
||
|
||
|
||
=> We are still missing spec to handle Keyframes
|