design: More specifications of compositing and material handling

Doing it this way will enable us to handle:
* Output conforming (proper scaling/conversion at the right place)
* Compositing in an easy way at the layer level
* Avoid having too many transformation elements
This commit is contained in:
Edward Hervey 2011-09-02 18:20:00 +02:00
parent 7cb5eb1187
commit 6f0a692215

View file

@ -37,6 +37,7 @@ Index of features:
* Plugin system * Plugin system
* Project file load/save support (GESFormatter) * Project file load/save support (GESFormatter)
Status: Status:
@ -136,6 +137,10 @@ Index of features:
but could also contain extra information provided by 3rd party but could also contain extra information provided by 3rd party
modules. modules.
The information regarding the various streams (and obtained through
optionally running GstDiscoverer) is not stored and has to be
re-analyzed else where.
Definition: Definition:
Material: n, The substance or substances out of which a thing is or Material: n, The substance or substances out of which a thing is or
can be made. can be made.
@ -146,6 +151,14 @@ Index of features:
A Material object contains all the information which is independent A Material object contains all the information which is independent
of the usage of that material in a timeline. of the usage of that material in a timeline.
A Material contains the list of 'streams' that can be provided with
as much information as possible (ex: contains audio and video
streams with full caps information, or better yet the output of
GstDiscoverer).
A Material contains the various Metadata (author, title, origin,
copyright ,....).
A Material object can specify the TimelineSource class to use in a A Material object can specify the TimelineSource class to use in a
Layer. Layer.
@ -180,10 +193,16 @@ Index of features:
resolution/quality/... version for the editing phase and the resolution/quality/... version for the editing phase and the
original material for final rendering phase). original material for final rendering phase).
Requires:
GESMaterial
* Editing modes (Ripple/Roll/Slip/Slide) * Editing modes (Ripple/Roll/Slip/Slide)
Status:
Not implemented.
Problems: Problems:
Most editing relies on heavy usage of 4 editing tools which editors Most editing relies on heavy usage of 4 editing tools which editors
will require. Ripple/Roll happen on edit points (between two clips) will require. Ripple/Roll happen on edit points (between two clips)
@ -234,6 +253,9 @@ Index of features:
aspect ratio of 4:3, we will make the width of the two videos aspect ratio of 4:3, we will make the width of the two videos
be equal without distorting their respective aspect-ratios. be equal without distorting their respective aspect-ratios.
Requires:
GESMaterial
See also: See also:
Video compositing and audio mixing Video compositing and audio mixing
@ -241,6 +263,11 @@ Index of features:
* Video compositing and audio mixing * Video compositing and audio mixing
Status:
Not implemented. The bare minimum to implement are the static
absolute property handling. Relative/variable properties and group
handling can be done once we know how to handle object grouping.
Problems: Problems:
Editing requires not only a linear combination of cuts and Editing requires not only a linear combination of cuts and
sequences, but also mixing various content/effect at the same sequences, but also mixing various content/effect at the same
@ -301,10 +328,29 @@ Index of features:
channels in the case of 7.1 upmixing) WITHOUT any extra loss in channels in the case of 7.1 upmixing) WITHOUT any extra loss in
quality. quality.
Add a set of extensible properties to the base source classes
relative to compositing and mixing.
The properties values can be both set/stored as 'relative' values Create a new 'CompositingProperties' object for audio and video
which is an extensible set of properties for media-specific
positioning. This contains the properties mentionned above.
Add the CompositingProperties object to the base GESTrackObject
which points to the audio or video CompositingProperties
object (depending on what format that object is handling).
Provide convenience functions to retrieve and set the audio or video
compositing properties of a GESTrackObject. Do the same for the
GESTimelineObject, which proxies it to the relevant GESTrackObject.
Create a new GESTrack{Audio|Video}Compositing GstElement which will
be put in each track as a priority 0 expandable GnlOperation.
That object will be able to figure out which
mixing/scaling/conversion elements to use at any given time by
inspecting:
* The various GESTrackObject Compositing Properties
* The various GESTrackObject GESMaterial stream properties
* The GESTrack target output GstCaps
The properties values could be both set/stored as 'relative' values
and as 'absolute' values in order to handle any input/output formats and as 'absolute' values in order to handle any input/output formats
or setting. or setting.
@ -318,9 +364,8 @@ Index of features:
changes, all object positioning will change relatively to that changes, all object positioning will change relatively to that
format. format.
Add {audio|video}mixer elements in the Tracks where applicable, and Requires:
take into account input/output formats and relative values to apply GESMaterial
the most efficient processing.
See also: See also:
Coherent handling of Content in different formats Coherent handling of Content in different formats
@ -334,7 +379,8 @@ Index of features:
logos/videos, bluescreen, ...). logos/videos, bluescreen, ...).
Those streams need to be handle-able by the user just like Those streams need to be handle-able by the user just like
non-alpha videos. non-alpha videos without losing the transparency regions (i.e. it
should be properly blended with the underlying regions).