mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-09-08 13:18:52 +00:00
Original commit message from CVS: 2005-11-21 Andy Wingo <wingo@pobox.com> * *.h: * *.c: Ran scripts/update-macros. Oh yes. * gst/gstobject.h (GST_OBJECT_GET_LOCK, GST_OBJECT_LOCK) (GST_OBJECT_TRYLOCK, GST_OBJECT_UNLOCK): Renamed from GST_GET_LOCK, etc. * scripts/update-macros: New script. Run it on your files to change GST_LOCK to GST_OBJECT_LOCK, and the same for UNLOCK as well.
676 lines
29 KiB
Text
676 lines
29 KiB
Text
Fixing Threading
|
|
----------------
|
|
|
|
1) Observations
|
|
|
|
The following observations are made when considering the current (17/11/2004)
|
|
problems in gstreamer.
|
|
|
|
- Bin state changes.
|
|
Currently the state of a bin is determined by the highest state of the
|
|
children, This is in particular a problem for GstThread because a thread
|
|
should start/stop spinning at any time depending on the state of a child.
|
|
|
|
ex 1:
|
|
|
|
+-------------------------------------+
|
|
| GstThread |
|
|
| +--------+ +---------+ +------+ |
|
|
| | src | | decoder | | sink | |
|
|
| | src-sink src-sink | |
|
|
| +--------+ +---------+ +------+ |
|
|
+-------------------------------------+
|
|
|
|
When performing the state change on the GstThread to PLAYING, one of the
|
|
children (at random) will go to PLAYING first, this will trigger a method
|
|
in GstThread that will start spinning the thread. Some elements are not yet
|
|
in the PLAYING state when the scheduler starts iterating elements. This
|
|
is not a clean way to start the data passing.
|
|
|
|
State changes also trigger negotiation and scheduling (in the other thread)
|
|
can do too. This creates races in negotiation.
|
|
|
|
- ERROR and EOS conditions triggering a state change
|
|
|
|
A typical problem is also that since scheduling starts while the state change
|
|
happens, it is possible that the elements go to EOS or ERROR before the
|
|
state change completes. Currently this makes the elements go to PAUSED again,
|
|
creating races with the state change in progress. This also gives the
|
|
impression to the core that the state change failed.
|
|
|
|
- no locking whatsoever
|
|
|
|
When an element does a state change, it is possible for another thread to
|
|
perform a conflicting state change.
|
|
|
|
- negotiation is not designed to work over multithread boundaries.
|
|
|
|
negotiation over a queue is not possible. There is no method or policy of
|
|
discovering a media type and then commiting it. It is also not possible to
|
|
tie the negotiated media to the relevant buffer.
|
|
|
|
ex1:
|
|
it Should be possible to queue the old and the new formats in a queue.
|
|
The element connected to the sinkpad of the queue should be able to
|
|
find out that the new format will be accepted by the element connected
|
|
on the srcpad of the queue, even if that element is streaming the old
|
|
format.
|
|
|
|
+------------------------------+
|
|
| GstQueue |
|
|
| +++++++++++++++++++++++++ |
|
|
-sink |B|B|B|B|B|B|A|A|A|A|A|A| src-
|
|
| +++++++++++++++++++++++++ |
|
|
+------------------------------+
|
|
+----------+ +----------+
|
|
buffers in buffers in
|
|
new format old format
|
|
|
|
- element properties are not threadsafe
|
|
|
|
When setting an element property while streaming, the element does no
|
|
locking whatsoever to guarantee its internal consistency.
|
|
|
|
- No control over streaming.
|
|
|
|
When some GstThread is iterating and you want to reconnect a pad, there
|
|
is no way to block the pad, perform the actions and then unblock it
|
|
again. This leads to thread problems where a pad is negotiation at the
|
|
same time that it is passing data.
|
|
|
|
This is currently solved by PAUSING the pipeline or performing the actions
|
|
in the same threadcontext as the iterate loop.
|
|
|
|
- race conditions in synchronizing the clocks and spinning up the pipeline.
|
|
Currently the clock is started as soon as the pipeline is set to playing.
|
|
Because some time elaspes before the elements are negotiated, autoplugged
|
|
and streaming, the first frame/sample almost always arrives late at the
|
|
sinks. Hacks exist to adjust the element base time to compensate for the
|
|
delay but this clearly is not clean.
|
|
|
|
- race conditions when performing seeks in the pipeline. Since the elements
|
|
have no control over the streaming threads, they cannot block them or
|
|
resync them to the new seek position. It is also hard to synchronize them
|
|
with the clock.
|
|
|
|
- race conditions when sending tags and error messages up the pipeline
|
|
hierarchy. These races are either caused by glib refcounting problems and
|
|
by not properly locking.
|
|
|
|
- more as changes are implemented and testcases are written
|
|
|
|
2) possible solutions
|
|
|
|
- not allowing threading at all
|
|
|
|
Run the complete pipeline in a single thread. Complications to solve include
|
|
handling of blocking operations like source elements blocking in kernel
|
|
space, sink elements blocking on the clock or kernel space, etc.. In practice,
|
|
all operations should be made non-blocking because a blocking element can
|
|
cause the rest of the pipeline to block as well and cause it to miss a deadline.
|
|
A non-blocking model needs cooperation from the kernel (with callbacks) or
|
|
requires the use of a polling mechanism, both of which are either impractical
|
|
or too CPU intensive and in general not achievable for a general purpose
|
|
Multimedia framework. For this reason we will not go further with this
|
|
solution.
|
|
|
|
- Allow threading.
|
|
|
|
To make this work, We propose the following changes:
|
|
|
|
- Remove GstThread, it does not add anything useful in a sense that you cannot
|
|
arbitrarily place the thread element, it needs decoupled elements around the
|
|
borders.
|
|
|
|
- Simplify the state changes of bins elements. A bin or element never changes
|
|
state automatically on EOS and ERROR.
|
|
|
|
- Introduce the concept of the application and the streaming thread. All data
|
|
passing is done in the streaming thread. This also means that all operations
|
|
either are performed in the application thread or streaming thread and that
|
|
they should be protected against competing operations in other threads.
|
|
This would define a policy for adding appropriate locking.
|
|
|
|
- Move the creation of threads into source and loop-based elements. This will
|
|
make it possible for the elements in control of the threads to perform the
|
|
locking when needed. One particular instance is for example the state changes,
|
|
by creating the threads in the element, it is possible to sync the streaming
|
|
and the application thread (which does the state change).
|
|
|
|
- Remove negotiation from state changes. This will remove the conflict between
|
|
streaming and negotiating elements.
|
|
|
|
- add locks around pad operations like negotiation, streaming, linking, etc. This
|
|
will remove races between these conflicting operations. This will also make it
|
|
possible to un/block dataflow.
|
|
|
|
- add locks around bin operations like add/removing elements.
|
|
|
|
- add locks around element operations like state changes and property changes.
|
|
|
|
- add a 2-phase directed negotiation process. The source pad queries and figures
|
|
out what the sinkpad can take in the first phase. In the second phase it sends
|
|
the new format change as an event to the peer element. This event can be
|
|
interleaved with the buffers and can travel over queues inbetween the buffers.
|
|
Need to rethink this wrt bufferpools (see DShow and old bufferpool implementation)
|
|
|
|
- add a preroll phase that will be used to spin up the pipeline and align frames/samples
|
|
in the sinks. This phase will happen in the PAUSED state. This also means that
|
|
dataflow will happen in the PAUSED state. Sinks will not sink samples in the PAUSED
|
|
state but will complete their state change asynchronously. This will allow
|
|
us to have perfect synchronisation with the clock.
|
|
|
|
- a two phase seek policy. First the event travels upstream, putting all elements in
|
|
the seeking phase and making them synchronize to the new position. In the
|
|
second phase the DISCONT event signals the end of the seek and all filters can
|
|
continue with the new position.
|
|
|
|
- Error messages, EOS, tags and other events in the pipeline should be sent to a
|
|
mainloop. The app then has an in-thread mechanism for getting information about
|
|
the pipeline. It should also be possible to get the messages directly from the
|
|
elements itself, like signals. The application programmer has to know that
|
|
working these events come from another thread and should handle them accordingly.
|
|
|
|
- Add return values to push/pull so that errors upstream or downstream can be noted
|
|
by other elements so that they can disable themselves or propagate the error.
|
|
|
|
|
|
3) detailed explanation
|
|
|
|
a) Pipeline construction
|
|
|
|
Pipeline construction includes:
|
|
|
|
- adding/removing elements to the bin
|
|
- finding elements in a bin by name and interface
|
|
- setting the clock on a pipeline.
|
|
- setting properties on objects
|
|
- linking/unlinking pads
|
|
|
|
These operations should take the object lock to make sure it can be
|
|
executed from different threads.
|
|
|
|
When connecting pads to other pads from elements inside another bin,
|
|
we require that the bin has a ghostpad for the pad. This is needed so
|
|
that the bin looks like a self-contained element.
|
|
|
|
not allowed:
|
|
+---------------------+
|
|
| GstBin |
|
|
+---------+ | +--------+ |
|
|
| element | | | src | |
|
|
sink src------sink src- ... |
|
|
+---------+ | +--------+ |
|
|
+---------------------+
|
|
|
|
allowed:
|
|
+-----------------------+
|
|
| GstBin |
|
|
| +--------+ |
|
|
+---------+ | | src | |
|
|
| element | | sink src- ... |
|
|
sink src---sink/ +--------+ |
|
|
+---------+ +-----------------------+
|
|
|
|
This requirement is important when we need to sort the elements in the
|
|
bin to perfrom the state change.
|
|
|
|
|
|
testcases:
|
|
|
|
- create a bin, add/remove elements from it
|
|
- add/remove from different threads and check the bin integrity.
|
|
|
|
b) state changes
|
|
|
|
An element can be in one of the four states NULL, READY, PAUSED, PLAYING.
|
|
|
|
NULL: starting state of the element
|
|
READY: element is ready to start running.
|
|
PAUSED: element is streaming data, has opened devices etc.
|
|
PLAYING: element is streaming data and clock is running
|
|
|
|
Note that data starts streaming even in the PAUSED state. The only difference
|
|
between the PAUSED and PLAYING state is that the clock is running in the
|
|
PLAYING state. This mostly has an effect on the renderers which will block on
|
|
the first sample they receive when in PAUSED mode. The transition from
|
|
READY->PAUSED is called the preroll state. During that transition, media is
|
|
queued in the pipeline and autoplugging is done.
|
|
|
|
Elements are put in a new state using the _set_state function. This function
|
|
can return the following return values:
|
|
|
|
typedef enum {
|
|
GST_STATE_FAILURE = 0,
|
|
GST_STATE_PARTIAL = 1,
|
|
GST_STATE_SUCCESS = 2,
|
|
GST_STATE_ASYNC = 3
|
|
} GstElementStateReturn;
|
|
|
|
GST_STATE_FAILURE is returned when the element failed to go to the
|
|
required state. When dealing with a bin, this is returned when one
|
|
of the elements failed to go to the required state. The other elements
|
|
in the bin might have changed their states succesfully. This return
|
|
value means that the element did _not_ change state, for bins this
|
|
means that not all children have changed their state.
|
|
|
|
GST_STATE_PARTIAL is returned when some elements in a bin where in the
|
|
locked state and therefore did not change their state. Note that the
|
|
state of the bin will be changed regardless of this PARTIAL return value.
|
|
|
|
GST_STATE_SUCCES is returned when all the elements successfully changed their
|
|
states.
|
|
|
|
GST_STATE_ASYNC is returned when an element is going to report the success
|
|
or failure of a state change later.
|
|
|
|
The state of a bin is not related to the state of its children but only to
|
|
the last state change directly performed on the bin or on a parent bin. This
|
|
means that changing the state of an element inside the bin does not affect
|
|
the state of the bin.
|
|
|
|
Setting the state on a bin that is already in the correct state will
|
|
perform the requested state change on the children.
|
|
|
|
Elements are not allowed to change their own state. For bins, it is allowed
|
|
to change the state of its children. This means that the application
|
|
can only know about the states of the elements it has explicitly set.
|
|
|
|
There is a difference in the way a pipeline and a bin handles the state
|
|
change of its children:
|
|
|
|
- a bin returns GST_STATE_ASYNC when one of its children returns an
|
|
ASYNC reply.
|
|
|
|
- a pipeline never returns GST_STATE_ASYNC but returns from the state
|
|
change function after all ASYNC elements completed the state change.
|
|
This is done by polling the ASYNC elements until they return their
|
|
final state.
|
|
|
|
The state change function must be fast an cannot block. If a blocking behaviour
|
|
is unavoidable, the state change function must perform an async state change.
|
|
Sink elements therefore always use async state changes since they need to
|
|
wait before the first buffer arrives at the sink.
|
|
|
|
A bin has to change the state of its children elements from the sink to the
|
|
source elements. This makes sure that a sink element is always ready to
|
|
receive data from a source element in the case of a READY->PAUSED state change.
|
|
In the case of a PAUSED->READY state, the sink element will be set to READY
|
|
first so that the source element will receive an error when it tries to push
|
|
data to this element so that it will shut down as well.
|
|
|
|
For loop based elements we have to be careful since they can pull a buffer
|
|
from the peer element before it has been put in the right state.
|
|
The state of a loop based element is therefore only changed after the source
|
|
element has been put in the new state.
|
|
|
|
c) Element state change functions
|
|
|
|
The core will call the change_state function of an element with the element
|
|
lock held. The element is responsible for starting any streaming tasks/threads
|
|
and making sure that it synchronizes them to the state change function if
|
|
needed.
|
|
|
|
This means that no other thread is allowed to change the state of the element
|
|
at that time and for bins, it is not possible to add/remove elements.
|
|
|
|
When an element is busy doing the ASYNC state change, it is possible that another
|
|
state change happens. The elements should be prepared for this.
|
|
|
|
An element can receive a state change for the same state it is in. This
|
|
is not a problem, some elements (like bins) use this to resynchronize their
|
|
children. Other elements should ignore this state change and return SUCCESS.
|
|
|
|
When performing a state change on an element that returns ASYNC on one of
|
|
the state changes, ASYNC is returned and you can only proceed to the next
|
|
state change change when this ASYNC state change completed. Use the
|
|
gst_element_get_state function to know when the state change completed.
|
|
An example of this behaviour is setting a videosink to PLAYING, it will
|
|
return ASYNC in the state change from READY->PAUSED. You can only set
|
|
it to PLAYING when this state change completes.
|
|
|
|
Bins will perform the state change code listed in d).
|
|
|
|
For performing the state change, two variables are used: the current state
|
|
of the element and the pending state. When the element is not performing a
|
|
state change, the pending state == None. The state change variables are
|
|
protected by the element lock. The pending state != None as long as the
|
|
state change is performed or when an ASYNC state change is running.
|
|
|
|
The core provides the following function for applications and bins to
|
|
get the current state of an element:
|
|
|
|
bool gst_element_get_state(&state, &pending, timeout);
|
|
|
|
This function will block while the state change function is running inside
|
|
the element because it grabs the element lock.
|
|
When the element did not perform an async state change, this function returns
|
|
TRUE immediatly with the state updated to reflect the current state of the
|
|
element and pending set to None.
|
|
When the element performed an async state change, this function will block
|
|
for the value of timeout and will return TRUE if the element completed the
|
|
async state change within that timeout, otherwise it returns FALSE, with
|
|
the current and pending state filled in.
|
|
|
|
The algorithm is like this:
|
|
|
|
bool gst_element_get_state(elem, &state, &pending, timeout)
|
|
{
|
|
g_mutex_lock (ELEMENT_LOCK);
|
|
if (elem->pending != none) {
|
|
if (!g_mutex_cond_wait(STATE, ELEMENT_LOCK, timeout) {
|
|
/* timeout triggered */
|
|
*state = elem->state;
|
|
*pending = elem->pending;
|
|
ret = FALSE;
|
|
}
|
|
}
|
|
if (elem->pending == none) {
|
|
*state = elem->state;
|
|
*pending = none;
|
|
ret = TRUE;
|
|
}
|
|
g_mutex_unlock (ELEMENT_LOCK);
|
|
|
|
return ret;
|
|
}
|
|
|
|
For plugins the following function is provided to commit the pending state,
|
|
the ELEMENT_LOCK should be held when calling this function:
|
|
|
|
gst_element_commit_state(element)
|
|
{
|
|
if (pending != none) {
|
|
state = pending;
|
|
pending = none;
|
|
}
|
|
g_cond_broadcast (STATE);
|
|
}
|
|
|
|
For bins the gst_element_get_state() works slightly different. It will run
|
|
the function on all of its children, as soon as one of the children returns
|
|
FALSE, the method returns FALSE with the state set to the current bin state
|
|
and the pending set to pending state.
|
|
|
|
For bins with elements that did an ASYNC state change, the _commit_state()
|
|
is only executed when actively calling _get_state(). The reason for this is
|
|
that when a child of the bin commits its state, this is not automatically
|
|
reported to the bin. This is not a problem since the _get_state() function
|
|
is the only way to get the current and pending state of the bin and is always
|
|
consistent.
|
|
|
|
d) bin state change algorithm
|
|
|
|
In order to perform the sink to source state change a bin must be able to sort
|
|
the elements. To make this easier we require that elements are connected to
|
|
bins using ghostpads on the bin.
|
|
|
|
The algoritm goes like this:
|
|
|
|
d = [ ] # list of delayed elements
|
|
p = [ ] # list of pending async elements
|
|
q = [ elements without srcpads ] # sinks
|
|
while q not empty do
|
|
e = dequeue q
|
|
s = [ all elements connected to e on the sinkpads ]
|
|
q = q append s
|
|
if e is entry point
|
|
d = d append e
|
|
else
|
|
r = state change e
|
|
if r is ASYNC
|
|
p = p append e
|
|
done
|
|
while d not empty do
|
|
e = dequeue d
|
|
r = state change e
|
|
if r is ASYNC
|
|
p = p append e
|
|
done
|
|
# a bin would return ASYNC here if p is not empty
|
|
|
|
# this last part is only performed by a pipeline
|
|
while p not empty do
|
|
e = peek p
|
|
if state completed e
|
|
dequeue e from p
|
|
done
|
|
|
|
The algorithm first tries to find the sink elements, ie. ones without
|
|
sinkpads. Then it changes the state of each sink elements and queues
|
|
the elements connected to the sinkpads.
|
|
|
|
The entry points (loopbased and getbased elements) are delayed as we
|
|
first need to change the state of the other elements before we can activate
|
|
the entry points in the pipeline.
|
|
|
|
The pipeline will poll the async children before returning.
|
|
|
|
e) The GstTask infrastructure
|
|
|
|
A new component: GstTask is added to the core. A task is created by
|
|
an instance of the abstract GstScheduler class.
|
|
|
|
Each schedulable element (when added to a pipeline) is handed a
|
|
reference to a GstScheduler. It can use this object to create
|
|
a GstTask, which is basically a managed wrapper around a threading
|
|
library like GThread. It should be possible to write a GstScheduler
|
|
instance that uses other means of scheduling, like one that does not
|
|
use threads but implements task switching based on mutex locking.
|
|
|
|
When source and loopbased elements want to create the streaming thread
|
|
they create an instance of a GstTask, which they pass a pointer to
|
|
a loop-function. This function will be called as soon as the element
|
|
performs GstTask.start(). The element can stop and uses mutexes to
|
|
pause the GstTask from, for example, the state change function or the
|
|
event functions.
|
|
|
|
The GstTasks implement the streaming threads.
|
|
|
|
f) the preroll phase
|
|
|
|
Element start the streaming threads in the READY->PAUSED state. Since
|
|
the elements that start the threads are put in the PAUSED state last,
|
|
after their connected elements, they will be able to deliver data to
|
|
their peers without problems.
|
|
|
|
Sink elements like audio and videosinks will return an async state change
|
|
reply and will only commit the state change after receiving the first
|
|
buffer. This will implement the preroll phase.
|
|
|
|
The following pseudo code shows an algorithm for commiting the state
|
|
change in the streaming method.
|
|
|
|
GST_OBJECT_LOCK (element);
|
|
/* if we are going to PAUSED, we can commit the state change */
|
|
if (GST_STATE_TRANSITION (element) == GST_STATE_READY_TO_PAUSED) {
|
|
gst_element_commit_state (element);
|
|
}
|
|
/* if we are paused we need to wait for playing to continue */
|
|
if (GST_STATE (element) == GST_STATE_PAUSED) {
|
|
|
|
/* here we wait for the next state change */
|
|
do {
|
|
g_cond_wait (element->state_cond, GST_OBJECT_GET_LOCK (element));
|
|
} while (GST_STATE (element) == GST_STATE_PAUSED);
|
|
|
|
/* check if we got playing */
|
|
if (GST_STATE (element) != GST_STATE_PLAYING) {
|
|
/* not playing, we can't accept the buffer */
|
|
GST_OBJECT_UNLOCK (element);
|
|
gst_buffer_unref (buf);
|
|
return GST_FLOW_WRONG_STATE;
|
|
}
|
|
}
|
|
GST_OBJECT_UNLOCK (element);
|
|
|
|
|
|
|
|
g) return values for push/pull
|
|
|
|
To recover from pipeline errors in a more elegant manner than just
|
|
shutting down the pipeline, we need more finegrained error messages
|
|
in the data transport. The plugins should be able to know what goes
|
|
wrong when interacting with their outside environment. This means
|
|
that gst_pad_push/gst_pad_pull and gst_event_send should return a
|
|
result code.
|
|
|
|
Possible return values include:
|
|
|
|
- GST_OK
|
|
- GST_ERROR
|
|
- GST_NOT_CONNECTED
|
|
- GST_NOT_NEGOTIATED
|
|
- GST_WRONG_STATE
|
|
- GST_UNEXPECTED
|
|
- GST_NOT_SUPPORTED
|
|
|
|
GST_OK
|
|
Data transport was successful
|
|
|
|
GST_ERROR
|
|
An error occured during transport, such as a fatal decoding error,
|
|
the pad should not be used again.
|
|
|
|
GST_NOT_CONNECTED
|
|
The pad was not connected
|
|
|
|
GST_NOT_NEGOTIATED
|
|
The peer does not know what datatype is going over the pipeline.
|
|
|
|
GST_WRONG_STATE
|
|
The peer pad is not in the correct state.
|
|
|
|
GST_UNEXPECTED
|
|
The peer pad did not expect the data because it was flushing or
|
|
received an eos.
|
|
|
|
GST_NOT_SUPPORTED
|
|
The operation is not supported.
|
|
|
|
The signatures of the functions will become:
|
|
|
|
GstFlowReturn gst_pad_push (GstPad *pad, GstBuffer *buffer);
|
|
GstFlowReturn gst_pad_pull (GstPad *pad, GstBuffer **buffer);
|
|
|
|
GstResult gst_pad_push_event (GstPad *pad, GstEvent *event);
|
|
|
|
- push_event will send the event to the connected pad.
|
|
|
|
For sending events from the application:
|
|
|
|
GstResult gst_pad_send_event (GstPad *pad, GstEvent *event);
|
|
|
|
h) Negotiation
|
|
|
|
Implement a simple two phase negotiation. First the source queries the
|
|
sink if it accepts a certain format, then it sends the new format
|
|
as an event. Sink pads can also trigger a state change by requesting
|
|
a renegotiation.
|
|
|
|
i) Mainloop integration/GstBus
|
|
|
|
All error, warning and EOS messages from the plugins are sent to an event
|
|
queue. The pipeline reads the messages from the queue and will either
|
|
handle them or forward them to the main event queue that is read by the
|
|
application.
|
|
|
|
Specific pipelines can be written that deal with negotiation messages and
|
|
errors in the pipeline intelligently. The basic pipeline will stop the
|
|
pipeline when an error occurs.
|
|
|
|
Whenever an element posts a message on the event queue, a signal is also
|
|
fired that can be catched by the application. When dealing with those
|
|
signals the application has to be aware that they come from the streaming
|
|
threads and need to make sure they use proper locking to protect their
|
|
own data structures.
|
|
|
|
The messages will be implemented using a GstBus object that allows
|
|
plugins to post messages and allows the application to read messages either
|
|
synchronous or asynchronous. It is also possible to integrate the bus in
|
|
the mainloop.
|
|
|
|
The messages will derive from GstData to make them a lightweight refcounted
|
|
object. Need to figure out how we can extend this method to encapsulate
|
|
generic signals in messages too.
|
|
|
|
This decouples the streaming thread from the application thread and should
|
|
avoid race conditions and pipeline stalling due to application interaction.
|
|
|
|
It is still possible to receive the messages in the streaming thread context
|
|
if an application wants to. When doing this, special care has to be taken
|
|
when performing state changes.
|
|
|
|
j) EOS
|
|
|
|
When an element goes to EOS, it sends the EOS event to the peer plugin
|
|
and stops sending data on that pad. The peer element that received an EOS
|
|
event on a pad can refuse any buffers on that pad.
|
|
|
|
All elements without source pads must post the EOS message on the message
|
|
queue. When the pipeline receives an EOS event from all sinks, it will
|
|
post the EOS message on the application message queue so that the application
|
|
knows the pipeline is in EOS. Elements without any connected sourcepads
|
|
should also post the EOS message. This makes sure that all "dead-ends"
|
|
signalled the EOS.
|
|
|
|
No state change happens when elements go to EOS but the elements with the
|
|
GstTask will stop their tasks and so stop producing data.
|
|
|
|
An application can issue a seek operation which makes all tasks running
|
|
again so that they can start streaming from the new location.
|
|
|
|
|
|
|
|
A) threads and lowlatency
|
|
|
|
People often think it is a sin to use threads in low latency applications. This is true
|
|
when using the data has to pass thread boundaries but false when it doesn't. Since
|
|
source and loop based elements create a thread, it is possible to construct a pipeline
|
|
where data passing has to cross thread boundaries, consider this case:
|
|
|
|
+-----------------------------------+
|
|
| +--------+ +--------+ |
|
|
| |element1| |element2| |
|
|
| .. -sink src-sink src- .. |
|
|
| +--------+ +--------+ |
|
|
+-----------------------------------+
|
|
|
|
The two elements are loop base and thus create a thread to drive the pipeline. At the
|
|
border between the two elements there is a mutex to pass the data between the two
|
|
threads. When using these kinds of element in a pipeline, low-latency will not be
|
|
possible. For low-latency apps, don't use these constructs!
|
|
|
|
Note that in a typical pipeline with one get-based element and two chain-based
|
|
elements (decoder/sink) there is only one thread, no data is crossing thread
|
|
boundaries and thus this pipeline can be low-latency. Also note that while this
|
|
pipeline is streaming no interaction or locking is done between it and the main
|
|
application.
|
|
|
|
+-------------------------------------+
|
|
| +--------+ +---------+ +------+ |
|
|
| | src | | decoder | | sink | |
|
|
| | src-sink src-sink | |
|
|
| +--------+ +---------+ +------+ |
|
|
+-------------------------------------+
|
|
|
|
|
|
B) howto make non-threaded pipelines
|
|
|
|
For low latency it is required to not have datapassing cross any thread
|
|
borders. Here are some pointers for making sure this requirement is met:
|
|
|
|
- never connect a loop or chain based element to a loop based element, this
|
|
will create a new thread for the sink loop element.
|
|
|
|
- do not use queues or any other decoupled element, as they implicitly
|
|
create a thread boundary.
|
|
|
|
- At least one thread will be created for any source element (either in the
|
|
connected loop-based element or in the source itself) unless the source
|
|
elements are connected to the same loop based element.
|
|
|
|
- when designing sinks, make them non-blocking, use the async clock callbacks
|
|
to schedule media rendering in the same thread (if any) as the clock. Sinks that
|
|
provide the clock can be made blocking.
|
|
|