According to the OMX specification, implementations are allowed to call
callbacks in the context of their function calls. However, our callbacks
take locks and this causes deadlocks if the unerlying OMX implementation
uses this kind of in-context calls.
A solution to the problem would be a recursive mutex. However, a normal
recursive mutex does not fix the problem because it is not guaranteed
that the callbacks are called from the same thread. What we see in Broadcom's
implementation for example is:
- OMX_Foo is called
- OMX_Foo waits on a condition
- A callback is executed in a different thread
- When the callback returns, its calling function
signals the condition that OMX_Foo waits on
- OMX_Foo wakes up and returns
The solution I came up with here is to take a second lock inside the callback,
but only if recursion is expected to happen. Therefore, all calls to OMX
functions are guarded by calls to gst_omx_rec_mutex_begin_recursion() / _end_recursion(),
which effectively tells the mutex that at this point we want to allow calls
to _recursive_lock() to succeed, although we are still holding the master lock.
This happens on the Galaxy Nexus, and causes the pipeline to hang waiting
endlessly for a drain. The hack replaces the wait with a wait + 500ms timeout.
gst_omx_port_set_flushing() calls OMX_FillThisBuffer at the end of a flush
without releasing the port lock, and this can cause a deadlock with the
EventHandler. This patches fixes this by dropping the lock for the duration of
the fill buffer call.
We can't pass them back to OMX_FillThisBuffer() or OMX_EmptyThisBuffer()
but instead of doing nothing we have to put them back into our queue.
Otherwise the buffer is leaked and we will have too few buffers in
the future.
Some implementations don't reset the flags and the standard is not
really clear on the expected behaviour. Let's just always reset the
flags as they're not valid at this point anymore.
Some OMX implementations don't reset nOffset when the complete
buffer is emptied but instead only reset nFilledLen. We reset
nOffset to 0 if nFilledLen == 0, which is safe to do because
the offset *must* be 0 if the buffer is not filled at all.
Seen in QCOM's OMX implementation.
They should be deallocated by the caller before reaching the
Loaded state but to be on the safe side we will make sure
they're really deallocated here.
