Commit graph

111 commits

Author SHA1 Message Date
Jan Schmidt
39365948ff androidmedia: Invert the transform matrix from the decoder
The transform from mediacodec applies to the texture coords, but
GStreamer affine meta applies to the video geometry, which is the
opposite - so invert it to get display correct for decoders
that require transforming
2018-06-15 05:01:20 +10:00
Ursula Maplehurst
236398ee3f androidmedia: when flushing, better handle IllegalStateException received from getOutputBuffer
1. Similar to 880f3d8, don't consider not getting an output buffer as
   an error during flushing. I've seen the following sometimes when
   encoding:

   W GStreamer+amcvideoenc: java.lang.IllegalStateException
   W GStreamer+amcvideoenc:         at android.media.MediaCodec.getBuffer(Native Method)
   W GStreamer+amcvideoenc:         at android.media.MediaCodec.getOutputBuffer(MediaCodec.java:2886)

2. For amcvideodec/enc, call _find_nearest_frame (which grabs a fresh
   reference on a GstVideoCodecFrame) after we have an output buffer,
   so as to not leak the reference, in case getting an output buffer
   fails.

   Otherwise, if we get an error grabbing the output buffer, we leak
   the reference to the frame. This can cause issues with a
   v4l2bufferpool feeding the encoder not being able to clean itself
   up properly due to buffers still being marked as in-use.

https://bugzilla.gnome.org/show_bug.cgi?id=791258
2017-12-06 10:32:02 +02:00
Sebastian Dröge
732012a78e amcvideodec: Unref downstream caps after usage
https://bugzilla.gnome.org/show_bug.cgi?id=782771
2017-05-18 14:34:46 +03:00
shakin chou
febbaaddcf amcvideodec/enc: Correctly check for no PTS on input buffers
MediaCodec gives us a presentation timestamp of 0 if it does not know
anything, but GStreamer gives us GST_CLOCK_TIME_NONE. Don't mix up these
two.

https://bugzilla.gnome.org/show_bug.cgi?id=780190
2017-04-26 13:40:28 +03:00
Matthew Waters
956c4d0bde gl/format: use our own GL format enum's instead of gstvideo's
They can describe in more detail (such as component sizes) the requested format.
2017-03-13 21:10:58 +11:00
Edward Hervey
e99e69843a androidmedia: Use proper instance name 2017-01-13 14:46:36 +01:00
Matthew Waters
ed03fd6d3f amcvideodec: fix build error from change in libgstgl
gstamcvideodec.c: In function 'gst_amc_video_dec_src_query':
gstamcvideodec.c:2412:55: error: 'self' undeclared (first use in this function)
       if (gst_gl_handle_context_query ((GstElement *) self, query,
2017-01-13 21:17:33 +11:00
Matthew Waters
03b539bdfc gl: update sys dependants for function removals
4315a4b54d forgot to change the androidmedia/videotoolbox/caopengllayer
sources as required.
2017-01-13 13:01:28 +11:00
Sebastian Dröge
5619690ae1 androidmedia: Add support for VP9 2017-01-04 20:59:39 +02:00
Matthew Waters
41a6448918 gl: GST_GL_TYPE -> GST_TYPE_GL
Some deprecated symbols are kept for backwards compatibility
2016-11-03 16:16:12 +11:00
Thibault Saunier
2fb716409c Use the new API to post flow ERROR messages on the bus
https://bugzilla.gnome.org/show_bug.cgi?id=770158
2016-08-26 19:23:31 -03:00
Matthew Waters
989200820d glmemory: add the texture type to allocate to parameters
Rather than assuming something.  e.g. zerocopy on iOS with GLES3 requires
the use of Luminance/Luminance Alpha formats and does not work with
Red/RG textures.
2016-06-29 18:04:28 +10:00
Sebastian Dröge
610705cbb4 amc: Correctly handle NULL input buffers
https://bugzilla.gnome.org/show_bug.cgi?id=763401
2016-03-11 10:46:10 +02:00
Sebastian Dröge
476d5b6940 amcvideodec: When outputting on a surface, accept all color formats
We don't have to understand them, we handle them as a GL texture.

https://bugzilla.gnome.org/show_bug.cgi?id=762792
2016-02-28 11:10:39 +02:00
Matthew Waters
cd4a93da3b glsyncmeta: separate out gpu/cpu waits.
CPU waits are more expensive and are only required if the CPU is ever going to
access the data. GPU waits perform inter-context synchronisation and are cheaper
as they don't require CPU intervention.
2016-02-09 12:30:25 +11:00
Matthew Waters
9efdfb1a29 amcvideodec: pass the correct time value to wait_for_sync
When we are not waiting, we need to pass -1 to signal that we just want to check
that the frame was/n't rendered.  Avoids waiting for frames that will never be
rendered.

https://bugzilla.gnome.org/show_bug.cgi?id=761014
2016-02-04 15:46:22 +11:00
Matthew Waters
3c29dcaddb amcvideodec: advance the ready counter ourselves when render=FALSE
When not rendering the video frame, e.g. when freeing an unreleased sync frame,
we will not receive a frame listener callback.

Reduces the amount of 'on_frame_available miss detected' messages when dropping
frames.

https://bugzilla.gnome.org/show_bug.cgi?id=761014
2016-02-04 15:46:22 +11:00
Matthew Waters
b276de7827 amcvideodec: guard against not finding a valid frame in loop
Fixes sporadic crashes on finishing decoding a video.

https://bugzilla.gnome.org/show_bug.cgi?id=761014
2016-01-29 23:52:15 +11:00
Matthew Waters
1fceac8122 amcvideodec: set our data pointer in java to NULL on close
The frame available callback can be called after deconfiguring the amc codec.
Guard against this by setting the back pointer to NULL on close() and ignoring
any NULL data pointer.

https://bugzilla.gnome.org/show_bug.cgi?id=761014
2016-01-29 23:52:15 +11:00
Sebastian Dröge
f585a66bd1 amcvideodec: Only free GstAmcBuffer if it's not NULL
And also free it on GL errors.
2016-01-22 17:19:29 +02:00
Sebastian Dröge
5a3744e86c amc: MediaCodec::getOutputBuffer() can return NULL without exception
Happens when doing zerocopy rendering, or when passing a wrong index to it.
Handle this properly for zerocopy rendering, fail properly for the other
cases.

https://bugzilla.gnome.org/show_bug.cgi?id=760961
2016-01-22 16:13:45 +02:00
Matthew Waters
650e5c4fe5 glmemory: implement GstGLBaseMemory::alloc
- Create GstGLVideoAllocationParams which is a GstGLAllocationParams subclass.
- Make it possible to allocate glmemory objects directly if no frills are
  needed.
2015-12-17 13:49:55 +11:00
Matthew Waters
fd3203cfeb amcvideodec: update for GLMemory API changes 2015-12-14 19:15:14 +11:00
Mathias Hasselmann
d387bfd188 androidmedia: Don't add metadata to locked buffer
The video decoders tried calling gst_buffer_add_*meta() on non-writable
buffer resulting in warnings of this kind:

  gstamcvideodec.c:921 (_gl_sync_render_unlocked): WARNING: amcvideodec
  Failed to create the transformation meta for the gl_sync 0xabc03848
  buffer 0xabb01b40 (0)

https://bugzilla.gnome.org/show_bug.cgi?id=758694
2015-11-26 22:24:26 +11:00
Matthew Waters
ab3fd36d64 amcvideodec: move release_output_buffer into the sync meta
Some devices only ever keep one buffer available in the GL queue resulting in
multiple calls to release_output_buffer only causing one frame to be rendered.
If there is a queue after amcvideodec (even playsink's small one), then
multiple buffers are pushed but only a small fraction of them are actually
rendered on time.  The rest will either render some number of frames ahead of
where they are meant to be or timeout waiting for a frame that's already been
rendered.

Solved by moving the release_output_buffer into the sync_meta the is pushed
downstream.  When downstream renders, the custom sync implementation attempts
to release the current buffer (if not already released) and render. Once the
frame has been rendered to the screen, the next frame is released and is
hopefully available by the time the next frame is to be rendered.

This fixes a perceived frame jitter in the output.
2015-11-19 19:25:48 +11:00
Sebastian Dröge
96eb5ef33c amcvideodec: Don't require a non-zero buffer size when doing decoding to a surface
At least on some devices/Android versions the buffer size will always be zero
in these cases.

https://bugzilla.gnome.org/show_bug.cgi?id=758228
2015-11-18 09:23:49 +02:00
Sebastian Dröge
be7f4d6cd6 amcvideodec: Fix indentation 2015-11-17 16:21:10 +02:00
Matthew Waters
3bf7430411 glimagesink: wait on the correct sync meta when rendering 2015-11-17 15:27:26 +11:00
Matthew Waters
f2ca0eaf27 amcviddec: output external-oes textures
This provides a performance and power usage improvement by removing
the texture copy from an OES texture to 2D texture.

The flow is as follows
1. Generate the output buffer with the required sync meta with the incrementing
   push counter and OES GL memory
1.1 release_output_buffer (buf, render=true) and push downstream
2. Downstream waits for on the sync meta (timed wait) or drops the frame (no wait)
2.1 Timed wait for the frame number to reach the number of frame callbacks fired
2.2 Unconditionally update the image when the wait completes (success or fail).
    Sets the affine transformation matrix meta on the buffer.
3. Downstream renders as usual.

At *some* point through this the on_frame_callback may or may not fire.  If it
does fire, we can finish waiting early and render. Otherwise we have to
wait for a timeout to occur which may cause more buffers to be pused into the
internal GL queue which siginificantly decreases the chances of the
on_frame_callback to fire again.  This is because the frame callback only occurs
when the internal GL queue changes state from empty to non-empty.

Because there is no way to reliably correlate between the number of buffers
pushed and the number of frame callbacks received, there are a number of
workarounds in place.
1. We self-increment the ready counter when it falls behind the push counter
2. Time based waits as the frame callback may not be fired for a certain frame.
3. It is assumed that the device can render at speed or performs some QoS of
   the interal GL queue (which may not match the GStreamer QoS).

It holds that we call SurfaceTexture::updateTexImage for each buffer pushed
downstream however there's no guarentee that updateTexImage will result in
the exact next frame (it could skip or duplicate) so synchronization is not
guaranteed to be accurate although it seems to be close enough to be unable
to discern visually.  This has not changed from before this patch.  The current
requirement for synchronization is that updateTexImage is called at the point in
time when the buffers is to be rendered.

https://bugzilla.gnome.org/show_bug.cgi?id=757285
2015-11-17 15:27:26 +11:00
Luis de Bethencourt
1ba4d63d6a amcvideodec: use GST_STIME_ARGS for GstClockTimeDiff
No need to manually handle negative value of deadline, GST_STIME_ARGS does
exactly this.
2015-11-03 15:00:28 +00:00
Matthew Waters
c6f2426a7a androidmedia: fix build errors
printf %lld instead of G_GINT64_FORMAT.
Double ret variable declaration.
2015-10-21 05:33:13 +11:00
Matthew Waters
c322806227 amcviddec: use gstcontext to retreive the OpenGL context 2015-10-21 04:27:43 +11:00
Matthieu Bouron
7dbb6681a3 androidmedia: Only allow GL output if the decoder has unknown color formats
If GST_AMC_IGNORE_UNKNOWN_COLOR_FORMATS is set to yes, non-GL output
is still allowed.

https://bugzilla.gnome.org/show_bug.cgi?id=731204
2015-10-21 04:27:43 +11:00
Matthieu Bouron
43b63f304d androidmedia: Add support for GL output in amcvideodec
https://bugzilla.gnome.org/show_bug.cgi?id=731204
2015-10-21 04:27:43 +11:00
Matthieu Bouron
45e287840d androidmedia: Do not flush codec if it is not started 2015-10-21 04:27:43 +11:00
Matthieu Bouron
4ab1e66b2e androidmedia: Improve color format debug messages 2015-10-21 04:27:43 +11:00
Sebastian Dröge
f441b9d5d2 amcvideodec: Properly forward the return value of gst_video_decoder_negotiate()
https://bugzilla.gnome.org/show_bug.cgi?id=756578
2015-10-16 09:12:35 +03:00
Sebastian Dröge
b48524c357 amcvideodec: Implement support for COLOR_QCOM_FormatYVU420SemiPlanar32mMultiView for decoding
https://bugzilla.gnome.org/show_bug.cgi?id=756578
2015-10-16 09:12:35 +03:00
Sebastian Dröge
42a1a95f3e androidmedia: Add support for H265/HEVC 2015-07-08 11:42:48 +03:00
Sebastian Dröge
a6fb482247 androidmedia: Conditionally use get_{input,output}_buffer() Android 21 APIs
Also properly set limit/position on byte buffer, some codecs prefer to have
correct values there.
2015-05-31 21:27:27 +02:00
Sebastian Dröge
774866ad62 androidmedia: Handle queue/release errors non-fatal
They can happen sometimes as a transient problem, e.g. if there
is a problem in the stream. Only handle them as fatal if many of
them happen in a row.
2015-04-17 20:11:46 +02:00
Anuj Jaiswal
a3958ae65b androidmedia: break statement unnecessary
Code flow never arrives to the break statements due to the goto's. So the
breaks are unecessary.

https://bugzilla.gnome.org/show_bug.cgi?id=736942
2014-09-24 12:12:16 +01:00
Sebastian Dröge
8f25220b9c androidmedia: Make sure to unblock any thread waiting on the drain condition variable when errors happen 2014-08-14 15:27:21 +03:00
Sebastian Dröge
ca62186797 androidmedia: Fix draining logic to let the base class handle EOS events
https://bugzilla.gnome.org//show_bug.cgi?id=734775
2014-08-14 15:27:21 +03:00
Sebastian Dröge
aee9d12b86 androidmedia: Also don't report warnings for queue/dequeue errors during flushing in error cases
... and fix a case where we released an invalid buffer index.
2014-06-19 18:34:05 +02:00
Sebastian Dröge
880f3d8a18 androidmedia: Don't consider input buffer queueing failures as an error during flushing 2014-06-19 18:34:05 +02:00
Sebastian Dröge
71c0e0e19f androidmedia: Clean up flushing code and don't consider output buffer releasing failures during flushing as an error 2014-06-19 18:34:05 +02:00
Sebastian Dröge
f844af2320 androidmedia: Add exceptions from the Java API to error messages, and post more error/warning messages overall 2014-05-30 16:36:11 +02:00
Jorge Zapata
ad82575693 Release the codec before freeing it
On Samsung Galaxy S4 it is impossible to have more than one
hardware decoder at the same time. If we do not release it
explicitly the GC only releases it whenever the whole application
is finished not whenever the activity is finished and thus a player
will not be able to work correctly
2014-05-30 12:40:59 +02:00
Chen Jie
8e0bbc9e32 androidmedia: add gst_amc_color_format_copy
gst_amc_color_format_copy will copy in/out a frame resides at a
GstAmcBuffer. Lots of codes in gst_amc_video_*_fill_buffer are moved to
this new function.
2014-05-26 16:29:01 +02:00