Commit graph

98 commits

Author SHA1 Message Date
Sebastian Dröge
476d5b6940 amcvideodec: When outputting on a surface, accept all color formats
We don't have to understand them, we handle them as a GL texture.

https://bugzilla.gnome.org/show_bug.cgi?id=762792
2016-02-28 11:10:39 +02:00
Matthew Waters
cd4a93da3b glsyncmeta: separate out gpu/cpu waits.
CPU waits are more expensive and are only required if the CPU is ever going to
access the data. GPU waits perform inter-context synchronisation and are cheaper
as they don't require CPU intervention.
2016-02-09 12:30:25 +11:00
Matthew Waters
9efdfb1a29 amcvideodec: pass the correct time value to wait_for_sync
When we are not waiting, we need to pass -1 to signal that we just want to check
that the frame was/n't rendered.  Avoids waiting for frames that will never be
rendered.

https://bugzilla.gnome.org/show_bug.cgi?id=761014
2016-02-04 15:46:22 +11:00
Matthew Waters
3c29dcaddb amcvideodec: advance the ready counter ourselves when render=FALSE
When not rendering the video frame, e.g. when freeing an unreleased sync frame,
we will not receive a frame listener callback.

Reduces the amount of 'on_frame_available miss detected' messages when dropping
frames.

https://bugzilla.gnome.org/show_bug.cgi?id=761014
2016-02-04 15:46:22 +11:00
Matthew Waters
b276de7827 amcvideodec: guard against not finding a valid frame in loop
Fixes sporadic crashes on finishing decoding a video.

https://bugzilla.gnome.org/show_bug.cgi?id=761014
2016-01-29 23:52:15 +11:00
Matthew Waters
1fceac8122 amcvideodec: set our data pointer in java to NULL on close
The frame available callback can be called after deconfiguring the amc codec.
Guard against this by setting the back pointer to NULL on close() and ignoring
any NULL data pointer.

https://bugzilla.gnome.org/show_bug.cgi?id=761014
2016-01-29 23:52:15 +11:00
Sebastian Dröge
f585a66bd1 amcvideodec: Only free GstAmcBuffer if it's not NULL
And also free it on GL errors.
2016-01-22 17:19:29 +02:00
Sebastian Dröge
5a3744e86c amc: MediaCodec::getOutputBuffer() can return NULL without exception
Happens when doing zerocopy rendering, or when passing a wrong index to it.
Handle this properly for zerocopy rendering, fail properly for the other
cases.

https://bugzilla.gnome.org/show_bug.cgi?id=760961
2016-01-22 16:13:45 +02:00
Matthew Waters
650e5c4fe5 glmemory: implement GstGLBaseMemory::alloc
- Create GstGLVideoAllocationParams which is a GstGLAllocationParams subclass.
- Make it possible to allocate glmemory objects directly if no frills are
  needed.
2015-12-17 13:49:55 +11:00
Matthew Waters
fd3203cfeb amcvideodec: update for GLMemory API changes 2015-12-14 19:15:14 +11:00
Mathias Hasselmann
d387bfd188 androidmedia: Don't add metadata to locked buffer
The video decoders tried calling gst_buffer_add_*meta() on non-writable
buffer resulting in warnings of this kind:

  gstamcvideodec.c:921 (_gl_sync_render_unlocked): WARNING: amcvideodec
  Failed to create the transformation meta for the gl_sync 0xabc03848
  buffer 0xabb01b40 (0)

https://bugzilla.gnome.org/show_bug.cgi?id=758694
2015-11-26 22:24:26 +11:00
Matthew Waters
ab3fd36d64 amcvideodec: move release_output_buffer into the sync meta
Some devices only ever keep one buffer available in the GL queue resulting in
multiple calls to release_output_buffer only causing one frame to be rendered.
If there is a queue after amcvideodec (even playsink's small one), then
multiple buffers are pushed but only a small fraction of them are actually
rendered on time.  The rest will either render some number of frames ahead of
where they are meant to be or timeout waiting for a frame that's already been
rendered.

Solved by moving the release_output_buffer into the sync_meta the is pushed
downstream.  When downstream renders, the custom sync implementation attempts
to release the current buffer (if not already released) and render. Once the
frame has been rendered to the screen, the next frame is released and is
hopefully available by the time the next frame is to be rendered.

This fixes a perceived frame jitter in the output.
2015-11-19 19:25:48 +11:00
Sebastian Dröge
96eb5ef33c amcvideodec: Don't require a non-zero buffer size when doing decoding to a surface
At least on some devices/Android versions the buffer size will always be zero
in these cases.

https://bugzilla.gnome.org/show_bug.cgi?id=758228
2015-11-18 09:23:49 +02:00
Sebastian Dröge
be7f4d6cd6 amcvideodec: Fix indentation 2015-11-17 16:21:10 +02:00
Matthew Waters
3bf7430411 glimagesink: wait on the correct sync meta when rendering 2015-11-17 15:27:26 +11:00
Matthew Waters
f2ca0eaf27 amcviddec: output external-oes textures
This provides a performance and power usage improvement by removing
the texture copy from an OES texture to 2D texture.

The flow is as follows
1. Generate the output buffer with the required sync meta with the incrementing
   push counter and OES GL memory
1.1 release_output_buffer (buf, render=true) and push downstream
2. Downstream waits for on the sync meta (timed wait) or drops the frame (no wait)
2.1 Timed wait for the frame number to reach the number of frame callbacks fired
2.2 Unconditionally update the image when the wait completes (success or fail).
    Sets the affine transformation matrix meta on the buffer.
3. Downstream renders as usual.

At *some* point through this the on_frame_callback may or may not fire.  If it
does fire, we can finish waiting early and render. Otherwise we have to
wait for a timeout to occur which may cause more buffers to be pused into the
internal GL queue which siginificantly decreases the chances of the
on_frame_callback to fire again.  This is because the frame callback only occurs
when the internal GL queue changes state from empty to non-empty.

Because there is no way to reliably correlate between the number of buffers
pushed and the number of frame callbacks received, there are a number of
workarounds in place.
1. We self-increment the ready counter when it falls behind the push counter
2. Time based waits as the frame callback may not be fired for a certain frame.
3. It is assumed that the device can render at speed or performs some QoS of
   the interal GL queue (which may not match the GStreamer QoS).

It holds that we call SurfaceTexture::updateTexImage for each buffer pushed
downstream however there's no guarentee that updateTexImage will result in
the exact next frame (it could skip or duplicate) so synchronization is not
guaranteed to be accurate although it seems to be close enough to be unable
to discern visually.  This has not changed from before this patch.  The current
requirement for synchronization is that updateTexImage is called at the point in
time when the buffers is to be rendered.

https://bugzilla.gnome.org/show_bug.cgi?id=757285
2015-11-17 15:27:26 +11:00
Luis de Bethencourt
1ba4d63d6a amcvideodec: use GST_STIME_ARGS for GstClockTimeDiff
No need to manually handle negative value of deadline, GST_STIME_ARGS does
exactly this.
2015-11-03 15:00:28 +00:00
Matthew Waters
c6f2426a7a androidmedia: fix build errors
printf %lld instead of G_GINT64_FORMAT.
Double ret variable declaration.
2015-10-21 05:33:13 +11:00
Matthew Waters
c322806227 amcviddec: use gstcontext to retreive the OpenGL context 2015-10-21 04:27:43 +11:00
Matthieu Bouron
7dbb6681a3 androidmedia: Only allow GL output if the decoder has unknown color formats
If GST_AMC_IGNORE_UNKNOWN_COLOR_FORMATS is set to yes, non-GL output
is still allowed.

https://bugzilla.gnome.org/show_bug.cgi?id=731204
2015-10-21 04:27:43 +11:00
Matthieu Bouron
43b63f304d androidmedia: Add support for GL output in amcvideodec
https://bugzilla.gnome.org/show_bug.cgi?id=731204
2015-10-21 04:27:43 +11:00
Matthieu Bouron
45e287840d androidmedia: Do not flush codec if it is not started 2015-10-21 04:27:43 +11:00
Matthieu Bouron
4ab1e66b2e androidmedia: Improve color format debug messages 2015-10-21 04:27:43 +11:00
Sebastian Dröge
f441b9d5d2 amcvideodec: Properly forward the return value of gst_video_decoder_negotiate()
https://bugzilla.gnome.org/show_bug.cgi?id=756578
2015-10-16 09:12:35 +03:00
Sebastian Dröge
b48524c357 amcvideodec: Implement support for COLOR_QCOM_FormatYVU420SemiPlanar32mMultiView for decoding
https://bugzilla.gnome.org/show_bug.cgi?id=756578
2015-10-16 09:12:35 +03:00
Sebastian Dröge
42a1a95f3e androidmedia: Add support for H265/HEVC 2015-07-08 11:42:48 +03:00
Sebastian Dröge
a6fb482247 androidmedia: Conditionally use get_{input,output}_buffer() Android 21 APIs
Also properly set limit/position on byte buffer, some codecs prefer to have
correct values there.
2015-05-31 21:27:27 +02:00
Sebastian Dröge
774866ad62 androidmedia: Handle queue/release errors non-fatal
They can happen sometimes as a transient problem, e.g. if there
is a problem in the stream. Only handle them as fatal if many of
them happen in a row.
2015-04-17 20:11:46 +02:00
Anuj Jaiswal
a3958ae65b androidmedia: break statement unnecessary
Code flow never arrives to the break statements due to the goto's. So the
breaks are unecessary.

https://bugzilla.gnome.org/show_bug.cgi?id=736942
2014-09-24 12:12:16 +01:00
Sebastian Dröge
8f25220b9c androidmedia: Make sure to unblock any thread waiting on the drain condition variable when errors happen 2014-08-14 15:27:21 +03:00
Sebastian Dröge
ca62186797 androidmedia: Fix draining logic to let the base class handle EOS events
https://bugzilla.gnome.org//show_bug.cgi?id=734775
2014-08-14 15:27:21 +03:00
Sebastian Dröge
aee9d12b86 androidmedia: Also don't report warnings for queue/dequeue errors during flushing in error cases
... and fix a case where we released an invalid buffer index.
2014-06-19 18:34:05 +02:00
Sebastian Dröge
880f3d8a18 androidmedia: Don't consider input buffer queueing failures as an error during flushing 2014-06-19 18:34:05 +02:00
Sebastian Dröge
71c0e0e19f androidmedia: Clean up flushing code and don't consider output buffer releasing failures during flushing as an error 2014-06-19 18:34:05 +02:00
Sebastian Dröge
f844af2320 androidmedia: Add exceptions from the Java API to error messages, and post more error/warning messages overall 2014-05-30 16:36:11 +02:00
Jorge Zapata
ad82575693 Release the codec before freeing it
On Samsung Galaxy S4 it is impossible to have more than one
hardware decoder at the same time. If we do not release it
explicitly the GC only releases it whenever the whole application
is finished not whenever the activity is finished and thus a player
will not be able to work correctly
2014-05-30 12:40:59 +02:00
Chen Jie
8e0bbc9e32 androidmedia: add gst_amc_color_format_copy
gst_amc_color_format_copy will copy in/out a frame resides at a
GstAmcBuffer. Lots of codes in gst_amc_video_*_fill_buffer are moved to
this new function.
2014-05-26 16:29:01 +02:00
Chen Jie
557d6b974f androidmedia: move create_src|sink_caps to gstamc.c
Some hack logic needs also to be present in create_src|sink_caps, for
working around some broken codecs. These hacks are hidden
in color_format/video_format conversion -- the prototypes of these
functions are also changed to include more args for hack judgement.

Also in case of multi-color_formats mapped to one video_format, then
map that video_format back will not give the original color_format, which
causes gst_amc_codec_configure failed with something like
'does not support color format N'.

The new prototype involves with GstAmcCodecInfo and mime, which
ensures the converted color_format is supported by the codec.

A COLOR_FormatYCbYCr to GST_VIDEO_FORMAT_YUY2 mapping is also added, in
order to work around bugs in OMX.k3.video.decoder.avc(which incorrectly
reports supporting COLOR_FormatYCbYCr, which is actually
COLOR_FormatYUV420SemiPlanar). There are already hacks for this in
gst_amc_video_format_to_color_format, gst_amc_color_format_to_video_format
and gst_amc_color_format_info_set, but the codec will still not work(be
ignored because of "has unknown color formats") without adding this mapping.
2014-05-26 16:29:01 +02:00
Sebastian Dröge
cea218db6f amcvideodec: Store return value of gst_caps_merge_structure()
The inputs are both invalid afterwards.
2014-02-14 12:51:50 +01:00
Sebastian Dröge
faf48d26bf amcvideodec: Release buffer to Android if allocating output failed 2014-02-14 12:51:50 +01:00
Sebastian Dröge
30d80bf658 androidmedia: Don't handle FLUSHING or NOT_LINKED as error
And also just stop the task for FLUSHING.
2014-01-25 17:41:18 +01:00
Jorge Zapata
c9abe8c50d amcvideodec: Add the divx variant for mpeg4 video
Looks like all the mpeg4 video decoders also accept the divx
variants. So we better add those caps too

https://bugzilla.gnome.org/show_bug.cgi?id=711163
2014-01-14 20:49:24 +01:00
Sebastian Dröge
219275dbb6 amcvideodec: Use new gst_video_decoder_set_needs_format() API 2013-12-05 11:50:17 +01:00
Andoni Morales Alastruey
6b49683447 androidmedia: fix access to invalid buffers in the decoding loop
Flushing the decoder invalidates all buffers, so it should be done
after quiting the decoding loop. Otherwise we can jump into
"failed_release" and stop everything

https://bugzilla.gnome.org/show_bug.cgi?id=711156
2013-10-30 16:24:17 +01:00
Sebastian Dröge
2b94641a42 amcvideodec: Don't put the level restrictions on the sinkpad caps
They tend to be inaccurate and having them in the sinkpad caps
prevents playback of files that would otherwise play fine.
2013-09-12 13:23:28 +02:00
Sebastian Dröge
d065c09684 amcvideodec: Port tiled NV12 conversion to 1.0 2013-09-03 11:06:00 +02:00
Sebastian Dröge
c4e742f428 amcvideodec: Add FIXME comment 2013-09-03 11:06:00 +02:00
Jorge Luis Zapata
48484f04a2 androidmedia: add support for a new qualcomm colorspace 2013-09-03 11:06:00 +02:00
Sebastian Dröge
2e8af6973f ext: Use new flush vfunc of video codec base classes and remove reset implementations 2013-08-15 15:46:58 +02:00
Xavi Artigas
85c1510d6b androidmedia: Fix copying of raw video frames on Samsung Galaxy S3 with Exynos 4 SOC 2013-06-20 09:53:41 +02:00