Commit graph

97 commits

Author SHA1 Message Date
Alessandro Decina
7898bc5810 applemedia: changes to make GL memory mappable on CPU on iOS
This commit introduces IOSGLMemory which is a GLMemory that falls back to
GstAppleCoreVideoMemory for CPU access. This is a temporary solution until
IOSurface gets exposed as a public framework on iOS and so we can use
IOSurfaceMemory on both MacOS and iOS.

https://bugzilla.gnome.org/show_bug.cgi?id=769210
2016-08-26 17:44:48 +10:00
Matej Knopp
26c7ca4c24 vtdec: fix MPEG-2 video caps
Add systemstream=false to caps, otherwise the decoder
may be picked for MPEG-PS files. Also parsed=true,
as video toolbox expects entire frame in
VTDecompressionSessionDecodeFrame.

https://bugzilla.gnome.org/show_bug.cgi?id=770049
2016-08-22 20:25:03 +01:00
Alessandro Decina
9a026799f0 vtdec: minimize renegotiation to make hw decoding more reliable
The hardware decoder can become (temporarily) unavailable across
VTDecompressionSessionCreate/Destroy calls. During negotiation if the currently
configured caps are still accepted by downstream we keep using them so we don't
have to destroy and recreate the decoding session.

This indirectly fixes https://bugzilla.gnome.org/show_bug.cgi?id=767429, by
making vtdec stick to GLMemory.
2016-06-23 11:45:27 +10:00
Alessandro Decina
a037f13271 vtdec: always drain in ::negotiate
Move calling gst_vtdec_push_frames_if_needed from ::set_format to ::negotiate so
that we always drain even when renegotiation is triggered by downstream.
2016-06-07 17:22:01 +10:00
Alessandro Decina
7fea17a476 vtdec: try to preserve downstream caps order
vtdec specifies sysmem; GLMemory as template caps. When negotiating, we used to
call gst_pad_peer_query_caps (..., filter) with our template caps as filter. The
query does gst_caps_intersect (filter, peercaps) internally which gives
precedence to the order of the filter caps. While we want to output sysmem by
default, when negotiating with glimagesink which returns GLMemory; sysmem; we
do want to do GL, so we now query using a NULL filter and intersect the result
with our template caps giving precedence to downstream's caps.

tl;dr: make sure we end up negotiating GLMemory with glimagesink
2016-06-07 17:13:12 +10:00
Alessandro Decina
3d8d60baa8 vtdec: make vtdec_hw fallback to software on renegotiation
When renegotiating mid stream - for example with variable bitrate
streams - and therefore destroying and recreating VTSessions, the
hw decoder might become temporarily unavailable.

To deal with this and avoid erroring out on bitrate changes,
vtdec_hw now falls back to using the software decoder if the hw
one was available at some point but isn't anymore. At
renegotiation/bitrate change time, it will still retry to open
the hardware one.
2016-06-02 16:30:02 +10:00
Alessandro Decina
4a83686a57 vtdec: fix switching from GLMemory to Sysmem
When renegotiating from GLMemory to Sysmem do teardown the texture_cache.

Fixes: https://bugzilla.gnome.org/show_bug.cgi?id=766190
2016-06-02 13:15:05 +10:00
Alessandro Decina
bf444301f9 vtdec: optimize renegotiation
::negotiate can be called several times before the CAPS event is sent downstream
so use the currently configured output state caps instead of the pad current
caps when deciding whether to recreate the VTSession or not.

This leads to creating/destroying less VTSessions which makes renegotiation more
reliable especially when using hw decoding.
2016-06-02 13:15:05 +10:00
Alessandro Decina
5228238715 applemedia: vtdec: output sysmem by default 2016-05-04 11:33:16 +10:00
Vineeth TM
8cdfb13658 bad: use new gst_element_class_add_static_pad_template()
https://bugzilla.gnome.org/show_bug.cgi?id=763081
2016-03-24 14:56:51 +02:00
Sebastian Dröge
924795c85c vtdec: Negotiate the decoder in set_format() already
Don't wait until later, we want to know here if the codec can be opened or not
for the requested format. This was removed (accidentially?) by
  119e09eac3

Without this decodebin has no way to switch to a different decoder if this one
does not work.

https://bugzilla.gnome.org/show_bug.cgi?id=762613
2016-02-25 11:38:07 +02:00
Alessandro Decina
99f29ec56e applemedia: vtdec: implement GstElement::set_context 2016-02-09 14:13:26 +11:00
Alessandro Decina
915124b363 applemedia: vtdec: update copyright 2016-02-09 14:13:26 +11:00
Alessandro Decina
045a935871 applemedia: refactor GL context code
Rework the GL context code. Now both avfvideosrc and vtdec can create an
internal GL context for pushing textures. Both elements will still try to
use/switch to a local context where available (including after RECONFIGURE
events).
2016-01-29 15:10:32 +11:00
Alessandro Decina
f6154b4cc0 applemedia: always fill GstBuffers with GstMemory
Always fill buffers with our custom memory. The custom memory will avoid
mapping CV/CM buffers unless necessary.
2016-01-19 16:13:04 +11:00
Alessandro Decina
fe42e302dd applemedia: vtdec: fix texture-target on iOS 2015-12-16 17:03:04 +11:00
Alessandro Decina
89b6c296ad applemedia: vtdec: remove obsolete FIXME 2015-12-16 17:03:04 +11:00
Alessandro Decina
9981cd8327 applemedia: vtdec: remove the internal GstGLColorConvert
It was added back in the day to make texture sharing work by default with
glimagesink inside playbin. These days glimagesink accepts (and converts) YUV
internally so it's no longer needed.
2015-12-16 17:03:03 +11:00
Alessandro Decina
07bf3796aa applemedia: vtdec: decouple outputting textures from outputting RGBA
We're going to be able to output NV12 textures soon.
2015-12-16 17:03:03 +11:00
Alessandro Decina
8ae0033261 applemedia: vtdec: switch to IOSurface on Mac
Switch to using IOSurface instead of CVOpenGLTextureCache on OSX. The latter can't be
used anymore to do YUV => RGB with opengl3 on El Capitan as GL_YCBCR_422_APPLE
has been removed from the opengl3 driver. Also switch to NV12 from UYVY, which
was the only YUV format supported by CVOpenGLTextureCache.
2015-12-16 17:03:03 +11:00
Alessandro Decina
e730b0a8ab applemedia: rename GstCoreVideoTextureCache to GstVideoTextureCache
First of a few commits to stop using CVOpenGLTextureCache on OSX and use
IOSurfaces directly instead. CVOpenGLTextureCache hasn't been updated for OpenGL
3 which is why texture sharing is currently disabled on OSX.
2015-12-16 17:03:03 +11:00
Alessandro Decina
a8b01eb6d1 Revert "vtdec: disable the texture cache on OSX"
This reverts commit f02425c4af.
2015-12-16 17:03:03 +11:00
Vineeth TM
7c42ba97d7 plugins-bad: Fix example pipelines
rename gst-launch --> gst-launch-1.0
replace old elements with new elements(ffmpegcolorspace -> videoconvert, ffenc_** -> avenc_**)
fix caps in examples

https://bugzilla.gnome.org/show_bug.cgi?id=759432
2015-12-15 10:30:49 +00:00
Alessandro Decina
3d5a1ccbf4 applemedia: vtdec: fix negotiation more
Year 12: I still don't understand how negotiation works.

Apparently gst_pad_query_caps doesn't do what I thought it did. To get the
actual caps that can flow through vtdec:src we must call gst_pad_peer_query_caps
with the template caps as filter.

Fixes negotiation with stuff that doesn't understand GLMemory (hello videoscale).
2015-11-17 16:15:53 +11:00
Alessandro Decina
94cd5316f3 applemedia: vtdec: minor texture cache fixes
Small fix on how the texture cache is cleaned up / setup in case of renegotiation
2015-11-17 16:15:53 +11:00
Alessandro Decina
35052fdfd0 applemedia: vtdec: fix setting internal SKIP / DROP flags 2015-11-17 15:19:01 +11:00
Alessandro Decina
119e09eac3 applemedia: vtdec: improve negotiation
Rework negotiation implementing GstVideoDecoder::negotiate. Make it possible to
switch texture sharing on and off at runtime. Useful to (eventually) turn
texture sharing on in pipelines where glimagesink is linked only after
decoding has already started (for example OWR).
2015-11-17 12:59:58 +11:00
Alessandro Decina
8f14882b44 applemedia: vtdec: improve handing of decode errors/frame drops
Improve decode error handling by avoiding calling into GstVideoDecoder from the
VT decode callback. This removes contention on the GST_VIDEO_DECODER_STREAM_LOCK
which used to make the decode callback slow enough for VT to start dropping lots
of frames once the first frame was dropped.
2015-11-17 12:59:58 +11:00
Alessandro Decina
8a171754e5 applemedia: vtdec: fix uninitialized variable warning 2015-11-17 12:59:58 +11:00
Matthew Waters
ccce217502 glcolorconvert: add support for converting texture targets
Solved with a simple shader templating mechanism and string replacements
of the necessary sampler types/texture accesses and texture coordinate
mangling for rectangular and external-oes textures.
2015-10-30 14:24:53 +11:00
Matthew Waters
f02425c4af vtdec: disable the texture cache on OSX
This a workaround until the incompatibility with CVOpenGLTextureCache
and opengl3 contexts is fixed.

https://bugzilla.gnome.org/show_bug.cgi?id=754786
2015-09-18 21:31:15 +10:00
Matthew Waters
578b8432b6 vtdec: gst-indent file 2015-09-18 16:58:21 +10:00
Matthew Waters
3772c8ece6 applemedia/vtdec: fix negotiation errors when resizing glimagesink
GstVideoDecoder has its own logic for detecting when to reconfigure
which ultimately calls decide_allocation and results in a new
texture cache that has not been configured from our reconfigure check.

https://bugzilla.gnome.org/show_bug.cgi?id=755156
2015-09-17 21:14:24 +10:00
Jan Schmidt
0d7d7f6c92 vtdec: Prefer GL output caps.
Prefer GL caps by putting them first in the pad template and
intersecting using GST_CAPS_INTERSECT_FIRST.
2015-09-12 04:33:34 +10:00
Ilya Konstantinov
bfa054a733 vtdec: handle non-consecutive GstBuffer input without copying
CMBlockBuffer offers a model similar to GstBuffer, as it can
consist of multiple non-consecutive memory blocks.

Prior to this change, what we were doing was:

 1) Incorrect:

   CMBlockBufferCreateWithMemoryBlock does not copy the data,
   but we gst_buffer_unmap'd right away.

 2) Inefficient:

   If the GstBuffer consisted of non-contiguous memory blocks,
   gst_buffer_map resulted in malloc / memcpy.

With this change, we construct a CMBlockBuffer out of individual mapped
GstMemory objects. CMBlockBuffer is made to retain the GstMemory
objects (through the use of CMBlockBufferCustomBlockSource), so the
original GstBuffer can be unref'd.

https://bugzilla.gnome.org/show_bug.cgi?id=751241
2015-07-13 11:11:04 -04:00
Ilya Konstantinov
cc3d79f7cc vtdec: fix refcount error in error handling
https://bugzilla.gnome.org/show_bug.cgi?id=751641
2015-07-07 20:18:50 +03:00
Ilya Konstantinov
1f7681dd0a vtdec: don't copy meta from input to output
Copying arbitrary metas is going to cause problems and this should really be
handled by the base class. It overrides most other things already anyway,
including timestamp and duration. Those are just set here now so we can
insert the frame sorted into the queue.

https://bugzilla.gnome.org/show_bug.cgi?id=748922
2015-06-09 16:20:06 +02:00
Nicolas Dufresne
4cca6efda8 vtdec: Require width and height field for H264
This decoder does not work if width and height field are not set
in the sinkpad caps. Let's make this explicit by adding them to
the template caps.

https://bugzilla.gnome.org/show_bug.cgi?id=749655
2015-06-01 11:23:41 -04:00
Sebastian Dröge
7020154f1c vtdec: We shouldn't manually check reconfigure flag on pads but instead implement ::negotiate()
Add FIXME comment related to that.
2015-05-08 15:08:18 +02:00
Alessandro Decina
edf9035d02 applemedia: avoid implicit color conversions
Rework the GL texture code a little to avoid implicit color conversion inside
AVF/VT on both iOS and OSX.
2015-04-20 17:09:58 +10:00
Alessandro Decina
a10232f7da applemedia: vtdec: fix GL negotiation 2015-04-14 20:33:25 +10:00
Sebastian Dröge
160df421ea vtdec: Register a hardware-only vtdec_hw on OSX and give it a higher rank
while having the default vtdec at secondary rank. This allows decodebin/playbin
to prefer the hardware based decoders, and if that fails to initialize because
hardware resources are busy to fall back to e.g. the libav based h264 decoder
instead of the software based vtdec (which is slower), and only fall back to
the software based vtdec if there is no higher ranked decoder available.
2015-03-07 10:31:26 +01:00
Matthew Waters
c873c57430 vtdec: only attempt use of the texture cache with GLMemory caps features
Otherwise we send rectangle textures to glimagesink
2015-02-25 00:03:34 +11:00
Alessandro Decina
052e4804eb vtdec: fix texture sharing on iOS
Move GLES calls to the context thread. Fix
CVOpenGLESTextureCacheCreateTextureFromImage usage on iOS.
2015-02-09 21:18:24 +11:00
Sebastian Dröge
dac3e4ca06 vtdec: Don't dereference NULL frames when draining/flushing 2015-02-03 15:38:40 +01:00
Alessandro Decina
f577b52e5d applemedia: rework GL texture sharing
Use YUV instead of RGB textures, then convert using the new apple specific
shader in GstGLColorConvert. Also use GLMemory directly instead of using the
GL upload meta, avoiding an extra texture copy we used to have before.
2015-01-30 00:34:08 +11:00
Alessandro Decina
77d48abfab applemedia: vtdec: fix CMBlockBufferRef leak
Fix huge leak that went unnoticed for too long. Oops.
2015-01-30 00:34:08 +11:00
Alessandro Decina
e91989136a applemedia: corevideobuffer: make mapping the CVPixelBuffer in CPU optional
Mapping is not necessary and impacts performance when doing texture sharing with
downstream
2015-01-30 00:34:08 +11:00
Alessandro Decina
d95a12e185 vtdec: implement the GL texture upload meta 2015-01-15 14:10:43 +11:00
Alessandro Decina
d0ce9a6f7c vtdec: disable the reorder queue for baseline streams 2014-11-20 16:06:46 +01:00