Commit graph

359 commits

Author SHA1 Message Date
Matthew Waters
9a325f4a09 vtdec: add support for zerocopy decoding with GLES 3.x
It seems that Red/RG textures aren't supported anymore for zerocopy decoding
with GLES 3.x.  Instead use Luminance/Luminance Alpha textures which are.
2016-06-29 18:04:28 +10:00
Matthew Waters
989200820d glmemory: add the texture type to allocate to parameters
Rather than assuming something.  e.g. zerocopy on iOS with GLES3 requires
the use of Luminance/Luminance Alpha formats and does not work with
Red/RG textures.
2016-06-29 18:04:28 +10:00
Alessandro Decina
9a026799f0 vtdec: minimize renegotiation to make hw decoding more reliable
The hardware decoder can become (temporarily) unavailable across
VTDecompressionSessionCreate/Destroy calls. During negotiation if the currently
configured caps are still accepted by downstream we keep using them so we don't
have to destroy and recreate the decoding session.

This indirectly fixes https://bugzilla.gnome.org/show_bug.cgi?id=767429, by
making vtdec stick to GLMemory.
2016-06-23 11:45:27 +10:00
Sebastian Dröge
a913a0b967 avfassetsrc: Don't escape the URI before passing it to NSURL
The URI must already be escaped by the caller, we don't support passing around
invalid (unescaped) URIs via the GstURIHandler interface.

Also it will escape too much of the URI in this case, e.g.
  ipod-library://item/item.m4a?id=3143338395173862951
becomes
  ipod-library://item/item.m4a%3Fid%3D3143338395173862951

https://bugzilla.gnome.org/show_bug.cgi?id=767492
2016-06-13 09:21:50 +03:00
Sebastian Dröge
0275007107 iosassetsrc: Print the correct URI if it was rejected
We were printing the previously set URI instead of the new one.
2016-06-13 09:21:50 +03:00
Alessandro Decina
a037f13271 vtdec: always drain in ::negotiate
Move calling gst_vtdec_push_frames_if_needed from ::set_format to ::negotiate so
that we always drain even when renegotiation is triggered by downstream.
2016-06-07 17:22:01 +10:00
Alessandro Decina
7fea17a476 vtdec: try to preserve downstream caps order
vtdec specifies sysmem; GLMemory as template caps. When negotiating, we used to
call gst_pad_peer_query_caps (..., filter) with our template caps as filter. The
query does gst_caps_intersect (filter, peercaps) internally which gives
precedence to the order of the filter caps. While we want to output sysmem by
default, when negotiating with glimagesink which returns GLMemory; sysmem; we
do want to do GL, so we now query using a NULL filter and intersect the result
with our template caps giving precedence to downstream's caps.

tl;dr: make sure we end up negotiating GLMemory with glimagesink
2016-06-07 17:13:12 +10:00
Edward Hervey
09e3bd0fb3 applemedia: Only use the OpenGL framework on OSX
It's not available on ios (uses OpenGLES already)

https://bugzilla.gnome.org/show_bug.cgi?id=766973
2016-06-03 07:11:55 +02:00
Havard Graff
f615a84646 applemedia: CGLTexImageIOSurface2D needs the OpenGL framework on OSX
https://bugzilla.gnome.org/show_bug.cgi?id=766973
2016-06-02 22:42:22 +01:00
Havard Graff
ba06fc96b7 avsamplevideosink: check we are compiling for 10.1 up to 10.4
This API was deprecated in 10.4, so don't use it for 10.5 and onwards.

https://bugzilla.gnome.org/show_bug.cgi?id=766973
2016-06-02 22:41:54 +01:00
Heinrich Fink
ab70d79c43 applemedia: vtenc: Register a hardware-only vtenc_h264_hw element on OSX
Similar to vtdec_hw, this commit adds a vtenc_h264_hw element that fails
caps negotiation unless a hardware encoder could actually be acquired.
This is useful in situations where a fallback to a software encoder
other than the vtenc_h264 software encoder is desired (e.g. to x264enc).

https://bugzilla.gnome.org/show_bug.cgi?id=767104
2016-06-02 11:22:09 +03:00
Alessandro Decina
3d8d60baa8 vtdec: make vtdec_hw fallback to software on renegotiation
When renegotiating mid stream - for example with variable bitrate
streams - and therefore destroying and recreating VTSessions, the
hw decoder might become temporarily unavailable.

To deal with this and avoid erroring out on bitrate changes,
vtdec_hw now falls back to using the software decoder if the hw
one was available at some point but isn't anymore. At
renegotiation/bitrate change time, it will still retry to open
the hardware one.
2016-06-02 16:30:02 +10:00
Alessandro Decina
4a83686a57 vtdec: fix switching from GLMemory to Sysmem
When renegotiating from GLMemory to Sysmem do teardown the texture_cache.

Fixes: https://bugzilla.gnome.org/show_bug.cgi?id=766190
2016-06-02 13:15:05 +10:00
Alessandro Decina
bf444301f9 vtdec: optimize renegotiation
::negotiate can be called several times before the CAPS event is sent downstream
so use the currently configured output state caps instead of the pad current
caps when deciding whether to recreate the VTSession or not.

This leads to creating/destroying less VTSessions which makes renegotiation more
reliable especially when using hw decoding.
2016-06-02 13:15:05 +10:00
Philippe Normand
c78819b26a applemedia: fix core_video_meta init registration 2016-05-10 19:22:33 +03:00
Sebastian Dröge
43487482e5 applemedia: Ship all headers in tarballs
Some were missing from noinst_HEADERS.

https://bugzilla.gnome.org/show_bug.cgi?id=766163
2016-05-09 12:50:47 +03:00
Alessandro Decina
5228238715 applemedia: vtdec: output sysmem by default 2016-05-04 11:33:16 +10:00
Hyunjun Ko
45b258127c applemedia: vtenc: fix build break on iOS
Fix 'conflicting types' error
https://bugzilla.gnome.org/show_bug.cgi?id=765292
2016-04-21 09:28:01 +03:00
Hyunjun Ko
5023a7c85d applemedia: vtenc: Fix a warning by weak-link symbol
Fix this warning
address of function 'VTCompressionSessionPrepareToEncodeFrames' will always evaluate to 'true' [-Wpointer-bool-conversion]

https://bugzilla.gnome.org/show_bug.cgi?id=765292
2016-04-20 17:16:02 +03:00
Sebastian Dröge
4ad32506c6 applemedia: GstCoreVideoMeta in the plugin using only public API has no ctx member 2016-04-13 13:10:12 +03:00
Sebastian Dröge
d00c458592 applemedia: GstCoreMediaMeta in the plugin using only public API has no ctx member 2016-04-13 13:09:05 +03:00
Sebastian Dröge
ccc068576a meta: Initialize all GstMeta fields
During allocation they are not initialized to all zeroes.

https://bugzilla.gnome.org/show_bug.cgi?id=764902
2016-04-13 10:17:24 +03:00
Vineeth TM
8cdfb13658 bad: use new gst_element_class_add_static_pad_template()
https://bugzilla.gnome.org/show_bug.cgi?id=763081
2016-03-24 14:56:51 +02:00
Sebastian Dröge
924795c85c vtdec: Negotiate the decoder in set_format() already
Don't wait until later, we want to know here if the codec can be opened or not
for the requested format. This was removed (accidentially?) by
  119e09eac3

Without this decodebin has no way to switch to a different decoder if this one
does not work.

https://bugzilla.gnome.org/show_bug.cgi?id=762613
2016-02-25 11:38:07 +02:00
Joe Gorse
2b63a88f26 avfvideosrc: Frame durations as CTime to the API, not double
Newer iOS seems to automatically convert, older iOS/OSX just crashes.

https://bugzilla.gnome.org/show_bug.cgi?id=762575
2016-02-25 11:34:40 +02:00
Alessandro Decina
4aaaf23868 applemedia: videotesturecache: let CVOpenGLESTextureCache flush every 1s
Leave kCVOpenGLESTextureCacheMaximumTextureAgeKey to the default (1s). We used
to set it to 0 and flush manually, but apparently (looking at the GLES profiler)
0 means "disable the cache entirely".
2016-02-10 15:40:02 +11:00
Alessandro Decina
99f29ec56e applemedia: vtdec: implement GstElement::set_context 2016-02-09 14:13:26 +11:00
Alessandro Decina
915124b363 applemedia: vtdec: update copyright 2016-02-09 14:13:26 +11:00
Alessandro Decina
e34fe9fe70 applemedia: avfvideosrc: implement GstElement::set_context
Needed to properly handle gst.gl.app_context(s).
2016-02-09 14:13:26 +11:00
Alessandro Decina
022e060563 applemedia: avfvideosrc: add myself to the copyright holders 2016-02-09 14:13:26 +11:00
Alessandro Decina
045a935871 applemedia: refactor GL context code
Rework the GL context code. Now both avfvideosrc and vtdec can create an
internal GL context for pushing textures. Both elements will still try to
use/switch to a local context where available (including after RECONFIGURE
events).
2016-01-29 15:10:32 +11:00
Alessandro Decina
6f80d5c59a applemedia: avfvideosrc: actually set the configured framerate
Actually set the configured framerate. Before we only used to set the first
matching framerate range. On iOS where the camera reports ranges [2, 60], we
used to configure the camera to output anything between 2 and 60fps.
2016-01-27 14:38:36 +11:00
Alessandro Decina
c2a57f3555 applemedia: texture cache: copy the input metas
Copy the input metas so avfvideosrc and vtenc can fast path to using
CVPixelBuffer(s) even when using GLMemory.
2016-01-22 13:53:32 +11:00
Alessandro Decina
5c7fb644cd applemedia: vtenc: fix build on iOS 2016-01-19 19:18:43 +11:00
Alessandro Decina
f6154b4cc0 applemedia: always fill GstBuffers with GstMemory
Always fill buffers with our custom memory. The custom memory will avoid
mapping CV/CM buffers unless necessary.
2016-01-19 16:13:04 +11:00
Ilya Konstantinov
8577224c74 applemedia: implement copying of meta
Before this, buffers would lose their Core Video / Core Media meta
over intervideo* boundary.

https://bugzilla.gnome.org/show_bug.cgi?id=747216
2016-01-19 13:06:31 +11:00
Ilya Konstantinov
936b2fdfbc applemedia: implement GstAppleCoreVideoMemory
Implement a new memory type wrapping CVPixelBuffer.

There are two immediate advantages:
 a) Make the GstMemory itself retain the CVPixelBuffer. Previously,
    the containing GstBuffer was solely responsible for the lifetime of
    the backing CVPixelBuffer.

    With this change, we remove the GST_MEMORY_FLAG_NO_SHARE so that
    GstMemory objects be referenced by multiple GstBuffers (doing away
    with the need to copy.)

  b) Delay locking CVPixelBuffer into CPU memory until it's actually
     mapped -- possibly never.

The CVPixelBuffer object is shared among references, shares and
(in planar formats) planes, so a wrapper GstAppleCoreVideoPixelBuffer
structure was introduced to manage locking.

https://bugzilla.gnome.org/show_bug.cgi?id=747216
2016-01-19 12:58:43 +11:00
Alessandro Decina
ed7df2da3c applemedia: iosurfacememory: alloc with g_new0
...since the base class doesn't use g_slice anymore
2016-01-19 12:56:40 +11:00
Alessandro Decina
05ccc2f6ea applemedia: avfvideosrc: optionally get the GL context from gst_element_set_context
Allow setting the GL context with gst_element_set_context. When available, the
local context obtained via the context query still has the precedence.
2016-01-12 14:36:02 +11:00
Alessandro Decina
45b40b8467 applemedia: iosurfacememory: remove NO_SHARE flag
The full memory can be safely shared. Sharing a sub region can't be done and
will fail in the base allocator, triggering a copy.
2016-01-12 14:05:01 +11:00
Alessandro Decina
b0f5d4f1d5 applemedia: avfvideosrc: enable renegotiation
We can actually renegotiate now, so remove old check which disabled it.
2016-01-06 16:51:12 +11:00
Alessandro Decina
85df5648bb applemedia: avfvideosrc: small negotiation fix
Fix negotiation when GLMemory is requested but a context is not available
2016-01-06 13:20:17 +11:00
Alessandro Decina
e0d65937bc applemedia: vtenc: accept UYVY on Mac
When doing GLMemory avfvideosrc negotiates UYVY. This change allows avfvideosrc
! tee name=t ! ... ! glimagesink t. ! ... ! gldownload ! vtenc_h264 ! ...
to do GLMemory and 0-copy with the encoder (with the CV meta).
2016-01-06 11:43:37 +11:00
Alessandro Decina
a15d8acff0 applemedia: avfvideosrc: change texture format from BGRA to NV12 on iOS
Change texture format from BGRA to NV12. This allows a pipeline like avfvideosrc
! tee name=t ! ... ! glimagesink t. ! ... ! gldownload ! vtenc_h264 ! ...  to
negotiate GLMemory. This makes the glimagesink branch much faster (obviously)
and triggers the 0-copy path between avfvideosrc and vtenc (using the CV meta).
Combined this results in a huge perf improvement on iOS (25-30% of CPU time in a
pipeline like the one above).

Note that this doesn't introduce a new shader conversion in the sink, since BGRA
textures had to be copied/converted from format=BGRA,texture-target=RECTANGLE to
format=RGBA,texture-target=2D anyway.
2016-01-06 11:43:37 +11:00
Alessandro Decina
bf86d7fef1 applemedia: avfvideosrc: output at highest fps/resolution by default
Fixate to the highest possible resolution and fps. Otherwise by default we end
up fixating at 2fps and the lowest supported resolution, which is hardly what
someone who bought an overpriced smartphone wants.
2015-12-29 10:57:36 +11:00
Julien Isorce
89229bcfed applemedia: set -mmacosx-version-min to 10.8
Otherwise qtkitvideosrc fails to build on OSX 10.10.4
because QTKit has been deprecated since OS X 10.9.

Also set -mmacosx-version-min=10.8 in front to allow
the user or cerbero to override the version.

https://bugzilla.gnome.org/show_bug.cgi?id=745564
2015-12-18 10:30:25 +00:00
Alessandro Decina
6961945110 glmemory: add gst_gl_memory_allocator_get_default
Add gst_gl_memory_allocator_get_default to get the default allocator based on
the opengl version. Allows us to stop hardcoding the PBO allocator which isn't
supported on gles2.

Fixes GL upload on iOS9 among other things.
2015-12-18 14:25:32 +11:00
Matthew Waters
5dcd500257 gl*memory*: reverse the parameter order of user_data and destroy notify
The convention is to have the destroy notify last after any user data
2015-12-17 15:44:22 +11:00
Alessandro Decina
491e2eefcb applemedia: fix build on iOS
Fix build after 779dc3132c
2015-12-17 15:07:33 +11:00
Alessandro Decina
4f550ff50f applemedia: avfvideosrc: do fixate 2015-12-17 14:49:13 +11:00