Commit graph

284 commits

Author SHA1 Message Date
Matthew Waters
34c523c3b9 vtdec: fix compilation error
Undefined symbols for architecture x86_64:
  "__gl_target_to_gst", referenced from:
      __do_get_gl_buffer in libgstapplemedia_la-corevideotexturecache.o
2015-10-30 16:38:17 +11:00
Matthew Waters
ccce217502 glcolorconvert: add support for converting texture targets
Solved with a simple shader templating mechanism and string replacements
of the necessary sampler types/texture accesses and texture coordinate
mangling for rectangular and external-oes textures.
2015-10-30 14:24:53 +11:00
Matthew Waters
e61d504556 glmemory: add support for rectangle textures
Add the various tokens/strings for the differnet texture types (2D, rect, oes)

Changes the GLmemory api to include the GstGLTextureTarget in all relevant
functions.

Update the relevant caps/templates for 2D only textures.
2015-10-30 14:24:53 +11:00
Aleix Conchillo Flaqué
12a2acd7e8 applemedia: dist missing header files
https://bugzilla.gnome.org/show_bug.cgi?id=755852
2015-09-30 09:05:08 +01:00
Matthew Waters
f02425c4af vtdec: disable the texture cache on OSX
This a workaround until the incompatibility with CVOpenGLTextureCache
and opengl3 contexts is fixed.

https://bugzilla.gnome.org/show_bug.cgi?id=754786
2015-09-18 21:31:15 +10:00
Matthew Waters
578b8432b6 vtdec: gst-indent file 2015-09-18 16:58:21 +10:00
Matthew Waters
3772c8ece6 applemedia/vtdec: fix negotiation errors when resizing glimagesink
GstVideoDecoder has its own logic for detecting when to reconfigure
which ultimately calls decide_allocation and results in a new
texture cache that has not been configured from our reconfigure check.

https://bugzilla.gnome.org/show_bug.cgi?id=755156
2015-09-17 21:14:24 +10:00
Jan Schmidt
9f890c5935 vtdec: Use R/RG textures for NV12 when the context provides them
Fixes playback to GL memory on iOS, where the colours are messed
up by passing Luminance/LuminanceAlpha textures where
color convert expects R/RG textures.

https://bugzilla.gnome.org/show_bug.cgi?id=754504
2015-09-12 04:33:34 +10:00
Jan Schmidt
0d7d7f6c92 vtdec: Prefer GL output caps.
Prefer GL caps by putting them first in the pad template and
intersecting using GST_CAPS_INTERSECT_FIRST.
2015-09-12 04:33:34 +10:00
Heinrich Fink
0cc6d16c94 applemedia/avsample: Fix racy cleanup of CA layer
The block that is dispatched async to the main thread assumed the
wrapping GstAvSampleVideoSink to be alive. However, at the time of
the block execution the GstObject instance that is deferenced to access
the CA layer might already be freed, which caused occasional crashes.
Instead, we now only pass the CoreAnimation layer that needs to be
released to the block. We use __block to make sure the block is not
increasing the refcount of the CA layer again on its own.

https://bugzilla.gnome.org/show_bug.cgi?id=753081
2015-07-31 12:59:00 +10:00
Ilya Konstantinov
bfa054a733 vtdec: handle non-consecutive GstBuffer input without copying
CMBlockBuffer offers a model similar to GstBuffer, as it can
consist of multiple non-consecutive memory blocks.

Prior to this change, what we were doing was:

 1) Incorrect:

   CMBlockBufferCreateWithMemoryBlock does not copy the data,
   but we gst_buffer_unmap'd right away.

 2) Inefficient:

   If the GstBuffer consisted of non-contiguous memory blocks,
   gst_buffer_map resulted in malloc / memcpy.

With this change, we construct a CMBlockBuffer out of individual mapped
GstMemory objects. CMBlockBuffer is made to retain the GstMemory
objects (through the use of CMBlockBufferCustomBlockSource), so the
original GstBuffer can be unref'd.

https://bugzilla.gnome.org/show_bug.cgi?id=751241
2015-07-13 11:11:04 -04:00
Matthew Waters
c1906f1cfe applemedia/avsample: fix unconditional use of OSX 10.10 API
We can just not use the error/status properties

https://bugzilla.gnome.org/show_bug.cgi?id=746631
2015-07-13 23:26:14 +10:00
Ilya Konstantinov
cc3d79f7cc vtdec: fix refcount error in error handling
https://bugzilla.gnome.org/show_bug.cgi?id=751641
2015-07-07 20:18:50 +03:00
Luis de Bethencourt
bf2f4f5c8c docs: decodebin2 -> decodebin 2015-06-25 11:02:45 +01:00
Ilya Konstantinov
ad8bf99f04 applemedia: enable sharing of CMBlockBuffer data
Instead of wrapping with GST_MEMORY_FLAG_NO_SHARE, we make the GstMemory
object retain the underlying CMBlockBuffer.

https://bugzilla.gnome.org/show_bug.cgi?id=751072
2015-06-25 10:49:05 +02:00
Ilya Konstantinov
8cd65c3250 applemedia: CMBlockBuffer can be non-contiguous
CMBlockBufferGetDataLength would return the entire data length, while
size of individual blocks can be smaller. Iterate over the block buffer
and add the individual (possibly non-contiguous) memory blocks.

https://bugzilla.gnome.org/show_bug.cgi?id=751071
2015-06-25 10:33:54 +02:00
Ilya Konstantinov
58ce6d50a0 avfvideosrc: add frame rate range to caps
When AVFoundation indicates a supported frame rate range, add it to
the caps. This is important for devices such as the iPhone 6, which
indicate a single AVFrameRateRange of 2fps - 60fps.

https://bugzilla.gnome.org/show_bug.cgi?id=751048
2015-06-22 18:22:57 +02:00
Ilya Konstantinov
29c79d7595 vtenc: always enqueue frames, even on error
Even when we fail to encode frame, we should still enqueue it so
it could be passed into handle_frame (with output_buffer == NULL).
Otherwise, we risk GstVideoEncoder's queue of frames growing unbounded.

Note: We're slightly changing the renegotiation code to accommodate for
frames without output buffers, but this commit takes no ownership over
the way negotiation is being done.

https://bugzilla.gnome.org/show_bug.cgi?id=750669
2015-06-10 22:23:06 +02:00
Ilya Konstantinov
6101fc57b8 vtenc: free input buffer ASAP
VTCompressionSessionEncodeFrame retains the CVPixelBuffer during
encoding, and will release it as soon as it can (e.g. before it even
calls our callback). This means we can safely release input buffer
at this point, possibly allowing the system to reuse it sooner.

https://bugzilla.gnome.org/show_bug.cgi?id=750671
2015-06-10 10:30:23 +02:00
Ilya Konstantinov
f5c29c7379 vtenc: Add missing gst_vtenc_frame_free() in error path
https://bugzilla.gnome.org/show_bug.cgi?id=750668
2015-06-10 10:26:28 +02:00
Ilya Konstantinov
5db08a8b89 vtenc: add alignment=au to sink caps
VT compression callback gets a CMSampleBuffer per frame, so vtenc_h264
should set 'alignment=au' in sink caps to indicate this.

https://bugzilla.gnome.org/show_bug.cgi?id=750635
2015-06-10 10:20:44 +02:00
Ilya Konstantinov
05cc418ca2 avfassetsrc: fix huge memory leak
CMSampleBuffers were retained -> huge memory leak.

https://bugzilla.gnome.org/show_bug.cgi?id=750638
2015-06-09 16:36:41 +02:00
Ilya Konstantinov
faf903720a vtenc: no need for queue_length with try_pop 2015-06-09 16:20:06 +02:00
Ilya Konstantinov
1f7681dd0a vtdec: don't copy meta from input to output
Copying arbitrary metas is going to cause problems and this should really be
handled by the base class. It overrides most other things already anyway,
including timestamp and duration. Those are just set here now so we can
insert the frame sorted into the queue.

https://bugzilla.gnome.org/show_bug.cgi?id=748922
2015-06-09 16:20:06 +02:00
Thiago Santos
045bfa10fe Fix a common typo: retreive -> retrieve
Seems to have been copy pasted around a few places
2015-06-05 09:43:35 -03:00
Nicolas Dufresne
4cca6efda8 vtdec: Require width and height field for H264
This decoder does not work if width and height field are not set
in the sinkpad caps. Let's make this explicit by adding them to
the template caps.

https://bugzilla.gnome.org/show_bug.cgi?id=749655
2015-06-01 11:23:41 -04:00
Ilya Konstantinov
94e7ed1323 vtenc: fix keyframe request race condition
It is incorrect to modify the frame properties after passing them, since
VTCompressionSessionEncodeFrame takes reference and we have no control
over when it's being used.

In fact, the code can be simplified. We just preallocate the frame
properties for keyframe requests, and pass NULL otherwise.

https://bugzilla.gnome.org/show_bug.cgi?id=748467
2015-06-01 13:32:03 +02:00
Ilya Konstantinov
c948484c7f avfvideosrc: fix unconditional buffer queue unlock
Unless stopRequest is set, we should unlock conditionally -- otherwise,
the 'create:' method can wake up to an empty buffer queue
and pull a nil buffer.

https://bugzilla.gnome.org/show_bug.cgi?id=748054
2015-05-11 21:11:30 +10:00
Sebastian Dröge
7020154f1c vtdec: We shouldn't manually check reconfigure flag on pads but instead implement ::negotiate()
Add FIXME comment related to that.
2015-05-08 15:08:18 +02:00
Ilya Konstantinov
528871f571 avfvideosrc: drop frames we get before we have a clock
https://bugzilla.gnome.org/show_bug.cgi?id=748054
2015-04-26 20:37:11 +02:00
Yujin Lee
a8ea7a9cfb vtdec: fix build error with clang
This commit fixes a simple build error by clang with
by clang with [-Werror,-Wtautological-pointer-compare] options.

https://bugzilla.gnome.org/show_bug.cgi?id=748110
2015-04-26 19:23:06 +02:00
Alessandro Decina
edf9035d02 applemedia: avoid implicit color conversions
Rework the GL texture code a little to avoid implicit color conversion inside
AVF/VT on both iOS and OSX.
2015-04-20 17:09:58 +10:00
Ilya Konstantinov
c98eb6f2f5 corevideotexturecache: free texture cache on iOS
https://bugzilla.gnome.org/show_bug.cgi?id=748122
2015-04-19 14:48:43 +02:00
Ilya Konstantinov
472d1959a2 avfvideosrc: check for failure to create buffer
gst_core_media_buffer_new and gst_core_video_texture_cache_get_gl_buffer
can fail for various platform reasons.

https://bugzilla.gnome.org/show_bug.cgi?id=748122
2015-04-19 14:48:11 +02:00
Yujin Lee
4a88951c52 avfvideosrc: fix build failure with clang
Use fabs for floating point input.

https://bugzilla.gnome.org/show_bug.cgi?id=748128
2015-04-19 12:54:32 +01:00
Ilya Konstantinov
a02a948a5d avfvideosrc: get connection and input clock early 2015-04-18 00:33:42 +10:00
Ilya Konstantinov
d1120c591c avfvideosrc: remove unused variables
Complement commit 5496fd3e75 by removing
the involved variables.
2015-04-15 10:16:17 +10:00
Alessandro Decina
a10232f7da applemedia: vtdec: fix GL negotiation 2015-04-14 20:33:25 +10:00
Alessandro Decina
3e0391a1f1 avfvideosrc: log timestamps 2015-04-14 18:05:58 +10:00
Ilya Konstantinov
5bc17dac29 avfvideosrc: subtract time spent in AVF queues
The time spent in AVF queues was being incorrectly added to running time
rather than subtracted.
2015-04-14 18:05:58 +10:00
Ilya Konstantinov
1b27538648 avfvideosrc: fix GL texture negotiation
GST_VIDEO_GL_TEXTURE_UPLOAD_META_API_TYPE should no longer be used.
Instead, just get the GL context.

https://bugzilla.gnome.org/show_bug.cgi?id=747352
2015-04-14 18:05:57 +10:00
Ilya Konstantinov
4fd0f7277a avfvideosrc: fix setting of buffer offset end
Don't set offset end to random values off the stack.

https://bugzilla.gnome.org/show_bug.cgi?id=747352
2015-04-07 12:57:44 +01:00
ijsf
f62dda74cc Restricted activeVideoMaxFrameDuration to fix frame rate 2015-03-13 14:43:14 +00:00
Sebastian Dröge
160df421ea vtdec: Register a hardware-only vtdec_hw on OSX and give it a higher rank
while having the default vtdec at secondary rank. This allows decodebin/playbin
to prefer the hardware based decoders, and if that fails to initialize because
hardware resources are busy to fall back to e.g. the libav based h264 decoder
instead of the software based vtdec (which is slower), and only fall back to
the software based vtdec if there is no higher ranked decoder available.
2015-03-07 10:31:26 +01:00
Florian Zwoch
a8ae57480c avfvideosrc: allow "capture-screen" mode to select which screen to capture via the "device-index" option
https://bugzilla.gnome.org/show_bug.cgi?id=745161
2015-03-06 10:37:37 +01:00
Matthew Waters
0593480381 applemedia/avsamplesink: utilise a pull based approach of displaying video frames
Using requestMediaDataWhenReadyOnQueue the layer will execute a block
when it would like more frames. Using this we can provide the current
frame and avoid needlessly filling the layer's buffer queue causing
older frames to be displayed when under resource pressure.
2015-03-04 14:48:03 +11:00
Matthew Waters
c873c57430 vtdec: only attempt use of the texture cache with GLMemory caps features
Otherwise we send rectangle textures to glimagesink
2015-02-25 00:03:34 +11:00
Matthew Waters
e505f6257c applemedia: new AVSampleBufferLayerSink
Renders buffers using the CALayer subclass AVSampleBufferDisplayLayer
which can be placed inside a Core Animation render tree.
2015-02-24 19:12:10 +11:00
Sebastian Dröge
84ab11afdb avfvideosrc: Only set latency if we already know our caps
Otherwise we might set bogus values or GST_CLOCK_TIME_NONE.

Also make sure to reset the caps field to NULL after unreffing
the caps to prevent accidential use afterwards, and unref any
old caps before we remember new caps.
2015-02-19 01:03:35 +02:00
Sebastian Dröge
0c9f35653f vtenc: Check if VTSessionCopyProperty() succeeds before using the result
https://bugzilla.gnome.org/show_bug.cgi?id=744585
2015-02-16 12:35:09 +02:00