Commit graph

79 commits

Author SHA1 Message Date
Sebastian Dröge
2bcd155a6a avfvideosrc: Use AVCaptureDeviceTypeBuiltInDuoCamera instead of DualCamera for now
The former is deprecated in 10.2 but we don't require that SDK yet.
2017-03-09 10:10:37 +02:00
Nick Kallen
82c132aa50 applemedia/gl: Fix compile issues for OSX
https://bugzilla.gnome.org/show_bug.cgi?id=778333
2017-03-08 15:23:42 +02:00
Nick Kallen
0968bad8a4 applemedia: avfvideosrc supports position, device-type, and orientation
Avfvideosrc represents an iphone camera or, on mac, a screencapture session.
The old API allowed you to select an input device by device index only. The new
API adds the ability to select the position (front or back facing) and
device-type (wide angle, telephoto, etc.). Furthermore, you can now specify
the orientation (portrait, landscape, etc.) of the videostream.

https://bugzilla.gnome.org/show_bug.cgi?id=778333
2017-03-07 13:39:19 +02:00
Nick Kallen
46bbc60d24 applemedia/gl: Update code to use ARC
All code interacting with Objective-C objects should now use Automated
Reference Counting rather than manual memory management or Garbage
Collection. Because ARC prohibits C-structs from containing
references to Objective-C objects, all such fields are now typed
'gpointer'. Setting and gettings Objective-C fields on such a
struct now uses explicit __bridge_* calls to tell ARC about
object lifetimes.

https://bugzilla.gnome.org/show_bug.cgi?id=777847
2017-03-07 13:37:07 +02:00
Alessandro Decina
7898bc5810 applemedia: changes to make GL memory mappable on CPU on iOS
This commit introduces IOSGLMemory which is a GLMemory that falls back to
GstAppleCoreVideoMemory for CPU access. This is a temporary solution until
IOSurface gets exposed as a public framework on iOS and so we can use
IOSurfaceMemory on both MacOS and iOS.

https://bugzilla.gnome.org/show_bug.cgi?id=769210
2016-08-26 17:44:48 +10:00
Florian Zwoch
6de83e0515 avfvideosrc: account for retina displays when capturing screens
Fixes: https://bugzilla.gnome.org/show_bug.cgi?id=770030
2016-08-19 19:46:41 +10:00
Vineeth TM
8cdfb13658 bad: use new gst_element_class_add_static_pad_template()
https://bugzilla.gnome.org/show_bug.cgi?id=763081
2016-03-24 14:56:51 +02:00
Joe Gorse
2b63a88f26 avfvideosrc: Frame durations as CTime to the API, not double
Newer iOS seems to automatically convert, older iOS/OSX just crashes.

https://bugzilla.gnome.org/show_bug.cgi?id=762575
2016-02-25 11:34:40 +02:00
Alessandro Decina
e34fe9fe70 applemedia: avfvideosrc: implement GstElement::set_context
Needed to properly handle gst.gl.app_context(s).
2016-02-09 14:13:26 +11:00
Alessandro Decina
022e060563 applemedia: avfvideosrc: add myself to the copyright holders 2016-02-09 14:13:26 +11:00
Alessandro Decina
045a935871 applemedia: refactor GL context code
Rework the GL context code. Now both avfvideosrc and vtdec can create an
internal GL context for pushing textures. Both elements will still try to
use/switch to a local context where available (including after RECONFIGURE
events).
2016-01-29 15:10:32 +11:00
Alessandro Decina
6f80d5c59a applemedia: avfvideosrc: actually set the configured framerate
Actually set the configured framerate. Before we only used to set the first
matching framerate range. On iOS where the camera reports ranges [2, 60], we
used to configure the camera to output anything between 2 and 60fps.
2016-01-27 14:38:36 +11:00
Alessandro Decina
f6154b4cc0 applemedia: always fill GstBuffers with GstMemory
Always fill buffers with our custom memory. The custom memory will avoid
mapping CV/CM buffers unless necessary.
2016-01-19 16:13:04 +11:00
Alessandro Decina
05ccc2f6ea applemedia: avfvideosrc: optionally get the GL context from gst_element_set_context
Allow setting the GL context with gst_element_set_context. When available, the
local context obtained via the context query still has the precedence.
2016-01-12 14:36:02 +11:00
Alessandro Decina
b0f5d4f1d5 applemedia: avfvideosrc: enable renegotiation
We can actually renegotiate now, so remove old check which disabled it.
2016-01-06 16:51:12 +11:00
Alessandro Decina
85df5648bb applemedia: avfvideosrc: small negotiation fix
Fix negotiation when GLMemory is requested but a context is not available
2016-01-06 13:20:17 +11:00
Alessandro Decina
a15d8acff0 applemedia: avfvideosrc: change texture format from BGRA to NV12 on iOS
Change texture format from BGRA to NV12. This allows a pipeline like avfvideosrc
! tee name=t ! ... ! glimagesink t. ! ... ! gldownload ! vtenc_h264 ! ...  to
negotiate GLMemory. This makes the glimagesink branch much faster (obviously)
and triggers the 0-copy path between avfvideosrc and vtenc (using the CV meta).
Combined this results in a huge perf improvement on iOS (25-30% of CPU time in a
pipeline like the one above).

Note that this doesn't introduce a new shader conversion in the sink, since BGRA
textures had to be copied/converted from format=BGRA,texture-target=RECTANGLE to
format=RGBA,texture-target=2D anyway.
2016-01-06 11:43:37 +11:00
Alessandro Decina
bf86d7fef1 applemedia: avfvideosrc: output at highest fps/resolution by default
Fixate to the highest possible resolution and fps. Otherwise by default we end
up fixating at 2fps and the lowest supported resolution, which is hardly what
someone who bought an overpriced smartphone wants.
2015-12-29 10:57:36 +11:00
Alessandro Decina
4f550ff50f applemedia: avfvideosrc: do fixate 2015-12-17 14:49:13 +11:00
Alessandro Decina
bbdb22efba applemedia: avfvideosrc: rework GLMemory negotiation
Only do GLMemory when a GstGLContext is available AND GLMemory is actually
negotiated with downstream.
2015-12-16 17:03:04 +11:00
Alessandro Decina
0a73abd448 applemedia: avfvideosrc: update after IOSurface changes
Prefer GLMemory over sysmem. Also now when pushing GLMemory we push the
original formats (UYVY in OSX, BGRA in iOS) and leave it to downstream to
convert.
2015-12-16 17:03:04 +11:00
Alessandro Decina
e730b0a8ab applemedia: rename GstCoreVideoTextureCache to GstVideoTextureCache
First of a few commits to stop using CVOpenGLTextureCache on OSX and use
IOSurfaces directly instead. CVOpenGLTextureCache hasn't been updated for OpenGL
3 which is why texture sharing is currently disabled on OSX.
2015-12-16 17:03:03 +11:00
Ilya Konstantinov
58ce6d50a0 avfvideosrc: add frame rate range to caps
When AVFoundation indicates a supported frame rate range, add it to
the caps. This is important for devices such as the iPhone 6, which
indicate a single AVFrameRateRange of 2fps - 60fps.

https://bugzilla.gnome.org/show_bug.cgi?id=751048
2015-06-22 18:22:57 +02:00
Ilya Konstantinov
c948484c7f avfvideosrc: fix unconditional buffer queue unlock
Unless stopRequest is set, we should unlock conditionally -- otherwise,
the 'create:' method can wake up to an empty buffer queue
and pull a nil buffer.

https://bugzilla.gnome.org/show_bug.cgi?id=748054
2015-05-11 21:11:30 +10:00
Ilya Konstantinov
528871f571 avfvideosrc: drop frames we get before we have a clock
https://bugzilla.gnome.org/show_bug.cgi?id=748054
2015-04-26 20:37:11 +02:00
Alessandro Decina
edf9035d02 applemedia: avoid implicit color conversions
Rework the GL texture code a little to avoid implicit color conversion inside
AVF/VT on both iOS and OSX.
2015-04-20 17:09:58 +10:00
Ilya Konstantinov
472d1959a2 avfvideosrc: check for failure to create buffer
gst_core_media_buffer_new and gst_core_video_texture_cache_get_gl_buffer
can fail for various platform reasons.

https://bugzilla.gnome.org/show_bug.cgi?id=748122
2015-04-19 14:48:11 +02:00
Yujin Lee
4a88951c52 avfvideosrc: fix build failure with clang
Use fabs for floating point input.

https://bugzilla.gnome.org/show_bug.cgi?id=748128
2015-04-19 12:54:32 +01:00
Ilya Konstantinov
a02a948a5d avfvideosrc: get connection and input clock early 2015-04-18 00:33:42 +10:00
Ilya Konstantinov
d1120c591c avfvideosrc: remove unused variables
Complement commit 5496fd3e75 by removing
the involved variables.
2015-04-15 10:16:17 +10:00
Alessandro Decina
3e0391a1f1 avfvideosrc: log timestamps 2015-04-14 18:05:58 +10:00
Ilya Konstantinov
5bc17dac29 avfvideosrc: subtract time spent in AVF queues
The time spent in AVF queues was being incorrectly added to running time
rather than subtracted.
2015-04-14 18:05:58 +10:00
Ilya Konstantinov
1b27538648 avfvideosrc: fix GL texture negotiation
GST_VIDEO_GL_TEXTURE_UPLOAD_META_API_TYPE should no longer be used.
Instead, just get the GL context.

https://bugzilla.gnome.org/show_bug.cgi?id=747352
2015-04-14 18:05:57 +10:00
Ilya Konstantinov
4fd0f7277a avfvideosrc: fix setting of buffer offset end
Don't set offset end to random values off the stack.

https://bugzilla.gnome.org/show_bug.cgi?id=747352
2015-04-07 12:57:44 +01:00
ijsf
f62dda74cc Restricted activeVideoMaxFrameDuration to fix frame rate 2015-03-13 14:43:14 +00:00
Florian Zwoch
a8ae57480c avfvideosrc: allow "capture-screen" mode to select which screen to capture via the "device-index" option
https://bugzilla.gnome.org/show_bug.cgi?id=745161
2015-03-06 10:37:37 +01:00
Sebastian Dröge
84ab11afdb avfvideosrc: Only set latency if we already know our caps
Otherwise we might set bogus values or GST_CLOCK_TIME_NONE.

Also make sure to reset the caps field to NULL after unreffing
the caps to prevent accidential use afterwards, and unref any
old caps before we remember new caps.
2015-02-19 01:03:35 +02:00
Alessandro Decina
6e325be516 applemedia: avfvideosrc: fix crash when non-GL memory is used 2015-02-13 19:12:54 +11:00
Sebastian Dröge
31f59cb344 avfvideosrc: Only use the duration of the sample if it is valid 2015-02-12 11:12:09 +02:00
Alessandro Decina
f577b52e5d applemedia: rework GL texture sharing
Use YUV instead of RGB textures, then convert using the new apple specific
shader in GstGLColorConvert. Also use GLMemory directly instead of using the
GL upload meta, avoiding an extra texture copy we used to have before.
2015-01-30 00:34:08 +11:00
Alessandro Decina
8734abb1ce applemedia: don't call CVPixelBufferLockBaseAddress when doing texture sharing
When doing texture sharing we don't need to call CVPixelBufferLockBaseAddress to
map the buffer in CPU. This cuts about 10% relative cpu time from a vtdec !
glimagesink pipeline.
2015-01-30 00:34:08 +11:00
Alessandro Decina
cdfe90aaba avfvideosrc: implement the GL texture upload meta 2015-01-15 14:10:43 +11:00
Alessandro Decina
5496fd3e75 avfvideosrc: improve timestamp code
Use a combination of the running time and CMClockGetTime from AVF to compute
timestamps.
2014-12-27 12:43:45 +11:00
Alessandro Decina
0156725480 avfvideosrc: improve timestamping and latency reporting
Use AVF provided timings to timestamp output buffers. Use the running time at
the time the first buffer is produced to base timestamps on. Report 1-frame
latency based on the negotiated framerate instead of hardcoding 4ms latency.
2014-12-17 12:24:05 +11:00
Sebastian Dröge
98b856d630 avfvideosrc: Don't pretend to support reconfiguration
We will run into an assertion in set_caps() if we try to change
caps while the source is already running. Don't try to find new
caps in GstBaseSrc::negotiate() to prevent caps changes.
2014-11-09 10:41:52 +01:00
Tim-Philipp Müller
ffe59820d8 avfvideosrc: fix bogus BGRA caps
Remove 0.10-style endianness/*_mask/bpp/depth fields.
2014-09-10 16:38:52 +01:00
Matthieu Bouron
e728ee8208 coremediabuffer: handle stride alignment
Handle stride alignment through the use of the video meta API. The
code is based on the corevideobuffer implementation.

If the video meta API is not supported and the underlying buffer
contains padding, the core media buffer is copied to a system memory
buffer.

https://bugzilla.gnome.org/show_bug.cgi?id=727885
2014-04-24 12:27:59 +02:00
Matthieu Bouron
c6572df9ff avfvideosrc: check if downstream supports the video meta API
Check if downstream supports the video meta API, so we can use it later
with the core media buffers.

https://bugzilla.gnome.org/show_bug.cgi?id=727953
2014-04-24 10:46:51 +01:00
Matthieu Bouron
58be5ce16d avfvideosrc: declare missing instance variables
Fixes build on 32-bit system.

https://bugzilla.gnome.org/show_bug.cgi?id=711844
2013-11-11 14:36:19 +01:00
Andoni Morales Alastruey
98e849a092 avfvideosrc: fix build in iOS with isEqualTo doesn't exists 2013-11-07 15:24:28 +01:00