Commit graph

3674 commits

Author SHA1 Message Date
Stéphane Cerveau
6bc0e9527e remove various useless linefeed in logs 2019-12-11 10:51:29 +01:00
Haihao Xiang
b18be3562c msdk: ignore MFX_ERR_UNDEFINED_BEHAVIOR when loading a plugin
An issue can be seen when using msdkh265enc with bitrate change in
playing state. The root cause is the corresponding plugin is loaded
again.

Returning MFX_ERR_UNDEFINED_BEHAVIOR from MSDK just means the plugin has
been loaded, so we may ignore this error when doing configuation again
in the sub class, otherwise the pipeline will be interrupted
2019-12-10 17:23:53 +00:00
Seungha Yang
d963d442c5 d3d11videosink: Fix broken fallback rendering
Make fallback rendering work
2019-12-10 04:06:28 +00:00
Seungha Yang
abc1443cf0 d3d11videosink: Add debug message for fallback reason 2019-12-10 04:06:28 +00:00
Seungha Yang
9835289239 d3d11window: Do not check shader resource view if not doing conversion
If d3d11window does not convert format internally, shader resource view
is not required. Note that shader resource view is used for
color conversion using shader but when conversion is not required,
we just copy input input texture to backbuffer.
2019-12-10 04:06:28 +00:00
Julien Isorce
b681d6f62e msdkvpp: check mfx surface nullity for the input buffer
In theory it should not happen but it happened to me
in some cases where it failed to allocate some video
buffers so this was a consequence of a corner case.
Better to be safe than sorry.
2019-12-10 03:00:13 +00:00
Seungha Yang
8154b22a88 d3d11window: Clear old swapchain per prepare
_prepare() might be called multiple times for given window, so clear
old swapchain if any.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1138
2019-12-10 10:58:28 +09:00
Julien Isorce
3bae199863 msdkdec: fix assertion 'frame->ref_count > 0' failed
Can happen if the oldest frame is the current frame
and if gst_video_decoder_finish_frame failed in which
case the current is unref and then drop instead of
just drop.
This patch also removes some assumptions, it was strange
to call unref and finish_frame in gst_msdkdec_finish_task.
In principle when owning a frame, the code should either
unref, or drop or finish.
2019-12-09 16:10:51 +00:00
Seungha Yang
5bf10d8781 d3d11window: Always set dxgi colorspace if possible
Not only for the 10 bits RGB case with HDR metadata, set proper color space
to swapchain in order to make color representaion correct
2019-12-09 12:22:15 +00:00
Matthew Waters
526afac736 vtdec: add support for outputing vulkan images 2019-12-09 01:49:31 +00:00
Matthew Waters
acac78c1ce applemedia: rename IOSurfaceMemory -> IOSurfaceGLMemory 2019-12-09 01:49:30 +00:00
Matthew Waters
0560946c82 videotexturecache: implement GL specifics as a subclass 2019-12-09 01:49:30 +00:00
Matthew Waters
ff5ba2f126 applemedia/videocache: make a GObject subclass 2019-12-09 01:49:30 +00:00
Matthew Waters
27f4fb060d avfvideosrc: don't leak texture caches
If decide_allocation is called multiple times, then we would continually
overwrite the previous cache without freeing
2019-12-09 01:49:30 +00:00
Aaron Boxer
e9891dfe14 d3d11: translate windows error codes to human readable strings
This will help with debugging.
2019-12-07 16:09:03 -06:00
Xavier Claessens
da584995f6 mlaudiosink: Fix crash when stopping pipeline
This is a bug in LuminOS 0.97.0.
2019-12-06 15:29:29 +00:00
Xavier Claessens
380ab8123b Add mlaudiosink element 2019-12-06 15:29:29 +00:00
Sebastian Dröge
f776e2df96 decklink: Return new references to the devices from get_devices()
Otherwise the caller will free the devices we store internally.
2019-12-05 21:22:18 +02:00
Seungha Yang
4eac9d0ae6 d3dvideosink: Fix warning for unhandled enum in switch
d3dhelpers.c:135:3: warning: enumeration value 'D3DFMT_D32_LOCKABLE' not handled in switch [-Wswitch]
2019-12-05 04:31:54 +00:00
Seungha Yang
199caccc41 d3d11: Initialize debug categories of non-GstElement implementation in plugin init
That's the way to expose debug category to --gst-debug-help
2019-12-05 02:29:18 +00:00
Seungha Yang
71befc477f d3d11window: Invoke resize task with high priority
On resize event, swapchain should be configured with higher priority
than the other tasks since it's directly related to visual artifacts.
2019-12-05 02:29:18 +00:00
Seungha Yang
d2b85683de d3d11device: Add gst_d3d11_device_thread_add_full method
I would be used to invoke GPU task with specified priority.
2019-12-05 02:29:18 +00:00
Seungha Yang
7cab47ddda d3d11videosinkbin: Drop d3d11 color convert element
d3d11videosink can convert color space now, so the conversion element
seems to be redundant.
2019-12-05 02:29:18 +00:00
Seungha Yang
ca3ddf7848 d3d11videosink: Add color conversion support
Draw to back buffer texture directly. It would reduce the number of
copy at least once when color conversion is required.
2019-12-05 02:29:18 +00:00
Seungha Yang
622733ed0d d3d11converter: Skip setup converter if input and output formats are equal 2019-12-05 02:29:18 +00:00
Seungha Yang
c9cb08fcd4 d3d11colorconvert: Split color space converter to reuse code 2019-12-05 02:29:18 +00:00
Seungha Yang
42ca123f0f d3d11: Split shader to reuse code 2019-12-05 02:29:18 +00:00
Seungha Yang
59d9589722 d3d11videosink: Draw window with cached texture on resize
This would render nicer than presenting scene with broken aspect ratio,
especially in case of low framerate.
2019-12-05 02:29:18 +00:00
Seungha Yang
ddd13fc7c0 d3d11: Add support for D3D11_USAGE_DYNAMIC
D3D11 dynamic texture is a special memory type, which is mainly used for
frequent CPU write access to the texture. For now, this texture type
does not support gst_memory_{map,unmap}
2019-12-05 02:29:18 +00:00
Seungha Yang
5ce4748cc0 d3d11device: Add debug message for CreateTexture2D failure 2019-12-05 02:29:18 +00:00
Seungha Yang
914d2df79e d3d11device: Fix misreading debug message 2019-12-05 02:29:18 +00:00
Seungha Yang
71ae632b25 d3d11colorconvert: Disable color conversion with software rasterizer
It's much slower than our CPU based color-converter in most case.
2019-12-05 02:29:18 +00:00
Seungha Yang
79e88b76da d3d11videosink: Handle context query
Propagate d3d11device via context query.
2019-12-05 02:29:18 +00:00
Seungha Yang
5edf64d655 d3d11colorconvert: Avoid copy in/out texture if possible
When input and/or output d3d11memory has its own view, use them for
rendering to avoid texture copy
2019-12-05 02:29:18 +00:00
Seungha Yang
da3545e8c8 d3d11memory: Allocate texture with required views
Store required resource views with d3d11 texture when it requested.
2019-12-05 02:29:18 +00:00
Seungha Yang
4e4fb8a577 d3d11: Add d3d11videosinkbin element
New wrapper element to support d3d11 memory upload, color conversion, and
rendering at once.
2019-12-05 02:29:18 +00:00
Seungha Yang
79ac41ec5b d3d11: Add color space conversion element using shader
Add d3d11colorconvert element for color space conversion.
Currently {RGB, YUV} to RGB conversion is supported.
2019-12-05 02:29:18 +00:00
Seungha Yang
01903c47f7 d3d11memory: Register mininal compare function for GstD3D11AllocationParams
The d3d11 allocation param is stored in GstStructure so it should
provide compare function for later use such as gst_structure_is_equal()
2019-12-05 02:29:18 +00:00
Seungha Yang
0778fa8043 d3d11window: Set render target whenever drawing
Device context's render target might be updated for offscreen rendering
2019-12-05 02:29:18 +00:00
Seungha Yang
a687f06b59 d3d11videosink: Specify d3d11 memory caps feature on sinkpad template 2019-12-05 02:29:18 +00:00
Seungha Yang
c17c1346ee d3d11: Introduce d3d11 upload/download element
That's equivalent to glupload and gldownload elements but for d3d11
2019-12-05 02:29:18 +00:00
Seungha Yang
6c3311a39e d3d11: Refactor d3d11 memory and dxgi format usage
* Create staging texture only when the CPU access is requested.
Note that we should avoid the CPU access to d3d11 memory as mush as possible.
Incoming d3d11upload and d3d11download will take this GPU memory upload/download.

* Upload/Download texture memory from/to staging only if it needed, similar to
GstGL PBO implementation.

* Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder).
Because I420_* formats are not supported formats by dxgi, each plane should
be handled likewise GstGL separately, but NV12/P10 formats might be supported ones.
So we decide the number of d3d11memory per GstBuffer for video memory depending on
OS version and dxgi format. For instance, if NV12 is supported by OS,
only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit.
One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data
to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes.
Such behavior will be controlled via configuration of GstD3D11BufferPool and
default configuration is separate resources per plane.
2019-12-05 02:29:18 +00:00
Seungha Yang
d4c75f32dd d3d11: Add method for querying d3d11 memory type
... and fix misreading "private" annotation.
2019-12-05 02:29:18 +00:00
Seungha Yang
8da5237e22 d3d11device: Store selected d3d feature level
Depending on selected feature level, d3d11 API usage can be different.
Instead of querying the selected feature level by user whenever required,
store it once by d3d11device.
2019-12-05 02:29:18 +00:00
Seungha Yang
4ff0e62b72 d3d11: Don't share d3d11 device context unconditionally without checking adapter
Do not accept any GstD3D11Device context which has different adapter
index from the required one. For example, if a d3d11 element is expecting
d3d11 device with adapter 1 (i.e., the second GPU), any d3d11 device
context having different adapter could not be shared with
the d3d11 element.
2019-12-05 02:29:18 +00:00
Seungha Yang
7ae4105c6e d3d11device: Don't allow auto adapter selection
Change to d3d11device to be created with explicit target adapter index,
and expose some readonly device properties
2019-12-05 02:29:18 +00:00
Seungha Yang
494ec13316 d3d11: Rename native handle getter and context utils functions
Make them consistent with cuda context utils functions.
Put in-only parameter before all in-out parameters, and add _handle()
suffix to native handle getter functions.
2019-12-05 02:29:18 +00:00
Aaron Boxer
d3426cf15b d3dvideosink: use parent buffer pool to release buffers
In certain cases, the sink's buffer pool will not call the parent's
release_buffer method, so the pool does not clean up properly
after the buffer is released.
2019-12-04 23:00:35 +00:00
Sebastian Dröge
8ba4d1a480 decklink: Add simple device provider for Decklink devices 2019-12-03 09:16:22 +00:00
o0Ignition0o
d29012a760 avfvideosrc: Explicitly request device video permissions for macOS 10.14+
Since macOS Mojave (10.14), video permissions have to be explicitly
granted by a user in order to open a video device such as a camera.
This commit adds a check for the current permission status, and tries
to request for permission if applicable.
2019-12-02 08:52:05 +01:00
Nirbheek Chauhan
d6471b0251 wasapisrc: Correctly handle BUFFERFLAGS_SILENT
We need to ignore the data we get from WASAPI in this case and write
out silence (zeroes).

Initially reported at https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/808
2019-11-28 08:59:41 +00:00
Nirbheek Chauhan
08d5bdc7b5 wasapisrc: Try harder to avoid debug output in the hot loop
The whole `src_read()` function is a hot loop since the ringbuffer
thread is waiting on us, and printing to the console from inside it
can easily cause us to miss our deadline.

F.ex., if you had GST_DEBUG=3 and we accidentally missed a device
period, we'd trigger the "reported glitch" warning, which would cause
us to miss another device period, and so on. Let's reduce the log
level so that GST_DEBUG=3 is more usable, and only print buffer flag
info when it's actually relevant.
2019-11-28 08:59:41 +00:00
Nirbheek Chauhan
6cbff552fe wasapisrc: Fix capturing from some buggy audio drivers
Some audio drivers return varying amounts of data per ::GetBuffer
call, instead of following the device period that they've told us
about in `src_prepare()`.

Previously, we would just drop those extra buffers hoping that the
extra buffers were temporary (f.ex., a startup 'burst' of audio data).
However, it seems that some audio drivers, particularly on older
Windows versions (such as Windows 10 1703 and older) consistently
return varying amounts of data.

Use GstAdapter to smooth that out, and hope that the audio driver is
locally varying but globally periodic.

Initially reported in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/808
2019-11-28 08:59:41 +00:00
Nirbheek Chauhan
5d9c060ca7 wasapisrc: Clarify that nBlockAlign is actually bpf
bpf = bytes per frame.
2019-11-28 08:59:41 +00:00
Nirbheek Chauhan
6d27c0ac08 wasapisrc: Fix glitching and clock skew issues
We were miscalculating the device period, i.e. the number of frames
we'll get from WASAPI in each IAudioClient::GetBuffer call, due to
a calculation mistake (truncate instead of round).

For example, on my machine when the aux input is set to 44.1KHz, the
reported device period is 101587, which comes out to 447.998 frames
per ::GetBuffer call. In reality we will, of course, get 448 frames
per call, but we were truncating, so we expected 447 and were
discarding one frame every time. This led to glitching, and skew over
time.

Interestingly, I can only see this with 44.1Khz. 48Khz/96Khz are fine,
because the device period is a more 'even' number.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/806
2019-11-28 08:59:41 +00:00
Seungha Yang
00cb881431 msdk: Fix mixed declarations warning
msdk.c:194:3: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
2019-11-27 12:25:40 +09:00
Julien Isorce
2af672d00a msdkdec: log an error if leaking surfaces in finalize
The for loop in gst_msdkdec_handle_frame is error prone
about how it manages surfaces. Because sometimes it sets
the surface variable to NULL and sometimes it needs to free
it right away. So better to print an error if surfaces are
leaked to help with any change around the loop.
2019-11-25 17:14:20 +00:00
Aaron Boxer
348494fd34 msdk: query hardware when checking MFX availability
msdk plugin is not used for sofware encode/decode as there are better
solutions available. Also, with MFX_IMPL_AUTO_ANY, if software decode
is not supported, the plugin will still load, but will then fail when trying to
run the (autoplugged) pipeline. With MFX_IMPL_HARDWARE_ANY,
the plugin fails and a better software decoder is auto-plugged.
2019-11-22 05:18:42 +00:00
Aaron Boxer
2f1256ada7 msdk: query platform when opening session 2019-11-22 05:18:42 +00:00
Seungha Yang
0cf67c3be7 nvenc: Fix crash when nvenc was reused then freed without encoding
GstNvBaseEnc::n_bufs was set from the previous encoding session
but it wasn't cleared after stop. That might result to invalid memory
access at the next start (no encoded data) and then stop sequence.
Instead of defining a variable for array length, use GArray::len
directly to avoid such confusion.
2019-11-22 03:02:57 +00:00
Seungha Yang
aef414375a nvenc: Remove unused code path
refilling queue would not happen
2019-11-22 03:02:57 +00:00
Julien Isorce
a7dc407041 msdkdec: fix surface leak in msdkdec_handle_frame
Can be reproduced with:
  videotestsrc ! x264enc key-int-max=$N ! \
  h264parse ! msdkh264dec ! fakesink sync=1

It happens with any gop size but the smaller is the distance N
between key frames, the quicker it is leaking.

Fixes #1023
2019-11-19 09:14:51 -08:00
Andrew Branson
397a9edb35 androidmedia: Add new effects and scene modes to Camera parameters 2019-11-19 07:54:16 +01:00
Aaron Boxer
842bd66bd8 d3dvideosink: use sink dimensions when calculating overlay scaling 2019-11-18 03:32:57 +00:00
Aaron Boxer
20f3f63540 d3dvideosink: use explicit system memory feature in overlay composition caps 2019-11-18 02:29:45 +00:00
Jochen Henneberg
3ecd8666d9 msdk: Fixes for meson include directory setup
In case of pkg-config we need to create the include directories object
from the path using include_directories(). For INTELMEDIASDKROOT or
MFX_HOME we need to add the alternate include path ./include/mfx as
Intel MediaSDK now puts the headers there.
2019-11-17 17:39:44 +00:00
Seungha Yang
e52ed98197 d3d11window: Fix subclass procedure recursion
subclass window precedure should be cleared if it's no more used.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1115
2019-11-14 01:26:09 +00:00
Aaron Boxer
bfe2447a3b d3d11videosink: only destroy loop if external_win_id is false
Loop was created with this condition, so it should also be
destroyed under the same condition
2019-11-12 19:54:46 -06:00
Nicolas Dufresne
80b599c81b kmssink: Do not drain if imported buffer are from KMS
This adds a check to avoid draining when the imported buffers are in
fact own by kmssink. This happens since we export our kms buffer as
DMABuf. They are not really imported back as we pre-fill the cache,
but uses the same format as if they were external. This fixes
performance issues seen with videocrop2-test (found in -good).
2019-11-11 22:20:28 +00:00
Nicolas Dufresne
8574154cc8 kmssink: Avoid drain on caps changes
Draining systematically on caps changes was a hack. Instead, properly
save the render information used to render last_render, and use that
information to drain. This fixes performance issues met with video crop
meta and per frame caps changes.
2019-11-11 22:20:28 +00:00
Nicolas Dufresne
a83e0036ea kmssink: Ensure we have an allocator before importing
This fixes cases where the kms allocator API was called with a null
pointer.
2019-11-11 22:20:28 +00:00
Xavier Claessens
7d31f7a9e5 amc: Change plugin name if it's built for Magic Leap 2019-11-08 13:40:14 -05:00
Xavier Claessens
90223a2702 amc: Add MLSDK implementation 2019-11-08 13:40:14 -05:00
Haihao Xiang
dca3e47bce msdk: remove msdkvp8enc
MediaSDK doesn't support vp8 encode which is not going to be
implemented [1], so remove msdkvp8enc from this plugin

[1]: https://github.com/Intel-Media-SDK/MediaSDK/issues/947
2019-11-08 08:44:46 +00:00
Niels De Graef
d8f61515d8 Don't pass default GLib marshallers for signals
By passing NULL to `g_signal_new` instead of a marshaller, GLib will
actually internally optimize the signal (if the marshaller is available
in GLib itself) by also setting the valist marshaller. This makes the
signal emission a bit more performant than the regular marshalling,
which still needs to box into `GValue` and call libffi in case of a
generic marshaller.

Note that for custom marshallers, one would use
`g_signal_set_va_marshaller()` with the valist marshaller instead.
2019-11-06 14:27:46 +00:00
Ignacio Casal Quinteiro
a87ea01992 wasapi: minor cleanup 2019-11-06 08:18:59 +00:00
Aaron Boxer
86b01bb291 d3dvideosink: support OverlayComposition for GPU overlay compositing 2019-11-06 01:05:33 +00:00
Aaron Boxer
6d3429af34 documentation: fixed a heap o' typos 2019-11-05 09:11:25 -05:00
Roman Shpuntov
f84ebbd95f vtdec: fix leak of CMFormatDescription
There was consideration for replacing the CMFormatDescription used
on format changes in `set_format()` however on shutdown, we were
leaking the CMFormatDescription at the end of processing.

'https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1106'
2019-11-04 12:12:57 +00:00
Roman Shpuntov
b0543d6ea5 vtdec: memory leaks bugfix
release 'videoDecoderSpecification' variable in 'gst_vtdec_create_session' function.
release 'extensions' variable in 'create_format_description_from_codec_data' function.
2019-11-02 13:58:56 +00:00
Haihao Xiang
1dd3ef24b1 msdkmjpegenc: convert UYVY to YUY2 instead of NV12
Before this fix, the chroma subsampling of the output is 4:2:0. It is
4:2:2 with this fix, which is better for UYVY input
2019-11-02 09:52:36 +00:00
Haihao Xiang
e1b8b81e84 msdkmjpegenc: don't need conversion for BGRA and YUY2
jpeg encoder in MSDK can accept BGRA and YUY2 input.
2019-11-02 09:52:36 +00:00
Haihao Xiang
fbeaa37b79 msdkenc: add need_conversion method
In future, a sub class of GstMsdkEncClass may decide a native format by
using this method, e.g. JPEG encoder may accept YUY2 input, however the
current implemation needs a conversion from YUY2 to NV12 before encoding.

In addtion, a sub class may choose a format for encoding if the input
format is not supported by MSDK, e.g. the current implemation does
UYVY->NV12 if the input format is UYVY. We may do UYVY->YUY2 for JPEG
encoder in future
2019-11-02 09:52:36 +00:00
Haihao Xiang
afce02b392 msdk: support for MFX_FOURCC_BGR4 frame allocation
MFX_FOURCC_BGR4 is mapped to VA_FOURCC_ABGR and JPEG encoder needs a
MFX_FOURCC_BGR4 frame for internal usage when the input format is
MFX_FOURCC_RGB4

This is a preparation for supporting native formats of JPEG encoder
2019-11-02 09:52:36 +00:00
Víctor Manuel Jáquez Leal
10d4c0c511 msdkdec: use decoder base class packetized flag
Instead of using a proxy of `is_packetized` flag this patch
replaces it with the accessor to that flag in decoder base class,
avoiding probable mismatches.
2019-10-29 14:58:54 +00:00
Haihao Xiang
cd883427e9 msdkdec: add parse callback for non-packetized input
commit 55c0d720 added the capability to handle non-packetized bitstream,
and there is a loop to handle multiple frames in a non-packetized buffer
in gst_msdkdec_handle_frame. However it is possible that a
non-packetized buffer still contains valid data but there is no long any
pending unfinished frame. Currently gst_video_decoder_decode_frame is
invoked to send a new frame with new input data, the situaltion is
repeated till an EOS is received. An application has to exit when
receiving an EOS, however there is still valid data in a
non-packetezied input buffer, hence some frames are dropped.

This fix adds a parse callback for non-packeteized input, a new frame
will be sent to the subclass as soon as the input buffer has valid data

This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/665
2019-10-29 14:58:54 +00:00
Haihao Xiang
a2cafcfc3e msdkh265enc: Add support for tiled encoding
Add num-tile-rows and num-tile-cols properties to set the number of rows
and columns
2019-10-27 18:40:15 +00:00
Haihao Xiang
9fcd3bdd2b msdk: support P010_10LE in DMABuf mode for VP9/HEVC encoding 2019-10-25 16:54:57 +00:00
Cheng-Chang Wu
b5e506d12b androidmedia: Make sure that the first member of GstAmcSurfaceTextureJNI is correct
It's a subclass of GstAmcSurfaceTexture so should have that as first
struct member or otherwise it won't work.

Fixes #1105
2019-10-25 13:56:13 +00:00
Cheng-Chang Wu
e71f620eb0 androidmedia: Call JNI method with the actual Java object instead of our wrapper struct
Fixes #1105
2019-10-25 13:56:13 +00:00
Haihao Xiang
351a3934ec msdkvpp: add support Y410 and Y210 formats
And clean up some macros
2019-10-24 13:57:51 +00:00
U. Artie Eoff
b7ee6dc49a msdkvpp: allow cropping via properties
Add crop-left, crop-right, crop-top and crop-bottom
properties to msdkvpp and apply to input surface at
vpp transform run-time.

Implements #1097
2019-10-24 12:44:49 +00:00
Javier Celaya
8684dffe50 Fix get_supported_types function name 2019-10-16 06:42:37 +00:00
Javier Celaya
f267adeb4a Fix releasing local refs 2019-10-16 06:42:37 +00:00
Javier Celaya
3f0d60515a Fix names of MediaCodecInfo inner classes 2019-10-16 06:42:37 +00:00
Javier Celaya
26ea6d7fbb Fix getting MediaCodecList static methods 2019-10-16 06:42:37 +00:00
Julien Isorce
5de00b9ffb msdkenc: close encoder upon bitrate change
Upon bitrate change, make sure to close the encoder otherwise
the encoder is not re-initialized and the target bitrate is
never reached, and the encoder was flushed at each frame
from this moment.

Regression introduced in f2b35abcab which replaced the call
that was closing the encoder by an early return to avoid
re-initialization.
2019-10-15 15:53:20 -07:00
Seungha Yang
76654539b9 wasapi: Fix build warnings
gstwasapiutil.c(173) : warning C4715: 'gst_wasapi_device_role_to_erole': not all control paths return a value
gstwasapiutil.c(188) : warning C4715: 'gst_wasapi_erole_to_device_role': not all control paths return a value
2019-10-14 14:43:59 +00:00
Seungha Yang
baaa965ebc wasapi: Don't cast GstDeviceProvider to GstElement
The GstDeviceProvider isn't subclass of GstElement.

(gst-device-monitor-1.0:49356): GLib-GObject-WARNING **: 20:21:18.651:
invalid cast from 'GstWasapiDeviceProvider' to 'GstElement'
2019-10-14 14:43:59 +00:00
Tim-Philipp Müller
f218ec2794 Remove autotools build system 2019-10-14 13:54:27 +01:00
Matthew Waters
4d528776b6 decklink: fix timestamp tracking when pausing
play->pause->sleep()->play would result in late frames being scheduled
2019-10-14 19:22:36 +11:00
Haihao Xiang
e5279cd97a msdk: return the right pointer
The first channel in memory for MFX_FOURCC_RGB4 (VA_FOURCC_ARGB or
GST_VIDEO_FORMAT_BGRA) is B, not A. In MSDK, channle B is used to access
data for RGB4 surface. In addition, the returned pointers for
MFX_FOURCC_AYUV and MFX_FOURCC_Y410 in gst_msdk_video_memory_map_full
were wrong too before this fix.
2019-10-11 14:07:17 +00:00
Josep Torra
f2c75d548d msdkenc: handle bitrate changes in playing state
When the bitrate is changed in playing state the encoder issues a reconfig
that drains and recreates the underlaying hw encoder instance.
With this set of changes we ensure that all this work is only made when
the bitrate did actually change. It also tries to reuse the vpp buffer
pool and fixes the pool leak spotted when testing this feature.
2019-10-08 06:37:48 -07:00
Haihao Xiang
4525f8707f msdkh265enc: add the missing profile string 2019-10-08 09:40:06 +08:00
Seungha Yang
5e8755cf07 d3d11window: Protect registration/creation of WIN32 window
Querying existing window class info and creation of window object
should be protected since they are not thread safe.
2019-10-04 21:15:25 +09:00
Seungha Yang
44cb5b4ff7 d3d11videosink: Don't try to post error message by d3d11window self
The d3d11window isn't GstElement. To post error message, proxy it
to d3d11videosink instead.
2019-10-04 21:15:15 +09:00
Víctor Manuel Jáquez Leal
5140bf1a1f msdkenc: guard MFX_FOURCC_Y410 with MFX version 1027 2019-10-03 13:50:15 +00:00
Haihao Xiang
71ff8236b5 msdkdec: set lockable flag
When postpone_free_surface is TRUE, the output buffer is not writable,
however the base decoder needs a writable buffer as output buffer,
otherwise it will make a copy of the output buffer. As the underlying
memory is always lockable, so we may set the LOCKABLE flag for this buffer
to avoid buffer copy in the base class.

The refcount of the output buffer is 1 when postpone_free_surface is
FALSE, so needn't set the LOCKABLE flag for this case.
2019-09-30 18:15:38 +00:00
Haihao Xiang
83100b3504 msdkh265enc: add support for Y410 in sink pad
Sample pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=Y410 !
msdkh265enc low-power=1 ! filesink location=output.h265
2019-09-30 10:44:50 +08:00
Haihao Xiang
0507547646 msdkh265enc: add support for VUYA in sink pad
sample pipelie:
gst-launch-1.0 videotestsrc ! video/x-raw,format=VUYA !
msdkh265enc low-power=1 ! filesink location=output.h265
2019-09-30 10:44:38 +08:00
Haihao Xiang
85914997f5 msdkenc: clean up code by using switch
It is convenient for us to add support for new formats in future, there
is no change in functionality.
2019-09-30 10:42:41 +08:00
Haihao Xiang
0c1c2ce45a msdk: remove some dead code 2019-09-30 09:10:31 +08:00
Seungha Yang
22d8d77b13 d3d11videosink: Use pixel aspect ratio to setup window draw area
... instead of calculated display ratio from given PAR and DAR.

d3d11window calculates output display ratio
to decide padding area per window resize event. In the formula,
actual PAR is required to handle both 1:1 PAR and non-1:1 PAR.
2019-09-26 10:23:46 +00:00
Haihao Xiang
ce4f9ae531 msdk: fix for mfx frame alloc response
Both MSDK and this plugin use mfxFrameAllocResponse for video and DMABuf
memory, it is possible that some GST buffers are still in use when calling
gst_msdk_frame_free, so add a reference count in the wrapper of
mfxFrameAllocResponse (GstMsdkAllocResponse) to make sure the underlying
mfx resources are still available if the corresponding buffer pool is in
use.

In addtion, currently all allocators for input or output share the same
mfxFrameAllocResponse pointer in an element, so it is possible that
the content of mfxFrameAllocResponse is updated for a new caps then all
GST buffers allocated from an old allocator will use this new content of
mfxFrameAllocResponse, which will result in unexpected behavior. In this
fix, we save the the content of mfxFrameAllocResponse in the corresponding
tructure to avoid such issue

Sample pipeline:

gst-launch-1.0 filesrc location=vp9_multi_resolutions.ivf ! ivfparse ! msdkvp9dec !
msdkvpp ! video/x-raw\(memory:DMABuf\),format=NV12 ! glimagesink
2019-09-23 09:58:28 +08:00
Haihao Xiang
e50011c064 msdk: don't cache mfxFrameAllocResponse pointer
Otherwise it is possible that different wrappers share the same
mfxFrameAllocResponse pointer, so instead of caching the pointer, we may
cache the content of mfxFrameAllocResponse
2019-09-21 19:51:38 +00:00
Haihao Xiang
f840e304f3 msdkdec: postpone surface free for VC1
For a skipped frame in VC1, MSDK returns the mfx surface of the reference
frame, so we have to make sure the corresponding surface for the
reference frame is not freed. In this fix, we postpone surface free because
we don't know whether a surface is referenced

Before this fix, the error is like as below:

New clock: GstSystemClock
0:00:00.181793130 23098 0x55f8a9d622d0 ERROR                msdkdec
gstmsdkdec.c:622:gst_msdkdec_finish_task:<msdkvc1dec0> Couldn't find the
cached MSDK surface

Sample pipeline:
gst-launch-1.0 filesrc location=input_has_skipped_frame.wmv ! asfdemux !
vc1parse ! msdkvc1dec ! glimagesink
2019-09-21 18:05:42 +00:00
Haihao Xiang
c689c94458 msdkdec: release the surface if this surface is not in use
If the surface is not in use, we may release it even if GST_FLOW_OK is going
to be returned, which may avoid the issue of failing to get surface
available

This fixes the regression caused by commit c05acf4
2019-09-21 17:34:23 +00:00
Haihao Xiang
8daac1c09a msdk: adjust the stride align
GstAllocationParams::align is set to 31 in msdkdec/msdken/msdkvpp, hence
the stride align should be greater than or equal to 31, otherwise it
will result in issue
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/861
(msdk: "GStreamer-CRITICAL: gst_buffer_resize_range failed" SPAM),

In addition, the stride should match the pitch alignment in the media driver,
otherwise it will result in some issues when a buffer is shared between
different elements, e.g. the NV12 issue mentioned in commit 3f2314a, which
can be reproduced by `gst-launch-1.0 vidoetestsrc ! msdkvpp !
video/x-raw\(memory:DMABuf\),format=NV12 ! glimagesink`

Fixed https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/861
2019-09-21 16:59:10 +00:00
Seungha Yang
e0b2d34237 d3d11: Use g_cond_broadcast() to wake up waiting threads
gst_d3d11_device_thread_add() can be called concurrently from different
threads so the cond signal should reach to all waiting threads.
2019-09-18 18:47:44 +09:00
Haihao Xiang
0b0d5afb53 msdkenc: remove unneeded label 2019-09-16 10:09:00 +00:00
Matthew Waters
52cd9846eb applemedia/videocache: remove unimplemented/used upload function 2019-09-16 03:15:39 +00:00
Haihao Xiang
c05acf40a8 msdkdec: set pointer to NULL to avoid freeing an used surface
The surface should be cached in the surface list when GST_FLOW_OK is
going to be returned

This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1051
2019-09-13 17:32:19 +00:00
Haihao Xiang
38c55de132 msdk: free mfx frames to avoid memory leaks in encoder and vpp 2019-09-13 16:54:21 +00:00
Haihao Xiang
2d4b0a5649 msdkdec: unref the caps to avoid memory leaks 2019-09-13 16:54:21 +00:00
Haihao Xiang
cc6ca4ec61 msdkvpp: free msdk surfaces to avoid memory leak 2019-09-13 16:54:21 +00:00
Haihao Xiang
93f72a5041 msdkvpp: unref buffer pools
Unref the pools first in caps re-negotiation to avoid memory leak
2019-09-13 16:54:21 +00:00
Haihao Xiang
3b171f70af msdk: fix memory leaks in msdk allocators 2019-09-13 16:54:21 +00:00
Fuwei Tang
9b2dc96b99 msdkenc: work-around to avoid zero fps in MediaSDK structure 2019-09-13 16:28:00 +00:00
Fuwei Tang
11bbd6f721 msdkenc: add an extra surface for hevc encoding
For some hevc 10bit 4K encoding cases, the encoding process may be
slow, and MediaSDK surface can't be released in time before one other
available surface is needed. So add an extra surface for hevc encoding
to avoid this issue.
2019-09-12 02:20:11 +00:00
Seungha Yang
52dfbbe5da nvenc: Early terminate handle_frame if the last flow was not GST_FLOW_OK
If the last flow was not GST_FLOW_OK, the encoding thread is not running
and there is nothing to pop from GAsyncQueue (this causes deadlock).

To prevent deadlock, just return the handle_frame without further encoding
process if the last flow was not GST_FLOW_OK. Note that the last flow
will be cleared per FLUSH_STOP and STREAM_START event.
2019-09-11 15:21:03 +00:00
Seungha Yang
68a51abdcd nvenc: Add support VUYA format
The addition is very simple. Map NV_ENC_BUFFER_FORMAT_AYUV format
to GST_VIDEO_FORMAT_VUYA and add a condition for the VUYA format.
2019-09-11 14:33:54 +00:00
Seungha Yang
14b9a1cffd nvdec: Add support for mpeg4 video decoding with codec_data
Decoder should handle codec_data of mpeg4 video which includes essential
config data.
2019-09-11 14:00:48 +00:00
Seungha Yang
af77988b9f nvenc: Reduce the number of pre-allocated device memory
The hard-coded upper bound 32 (or 48 depending on resolution) might waste
GPU memory and high resolution encoding causes OUT-OF-MEMORY allocation error
quite easily. This commit calculates the number of required pre-allocated
device memory based on encoding options and it can reduce the amount of device memory
used by nvenc.
2019-09-11 11:44:03 +00:00
Seungha Yang
e3508a4f26 nvdec: Update plugin description and fix typo
Use consistent description with nvenc, and fix typo s/devide/device/g
2019-09-11 15:16:45 +09:00
Seungha Yang
1cbb23cf79 nvenc: Adjust DTS when bframe is enabled
NVDEC driver always uses input timestamp without adjustment
even if bframe encoding was enabled.
So DTS can be larger than PTS when bframe was enabled.
To ensure PTS >= DTS, we should adjust the timestamp manually
based on the PTS difference between the first
encoded frame and the second one. That's also the maximum PTS/DTS
difference.
2019-09-11 13:18:12 +09:00
Seungha Yang
83a1c7a9a6 nvenc: Add qp-{min,max,const}-{i,p,b} properties
This new properties allows more detailed target QP value setting
2019-09-11 13:18:12 +09:00
Seungha Yang
d3a909ccdd nvenc: Add properties to support bframe encoding if device supports it
Note that bframe encoding capability varies with GPU architecture
2019-09-11 13:18:12 +09:00
Seungha Yang
94f2843774 nvenc: Refactoring internal buffer pool structure
To support rc-lookahead and bframe encoding, nvenc needs one more
staging queue, because NvEncEncodePicture can return NV_ENC_ERR_NEED_MORE_INPUT
but which was not considered so far.
As documented by NVENC programming guide, pending buffers should wait
other inputs until NvEncEncodePicture returns success.

New encoding flow is
- Submit raw picture buffer to encoder with NvEncEncodePicture
- The submitted input/output buffer pair will be queued to pending_queue
  - If NvEncEncodePicture returned success, then move all pair in pending_queue
    to final stage
  - Otherwise, wait more input raw pictures.

Another change is dropping NV_ENC_LOCK_INPUT_BUFFER usage.
So now nvenc always uses CUDA memory input buffer. As a result,
both opengl and system memory handling are unified.
2019-09-11 13:18:12 +09:00
Seungha Yang
e73acbaa5c nvenc: Remove pointless iteration and cleanup some code
* The number of iteration is always one so the iteration is useless
and that makes code complicated.
* Also defining named structure can code mroe readable.
* g_free is null safe
2019-09-11 13:18:12 +09:00
Seungha Yang
81272eaa82 nvenc: Add more rate-control options
New rate-control modes are introduced (if device can support)
* cbr-ld-hr: CBR low-delay high quality
* cbr-hq: CBR high quality
* vbr-hq: VBR high quality

Also, various configurable rate-control related properties are added.
2019-09-11 13:18:12 +09:00
Seungha Yang
ea19a7c715 nvenc: Add support for weighted prediction option
Note that this property will be exposed only if the device
supports the weighted prediction.
2019-09-11 13:18:12 +09:00
Seungha Yang
d05cbdbd72 nvenc: Add property for AUD insertion
Make AUD insertion configurable option
2019-09-11 13:18:12 +09:00
Seungha Yang
b3b723462e nvenc: Refactor class hierarchy to handle device capability dependent options
Introducing new dynamic class between GstNvBaseEncClass and
each subclass to be able to access device specific properties and
capabilities from each subclass implementation side.
2019-09-11 13:18:09 +09:00
Mathieu Duponchelle
f402cb07f9 msdk: actually use the include dir we compute 2019-09-11 00:14:08 +00:00
Marc Leeman
3ef503346a nvcodec: minor spell corrects in log messages 2019-09-10 23:13:17 +00:00
Seungha Yang
54c5ebbd53 msdk: Remove all DMABuf caps features on Windows
Add new macro for sink/src pad template to ensure no DMABuf caps
features are exposed on Windows. Some DMABuf caps features
were not handled by the commit 9ec62418c3
2019-09-10 13:29:11 +00:00
Seungha Yang
eb3a117548 msdkdec: Do not use video memory on Windows
Like msdkenc, do not use video memory by default on Windows.
2019-09-10 13:29:11 +00:00
Seungha Yang
d4112d3d7b msdk: Allow video and system memory share among buffers
gst_buffer_make_writable() requires exclusive reference to the
GstMemory so the _make_writable() for the msdk buffer will result
to fallback system memory copy, because the msdk memory were initialized
with GST_MEMORY_FLAG_NO_SHARE flag.

Note that, disable sharing GstMemory brings high overhead but actually
the msdk memory objects can be shared over multiple buffers.
If the memory is not shareable, newly added GstAllocator::mem_copy will
create copied msdk memory.
2019-09-10 13:29:11 +00:00
Haihao Xiang
04b77a8c9d msdkh265dec: remove the requirement on profile
Sometimes a HEVC/H265 stream doesn't have a valid profile but MSDK can
handle this stream. Like vaapih265dec, msdkh265dec may advertise the sink
caps without profile
2019-09-10 12:54:39 +00:00
Haihao Xiang
18728310c9 msdkdec: output the decoded frame immediately if decoded order is required
DecodedOrder was deprecated in msdk-2017 version, but some customers
still use this for low-latency streaming of non-b-frame encoded streams,
which needs to output the frame at once
2019-09-04 05:25:13 +00:00