Fixes the time calculations when dealing with a slaved clock (as
will occur with more than one decklink video sink), when performing
flushing seeks causing stalls in the output timeline, pausing.
Tighten up the calculations by relying solely on the internal time
(from the internal clock) for determining when to schedule display
frames instead attempting to track pause lengths from the external
clock and converting to internal time. This results in a much easier
offset calculation for choosing the output time and ensures that the
clock is always advancing when we need it to.
This is fixup to the 'monotonically increasing output timestamps' goal
in: bf849e9a69
CodecProfile will be set in MFXVideoDECODE_DecodeHeader() to match
the input stream. Setting the hard-coded profile here will mislead
user that msdkh265dec supports a special profile only.
Previously alloc_info is initialized when both thiz->initialized
and thiz->allocation_caps are true, but only thiz->initialized is
checked when alloc_info is used.
Instead of relying on buffers after a state change to PLAYING to always start
from 0, track the amount of time we have spent outside playing but not changed
state to PAUSED.
Note that, since Nvidia does not provide nvEncodeAPI.lib file,
find_library() couldn't be used for build on Windows.
This patch changes to load nvEncodeAPI(64).dll or libnvidia-encode.so
in runtime
dynlink_* was introduced since CUDA Toolkit 9.x but it's deprecated from 10.0.
Instead of using #ifdef hack, shipping nvidia headers of NVIDA CODEC SDK
can make build/code simple
MediaSDK has been released as open source [1], but the directories
where it installs its files, are different from the binary only
distribution.
This patch adds to the libraries path the directory /lib. Also it
is defined in meson if the include directory has the mfx/ prefix,
something that is already handled in autotools.
1. https://github.com/Intel-Media-SDK/MediaSDK
"required" keyword is not a valid argument for has_header()
WARNING: Passed invalid keyword argument "required".
WARNING: This will become a hard error in the future.
These function are not used at all, using them together with the
transport-volume property from avdtpsrc may end up in a binding loop so
we better remove the functions.
If properties are proxied through GBinding this can work only if the
proxied property keeps it's own value. The previous implementation will
read the original value if the proxied property signals a change and
thus nothing will happen.
Right now this only works for video. An attempt was made at adding
monitoring following the example winks, but it seems the only devices that
can be easily detected are KS sources, which winks already handles.
The previous behaviour had issues when setting one of the device properties
after _get_caps had been called. The device shouldn't be locked in until after
_start has been called.
the 2018.3.1 intel sdk release places libraries into /lib64 instead of
/lib/lin_x64 or /lib/x64, this commit adds /lib64 to the libdir
locations list
Fixes#815
Before this patch, if mode=auto and video-format!=auto, video-format would
always be ignored, and get set to 8bit-yuv, or if detected to be RGB444, then
it would be set to 8bit-argb. This change respects video-format if it is set
to 10bit-yuv (v210) or 8bit-bgra, even when mode=auto.
Closes#772
There's a race condition when is-live is set to true and the shmsrc
element releases the pipe in the transition from PLAYING to PAUSED.
To avoid it this change ensures that _create method takes the pipe
and increases the use_count in one operation protected by object lock.
Also perform apropriate protections when releasing the pipe.
https://bugzilla.gnome.org/show_bug.cgi?id=797203
../sys/decklink/gstdecklinkvideosink.cpp:1006:11: error: ‘GstDecklinkVideoSink {aka struct _GstDecklinkVideoSink}’ has no member named ‘scheduled_stop_time’
self->scheduled_stop_time = start_time;
^
Decklink sometimes does not notify us through the callback that it has
stopped scheduled playback either because it was uncleanly shutdown
without an explicit stop or for unknown other reasons.
Wait on the cond for a short amount of time before checking if scheduled
playback has stopped without notification.
https://bugzilla.gnome.org/show_bug.cgi?id=797130
This is part of a much larger goal to always keep the frames we schedule to
decklink be always increasing. This also allows us to avoid using both the
sync and async frame display functions which aren't recomended to be used
together.
If the output timestatmsp is not always increasing decklink seems to hold
onto the latest frame and may cause a flash in the output if the played
sequence has a framerate less than the video output.
Scenario is play for N seconds, pause, flushing seek to some other position,
play again. Each of the play sequences would normally start at 0 with
the decklink time. As a result, the latest frame from the previous sequence
is kept alive waiting for it's timestamp to pass before either dropping
(if a subsequent frame in the new sequence overrides it) or displayed
causing the out of place frame to be displayed.
This is also supported by the debug logs from the decklink video sink
element where a ScheduledFrameCompleted() callback would not occur for
the frame until the above had happened.
It was timing related as to whether the frame was displayed based
on the decklink refresh cycle (which seems to be 16ms here),
when the frame was scheduled by the sink and the difference between
the 'time since vblank' of the two play requests (and thus start times
of scheduled playback).
https://bugzilla.gnome.org/show_bug.cgi?id=797130
This is now handled directly in gstaudiosrc/sink, and we were setting
it in the wrong thread anyway. prepare() is not the same thread as
sink_write() or src_read().
This adds "restore-crtc" property using which one
can restore previous crtc mode.
By default it is enabled, if CRTC was already
active with a valid mode and kmssink set a new mode
on CRTC using force-modesetting.
This helps user restore previous crtc mode and get
the previous session back after running a kmssink
pipeline involving a force-modesetting.
For e.g. When running a kmssink pipeline on rpi
using force-modesetting on tty console, it was giving
a blank screen after pipeline, and now with help of restore-crtc
functionality, CRTC is set with previous crtc mode
previously active on tty console.
Edited-by: Nicolas Dufresne <nicolas.dufresne@collabora.com>
https://bugzilla.gnome.org/show_bug.cgi?id=797025
This allow setting properties that contains spaces. The spaces are
replaced with '-'. As an example, one can set the connector proper
"scaling mode" with the following:
... ! kmssink connector-properties="s,scaling-mode=1"
https://bugzilla.gnome.org/show_bug.cgi?id=797027
Can be used to pass custom connector properties to DRM. Properties can
be enumerated using modetest tool. These properties can then be applied
with the following gst-launch-1.0 syntax. Note that the name of the
structure is ignored.
... ! kmssink connector-properties="s,props1=value,props2=value"
https://bugzilla.gnome.org/show_bug.cgi?id=797027
drmModeGetFB returns -EINVAL for multi-planar framebuffers. Instead of
depending on the framebuffer dimensions to select the mode, use width
and height from GstVideoInfo, which was used to create the framebuffer
in the first place. This enables kmssink to display multi-planar
formats such as I420 or NV12 with modesetting enabled.
https://bugzilla.gnome.org/show_bug.cgi?id=796985
Since both audio and video capture devices declare the KSCATEGORY_CAPTURE interface,
plugging a camera that supports both could result in an audio device being mistaken
for a video one.
https://bugzilla.gnome.org/show_bug.cgi?id=796958
With the Windows 8.1 SDK, the v1 of the AUDCLNT_STREAMOPTIONS enum is
defined which only has NONE and RAW, so it's not only defined when
AudioClient3 is available.
Add a meson check for the symbol. This is not needed for Autotools
because there we build against the MinGW audioclient.h which is still
at v1 of the AudioClient interface.
The mxsfb-drm driver has been added to the kernel long ago and will now
be the default display driver for NXP i.MX28, i.MX6SX and i.MX7D
processors so now is a good time to add it to kmssink.
Also, this is used in the upcoming i.MX8MQ and i.MX8MM processors.
https://bugzilla.gnome.org/show_bug.cgi?id=796873
Otherwise decklink seems to hold onto the latest frame and may cause a
flash in the output if the played sequence has a framerate less than the
video output.
Scenario is play for N seconds, pause, flushing seek to some other position,
play again. Each of the play sequences would normally start at 0 with
the decklink time. As a result, the latest frame from the previous sequence
is kept alive waiting for it's timestamp to pass before either dropping
(if a subsequent frame in the new sequence overrides it) or displayed
causing the out of place frame to be displayed.
This is also supported by the debug logs from the decklink video sink
element where a ScheduledFrameCompleted() callback would not occur for
the frame until the above had happened.
It was timing related as to whether the frame was displayed based
on the decklink refresh cycle (which seems to be 16ms here),
when the frame was scheduled by the sink and the difference between
the 'time since vblank' of the two play requests (and thus start times
of scheduled playback).
Use async_depth for latency calcuation instead of
the length of Tasks array which could be NULL since we
don't do the msdk decoder init in set_format().
According to MediaSDK specification,
Width must be a multiple of 16 and Height must be a multiple
of 16 for progressive frame sequence and a multiple of 32 otherwise.
This patch sets a 16 bit alignment for width and 32 bit alignment
for height as default.
https://bugzilla.gnome.org/show_bug.cgi?id=796566
In cases where we do hard resest, the current code destroys the frame
which has new resolution bit early and this causes buffer_unmap
warnings. Keep an extra ref to the frame internally to avoid this.
The gst-msdk decoders only support packetized formats for
all codecs except VC1. For VC1, it supports codec_data for advanced
profiles and this codec_data wan't submitting to MSDK's DecodeHeader APIs.
Make sure the subclass deocders correctly configured so that
the codec_data buffers are in place in the internal adapter for
MediaSDK's DecoderHeader usage.
Currently we use the gst_video_decoder_get_oldest_frame()
to get the old pending frame to output. But this is not correct
if pts re-ordering required. This patch uses a custom made
get_old_frame() which accounts the PTS too similar to the
v4l2decoder.
https://bugzilla.gnome.org/show_bug.cgi?id=796699
The patch adds a serios of changes to support dynamic resolution
change and efficient utilization of resources.
Major changes:
-- Use MSDK's apis to retrieve the headers instead of only relying
on upsteram notification. For eg: avc decoder requires SEI header
information for dpb count calculation which we don't get from caps.
-- For all codecs other than VP9, we force the reset of decoder
if resoultion changes to fit with gstreamer flow. VP9 enfource
the hard reset only if the new resolution is bigger.
-- delay the src caps setting till msdk api's invokation in
handle_frame to avoid caching multiple configuration values
-- ensure pool negotiation is based on decoder's allocation_caps.
--dynamic resoluttion change use an explicit allocation_query
to reclaim the buffers before closing the decoder (thanks to v4l2dec)
--In case if we don't get upstream notification of res change (for eg,
this can can happen for vp9 frames with ivfheader where ivfparse
is not able to notify the dynamic changes), we handle the the case
based on MFX_ERR_INCOMPATIBLE_VIDEO_PARAM which is the return value
of MFXVideoDECODE_DecodeFrameAsync
-- calculate the minimum surfaces to be preallocated based on
msdk suggestion, downstream requirement, async depth and scratch surface
count for smooth display.
https://bugzilla.gnome.org/show_bug.cgi?id=796566
The transform from mediacodec applies to the texture coords, but
GStreamer affine meta applies to the video geometry, which is the
opposite - so invert it to get display correct for decoders
that require transforming
According to msdk spec, there are two ways to enable filters:
1: Filters can be enabled by adding a filter ID
to mfxExtVPPDoUse. In this case, default filter parameters are used
2: Add filter configuration structures directly to mfxVideoParam.
Using 1 with 2 is optional but legal. Unfortunately it won't work
with some specific use cases like Detail/EdgeEnhancement.
Let's stick with option2 which works fine for all VPP operations.
https://bugzilla.gnome.org/show_bug.cgi?id=796468
Since we do the MSDK initializing in set_caps(), a FALSE
return may still cause the invokation of set_caps() again
and this will leads to buffer allocation and other mess-up.
So make sure the msdk initialized correctly before trying
to do any buffer allocation.
https://bugzilla.gnome.org/show_bug.cgi?id=796465
Make sure all the enabled filter structures are added in the
mfxVideoParm before doing the VPPQuery so that msdk
can do the input param validation
https://bugzilla.gnome.org/show_bug.cgi?id=796465
Update the license blurb in camconditionalaccess.[hc] from GPL to LGPL.
The plugin is LGPL and the GPL header in those two files was just a
copy/paste mistake.
Using NV12 layout in dmabuf mode giving mis-aligned
VPP output with the media-driver. Keep the NV12 support
(so that we can file the bug agianst msdk or mediadriver),
but lower the ordering so that BGRA picks as default.
NV12 issue can be reproduced with explicit capfilter:
vidoetestsrc ! msdkvpp ! video/x-raw\(memory:DMABuf\),format=NV12 ! glimagesink
Added a utility method to replace the MemID (interanl VASurfaceID)
associated with the mfxFrameSurface. This is usefull for dmabuf-import
where we need to replace the memID dynamically
https://bugzilla.gnome.org/show_bug.cgi?id=794817
Exporting DRM_PRIME fd to VASurface requires direct
invocation of VA api VACreateSurface with
VASurfaceAttribExternalBufferDescriptor and other
necessary surface attributes.
https://bugzilla.gnome.org/show_bug.cgi?id=794817
In case the wasapi buffer levels got low in shared mode we would still wait until
more buffer is available until writing something in it, which means we could never
catch up and recover.
Instead only wait for a new buffer in case the existing one is full and always write
what we can. Also don't loop until all data is written since the base class can handle
that for us and under normal circumstances this doesn't happen anyway.
This only works in shared mode, as in exclusive mode we have to exactly
fill the buffer and always have to wait first.
This fixes noisy (buffer underrun) playback with the wasapisink under load.
https://bugzilla.gnome.org/show_bug.cgi?id=796354
The calculation for the frame count in the non-aligned case resulted in
a one too low buffer frame count.
This resulted in:
1) exclusive mode not working as the frame count has to match
exactly there.
2) Buffer underruns in shared mode as the current write() code doesn't
handle catching up to low buffer levels (fixed in the next commit)
To fix just use the wasapi API to get the buffer size which will always
be correct.
https://bugzilla.gnome.org/show_bug.cgi?id=796354
S_FALSE is a valid return value which does not indicate an error.
For example IAudioClient_Stop() returns S_FALSE when it is already stopped.
Use the FAILED macro instead which just checks if an error occured or not.
This fixes spurious warnings when using the wasapisink element.
https://bugzilla.gnome.org/show_bug.cgi?id=796280
This is a new warning introduced by gcc 8
We already check just before that we have enough space, just do a regular
memcpy with the full string size.
camswclient.c:87:3: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
'cuDeviceComputeCapability' was deprecated as of CUDA 5.0
gstnvenc.c: In function ‘gst_nvenc_create_cuda_context’:
gstnvenc.c:290:9: error: ‘cuDeviceComputeCapability’ is deprecated [-Werror=deprecated-declarations]
&& cuDeviceComputeCapability (&maj, &min, cdev) == CUDA_SUCCESS) {
^
https://bugzilla.gnome.org/show_bug.cgi?id=796203
The new property "output-order" can be set to either "display" order
which is the default where frames will be outputting in display order,
or "decoded-order" which will be outputting the frames in decoded order.
The "decoded order" output is generally useful for debugging. But there
are few
customers who use it for low-latency streaming. For eg if the customer
already knows that the stream doesn't have b-frames (which means no
algorithm requires for display order calculation), then they can use
"decoded-order"
output to skip some of the DPB logic to avoid the frame accumulation at
start-up.
The root cause of the above issue is a bit of unclarity in h264 spec +
lazy implementation of many H264 encoders; This is well handled in
gstreamer-vaapi using "low-latency" property:
https://bugzilla.gnome.org/show_bug.cgi?id=762509https://bugzilla.gnome.org/show_bug.cgi?id=795783
For packetized input, inform the msdk that the buffer has
a complete frame or complementary field pairs. For decoding,
this means that the decoder can proceed with this buffer without
waiting for the start of the next frame, which effectively reduces
decoding latency.
https://bugzilla.gnome.org/show_bug.cgi?id=795783
Currently we use an async depth of 4 as default (based on
recommendations
in msdk apps), which indicates how many asynchronous operations an
application performs
before the application explicitly synchronizes the result. As a result,
we
queue four frames in decoder which might not be good approach for
live streaming.
This patch reset the async-depth to 1 as default so that we do sync for
each frame we decode without queuing. Customer can play with already
exposed "async-depth" property for other use cases
https://bugzilla.gnome.org/show_bug.cgi?id=795783
So far msdk produced dmabuf fds are non-mappable.
If user wants to download the content of underlined surfaces,
dmabufcapsfeature negotiated pipeline will fail. So if the input surface
is dmabuf and downstream doesn't have support for dmabuf capsfeatures,
we do the vpp (no passthrough) and produce the mappable videomemory
buffers.
https://bugzilla.gnome.org/show_bug.cgi?id=794946
