A post_configure virtual method is added to allow
codec subclasses to adjust the initialized parameters
after MFXVideoDECODE_DecodeHeader is called from the
gstmsdkdec::gst_msdkdec_handle_frame function.
This is useful if codecs want to adjust the output
parameters based on the codec-specific decoding
options that are present in the mfxInfoMFX structure
after MFXVideoDECODE_DecodeHeader initializes them.
We were not setting self->segment and we are using it
when notifying downstream that we handled a REQUEST_KEY_UNIT
event, leading to all sort of criticals.
It's been replaced by NVENC/NVDEC and even NVIDIA doesn't
support VDPAU any longer and hasn't for quite some time.
The plugin has been unmaintained and unsupported for a very
long time, and given the track record over the last 10 years
it seems highly unlikely anyone is going to make it work well,
not to mention adding plumbing for proper zero-copy or
gst-gl integration.
Closes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/828
All DRM ioctl uses errno to report the error and simply returns -1
when some error occured. This patch fixes all usage of the return
value instead of errno to trace the error type and moves to g_strerror
instead of string.h strerror in order to be consistent with the rest
of GStreamer.
separateColourPlaneFlag is mapped to separate_colour_plane_flag which
means Y, U and V planes are separately processed as monochrome sampled pictures.
So encoder shouldn't set that flag for normal 4:4:4 encoding.
Also for 4:4:4 encoding, NV_ENC_H264_PROFILE_HIGH_444_GUID profile must be
explicitly set.
The workaround for https://github.com/Intel-Media-SDK/MediaSDK/issues/1139
is required for vp8 only, so move this workaround to the corresponding
postinit_decoder function
The pipeline below works with this change
gst-launch-1.0 filesrc location=SA10104.vc1 ! \
'video/x-wmv,profile=(string)advanced',width=720,height=480,framerate=14/1 ! \
msdkvc1dec ! fakesink
MFXVideoDECODE_DecodeHeader only parses the sequence layer for VC1, so
the structure is unknown for a stream with interlace flag set in the
sequence layer. If forcing the struct to progressive in this plugin,
MediaSDK will fail to decode such streams.
It is possible MFXVideoDECODE_DecodeFrameAsync returns MFX_ERR_INCOMPATIBLE_VIDEO_PARAM
and this error can't be recovered by retrying MFXVideoDECODE_DecodeFrameAsync
in some cases, so we need to limit the number of retries to avoid infinite loop.
This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/909
To drain all queued encoding items, encoder should gracefully
wait the encoding thread without stealing queued items.
Otherwise, some input frames can be dropped.
../subprojects/gst-plugins-bad/sys/msdk/msdk.c(61): error C2065: 'MFX_FOURCC_RGB565'
The minimum required version for the format seems to MFX_VERSION >= 1028
Returning MFX_ERR_INCOMPATIBLE_VIDEO_PARAM from
MFXVideoDECODE_DecodeFrameAsync means the allocated mfx surface is not
suitable for the current frame, we need a new mfx surface and try
MFXVideoDECODE_DecodeFrameAsync again.
When MFXVideoDECODE_DecodeFrameAsync () returns MFX_WRN_DEVICE_BUSY with
an output surface, a new input surface is required when retrying
MFXVideoDECODE_DecodeFrameAsync ().
This fixes the out-of-surface issue mentioned in
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/890
This gets rid of annoying message in the log, e.g. run the pipeline
below:
gst-launch-1.0 videotestsrc num-buffers=100 ! \
video/x-raw,format=NV12,width=352,height=288 ! msdkh264enc ! filesink \
location=test.h264
[LIBVA]:CRITICAL - DdiMedia_DestroyImage:4357: Invalid image
This make the pipeline below works:
gst-launch-1.0 videotestsrc num-buffers=1 ! msdkvpp ! \
video/x-raw,format=UYVY ! filesink location=a.yuv
Once https://github.com/intel/media-driver/pull/526 in the media-driver
is merged, the pipeline below also works:
gst-launch-1.0 videotestsrc num-buffers=1 ! msdkvpp ! \
video/x-raw\(memory:DMABuf\),format=UYVY ! filesink location=a.yuv
The VCD source was ported in 2014 (commit 89eb1e9), but the necessary
"cdxaparse" plugin, which is used to "Parse a .dat file (VCD) into
raw mpeg1" was never ported.
This means that the probable main user for the feature, totem, hasn't
actually been able to play back VCDs, since 2012, when it switched to
using GStreamer 1.0.
Note that even if cdxaparse was finally ported, a lot of work would
still be necessary before it is considered usable. Notably, it is
missing disc image support [1] and some VCDs just cannot be opened for
reading [2].
[1]: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/898
[2]: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/899
gstladspa.c:360:5: error: zero-length ms_printf format string [-Werror=format-zero-length]
vad_private.c:108:3: error: this decimal constant is unsigned only in ISO C90 [-Werror]
gstdecklinkvideosink.cpp:478:32: error: comparison between 'BMDTimecodeFormat {aka enum _BMDTimecodeFormat}' and 'enum GstDecklinkTimecodeFormat' [-Werror=enum-compare]
win/DeckLinkAPI_i.c:72:8: error: extra tokens at end of #endif directive [-Werror]
win/DeckLinkAPIDispatch.cpp:35:10: error: unused variable 'res' [-Werror=unused-variable]
gstwasapiutil.c:733:3: error: format '%x' expects argument of type 'unsigned int', but argument 8 has type 'DWORD' [-Werror=format]
gstwasapiutil.c:733:3: error: format '%x' expects argument of type 'unsigned int', but argument 9 has type 'guint64' [-Werror=format]
kshelpers.c:446:3: error: missing braces around initializer [-Werror=missing-braces]
kshelpers.c:446:3: error: (near initialization for 'known_property_sets[0].guid.Data4') [-Werror=missing-braces]
An output surface is returned but without sync point when when
MFXVideoDECODE_DecodeFrameAsync () returns MFX_ERR_MORE_DATA, this
surface should be released too, otherwise the surface is occupied
and it is easy to exhaust all pre-allocated mfx surfaces.
Example pipeline (input_vp8.webm contains lots of frame with show_frame
set to 0):
gst-launch-1.0 filesrc location=input_vp8.webm ! matroskademux !
msdkvp8dec ! msdkvpp ! fakesink
0:00:05.995959693 19866 0x563f30f14590 ERROR default
gstmsdkvideomemory.c:77:gst_msdk_video_allocator_get_surface: failed to
get surface available
ERROR: from element
/GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0: Internal data
stream error.
The input buffer is released in gst_msdkdec_finish_task () when decoding
some special clips however this buffer is still in use, so ref the input
buffer before gst_msdkdec_finish_task () and unref it at the end of
gst_msdkdec_handle_frame ().
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/862
The picture structure in the output parameters from
MFXVideoDECODE_Query is set to MFX_PICSTRUCT_UNKNOWN for some codecs, so
the structure of the corresponding mfx surfaces created for decoding are
unknown. The pipeline will be broken when these surfaces are used as the
input for msdkvpp.
Example pipeline:
gst-launch-1.0 filesrc location=input_vp8.webm ! matroskademux !
msdkvp8dec ! msdkvpp ! fakesink
Error message:
0:00:00.031568911 14259 0x55b79dc684a0 ERROR msdkvpp
gstmsdkvpp.c:728:gst_msdkvpp_transform:<msdkvpp0> MSDK Failed to do VPP
ERROR: from element
/GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0: Internal data
stream error.
This is a workaround for the above issue
The memory type was used as bitwise enum, but the enum was not
defined in that way.
Nonetheless, most of the usage of the memory type was as mutually
exclusive options, rather than option composition.
This patch refactor how the memory type is defined, so it is kept
the mutual exclusion among options.
'channel-mask' field should not be put in caps if channel mask is 0x0
Mapping WASAPI channel mask to GST equivalent was going only over
first nChannels elements of wasapi_to_gst_pos array, translating, for
example, WASAPI's 0x63f to GST's 0x3f instead of 0xc3f.
When 'channel-mask' is specified as NULL, it signifies that there's
need to do downmix or upmix and it makes caps negotiation with
audioconvert element impossible. Just omit it.
Signed-off-by: Nirbheek Chauhan <nirbheek@centricular.com>
When building the msdk plugin even if libmfx is found, unless the
plugin is explicitly enabled we should not error out if msdk
dependencies are not found.
Also give an error message when we don't build the plugin on Windows
because we're not building with MSVC.
When the audio device goes away during playback or capture, we were
going into an infinite loop of AUDCLNT_E_DEVICE_INVALIDATED. Return -1
and post an error message so the ringbuffer thread exits with an error.
BRCParamMultiplier in mfxInfoMFX is a parameter which specifies a
multiplier for bitrate control parameters [1], it impacts TargetKbps,
MaxKbps, BufferSizeInKB and InitialDelayInKB.
[1]: https://software.intel.com/en-us/node/628473
According to the MSDK documation[1], MFXCloneSession is a light-weight
equivalent of MFXJoinSession after MFXInit, so MFXJoinSession call isn't
needed in the msdk plugin, otherwise the cloned session is joined to the
parent session twice, and we will get a MFX error when closing the
parent session
example pipeline:
gst-launch-1.0 videotestsrc num-buffers=100 ! \
video/x-raw,format=NV12,width=352,height=288 ! msdkh264enc ! msdkh264dec ! \
msdkh264enc ! fakesink
Error message:
0:00:00.211948518 21733 0x5586ee741c60 ERROR msdk
msdk.c:148:msdk_close_session: Close failed (undefined behavior)
[1]: https://software.intel.com/en-us/node/628429#MFXCloneSession
In gst-msdk, a mfx session may be shared between different gst
elements, each element tries to set the frame allocator. However, per
the MSDK documation[1], the behavior is undefined if reset the frame
allocator while the previous allocator is in use. Fortunately all
elements use the same frame allocator, so we can avoid to call
MFXVideoCORE_SetFrameAllocator again.
[1]: https://software.intel.com/en-us/node/628430#MFXVideoCORE3
If so, BGRA is the preferred output format hence BGRA will be selected
as input format by default, e.g. in the pipleline below, BGRA instead of
NV12 is selected without renegotiation, so we can avoid the NV12 issue
(see commit 3f2314a) by default.
gst-launch-1.0 videotestsrc ! msdkvpp ! glimagesink
Otherwise MFXVideoVPP_Init will fail because it is called twice without
a close.
Example pipeline:
gst-launch-1.0 videotestsrc ! msdkvpp ! glimagesink
Sometimes glimagesink emits GST_EVENT_RECONFIGURE event which results
in that MFXVideoVPP_Init is called twice, then get the negotiation
failure below:
0:00:00.093715518 21218 0x558ef56231e0 ERROR msdkvpp
gstmsdkvpp.c:995:gst_msdkvpp_initialize:<msdkvpp0> Init failed
(undefined behavior)
WARNING: from element /GstPipeline:pipeline0/GstMsdkVPP:msdkvpp0: not
negotiated
After applying this commit, the pipeline above may run without
negotiation failure, however NV12 layout in dmabuf mode is selected in
renegotiation, the display image is corrupted due to the NV12 issue which
was mentioned in commit 3f2314a. Some other fixes are needed to avoid
renegotiation by default
In general, we should assume any unhandled error is
non-recoverable.
In the flush frames loop, some error states can cause us
to never increment the task and therefore we get stuck
in an infinite loop and generate GST_ELEMENT_ERROR
over and over again. This eventually consumes all
system memory and triggers OOM. Thus, assume the worst
and break out of the loop upon the first "unhandled" error.
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/859
When either the source or sink goes from PLAYING -> NULL -> PLAYING,
we call _reset() which sets client_needs_restart, and then we call
prepare() which calls IAudioClient_Start(), so we don't need to call
it again in src_read() or sink_write(). Unlike when we're just going
PLAYING -> PAUSED -> PLAYING.
configure_mode_setting() keeps a ref on tmp_kmsmem which is released in
gst_kms_sink_show_frame().
But if for some reason configure_mode_setting() is re-called before
showing a frame or if none is showed this memory was leaked.
ACM is an ancient legacy API, and there's no point in
keeping it around for a licensed mp3 decoder now that
mp3 patents have expired and we have a decoder in -good.
We didn't ship this in cerbero anyway. If there's a good
case for the AAC encoder (which is LC only anyway) someone
should write a new plugin based on current APIs, that can
actually be built out of the box.
Fixes#850
As a side-effect we can now actually store the line offset in the
line21dec element, and have to perform fewer transformations in the
decklink elements (which were also buggy as they assumed a single byte
triplet per meta).
We will add more profiles in the sink caps of msdkh265enc, so let
msdkh265enc re-add the sink pad template. Note this change doesn't
impact any capability
Fixes the time calculations when dealing with a slaved clock (as
will occur with more than one decklink video sink), when performing
flushing seeks causing stalls in the output timeline, pausing.
Tighten up the calculations by relying solely on the internal time
(from the internal clock) for determining when to schedule display
frames instead attempting to track pause lengths from the external
clock and converting to internal time. This results in a much easier
offset calculation for choosing the output time and ensures that the
clock is always advancing when we need it to.
This is fixup to the 'monotonically increasing output timestamps' goal
in: bf849e9a69
CodecProfile will be set in MFXVideoDECODE_DecodeHeader() to match
the input stream. Setting the hard-coded profile here will mislead
user that msdkh265dec supports a special profile only.
Previously alloc_info is initialized when both thiz->initialized
and thiz->allocation_caps are true, but only thiz->initialized is
checked when alloc_info is used.
Instead of relying on buffers after a state change to PLAYING to always start
from 0, track the amount of time we have spent outside playing but not changed
state to PAUSED.
Note that, since Nvidia does not provide nvEncodeAPI.lib file,
find_library() couldn't be used for build on Windows.
This patch changes to load nvEncodeAPI(64).dll or libnvidia-encode.so
in runtime
dynlink_* was introduced since CUDA Toolkit 9.x but it's deprecated from 10.0.
Instead of using #ifdef hack, shipping nvidia headers of NVIDA CODEC SDK
can make build/code simple
