Commit graph

90 commits

Author SHA1 Message Date
Víctor Manuel Jáquez Leal
f974246647 msdkdec: explain the the extra ref of input buffer 2019-02-02 17:52:58 +01:00
Haihao Xiang
74f297f31b msdkdec: avoid releasing the input buffer when it is still in use
The input buffer is released in gst_msdkdec_finish_task () when decoding
some special clips however this buffer is still in use, so ref the input
buffer before gst_msdkdec_finish_task () and unref it at the end of
gst_msdkdec_handle_frame ().

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/862
2019-02-02 16:32:02 +00:00
Haihao Xiang
546797cecf msdkdec: set the mfx picture structure to MFX_PICSTRUCT_PROGRESSIVE if it is unknown
The picture structure in the output parameters from
MFXVideoDECODE_Query is set to MFX_PICSTRUCT_UNKNOWN for some codecs, so
the structure of the corresponding mfx surfaces created for decoding are
unknown. The pipeline will be broken when these surfaces are used as the
input for msdkvpp.

Example pipeline:

gst-launch-1.0 filesrc location=input_vp8.webm ! matroskademux !
msdkvp8dec ! msdkvpp ! fakesink

Error message:

0:00:00.031568911 14259 0x55b79dc684a0 ERROR                msdkvpp
gstmsdkvpp.c:728:gst_msdkvpp_transform:<msdkvpp0> MSDK Failed to do VPP
ERROR: from element
/GstPipeline:pipeline0/GstMatroskaDemux:matroskademux0: Internal data
stream error.

This is a workaround for the above issue
2019-01-25 12:45:43 +08:00
Víctor Manuel Jáquez Leal
5d030ed776 msdk: fix some comments typos 2019-01-24 08:49:39 +00:00
Víctor Manuel Jáquez Leal
5f67b4946e msdk: decoder: fatal failure if negotiations fails
Otherwise it crashes when no parser is used placed upstream, v.gr.

gst-launch-1.0 filesrc location=file.264 ! msdkh264dec ! fakesink
2019-01-24 08:49:39 +00:00
Víctor Manuel Jáquez Leal
f238066288 msdk: decoder: comment to explain array reset 2019-01-24 08:49:39 +00:00
Víctor Manuel Jáquez Leal
882befb5db msdk: decoder: fix an usage-after-free pointer bug 2019-01-24 08:49:39 +00:00
Haihao Xiang
ba7f4de298 msdk: release resources if failing to create the buffer pool
Needn't check the pointers against NULL because the pointers are
non-NULL
2019-01-09 09:42:56 +08:00
Haihao Xiang
718b7d0b5d msdk: check the created context against NULL pointer
gst_msdk_context_new_with_parent() may return NULL
2018-12-12 17:24:00 +00:00
Xiang, Haihao
fda4918cc8 msdk: don't use hard-coded video format
Some codecs may support varied formats, e.g. HEVC may support NV12
and P010_10LE etc
2018-11-26 16:47:17 +00:00
Xiang, Haihao
1bfd1b2bf9 msdk: Make sure a variable is initialized when it is used
Previously alloc_info is initialized when both thiz->initialized
and thiz->allocation_caps are true, but only thiz->initialized is
checked when alloc_info is used.
2018-11-26 16:13:44 +00:00
Wangfei
409d4ff30e msdkdec: move output-order out of decode bass class.
Since output-order is a deprecated attribute, move it out of decode
bass class and configure it in each sub decoder class who need it.

https://bugzilla.gnome.org/show_bug.cgi?id=796853
2018-11-23 12:26:40 +00:00
Sreerenj Balachandran
dd981d4953 msdkdec: Fix latency calculation
Use async_depth for latency calcuation instead of
the length of Tasks array which could be NULL since we
don't do the msdk decoder init in set_format().
2018-07-03 18:38:04 -08:00
Sreerenj Balachandran
84c33be0c0 msdk: Set 16 bit alignment for width
According to MediaSDK specification,
Width must be a multiple of 16 and Height must be a multiple
of 16 for progressive frame sequence and a multiple of 32 otherwise.

This patch sets a 16 bit alignment for width and 32 bit alignment
for height as default.

https://bugzilla.gnome.org/show_bug.cgi?id=796566
2018-07-02 16:50:46 -08:00
Sreerenj Balachandran
d63a1b4e3f msdkdec: avoid early destruction of frame in dynamic resolution change
In cases where we do hard resest, the current code destroys the frame
which has new resolution bit early and this causes buffer_unmap
warnings. Keep an extra ref to the frame internally to avoid this.
2018-07-02 16:50:02 -08:00
Sreerenj Balachandran
ad6162e99b msdkdec: Fix advanced profile vc1 decode when codec_data presents
The gst-msdk decoders only support packetized formats for
all codecs except VC1. For VC1, it supports codec_data for advanced
profiles and this codec_data wan't submitting to MSDK's DecodeHeader APIs.
Make sure the subclass deocders correctly configured so that
the codec_data buffers are in place in the internal adapter for
MediaSDK's DecoderHeader usage.
2018-07-02 16:48:11 -08:00
Sreerenj Balachandran
9efb4c9179 msdkdec: Fix the PTS of output frames
Currently we use the gst_video_decoder_get_oldest_frame()
to get the old pending frame to output. But this is not correct
if pts re-ordering required. This patch uses a custom made
get_old_frame() which accounts the PTS too similar to the
v4l2decoder.

https://bugzilla.gnome.org/show_bug.cgi?id=796699
2018-07-02 16:42:20 -08:00
Sreerenj Balachandran
5c88da4a4c msdkdec: Remove dead code
We are not using any ExtendedParams for decoding.
2018-07-02 16:41:58 -08:00
Sreerenj Balachandran
1e95c03c7d msdk: dec: Add dynamic-configuration change support
The patch adds a serios of changes to support dynamic resolution
change and efficient utilization of resources.
Major changes:

-- Use MSDK's apis to retrieve the headers instead of only relying
on upsteram notification. For eg: avc decoder requires SEI header
information for dpb count calculation which we don't get from caps.

-- For all codecs other than VP9, we force the reset of decoder
if resoultion changes to fit with gstreamer flow. VP9 enfource
the hard reset only if the new resolution is bigger.

-- delay the src caps setting till msdk api's invokation in
handle_frame to avoid caching multiple configuration values

-- ensure pool negotiation is based on decoder's allocation_caps.

--dynamic resoluttion change use an explicit allocation_query
to reclaim the buffers before closing the decoder (thanks to v4l2dec)

--In case if we don't get upstream notification of res change (for eg,
this can can happen for vp9 frames with ivfheader where ivfparse
is not able to notify the dynamic changes), we handle the the case
based on MFX_ERR_INCOMPATIBLE_VIDEO_PARAM which is the return value
of MFXVideoDECODE_DecodeFrameAsync

-- calculate the minimum surfaces to be preallocated based on
msdk suggestion, downstream requirement, async depth and scratch surface
count for smooth display.

https://bugzilla.gnome.org/show_bug.cgi?id=796566
2018-07-02 16:17:49 -08:00
Sreerenj Balachandran
54482a54d8 msdk:dec: Add new propery to dump frames in decoded order
The new property "output-order" can be set to either "display" order
which is the default where frames will be outputting in display order,
or "decoded-order" which will be outputting the frames in decoded order.

The "decoded order" output is generally useful for debugging. But there
are few
customers who use it for low-latency streaming. For eg if the customer
already knows that the stream doesn't have b-frames (which means no
algorithm requires for display order calculation), then they can use
"decoded-order"
output to skip some of the DPB logic to avoid the frame accumulation at
start-up.

The root cause of the above issue is a bit of unclarity in h264 spec +
lazy implementation of many H264 encoders; This is well handled in
gstreamer-vaapi using "low-latency" property:
https://bugzilla.gnome.org/show_bug.cgi?id=762509

https://bugzilla.gnome.org/show_bug.cgi?id=795783
2018-05-07 14:12:10 -08:00
Sreerenj Balachandran
a372c05f06 msdk: dec: inform msdk if the buffer contains a complete frame
For packetized input, inform the msdk that the buffer has
a complete frame or complementary field pairs. For decoding,
this means that the decoder can proceed with this buffer without
waiting for the start of the next frame, which effectively reduces
decoding latency.

https://bugzilla.gnome.org/show_bug.cgi?id=795783
2018-05-07 14:11:34 -08:00
Sreerenj Balachandran
978bcf8aa6 msdk: dec: reset async depth to one
Currently we use an async depth of 4 as default (based on
recommendations
in msdk apps), which indicates how many asynchronous operations an
application performs
before the application explicitly synchronizes the result. As a result,
we
queue four frames in decoder which might not be good approach for
live streaming.

This patch reset the async-depth to 1 as default so that we do sync for
each frame we decode without queuing. Customer can play with already
exposed "async-depth" property for other use cases

https://bugzilla.gnome.org/show_bug.cgi?id=795783
2018-05-07 14:11:14 -08:00
Hyunjun Ko
35b6411d4d msdk: dec: rename the function to what it means more exactly.
https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-04-02 15:55:55 -08:00
Hyunjun Ko
cdc591dbc0 msdkdec: use dmabuf if possible
https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:06:40 -08:00
Hyunjun Ko
1955ffed3f msdk: dec: set framerate to the driver only if provided
For example, if framerate 0/1 is provided from upstream, the driver
fails to configure and complain about it.

We can let it go and make the driver assuming framerate itself.

https://bugzilla.gnome.org/show_bug.cgi?id=789752
2018-03-29 12:41:48 -08:00
Hyunjun Ko
8a3630ffd7 msdk: dec: fix leaks when flushing
https://bugzilla.gnome.org/show_bug.cgi?id=793708
2018-03-08 11:38:52 -09:00
Hyunjun Ko
37ef61586a msdk: dec: remove code to manage buffers with locked surface
https://bugzilla.gnome.org/show_bug.cgi?id=793413
2018-03-08 11:37:52 -09:00
Sreerenj Balachandran
55c0d7205d msdk: dec: Add non-packetized stream handling support
The gst-msdk decoders prefer packetized streams as input
and in this case we can avoid unnecessary input bitstream copy
to mfxBitstream. This works fine for codecs like h264 where
we only support byte-stream with au alignment. Other format
conversions should be done thorugh parsers. But this won't work
for codecs like vc1 where we don't have an autoplugged parser.
Even the parser is not capable to do format conversions.

Packetizing through base decoders parse() routine will bring a
lot of uncecessary of complexities and codecparser libraray dependency.
So we just use an interal gst_adaper to keep track of bitstream
which is not consumed by msdk durig AsynchronusDecoding.
This adapter will get used only if subclass implementations
set the "is_packetized" to FALSE for msdk base encoder.

https://bugzilla.gnome.org/show_bug.cgi?id=792589
2018-02-13 14:41:20 -09:00
Hyunjun Ko
72c6cd5545 msdkdec: use video memory if there's another MSDK context in a pipeline
1\ If downstream's pool is MSDK bufferpool,
2\ If there's shared GstMsdkContext in the pipeline,

a decoder decides to use video memory.

This policy should be improved to handle more cases.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:53:02 -09:00
Hyunjun Ko
375a50a876 msdk: add async depth from each msdk element to GstMsdkContext to be shared
In case that pipeline is like ".. ! decoder ! encoder ! ..." with using
video memory,
decoder needs to know the async depth of the following msdk element so
that it could
allocate the correct number of video memory.

Otherwise, decoder's memory is exhausted while processing.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:52:14 -09:00
Hyunjun Ko
f2b35abcab msdkdec/enc: query GstContext to share GstMsdkContext
How to share/create GstMsdkcontext is the following:

- Search GstMsdkContext if there's in the pipeline.
  - If found, check if it's decoder, encoder or vpp by job type.
    - If it's same job type, it creates another instance of
GstMsdkContext
     with joined-session.
    - Otherwise just use the shared GstMsdkContext.
  - If not found, just creates new instance of GstMsdkContext.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:51:18 -09:00
Hyunjun Ko
dc82ccb9a2 msdk: context: add job type to figure out if joining session is necessary
According to the driver's instruction, if there are two or more encoders
or decoders in a process, the session should be joined by
MFXJoinSession.

To achieve this successfully by GstContext, this patch adds job type
specified if it's encoder, decoder or vpp.

If a msdk element gets to know if joining session is needed by the
shared context,
it should create another instance of GstContext with joined session,
which
is not shared.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:50:48 -09:00
Hyunjun Ko
a66d5620f3 msdkdec: use bufferpool
1\ In decide_allocation, it makes its own msdk bufferpool.
  - If downstream supports video meta, it just replace it with the msdk
bufferpool.
  - If not, it uses the msdk bufferpool as a side pool, which will be
decoded into.
    and will copy it to downstream's bufferpool.

2\ Decide if using video memory or system memory.
  - This is not completed in this patch.
  - It might be decided in update_src_caps.
  - But tested for both system memory and video memory cases.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:49:28 -09:00
Hyunjun Ko
3ed9f7c5f1 msdkdec: fix typo
https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 12:43:00 -09:00
Hyunjun Ko
6ce9a66b80 msdk: implements GstMsdkContext.
Makes GstMsdkContext to be a descendant of GstObject so that
we could track the life-cycle of the session of the driver.

Also replaces MsdkContext with this one.
Keeps msdk_d3d.c alive for the future.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 12:41:28 -09:00
Hyunjun Ko
8f0450dad4 msdk: move and rename the function msdk_video_alignment
Move the msdk_video_alignment function from decoder
to msdk.c and rename so that others could call this function
without duplicated declaration.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 12:36:46 -09:00
Hyunjun Ko
3301dd34b5 msdkdec: keep draining even if a finish_task fails
Should continue draining so that it could try to
discard the rest of pending frames even if a finish_task fails.

https://bugzilla.gnome.org/show_bug.cgi?id=790312
2017-11-23 10:08:13 +02:00
Hyunjun Ko
0933b8b45a msdkdec: fix buffer leaks during drain and a leak of videobufferpool
https://bugzilla.gnome.org/show_bug.cgi?id=790312
2017-11-22 17:30:07 +02:00
Scott D Phillips
732a157cb7 msdk: Propagate GstFlowReturn values
In some places a GST_FLOW_FLUSHING result was return as a FALSE
gboolean and then returned from a parent function as
GST_FLOW_ERROR. This prevented seeking from working.

https://bugzilla.gnome.org/show_bug.cgi?id=776360
2017-01-20 11:11:50 -08:00
Scott D Phillips
83774c3eb9 msdk: Add H.264 decoder
The decoder only supports system memory output presently.

https://bugzilla.gnome.org/show_bug.cgi?id=774587
2016-12-12 23:16:11 +01:00