Commit graph

108 commits

Author SHA1 Message Date
Sreerenj Balachandran
54482a54d8 msdk:dec: Add new propery to dump frames in decoded order
The new property "output-order" can be set to either "display" order
which is the default where frames will be outputting in display order,
or "decoded-order" which will be outputting the frames in decoded order.

The "decoded order" output is generally useful for debugging. But there
are few
customers who use it for low-latency streaming. For eg if the customer
already knows that the stream doesn't have b-frames (which means no
algorithm requires for display order calculation), then they can use
"decoded-order"
output to skip some of the DPB logic to avoid the frame accumulation at
start-up.

The root cause of the above issue is a bit of unclarity in h264 spec +
lazy implementation of many H264 encoders; This is well handled in
gstreamer-vaapi using "low-latency" property:
https://bugzilla.gnome.org/show_bug.cgi?id=762509

https://bugzilla.gnome.org/show_bug.cgi?id=795783
2018-05-07 14:12:10 -08:00
Sreerenj Balachandran
a372c05f06 msdk: dec: inform msdk if the buffer contains a complete frame
For packetized input, inform the msdk that the buffer has
a complete frame or complementary field pairs. For decoding,
this means that the decoder can proceed with this buffer without
waiting for the start of the next frame, which effectively reduces
decoding latency.

https://bugzilla.gnome.org/show_bug.cgi?id=795783
2018-05-07 14:11:34 -08:00
Sreerenj Balachandran
978bcf8aa6 msdk: dec: reset async depth to one
Currently we use an async depth of 4 as default (based on
recommendations
in msdk apps), which indicates how many asynchronous operations an
application performs
before the application explicitly synchronizes the result. As a result,
we
queue four frames in decoder which might not be good approach for
live streaming.

This patch reset the async-depth to 1 as default so that we do sync for
each frame we decode without queuing. Customer can play with already
exposed "async-depth" property for other use cases

https://bugzilla.gnome.org/show_bug.cgi?id=795783
2018-05-07 14:11:14 -08:00
Sreerenj Balachandran
cb1eb650c6 msdk: enc: Add dmabuf-export support
Current implementation is only supporting dmabuf-export
through DMABufCapsfeatures.
MSDK dmabuf fds are not mappable and dmabuf-import
is not yet supported too (#794817).

https://bugzilla.gnome.org/show_bug.cgi?id=795707
2018-05-02 14:52:24 -08:00
Sreerenj Balachandran
e1a90f1ec9 msdkvpp: Disable passthrough if memory capsfeature changes
So far msdk produced dmabuf fds are non-mappable.
If user wants to download the content of underlined surfaces,
dmabufcapsfeature negotiated pipeline will fail. So if the input surface
is dmabuf and downstream doesn't have support for dmabuf capsfeatures,
we do the vpp (no passthrough) and produce the mappable videomemory
buffers.

https://bugzilla.gnome.org/show_bug.cgi?id=794946
2018-04-30 12:40:32 -08:00
Sreerenj Balachandran
ef6e186801 msdk: vpp: Add dmabuf-export support
Currenly, the dmabuf buffer pool can be negotiated
only through DMABuf capsfeatures.
This will not allow to negotiate dmabuf support with
v4l2src (v4l2src ! msdkvpp) where v4l2src always export
the dmabuf based memory with out using the DMABuf capsfeatures.
So it requires fix based on:
https://bugzilla.gnome.org/show_bug.cgi?id=794817

https://bugzilla.gnome.org/show_bug.cgi?id=794946
2018-04-30 12:39:52 -08:00
Sreerenj Balachandran
76bbefe3b0 msdk: vpp: Add YV12, YUY2 and BGRx formats to template 2018-04-25 12:33:16 -08:00
Sreerenj Balachandran
96c6a04d7a msdk: Add more video format mapping
BGRx format can be supported with Msdk's RGB4
2018-04-25 12:33:08 -08:00
Sreerenj Balachandran
5184f85d77 msdk: vpp: Allocation query fixes
prpose_allocation:
-- always instantiate a pool for for upstream
-- use async_depth + 1 as min buffer count

decide_allocation:
-- always create a new bufferpool for source pad.
Each of the msdk element has to create it's own mfxsurfacepool
which is an msdk contraint. For eg: Each Msdk component (vpp, dec and
enc)
will invoke the external Frame allocator for video-memory usage
So sharing the pool between gst-msdk elements might not be a good idea.

https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-25 12:33:00 -08:00
Xavier Claessens
83d0623293 Meson: Generate pc file for all plugins in bad
https://bugzilla.gnome.org/show_bug.cgi?id=794568
2018-04-25 11:08:09 +01:00
Sreerenj Balachandran
142ad9dbad msdk: jpegdec: Fix non-interleaved sample decode
Using the default value (InterleavedDec == MFX_SCANTYPE_UNKNOWN)
causing issues with non-interleaved sample decode. Ideally the usage
of MFXVideoDECODE_DecodeHeader should fix these type of issue, but
it seems to be not. But hardcoding the InterleaveDec to
MFX_SCANTYPE_NONINTERLEAVED
is fixing the problem and fortunately msdk seems to be taking care of
Interleaved samples
too .So let's hardcode it for now.

https://bugzilla.gnome.org/show_bug.cgi?id=793787
2018-04-16 14:37:21 -08:00
U. Artie Eoff
275d754156 msdk: fix plugin load on implementations with only HW support
We can't assume that MSDK always supports SW implementation
on all platforms.  Thus, msdk_is_available should check for
ANY implementation.

https://bugzilla.gnome.org/show_bug.cgi?id=794991
2018-04-04 17:31:14 -08:00
Tim-Philipp Müller
f7352ecc5c msdk: fix meson syntax 2018-04-03 19:22:01 +01:00
Sreerenj Balachandran
e4b4f09496 msdk: vpp : Add frame rate control
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 11:10:20 -08:00
Sreerenj Balachandran
c0ea4bdafb msdk: vpp : Add force-aspect-ratio property
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:45 -08:00
Sreerenj Balachandran
fb8c536393 msdk: Add more scaling filter algorithms
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:35 -08:00
Sreerenj Balachandran
51b6345dc4 msdk: vpp: Add support for horizontal and vertical mirroring
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:24 -08:00
Sreerenj Balachandran
108c8fde7f msdk: vpp: Add detail/edge enhancement tuning
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:13 -08:00
Sreerenj Balachandran
93c5dd2478 msdk: vpp: Add ProAmp(colorbalance) support
Added Hue, Saturation, Brightness and Contrast tuning support.

Fixme: Add GstColorBalanceInterface support

https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:39:02 -08:00
Sreerenj Balachandran
f5a3d3d799 msdk: vpp: Add deinterlacing support
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:38:52 -08:00
Sreerenj Balachandran
36e81744d1 msdk: vpp:Add more filters
-- Add Denoise
-- Add Rotation

https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:38:41 -08:00
Sreerenj Balachandran
d3d89f02b3 msdk: Add VPP element
https://bugzilla.gnome.org/show_bug.cgi?id=793705
2018-04-03 10:38:19 -08:00
Hyunjun Ko
35b6411d4d msdk: dec: rename the function to what it means more exactly.
https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-04-02 15:55:55 -08:00
Hyunjun Ko
90491d889a msdk: allocator: libva: check if it's already using dmabuf when mapping
As long as we negotiate the "DMABuf" capsfeatures for now, map can't be
working. So we need to confirm not to do it if using DMABuf memory.

https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-04-02 15:49:32 -08:00
Hyunjun Ko
cdc591dbc0 msdkdec: use dmabuf if possible
https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:06:40 -08:00
Hyunjun Ko
ea92da6e54 msdk: dmabuf support
This patch includes:
1\ Implements MsdkDmaBufAllocator and allocation of msdk dmabuf memroy.
2\ Each msdk dmabuf memory include its own msdk surface kept by GQuark.
3\ Adds new option GST_BUFFER_POOL_OPTION_MSDK_USE_DMABUF

https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:06:05 -08:00
Hyunjun Ko
5df06545ef msdk: adds new function to get dmabuf information from surface.
https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:05:16 -08:00
Hyunjun Ko
bd8ffcb29d msdk: allocator: get dmabuf handle during allocation if required
https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:04:28 -08:00
Hyunjun Ko
c3438b5a0f msdk: generalize the parameter of msdk video memory functions
There needs to be generalized for the parameter from
GstVideoMsdkVideoMemory to GstMemory.

Thus we can call these functions if using DMABuf memory.

https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:03:00 -08:00
Hyunjun Ko
762eb97092 msdk: specify the way to find a proper cached response by request
The current way to find proper response by just comparing request's
value is wrong.  We need to compare the size of a frame and the
number of suggested frames.

Refer to the sample in https://github.com/Intel-Media-SDK/samples.

https://bugzilla.gnome.org/show_bug.cgi?id=793707
2018-03-30 11:02:26 -08:00
Sreerenj Balachandran
e40dc51fcd msdk: dec: remove framerate field from sink caps template
Removes unessential field framerate for decoder so that negotiation
works even if framerate is not provided from upstream.

https://bugzilla.gnome.org/show_bug.cgi?id=789752
2018-03-29 13:06:41 -08:00
Hyunjun Ko
1955ffed3f msdk: dec: set framerate to the driver only if provided
For example, if framerate 0/1 is provided from upstream, the driver
fails to configure and complain about it.

We can let it go and make the driver assuming framerate itself.

https://bugzilla.gnome.org/show_bug.cgi?id=789752
2018-03-29 12:41:48 -08:00
Hyunjun Ko
0eaf3bdcd9 msdk: h265dec: remove framerate field from sink caps template
Removes unessential field framerate for decoder so that negotiation
works even if framerate is not provided from upstream.

https://bugzilla.gnome.org/show_bug.cgi?id=789752
2018-03-29 12:40:34 -08:00
Sreerenj Balachandran
f1f148fe7c msdk: Don't set extended coding options for JPEG encode
MJPEG doesn't have support for extended coding options

https://bugzilla.gnome.org/show_bug.cgi?id=793873
2018-03-29 11:56:19 -08:00
Hyunjun Ko
52f669bf43 msdk: libva: remove unnecessary code and comments
https://bugzilla.gnome.org/show_bug.cgi?id=794276
2018-03-13 14:21:40 -08:00
Hyunjun Ko
6547b638c5 msdk: adds new debug category
https://bugzilla.gnome.org/show_bug.cgi?id=794276
2018-03-13 14:20:50 -08:00
Hyunjun Ko
443b3c98fa msdk: fix typo
https://bugzilla.gnome.org/show_bug.cgi?id=794276
2018-03-13 14:18:23 -08:00
Wang,Fei
0c69867d52 msdk: Fix the I420 video format support
Make sure I420 surface mapping works as expected by using
YV12 format and swap U/V plane's offset and pitches.

https://bugzilla.gnome.org/show_bug.cgi?id=793865
2018-03-13 13:54:17 -08:00
Tim-Philipp Müller
c2bcc2711a meson: fix build when msdk is not found 2018-03-09 23:59:16 +00:00
Sreerenj Balachandran
bdbc1d0bef msdk: Fix the misspelled file name in meson build 2018-03-09 10:33:22 -09:00
Hyunjun Ko
4918430858 msdk: enc: fix missing some frames to be encoded
There was not handling the end of encoding sequence in encoder.
This patch does drain any remaining internal streams while decoder
already does this.

Document says:
"To mark the end of the encoding sequence, call this function with a
NULL surface
pointer. Repeat the call to drain any remaining internally cached
bitstreams—one
frame at a time—until MFX_ERR_MORE_DATA is returned."

https://bugzilla.gnome.org/show_bug.cgi?id=793236
2018-03-08 11:39:25 -09:00
Hyunjun Ko
8a3630ffd7 msdk: dec: fix leaks when flushing
https://bugzilla.gnome.org/show_bug.cgi?id=793708
2018-03-08 11:38:52 -09:00
Hyunjun Ko
c9faf0d612 msdk: manage child sessions on parent GstMsdkContext
Sometimes parent context is released before its children get released.
In this case MFXClose of parent session fails.

To make sure that child sessions are closed before closing a parent
session,
Parent context needs to manage child sessions and close them first when
it's released.

https://bugzilla.gnome.org/show_bug.cgi?id=793412
2018-03-08 11:38:30 -09:00
Hyunjun Ko
37ef61586a msdk: dec: remove code to manage buffers with locked surface
https://bugzilla.gnome.org/show_bug.cgi?id=793413
2018-03-08 11:37:52 -09:00
Hyunjun Ko
b08b8ddae3 msdk: manage MSDK surfaces seperately
Currently a gst buffer has one mfxFrameSurface when it's allocated and
can't be changed.
This is based on that the life of gst buffer and mfxFrameSurface would
be same.
But it's not true. Sometimes even if a gst buffer of a frame is finished
on downstream,
mfxFramesurface coupled with the gst buffer is still locked, which means
it's still being used in the driver.

So this patch does this.
Every time a gst buffer is acquired from the pool, it confirms if the
surface coupled with the buffer is unlocked.
If not, replace it with new unlocked one.
In this way, user(decoder or encoder) doesn't need to manage gst buffers
including locked surface.

To do that, this patch includes the following:
1. GstMsdkContext
- Manages MSDK surfaces available, used, locked respectively as the
following:
  1\ surfaces_avail : surfaces which are free and unused anywhere
  2\ surfaces_used : surfaces coupled with a gst buffer and being used
now.
  3\ surfaces_locked : surfaces still locked even after the gst buffer
is released.

- Provide an api to get MSDK surface available.
- Provide an api to release MSDK surface.

2. GstMsdkVideoMemory
- Gets a surface available when it's allocated.
- Provide an api to get an available surface with new unlocked one.
- Provide an api to release surface in the msdk video memory.

3. GstMsdkBufferPool
- In acquire_buffer, every time a gst buffer is acquired, get new
available surface from the list.
- In release_buffer, it confirms if the buffer's surface is unlocked or
not.
  - If unlocked, it is put to the available list.
  - If still locked, it is put to the locked list.

This also fixes bug #793525.

https://bugzilla.gnome.org/show_bug.cgi?id=793413
https://bugzilla.gnome.org/show_bug.cgi?id=793525
2018-03-08 11:37:12 -09:00
Hyunjun Ko
14f0186741 msdk: remove unused code
There's unused code remaining since MSDK bufferpool patches landed.

https://bugzilla.gnome.org/show_bug.cgi?id=793741
2018-02-23 14:30:56 -09:00
Sreerenj Balachandran
1c81bf4bdc msdkenc: remove unnecessary memset
https://bugzilla.gnome.org/show_bug.cgi?id=791479
2018-02-22 12:32:45 -09:00
Sreerenj Balachandran
3cc61f98b1 msdk: enc: Support force-key-unit events
https://bugzilla.gnome.org/show_bug.cgi?id=791479
2018-02-22 12:32:20 -09:00
Sreerenj Balachandran
a50ecfe46c msdk: enc: Fix typo 2018-02-20 17:22:35 -09:00
Sreerenj Balachandran
e924dec4e1 msdk: h264_enc: Enable B-pyramid prediction support
Since there is already an "adaptive-B" option, just
use boolean property for B-pyramid enabling.

Fixme: Not sure whether this can be supported in vp8 and vp9.
It could be possible through GPB (b without backward ref) but
can't verify currently. We can move this as common property
once verified with vp8 and vp9 without breaking any backward
compatibility.

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:41:18 -09:00