Commit graph

41 commits

Author SHA1 Message Date
Seungha Yang
9ec62418c3 msdk: Do not expose DMA buffer caps feature on Windows
On Windows, DMA buffer is not supported. PadTemplate with actually
supported feature seems to more make sense.
2019-07-22 16:13:43 +00:00
Haihao Xiang
1221898404 msdkenc: allow encode element requires extra frames
The call of MFXVideoENCODE_EncodeFrameAsync may not generate output and
the function returns MFX_ERR_MORE_DATA with NULL sync point, the input
frame is cached in this case, so it is possible that all allocated
frames go into the surfaces_used list after calling
MFXVideoENCODE_EncodeFrameAsync a few times, then the encoder will fail
to get an available surface before releasing used frames

This patch adds a new field of num_extra_frames to GstMsdkEnc and allows
encode element requires extra frames, the default value is 0.

This patch is the preparation for msdkvp9enc element.
2019-07-07 15:16:28 +00:00
Haihao Xiang
98e4967337 msdk: don't share context between msdkvpp and msdkenc
msdkenc supports CSC implicitly, so it is possible that two VPP
processes are required when a pipeline contains msdkvpp and msdkenc.
Before this fix, msdkvpp and msdkenc may share the same context, hence
the same mfx session, which results in MFX_ERR_UNDEFINED_BEHAVIOR
in MSDK because a mfx session has at most one VPP process only

This fixes the broken pipelines below:

gst-launch-1.0 videotestsrc ! video/x-raw,format=I420 ! msdkh264enc ! \
msdkh264dec ! msdkvpp ! video/x-raw,format=YUY2 ! fakesink

gst-launch-1.0 videotestsrc ! msdkvpp ! video/x-raw,format=YUY2 ! \
msdkh264enc ! fakesink
2019-06-30 06:08:31 +00:00
Peter Seiderer
e766aaf2b1 msdkenc: fix error handling in case of unsupported hardware
Check the return value of gst_msdk_context_ensure_context and
abort in case of failure.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/945
2019-04-10 15:54:31 +00:00
Haihao Xiang
9aa68dea51 msdk: set the right BRCParamMultiplier
BRCParamMultiplier in mfxInfoMFX is a parameter which specifies a
multiplier for bitrate control parameters [1], it impacts TargetKbps,
MaxKbps, BufferSizeInKB and InitialDelayInKB.

[1]: https://software.intel.com/en-us/node/628473
2019-01-11 09:57:44 +00:00
Haihao Xiang
5fc160fa59 msdk: set the upper bound of max-vbv-bitrate to 2048000 kbps
The upper bound of bitrate is also 2048000 kbps which should be large
enough in practice.

Fix https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/863
2019-01-11 09:57:44 +00:00
Haihao Xiang
6c887f982e msdk: check whether mfx function call fails
And log the failures for debugging
2019-01-09 12:43:56 +08:00
Haihao Xiang
d7606a7667 msdkenc: return NULL instead of FALSE
The returned type is a pointer
2019-01-09 09:50:49 +08:00
Haihao Xiang
ba7f4de298 msdk: release resources if failing to create the buffer pool
Needn't check the pointers against NULL because the pointers are
non-NULL
2019-01-09 09:42:56 +08:00
U. Artie Eoff
e91a53f693 msdkenc: break out of flush frames loop on error
In general, we should assume any unhandled error is
non-recoverable.

In the flush frames loop, some error states can cause us
to never increment the task and therefore we get stuck
in an infinite loop and generate GST_ELEMENT_ERROR
over and over again.  This eventually consumes all
system memory and triggers OOM.  Thus, assume the worst
and break out of the loop upon the first "unhandled" error.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/859
2019-01-06 19:07:35 -08:00
Haihao Xiang
88586b4149 msdk: change the wait time for encoder and vpp
In MSDK samples, the wait time for encoder, decoder and vpp is
300000. Let's set the wait time to the same value in msdk plugin
2018-12-12 17:24:00 +00:00
Haihao Xiang
718b7d0b5d msdk: check the created context against NULL pointer
gst_msdk_context_new_with_parent() may return NULL
2018-12-12 17:24:00 +00:00
Xiang, Haihao
f45d5dc8bc msdkenc: handle P010_10LE input format
Note it is up to each codec to support P010_10LE format
2018-12-12 09:00:13 +00:00
Xiang, Haihao
506835a484 msdkenc: use macro GST_VIDEO_INFO_FORMAT if possible 2018-12-12 09:00:13 +00:00
Xiang, Haihao
0e7e425bb9 msdk: add missing breaks 2018-12-03 07:09:56 +00:00
Seungha Yang
f106a10994 msdk: Fix building on MSVC
And add missing #ifdef _WIN32 for MSVC build

https://bugzilla.gnome.org/show_bug.cgi?id=797211
2018-10-02 04:04:24 +05:30
Sreerenj Balachandran
84c33be0c0 msdk: Set 16 bit alignment for width
According to MediaSDK specification,
Width must be a multiple of 16 and Height must be a multiple
of 16 for progressive frame sequence and a multiple of 32 otherwise.

This patch sets a 16 bit alignment for width and 32 bit alignment
for height as default.

https://bugzilla.gnome.org/show_bug.cgi?id=796566
2018-07-02 16:50:46 -08:00
Sreerenj Balachandran
57b9875260 msdk: enc: Add supprot for dmabuf-import
MediaSDK requires all the input buffers to be
pre-allocated during init phase and this won't work with
current design of GStreamer or gst-msdk. But this can be
done in future once we have a solution for:
https://bugzilla.gnome.org/show_bug.cgi?id=795747

There is a workaround possible as per
https://github.com/Intel-Media-SDK/MediaSDK/issues/155#issuecomment-381790504
by faking the mem-id during MFXInit.
This patch enabling it in gst-msdk by replacing the MemID of mfxSurface
with dmabuf-backed vasurface dynamically.

Important: v4l2 ! msdkenc won't work without a copy because
of the GMMLib (https://github.com/intel/gmmlib) memory restrictions.

https://bugzilla.gnome.org/show_bug.cgi?id=794817
2018-05-30 16:26:27 -08:00
Sreerenj Balachandran
cb1eb650c6 msdk: enc: Add dmabuf-export support
Current implementation is only supporting dmabuf-export
through DMABufCapsfeatures.
MSDK dmabuf fds are not mappable and dmabuf-import
is not yet supported too (#794817).

https://bugzilla.gnome.org/show_bug.cgi?id=795707
2018-05-02 14:52:24 -08:00
Sreerenj Balachandran
f1f148fe7c msdk: Don't set extended coding options for JPEG encode
MJPEG doesn't have support for extended coding options

https://bugzilla.gnome.org/show_bug.cgi?id=793873
2018-03-29 11:56:19 -08:00
Hyunjun Ko
4918430858 msdk: enc: fix missing some frames to be encoded
There was not handling the end of encoding sequence in encoder.
This patch does drain any remaining internal streams while decoder
already does this.

Document says:
"To mark the end of the encoding sequence, call this function with a
NULL surface
pointer. Repeat the call to drain any remaining internally cached
bitstreams—one
frame at a time—until MFX_ERR_MORE_DATA is returned."

https://bugzilla.gnome.org/show_bug.cgi?id=793236
2018-03-08 11:39:25 -09:00
Sreerenj Balachandran
1c81bf4bdc msdkenc: remove unnecessary memset
https://bugzilla.gnome.org/show_bug.cgi?id=791479
2018-02-22 12:32:45 -09:00
Sreerenj Balachandran
3cc61f98b1 msdk: enc: Support force-key-unit events
https://bugzilla.gnome.org/show_bug.cgi?id=791479
2018-02-22 12:32:20 -09:00
Sreerenj Balachandran
a50ecfe46c msdk: enc: Fix typo 2018-02-20 17:22:35 -09:00
Sreerenj Balachandran
07c05a75a5 msdk: Add more tuning options
Added tuning options for mb level bitrate control,
adaptive I-frame insertion, and adaptive B-frame insertion.

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:41:08 -09:00
Sreerenj Balachandran
ddd02be0de msdk: move enum definitions to separte file
Move enum value defintions which are (or in future) supported
by more than one codec into a common file.

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:40:50 -09:00
Sreerenj Balachandran
ae8d956c2c msdk: encode: Add more rate control options
MediaSDK has support for a number of rate control algorithms.
Adding all possible options to the property rate-control.

Fixme1: In case of failure, currently we don't have a proper method
to show which rate-control has been failed. It could be better
to add some extensive validation on EncQuery output in case of error.
Unfortunately, not all ratecontrol methods are supported by every codecs
and we don't have the dynamic detection of supported ratecontrol methods yet.

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:39:28 -09:00
Sreerenj Balachandran
f076f1948e msdk: encode: Add property to set slice/partitioning
Adding a new property num-slices to set the number of
slices/partitions per frame. Adding it as a general
property for all codecs (except jpeg).

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:39:13 -09:00
Sreerenj Balachandran
fa0911c3f6 msdk: encoder: register only the required properties
The base encoder common properties are not valid for
mjpeg encoder where there is no motion compensation or rate control.
Delaying the property installation on the base gobject
untill the subclass class_init get invoked.

https://bugzilla.gnome.org/show_bug.cgi?id=791637
2018-02-20 12:38:28 -09:00
Hyunjun Ko
375a50a876 msdk: add async depth from each msdk element to GstMsdkContext to be shared
In case that pipeline is like ".. ! decoder ! encoder ! ..." with using
video memory,
decoder needs to know the async depth of the following msdk element so
that it could
allocate the correct number of video memory.

Otherwise, decoder's memory is exhausted while processing.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:52:14 -09:00
Hyunjun Ko
f2b35abcab msdkdec/enc: query GstContext to share GstMsdkContext
How to share/create GstMsdkcontext is the following:

- Search GstMsdkContext if there's in the pipeline.
  - If found, check if it's decoder, encoder or vpp by job type.
    - If it's same job type, it creates another instance of
GstMsdkContext
     with joined-session.
    - Otherwise just use the shared GstMsdkContext.
  - If not found, just creates new instance of GstMsdkContext.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:51:18 -09:00
Hyunjun Ko
dc82ccb9a2 msdk: context: add job type to figure out if joining session is necessary
According to the driver's instruction, if there are two or more encoders
or decoders in a process, the session should be joined by
MFXJoinSession.

To achieve this successfully by GstContext, this patch adds job type
specified if it's encoder, decoder or vpp.

If a msdk element gets to know if joining session is needed by the
shared context,
it should create another instance of GstContext with joined session,
which
is not shared.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:50:48 -09:00
Hyunjun Ko
580a52ec49 msdkenc: use bufferpool
1\ Proposes msdk bufferpool to upstream.
  - If upstream has accepted the proposed msdk bufferpool,
    encoder can get msdk surface from the buffer directly.
  - If not, encoder get msdk surface its own msdk bufferpool
    and copy from upstream's frame to the surface.

2\ Replace arrays of surfaces with msdk bufferpool.

3\ In case of using VPP, there should be another msdk bufferpool
   with NV12 info so that it could convert first and encode.

Calls gst_msdk_set_frame_allocator and uses video memory only on linux.
and uses system memory on Windows until d3d allocator is implemented.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 13:48:32 -09:00
Hyunjun Ko
6ce9a66b80 msdk: implements GstMsdkContext.
Makes GstMsdkContext to be a descendant of GstObject so that
we could track the life-cycle of the session of the driver.

Also replaces MsdkContext with this one.
Keeps msdk_d3d.c alive for the future.

https://bugzilla.gnome.org/show_bug.cgi?id=790752
2018-02-13 12:41:28 -09:00
Hyunjun Ko
ddd9355767 msdkenc: handle the MORE_DATA case
If the driver requires more data, just unref the frame at the moment
then retreive/finish the frame after encoding is finished.

This also fixes a memory leak.

https://bugzilla.gnome.org/show_bug.cgi?id=790312
2017-11-22 17:30:07 +02:00
Hyunjun Ko
3c611da315 msdkenc: Fix handling of YUY2, UYVY, BGRA surfaces
https://bugzilla.gnome.org/show_bug.cgi?id=789847
2017-11-20 14:38:34 +02:00
Sebastian Dröge
d3eeb98f0c msdkenc: Add support for YV12, YUY2, UYVY and BGRA
By doing conversion with VPP to NV12 before the actual encoding.

https://bugzilla.gnome.org/show_bug.cgi?id=789847
2017-11-20 14:38:25 +02:00
Scott D Phillips
dcd9032d95 msdk: enc: set pts and dts, fix inverted sync_point flag
Set the pts and dts on the frame that we receive from the msdk.
Also fix the inverted logic in setting sync points, previously we
were marking all frames as sync points except IDRs.

https://bugzilla.gnome.org/show_bug.cgi?id=782801
2017-06-07 19:23:28 +03:00
Scott D Phillips
71df82db63 msdk: Initial windows build support
https://bugzilla.gnome.org/show_bug.cgi?id=770990
2016-11-14 15:17:41 +01:00
Scott D Phillips
2905a7e33e msdk: Width and Height need to be integer multiples of 32
https://bugzilla.gnome.org/show_bug.cgi?id=770990
2016-11-14 15:17:09 +01:00
Josep Torra
7d68d40239 msdk: Create plugin for Intel's Media SDK
[scott.d.phillips@intel.com: gst-indent *.c *.h]
[scott.d.phillips@intel.com: link with libva at build time]
[scott.d.phillips@intel.com: remove unused includes]
[scott.d.phillips@intel.com: Update README]

https://bugzilla.gnome.org/show_bug.cgi?id=770990
2016-11-14 15:16:45 +01:00