Commit graph

5 commits

Author SHA1 Message Date
Seungha Yang
84db4b68fe mfvideoenc: Add support for Direct3D11 texture
Initial support for d3d11 texture so that encoder can copy
upstream d3d11 texture into encoder's own texture pool without
downloading memory.

This implementation requires MFTEnum2() API for creating
MFT (Media Foundation Transform) object for specific GPU but
the API is Windows 10 desktop only. So UWP is not target
of this change.
See also https://docs.microsoft.com/en-us/windows/win32/api/mfapi/nf-mfapi-mftenum2

Note that, for MF plugin to be able to support old OS versions
without breakage, this commit will load MFTEnum2() symbol
by using g_module_open()

Summary of required system environment:
- Needs Windows 10 (probably at least RS 1 update)
- GPU should support ExtendedNV12SharedTextureSupported feature
- Desktop application only (UWP is not supported yet)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1903>
2021-01-13 20:15:04 +00:00
Seungha Yang
6195fcf857 mfvideoenc: Improve latency performance for hardware encoder
Unlike software MFT (Media Foundation Transform) which is synchronous
in terms of processing input and output data, hardware MFT works
in asynchronous mode. output data might not be available right after
we pushed one input data into MFT.
Note that async MFT will fire two events, one is "METransformNeedInput"
which happens when MFT can accept more input data,
and the other is "METransformHaveOutput", that's for signaling
there's pending data which can be outputted immediately.

To listen the events, we can wait synchronously via
IMFMediaEventGenerator::GetEvent() or make use of IMFAsyncCallback
object which is asynchronous way and the event will be notified
from Media Foundation's internal worker queue thread.

To handle such asynchronous operation, previous working flow was
as follows (IMFMediaEventGenerator::GetEvent() was used for now)
- Check if there is pending output data and push the data toward downstream.
- Pulling events (from streaming thread) until there's at least
  one pending "METransformNeedInput" event
- Then, push one data into MFT from streaming thread
- Check if there is pending "METransformHaveOutput" again.
  If there is, push new output data to downstream
  (unlikely there is pending output data at this moment)

Above flow was processed from upstream streaming thread. That means
even if there's available output data, it could be outputted later
when the next buffer is pushed from upstream streaming thread.
It would introduce at least one frame latency in case of live stream.

To reduce such latency, this commit modifies the flow to be fully
asynchronous like hardware MFT was designed and to be able to
output encoded data whenever it's available. More specifically,
IMFAsyncCallback object will be used for handling
"METransformNeedInput" and "METransformHaveOutput" events from
Media Foundation's internal thread, and new output data will be
also outputted from the Media Foundation's thread.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1520>
2020-12-19 18:56:33 +00:00
Seungha Yang
cee619486a mediafoundation: Add support for AAC encoding
Add MediaFoundation AAC encoder element.

Before Windows 10, mono and stereo channels were supported audio channels
configuration by AAC encoder MFT. However, on Windows 10,
5.1 channels support was introduced.

To expose correct range of support format by this element
whatever the OS version is, this element will enumerate
all the supported format by the AAC encoder MFT
and then will configure sink/src templates while plugin init.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1280>
2020-05-27 10:34:47 +00:00
Seungha Yang
c29c71ae9d mediafoundation: Remove COM thread constraints from GstMFTransform object
Move CoInitializeEx/CoUninitialize pair into our dedicated thread so that
we can ensure COM thread is MTA. This will remove thread constraints
around plugin init.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1269>
2020-05-15 09:03:13 +00:00
Seungha Yang
9625d19279 mediafoundation: Add h264 encoder
Add Media Foundation h264 encoder. If hardware encoders are available
on system, they will have higher rank than software encoder.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/760>
2020-04-28 14:37:31 +00:00