Commit graph

12 commits

Author SHA1 Message Date
Matthew Waters
640a65bf96 gst: don't use volatile to mean atomic
volatile is not sufficient to provide atomic guarantees and real atomics
should be used instead.  GCC 11 has started warning about using volatile
with atomic operations.

https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1719

Discovered in https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/issues/868

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2098>
2021-03-22 14:34:36 +11:00
Seungha Yang
1d6cdbdc90 mediafoundation: Fix resource leak
IMFActivate would hold its internal objects unless user call ShutdownObject(),
even if we release the IMFActivate. Here internal objects may
include Direct3D objects, such as texture, device handle for example.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2087>
2021-03-16 09:47:41 +00:00
Seungha Yang
84db4b68fe mfvideoenc: Add support for Direct3D11 texture
Initial support for d3d11 texture so that encoder can copy
upstream d3d11 texture into encoder's own texture pool without
downloading memory.

This implementation requires MFTEnum2() API for creating
MFT (Media Foundation Transform) object for specific GPU but
the API is Windows 10 desktop only. So UWP is not target
of this change.
See also https://docs.microsoft.com/en-us/windows/win32/api/mfapi/nf-mfapi-mftenum2

Note that, for MF plugin to be able to support old OS versions
without breakage, this commit will load MFTEnum2() symbol
by using g_module_open()

Summary of required system environment:
- Needs Windows 10 (probably at least RS 1 update)
- GPU should support ExtendedNV12SharedTextureSupported feature
- Desktop application only (UWP is not supported yet)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1903>
2021-01-13 20:15:04 +00:00
Seungha Yang
6195fcf857 mfvideoenc: Improve latency performance for hardware encoder
Unlike software MFT (Media Foundation Transform) which is synchronous
in terms of processing input and output data, hardware MFT works
in asynchronous mode. output data might not be available right after
we pushed one input data into MFT.
Note that async MFT will fire two events, one is "METransformNeedInput"
which happens when MFT can accept more input data,
and the other is "METransformHaveOutput", that's for signaling
there's pending data which can be outputted immediately.

To listen the events, we can wait synchronously via
IMFMediaEventGenerator::GetEvent() or make use of IMFAsyncCallback
object which is asynchronous way and the event will be notified
from Media Foundation's internal worker queue thread.

To handle such asynchronous operation, previous working flow was
as follows (IMFMediaEventGenerator::GetEvent() was used for now)
- Check if there is pending output data and push the data toward downstream.
- Pulling events (from streaming thread) until there's at least
  one pending "METransformNeedInput" event
- Then, push one data into MFT from streaming thread
- Check if there is pending "METransformHaveOutput" again.
  If there is, push new output data to downstream
  (unlikely there is pending output data at this moment)

Above flow was processed from upstream streaming thread. That means
even if there's available output data, it could be outputted later
when the next buffer is pushed from upstream streaming thread.
It would introduce at least one frame latency in case of live stream.

To reduce such latency, this commit modifies the flow to be fully
asynchronous like hardware MFT was designed and to be able to
output encoded data whenever it's available. More specifically,
IMFAsyncCallback object will be used for handling
"METransformNeedInput" and "METransformHaveOutput" events from
Media Foundation's internal thread, and new output data will be
also outputted from the Media Foundation's thread.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1520>
2020-12-19 18:56:33 +00:00
Seungha Yang
52a82f5b84 mftransform: Fix deadlock when MFT requests processing output twice
This sequence of event/data flow might happen

1) Initially we have one pending output event
  1-1) Then, process the pending output data
2) No pending input event, then we should wait new pending input event
  2-1) Wakeup by new pending event (but it's pending output event)

In above case, MFT will not report new pending input event
if pending output is not processed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1325>
2020-06-08 23:46:43 +09:00
Seungha Yang
309c679c43 mftransform: Add some debug log
Add some trace level log for debugging

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1325>
2020-06-08 23:42:22 +09:00
Seungha Yang
cec6858401 mftransform: Don't try to drain if MFT is not running
Otherwise MFT will be blocked forever as no event can be generated by
IMFMediaEventGenerator.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1322>
2020-06-06 21:03:05 +09:00
Seungha Yang
cee619486a mediafoundation: Add support for AAC encoding
Add MediaFoundation AAC encoder element.

Before Windows 10, mono and stereo channels were supported audio channels
configuration by AAC encoder MFT. However, on Windows 10,
5.1 channels support was introduced.

To expose correct range of support format by this element
whatever the OS version is, this element will enumerate
all the supported format by the AAC encoder MFT
and then will configure sink/src templates while plugin init.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1280>
2020-05-27 10:34:47 +00:00
Seungha Yang
50b36ce257 mediafoundation: Use G_BEGIN_DECLS/G_END_DECLS pair everywhere
... instead of extern "c" {} block.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1241>
2020-05-25 15:09:20 +00:00
Seungha Yang
3efc8f5de9 mftransform: Clear unused output IMediaSample
If MFT doesn't produce encoded output, need to free allocated
output sample and buffer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1297>
2020-05-25 10:25:37 +00:00
Seungha Yang
c29c71ae9d mediafoundation: Remove COM thread constraints from GstMFTransform object
Move CoInitializeEx/CoUninitialize pair into our dedicated thread so that
we can ensure COM thread is MTA. This will remove thread constraints
around plugin init.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1269>
2020-05-15 09:03:13 +00:00
Seungha Yang
9625d19279 mediafoundation: Add h264 encoder
Add Media Foundation h264 encoder. If hardware encoders are available
on system, they will have higher rank than software encoder.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/760>
2020-04-28 14:37:31 +00:00