gstd3d11window_corewindow.cpp(408): warning C4189:
'storage': local variable is initialized but not referenced
gstd3d11window_corewindow.cpp(490): warning C4189:
'self': local variable is initialized but not referenced
gstd3d11window_swapchainpanel.cpp(481): warning C4189:
'self': local variable is initialized but not referenced
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1962>
Some GPUs (especially NVIDIA) are complaining that GPU is still busy
even we did 50 times of retry with 1ms sleep per failure.
Because DXVA/D3D11 doesn't provide API for "GPU-IS-READY-TO-DECODE"
like signal, there seems to be still no better solution other than sleep.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1913>
The vabasedec's display and decoder are created/destroyed between
the gst_va_base_dec_open/close pair. All the data and event handling
functions are between this pair and so the accessing to these pointers
are safe. But the query function can be called anytime. So we need to:
1. Make these pointers operation in open/close and query atomic.
2. Hold an extra ref during query function to avoid it destroyed.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1957>
Initial support for d3d11 texture so that encoder can copy
upstream d3d11 texture into encoder's own texture pool without
downloading memory.
This implementation requires MFTEnum2() API for creating
MFT (Media Foundation Transform) object for specific GPU but
the API is Windows 10 desktop only. So UWP is not target
of this change.
See also https://docs.microsoft.com/en-us/windows/win32/api/mfapi/nf-mfapi-mftenum2
Note that, for MF plugin to be able to support old OS versions
without breakage, this commit will load MFTEnum2() symbol
by using g_module_open()
Summary of required system environment:
- Needs Windows 10 (probably at least RS 1 update)
- GPU should support ExtendedNV12SharedTextureSupported feature
- Desktop application only (UWP is not supported yet)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1903>
Move d3d11 device, memory, buffer pool and minimal method
to gst-libs so that other plugins can access d3d11 resource.
Since Direct3D is primary graphics API on Windows, we need
this infrastructure for various plugins can share GPU resource
without downloading GPU memory.
Note that this implementation is public only for -bad scope
for now.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/464>
Add the helper function _get_surface_id() which extracts the
VASurfaceID from the passed picture. This function gets the surface of
the next and previous reference picture.
Instead of if-statements, this refactor uses a switch-statement with a
fall-through, for P-type pictures, making the code a bit more readable.
Also it adds quirks for gallium driver, which cannot handle invalid
surfaces as forwarding nor backwarding references, so the function fails.
Also iHD cannot handle them, but to avoid failing, the current picture
is used as self-reference.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1939>
When missing the reference frames, we should not just discard the current
frame. Some streams have group of picture header. It is an optional header
that can be used immediately before a coded I-frame to indicate to the decoder
if the first consecutive B-pictures immediately following the coded I-frame can
be reconstructed properly in the case of a random access.
In that case, the B frames may miss the previous reference and can still be
correctly decoded. We also notice that the second field of the I frame may
be set to P type, and it only ref its first field.
We should not skip all those frames, and even the frame really misses the
reference frame, some manner such as inserting grey picture should be used
to handle these cases.
The driver crashes when it needs to access the reference picture while we set
forward_reference_picture or backward_reference_picture to VA_INVALID_ID. We
now set it to current picture to avoid this. This is just a temp manner.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1929>
The behavior for zero AVEncMPVGOPSize value would be
varying depending on GPU vendor implementation and some
GPU will produce keyframe only once at the beginning of encoding.
That's unlikely expected result for users.
To make this property behave consistently among various GPUs,
this commit will change default value of "gop-size" property to -1
which means "auto". When "gop-size" is unspecified, then
mfvideoenc will calculate GOP size based on framerate
like that of our x264enc implementation.
See also
https://docs.microsoft.com/en-us/windows/win32/directshow/avencmpvgopsize-property
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1911>
Add a new video source element "d3d11desktopdupsrc" for capturing desktop image
via Desktop Duplication based on Microsoft's Desktop Duplication sample available at
https://github.com/microsoft/Windows-classic-samples/tree/master/Samples/DXGIDesktopDuplication
This element is expected to be a replacement of existing dxgiscreencapsrc
element in winscreencap plugin.
Currently this element can support (but dxgiscreencapsrc cannot)
- Copying captured D3D11 texture to output buffer without download
- Support desktop session transition
e.g., can capture desktop without error even in case that
"Lock desktop" and "Permission dialog"
- Multiple d3d11desktopdupsrc elements can capture the same monitor
Not yet implemented features
- Cropping rect is not implemented, but that can be handled by downstream
- Mult-monitor is not supported. But that is also can be implemented by
downstream element for example via multiple d3d11desktopdup elements
with d3d11compositor
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1855>
Hide most of symbols of GstD3D11Memory object.
GstD3D11Memory is one of primary resource for imcoming d3d11 library
and it's expected to be a extensible feature.
Hiding implementation detail would be helpful for later use case.
Summary of this commit:
* Now all native Direct3D11 resources are private of GstD3D11Memory.
To access native resources, getter methods need to be used
or generic map (e.g., gst_memory_map) API should be called
apart from some exceptional case such as d3d11decoder case.
* Various helper methods are added for GstBuffer related operations
and in order to remove duplicated code.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1892>
... instead of READY state. READY state is too early for setting
overlay window handle especially playbin/playsink scenario
since playsink will set given overlay handle on videosink once
READY state change of videosink is ensured.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1893>
Unlike software MFT (Media Foundation Transform) which is synchronous
in terms of processing input and output data, hardware MFT works
in asynchronous mode. output data might not be available right after
we pushed one input data into MFT.
Note that async MFT will fire two events, one is "METransformNeedInput"
which happens when MFT can accept more input data,
and the other is "METransformHaveOutput", that's for signaling
there's pending data which can be outputted immediately.
To listen the events, we can wait synchronously via
IMFMediaEventGenerator::GetEvent() or make use of IMFAsyncCallback
object which is asynchronous way and the event will be notified
from Media Foundation's internal worker queue thread.
To handle such asynchronous operation, previous working flow was
as follows (IMFMediaEventGenerator::GetEvent() was used for now)
- Check if there is pending output data and push the data toward downstream.
- Pulling events (from streaming thread) until there's at least
one pending "METransformNeedInput" event
- Then, push one data into MFT from streaming thread
- Check if there is pending "METransformHaveOutput" again.
If there is, push new output data to downstream
(unlikely there is pending output data at this moment)
Above flow was processed from upstream streaming thread. That means
even if there's available output data, it could be outputted later
when the next buffer is pushed from upstream streaming thread.
It would introduce at least one frame latency in case of live stream.
To reduce such latency, this commit modifies the flow to be fully
asynchronous like hardware MFT was designed and to be able to
output encoded data whenever it's available. More specifically,
IMFAsyncCallback object will be used for handling
"METransformNeedInput" and "METransformHaveOutput" events from
Media Foundation's internal thread, and new output data will be
also outputted from the Media Foundation's thread.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1520>
Add a new property "render-stats" to allow rendering statistics
data on window for debugging and/or development purpose.
Text rendering will be accelerated by GPU since this implementation
uses Direct2D/DirectWrite API and Direct3D inter-op for minimal overhead.
Specifically, text data will be rendered on swapchain backbuffer
directly without any copy/allocation of extra texture.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1830>
