Commit graph

3925 commits

Author SHA1 Message Date
Seungha Yang
b70a192220 d3d11h264dec: Make use of max_dpb_size passed by baseclass
Most likely this modification can save GPU memory.
2020-03-08 16:03:35 +09:00
Seungha Yang
7c153b6e93 d3d11h265dec: Fix scaling list parsing
Scaling list can be signalled via sps or pps. Decoder should check
both of them.
2020-03-07 12:57:38 +09:00
Seungha Yang
cfb6283613 d3d11h265dec: Properly signal the ucNumDeltaPocsOfRefRpsIdx
ucNumDeltaPocsOfRefRpsIdx should be the NumDeltaPocs[RefRpsIdx].
2020-03-07 12:20:03 +09:00
Seungha Yang
83b056cb77 h265decoder: Port to nal type classification macro
... and remove namespaceless macro methods from baseclass
2020-03-05 23:22:39 +09:00
Nicolas Dufresne
020ee62c59 d3d11: Use gst_video_decoder_get_frame()
Now that the system_frame_number is saved on the pictures we can use
gst_video_decoder_get_frame() helper instead of getting the full list
and looping over it.
2020-03-05 03:06:16 +00:00
Nicolas Dufresne
c6b8157dd6 codecs: Pass the max_dpb_size to new_segment virtual
On new_segment, the decoder is expected to negotiate. The decoder may want to
pre-allocate the needed buffers. Pass the max_dpb_size as this is needed to
determin how many buffers should be allocated.
2020-03-05 03:06:16 +00:00
Nicolas Dufresne
00d04784d3 Move CODEC base classes into it's own library
This introduce a library which contains a set of base classes which
handles the parsing and the state tracking for the purpose of decoding
different CODECs. Currently H264, H265 and VP9 are supported. These
bases classes are used to decode with low level decoding API like DXVA,
NVDEC, VDPAU, VAAPI and V4L2 State Less decoders. The new library is
named gstreamer-codecs-1.0 / libgstcodecs.
2020-03-05 03:06:16 +00:00
Seungha Yang
be3de5a3db d3d11vp9dec: Add support for reverse playback
This commit moves parsing code for superframe and frame header into
handle_frame() method, and removes parse() implementation from vp9decoder
baseclass.

The combination of
- multiple frames are packed in a given input buffer (i.e., superframe)
- reverse playback
seems to be complicated and also it doesn't work as intended in some case
2020-03-03 12:15:33 +00:00
Matthew Read
3e2d86ea18 openslessink: Allow openslessink to handle 48kHz streams.
The most common audio sample rate in AV streams is 48kHz, and the most
common device output sample rate is 48kHz. This allows handing of 48kHz
input streams without resampling.

Remove comments about avoiding the use of 48kHz.
2020-03-03 08:59:01 +00:00
Heinrich Kruger
03cf2d42a5 decklink: Add support for 2K DCI video modes
Extend the video modes supported by the decklink plugin to include 2K
DCI video modes.
2020-03-03 08:05:56 +00:00
Heinrich Kruger
796fffa441 decklink: Update Decklink SDK to 10.11.4
This change is needed to support 2K DCI video modes.

Version 10.8 of the Decklink SDK supported DCI video modes for output
only. This updated version drops that restriction.

The current latest version of the Decklink SDK is 11.5, however
the gstreamer decklink plugin is not compatible with API changes
introduced in version 11 of the SDK. Therefore I have opted to upgrade
to the latest 10.x version instead.
2020-03-03 08:05:56 +00:00
Seungha Yang
9b7c20bfca d3d11videosink: Clarify the meaning of various width and height variables
* Remove redundant variables for width/height and par from GstD3D11Window.
  GstVideoInfo holds all the values.
* Don't need to pass par to gst_d3d11_window_prepare().
  It will be parsed from caps again
* Remove duplicated math

Fixing regression of the commit 9dada90108
2020-03-02 22:01:43 +09:00
Yeongjin Jeong
25e9ee10b0 d3d11vp9dec: Port to GstVP9SuperframeInfo
The vp9parser is now exposed new API for parsing superframe info.
2020-03-02 01:55:36 +09:00
Seungha Yang
9575d835ba d3d11decoder: Do not print warning message if retry count is in expected range
gst_d3d11_result() will print warning message when HRESULT != S_OK.
However, since the retry is trivial stuff, check hr == E_PENDING first
and do not warn it.
2020-02-28 13:21:04 +09:00
Seungha Yang
418e6991c1 d3d11decoder: Check decoder status report
... and if h/w decoder reports error, increase error count.
2020-02-28 13:21:00 +09:00
Seungha Yang
f2e322197e d3d11window: Set DXGI_PRESENT_ALLOW_TEARING only in fullscreen mode
The DXGI_PRESENT_ALLOW_TEARING flag might cause unexpected tearing
side effect. Setting it in fullscreen mode only seems to be
the correct usage as in the Microsoft's direct3d examples.
2020-02-26 13:18:16 +09:00
Seungha Yang
4643dc1c12 d3d11decoder: Ensure the written bitstream buffer size is 128 bytes aligned
DXVA spec is saying that the size of bitstream buffer provided by hardware decoder
should be 128 bytes aligned. And also the host software decoder should
align the size of written buffer to 128 bytes. That means if the slice
(or frame in case of VP9) size is not aligned with 128 bytes,
the rest of non 128 bytes aligned memory should be zero-padded.

In addition to aligning implementation, some variables are renamed
to be more intuitive by this commit.
2020-02-24 14:46:27 +00:00
Haihao Xiang
acc14225c9 msdk: libva: Don't set the hint if MFX_VERSION is lower than 1025
MFX_MEMTYPE_VIDEO_MEMORY_ENCODER_TARGET is defined since MFX_VERSION
1025.
2020-02-24 01:26:58 +00:00
Seungha Yang
fbe7917a94 d3d11decoder: Add padding space on decoder output view when it's not aligned
Most H/W decoders have required alignment and dxva is also the case.
2020-02-20 17:32:42 +09:00
Seungha Yang
fe72bf6053 d3d11decoder: Register elements per GPU device with capability check
This implementation is similar to what we've done for nvcodec plugin.
Since supported resolution, profiles, and formats are device dependent ones,
single template caps cannot represent them, so this modification
will help autoplugging and fallback.

Note that the legacy gpu list and list of resolution to query were
taken from chromium's code.
2020-02-18 11:58:45 +00:00
Seungha Yang
13586bc77a d3d11device: Fix typo
s/vender/vendor
2020-02-18 11:58:45 +00:00
Seungha Yang
8ead80eecd d3d11device: Adjust debug level for when _new() fails
gst_d3d11_device_new might be used to enumerate device.
2020-02-18 11:58:45 +00:00
Seungha Yang
36fb790243 d3d11videosink: Ensure upload staging texture to fallback render texture
gst_video_frame_copy will copy input frame to stating texture
of fallback frame. Then, we need to map fallback texture with GST_MAP_D3D11
flag to upload the staging texture to render texture. Otherwise
the render texture wouldn't be updated.
2020-02-16 21:29:08 +09:00
Seungha Yang
9bf4746e2f d3d11decoder: Fix copying decoder view to staging
Source texture (decoder view) might be larger than destination (staging) texture.
In that case, D3D11_BOX structure should be passed to CopySubresourceRegion method
in order to specify the exact target area.
2020-02-13 21:25:15 +09:00
Seungha Yang
f6cdb91f55 d3d11window: Fix for broken dirty rect drawing on Windows 7
DXGI_SWAP_EFFECT_DISCARD cannot be used with dirty rect drawing feature
of IDXGISwapChain1::Present().
Note that IDXGISwapChain1 interface is available on Platform Update for Windows 7
and DXGI_SWAP_EFFECT_FLIP_SEQUENTIAL is also the case.
2020-02-12 22:38:53 +09:00
Seungha Yang
4383b387b7 d3d11window: Fix for dxva decoder output view rendering
Use resolution specified in caps for input_rect instead of
passed width and height value. The width and height might be modified
ones by d3d11videosink, then frame resolution might be different.
2020-02-12 12:34:58 +00:00
Seungha Yang
a39a5bf131 d3d11decoder: Refactor decoding process
* Move decoding process to handle_frame
* Remove GstVideoDecoder::parse implementation
* Clarify flush/drain/finish usage

In forward playback case, have_frame() call will be followed by
handle_frame() but reverse playback is not the case.
To ensure GstVideoCodecFrame, the decoding process should be placed inside
of handle_frame(), instead of parse().

Since we don't support alignment=nal, the parse() implementation is not worth.
In order to fix broken reverse playback, let's remove the parse()
implementation and revisit it when adding alignment=nal support.
2020-02-12 12:34:58 +00:00
Seungha Yang
3e78afbe0a d3d11decoder: Move handle_frame implementation to baseclass
... and remove unused start, stop method from subclass.

Current implementation does not require subclass specific behavior
for the handle_frame() method.
2020-02-12 12:34:58 +00:00
Seungha Yang
6da90b59f4 d3d11videosink: Remove max size condition from pool
Actually our buffer pool size and the number of backbuffer are
independent. In case of reverse playback, upstream might request
a lot of buffers (up to GOP size).
2020-02-12 12:34:58 +00:00
Nirbheek Chauhan
266dc41596 nvcodec: Mark class data as may-be-leaked to quiet the leaks tracer
The class data with the caps in it will be leaked if the element is
registered but never instantiated. There is no way around this. Mark
the caps as such so that the leaks tracer does not warn about it.

This is the same as pad template caps getting leaked, which are also
marked as may-be-leaked. These objects are initialized exactly once,
and are 'global' data.
2020-02-12 00:00:51 +05:30
Haihao Xiang
50ae5061f9 msdkvp9enc: output raw vp9 stream instead of IVF stream
video/x-vp9 is required in the src pad, however the output includes a
IVF header, which makes the pipeline below doesn't work

  gst-launch-1.0 videotestsrc ! msdkvp9enc ! msdkvp9dec ! fakesink

Since mfx 1.26, the VP9 encoder supports bitstream without IVF header,
so in this patch, the mfx version is checked and msdkvp9enc is enabled
only if mfx 1.26+ is available
2020-02-10 06:46:28 +00:00
Jan Schmidt
6c1e5ab311 androidmedia: Support float i-frame-interval
Android 25 added support for i-frame-interval to be a floating
point value. Store the property as a float and use the newer
version when it's available.
2020-02-09 02:19:12 +11:00
Jan Schmidt
29e3d09014 androidmedia: Allow dynamic bitrate changes on Android >= 19
Android 19 added an API for dynamically changing the bitrate in a running
codec.

Also make it so that even when not update-able at runtime, parameters will at least
be stored so that they take effect the next the codec is restarted.
2020-02-09 02:19:12 +11:00
Jan Schmidt
1b8bf1be01 androidmedia: Handle force-keyunit requests
Use API from Android 19 to request a keyframe from the MediaCodec
when indicated by the base class.
2020-02-09 02:19:12 +11:00
Jan Schmidt
cfe318ea03 androidmedia: Permit Codec surface to be NULL
The AMC encoder wrapper doesn't support input surfaces yet,
and passes NULL when configuring the underlying codec.

This was broken in commit 7fcf3e
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1102
2020-02-09 02:19:12 +11:00
Nirbheek Chauhan
3ca87d9988 nvcodec: Fix crash in decoder on 32-bit Windows
Same fix as 1a7ea45ffd, but I didn't
test the decoder so I missed that the function pointers here weren't
using the correct calling convention too.
2020-02-06 13:39:52 +00:00
Seungha Yang
478fb29974 d3d11window: Clear cached buffer per new caps
d3d11window holds one buffer to redraw client area per resize event.

When the input format is being changed, this buffer should be cleared
to avoid mismatch beween newly configured shader/videoprocessor and
the format of previously cached buffer.
2020-02-06 00:46:46 +09:00
Sebastian Dröge
5b8ff98f96 nvdec: Don't leak template caps when registering elements with old NVIDIA driver 2020-02-05 09:49:20 +00:00
Seungha Yang
371b181292 d3d11decoder: Use consistent resolution between output caps and video meta
h264/h265 decoded buffer might have crop area then we need to
adjust video meta based on the padding space
2020-02-05 00:52:48 +00:00
Seungha Yang
2f32f30b62 d3d11memory: Add a method to specify padding space 2020-02-05 00:52:48 +00:00
Seungha Yang
f852ce01e4 d3d11decoder: Add support for zero-copy playback
When downstream support d3d11 memory with forward playback case,
expose decoder output view memory objects without copying.
2020-02-05 00:52:48 +00:00
Seungha Yang
69f7f958a0 d3d11decoder: Create decoder output view whenever it's required
Whatever the reason, buffer in pool might be freed then we need to
configure decoder output views again.
2020-02-05 00:52:48 +00:00
Seungha Yang
567575e33d d3d11decoder: Need to zero initilized for g_once
A vairable to be used for g_once, it should be zero initialized
2020-02-05 00:52:48 +00:00
Seungha Yang
8ff667e463 d3d11allocator: Work as if buffer pool when running on texture array mode
Because the size of texture array cannot be updated dynamically,
allocator should block the allocation request. This cannot be
done at buffer pool side if this d3d11 memory is shared among
multiple buffer objects. Note that setting NO_SHARE flag to
d3d11 memory is very inefficient. It would cause most likey
copy of the d3d11 texture.
2020-02-05 00:52:48 +00:00
Seungha Yang
4d02858f66 d3d11videosink: Fix fallback buffer copy
Since we don't use dynamic texture now, cpu access to the fallback
texture should not happen.
2020-02-05 00:52:48 +00:00
Seungha Yang
2aa9f0bd6c d3d11window: Don't create swapchain again per caps change
Creating swapchain is relatively heavy operation. If output dxgi format
is not being chagned, we don't need to destroy and create swachain again.
2020-02-05 00:52:48 +00:00
Seungha Yang
c1d2d9171d d3d11window: Invoke initial resize method from baseclass
... instead of calling from subclass in order for baseclass to handle
more things between swapchain creation and resource creation.
2020-02-05 00:52:48 +00:00
Seungha Yang
7aad9187e4 d3d11videosink: Use ID3D11VideoProcessor interface
...for color space conversion if available

ID3D11VideoProcessor is equivalent to DXVA-HD video processor
which might use specialized blocks for video processing
instead of general GPU resource. In addition to that feature,
we need to use this API for color space conversion of DXVA2 decoder
output memory, because any d3d11 texture arrays that were
created with D3D11_BIND_DECODER cannot be used for shader resource.

This is prework for d3d11decoder zero-copy rendering and also
for conditional HDR tone-map support.
Note that some Intel platform is known to support tone-mapping
at the driver level using this API on Windows 10.
2020-02-05 00:52:48 +00:00
Seungha Yang
122a9b93eb d3d11: Add video processor object
ID3D11VideoProcessor interface provides various image conversion
methods. Note that it's analogous to VAAPI VPP.
2020-02-05 00:52:48 +00:00
Seungha Yang
a967db3b20 d3d11format: Add util methods for mapping DXGI color space with ours
Move color space mapping and hdr10 metadata conversion methods to
d3d11format in order to reuse the code.
2020-02-05 00:52:48 +00:00
Seungha Yang
3f4a84bd32 nvenc: Query maximum supported API version
We've been using NvEncodeAPICreateInstance method to find the supported API
version, but that seems to be insufficient since there is a case
where plugin failed in opening encoding session even if NvEncodeAPICreateInstance
succeeded. Asking driver about the version would be the most certain way.
2020-02-03 14:15:28 +00:00
Nicolas Dufresne
115b65d1ab kmssink: Fix crash with force-modesetting=1
This is a master regression, we would allocate a bo without having
created the allocator yet. As of now, we lazily create the allocator.
2020-01-31 10:01:51 -05:00
Seungha Yang
5d160a8968 msdkdec: Fix GstMsdkContext leak 2020-01-29 15:35:21 +09:00
Seungha Yang
a8fff639c8 msdk: Clear reference counted object in dispose() method
Follow GObject's memory management model
2020-01-29 15:35:21 +09:00
Haihao Xiang
ac9c9d8efc msdk: use cached response for DMABuf when the frame size is same
User is seeing corrupted display when running `videotestsrc !
video/x-raw,format=NV12,width=xxx,height=xxx ! msdkh265enc ! msdkh265dec
! glimagesink` with changed frame size, e.g. from 1920x1080 to 1920x240

The root cause is a same dmabuf fd is used for frames with
different size, which causes some unexpected result. This patch requires
cached response is used for frames with same size only for DMABuf, so a
dmabuf fd can't be used for frames with different size any more.
2020-01-28 03:46:04 +00:00
Seungha Yang
e97ef8a562 d3d11window_win32: Let DXGI choose client area
Don't specify the resolution of backbuffer. Then dxgi will let us know the
actual client area. When upstream resolution is chagned, updating the size
of backbuffer without the consideration for client size would cause mismatch
between them.
2020-01-26 12:13:24 +00:00
Nicolas Dufresne
d393232bc2 nvdec: Do not map GStreamer discont to CUVid discont
Setting the CUVID_PKT_DISCONTINUITY implies clearing any past information
about the stream in the decoder. The GStreamer discont flag is used for
discontinuity caused by a seek, for first buffer and if a buffer was
dropped. In the first two cases, the parsers and demuxers should ensure we
start from a synchronization point, so it's unlikely that delta will be
matched against the wrong state.

For packet lost, the discontinuity flag will prevent the decoder from doing
any concealment, with a result that ca be much worst visually, or freeze the
playback until an IDR is met. It's better to let the decoder handle that for
us.

Removing this flag, also workaround a but in NVidia parser that makes it
ignore our ENDOFFRAME flag and increase the latency by one frame.
2020-01-25 13:39:03 +00:00
Nicolas Dufresne
a28ce16b3f nvdec: Tell the parser we have complete pictures
This sets the CUVID_PKT_ENDOFPICTURE flag in order to inform the decoder that
we have a complete picture. This should remove one frame latency otherwise
introduce by NVidia parser.
2020-01-25 13:39:03 +00:00
Seungha Yang
20d85c95d0 d3d11window_corewindow: Always call methods of CoreWindow interface from UI thread
Like swapchain panel implementation, most methods of CoreWindow
should be called from UI thread.
2020-01-23 03:49:19 +00:00
Haihao Xiang
e3ebebc20b msdk: Fix compiler warning
This patch fixed compiler warning below:

[1/4] Compiling C object 'sys/msdk/dc44ea0@@gstmsdk@sha/gstmsdkvpp.c.o'.
../../gst-plugins-bad/sys/msdk/gstmsdkvpp.c: In function
‘gst_msdkvpp_context_prepare’:
../../gst-plugins-bad/sys/msdk/gstmsdkvpp.c:214:7: warning: suggest
parentheses around operand of ‘!’ or change ‘&’ to ‘&&’ or ‘!’ to ‘~’
[-Wparentheses]
2020-01-21 18:37:10 +00:00
Seungha Yang
a10f26aa3a nvenc: Do not access to broken encode session
If an encode session failed in initializing, the encode
session would be broken and the next nvenc API will cause crash.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1179
2020-01-21 16:34:41 +09:00
Nirbheek Chauhan
1bcf44bb1f msdk: Fix increasing memory usage in dynamic pipelines
Our context is non-persistent, and we propagate it throughout the
pipeline. This means that if we try to reuse any gstmsdk element by
removing it from the pipeline and then re-adding it, we'll clone the
mfxSession and create a new gstmsdk context as a child of the old one
inside `gst_msdk_context_new_with_parent()`.

Normally this only allocates a few KB inside the driver, but on
Windows it seems to allocate tens of MBs which leads to linearly
increasing memory usage for each PLAYING->NULL->PLAYING state cycle
for the process. The contexts will only be freed when the pipeline
itself goes to `NULL`, which would defeat the purpose of dynamic
pipelines.

Essentially, we need to optimize the case in which the element is
removed from the pipeline and re-added and the same context is re-set
on it. To detect that case, we set the context on `old_context`, and
compare it to the new one when preparing the context. If they're the
same, we don't need to do anything.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/946
2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
6834a12112 msdk: Reorganize context preparation code
Split it out into a separate function with early exits to make the
flow clearer, and document what the function is doing clearly.
No functional changes.
2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
e83d5fd8d6 msdk: Fix warning about unused variable on Windows 2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
c0d778c2c7 msdk: Use gst_clear_object()
`gst_object_replace()` is not supposed to be used for unreffing and
NULLing objects.
2020-01-21 00:38:41 +00:00
Nirbheek Chauhan
bda687344b nvcodec: Print debug info when initializing nvenc
We weren't printing the return value.
2020-01-20 17:15:55 +05:30
Nirbheek Chauhan
1a7ea45ffd nvcodec: Fix crash on 32-bit Windows
We weren't using the correct calling convention when calling CUDA and
CUVID APIs. `CUDAAPI` is `__stdcall` on Windows. This was working fine
on x64 because `__stdcall` is ignored and there's no special calling
convention. However, on x86, we need to use `__stdcall`.
2020-01-20 17:15:55 +05:30
Nirbheek Chauhan
7e93ae0638 nvcodec: cuda.h only needs glib.h, not gst.h
Just a nitpick. Also, force the compiler to use our stub header
instead of searching for it in the include paths.
2020-01-20 15:10:51 +05:30
Josep Torra
ef37446460 msdkdec: align frame list using decoded timestamp
Before this change decoder used the oldest frame in the list to pair it
with the decoded surface. This only works when there's a perfect stream
like HEADERS,SYNCPOINT,DELTA...

When playing RTSP streams we can get imperfect streams like HEADERS,
DELTA,SYNCPOINT,DELTA... In this case decoder drops the frames
between HEADERS and SYNCPOINT which leads into using wrong PTS on
the output frames.

With this change we inject the input PTS into the bitstream and use it
to align the internal frame list with the actually decoded position.

Fixes playback with:
```
gst-launch-1.0 rtspsrc location=... latency=0 drop-on-latency=1 ! ...
```
2020-01-19 04:26:15 +00:00
Seungha Yang
e8d527df93 nvenc: Query supported minimum resolution
Hard-coded 16x16 resolution is likely to differ from the device's support
in most cases. If we can use NV_ENC_CAPS_WIDTH_MIN and NV_ENC_CAPS_HEIGHT_MIN,
update pad template with returned value.
2020-01-16 15:24:03 +00:00
Seungha Yang
58a663c1e5 nvcodec: Bump SDK header to version 9.1
Update header to query minimum resolution of encoder and to control
the number of reference frame if it's supported
2020-01-16 15:24:03 +00:00
Haihao Xiang
52abbeeefb msdkdec: support transform_meta virtual method
Allow the base class to copy GstVideoRegionOfInterestMeta data to the
output buffer when calling gst_video_decoder_finish_frame
2020-01-15 00:47:12 +00:00
Haihao Xiang
8512624a41 msdkenc: set ROI region for msdk{h264, h265}enc
A reconfig is needed when ROI is changed, otherwise the ROI parameters won't
take effect
2020-01-15 00:47:12 +00:00
Haihao Xiang
84e234a8c7 msdkenc: add set_extra_params virtual method
set_extra_params is added to allow sub class to add extra mfx parameters
for changed input frame or meta data
2020-01-15 00:47:12 +00:00
Haihao Xiang
0e2fc39f24 msdkenc: add need_reconfig virtual method
need_reconfig is added to allow sub class requires a reconfig when
the input frame or the MetaData (e.g. GstVideoRegionOfInterestMeta)
attached to the input frame is changed.
2020-01-15 00:47:12 +00:00
Seungha Yang
6e73e762c5 d3d11h265dec: Fix wrong NoRaslOutputFlag setting
... and handle EOS and EOB nals.

Only the first CRA picture should be associated with NoRaslOutputFlag
as the comment in code.
2020-01-14 08:47:30 +00:00
Nirbheek Chauhan
b7d91711f7 ipcpipeline: Minimal fixes that allow building with MSVC 2020-01-14 09:23:02 +05:30
Nirbheek Chauhan
c8db7a9127 ipcpipeline: Rework compiler checks
`pipe()` isn't used since 15927b6511,
and `socketpair()` from `#include <sys/socket.h>` is used only in the
examples. In practice, you can use probably also use anything that
allows you to create fd pairs, such as named pipes or anonymous pipes.

We use the cross-platform GstPollFD API in the plugin.
2020-01-14 09:23:02 +05:30
Seungha Yang
96f0f4b613 d3d11memory: Always use native DXGI format if device support it
Use consistent memory layout between dxva and other shader use case.
For example, use DXGI_FORMAT_NV12 texture format instead of
two textures with DXGI_FORMAT_R8_UNORM and DXGI_FORMAT_R8G8_UNORM.
2020-01-13 01:58:08 +00:00
Seungha Yang
9ee40679b5 Revert "d3d11: Add support for D3D11_USAGE_DYNAMIC"
This reverts commit ddd13fc7c0

Dynamic usage can reduce the number of copy per frame but make
things complicated and the benefit seems to not significant.
Also since we don't provide _map() method for the dynamic usage,
application cannot read buffers which make "last-sample" property
unusable in case of d3d11videosink.
2020-01-13 01:58:08 +00:00
Seungha Yang
e188893963 d3d11: Don't register decoders if unavailable
DXVA requires a hardware interface but may not be available,
such as in the case of VMs or when the GPU vendor does not provide a decoder interface.
2020-01-11 17:42:04 +09:00
Seungha Yang
616082d14a d3d11decoder: Don't return not initialized object from _new() method
... and change some debug levels since initialization failure might not be fatal.
2020-01-11 17:42:04 +09:00
Nicolas Dufresne
6d6e897762 decklink: Silence no driver / no SO messages
This g_once() is called everywhere, even in provider. This cause
spurious error when device monitor is used. Just silence or remove
the spurious logs.
2020-01-10 08:00:05 +00:00
Haihao Xiang
bdf9a76226 Revert "msdkdec: make sure to use video memory on Linux"
Commit a1584b6 caused big performance drop if the downstream element
is not a msdk element because it is very slow to read data from video
memory directly.

This reverts commit a1584b6f99.
2020-01-10 01:36:40 +00:00
Seungha Yang
49a1f022fd d3d11colorconverter: Handle P016_LE format
P016 format is no different than P010. Not much things to add code.
2020-01-09 16:29:47 +00:00
Seungha Yang
bbab229905 d3d11colorconverter: Add support for YUV to YUV conversion 2020-01-09 16:29:47 +00:00
Seungha Yang
a122f305f9 d3d11colorconverter: Add support for RGB to YUV conversion
... and remove code for RGBx since it's not supported format
by our d3d11 implementation for now.
2020-01-09 16:29:47 +00:00
Seungha Yang
da50cc4d92 d3d11format: Remove invalid format from supported format list 2020-01-09 16:29:47 +00:00
Sebastian Dröge
e4389146d9 decklinkvideosink: Always configure 10 bit YUV for VANC frames
If 8 bit are required by the device/mode then it will be converted internally
by the SDK, but the SDK won't automatically convert from 8 to 10 bit. As
such, always use 10 bit VANC.

Some devices require configuring also a 10 bit video format when using
10 bit VANC is required but those would fail regardless and the
application would have to configure the correct video format.

With newer versions of the SDK this information can be retrieved via the
BMDDeckLinkVANCRequires10BitYUVVideoFrames flag but we don't use a new
enough SDK version yet to extract this information.
2020-01-09 12:54:04 +02:00
Seungha Yang
e4daa2ef43 d3d11: Add support for Universal Windows Platform
Initial UWP support via new window (CoreWindow and SwapChainPanel) implementation.
2020-01-06 20:14:51 +09:00
Seungha Yang
43a8eb9e92 d3d11decoder: Fix build on non-desktop target
Although the target platform of D3D11 decoding API are both desktop and UWP app,
DXVA header is blocked by "WINAPI_FAMILY_PARTITION(WINAPI_PARTITION_DESKTOP)"
which is meaning that that's only for desktop app.
To workaround this inconsistent annoyingness, we need to define WINAPI_PARTITION_DESKTOP
regardless of target WinAPI partition.
2020-01-06 18:45:56 +09:00
Haihao Xiang
9064de27f9 msdkvp9enc: fix 10bit encoding
The codec profile should be consistent with the frame fourcc code, this
fixes pipeline below:

gst-launch-1.0 videotestsrc ! \
video/x-raw,width=320,height=240,format=P010_10LE ! msdkvp9enc ! \
fakesink
2020-01-06 03:54:57 +00:00
Haihao Xiang
da22ff3846 msdkvp9enc: fix width and height
The frame width and height is rounded up to 128 and 32 since commit
8daac1c, so the width, height for initialization should be rounded up to
128 and 32 too because the MSDK VP9 encoder will do some check on width
and height.

Sample pipeline:
gst-launch-1.0 videotestsrc ! \
video/x-raw,width=320,height=240,format=NV12 ! msdkvp9enc ! fakesink
2020-01-06 03:54:57 +00:00
Haihao Xiang
d6fbec5a44 msdk: make sure the found response meets the request
Otherwise the re-used msdk response doesn't have enough frames so it is
possible the pipeline will be broken
2020-01-06 02:16:53 +00:00
Julien Isorce
f578eab6fd msdkenc: renegotiate when video info changes
Renegotiation was implemented for bitrate change. We can re-use
the same sequence when video info changes except that this can be
executed right away when receiving the new input format. I.e. no
need to wait for the next call to handle_frame.
2020-01-06 01:13:28 +00:00
Philippe Normand
9f1fbd3649 decklink: Fix crash when probing without driver
If there is no decklink hardware/driver, the devices list is empty (NULL), so
this needs to be checked before iterating over the list.
2020-01-05 16:20:18 +00:00
Julien Isorce
4e6a1b9634 msdkdec: trigger renegotiation if video info changes
Useful when framerate changes. Previously it was only checking
for resolution change but renego should happen if any video
info changes.
2020-01-03 10:19:57 -08:00
Julien Isorce
a1584b6f99 msdkdec: make sure to use video memory on Linux
The block that sets use_video_memory flag is after the
the condition `if gst_msdk_context_prepare` but it
always returns false when there is no other msdk elements.
So the decoder ends up with use_video_memory as FALSE.
Note that msdkvpp always set use_video_memory as TRUE.

When use_video_memory is FALSE then the msdkdec allocates
the output frames with posix_memalign (see gstmsdksystemmemory.c).
The result is then copied back to the GstVideoPool's buffers
(or to the downstream pool's buffers if any).
When use_video_memory is TRUE then the msdkdec uses vaCreateSurfaces
to create vaapi surfaces for the hw decoder to decode into
(see gstmsdkvideomemory.c). The result is then copied to either
the internal GstVideoPool and to the downstream pool if any.
(vaDeriveImage/vaMapBuffer is used in order to read the surfaces)
2020-01-03 07:08:23 +00:00
Seungha Yang
b88af2e4fe msdkdec: Correct return value of GstVideoDecoder::flush()
Use boolean instead of GstFlowReturn as declared.
Note that since base class does not check return value of GstVideoDecoder::flush(),
this would not cause any change of behavior.
2020-01-02 12:22:07 +09:00
Haihao Xiang
a819f05851 msdkdec: free unlocked msdk surface before output buffer allocation
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/merge_requests/924
is trying to use video memory for decoding on Linux, which reveals a
hidden bug in msdkdec.

For video memory, it is possible that a locked mfx surface is not used
indeed and it will be un-locked later in MSDK, so we have to check the
associated MSDK surface to find out and free un-used surfaces, otherwise
it is easy to exhaust all pre-allocated mfx surfaces and get errors below:

0:00:00.777324879 27290 0x564b65a510a0 ERROR                default
gstmsdkvideomemory.c:77:gst_msdk_video_allocator_get_surface: failed to
get surface available
0:00:00.777429079 27290 0x564b65a510a0 ERROR         msdkbufferpool
gstmsdkbufferpool.c:260:gst_msdk_buffer_pool_alloc_buffer:<msdkbufferpool0>
failed to create new MSDK memory

Note the sample code in MSDK does similar thing in
CBuffering::SyncFrameSurfaces()
2020-01-02 00:43:36 +00:00
Seungha Yang
a139c8c7e8 d3d11: Add h265 decoder element
Some DPB management implementation is taken from gstreamer-vaapi
2019-12-31 02:13:48 +00:00
Seungha Yang
0e7b6526b8 d3d11: Add vp9 decoder element
Based on gstreamer-vaapi and Chromium implemetation.
2019-12-31 02:13:48 +00:00
Seungha Yang
586390b1ba d3d11: Add h264 decoder element
New decoder implementation based on dxva2 on d3d11 APIs. The DPB
management implementation is taken from Chromium.
2019-12-31 02:13:48 +00:00
Seungha Yang
7bfdeaf161 d3d11: Add support for Array typed texture memory
A ID3D11Texture2D memory can consist of multiple planes with array.
For array typed memory, GstD3D11Allocator will allocate new GstD3D11Memory
with increased reference count to the ID3D11Texture2D but different array index.
2019-12-31 02:13:48 +00:00
Stéphane Cerveau
add7878e14 bad: use of g_value_dup_string
Use helper method to get string from GValue.
2019-12-30 14:13:03 +00:00
Seungha Yang
81dde0f5b8 d3d11: Aggregate d3d11 memory usage query for dynamic-usage type decision
Even if one of downstream d3d11 elements can support dynamic-usage memory,
another one might not support it. Also, to support dynamic-usage,
both upstream and downstream d3d11device must be the same object.
2019-12-28 05:43:44 +00:00
Seungha Yang
d731bcb18a d3d11colorconvert: Remove device type dependent behavior
If d3d11colorconvert element is configured, do color space conversion
regardless of the device type whether it's S/W emulation or real H/W.
Since d3d11colorconvert is no more a child of d3d11videosinkbin,
we don't need this behavior. Note that previous code was added to
avoid color space conversion from d3d11videosink if no hardware
device is available (S/W emulation of d3d11 is too slow).
2019-12-28 05:43:44 +00:00
Seungha Yang
46186356ca d3d11colorconverter: Fix unmatched lock/unlock pair 2019-12-28 05:43:44 +00:00
Seungha Yang
ab80c771df d3d11upload: Don't hard overwrite caps features to support d3d11 memory passthrough
d3d11upload should be able to support upstream d3d11 memory, not only system memory.

Fix for following pipeline
d3d11upload ! "video/x-raw(memory:D3D11Memory)" ! d3d11videosink
2019-12-28 05:43:44 +00:00
Seungha Yang
5298d95195 d3d11window: Make use of partial presentation with IDXGISwapChain1::Present1
Since we might draw on partial area of backbuffer in case of force-aspect-ratio,
presenting only updated area is more efficient way.

See also https://docs.microsoft.com/ko-kr/windows/win32/direct3ddxgi/dxgi-1-2-presentation-improvements
2019-12-24 20:10:54 +09:00
Seungha Yang
487a41d312 d3d11videosink: Add support for overlay composition
Add d3d11overlaycompositor object to draw overlay image
on render target using Blend method.
2019-12-24 19:00:45 +09:00
Seungha Yang
a5295509af d3d11: Enable dxgi debug layer and always try to pop d3d11/dxgi debug message if possible
Note that dxgi and d3d11 sdk debug will be enabled on debug build
2019-12-24 19:00:40 +09:00
Seungha Yang
49bccf0433 nvcodec: Refactor plugin initialization
Create CUDA context per device, instead of per codec and encoder/decoder.
Allocating CUDA context is heavy operation so we should reuse it
as much as possible.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1130
2019-12-24 08:10:14 +00:00
Seungha Yang
0e013fe539 d3d11videosink: Use dynamic texture for fallback buffer
Upload CPU memory to texture directly by using dynamic usage texture.
This will reduce at least one step of staging copy per frame.
2019-12-20 19:21:02 +09:00
Seungha Yang
65cace22e2 d3d11colorconvert: Set TRANSFER_NEED_DOWNLOAD flag to output memory for later cpu access
Otherwise CPU cannot access texture via gst_memory_map()
2019-12-20 19:21:02 +09:00
Seungha Yang
18e13404a2 d3d11colorconvert: Use shader resource and render target bind flags together
The output of d3d11colorconvert would be used for rendering (i.e., shader resource)
2019-12-20 19:21:02 +09:00
Seungha Yang
62c8d19e01 d3d11window: Change aspect-ratio mode from window thread
Call DXGI API from window thread as much as possible
2019-12-20 19:21:02 +09:00
Seungha Yang
1b687d556a d3d11videosink: Add support for full screen mode
borderless top-most style full screen mode support.
Basically fullscreen toggle mode is disabled by default. To enable it
use "fullscreen-toggle-mode" property to allow fullscreen mode change
by user input and/or property.
2019-12-20 19:21:02 +09:00
Seungha Yang
0788492461 d3d11: Use GRecMutex to protect immediate context and dxgi API call
In some cases, rendering and dxgi (e.g., swapchain) APIs should be
called from window message pump thread, but current design (dedicated d3d11 thread)
make it impossible. To solve it, change concurrency model to locking based one
from single-thread model.
2019-12-20 19:21:03 +09:00
Seungha Yang
a0a85cd80c d3d11window: Prefer to use flip-sequential over discard
flip-sequential is more efficient than discard
2019-12-20 11:15:12 +09:00
Seungha Yang
1424de0a4a d3d11window: Don't hold backbuffer interface
We don't need to hold it
2019-12-20 11:15:12 +09:00
Seungha Yang
32d618c677 d3d11window: Use CreateSwapChainForHwnd if available
That's recommended way from MS and CreateSwapChainForHwnd supports
more options than CreateSwapChain
2019-12-20 11:15:12 +09:00
Seungha Yang
9fd0b62f2d d3d11: Use configuration file for dxgi header version check 2019-12-20 11:15:12 +09:00
Seungha Yang
23b47a7ec1 d3d11window: Use allow-tearing mode if device supports it
As the recommendation from MS.
2019-12-20 11:15:12 +09:00
Seungha Yang
72b6b3557d d3d11window: Disable fullscreen mode change by alt + enter
Disable full screen mode change until proper handling is implemented
2019-12-20 11:15:12 +09:00
Seungha Yang
9dada90108 d3d11videosink: Remove resizing window hack and unify resizing flow
In earlier implementation of d3d11videosink where no shader was implemented,
the aspect ratio and render size were adjusted by manipulating the backbuffer size
with unintuitive formula. Since now we do color conversion and resize using
shader, we can remove the hack.
2019-12-20 11:15:12 +09:00
Seungha Yang
61cb6b2bbe d3d11: Pass GstD3D11Memory object to ensure_{shader_resource,render_target}_view methods
The method name indicates it should be d3d11memory, so passing
GstD3D11Memory seems to make more sense than GstMemory.
2019-12-20 11:15:12 +09:00
Seungha Yang
7c041f1934 d3d11videosink: Don't specify crop meta to allocation query
d3d11videosink could not handle it for now
2019-12-20 11:15:12 +09:00
Aaron Boxer
c8193b1615 d3dvideosink: hold class lock for entire duration of class destruction
This avoids a race condition currently when temporarily releasing the
lock and then re-acquiring.
2019-12-18 13:15:06 -05:00
Aaron Boxer
7cb6dbefba d3dvideosink: improve concurrency on hidden window create/destroy 2019-12-18 13:15:06 -05:00
Aaron Boxer
eff0117b5a d3dvideosink: use thread pool to handle events from hidden window event queue
window event queue now does not lock on the class lock, so we can now shut
it down without releasing the class lock, thus avoiding a potential race when
stopping the sink.
2019-12-18 13:15:06 -05:00
Aaron Boxer
027eb5ef20 d3dvideosink: improve concurrency on internal window create/destroy
Remove timeout and rely on condition variable instead to indicate thread
start.
2019-12-18 13:15:06 -05:00
Aaron Boxer
adfbce336a d3dvideosink: only warn about HWND already set if new HWND is not NULL 2019-12-18 13:15:06 -05:00
Aaron Boxer
84403dbae3 d3dvideosink: add more null checks 2019-12-18 13:15:05 -05:00
Aaron Boxer
06862d7ff5 d3dvideosink: check ref count and device before resetting display 2019-12-18 13:15:05 -05:00
Aaron Boxer
85c1550b92 d3dvideosink: use class lock when checking d3d formats, and cache format list 2019-12-18 13:15:05 -05:00
Aaron Boxer
d7af30bab9 d3dvideosink: destroy device just before final d3d release call
and free overlays after swap chain is released
2019-12-18 13:15:05 -05:00
Aaron Boxer
5e3520a302 d3dvideosink: add two forward declarations 2019-12-18 13:15:05 -05:00
Aaron Boxer
49211bdf74 d3dvideosink: use class lock when setting pool config 2019-12-18 13:15:05 -05:00
Aaron Boxer
fb6c2e8533 d3dvideosink: use class lock when allocating pool buffer 2019-12-18 13:15:05 -05:00
Aaron Boxer
52254ad31c d3dvideosink: use class lock when creating overlay textures 2019-12-18 13:15:05 -05:00
Aaron Boxer
51f823c98a d3dvideosinnk: check d3d device exists before creating overlay vertex buffer 2019-12-18 13:15:05 -05:00
Aaron Boxer
8710eb0656 d3dvideosink: pass hidden window hwnd to GST_DEBUG, rather than NULL 2019-12-18 13:15:05 -05:00
Aaron Boxer
54ce3d5011 d3dvideosink: fix typo 2019-12-18 13:15:05 -05:00
Seungha Yang
64eb1d46b5 d3d11window: Always draw to internal window
... and use SetParent() WIN32 API when external window is used.
Depending on DXGI swap effect, the external window might not be
reusable by another backend. To preserve the external window's property
and setting, drawing to internal window seems to be safer way.
2019-12-16 22:24:29 +09:00
Seungha Yang
638c6d6931 d3d11window: Always set error reason to GError object 2019-12-16 18:21:02 +09:00
Julien Isorce
93bc74284f msdkdec: call finalize on the parent class
Otherwise GstVideoDecoder is not finalized and
resources are leaked.

Somehow GST_TRACERS="leaks" GST_DEBUG="GST_TRACER:7" did not catch it.

Valgrind output:

==31645== 22,480 (1,400 direct, 21,080 indirect) bytes in 5 blocks are definitely lost in loss record 5,042 of 5,049
==31645==    at 0x4C2FB0F: malloc
==31645==    by 0x51D9E88: g_malloc
==31645==    by 0x51FA7B5: g_slice_alloc
==31645==    by 0x51FAC68: g_slice_alloc0
==31645==    by 0x58D9984: g_type_create_instance
==31645==    by 0x58BA344: g_object_new_with_properties
==31645==    by 0x58BADA0: g_object_new
==31645==    by 0x8ECA966: gst_video_decoder_init
==31645==    by 0x58D99E7: g_type_create_instance
==31645==    by 0x58BA344: g_object_new_with_properties
2019-12-13 15:36:26 -08:00
Roman Shpuntov
5da2938244 avfvideosrc: element requests camera permissions even with capture-screen property is true
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1137

macOS has different dialogs for camera capture and screen capture.
No need to request screen capture permissions, the system detect
screen capture automatically and create request dialog.
2019-12-11 17:31:03 +00:00
Seungha Yang
a400126889 d3d11window: Initialize surface rect with given external window handle
Unlike internal window, resize event might not happen with external window.
2019-12-11 23:04:55 +09:00
Seungha Yang
a8ec409b84 d3d11device: Report alive d3d11 objects at the end
It would be useful for debugging d3d11 object leak.
2019-12-11 23:04:49 +09:00
Seungha Yang
59295b4271 d3d11: Fix ID3D11SamplerState leak 2019-12-11 12:57:46 +00:00
Stéphane Cerveau
6bc0e9527e remove various useless linefeed in logs 2019-12-11 10:51:29 +01:00
Haihao Xiang
b18be3562c msdk: ignore MFX_ERR_UNDEFINED_BEHAVIOR when loading a plugin
An issue can be seen when using msdkh265enc with bitrate change in
playing state. The root cause is the corresponding plugin is loaded
again.

Returning MFX_ERR_UNDEFINED_BEHAVIOR from MSDK just means the plugin has
been loaded, so we may ignore this error when doing configuation again
in the sub class, otherwise the pipeline will be interrupted
2019-12-10 17:23:53 +00:00
Seungha Yang
d963d442c5 d3d11videosink: Fix broken fallback rendering
Make fallback rendering work
2019-12-10 04:06:28 +00:00
Seungha Yang
abc1443cf0 d3d11videosink: Add debug message for fallback reason 2019-12-10 04:06:28 +00:00
Seungha Yang
9835289239 d3d11window: Do not check shader resource view if not doing conversion
If d3d11window does not convert format internally, shader resource view
is not required. Note that shader resource view is used for
color conversion using shader but when conversion is not required,
we just copy input input texture to backbuffer.
2019-12-10 04:06:28 +00:00
Julien Isorce
b681d6f62e msdkvpp: check mfx surface nullity for the input buffer
In theory it should not happen but it happened to me
in some cases where it failed to allocate some video
buffers so this was a consequence of a corner case.
Better to be safe than sorry.
2019-12-10 03:00:13 +00:00
Seungha Yang
8154b22a88 d3d11window: Clear old swapchain per prepare
_prepare() might be called multiple times for given window, so clear
old swapchain if any.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1138
2019-12-10 10:58:28 +09:00
Julien Isorce
3bae199863 msdkdec: fix assertion 'frame->ref_count > 0' failed
Can happen if the oldest frame is the current frame
and if gst_video_decoder_finish_frame failed in which
case the current is unref and then drop instead of
just drop.
This patch also removes some assumptions, it was strange
to call unref and finish_frame in gst_msdkdec_finish_task.
In principle when owning a frame, the code should either
unref, or drop or finish.
2019-12-09 16:10:51 +00:00
Seungha Yang
5bf10d8781 d3d11window: Always set dxgi colorspace if possible
Not only for the 10 bits RGB case with HDR metadata, set proper color space
to swapchain in order to make color representaion correct
2019-12-09 12:22:15 +00:00
Matthew Waters
526afac736 vtdec: add support for outputing vulkan images 2019-12-09 01:49:31 +00:00
Matthew Waters
acac78c1ce applemedia: rename IOSurfaceMemory -> IOSurfaceGLMemory 2019-12-09 01:49:30 +00:00
Matthew Waters
0560946c82 videotexturecache: implement GL specifics as a subclass 2019-12-09 01:49:30 +00:00
Matthew Waters
ff5ba2f126 applemedia/videocache: make a GObject subclass 2019-12-09 01:49:30 +00:00
Matthew Waters
27f4fb060d avfvideosrc: don't leak texture caches
If decide_allocation is called multiple times, then we would continually
overwrite the previous cache without freeing
2019-12-09 01:49:30 +00:00
Aaron Boxer
e9891dfe14 d3d11: translate windows error codes to human readable strings
This will help with debugging.
2019-12-07 16:09:03 -06:00
Xavier Claessens
da584995f6 mlaudiosink: Fix crash when stopping pipeline
This is a bug in LuminOS 0.97.0.
2019-12-06 15:29:29 +00:00
Xavier Claessens
380ab8123b Add mlaudiosink element 2019-12-06 15:29:29 +00:00
Sebastian Dröge
f776e2df96 decklink: Return new references to the devices from get_devices()
Otherwise the caller will free the devices we store internally.
2019-12-05 21:22:18 +02:00
Seungha Yang
4eac9d0ae6 d3dvideosink: Fix warning for unhandled enum in switch
d3dhelpers.c:135:3: warning: enumeration value 'D3DFMT_D32_LOCKABLE' not handled in switch [-Wswitch]
2019-12-05 04:31:54 +00:00
Seungha Yang
199caccc41 d3d11: Initialize debug categories of non-GstElement implementation in plugin init
That's the way to expose debug category to --gst-debug-help
2019-12-05 02:29:18 +00:00
Seungha Yang
71befc477f d3d11window: Invoke resize task with high priority
On resize event, swapchain should be configured with higher priority
than the other tasks since it's directly related to visual artifacts.
2019-12-05 02:29:18 +00:00
Seungha Yang
d2b85683de d3d11device: Add gst_d3d11_device_thread_add_full method
I would be used to invoke GPU task with specified priority.
2019-12-05 02:29:18 +00:00
Seungha Yang
7cab47ddda d3d11videosinkbin: Drop d3d11 color convert element
d3d11videosink can convert color space now, so the conversion element
seems to be redundant.
2019-12-05 02:29:18 +00:00
Seungha Yang
ca3ddf7848 d3d11videosink: Add color conversion support
Draw to back buffer texture directly. It would reduce the number of
copy at least once when color conversion is required.
2019-12-05 02:29:18 +00:00
Seungha Yang
622733ed0d d3d11converter: Skip setup converter if input and output formats are equal 2019-12-05 02:29:18 +00:00
Seungha Yang
c9cb08fcd4 d3d11colorconvert: Split color space converter to reuse code 2019-12-05 02:29:18 +00:00
Seungha Yang
42ca123f0f d3d11: Split shader to reuse code 2019-12-05 02:29:18 +00:00
Seungha Yang
59d9589722 d3d11videosink: Draw window with cached texture on resize
This would render nicer than presenting scene with broken aspect ratio,
especially in case of low framerate.
2019-12-05 02:29:18 +00:00
Seungha Yang
ddd13fc7c0 d3d11: Add support for D3D11_USAGE_DYNAMIC
D3D11 dynamic texture is a special memory type, which is mainly used for
frequent CPU write access to the texture. For now, this texture type
does not support gst_memory_{map,unmap}
2019-12-05 02:29:18 +00:00
Seungha Yang
5ce4748cc0 d3d11device: Add debug message for CreateTexture2D failure 2019-12-05 02:29:18 +00:00
Seungha Yang
914d2df79e d3d11device: Fix misreading debug message 2019-12-05 02:29:18 +00:00
Seungha Yang
71ae632b25 d3d11colorconvert: Disable color conversion with software rasterizer
It's much slower than our CPU based color-converter in most case.
2019-12-05 02:29:18 +00:00
Seungha Yang
79e88b76da d3d11videosink: Handle context query
Propagate d3d11device via context query.
2019-12-05 02:29:18 +00:00
Seungha Yang
5edf64d655 d3d11colorconvert: Avoid copy in/out texture if possible
When input and/or output d3d11memory has its own view, use them for
rendering to avoid texture copy
2019-12-05 02:29:18 +00:00
Seungha Yang
da3545e8c8 d3d11memory: Allocate texture with required views
Store required resource views with d3d11 texture when it requested.
2019-12-05 02:29:18 +00:00
Seungha Yang
4e4fb8a577 d3d11: Add d3d11videosinkbin element
New wrapper element to support d3d11 memory upload, color conversion, and
rendering at once.
2019-12-05 02:29:18 +00:00
Seungha Yang
79ac41ec5b d3d11: Add color space conversion element using shader
Add d3d11colorconvert element for color space conversion.
Currently {RGB, YUV} to RGB conversion is supported.
2019-12-05 02:29:18 +00:00
Seungha Yang
01903c47f7 d3d11memory: Register mininal compare function for GstD3D11AllocationParams
The d3d11 allocation param is stored in GstStructure so it should
provide compare function for later use such as gst_structure_is_equal()
2019-12-05 02:29:18 +00:00
Seungha Yang
0778fa8043 d3d11window: Set render target whenever drawing
Device context's render target might be updated for offscreen rendering
2019-12-05 02:29:18 +00:00
Seungha Yang
a687f06b59 d3d11videosink: Specify d3d11 memory caps feature on sinkpad template 2019-12-05 02:29:18 +00:00
Seungha Yang
c17c1346ee d3d11: Introduce d3d11 upload/download element
That's equivalent to glupload and gldownload elements but for d3d11
2019-12-05 02:29:18 +00:00
Seungha Yang
6c3311a39e d3d11: Refactor d3d11 memory and dxgi format usage
* Create staging texture only when the CPU access is requested.
Note that we should avoid the CPU access to d3d11 memory as mush as possible.
Incoming d3d11upload and d3d11download will take this GPU memory upload/download.

* Upload/Download texture memory from/to staging only if it needed, similar to
GstGL PBO implementation.

* Define more dxgi formats for future usage (e.g., color conversion, dxva2 decoder).
Because I420_* formats are not supported formats by dxgi, each plane should
be handled likewise GstGL separately, but NV12/P10 formats might be supported ones.
So we decide the number of d3d11memory per GstBuffer for video memory depending on
OS version and dxgi format. For instance, if NV12 is supported by OS,
only one d3d11memory with DXGI_FORMAT_NV12 texture can be allocated by this commit.
One use case of such texture is DXVA. In case DXVA decoder, it might need to produce decoded data
to one DXGI_FORMAT_NV12 instead of seperate Y and UV planes.
Such behavior will be controlled via configuration of GstD3D11BufferPool and
default configuration is separate resources per plane.
2019-12-05 02:29:18 +00:00
Seungha Yang
d4c75f32dd d3d11: Add method for querying d3d11 memory type
... and fix misreading "private" annotation.
2019-12-05 02:29:18 +00:00
Seungha Yang
8da5237e22 d3d11device: Store selected d3d feature level
Depending on selected feature level, d3d11 API usage can be different.
Instead of querying the selected feature level by user whenever required,
store it once by d3d11device.
2019-12-05 02:29:18 +00:00
Seungha Yang
4ff0e62b72 d3d11: Don't share d3d11 device context unconditionally without checking adapter
Do not accept any GstD3D11Device context which has different adapter
index from the required one. For example, if a d3d11 element is expecting
d3d11 device with adapter 1 (i.e., the second GPU), any d3d11 device
context having different adapter could not be shared with
the d3d11 element.
2019-12-05 02:29:18 +00:00
Seungha Yang
7ae4105c6e d3d11device: Don't allow auto adapter selection
Change to d3d11device to be created with explicit target adapter index,
and expose some readonly device properties
2019-12-05 02:29:18 +00:00
Seungha Yang
494ec13316 d3d11: Rename native handle getter and context utils functions
Make them consistent with cuda context utils functions.
Put in-only parameter before all in-out parameters, and add _handle()
suffix to native handle getter functions.
2019-12-05 02:29:18 +00:00
Aaron Boxer
d3426cf15b d3dvideosink: use parent buffer pool to release buffers
In certain cases, the sink's buffer pool will not call the parent's
release_buffer method, so the pool does not clean up properly
after the buffer is released.
2019-12-04 23:00:35 +00:00
Sebastian Dröge
8ba4d1a480 decklink: Add simple device provider for Decklink devices 2019-12-03 09:16:22 +00:00
o0Ignition0o
d29012a760 avfvideosrc: Explicitly request device video permissions for macOS 10.14+
Since macOS Mojave (10.14), video permissions have to be explicitly
granted by a user in order to open a video device such as a camera.
This commit adds a check for the current permission status, and tries
to request for permission if applicable.
2019-12-02 08:52:05 +01:00
Nirbheek Chauhan
d6471b0251 wasapisrc: Correctly handle BUFFERFLAGS_SILENT
We need to ignore the data we get from WASAPI in this case and write
out silence (zeroes).

Initially reported at https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/808
2019-11-28 08:59:41 +00:00
Nirbheek Chauhan
08d5bdc7b5 wasapisrc: Try harder to avoid debug output in the hot loop
The whole `src_read()` function is a hot loop since the ringbuffer
thread is waiting on us, and printing to the console from inside it
can easily cause us to miss our deadline.

F.ex., if you had GST_DEBUG=3 and we accidentally missed a device
period, we'd trigger the "reported glitch" warning, which would cause
us to miss another device period, and so on. Let's reduce the log
level so that GST_DEBUG=3 is more usable, and only print buffer flag
info when it's actually relevant.
2019-11-28 08:59:41 +00:00
Nirbheek Chauhan
6cbff552fe wasapisrc: Fix capturing from some buggy audio drivers
Some audio drivers return varying amounts of data per ::GetBuffer
call, instead of following the device period that they've told us
about in `src_prepare()`.

Previously, we would just drop those extra buffers hoping that the
extra buffers were temporary (f.ex., a startup 'burst' of audio data).
However, it seems that some audio drivers, particularly on older
Windows versions (such as Windows 10 1703 and older) consistently
return varying amounts of data.

Use GstAdapter to smooth that out, and hope that the audio driver is
locally varying but globally periodic.

Initially reported in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/808
2019-11-28 08:59:41 +00:00
Nirbheek Chauhan
5d9c060ca7 wasapisrc: Clarify that nBlockAlign is actually bpf
bpf = bytes per frame.
2019-11-28 08:59:41 +00:00
Nirbheek Chauhan
6d27c0ac08 wasapisrc: Fix glitching and clock skew issues
We were miscalculating the device period, i.e. the number of frames
we'll get from WASAPI in each IAudioClient::GetBuffer call, due to
a calculation mistake (truncate instead of round).

For example, on my machine when the aux input is set to 44.1KHz, the
reported device period is 101587, which comes out to 447.998 frames
per ::GetBuffer call. In reality we will, of course, get 448 frames
per call, but we were truncating, so we expected 447 and were
discarding one frame every time. This led to glitching, and skew over
time.

Interestingly, I can only see this with 44.1Khz. 48Khz/96Khz are fine,
because the device period is a more 'even' number.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/806
2019-11-28 08:59:41 +00:00
Seungha Yang
00cb881431 msdk: Fix mixed declarations warning
msdk.c:194:3: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
2019-11-27 12:25:40 +09:00
Julien Isorce
2af672d00a msdkdec: log an error if leaking surfaces in finalize
The for loop in gst_msdkdec_handle_frame is error prone
about how it manages surfaces. Because sometimes it sets
the surface variable to NULL and sometimes it needs to free
it right away. So better to print an error if surfaces are
leaked to help with any change around the loop.
2019-11-25 17:14:20 +00:00
Aaron Boxer
348494fd34 msdk: query hardware when checking MFX availability
msdk plugin is not used for sofware encode/decode as there are better
solutions available. Also, with MFX_IMPL_AUTO_ANY, if software decode
is not supported, the plugin will still load, but will then fail when trying to
run the (autoplugged) pipeline. With MFX_IMPL_HARDWARE_ANY,
the plugin fails and a better software decoder is auto-plugged.
2019-11-22 05:18:42 +00:00
Aaron Boxer
2f1256ada7 msdk: query platform when opening session 2019-11-22 05:18:42 +00:00
Seungha Yang
0cf67c3be7 nvenc: Fix crash when nvenc was reused then freed without encoding
GstNvBaseEnc::n_bufs was set from the previous encoding session
but it wasn't cleared after stop. That might result to invalid memory
access at the next start (no encoded data) and then stop sequence.
Instead of defining a variable for array length, use GArray::len
directly to avoid such confusion.
2019-11-22 03:02:57 +00:00
Seungha Yang
aef414375a nvenc: Remove unused code path
refilling queue would not happen
2019-11-22 03:02:57 +00:00
Julien Isorce
a7dc407041 msdkdec: fix surface leak in msdkdec_handle_frame
Can be reproduced with:
  videotestsrc ! x264enc key-int-max=$N ! \
  h264parse ! msdkh264dec ! fakesink sync=1

It happens with any gop size but the smaller is the distance N
between key frames, the quicker it is leaking.

Fixes #1023
2019-11-19 09:14:51 -08:00
Andrew Branson
397a9edb35 androidmedia: Add new effects and scene modes to Camera parameters 2019-11-19 07:54:16 +01:00
Aaron Boxer
842bd66bd8 d3dvideosink: use sink dimensions when calculating overlay scaling 2019-11-18 03:32:57 +00:00
Aaron Boxer
20f3f63540 d3dvideosink: use explicit system memory feature in overlay composition caps 2019-11-18 02:29:45 +00:00
Jochen Henneberg
3ecd8666d9 msdk: Fixes for meson include directory setup
In case of pkg-config we need to create the include directories object
from the path using include_directories(). For INTELMEDIASDKROOT or
MFX_HOME we need to add the alternate include path ./include/mfx as
Intel MediaSDK now puts the headers there.
2019-11-17 17:39:44 +00:00
Seungha Yang
e52ed98197 d3d11window: Fix subclass procedure recursion
subclass window precedure should be cleared if it's no more used.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1115
2019-11-14 01:26:09 +00:00
Aaron Boxer
bfe2447a3b d3d11videosink: only destroy loop if external_win_id is false
Loop was created with this condition, so it should also be
destroyed under the same condition
2019-11-12 19:54:46 -06:00
Nicolas Dufresne
80b599c81b kmssink: Do not drain if imported buffer are from KMS
This adds a check to avoid draining when the imported buffers are in
fact own by kmssink. This happens since we export our kms buffer as
DMABuf. They are not really imported back as we pre-fill the cache,
but uses the same format as if they were external. This fixes
performance issues seen with videocrop2-test (found in -good).
2019-11-11 22:20:28 +00:00
Nicolas Dufresne
8574154cc8 kmssink: Avoid drain on caps changes
Draining systematically on caps changes was a hack. Instead, properly
save the render information used to render last_render, and use that
information to drain. This fixes performance issues met with video crop
meta and per frame caps changes.
2019-11-11 22:20:28 +00:00
Nicolas Dufresne
a83e0036ea kmssink: Ensure we have an allocator before importing
This fixes cases where the kms allocator API was called with a null
pointer.
2019-11-11 22:20:28 +00:00
Xavier Claessens
7d31f7a9e5 amc: Change plugin name if it's built for Magic Leap 2019-11-08 13:40:14 -05:00
Xavier Claessens
90223a2702 amc: Add MLSDK implementation 2019-11-08 13:40:14 -05:00
Haihao Xiang
dca3e47bce msdk: remove msdkvp8enc
MediaSDK doesn't support vp8 encode which is not going to be
implemented [1], so remove msdkvp8enc from this plugin

[1]: https://github.com/Intel-Media-SDK/MediaSDK/issues/947
2019-11-08 08:44:46 +00:00
Niels De Graef
d8f61515d8 Don't pass default GLib marshallers for signals
By passing NULL to `g_signal_new` instead of a marshaller, GLib will
actually internally optimize the signal (if the marshaller is available
in GLib itself) by also setting the valist marshaller. This makes the
signal emission a bit more performant than the regular marshalling,
which still needs to box into `GValue` and call libffi in case of a
generic marshaller.

Note that for custom marshallers, one would use
`g_signal_set_va_marshaller()` with the valist marshaller instead.
2019-11-06 14:27:46 +00:00
Ignacio Casal Quinteiro
a87ea01992 wasapi: minor cleanup 2019-11-06 08:18:59 +00:00
Aaron Boxer
86b01bb291 d3dvideosink: support OverlayComposition for GPU overlay compositing 2019-11-06 01:05:33 +00:00
Aaron Boxer
6d3429af34 documentation: fixed a heap o' typos 2019-11-05 09:11:25 -05:00
Roman Shpuntov
f84ebbd95f vtdec: fix leak of CMFormatDescription
There was consideration for replacing the CMFormatDescription used
on format changes in `set_format()` however on shutdown, we were
leaking the CMFormatDescription at the end of processing.

'https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/1106'
2019-11-04 12:12:57 +00:00
Roman Shpuntov
b0543d6ea5 vtdec: memory leaks bugfix
release 'videoDecoderSpecification' variable in 'gst_vtdec_create_session' function.
release 'extensions' variable in 'create_format_description_from_codec_data' function.
2019-11-02 13:58:56 +00:00
Haihao Xiang
1dd3ef24b1 msdkmjpegenc: convert UYVY to YUY2 instead of NV12
Before this fix, the chroma subsampling of the output is 4:2:0. It is
4:2:2 with this fix, which is better for UYVY input
2019-11-02 09:52:36 +00:00
Haihao Xiang
e1b8b81e84 msdkmjpegenc: don't need conversion for BGRA and YUY2
jpeg encoder in MSDK can accept BGRA and YUY2 input.
2019-11-02 09:52:36 +00:00
Haihao Xiang
fbeaa37b79 msdkenc: add need_conversion method
In future, a sub class of GstMsdkEncClass may decide a native format by
using this method, e.g. JPEG encoder may accept YUY2 input, however the
current implemation needs a conversion from YUY2 to NV12 before encoding.

In addtion, a sub class may choose a format for encoding if the input
format is not supported by MSDK, e.g. the current implemation does
UYVY->NV12 if the input format is UYVY. We may do UYVY->YUY2 for JPEG
encoder in future
2019-11-02 09:52:36 +00:00
Haihao Xiang
afce02b392 msdk: support for MFX_FOURCC_BGR4 frame allocation
MFX_FOURCC_BGR4 is mapped to VA_FOURCC_ABGR and JPEG encoder needs a
MFX_FOURCC_BGR4 frame for internal usage when the input format is
MFX_FOURCC_RGB4

This is a preparation for supporting native formats of JPEG encoder
2019-11-02 09:52:36 +00:00
Víctor Manuel Jáquez Leal
10d4c0c511 msdkdec: use decoder base class packetized flag
Instead of using a proxy of `is_packetized` flag this patch
replaces it with the accessor to that flag in decoder base class,
avoiding probable mismatches.
2019-10-29 14:58:54 +00:00
Haihao Xiang
cd883427e9 msdkdec: add parse callback for non-packetized input
commit 55c0d720 added the capability to handle non-packetized bitstream,
and there is a loop to handle multiple frames in a non-packetized buffer
in gst_msdkdec_handle_frame. However it is possible that a
non-packetized buffer still contains valid data but there is no long any
pending unfinished frame. Currently gst_video_decoder_decode_frame is
invoked to send a new frame with new input data, the situaltion is
repeated till an EOS is received. An application has to exit when
receiving an EOS, however there is still valid data in a
non-packetezied input buffer, hence some frames are dropped.

This fix adds a parse callback for non-packeteized input, a new frame
will be sent to the subclass as soon as the input buffer has valid data

This fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/665
2019-10-29 14:58:54 +00:00
Haihao Xiang
a2cafcfc3e msdkh265enc: Add support for tiled encoding
Add num-tile-rows and num-tile-cols properties to set the number of rows
and columns
2019-10-27 18:40:15 +00:00
Haihao Xiang
9fcd3bdd2b msdk: support P010_10LE in DMABuf mode for VP9/HEVC encoding 2019-10-25 16:54:57 +00:00
Cheng-Chang Wu
b5e506d12b androidmedia: Make sure that the first member of GstAmcSurfaceTextureJNI is correct
It's a subclass of GstAmcSurfaceTexture so should have that as first
struct member or otherwise it won't work.

Fixes #1105
2019-10-25 13:56:13 +00:00
Cheng-Chang Wu
e71f620eb0 androidmedia: Call JNI method with the actual Java object instead of our wrapper struct
Fixes #1105
2019-10-25 13:56:13 +00:00
Haihao Xiang
351a3934ec msdkvpp: add support Y410 and Y210 formats
And clean up some macros
2019-10-24 13:57:51 +00:00
U. Artie Eoff
b7ee6dc49a msdkvpp: allow cropping via properties
Add crop-left, crop-right, crop-top and crop-bottom
properties to msdkvpp and apply to input surface at
vpp transform run-time.

Implements #1097
2019-10-24 12:44:49 +00:00
Javier Celaya
8684dffe50 Fix get_supported_types function name 2019-10-16 06:42:37 +00:00
Javier Celaya
f267adeb4a Fix releasing local refs 2019-10-16 06:42:37 +00:00
Javier Celaya
3f0d60515a Fix names of MediaCodecInfo inner classes 2019-10-16 06:42:37 +00:00
Javier Celaya
26ea6d7fbb Fix getting MediaCodecList static methods 2019-10-16 06:42:37 +00:00
Julien Isorce
5de00b9ffb msdkenc: close encoder upon bitrate change
Upon bitrate change, make sure to close the encoder otherwise
the encoder is not re-initialized and the target bitrate is
never reached, and the encoder was flushed at each frame
from this moment.

Regression introduced in f2b35abcab which replaced the call
that was closing the encoder by an early return to avoid
re-initialization.
2019-10-15 15:53:20 -07:00
Seungha Yang
76654539b9 wasapi: Fix build warnings
gstwasapiutil.c(173) : warning C4715: 'gst_wasapi_device_role_to_erole': not all control paths return a value
gstwasapiutil.c(188) : warning C4715: 'gst_wasapi_erole_to_device_role': not all control paths return a value
2019-10-14 14:43:59 +00:00
Seungha Yang
baaa965ebc wasapi: Don't cast GstDeviceProvider to GstElement
The GstDeviceProvider isn't subclass of GstElement.

(gst-device-monitor-1.0:49356): GLib-GObject-WARNING **: 20:21:18.651:
invalid cast from 'GstWasapiDeviceProvider' to 'GstElement'
2019-10-14 14:43:59 +00:00