Commit graph

25525 commits

Author SHA1 Message Date
Nicolas Dufresne
e70993bf43 v4l2codecs: Wait for buffers to come back
This code add required mechanism to try and allocate (not implemented yet)
otherwise wait for more buffers. This also comes with mechanism to terminate
the wait on flush or PAUSED_TO_READY transitions.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
09a9ffcda6 v4l2codecs: Implement flushing sequence
This simply consit of cycling through STREAMOFF/STREAMON with stateless
decoders.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
f69283819a v4l2codecs: allocator: Add method to wait for more buffers
This add function to wait for buffers to get back into the pool along with a
set_flushing() method to allow unblocking this wait.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
bc0a8ff40d v4l2codecs: pool: Create new buffer when pool is empty
This simply create an empty GstBuffer when the pool is empty. This way it's up
to the allocator to grow or wait if we ran out of memory.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
ee26c9c9a1 v4l2slh264dec: Improve end_picture() robustness
Use a goto to ensure that for all cases we cleanup the current picture state.
And move the src buffer allocation higher, so we don't queue a bitstream
buffer if we don't have a picture buffer to decode into.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
bd7db13412 codecs: h264: Do not ignore end_picture() return value
If decoding failed because end_picture() failed, set the picture to
nonexisting, this way output_picture() will be skipped. This avoids confusing
special cases in output_picture() implementation.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
390cbe1f00 v4l2slh264dec: Add output format negotiation
This allow negotiating the output format through caps. Some drivers can
pipeline the decoder buffer through an image processor. This only support
colorspace conversion for now.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
6494d7b056 v4l2format: Convert between V4L2 and GST video format
This will be needed in the output format negotiation.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
2a1836f183 v4l2codecs: Read driver provided stride
This implements driver stride support but only for single allocation buffers.
This code is imported from the original v4l2 plugin and adapted to the new
helper context.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
21652a8e52 v4l2h264dec: Copy frames when GstVideoMeta is not supported
In some case, when downstream does not support GstVideoMeta, we need to
normalize the stride and offset of the buffer so that downstream can render
properly with a GstVideoMeta. This code is not called when GstVideoMeta is
supported downstream.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
3c2d25eebf v4l2codec: Add initial GstVideoMeta support
In this patch we strictly set the GstVideoMeta width/height to the coded width
and height. Further patches will add stride support and frame copying when
downstream does not support GstVideoMeta.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
f6690420ce v4l2slh264dec: Prevent spurious renegotiation
Don't let downstream cause a renegotiation at random point in time. This would
lead to spurious renegotiation and the decoder state may not be recoverable.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
b47929a74d v4l2slh264dec: Hold on picture buffers
These buffers should not be reused for decoding until they are no longer
referenced.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
b107c20198 v4l2codecs: Fix bytesused value
Pass the actual amount of bytes we have copied into the bitstream buffer. Also
unmap the memory before queuing.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
0d740a184e v4l2codech264dec: Implement finish frame
This enables the request to be processed by the HW (STREAMON) and to be pushed
downstream for further processing.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
3cc6643050 v4l2codecdecoder: Add support for dequeuing buffers
This includings polling of the request and streamon/streamoff.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
4305771178 v4l2codech264dec: Implement queuing
We now pass the controls, associated to a request, queue the bitstream, qeueue
a picture buffer to decode into and finally queue the request. This now runs
until the buffer pool is exhausted. The next step will be to dequeue.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
a307499437 v4l2decoder: Add helpers to queue buffer and requests 2020-03-31 09:34:05 -04:00
Nicolas Dufresne
db787adfa1 v4l2codec: Add getter for buffer index in allocator and pool 2020-03-31 09:34:05 -04:00
Nicolas Dufresne
527f040bfc v4l2h264codecdec: Copy bitstream parameter and data
In this patch we fill the control structure with the bitstream paramter and
copy the bitstream data into V4L2 memory. Slice paramters are only the subset
of what Hantro needs, without any support for interlaced content.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
2bac52a6fb v4l2slh264dec: Use the newly introduced allocator / pool
We now allocate and free the memory needed for streaming.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
c120f8b041 v4l2codec: Add allocator and pool implementation
This is a pooling allocator and the buffer pool does nothing other then
reusing the GstBuffer structure. Note that the pool is an internal pool, so
the start/stop/set_config virtual functions are not implemented.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
d476e6d689 v4l2codec: decoder: Add allocation helper 2020-03-31 09:34:05 -04:00
Nicolas Dufresne
7ae57631d4 v4l2codecs: Implement H264 format negotiation 2020-03-31 09:34:05 -04:00
Nicolas Dufresne
020a019fd6 v4l2codecs: Add initial formats helpers 2020-03-31 09:34:05 -04:00
Nicolas Dufresne
a007e2dd9d v4l2codecs: Add skeleton of H264 decoder
This introduces the skeleton of the H264 decoder. The plugin will list the
devices and register a subclass of the GstV4L2CodecH264Dec base class. The
subclass will pick the required specific information from the GstV4L2Device
stored in the subclass structure.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
b7762fc332 v4l2codec: Add GstV4L2Deocder helper object
This is a GstObject which will be used to hold on media and video device file
descriptor and provide abstracted ioctl calls with these descriptor. At the
moment this helper contains just enough to enumerate the supported format.
This part will be used by the plugin to register the CODEC specific elements..
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
dd2b0302b3 v4l2codecs: Copy all needed Linux kernel headers
Most of the features we need are very early or not expose yet in the uAPI.
Using an internal copy ensure that we everything we need is defined avoiding
to add load of checks and conditionnal code.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
6d9c98ae5a v4l2codecs: Add device enumeration
This introduces a GstV4L2CodecDevice structure and helper to retrieve a
list of CODEC device drivers. In order to find the device driver we
enumerate all media devices with UDEV. We then get the media controller
topology and locate a entity with function encoder or decoder and make
sure it is linked to two V4L2 IO entity pointing to the same device
node.
2020-03-31 09:34:05 -04:00
Nicolas Dufresne
6958fb6d59 v4l2codecs: Introdude a V4L2 CODECs Accelerator plugin
This plugin will support new CODECs uAPI in the Linux kernel. These
consist of an accelerator interface similar to DXVA, NVDEC, VDPAU and
VAAPI.
2020-03-31 09:34:05 -04:00
Seungha Yang
882531aa54 msdkh264enc: Port to h264parser API for frame packing SEI injection
Create frame packing SEI memory only once per set_format() and
reuse it if possible.
2020-03-31 08:30:50 +00:00
Seungha Yang
d81438a69e h264parse: Print all the syntax elements of frame packing for debugging
Other values might be useful for debugging
2020-03-31 08:30:50 +00:00
Haihao Xiang
326b755332 msdkh265enc: support 8-bit 422 encoding
The media driver can support HEVC 8-bit 422 encoding for non-lowpower
mode since ICL[1], so VPP is not needed for this case.

Sample pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=YUY2 ! msdkh265enc ! \
filesink location=output.h265

[1] https://github.com/intel/media-driver#decodingencoding-features
2020-03-31 09:27:03 +08:00
Haihao Xiang
7c91c8c980 msdkh265enc: add support for Y210 in sink pad
Sample pipeline:
gst-launch-1.0 videotestsrc ! video/x-raw,format=Y210 !
msdkh265enc ! filesink location=output.h265
2020-03-31 09:27:03 +08:00
Haihao Xiang
fbd1d0cecb msdk: add a helper function to get codename of the platform
The features supported in MSDK vary from platform to platform. We may
support some features based on the codename of the platform in future.
2020-03-31 09:27:03 +08:00
Seungha Yang
afb042cbb9 msdkenc: Try to find corresponding codec frame for encoded output buffer
The input and output buffers should be matched as much as possible
so that various metadata and its ordering to be preserved.
2020-03-30 07:59:10 +00:00
Seungha Yang
c0787ed1cd msdkh264enc: Add support for CEA708 closed caption insertion
Currently supported caption format is CEA708_RAW
2020-03-30 07:59:10 +00:00
Seungha Yang
a1e8109aa8 h264parser: Add a helper method to create and inject raw SEI data
Add an API to create raw SEI nal unit. This would be useful in case
an user want to create SEI nal data and inject the SEI nal data
into bitstream.
2020-03-30 07:59:10 +00:00
Seungha Yang
f94157c949 nalutils: Introduce NAL writer helper methods
Add helper methods for writing h264 and h265 NAL
2020-03-30 07:59:10 +00:00
Seungha Yang
5f6884a1c0 h264parser: Expose SEI clear function to public
gsth265parser does it already. Although corresponding API of h265parser is
gst_h265_sei_free, _clear suffix is more consistent naming for h264parser
since there are gst_h264_{sps,pps}_clear().
2020-03-30 07:59:10 +00:00
Seungha Yang
429de2c6d8 d3d11decoder: Add support for array of texture DPB
DXVA supports two kinds of texture structure for DPB, one is
"1) texture array" and the other is "2) array of texture".
1) is a type of texture which is single ID3D11Texture2D object having
ArraySize greater than one. So the ID3D11Texture2D itself is a set of texture.
Each sub texture of this type mush have identical resolution, format and so on,
and the number of sub texture in a texture array is fixed.

2) is an array of usual ID3D11Texture2D object. That means each
ID3D11Texture2D is independent each other and might have different resolution as well.
Moreover, we can modify the number of frames of the array dynamically.
This type is more flexible than "1) texture array" in terms of dynamic
behavior and also this type of texture can be used for shader resource view
but "1) texture array" couldn't be.

If "2) array of texture" is supported by driver, DXVA spec is saying that
it's preferred format over "1) texture array" in terms of performance.
2020-03-30 03:40:16 +00:00
Haihao Xiang
d796707dd4 msdkh265enc: slice size control
Add max-slice-size property to set the maximum slice size
2020-03-30 00:20:11 +00:00
Seungha Yang
3208e3d371 Revert "d3d11decoder: Check decoder status report"
This reverts commit 418e6991c1.

Not all drivers seem to be friendly to this API.
Revert to avoid incomprehensible crash with Nvidia.
2020-03-28 12:12:03 +00:00
Seungha Yang
b2a650cf28 d3d11window: Always reuse swapchain
DXGI format can be updated via ResizeBuffers()
2020-03-28 11:45:48 +00:00
Seungha Yang
f9f6916aa1 d3d11window: Reorganize display colorspace selection
The set of supported color space by DXGI is not full combination of
our colorimetry. That means we should convert color space to one
of supported color space by DXGI. This commit modifies the color space
selection step so that d3d11window can find the best matching DXGI color space
first and then the selected input/output color space will be referenced
by shader and/or d3d11videoprocessor.
2020-03-28 11:45:48 +00:00
Seungha Yang
7f5347a664 rtmp2src: Add idle-timeout property
Add new property to signalling that there is no incoming data
from peer. This can be useful if users want to stop the streaming
when the connection is alive but no packet is arriving.
2020-03-27 10:25:37 +00:00
Matthew Waters
8da177c0bf dtls/connection: fix EOF handling with openssl 1.1.1e
openssl 1.1.1e does some stricker EOF handling and will throw an error
if the EOF is unexpected (like in the middle of a record).  As we are
streaming data into openssl here, it is entirely possible that we push
data from multiple buffers/packets into openssl separately.

From the openssl changelog:

 Changes between 1.1.1d and 1.1.1e [17 Mar 2020]
  *) Properly detect EOF while reading in libssl. Previously if we hit an EOF
     while reading in libssl then we would report an error back to the
     application (SSL_ERROR_SYSCALL) but errno would be 0. We now add
     an error to the stack (which means we instead return SSL_ERROR_SSL) and
     therefore give a hint as to what went wrong.
     [Matt Caswell]

We can relax the EOF signalling to only return TRUE when we have stopped
for any reason (EOS, error).

Will also remove a spurious EOF error from previous openssl version.
2020-03-27 11:43:53 +11:00
Matthew Waters
319a5e5779 webrtc: mark streams as active on renegotiation as well.
Otherwise when bundling, only the changed streams would be considered as
to whether the bundled transport needs to be blocked as all streams are
inactive.

Scenario is one transceiver changes direction to inactive and as that is
the only change in transciever direction, the entire bundled transport would
be blocked even if there are other active transceivers inside the same bundled
transport that are still active.

Fix by always checking the activeness of a stream regardless of if the
transceiverr has changed direction.
2020-03-25 14:46:15 +11:00
Guillaume Desmottes
c3a9d2dc64 interlace: add alternate support
Allow downstream elements to negotiate the alternate interlace mode,
splitting each input buffer in two fields, each having their own buffer.
2020-03-24 09:57:53 +01:00
Guillaume Desmottes
2db7c4e22c interlace: factor out interlace_mode_from_pattern() 2020-03-24 09:53:43 +01:00