The previous code would run out of sync if there was packet lost
or clock skews. When that happened, the echo cancellation feature would
completely stop working. As this is crucial for audio calls, this patch
re-implement synchronization completely.
Instead of letting it drift until next discont, we now synchronize
against the record data at every iteration. This way we simply never
let the stream drift for longer then 10ms period. We also shorter the
delay by using the latency up the probe (basically excluding the sink
latency. This is a decent delay to avoid starving in the probe queue.
https://bugzilla.gnome.org/show_bug.cgi?id=768009
When echo cancel is enabled, we now fail the pipeline if there is
not echo probe. For this reason there is no need to check if probe
pointer is set anymore.
The byte-stream to avc conversion did not consider NAL sizes bigger than 2^16,
multiple layers, multiple NALs per layer, and various other things. This
caused corrupted streams in higher bitrates and other circumstances.
Let's just forward byte-stream as generated by the encoder and let h264parse
handle conversion to avc if needed. That way we only have to keep around one
version of the conversion and don't have to fix it in multiple places.
Rather than assuming something. e.g. zerocopy on iOS with GLES3 requires
the use of Luminance/Luminance Alpha formats and does not work with
Red/RG textures.
The saved timestamp is used to compute the delay of the probe data.
As it's used at the following incoming buffer, it needs to be offset
with the duration of the buffer to represent the end position. Also,
properly initialize the saved timestamp and protect against TIME_NONE.
Until now, we were synchronizing both DSP and Probe adapter by
waiting and clipping the probe adapter data. This increases the CPU
usage, can cause copies if the audio is not 10ms aligned and the worst
is that it prevents the processing from compensating for inaccurate
latency. This is also a step forward toward supporting playback
filters.
The current state of c++ ABI's on Window's and Gst's/Qt's conflicting
mingw builds means that we cannot use mingw for building the qt plugin.
Instead, a qmake .pro file is provided that is expected to be used with the
msvc binaries provided by Qt like so:
(with the PATH environment variable containing the path to the qt biniaries
and PKG_CONFIG_PATH containing the path to GStreamer modules)
cd /path/to/sources/gst-plugins-bad/ext/qt
qmake -tp vc
Then open the resulting VS project and build the library. Then
cp debug/libgstqtsink.dll /path/to/prefix/lib/gstreamer-1.0/libgstqtsink.cll
https://bugzilla.gnome.org/show_bug.cgi?id=761260
This DSP library can be used to enhance voice signal for real time
communication call. In implements multiple filters like noise reduction,
high pass filter, echo cancellation, automatic gain control, etc.
The webrtcdsp element can be used along, or with the help of the
webrtcechoprobe if echo cancellation is enabled. The echo probe should
be placed as close as possible to the audio sink, while the DSP is
generally place close to the audio capture. For local testing, one can
use an echo loop pipeline like the following:
autoaudiosrc ! webrtcdsp ! webrtcechoprobe ! autoaudiosink
This pipeline should produce a single echo rather then repeated echo.
Those elements works if they are placed in the same top level pipeline.
https://bugzilla.gnome.org/show_bug.cgi?id=767800
These two shader will calculate the vector length and use it as denominator.
But length could be zero which will cause undefine behaviour. Add protection for
this condition
https://bugzilla.gnome.org/show_bug.cgi?id=767635
We remove get-rollover-counter signal in favor of the "stats"
property. The "stats" property is a GstStructure with caps
application/x-srtp-encoder-stats that contains an array of
structures with caps application/x-srtp-stream.
Each stream structure contains "ssrc" and "roc" fields.
https://bugzilla.gnome.org/show_bug.cgi?id=733265
Previously GstCurlSmtpSink could cause the pipeline thread to end up
waiting for a stopped thread to perform work.
The scenario was that the sink could be rendering a buffer and waiting
for the curl transfer thread to have sent the data. As soon as the
transfer thread has copied all data to curl's data buffer in
gst_curl_base_sink_transfer_read_cb() then the render call would stop
waiting and return GST_FLOW_OK. While this takes place the transfer
thread may suffer from an error e.g. due gst_poll_wait() timing out.
This causes the transfer thread to record the error, claim (it is not
really true since there was an error) that the data has been sent and
that a response has been received by trying to signal the pipeline
thread (but this has already stopped waiting). Finally the transfer
thread stops itself. A short while later the pipeline thread may attempt
to push an EOS event into GstCurlSmtpSink. Since there is no check in
gst_curl_smtp_sink_event() to check if the sink has suffered from any
error it may attempt to add a final boundary and ask the, now deceased,
transfer thread to transfer the new data. Next the sink element would
have waited for the transfer to complete (using a different mechanism
than normal transfers through GstCurlBaseSink). In this case there was
an error check to avoid waiting if an error had already been seen.
Finally GstCurlSmtpSink would chain up to GstCurlBaseSink which would
then block waiting for a response (normally this would be prevented by
the transfer thread suffering the error claiming that it had been
received, but GstCurlSmtpSink clobbered this flag after the fact).
Now GstCurlSmtpSink avoids this by locking over the entire event handing
(preventing simultaneous changes to flags by the two threads) and also
by avoiding to initiate transfer of final boundary if an error has
already been seen.
Also add GST_FIXME() for remaining similar issue where the pipeline
thread may block indefinitely waiting for transfer thread to transfer
data but the transfer thread errors out and fails to notify the pipeline
thread that the transfer failed.
https://bugzilla.gnome.org/show_bug.cgi?id=767501
Use namespace only after it was actually defined by a header.
gstfacedetect.cpp:79:17: error: using directive refers to implicitly-defined namespace 'std' [-Werror]
using namespace std;
^
Replace the type and function prefix to follow the conventions:
- Use `GST_TYPE_DC1394_SRC` instead of `GST_TYPE_DC1394`.
- Use `GstDC1394Src` and `GstDC1394SrcClass` instead of
`GstDc1394` and `GstDc1394Class`.
- Use `gst_dc1394_src` instead of `gst_dc1394`.
https://bugzilla.gnome.org/show_bug.cgi?id=763026
The dc1394src is a PushSrc element for IIDC cameras based on libdc1394.
The implementation from the 0.x series is deffective:
caps negotiation does not work, and some video formats
provided by the camera are not supported.
Refactor the code to port it to 1.X and enhance the support
for the full set of video options of IIDC cameras:
- The IIDC specification includes a set of camera video modes
(video format, frame size, and frame rates).
They do not map perfectly to Gstreamer formats, but those that
do not match are very rare (if used at all by any camera).
In addition, although the specification includes a raw format,
some cameras use mono video formats to capture in Bayer format.
Map corresponding video modes to Gstreamer formats in capabilities,
allowing both gray raw and Bayer video formats for mono video modes.
- The specification includes scalable video modes (Format7),
where the frame size and rate can be set to arbitrary values
(within the limits of the camera and the bus transport).
Allow the use of such mode, using the frame size and rate
from the negotiatied caps, and set the camera frame rate
adjusting the packet size as in:
<http://damien.douxchamps.net/ieee1394/libdc1394/faq/#How_do_I_set_the_frame_rate>
The scalable modes also allow for a custom ROI offset.
Support for it can be easily added later using properties.
- Camera operation using libdc1394 is as follows:
1. Enumerate cameras on the system and open the camera
identified the enumeration index or by a GUID (64bit hex code).
2. Query the video formats supported by the camera.
3. Configure the camera for the desired video format.
4. Setup the capture resources for the configured video format
and start the camera transmission.
5. Capture frames from the camera and release them when not used.
6. Stop the camera transmission and clear the capture resources.
7. Close the camera freeing its resources.
Do steps 2 and 3 when getting and setting the caps respectively.
Ideally 4 and 6 would be done when going from PAUSED to PLAYING
and viceversa, but since caps might not be set yet, the video mode
is not properly configured leaving the camera in a broken state.
Hence, setup capture and start transmission in the set caps method,
and consequently clear the capture and stop the transmission
when going from PAUSED to READY (instead of PLAYING to PAUSED).
Symmetrycally, open the camera when going from READY to PAUSED,
allowing to probe the camera caps in the negotiation stage.
Implement that using the `start` and `stop` methods of `GstBaseSrc`,
instead of the `change-state` method of `GstElement`.
Stop the camera before setting new caps and restarting it again
to handle caps reconfiguration while in PLAYING (it has no effect
if the camera is not started).
- Create buffers copying the bytes of the captured frames.
Alternatively, the buffers could just wrap the bytes of the frames,
releasing the frame in the buffer's destroy notify function,
if all buffers were destroyed before going from PLAYING to PAUSED.
- No timestamp nor offset is set when creating buffers.
Timestamping is delegated to the parent class BaseSrc,
setting `gst_base_src_set_live` TRUE, `gst_base_src_set_format`
with GST_FORMAT_TIME and `gst_base_src_set_do_timestamp`.
Captured frames have a timestamp field with the system time
at the completion of the transmission of the frame,
but it is not sure that this comes from a monotonic clock,
and it seems to be left NULL in Windows.
- Use GUID and unit properties to select the camera to operate on.
The camera number used in version 0.X does not uniquely identify
the device (it depends on the set of cameras currently detected).
Since the GUID is 64bit identifier (same as MAC address),
handle it with a string property with its hexadecimal representation.
For practicality, operate on the first camera available if the GUID
is null (default) and match any camera unit number if unit is -1.
Alternatively, the GUID could be handed with an unsigned 64 bit
integer type property, using `0xffffffffffffffff` as default value
to select the first camera available (it is not a valid GUID value).
- Keep name `GstDc1394` and prefix `gst_dc1394` as in version 0.X,
although `GstDC1394Src` and `gst_dc1394_src` are more descriptive.
- Adjust build files to reenable the compilation of the plugin.
Remove dc1394 from the list of unported plugins in configure.ac.
Add the missing flags and libraries to Makefile.
Use `$()` for variable substitution, as many plugins do,
although other plugins use `@@` instead.
https://bugzilla.gnome.org/show_bug.cgi?id=763026
The heuristic to choose between packetise or not was changed to use the
segment format. The problem is that this change is reading the segment
during the caps event handling. The segment event will only be sent
after. That prevented the decoder to go in packetize mode, and avoid
useless parsing.
https://bugzilla.gnome.org/show_bug.cgi?id=736252
Use new gst_h264_video_calculate_framerate() API instead of fps_n/fps_d
fields in SPS struct which are to be removed.
Apparently H264 content in MSS is always non-interlaced/progressive,
so we can just pass 0 for field_pic_flag and don't need to parse any
slice headers first if there's no external signalling. But even if
that's not the case the new code is not worse than the existing code.
https://msdn.microsoft.com/en-us/library/cc189080%28VS.95%29.aspxhttps://bugzilla.gnome.org/show_bug.cgi?id=723352
_get_gl_context() can be called concurrently from either propose_allocation() or
decide_allocation(). If it so happens that this happens at the same time,
the check for whether we already had a GL context was outside the lock. Inside
the lock and loop, the first thing that happens is that we unref the current GL
context (if valid) as if there was a conflict adding it to the display. If the
timing was unlucky, subsequent use of the GL context would be referencing an
already unreffed GL context object resulting in a critical:
g_object_ref: assertion 'object->ref_count > 0' failed
https://bugzilla.gnome.org/show_bug.cgi?id=766703
The gltestsrc element uses two shaders: color_shader and snow_shader.
Those are alternatively assigned to the SrcShader->shader pointer and
their reference was transferred to it. Only the SrcShader->shader was
unreffed (in _src_shader_deinit()) so only one shader was properly
freed, the other one was leaked.
Fixed this by giving an extra ref to SrcShader->shader and unreffing the
2 shaders in _src_smpte_free().
https://bugzilla.gnome.org/show_bug.cgi?id=766661
Plugins can provide a set of named values for a control port. Ideally only those
values are set for the property. Check if all scalepoints are integers and if so
generate an enum type.
- add using namespace std; for std::vector
- use the cpp header imgproc.hpp file for the cv::ellipse function instead of
the C header
- Mat no longer takes IplImage in it's constructors, use the cvarrtomat()
function instead.
Fixes a couple of build errors:
gstfacedetect.cpp:140:30: error: ‘vector’ does not name a type
structure_and_message (const vector < Rect > &rectangles, const gchar * name,
^~~~~~
gstfacedetect.cpp:140:37: error: expected ‘,’ or ‘...’ before ‘<’ token
structure_and_message (const vector < Rect > &rectangles, const gchar * name,
^
gstfacedetect.cpp: In function ‘void structure_and_message(int)’:
gstfacedetect.cpp:143:13: error: ‘rectangles’ was not declared in this scope
Rect sr = rectangles[0];
[...]
gstfacedetect.cpp: In function ‘void
gst_face_detect_run_detector(GstFaceDetect*, cv::CascadeClassifier*, gint, gint,
cv::Rect, std::vector<cv::Rect_<int> >&)’:
gstfacedetect.cpp:562:31: error: no matching function for call to
‘cv::Mat::Mat(IplImage*&, cv::Rect&)’
Mat roi (filter->cvGray, r);
[...]
gstfacedetect.cpp: In function ‘GstFlowReturn
gst_face_detect_transform_ip(GstOpencvVideoFilter*, GstBuffer*, IplImage*)’:
gstfacedetect.cpp:594:44: error: no matching function for call to
‘cv::Mat::Mat(cv::Mat, bool)’
Mat mtxOrg (cv::cvarrToMat (img), false);
[...]
gstfacedetect.cpp:734:79: error: ‘ellipse’ was not declared in this scope
ellipse (mtxOrg, center, axes, 0, 0, 360, Scalar (cr, cg, cb), 3, 8,
0);
We can avoid a render pass if downstream supports the affine transformation meta
and increase the performance of some pipelines involving gltransformation.
Implemented by checking for the affine transformation in the allocation query
from downstream and combining our matrix with that of upstream's (or creating
our own).
Provide a function to get the affine matrix in the meta in terms of NDC
coordinates and use as a standard opengl matrix.
Also advertise support for the affine transformation meta in the allocation
query.
We were always failing the allocation query as a flag was never being set to
signal a successful negotiation. Fix by setting the required flag on a
successful caps event from upstream.
The port was trivial, and according to the NEWS file nothing else has changed,
but it is possible that other API was changed without proper notification.
OpenJPEG upstream has shipped a pkg-config file for the past 4 years, and all
distros should be shipping it by now.
https://bugzilla.gnome.org/show_bug.cgi?id=766213
Use the semaphores in the correct place, before and after the submission for
acquiring and presenting the swapchain buffer.
Waiting on the fence that only signals the command buffer completion rather than
the completion of the presentation is racy with the destruction of the vulkan
buffers associated with that image. Wait on the device to be idle instead after
presenting.
1.Porting the exist deinterlace shader and OpenGL callback
to be compatible with OpenGL ES.
2.Add a our blur vertical shader to gldeinterlace.
3.Add a property named “method” to let user choose which
deinterlace function to use. Default to choose blur vertical
method for better performance.
[Matthew Waters]: fix name of greedyh in method property (was greedhy) and port
to git master.
https://bugzilla.gnome.org/show_bug.cgi?id=764873
A realtime clock is used in many places, such as deciding which
fragment to select at start up and deciding how long to sleep
before a fragment becomes available. For example dashdemux needs
sample the client's estimate of UTC when selecting where to start
in a live DASH stream.
The problem with dashdemux calculating the client's idea of UTC is
that it makes it difficult to create unit tests, because the passage
of time is a factor in the test.
This commit changes dashdemux and adaptivedemux to use the
GstSystemClock, so that a unit test can replace the system clock when
it needs to be able to control the clock.
This commit makes no change to the behaviour under normal usage, as
GstSystemClock is based upon the system time.
https://bugzilla.gnome.org/show_bug.cgi?id=762147
When application change pipeline state NULL->READY and then READY->NULL,
glimagesink will not clear glsink->window_id. After that, when application
change state NULL->READY, the new_window_id is equal to window_id, glimagesink
will not set window handle. It will use the internal window but not the window
create by application.
https://bugzilla.gnome.org/show_bug.cgi?id=765241
Currently, gst_srtp_dec_sink_setcaps is happy if the "roc" field is not
provided in the caps. If it is not provided the stream will be properly
inserted in the hash table with a default "roc". Then, when the first
buffer arrives validate_buffer will find an existing stream in the hash
table and will not signal request-key, not allowing the user to provide
a "roc".
This patch expects "roc" in gst_srtp_dec_sink_setcaps, if not found a
request-key will be signaled and the user will be able to provide all
the srtp fields, including "roc".
https://bugzilla.gnome.org/show_bug.cgi?id=765079
