Refactor the encoder's caps query proxying function to a common place
and use it in the videodecoder to proxy downstream restrictions.
The new function is private to the gstvideo lib.
https://bugzilla.gnome.org/show_bug.cgi?id=741263
Update the new buffer size after alignment in the pool configuration
before calling the parent set_config. This ensures that the parent knows
about the buffer size that we will allocate and makes the size check
work in the release_buffer method.
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=741420
This reverts commit 406f32a946.
The problem was apparently that my video-orc.h was not updated and did not
include the prototype for that function. Only a "make clean" caused it to
be regenerated.
Avoid using a constant.
Avoid doing saturated adds, results are not supposed to overflow here.
Rework the C backup function a little in preparation for custom backup
functions in ORC.
See https://bugzilla.gnome.org/show_bug.cgi?id=741015
It will cause the frame to be initialized with inconsistent values that then
later can cause crashes or any other kind of interesting and hard to debug
bugs.
In cases where we just call orc directly this is somewhat
superfluous, but let's do it anyway for consistency. In
other cases the compiler can hopefully use this to optimise
memory access a little.
when using variable taps and when we are limiting the number of taps,
recalculate the lanczos parameters to match the clamped value.
Set the max number of taps to 128
Make a small object to hold a pool of allocated temp lines.
Keep track of how many temp lines each conversion stage needs and use
this to allocate just enough temp lines from the temp lines object. from
the temp lines object.
When dealing with mixed interlaced, setup a scaler and chroma-resampler
for both interlaced and progressive frames and switch between them
depending on the interlace mode of the input frame.
Add an option to limit the number of taps to use in automatic mode. The
problem is that for lanczos, we might use more taps than what we can
handle with the current precision.
Rework the other options a little to make it nicer to set defaults.
Refactor GstVideoInfo init, make function to set default colorimetry.
Call fill_planes after we configure the GstVideoInfo with parameters
from the caps.
The size of the chroma planes for interlaced vertically subsampled
formats needs to be rounded up to 2, we have 2 fields with each
the same anount of chroma lines.
Keep only 1 structure with all matrix information.
Add structure to hold gamma information.
Add more options to control gamma, primaries and color matrix handling.
Add functions to compute transformations to and from XYZ and use this
to convert between primaries.
Merge gamma into the convert to and from RGB stage.
Fix border val.
Simplify the fastpath table, remove unused fields, add some more checks.
Prepare for doing full gamma corrected conversion and scaling by first
splitting the conversions from and to RGB into separate steps.
split scaling in downscaling and upscaling steps to be performed before
and after conversion respectively.
Fix clipping of images that are partially left of the video
surface, they would get clipped on the right side instead of
the left side, because the video unpack functions currently
ignore the x offset parameter. Work around that until that
is implemented.
https://bugzilla.gnome.org/show_bug.cgi?id=739281
In case of overlay being completely or partially outside
the video frame, the offset calculations are not right,
which resulted in the overlay not being displayed as
expected, or crashes due to invalid memory access.
When the overlay rectangle is completely outside,
we need not render the overlay at all.
For partial display of overlay rectangles, src_yoff
was not being calculated, hence it was always clipping
the bottom half of the overlay, By calculating the
src_yoff, now the overlay is clipped properly.
https://bugzilla.gnome.org/show_bug.cgi?id=739281
Make an ORC version of the 2x vertical upsampling code.
Improve unit tests, test chroma up and down sampling.
memset buffer in conversion to make valgrind happy.
Combine multiplies in 4x filters.
Rename conversion functions to make them nicer in orc.
Add ORC versions for various downsampling algorithms
Add unit test chroma resampler
We only need to do the horizontal subsampling on 1 line if we do it
after vertical subsampling and we also avoid doing vertical subsampling
on unused pixels.
Rework the converter, keep track of the conversion steps by chaining the
cache objects together. We can then walk the chain and decide the
optimal allocation pattern.
Remove the free function, we're not going to need this anytime soon.
Keep track of what output line we're constructing so that we can let the
allocator return a line directly into the target image when possible.
Directly read from the source pixels when possible.
We need to allocate the templine with the amount of pixels we are going
to handle, which we only know for the vertical resampler when we are
asked to resample.
Add scaler functions for 16 bits formats.
Rename the scaler functions so that 16bits versions don't look too
weird.
Remove old unused h_2tap functions
Fix v_ntap functions, it was using 1 tap too little.
Rework the way we track the current state of the video through the
different conversion phases and use this to make sure we use the right
format and pstride where needed.
A faster version of 4tap horizontal scaling causes segfaults in ORC
presumably because it uses too many registers so disable it to avoid
crashing in the ORC tests.
video-scaler.c:151:58: error: implicit conversion from enumeration type
'GstVideoScalerFlags' to different enumeration type
'GstVideoResamplerFlags' [-Werror,-Wenum-conversion]
gst_video_resampler_init (&scale->resampler, method, flags, out_size,
~~~~~~~~~~~~~~~~~~~~~~~~ ^~~~~
Only apply an offset that is a multiple of the subsampling. To handle
arbitrary offsets in the future, we need to be able to chroma-resample
part of the borders.
Add support for cropping the source and placing the converted image
into a rectangle in the destination frame.
Add an option to add a border and border color.
Add the old ORC functions for nearest and linear. Label them as Low
quality because they are not as accurate but ORC lacks opcodes to
express this for now.
Add a video scaler object build on top of the resampler. It has
implementation to deal with interlaced video as well as horizontal and
vertical scaling functions.
Use a LineCache object to track and process lines between unpack,
upsample, convert, downsample and pack stages. This simplifies the
main core processing function a lot and allows for future additions
easily.
Add support for interlaced formats in chroma up and downsampling.
There are some few but certain conditions where it is possible for the
dest_width to be smaller than x. So we check this before assigning a negative
value to src_width, which is a unsigned and would be promoted to a number that
can segfault videoblend.
https://bugzilla.gnome.org/show_bug.cgi?id=738242
This was never reset when going from PAUSED->READY and resulted
in encoders being not reusable after EOS. They just rejected any
buffer because they received EOS in their previous life.
The flag wasn't used anywhere except for rejecting buffers after
EOS, and this is now handled by GstPad directly.
Move the conversion code used in videoconvert to the video library
and expose a simple but generic API to do arbitrary conversion. It can
currently do colorspace conversion but the plan is to add videoscale to
it as well.
See https://bugzilla.gnome.org/show_bug.cgi?id=732415
Reset last_timestamp_out when applying the output segment
change, to avoid decoder confusion over new timestamp timelines when
a seamless segment change happens.
Move some locks/unlocks to later when they're actually needed.
https://bugzilla.gnome.org/show_bug.cgi?id=734617
This fixes the reverse playback scenario when upstream is not fully
parsing the stream and does not send every keyframe chain separately
with the DISCONT flag on the keyframe.
To explain this, let's suppose we have this stream:
0 1 2 3 4 5 6 7 8
K K K
In most circumstances, the upstream parser will chain in the
decoder the buffers in the following order:
6 7 8 3 4 5 0 1 2
D D D
In this case, GstVideoDecoder will flush the parse queue every time
it receives discont (D) and we will eventually get in the output queue:
(flush here) 8 7 6 (flush here) 5 4 3 (flush here) 2 1 0
In case the upstream parser doesn't do this work, though,
GstVideoDecoder will receive the whole stream at once and will flush
the parse queue afterwards:
0 1 2 3 4 5 6 7 8
D
During the flush, it will look backwards for keyframes and will
decode in this order:
6 7 8 3 4 5 0 1 2
This is the same order that it would receive from upstream if
upstream was parsing and looking for the keyframes, only that now
there is no flushing of the output queue in between keyframes,
which will result in the output queue looking like this:
2 1 0 6 5 3 8 7 6
This will confuse downstream obviously and will play incorrectly.
This patch forces the decoder to flush the output queue every time
it picks a new keyframe to decode, so it will end up decoding 6 7 8
and then flushing before picking 3 for decoding, so the output will
get 8 7 6 before 6 5 3 and the video will play back correctly.
https://bugzilla.gnome.org/show_bug.cgi?id=734441
This prevent implementing allocation query, as the format need to be
known in order to determin the size and number of buffers needed.
Note: This may lead to few regressions that will need fixing
https://bugzilla.gnome.org/show_bug.cgi?id=732288
Fix gst_video_decoder_parse_available() to really parse any pending
source data that is still available in the adapter. This is a memory
optimization to avoid expansion of video packed added to the adapter,
but also a fix to EOS condition when the subclass parse() function
ultimately only needed to call into gvd_have_frame() and no additional
source bytes were consumed, i.e. gvd_add_to_frame() is not called.
This situation can occur when decoding H.264 streams in byte-stream/nal
mode for instance. A decoder always requires the next NAL unit to be
parsed so that to determine picture boundaries. When a new picture is
found, no byte is consumed (i.e. gvd_add_to_frame() is not called)
but gvd_have_frame() is called (i.e. priv->current_frame is gone).
Also make sure to avoid infinite loops caused by incorrect subclass
parse() implementations. This can occur when no byte gets consumed
and no appropriate indication (GST_VIDEO_DECODER_FLOW_NEED_DATA) is
returned.
https://bugzilla.gnome.org/show_bug.cgi?id=731974
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Buffer pool set_config() may return FALSE if requested configuration needed small
changes. Reget the config and try setting it again. This ensure we have a configured
pool if possible.
This should allow for more meaningful errors. Dereferencing NULL
is more useful information than dereferencing a random address
happened to be on the stack.
If gst_video_overlay_rectangle_apply_global_alpha is called with
a rectangle with unsuitable alpha, expanding the alpha plane will
fail, and thus lead to dereferencing a NULL src pointer. It's not
certain this will happen in practice, as the function is static
and callers might ensure suitable alpha before calling, but there
is no apparent explicit such check.
Add prologue asserts for proper alpha to explicitely prevent this.
Coverity 1139707
Videodecoder does late renegotiation, it will wait for the next
buffer before renegotiating its caps and bufferpool. It might happen
that downstream element switched from passthrough to non-passthrough
and sent a reconfigure upstream (that caused this renegotiation).
This downstream element will ask the video sink below for the bufferpool
with an allocation query and will get the same bufferpool that
videodecoder is holding, too.
When renegotiating, if videodecoder deactivates its bufferpool it
might be deactivating the bufferpool that some element downstream
is using and cause the pipeline to fail.
https://bugzilla.gnome.org/show_bug.cgi?id=727498
baseparse will reverse each GOP for us already, so the segment events can
be after our keyframe. Make sure to get it and all other relevant sticky
events before starting to decode.
This was a regression introduced by f52fd7a68, where we started using
the stride to encode the dimensions in tiles. This patch simply updates
offset and size calculation as described in the documentation,
part-mediatype-video-raw.txt.
Instead of using extra plane, we encode the number of tiles in x and y in the stride of
each planes (i.e. y_tiles << 16 | x_tiles) and introduce tile_mode, tile_width and
tile_height into GstVideoFormatInfo structure.
https://bugzilla.gnome.org/show_bug.cgi?id=707361
For reverse playback, the segment event will only be pushed when
the first buffer is actually pushed. But for decoding frames and storing
those into the list to be pushed the output_segment.rate value is used
to determine if it is forward or reverse playback.
In case a previous segment event (or none) is in use it will mistakenly
think it is doing forward playback and push the buffers immediatelly and
try to clip buffers based on an old segment (or an uninitialized one, leading
to an assertion)
This patch fixes this by copying the segment earlier if on reverse playback
https://bugzilla.gnome.org/show_bug.cgi?id=721666
And don't assume in other code that set_format() preserves any fields at
all. These assumptions were already made here for fields that were changed
by set_format().
Fixes "Unitialized Scalar Variable" issues reported by Coverity.
Has the added advantage of detecting whether somebody *does* use those
fields (ending up with a invalid address).
https://bugzilla.gnome.org/show_bug.cgi?id=720810
This must only ever be used in caps in combination with a non-system
memory GstCapsFeatures, and where it does not make sense to specify
any of the other video formats. Examples of this would be in gst-vaapi.
This reverts commit 5fcdabd907.
Instead of making it impossible to use the ENCODED format we should
just document that it must not be used for capsfeature-less caps.
Also this commit broke API/ABI.
GST_VIDEO_FORMAT_ENCODED was added to support *extracting* video-related
information (like width, height, framerate,...) from caps.
It is __NOT__ intended to be used as a format field on video/x-raw caps.
So that it avoids to send an allocation query twice.
One from an early call to gst_video_encoder_negotiate from a
subclass, then one from gst_video_encoder_allocate_output_frame.
Which means that previously gst_video_encoder_negotiate was not
clearing the GST_PAD_FLAG_NEED_RECONFIGURE even on success.
Fixes bug https://bugzilla.gnome.org/show_bug.cgi?id=719684
... so subclasses can release a frame all the way (also from frame list)
without having to pass through _finish_frame or _drop_frame.
The latter may not be applicable, or may or may not have already
been called for the frame in question.
See https://bugzilla.gnome.org/show_bug.cgi?id=693772
allows configuration of whether GstVideoGLTextureUploadMeta is
added to buffers resulting from a buffer pool. This is sperate
to the caps feature in that an element may want to add the upload
meta itself rather than allowing the buffer pool to.
https://bugzilla.gnome.org/show_bug.cgi?id=712798
We could have allocation query before caps event and even without caps inside
the query. In such cases , the downstream can return a bufferpool object with
out actually configuring it. This feature is helpful to negotiate the bufferpool
with out knowing the output video format. For eg: some hardware accelerated
decoders can interpret the o/p video format only after it finishes the decoding
of one buffer at least.
https://bugzilla.gnome.org/show_bug.cgi?id=687183
This avoids triggering plenty of extra code/methods/overhead downstream when
we can just quickly check whenever we want to set caps whether they are
identical or not
https://bugzilla.gnome.org/show_bug.cgi?id=706600
Either there was a flush before that resets everything anyway,
or resetting would make us lose information we might need if
it's just a segment update.
The subclass will be called with set_format() and there it can drain
if necessary and reset whatever is necessary. This is the same behaviour
as for the video decoder.
This reverts commit 28e1dadbfa.
Incrementing the offset to make the plane aligned causes the image to be
incompatible with what Xv expects. Rather that forcing a memcpy in the
xvimagesink we would like to do adjust the left padding instead.
In the unlikely case that the decoder drops a frame before the first
input frame is outputted, use the input segment (since it wasn't
carried over to the output segment yet)
https://bugzilla.gnome.org/show_bug.cgi?id=702502
Add functions to up/downsample chroma in horizontal and vertical
directions. These functions work in-placeand are meant to be used on the
input/output of the pack/unpack functions.
We have no way of tell the caller of the exact error (e.g. if we're flushing),
so will have to wait until the caller uses API that returns a GstFlowReturn,
for example when pushing this buffer.
https://bugzilla.gnome.org/show_bug.cgi?id=700006
For this release the corresponding GstVideoCodecFrame before
pushing the buffer. The buffer will now be writable unless
the subclass still holds another reference to the buffer or
the frame.
For this release the corresponding GstVideoCodecFrame before
pushing the buffer. The buffer will now be writable unless
the subclass still holds another reference to the buffer or
the frame.
When we get a new buffer, always call the parse function, even if it is a 0
sized buffer. For theora we need to also decode 0 sized buffers.
Ideally we would like to make theoradec be packetized but that fails currently
because of oggdemux and because of the assumptions that the base class makes.
This allows elements to specify a function to upload
a buffer content to a specific OpenGL texture ID. It
could be used by the vaapi elements to provide a way
for eglglessink or WebKit to upload a VA surface to
an GL texture without the respective sinks knowing
anything about VA.
Helps when using dvbsuboverlay in connection with vaapisink
or some other video sink that wants ARGB pixels (dvbsuboverlay
attaches pixels in AYUV format, and we then convert as needed).
Alignment should not be a problem here.
DTS and PTS usually have a non-zero offset between them in MPEG-TS,
so assigning DTS to PTS is almost always wrong. The other, newer
timestamp recovery code does it correctly if we leave it as invalid.
For interlaced vertically subsampled images we need to combine alternating
chroma lines with alternating luma lines. That is line 0 and 2 are combined
with the first line of chroma samples and line 1 and 3 with the second line
of chroma samples.
See also: https://bugzilla.gnome.org/show_bug.cgi?id=588535
The _1_0 suffixed environment variables override the
non-suffixed ones, so if we're in an environment that
sets the _1_0 suffixed ones, such as jhbuild, we need
to set those to make sure ours actually always get
used.
And only return the proportion. The earliest time already can be
retrieved from get_max_decode_time() and by renaming we allow this
to be more extensible in the future.
Add a getter for the QoS proportion and earliest_time to help
subclasses do better estimations based on the proportion.
API: gst_video_decoder_get_qos_info()
https://bugzilla.gnome.org/show_bug.cgi?id=687991
We only allocate 8 bits per component for our temp buffers, which
causes invalid memory accesses if we try to unpack formats that
unpack into a format with 16 bits per component such as e.g. v210.
We don't support blending onto those yet, so just bail out.
This reverts commit e39fbe6b7e.
Looks like we need to pass the full .la file after all in a setup
with libtool, or it might not find the library, e.g. like
ERROR: can't resolve libraries to shared libraries: gstfft-1.0
Conflicts:
gst-libs/gst/audio/Makefile.am
gst-libs/gst/pbutils/Makefile.am
Also see https://bugzilla.gnome.org/show_bug.cgi?id=603710
Monitor for reordered output timestamps, and then avoid oldest DTS
as PTS approach, and try for an oldest PTS as out PTS approach,
if at least all valid PTS available.
Avoids bogus estimating upon sparse available input PTS, and tries
to handle all-keyframe input, or input PTS which are actually DTS.
Hold both the stream and the object lock to modify the output_state,
this way it can be safely modified while hold either one or the other.
Also, only hold the object lock in the query
https://bugzilla.gnome.org/show_bug.cgi?id=684832
... by having some more timestamp tracking in a private frame field.
Not doing so would lead to (a.o.) losing the needed minimum timestamp in
an earlier sent frame.
... rather than to output segment, which will only be set
to current input segment if some output is produced
(coming from non-clipped input).
Also fixup debug message.
Don't try to take STREAM_LOCK on upstream events such as QOS.
Protect qos-related variables with object lock instead. Fixes
possible deadlock when shutting down in certain situations.
https://bugzilla.gnome.org/show_bug.cgi?id=684658
Drain out the decoder when encountering a gap. Needed for DVD 'still'
sequences which consist of a single video frame, and a large gap
while audio plays.
Only hold back events until the first buffer is generated, then just
send them directly. Otherwise, important events like 'still-frame' are
held forever, waiting for a frame that'll never arrive.
Expose the gst_video_info_align function that adds padding and does stride
alignment on a GstVideoInfo. Move this previously private function to
video-info.co
Move the definition of the alignment structure to video.h where it can be picked
up by both the bufferpool and the video-info.
Sometimes the encoder would need to use the allocator for something else than
just allocating output buffers, for example, querying different parameters.
This patch expose a getter accessor for the negotiated memory allocator.
Sometimes the decoder would need to use the pool or the allocator for
something else than just allocating output buffers. For example, the querying
for different parameters, such as asking for a bigger number of buffers to
allocate in the pool.
This patch expose a two getters accessors: one for the buffer pool and the
other for the memory allocator.
Unifies the code and ensures that:
* subclasses needing to use the frame_number on a void* field will
always work
* wraparounds will be automatically taken care of if we have to deal
with more than 2**32 frames
