We may need to drop the slices such as RASL pictures with the NoRaslOutputFlag, so
the current picture of h265decoder may be freed. We should not assign the frame->
output_buffer too early until we really output it. Or, the later coming slices will
allocate another picture and trigger the assert of:
gst_video_decoder_allocate_output_frame_with_params:
assertion 'frame->output_buffer == NULL' failed
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2421>
There is currently no way for users to receive incoming events from
appsink while keeping them properly serialized with the buffers flow.
This can be especially useful when application is injecting custom
downstream events into the pipeline and needs to know when they reached
appsink.
Solving this by adding a new signal notifying about new incoming events
and a set of action signals and method to pull those events.
The API is actually pulling the samples and events all together as they
are actually fetched from the same queue.
Having a specific API to pull only events would have the side effect of
discarding samples (and pulling samples would discard events) making
this API not convenient for users.
Partially fix#247
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1046>
In H265, the stream may have odd bit depth such as 9 or 11. And
the bit depth of luma and chroma may differ. For example, the
stream with luma depth of 8 and chroma depth of 9 should use the
10 bit rtformat as the decoded picture format.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2420>
The problem is that EGLNativeWindowSurface and wl_egl_surface are the
same object underneath, so we must recreate both together. As an
optimization, the EGLNativeWindowSurface wrapper is only re-created
if the window_handle changed.
On Mesa, this would cause crash, which will be fixed by:
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11979
And will lead to proper errors in the future or on other GL stack. This
issue was encounter using a permanent GstGLDisplay after cycling one of
multiple independent pipelines through NULL state.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1230>
In low_latency mode, try to bump the picture as soon as possible
without the frames disorder:
1. We can directly output the continuous non-reference frame.
2. Consider max_num_reorder_frames, which is special useful for
I-P mode.
3. Consider the leading pictures with negative POC.
4 Output small POC pictures when non-reference frame comes.
4. Output the POC increment<=2 pictures. This is not 100% safe,
but in practice this condition can be used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
The picture->ref field will change from time to time according to decoder's
state and reference sliding window. We need another flag to record whether
the picture is a reference picture when it is created, and this can help
the bumping check.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
Accord to spec, we should not add the current picture into the DPB
when we check whether it needs to bump, so the checks of the IDR and
the "memory_management_control_operation equal to 5" are no needed.
And the spec also says that the DPB only needs to bump when there is
no empty frame buffer left(We handle the IDR cases in other places).
We need to follow that and the max_num_reorder_frames is useless.
We also minus 1 in has_empty_frame_buffer because the current frame
has not been added yet.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
When current frame memory_management_control_operation equal to 5, that
means we need to drain the dpb and the current picture act as an IDR frame.
So it should have smaller poc than the later pictures to ensure the output
order.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
* If we have an index table for non-framed essence, we can handle it
* The demuxer has a state which indicates whether it will next fetch a KLV or
data contained *within* a KLV.
* The position on Essence Tracks always correspond to the next entry to fetch,
demuxer offset will be skipped accordingly whenever we switch between
partitions (in case of resyncs). A copy of the main clip/custom KLV for that
partition is kept to track the position within the essence of that partition.
* For clip/custom-wrapped raw audio, if the edit rate is too small (and would
cause plenty of tiny buffers to be outputted), specify a minimum number of edit
units per buffer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2371>
* For pull-based, this avoids pulling content if it's not needed (ex: skipping filler
packet, not downloading the content if we only need to know if/where an essence
packet is, etc...). Allows reducing i/o usage to the minimum.
* This also allows doing sub-klv position tracking, and opens the way for
non-frame-wrapping handling
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2371>
In order to figure out the exact start position (backed by a keyframe) accross
all tracks, we first figure out the backing keyframe position, and *then* seek
to that position.
Avoids ending up in situations where we would properly seek to the backing
keyframe on video ... but not on the audio streams (they would have been set to
the original non-keyframe position). Fixes key-unit seeking.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2371>
The picture essence coding matching was wrong. Use the proper "base" MXFUL for
video mpeg compression for matching.
Also handle the case where some old files would put the essence container label
in the essence coding field
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2371>
* Streamline offset <=> entry handling. Historically the demuxer didn't support
information from index tables and stored the discovered information in an array
per track. When index table support was added, a parallel system was setup for
that relationship. This commit unifies this into one system with the
`find_edit_entry()` and `find_entry_for_offset()` functions.
* By extension, per-track offset entry tables are only created/used if no index
table is present for those tracks.
* Use index table information as-is. The index table system from MXF is quite
complex and there are various ways to use the information contained
within. Instead of converting that information we store the data from the tables
as-is and extract the needed information when needed.
* Handle index tables without entries (i.e. all content package units are of the
same size).
* Allow collecting index table segments as we go instead of only once if a
random-index-pack is present. This also improves support of some files in
push-mode.
* When searching for keyframe entries, use the keyframe_offset if
present (speeds up searching).
* For interleaved content (i.e. several tracks in the sample essence container),
we use a system to be able to identify the position of each track in the delta
entries of index tables.
* Handle temporal offset only on tracks which *do* need it (as specified in the
delta entries of the index tables). If present, those offsets are stored in a
pre-processed table which allows computing PTS from DTS with a simple offset.
* Add a quirk for files which are known to be have wrongly stored temporal
offsets.
* Overall opens the way to handle more types of MXF files, especially those with
non-frame-wrapping.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2371>
This is similar to how the same issue was handled in qtdemux.
In order for the "DTS <= PTS" constraint to be respected, we calculate the
maximum temporal reordering that can happen (via index tables).
If there is a non-0 temporal reordering, we:
* Shift all outgoing PTS by that amount
* Shift segment for that stream by that amount
* Don't modify DTS (i.e. they might end up having negative running-time, before
the start of the segment)
Also ensure all entries have a valid PTS set, previously this wouldn't be set
for entries with a temporal offset of 0.
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/584
(and maybe a lot of other issues)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2371>
120 bytes in 5 blocks are definitely lost in loss record 7,615 of 9,510
at 0x484486F: malloc (vg_replace_malloc.c:380)
by 0x58A2938: g_malloc (gmem.c:106)
by 0x58BA1F4: g_slice_alloc (gslice.c:1069)
by 0x588F059: g_list_prepend (glist.c:335)
by 0x5B9C5C0: select_best_master_clock (gstptpclock.c:756)
by 0x5B9CA8E: cleanup_cb (gstptpclock.c:1930)
by 0x589AD20: g_timeout_dispatch (gmain.c:4889)
by 0x589A4CE: UnknownInlinedFun (gmain.c:3337)
by 0x589A4CE: g_main_context_dispatch (gmain.c:4055)
by 0x58EE4E7: g_main_context_iterate.constprop.0 (gmain.c:4131)
by 0x5899A92: g_main_loop_run (gmain.c:4329)
by 0x5B9BA4C: ptp_helper_main (gstptpclock.c:1980)
by 0x58C8C31: g_thread_proxy (gthread.c:826)
576 bytes in 24 blocks are definitely lost in loss record 8,782 of 9,510
at 0x484486F: malloc (vg_replace_malloc.c:380)
by 0x58A2938: g_malloc (gmem.c:106)
by 0x58BA1F4: g_slice_alloc (gslice.c:1069)
by 0x588F059: g_list_prepend (glist.c:335)
by 0x5B9C5C0: select_best_master_clock (gstptpclock.c:756)
by 0x5B9EFA0: handle_announce_message (gstptpclock.c:934)
by 0x5B9EFA0: handle_ptp_message (gstptpclock.c:1765)
by 0x5B9EFA0: have_stdin_data_cb (gstptpclock.c:1851)
by 0x589A4CE: UnknownInlinedFun (gmain.c:3337)
by 0x589A4CE: g_main_context_dispatch (gmain.c:4055)
by 0x58EE4E7: g_main_context_iterate.constprop.0 (gmain.c:4131)
by 0x5899A92: g_main_loop_run (gmain.c:4329)
by 0x5B9BA4C: ptp_helper_main (gstptpclock.c:1980)
by 0x58C8C31: g_thread_proxy (gthread.c:826)
by 0x5DA4298: start_thread (pthread_create.c:481)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/852>
Currently the decoder selection is very naive: The type with the highest
rank that matches the current caps is used. This works well for software
decoders. The exact supported caps are always known and the static caps are
defined accordingly.
With hardware decoders, e.g. vaapi, the situation is different. The decoder
may reject the caps later during a caps query. At that point, a new decoder
is created. However, the same type is chosen an after several tries,
decodebin fails.
To avoid this, do the caps query while adding the decoder and try again
with other decoder types if the query fails:
1. create the decoder from the next matching type
2. add and link the decoder
3. change the decoder state to READY
4. do the caps query
if it fails then remove the decoder again and go back to 1.
5. expose the source pad
6. sync the decoder state with the parent.
This way, the decoder is already part of the pipeline when the state change
to READY happens. So context handling should work as before.
Exposing the source pad after the query was successful is important:
Otherwise the thread from the decoder source pad may block in a blocked pad
downstream in the playsink waiting for other pads to be ready.
The thread now blocks trying to set the state back to NULL while holding
the SELECTION_LOCK. Other streams may block on the SELECTION_LOCK and the
playsink never unblocks the pad. The result is a deadlock.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1201>
Renamed gst_va_decoder_set_format() to
gst_va_decoder_set_frame_size_with_surfaces() which resembles better
the passed parameters. Internally it creates the vaContext.
Added gst_va_decoder_set_frame_size() which is an alias of
gst_va_decoder_set_frame_size_with_surfaces() without surfaces. This
is the function which replaces gst_va_decoder_set_format() where
used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2417>
We should use the NumPocTotalCurr value stored in decoder, which is a calculated
valid value, rather than use the invalid value in the slice header. Most of the
time, the NumPocTotalCurr is 0 and make the tmp_refs a very short length, and
causes the decoder's wrong result.
By the way, the NumPocTotalCurr is not the correct name specified in H265 spec,
its name should be NumPicTotalCurr. We change it to the correct name.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2414>
Some GPUs support BGRA format and it will be converted to subsampled
YUV format by GPU internally. Disable this implicit conversion
since the conversion parameters such as input/output colorimetry
are not exposed nor it's written in bitstream (e.g., VUI).
We prefer explicit conversion via our conversion elements.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2410>
When the window is called, we properly destroy all surfaces, which effectively
will unmap that surface and should make it disapear on screen, but we also
destroy the wl_source, a GSource that is resposibble of dispatching and executing
messages to/from the Wayland server.
As a side effect, the server never gets the message and the surfaces are
"leaked" on the server. We fix this using wl_display_flush() before destroying
the wl_source.
Fixes#815
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1226>
Some manifests use the ContentProtection node to store additional information
such as the license server url. Our MPD parser used to process the
ContentProtection node, extracting Playready PSSH boxes. However for other DRM
systems, only the `value` attribute was passed down to the protection event, so
for example, Widevine data was not parsed at all and "Widevine" was passed to
the event, which is not very useful for decryptors that require a PSSH init
data.
Parsing should now be done by decryptors which will receive the entire
ContentProtection XML node as a string. This gives more "freedom" to the
decryptor which can then detect and parse custom nodes as well.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2400>
The VP9 streams have the ability to change the resolution dynamically
at any time point. It does not send ad KEY frame before change the
resolution, even the INTER frame can change the resolution immediately.
So we need to check the resolution change for each frame and do the
re-negiotiation if needed.
Some insaned stream may play in resolution A first and then dynamically
changes to B, and after 1 or 2 frames, it use a show_existing_frame to
repeat the old frame of resolution A before. So, not only new_picture(),
but also duplicate_picture() need to check this.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2407>
Some codecs such as VP9, its config and context have the ability to
dynamically. When we only change the width and height, no need to
re-create the config and context. The helper function can just change
the resolution without re-creating config and context.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2407>
