The case was that the timeline state was being changed from the parent
composition's action thread before the timeline was committed, leading
to the SELECT_STREAM event to be pushed from the track to the nested
timeline from the wrong composition thread.
```
**
GES:ERROR:../subprojects/gst-editing-services/ges/ges-track.c:1263:ges_track_remove_element: assertion failed: (track->priv->valid_thread == g_thread_self())
Bail out! GES:ERROR:../subprojects/gst-editing-services/ges/ges-track.c:1263:ges_track_remove_element: assertion failed: (track->priv->valid_thread == g_thread_self())
Thread 1 (Thread 0x7f6ec2d43700 (LWP 1228982)):
#0 0x00007f6ed85b2a25 in raise () at /lib64/libc.so.6
#1 0x00007f6ed859b895 in abort () at /lib64/libc.so.6
#2 0x00007f6ed899cb8c in g_assertion_message (domain=<optimized out>, file=0x7f6ed8d7fd58 "../subprojects/gst-editing-services/ges/ges-track.c", line=<optimized out>, func=<optimized out>, message=<optimized out>) at ../glib/gtestutils.c:2914
#3 0x00007f6ed89fa9ff in g_assertion_message_expr (domain=domain@entry=0x7f6ed8d76875 "GES", file=file@entry=0x7f6ed8d7fd58 "../subprojects/gst-editing-services/ges/ges-track.c", line=line@entry=1263, func=func@entry=0x7f6ed8d805b0 <__func__.6> "ges_track_remove_element", expr=expr@entry=0x7f6ed8d801e8 "track->priv->valid_thread == g_thread_self()") at ../glib/gtestutils.c:2940
#4 0x00007f6ed8d2658f in ges_track_remove_element (track=track@entry=0x7f6eb4119b20 [GESAudioTrack], object=object@entry=0x106f240 [GESAudioUriSource]) at ../subprojects/gst-editing-services/ges/ges-track.c:1263
#5 0x00007f6ed8d10842 in ges_clip_empty_from_track (clip=0x7f6e7803ee80 [GESUriClip], track=track@entry=0x7f6eb4119b20 [GESAudioTrack]) at ../subprojects/gst-editing-services/ges/ges-clip.c:1086
#6 0x00007f6ed8d01453 in ges_timeline_remove_track (timeline=timeline@entry=0x7f6e6c01ae50 [GESTimeline], track=0x7f6eb4119b20 [GESAudioTrack]) at ../subprojects/gst-editing-services/ges/ges-timeline.c:2460
#7 0x00007f6ed8d0286b in ges_timeline_send_event (element=<optimized out>, event=<optimized out>) at ../subprojects/gst-editing-services/ges/ges-timeline.c:484
#8 0x00007f6ed8bf466c in gst_element_send_event (element=0x7f6e6c01ae50 [GESTimeline], event=event@entry=0x7f6eb410f9f0) at ../subprojects/gstreamer/gst/gstelement.c:1934
#9 0x00007f6ed8d242cd in ges_track_handle_message (bin=0xd846f0 [GESVideoTrack], message=0x7f6eb411ac90) at ../subprojects/gst-editing-services/ges/ges-track.c:477
#10 0x00007f6ed8bc9128 in bin_bus_handler (bus=<optimized out>, message=<optimized out>, bin=<optimized out>) at ../subprojects/gstreamer/gst/gstbin.c:3286
#11 0x00007f6ed8bdbae2 in gst_bus_post (bus=bus@entry=0xfdf440 [GstBus], message=message@entry=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstbus.c:359
#12 0x00007f6ed8bf1396 in gst_element_post_message_default (element=element@entry=0x10261d0 [NleComposition], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2067
#13 0x00007f6ed8bccbee in gst_bin_post_message (element=0x10261d0 [NleComposition], msg=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstmessage.h:376
#14 0x00007f6ed8bf4b66 in gst_element_post_message (element=0x10261d0 [NleComposition], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2110
#15 0x00007f6ed8bc9128 in bin_bus_handler (bus=<optimized out>, message=<optimized out>, bin=<optimized out>) at ../subprojects/gstreamer/gst/gstbin.c:3286
#16 0x00007f6ed8bdbae2 in gst_bus_post (bus=bus@entry=0xfdf2c0 [GstBus], message=message@entry=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstbus.c:359
#17 0x00007f6ed8bf1396 in gst_element_post_message_default (element=element@entry=0x1029110 [GstBin], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2067
#18 0x00007f6ed8bccbee in gst_bin_post_message (element=0x1029110 [GstBin], msg=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstmessage.h:376
#19 0x00007f6ed8bf4b66 in gst_element_post_message (element=0x1029110 [GstBin], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2110
#20 0x00007f6ed8bc9128 in bin_bus_handler (bus=<optimized out>, message=<optimized out>, bin=<optimized out>) at ../subprojects/gstreamer/gst/gstbin.c:3286
#21 0x00007f6ed8bdbae2 in gst_bus_post (bus=bus@entry=0xfdf500 [GstBus], message=message@entry=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstbus.c:359
#22 0x00007f6ed8bf1396 in gst_element_post_message_default (element=element@entry=0xd705e0 [NleSource], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2067
#23 0x00007f6ed8bccbee in gst_bin_post_message (element=0xd705e0 [NleSource], msg=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstmessage.h:376
#24 0x00007f6ed8bf4b66 in gst_element_post_message (element=0xd705e0 [NleSource], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2110
#25 0x00007f6ed8bc9128 in bin_bus_handler (bus=<optimized out>, message=<optimized out>, bin=<optimized out>) at ../subprojects/gstreamer/gst/gstbin.c:3286
#26 0x00007f6ed8bdbae2 in gst_bus_post (bus=bus@entry=0x1042400 [GstBus], message=message@entry=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstbus.c:359
#27 0x00007f6ed8bf1396 in gst_element_post_message_default (element=element@entry=0x1029450 [GstBin], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2067
#28 0x00007f6ed8bccbee in gst_bin_post_message (element=0x1029450 [GstBin], msg=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstmessage.h:376
#29 0x00007f6ed8bf4b66 in gst_element_post_message (element=0x1029450 [GstBin], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2110
#30 0x00007f6ed8bc9128 in bin_bus_handler (bus=<optimized out>, message=<optimized out>, bin=<optimized out>) at ../subprojects/gstreamer/gst/gstbin.c:3286
#31 0x00007f6ed8bdbae2 in gst_bus_post (bus=bus@entry=0x1042640 [GstBus], message=message@entry=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstbus.c:359
#32 0x00007f6ed8bf1396 in gst_element_post_message_default (element=element@entry=0x7f6eb42fc7a0 [GstURIDecodeBin], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2067
#33 0x00007f6ed8bccbee in gst_bin_post_message (element=0x7f6eb42fc7a0 [GstURIDecodeBin], msg=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstmessage.h:376
#34 0x00007f6ed8bf4b66 in gst_element_post_message (element=0x7f6eb42fc7a0 [GstURIDecodeBin], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2110
#35 0x00007f6ed8bc9128 in bin_bus_handler (bus=<optimized out>, message=<optimized out>, bin=<optimized out>) at ../subprojects/gstreamer/gst/gstbin.c:3286
#36 0x00007f6ed8bdbae2 in gst_bus_post (bus=bus@entry=0x7f6eb80a7130 [GstBus], message=message@entry=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstbus.c:359
#37 0x00007f6ed8bf1396 in gst_element_post_message_default (element=element@entry=0x7f6e6c02aa60 [GstDecodeBin], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2067
#38 0x00007f6ed8bccbee in gst_bin_post_message (element=0x7f6e6c02aa60 [GstDecodeBin], msg=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstmessage.h:376
#39 0x00007f6ed8bf4b66 in gst_element_post_message (element=0x7f6e6c02aa60 [GstDecodeBin], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2110
#40 0x00007f6ec8f1e00d in gst_decode_bin_handle_message (bin=0x7f6e6c02aa60 [GstDecodeBin], msg=<optimized out>) at ../subprojects/gst-plugins-base/gst/playback/gstdecodebin2.c:5667
#41 0x00007f6ed8bc9128 in bin_bus_handler (bus=<optimized out>, message=<optimized out>, bin=<optimized out>) at ../subprojects/gstreamer/gst/gstbin.c:3286
#42 0x00007f6ed8bdbae2 in gst_bus_post (bus=bus@entry=0x7f6eb4139110 [GstBus], message=message@entry=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstbus.c:359
#43 0x00007f6ed8bf1396 in gst_element_post_message_default (element=element@entry=0x7f6e54038c70 [GESDemux], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2067
#44 0x00007f6ed8bccbee in gst_bin_post_message (element=0x7f6e54038c70 [GESDemux], msg=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstmessage.h:376
#45 0x00007f6ed8bf4b66 in gst_element_post_message (element=0x7f6e54038c70 [GESDemux], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2110
#46 0x00007f6ed8bc9128 in bin_bus_handler (bus=<optimized out>, message=<optimized out>, bin=<optimized out>) at ../subprojects/gstreamer/gst/gstbin.c:3286
#47 0x00007f6ed8bdbae2 in gst_bus_post (bus=bus@entry=0x7f6eb4139350 [GstBus], message=message@entry=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstbus.c:359
#48 0x00007f6ed8bf1396 in gst_element_post_message_default (element=element@entry=0x7f6e6c01ae50 [GESTimeline], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2067
#49 0x00007f6ed8bccbee in gst_bin_post_message (element=0x7f6e6c01ae50 [GESTimeline], msg=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstmessage.h:376
#50 0x00007f6ed8bf4b66 in gst_element_post_message (element=element@entry=0x7f6e6c01ae50 [GESTimeline], message=0x7f6eb411ac90) at ../subprojects/gstreamer/gst/gstelement.c:2110
#51 0x00007f6ed8cfa221 in ges_timeline_change_state (element=0x7f6e6c01ae50 [GESTimeline], transition=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gst-editing-services/ges/ges-timeline.c:450
#52 0x00007f6ed8bf5d1e in gst_element_change_state (element=element@entry=0x7f6e6c01ae50 [GESTimeline], transition=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:3033
#53 0x00007f6ed8bf6868 in gst_element_continue_state (element=element@entry=0x7f6e6c01ae50 [GESTimeline], ret=ret@entry=GST_STATE_CHANGE_SUCCESS) at ../subprojects/gstreamer/gst/gstelement.c:2741
#54 0x00007f6ed8bf5d67 in gst_element_change_state (element=element@entry=0x7f6e6c01ae50 [GESTimeline], transition=transition@entry=GST_STATE_CHANGE_NULL_TO_READY) at ../subprojects/gstreamer/gst/gstelement.c:3072
#55 0x00007f6ed8bf6368 in gst_element_set_state_func (element=0x7f6e6c01ae50 [GESTimeline], state=GST_STATE_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:2987
#56 0x00007f6ed8bd2129 in gst_bin_element_set_state (next=GST_STATE_PAUSED, current=GST_STATE_PAUSED, start_time=0, base_time=0, element=0x7f6e6c01ae50 [GESTimeline], bin=0x7f6e54038c70 [GESDemux]) at ../subprojects/gstreamer/gst/gstbin.c:2615
#57 gst_bin_change_state_func (element=0x7f6e54038c70 [GESDemux], transition=GST_STATE_CHANGE_PAUSED_TO_PAUSED) at ../subprojects/gstreamer/gst/gstbin.c:2957
#58 0x00007f6ed8bf5d1e in gst_element_change_state (element=element@entry=0x7f6e54038c70 [GESDemux], transition=transition@entry=GST_STATE_CHANGE_PAUSED_TO_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:3033
#59 0x00007f6ed8bf6368 in gst_element_set_state_func (element=0x7f6e54038c70 [GESDemux], state=GST_STATE_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:2987
#60 0x00007f6ed8bd2129 in gst_bin_element_set_state (next=GST_STATE_PAUSED, current=GST_STATE_READY, start_time=0, base_time=0, element=0x7f6e54038c70 [GESDemux], bin=0x7f6e6c02aa60 [GstDecodeBin]) at ../subprojects/gstreamer/gst/gstbin.c:2615
#61 gst_bin_change_state_func (element=0x7f6e6c02aa60 [GstDecodeBin], transition=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstbin.c:2957
#62 0x00007f6ec8f1e84f in gst_decode_bin_change_state (element=0x7f6e6c02aa60 [GstDecodeBin], transition=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gst-plugins-base/gst/playback/gstdecodebin2.c:5482
#63 0x00007f6ed8bf5d1e in gst_element_change_state (element=element@entry=0x7f6e6c02aa60 [GstDecodeBin], transition=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:3033
#64 0x00007f6ed8bf6868 in gst_element_continue_state (element=element@entry=0x7f6e6c02aa60 [GstDecodeBin], ret=ret@entry=GST_STATE_CHANGE_SUCCESS) at ../subprojects/gstreamer/gst/gstelement.c:2741
#65 0x00007f6ed8bf5d67 in gst_element_change_state (element=element@entry=0x7f6e6c02aa60 [GstDecodeBin], transition=transition@entry=GST_STATE_CHANGE_NULL_TO_READY) at ../subprojects/gstreamer/gst/gstelement.c:3072
#66 0x00007f6ed8bf6368 in gst_element_set_state_func (element=0x7f6e6c02aa60 [GstDecodeBin], state=GST_STATE_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:2987
#67 0x00007f6ed8bf5ae8 in gst_element_sync_state_with_parent (element=0x7f6e6c02aa60 [GstDecodeBin]) at ../subprojects/gstreamer/gst/gstelement.c:2413
#68 0x00007f6ed89f17a0 in g_slist_foreach (list=<optimized out>, func=0x7f6ed8bf5a50 <gst_element_sync_state_with_parent>, user_data=user_data@entry=0x0) at ../glib/gslist.c:880
#69 0x00007f6ec8f37d45 in gst_uri_decode_bin_change_state (element=<optimized out>, transition=<optimized out>) at ../subprojects/gst-plugins-base/gst/playback/gsturidecodebin.c:2869
#70 0x00007f6ed8bf5d1e in gst_element_change_state (element=element@entry=0x7f6eb42fc7a0 [GstURIDecodeBin], transition=transition@entry=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:3033
#71 0x00007f6ed8bf6368 in gst_element_set_state_func (element=0x7f6eb42fc7a0 [GstURIDecodeBin], state=GST_STATE_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:2987
#72 0x00007f6ed8bd2129 in gst_bin_element_set_state (next=GST_STATE_PAUSED, current=GST_STATE_READY, start_time=0, base_time=0, element=0x7f6eb42fc7a0 [GstURIDecodeBin], bin=0x1029450 [GstBin]) at ../subprojects/gstreamer/gst/gstbin.c:2615
#73 gst_bin_change_state_func (element=0x1029450 [GstBin], transition=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstbin.c:2957
#74 0x00007f6ed8bf5d1e in gst_element_change_state (element=element@entry=0x1029450 [GstBin], transition=transition@entry=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:3033
#75 0x00007f6ed8bf6368 in gst_element_set_state_func (element=0x1029450 [GstBin], state=GST_STATE_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:2987
#76 0x00007f6ed8bd2129 in gst_bin_element_set_state (next=GST_STATE_PAUSED, current=GST_STATE_READY, start_time=0, base_time=0, element=0x1029450 [GstBin], bin=0xd705e0 [NleSource]) at ../subprojects/gstreamer/gst/gstbin.c:2615
#77 gst_bin_change_state_func (element=0xd705e0 [NleSource], transition=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstbin.c:2957
#78 0x00007f6ec805533f in nle_object_change_state (element=0xd705e0 [NleSource], transition=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gst-editing-services/plugins/nle/nleobject.c:748
#79 0x00007f6ed8bf5d1e in gst_element_change_state (element=element@entry=0xd705e0 [NleSource], transition=transition@entry=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:3033
#80 0x00007f6ed8bf6368 in gst_element_set_state_func (element=0xd705e0 [NleSource], state=GST_STATE_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:2987
#81 0x00007f6ed8bd2129 in gst_bin_element_set_state (next=GST_STATE_PAUSED, current=GST_STATE_READY, start_time=0, base_time=0, element=0xd705e0 [NleSource], bin=0x1029110 [GstBin]) at ../subprojects/gstreamer/gst/gstbin.c:2615
#82 gst_bin_change_state_func (element=0x1029110 [GstBin], transition=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstbin.c:2957
#83 0x00007f6ed8bf5d1e in gst_element_change_state (element=element@entry=0x1029110 [GstBin], transition=transition@entry=GST_STATE_CHANGE_READY_TO_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:3033
#84 0x00007f6ed8bf6368 in gst_element_set_state_func (element=0x1029110 [GstBin], state=GST_STATE_PAUSED) at ../subprojects/gstreamer/gst/gstelement.c:2987
#85 0x00007f6ed8bf5ae8 in gst_element_sync_state_with_parent (element=0x1029110 [GstBin]) at ../subprojects/gstreamer/gst/gstelement.c:2413
#86 0x00007f6ec8060356 in _activate_new_stack (toplevel_seek=<optimized out>, comp=0x10261d0 [NleComposition]) at ../subprojects/gst-editing-services/plugins/nle/nlecomposition.c:3117
#87 update_pipeline (comp=comp@entry=0x10261d0 [NleComposition], currenttime=<optimized out>, seqnum=<optimized out>, update_reason=update_reason@entry=COMP_UPDATE_STACK_INITIALIZE) at ../subprojects/gst-editing-services/plugins/nle/nlecomposition.c:3396
#88 0x00007f6ec80614f6 in _initialize_stack_func (comp=0x10261d0 [NleComposition], ucompo=0x108c800) at ../subprojects/gst-editing-services/plugins/nle/nlecomposition.c:732
#89 0x00007f6ed893788a in g_closure_invoke (closure=<optimized out>, return_value=<optimized out>, n_param_values=<optimized out>, param_values=<optimized out>, invocation_hint=<optimized out>) at ../gobject/gclosure.c:810
#90 0x00007f6ec805aaf6 in _execute_actions (comp=0x10261d0 [NleComposition]) at ../subprojects/gst-editing-services/plugins/nle/nlecomposition.c:412
#91 0x00007f6ed8c4c1cf in gst_task_func (task=0x7f6e6c01c290 [GstTask]) at ../subprojects/gstreamer/gst/gsttask.c:328
#92 0x00007f6ed89fc0f4 in g_thread_pool_thread_proxy (data=<optimized out>) at ../glib/gthreadpool.c:354
#93 0x00007f6ed89fb7f2 in g_thread_proxy (data=0x7f6eb0017800) at ../glib/gthread.c:807
#94 0x00007f6ed7e14432 in start_thread () at /lib64/libpthread.so.0
#95 0x00007f6ed86779d3 in clone () at /lib64/libc.so.6
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-editing-services/-/merge_requests/167>
Basically the tags we send before STREAM_START are now ignored, meaning
that we could not detect nested timelines anymore, this commits makes
sure that we send our tag event after getting pushing STREAM_START.
Implement lazy loading asset cache so gesdemux use the formatters
assets while GES hasn't been initialized.
And set extensions to temporary files as some formatters require
the information (otio)
Adding a property to let the application know
Also make sure that the duration of nested timeline assets is reported
as CLOCK_TIME_NONE as those are extended as necessary.
And make a difference between asset duration and their max duration
As nested timelines can be extended 'infinitely' those max duration
is GST_CLOCK_TIME_NONE, but their duration is the real duration of
the timeline.
When we have nested timelines, we need to make sure the underlying
formatted file is reloaded when commiting the main composition to
take into account the new timeline.
In other to make the implementation as simple as possible we make
sure that whenever the toplevel composition is commited, the decodebin
holding the gesdemux is torn down so that a new demuxer is created
with the new content of the timeline.
To do that a we do a NleCompositionQueryNeedsTearDown query to which
gesdemux answers leading to a full nlecomposition stack
deactivation/activation cycle.
First marshilling it to the main thread is dangerous as it is a blocking
operation and it should never happen there.
The asset cache is MT safe now so it is possible to load the timeline
from that thread directly
Making sure to have 1 GstDiscoverer per thread.
Use that new feature in gesdemux by loading the timeline directly from
the streaming thread. Modifying the timeline is not supported allowed
anyway.
/usr/bin/ld: .libs/libgstges_la-gesdemux.o: in function `ges_timeline_new_from_uri_from_main_thread':
./plugins/ges/gesdemux.c:279: undefined reference to `gst_discoverer_new'
/usr/bin/ld: ./plugins/ges/gesdemux.c:288: undefined reference to `gst_discoverer_start'
