gstreamer/docs/design/draft-keyframe-force.txt

Forcing keyframes
-----------------

Consider the following use case:

  We have a pipeline that performs video and audio capture from a live source,
  compresses and muxes the streams and writes the resulting data into a file.

  Inside the uncompressed video data we have a specific pattern inserted at
  specific moments that should trigger a switch to a new file, meaning, we close
  the existing file we are writing to and start writing to a new file.

  We want the new file to start with a keyframe so that one can start decoding
  the file immediatly.

Components:

  1) We need an element that is able to detect the pattern in the video stream.

  2) We need to inform the video encoder that it should start encoding a keyframe
     starting from exactly the frame with the pattern.

  3) We need to inform the demuxer that it should flush out any pending data and
     start creating the start of a new file with the keyframe as a first video
     frame.

  4) We need to inform the sink element that it should start writing to the next
     file. This requires application interaction to instruct the sink of the new
     filename. The application should also be free to ignore the boundary and
     continue to write to the existing file. The application will typically use
     an event pad probe to detect the custom event.

Implementation:

 The implementation would consist of generating a GST_EVENT_CUSTOM_DOWNSTREAM
 event that marks the keyframe boundary. This event is inserted into the
 pipeline by the application upon a certain trigger. In the above use case this
 trigger would be given by the element that detects the pattern, in the form of
 an element message.
 
 The custom event would travel further downstream to instruct encoder, muxer and
 sink about the possible switch.

 The information passed in the event consists of:

  name:  GstForceKeyUnit
	 (G_TYPE_UINT64)"timestamp"    : the timestamp of the buffer that
	                                 triggered the event.
	 (G_TYPE_UINT64)"stream-time"  : the stream position that triggered the
	                                 event.
	 (G_TYPE_UINT64)"running-time" : the running time of the stream when the 
	                                 event was triggered.
	 ....                          : optional other data fields.

  Note that this event is purely informational, no element is required to
  perform an action but it should forward the event downstream, just like any
  other event it does not handle.

  Elements understanding the event should behave as follows:

  1) The video encoder receives the event before the next frame. Upon reception
     of the event it schedules to encode the next frame as a keyframe. 
     Before pushing out the encoded keyframe it must push the GstForceKeyUnit
     event downstream.

  2) The muxer receives the GstForceKeyUnit event and flushes out its current state,
     preparing to produce data that can be used as a keyunit. Before pushing out
     the new data it pushes the GstForceKeyUnit event downstream.

  3) The application receives the GstForceKeyUnit on a sink padprobe of the sink
     and reconfigures the sink to make it perform new actions after receiving
     the next buffer. 


Upstream
--------

When using RTP packets can get lost or receivers can be added at any time,
they may request a new key frame.

An downstream element sends an upstream "GstForceKeyUnit" event up the
pipeline.

When an element produces some kind of key unit in output, but has
no such concept in its input (like an encoder that takes raw frames),
it consumes the event (doesn't pass it upstream), and instead sends
a downstream GstForceKeyUnit event and a new keyframe.