gst: New Stream listing/selection system

* GstStream
* GstStreamCollection
* GST_EVENT_SELECT_STREAMS
* GST_MESSAGE_STREAM_COLLECTION

docs/design/part-stream-selection.txt

@@ -0,0 +1,611 @@
+Stream selection
+----------------
+
+ History
+ v0.1: Jun 11th 2015
+    Initial Draft
+ v0.2: Sep 18th 2015
+    Update to reflect design changes
+ v1.0: Jun 28th 2016
+    Pre-commit revision
+
+  This document describes the events and objects involved in stream
+  selection in GStreamer pipelines, elements and applications
+
+
+0. Background
+----------------
+
+  This new API is intended to address the use cases described in
+  this section:
+
+   1) As a user/app I want an overview and control of the media streams
+      that can be configured within a pipeline for processing, even
+      when some streams are mutually exclusive or logical constructs only.
+
+   2) The user/app can disable entirely streams it's not interested
+      in so they don't occupy memory or processing power - discarded
+      as early as possible in the pipeline. The user/app can also
+      (re-)enable them at a later time.
+
+   3) If the set of possible stream configurations is changing,
+      the user/app should be aware of the pending change and
+      be able to make configuration choices for the new set of streams,
+      as well as possibly still reconfiguring the old set
+
+   4) Elements that have some other internal mechanism for triggering
+      stream selections (DVD, or maybe some scripted playback
+      playlist) should be able to trigger 'selection' of some particular
+      stream.
+
+   5) Indicate known relationships between streams - for example that
+      2 separate video feeds represent the 2 views of a stereoscopic
+      view, or that certain streams are mutually exclusive.
+
+Note: the streams that are "available" are not automatically
+the ones active, or present in the pipeline as pads. Think HLS/DASH
+alternate streams.
+
+Use case examples:
+
+  1) Playing an MPEG-TS multi-program stream, we want to tell the 
+     app that there are multiple programs that could be extracted
+     from the incoming feed. Further, we want to provide a mechanism
+     for the app to select which program(s) to decode, and once
+     that is known to further tell the app which elementary streams
+     are then available within those program(s) so the app/user can
+     choose which audio track(s) to decode and/or use.
+  
+  2) A new PMT arrives for an MPEG-TS stream, due to a codec or
+     channel change. The pipeline will need to reconfigure to 
+     play the desired streams from new program. Equally, there
+     may be multiple seconds of content buffered from the old
+     program and it should still be possible to switch (for example)
+     subtitle tracks responsively in the draining out data, as
+     well as selecting which subs track to play from the new feed.
+  
+     This same scenario applies when doing gapless transition to a
+     new source file/URL, except that likely the element providing
+     the list of streams also changes as a new demuxer is installed.
+  
+  3) When playing a multi-angle DVD, the DVD Virtual Machine needs to
+     extract 1 angle from the data for presentation. It can publish
+     the available angles as logical streams, even though only one
+     stream can be chosen.
+  
+  4) When playing a DVD, the user can make stream selections from the
+     DVD menu to choose audio or sub-picture tracks, or the DVD VM
+     can trigger automatic selections. In addition, the player UI
+     should be able to show which audio/subtitle tracks are available
+     and allow direct selection in a GUI the same as for normal
+     files with subtitle tracks in them.
+  
+  5) Playing a SCHC (3DTV) feed, where one view is MPEG-2 and the other
+     is H.264 and they should be combined for 3D presentation, or
+     not bother decoding 1 stream if displaying 2D.
+     (bug https://bugzilla.gnome.org/show_bug.cgi?id=719333)
+  
+  *) FIXME - need some use cases indicating what alternate streams in
+     HLS might require - what are the possibilities?
+
+
+1. Design Overview
+-----------
+
+  Stream selection in GStreamer is implemented in several parts:
+  1) Objects describing streams : GstStream
+  2) Objects describing a collection of streams : GstStreamCollection
+  3) Events from the app allowing selection and activation of some streams:
+     GST_EVENT_SELECT_STREAMS
+  4) Messages informing the user/application about the available
+     streams and current status:
+     GST_MESSAGE_STREAM_COLLECTION
+     GST_MESSAGE_STREAMS_SELECTED
+
+
+2. GstStream objects
+--------------------
+
+  API: GstStream
+  API: gst_stream_new(..)
+  API: gst_stream_get_*(...)
+  API: gst_stream_set_*()
+  API: gst_event_set_stream(...)
+  API: gst_event_parse_stream(...)
+
+  GstStream objects are a high-level convenience object containing
+  information regarding a possible data stream that can be exposed by
+  GStreamer elements.
+
+  They are mostly the aggregation of information present in other
+  GStreamer components (STREAM_START, CAPS, TAGS event) but are not
+  tied to the presence of a GstPad, and for some use-cases provide
+  information that the existing components don't provide.
+
+  The various properties of a GstStream object are:
+    - stream_id (from the STREAM_START event)
+    - flags (from the STREAM_START event)
+    - caps
+    - tags
+    - type (high-level type of stream: Audio, Video, Container,...)
+
+  GstStream objects can be subclassed so that they can be re-used by
+  elements already using the notion of stream (which is common for
+  example in demuxers).
+
+  Elements that create GstStream should also set it on the
+  GST_EVENT_STREAM_START event of the relevent pad. This helps
+  downstream elements to have all information in one location.
+
+
+3. Exposing collections of streams
+----------------------------------
+
+  API: GstStreamCollection
+  API: gst_stream_collection_new(...)
+  API: gst_stream_collection_add_stream(...)
+  API: gst_stream_collection_get_size(...)
+  API: gst_stream_collection_get_stream(...)
+  API: GST_MESSAGE_STREAM_COLLECTION
+  API: gst_message_new_stream_collection(...)
+  API: gst_message_parse_stream_collection(...)
+  API: GST_EVENT_STREAM_COLLECTION
+  API: gst_event_new_stream_collection(...)
+  API: gst_event_parse_stream_collection(...)
+
+  Elements that create new streams (such as demuxers) or can create
+  new streams (like the HLS/DASH alternative streams) can list the
+  streams they can make available with the GstStreamCollection object.
+
+  Other elements that might generate GstStreamCollections are the
+  DVD-VM, which handles internal switching of tracks, or parsebin and
+  decodebin3 when it aggregates and presents multiple internal stream
+  sources as a single configurable collection.
+
+  The GstStreamCollection object is a flat listing of GstStream objects.
+
+  The various properties of a GstStreamCollection are:
+    - 'identifier'
+        - the identifier of the collection (unique name)
+        - Generated from the 'upstream stream id' (or stream ids, plural)
+    - the list of GstStreams in the collection.
+    - (Not implemented) : Flags -
+        For now, the only flag is 'INFORMATIONAL' - used by container parsers to
+        publish information about detected streams without allowing selection of
+        the streams.
+    - (Not implemented yet) : The relationship between the various streams
+      This specifies which streams are exclusive (can not be selected at the
+      same time), are related (such as LINKED_VIEW or ENHANCEMENT), or need to
+      be selected together.
+
+  An element will inform outside components about that collection via:
+  * a GST_MESSAGE_STREAM_COLLECTION message on the bus.
+  * a GST_EVENT_STREAM_COLLECTION on each source pads.
+
+  Applications and container bin elements can listen and collect the
+  various stream collections to know the full range of streams
+  available within a bin/pipeline.
+
+  Once posted on the bus, a GstStreamCollection is immutable. It is
+  updated by subsquent messages with a matching identifier.
+
+  If the element that provided the collection goes away, there is no way
+  to know that the streams are no longer valid (without having the
+  user/app track that element). The exception to that is if the bin
+  containing that element (such as parsebin or decodebin3) informs that
+  the next collection is a replacement of the former one.
+
+  The mutual exclusion and relationship lists use stream-ids
+  rather than GstStream references in order to avoid circular
+  referencing problems.
+
+
+3.1 Usage from elements
+-----------------------
+
+  When a demuxer knows the list of streams it can expose, it
+  creates a new GstStream for each stream it can provide with the
+  appropriate information (stream id, flag, tags, caps, ...).
+
+  The demuxer then creates a GstStreamCollection object in which it
+  will put the list of GstStream it can expose.  That collection is
+  then both posted on the bus (via a GST_MESSAGE_COLLECTION) and on
+  each pad (via a GST_EVENT_STREAM_COLLECTION).
+
+  That new collection must be posted on the bus *before* the changes
+  are made available. i.e. before pads corresponding to that selection
+  are added/removed.
+
+  In order to be backwards-compatible and support elements that don't
+  create streams/collection yet, the new 'parsebin' element used by
+  decodebin3 will automatically create those if not provided.
+
+
+3.2 Usage from application
+--------------------------
+
+  Applications can know what streams are available by listening to the
+  GST_MESSAGE_STREAM_COLLECTION messages posted on the bus.
+
+  The application can list the available streams per-type (such as all
+  the audio streams, or all the video streams) by iterating the
+  streams available in the collection by GST_STREAM_TYPE.
+
+  The application will also be able to use these stream information to
+  decide which streams should be activated or not (see the stream
+  selection event below).
+
+
+3.3 Backwards compatibility
+---------------------------
+
+  Not all demuxers will create the various GstStream and
+  GstStreamCollection objects. In order to remain backwards
+  compatible, a parent bin (parsebin in decodebin3) will create the
+  GstStream and GstStreamCollection based on the pads being
+  added/removed from an element.
+
+  This allows providing stream listing/selection for any demuxer-like
+  element even if it doesn't implement the GstStreamCollection usage.
+
+
+4. Stream selection event
+-------------------------
+
+  API: GST_EVENT_SELECT_STREAMS
+  API: gst_event_new_select_streams(...)
+  API: gst_event_parse_select_streams(...)
+
+  Stream selection events are generated by the application and
+  sent into the pipeline to configure the streams.
+
+  The event carries:
+    * List of GstStreams to activate - a subset of the GstStreamCollection
+    * (Not implemented) - List of GstStreams to be kept discarded - a
+      subset of streams for which hot-swapping will not be desired,
+      allowing elements (such as decodebin3, demuxers, ...) to not parse or
+      buffer those streams at all.
+
+
+4.1. Usage from application
+---------------------------
+
+  There are two use-cases where an application needs to specify in a
+  generic fashion which streams it wants in output:
+  1) When there are several present streams of which it only wants a
+    subset (such as one audio, one video and one subtitle
+    stream). Those streams are demuxed and present in the pipeline.
+  2) When the stream the user wants require some element to undertake
+    some action to expose that stream in the pipeline (such as
+    DASH/HLS alternative streams).
+
+  From the point of view of the application, those two use-cases are
+  treated identically.  The streams are all available through the
+  GstStreamCollection posted on the bus, and it will select a subset.
+
+  The application can select the streams it wants by creating a
+  GST_EVENT_SELECT_STREAMS event with the list of stream-id of the
+  streams it wants. That event is then sent on the pipeline,
+  eventually travelling all the way upstream from each sink.
+
+  In some cases, selecting one stream may trigger the availability of
+  other dependent streams, resulting in new GstStreamCollection
+  messages. This can happen in the case where chosing a different DVB
+  channel would create a new single-program collection.
+
+
+4.2. Usage in elements
+----------------------
+
+  Elements that receive the GST_EVENT_SELECT_STREAMS event and that
+  can activate/deactivate streams need to look at the list of
+  stream-id contained in the event and decide if they need to do some
+  action.
+
+  In the standard demuxer case (demuxing and exposing all streams),
+  there is nothing to do by default.
+
+  In decodebin3, activating or deactivating streams is taken care of by
+  linking only the streams present in the event to decoders and output
+  ghostpad.
+
+  In the case of elements that can expose alternate streams that are
+  not present in the pipeline as pads, they will take the appropriate
+  action to add/remove those streams.
+
+  Containers that receive the event should pass it to any elements
+  with no downstream peers, so that streams can be configured during
+  pre-roll before a pipeline is completely linked down to sinks.
+
+
+5. decodebin3 usage and example
+-------------------------------
+
+  This is an example of how decodebin3 works by using the
+  above-mentioned objects/events/messages.
+
+  For clarity/completeness, we will consider a mpeg-ts stream that has
+  multiple audio streams. Furthermore that stream might have changes
+  at some point (switching video codec, or adding/removing audio
+  streams).
+
+
+5.1. Initial differences
+------------------------
+
+  decodebin3 is different, compared to decodebin2, in the sense that, by
+  default:
+  * it will only expose as output ghost source pads one stream of each
+    type (one audio, one video, ..).
+  * It will only decode the exposed streams
+
+  The multiqueue element is still used and takes in all elementary
+  (non-decoded) streams. If parsers are needed/present they are placed
+  before the multiqueue. This is needed in order for multiqueue to
+  work only with packetized and properly timestamped streams.
+
+  Note that the whole typefinding of streams, and optional depayloading,
+  demuxing and parsing is done in a new 'parsebin' element.
+
+  Just like the current implementation, demuxers will expose all
+  streams present within a program as source pads. They will connect
+  to parsers and multiqueue.
+
+  Initial setup. 1 video stream, 2 audio streams.
+
+    +---------------------+
+    | parsebin            |
+    | ---------           | +-------------+
+    | | demux |--[parser]-+-| multiqueue  |--[videodec]---[
+  ]-+-|       |--[parser]-+-|             |
+    | |       |--[parser]-+-|             |--[audiodec]---[
+    | ---------           | +-------------+
+    +---------------------+
+
+
+5.2. GstStreamCollection
+------------------------
+
+  When parsing the initial PAT/PMT, the demuxer will:
+  1) create the various GstStream objects for each stream.
+  2) create the GstStreamCollection for that initial PMT
+  3) post the GST_MESSAGE_STREAM_COLLECTION
+
+  Decodebin will intercept that message and know what the demuxer will
+  be exposing.
+
+  4) The demuxer creates the various pads and sends the corresponding
+  STREAM_START event (with the same stream-id as the corresponding
+  GstStream objects), CAPS event, and TAGS event.
+
+  parsebin will add all relevant parsers and expose those streams.
+
+  Decodebin will be able to correlate, based on STREAM_START event
+  stream-id, what pad corresponds to which stream. It links each stream
+  from parsebin to multiqueue.
+
+  Decodebin knows all the streams that will be available. Since by
+  default it is configured to only expose a stream of each type, it
+  will pick a stream of each for which it will complete the
+  auto-plugging (finding a decoder and then exposing that stream as a
+  source ghostpad.
+
+  Note:
+  If the demuxer doesn't create/post the GstStreamCollection,
+  parsebin will create it on itself, as explained in section 2.3
+  above.  
+
+
+5.3. Changing the active selection from the application
+-------------------------------------------------------
+
+  The user wants to change the audio track. The application received
+  the GST_MESSAGE_STREAM_COLLECTION containing the list of available
+  streams. For clarity, we will assume those stream-ids are
+  "video-main", "audio-english" and "audio-french".
+
+  The user prefers to use the french soundtrack (which it knows based
+  on the language tag contained in the GstStream objects).
+
+  The application will create and send a GST_EVENT_SELECT_STREAM event
+  containing the list of streams: "video-main", "audio-french".
+
+  That event gets sent on the pipeline, the sinks send it upstream and
+  eventually reach decodebin.
+
+  Decodebin compares:
+  * The currently active selection ("video-main", "audio-english")
+  * The available stream collection ("video-main", "audio-english",
+    "audio-french")
+  * The list of streams in the event ("video-main", "audio-french")
+
+  Decodebin determines that no change is required for "video-main",
+  but sees that it needs to deactivate "audio-english" and activate
+  "audio-french".
+
+  It unlinks the multiqueue source pad connected to the audiodec. Then
+  it queries audiodec, using the GST_QUERY_ACCEPT_CAPS, whether it can
+  accept as-is the caps from the "audio-french" stream.
+  1) If it does, the multiqueue source pad corresponding to
+     "audio-french" is linked to the decoder.
+  2) If it does not, the existing audio decoder is removed,
+     a new decoder is selected (like during initial
+     auto-plugging), and replaces the old audio decoder element.
+
+  The newly selected stream gets decoded and output through the same
+  pad as the previous audio stream.
+
+  Note:
+  The default behaviour would be to only expose one stream of each
+  type. But nothing prevents decodebin from outputting more/less of
+  each type if the GST_EVENT_SELECT_STREAM event specifies that. This
+  allows covering more use-case than the simple playback one.
+  Such examples could be :
+    * Wanting just a video stream or just a audio stream
+    * Wanting all decoded streams
+    * Wanting all audio streams
+    ...
+
+
+5.4. Changes coming from upstream
+---------------------------------
+
+  At some point in time, a PMT change happens. Let's assume a change
+  in video-codec and/or PID.
+
+  The demuxer creates a new GstStream for the changed/new stream,
+  creates a new GstStreamCollection for the updated PMT and posts it.
+
+  Decodebin sees the new GstStreamCollection message.
+
+  The demuxer (and parsebin) then adds and removes pads.
+  1) decodebin will match the new pads to GstStream in the "new"
+    GstStreamCollection the same way it did for the initial pads in
+    section 4.2 above.
+  2) decodebin will see whether the new stream can re-use a multiqueue
+    slot used by a stream of the same type no longer present (it
+    compares the old collection to the new collection).
+    In this case, decodebin sees that the new video stream can re-use
+    the same slot as the previous video stream.
+  3) If the new stream is going to be active by default (in this case
+    it does because we are replacing the only video stream, which was
+    active), it will check whether the caps are compatible with the
+    existing videodec (in the same way it was done for the audio
+    decoder switch in section 4.3).
+
+  Eventually, the stream that switched will be decoded and output
+  through the same pad as the previous video stream in a gapless fashion.
+
+
+5.5. Further examples
+---------------------
+
+5.5.1. HLS alternates
+---------------------
+
+    There is a main (multi-bitrate or not) stream with audio and
+    video interleaved in mpeg-ts. The manifest also indicates the
+    presence of alternate language audio-only streams.
+    HLS would expose one collection containing:
+       1) The main A+V CONTAINER stream (mpeg-ts), initially active,
+         downloaded and exposed as a pad
+       2) The alternate A-only streams, initially inactive and not
+         exposed as pads
+    the tsdemux element connected to the first stream will also
+    expose a collection containing
+       1.1) A video stream
+       1.2) An audio stream
+
+      [ Collection 1 ]         [ Collection 2 ]
+      [  (hlsdemux)  ]         [   (tsdemux)  ]
+      [ upstream:nil ]    /----[ upstream:main]
+      [              ]   /     [              ]
+      [ "main" (A+V) ]<-/      [ "video"  (V) ]  viddec1 : "video"
+      [ "fre"  (A)   ]         [ "eng"    (A) ]  auddec1 : "eng"
+      [ "kor"  (A)   ]         [              ]
+
+      The user might want to use the korean audio track instead of the
+      default english one.
+        => SELECT_STREAMS ("video", "kor")
+
+      
+      1) decodebin3 receives and sends the event further upstream
+      2) tsdemux sees that "video" is part of its current upstream,
+        so adds the corresponding stream-id ("main") to the event
+	and sends it upstream ("main", "video", "kor")
+      3) hlsdemux receives the event
+        => It activates "kor" in addition to "main"
+      4) The event travels back to decodebin3 which will remember the
+        requested selection. If "kor" is already present it will switch
+	the "eng" stream from the audio decoder to the "kor" stream.
+	If it appears a bit later, it will wait until that "kor" stream
+	is available before switching
+
+
+5.5.2 multi-program MPEG-TS
+---------------------------
+
+     Assuming the case of a mpeg-ts stream which contains multiple
+     programs.
+     There would be three "levels" of collection:
+        1) The collection of programs present in the stream
+	2) The collection of elementary streams present in a stream
+        3) The collection of streams decodebin can expose
+
+     Initially tsdemux exposes the first program present (default)
+
+      [ Collection 1 ]         [ Collection 2     ]        [ Collection 3    ]
+      [  (tsdemux)   ]         [   (tsdemux)      ]        [ (decodebin)     ]
+      [ id:Programs  ]<-\      [ id:BBC1          ]<-\     [ id:BBC1-decoded ]
+      [ upstream:nil ]   \-----[ upstream:Programs]   \----[ upstream:BBC1   ]
+      [              ]         [                  ]        [                 ]
+      [ "BBC1" (C)   ]         [ id:"bbcvideo"(V) ]        [ id:"bbcvideo"(V)]
+      [ "ITV"  (C)   ]         [ id:"bbcaudio"(A) ]        [ id:"bbcvideo"(A)]
+      [ "NBC"  (C)   ]         [                  ]        [                 ]
+
+     At some point the user wants to switch to ITV (of which we do not
+     know the topology at this point in time. A SELECT_STREAMS event
+     is sent with "ITV" in it and the pointer to the Collection1.
+     1) The event travels up the pipeline until tsdemux receives it
+        and begins the switch.
+     2) tsdemux publishes a new 'Collection 2a/ITV' and marks 'Collection 2/BBC'
+        as replaced.
+     2a) App may send a SELECT_STREAMS event configuring which demuxer output
+        streams should selected (parsed)
+     3) tsdemux adds/removes pads as needed (flushing pads as it removes them?)
+     4) Decodebin feeds new pad streams through existing parsers/decoders as
+        needed. As data from the new collection arrives out each decoder,
+        decodebin sends new GstStreamCollection messages to the app so it
+        can know that the new streams are now switchable at that level.
+     4a) As new GstStreamCollections are published, the app may override
+        the default decodebin stream selection to expose more/fewer streams.
+        The default is to decode and output 1 stream of each type.
+
+     Final state:
+
+      [ Collection 1 ]         [ Collection 4     ]        [ Collection 5    ]
+      [  (tsdemux)   ]         [   (tsdemux)      ]        [ (decodebin)     ]
+      [ id:Programs  ]<-\      [ id:ITV           ]<-\     [ id:ITV-decoded  ]
+      [ upstream:nil ]   \-----[ upstream:Programs]   \----[ upstream:ITV    ]
+      [              ]         [                  ]        [                 ]
+      [ "BBC1" (C)   ]         [ id:"itvvideo"(V) ]        [ id:"itvvideo"(V)]
+      [ "ITV"  (C)   ]         [ id:"itvaudio"(A) ]        [ id:"itvvideo"(A)]
+      [ "NBC"  (C)   ]         [                  ]        [                 ]
+
+6.0 TODO
+--------
+
+* Add missing implementation
+  - Add flags to GstStreamCollection
+  - Add mutual-exclusion and relationship API to GstStreamCollection
+
+* Add helper API to figure out whether a collection is a replacement of another
+  or a completely new one. This will require a more generic system to know whether
+  a certain stream-id is a replacement of another or not.
+
+
+7.0 OPEN QUESTIONS
+------------------
+
+* Is a FLUSHING flag for stream-selection required or not ?
+  This would make the handler of the SELECT_STREAMS event send FLUSH START/STOP
+  before switching to the other streams.
+  This is tricky when dealing where situations where we keep some streams and
+  only switch some others. Do we flush all streams ? Do we only flush the new
+  streams, potentially resulting in delay to fully switch ?
+  Furthermore, due to efficient buffering in decodebin3, the switching time has
+  been minimized extensively, to the point where flushing might not bring a
+  noticeable improvement.
+
+* Store the stream collection in bins/pipelines ?
+  A Bin/Pipeline could store all active collection internally, so that it
+   could be queried later on. This could be useful to then get, on any pipeline,
+   at any point in time, the full list of collections available without having
+   to listen to all COLLECTION messages on the bus.
+  This would require fixing the "is a collection a replacement or not" issue first.
+
+* When switching to new collections, should decodebin3 make any effort
+  to 'map' corresponding streams from the old to new PMT - that is,
+  try and stick to the 'english' language audio track, for example?
+  Alternatively, rely on the app to do such smarts with stream-select
+  messages ?

docs/gst/gstreamer-docs.sgml

@@ -99,6 +99,8 @@ Windows.  It is released under the GNU Library General Public License
     <xi:include href="xml/gstquery.xml" />
     <xi:include href="xml/gstregistry.xml" />
     <xi:include href="xml/gstsegment.xml" />
+    <xi:include href="xml/gststreams.xml" />
+    <xi:include href="xml/gststreamcollection.xml" />
     <xi:include href="xml/gststructure.xml" />
     <xi:include href="xml/gstsystemclock.xml" />
     <xi:include href="xml/gsttaglist.xml" />

docs/gst/gstreamer-sections.txt

@@ -1123,6 +1123,9 @@ gst_event_parse_stream_flags
 gst_event_set_group_id
 gst_event_parse_group_id
 
+gst_event_set_stream
+gst_event_parse_stream
+
 gst_event_new_segment
 gst_event_parse_segment
 gst_event_copy_segment
@@ -1169,6 +1172,12 @@ gst_event_parse_segment_done
 
 gst_event_new_protection
 gst_event_parse_protection
+
+gst_event_new_select_streams
+gst_event_parse_select_streams
+
+gst_event_new_stream_collection
+gst_event_parse_stream_collection
 <SUBSECTION Standard>
 GstEventClass
 GST_EVENT
@@ -1621,6 +1630,8 @@ gst_message_new_stream_start
 gst_message_set_group_id
 gst_message_parse_group_id
 
+gst_message_new_stream_collection
+gst_message_parse_stream_collection
 GstStructureChangeType
 gst_message_new_structure_change
 gst_message_parse_structure_change
@@ -1649,6 +1660,13 @@ gst_message_parse_device_removed
 
 gst_message_new_property_notify
 gst_message_parse_property_notify
+
+gst_message_new_streams_selected
+gst_message_parse_streams_selected
+gst_message_streams_selected_add
+gst_message_streams_selected_get_size
+gst_message_streams_selected_get_stream
+
 <SUBSECTION Standard>
 GstMessageClass
 GST_MESSAGE
@@ -1977,6 +1995,7 @@ gst_pad_create_stream_id_printf
 gst_pad_create_stream_id_printf_valist
 
 gst_pad_get_stream_id
+gst_pad_get_stream
 
 GstPadForwardFunction
 gst_pad_forward
@@ -2616,6 +2635,60 @@ gst_segment_flags_get_type
 <SUBSECTION Private>
 </SECTION>
 
+<SECTION>
+<FILE>gststreams</FILE>
+<TITLE>GstStream</TITLE>
+GstStream
+GstStreamClass
+GstStreamType
+gst_stream_new
+gst_stream_get_caps
+gst_stream_get_stream_flags
+gst_stream_get_stream_id
+gst_stream_get_stream_type
+gst_stream_get_tags
+gst_stream_set_caps
+gst_stream_set_stream_flags
+gst_stream_set_stream_type
+gst_stream_set_tags
+gst_stream_type_get_name
+<SUBSECTION Standard>
+GST_IS_STREAM
+GST_IS_STREAM_CLASS
+GST_STREAM
+GST_STREAM_CAST
+GST_STREAM_CLASS
+GST_STREAM_GET_CLASS
+GST_TYPE_STREAM
+GST_TYPE_STREAM_TYPE
+gst_stream_get_type
+gst_stream_type_get_type
+<SUBSECTION Private>
+GstStreamPrivate
+</SECTION>
+
+<SECTION>
+<FILE>gststreamcollection</FILE>
+<TITLE>GstStreamCollection</TITLE>
+GstStreamCollection
+GstStreamCollectionClass
+gst_stream_collection_new
+gst_stream_collection_add_stream
+gst_stream_collection_get_upstream_id
+gst_stream_collection_get_size
+gst_stream_collection_get_stream
+<SUBSECTION Standard>
+gst_stream_collection_get_type
+GST_IS_STREAM_COLLECTION
+GST_IS_STREAM_COLLECTION_CLASS
+GST_STREAM_COLLECTION
+GST_STREAM_COLLECTION_CAST
+GST_STREAM_COLLECTION_CLASS
+GST_STREAM_COLLECTION_GET_CLASS
+GST_TYPE_STREAM_COLLECTION
+<SUBSECTION Private>
+GstStreamCollectionPrivate
+</SECTION>
 
 <SECTION>
 <FILE>gststructure</FILE>

gst/Makefile.am

@@ -106,6 +106,8 @@ libgstreamer_@GST_API_VERSION@_la_SOURCES = \
 	gstregistrychunks.c	\
 	gstsample.c		\
 	gstsegment.c		\
+	gststreamcollection.c	\
+	gststreams.c		\
 	gststructure.c		\
 	gstsystemclock.c	\
 	gsttaglist.c		\
@@ -214,6 +216,8 @@ gst_headers =			\
 	gstquery.h		\
 	gstsample.h		\
 	gstsegment.h		\
+	gststreamcollection.h	\
+	gststreams.h		\
 	gststructure.h		\
 	gstsystemclock.h	\
 	gsttaglist.h		\

gst/gst.c

@@ -684,6 +684,7 @@ init_post (GOptionContext * context, GOptionGroup * group, gpointer data,
   g_type_class_ref (gst_lock_flags_get_type ());
   g_type_class_ref (gst_allocator_flags_get_type ());
   g_type_class_ref (gst_stream_flags_get_type ());
+  g_type_class_ref (gst_stream_type_get_type ());
 
   _priv_gst_event_initialize ();
   _priv_gst_buffer_initialize ();
@@ -1123,6 +1124,7 @@ gst_deinit (void)
   g_type_class_unref (g_type_class_peek (gst_pad_probe_return_get_type ()));
   g_type_class_unref (g_type_class_peek (gst_segment_flags_get_type ()));
   g_type_class_unref (g_type_class_peek (gst_scheduling_flags_get_type ()));
+  g_type_class_unref (g_type_class_peek (gst_stream_type_get_type ()));
 
   g_type_class_unref (g_type_class_peek (gst_control_binding_get_type ()));
   g_type_class_unref (g_type_class_peek (gst_control_source_get_type ()));

gst/gst.h

@@ -58,6 +58,7 @@
 #include <gst/gstmeta.h>
 #include <gst/gstminiobject.h>
 #include <gst/gstobject.h>
+#include <gst/gststreamcollection.h>
 #include <gst/gstpad.h>
 #include <gst/gstparamspecs.h>
 #include <gst/gstpipeline.h>
@@ -69,6 +70,7 @@
 #include <gst/gstregistry.h>
 #include <gst/gstsample.h>
 #include <gst/gstsegment.h>
+#include <gst/gststreams.h>
 #include <gst/gststructure.h>
 #include <gst/gstsystemclock.h>
 #include <gst/gsttaglist.h>

gst/gstevent.c

@@ -104,7 +104,9 @@ static GstEventQuarks event_quarks[] = {
   {GST_EVENT_UNKNOWN, "unknown", 0},
   {GST_EVENT_FLUSH_START, "flush-start", 0},
   {GST_EVENT_FLUSH_STOP, "flush-stop", 0},
+  {GST_EVENT_SELECT_STREAMS, "select-streams", 0},
   {GST_EVENT_STREAM_START, "stream-start", 0},
+  {GST_EVENT_STREAM_COLLECTION, "stream-collection", 0},
   {GST_EVENT_CAPS, "caps", 0},
   {GST_EVENT_SEGMENT, "segment", 0},
   {GST_EVENT_TAG, "tag", 0},
@@ -575,6 +577,77 @@ gst_event_parse_flush_stop (GstEvent * event, gboolean * reset_time)
             GST_QUARK (RESET_TIME)));
 }
 
+/**
+ * gst_event_new_select_streams:
+ * @streams: (element-type gchar) (transfer none): the list of streams to
+ * activate
+ *
+ * Allocate a new select-streams event.
+ *
+ * The select-streams event requests the specified @streams to be activated.
+ *
+ * The list of @streams corresponds to the "Stream ID" of each stream to be
+ * activated. Those ID can be obtained via the #GstStream objects present
+ * in #GST_EVENT_STREAM_START, #GST_EVENT_STREAM_COLLECTION or 
+ * #GST_MESSSAGE_STREAM_COLLECTION.
+ *
+ * Returns: (transfer full): a new select-streams event.
+ */
+GstEvent *
+gst_event_new_select_streams (GList * streams)
+{
+  GstEvent *event;
+  GValue val = G_VALUE_INIT;
+  GstStructure *struc;
+  GList *tmpl;
+
+  GST_CAT_INFO (GST_CAT_EVENT, "Creating new select-streams event");
+  struc = gst_structure_new_id_empty (GST_QUARK (EVENT_SELECT_STREAMS));
+  g_value_init (&val, GST_TYPE_LIST);
+  /* Fill struc with streams */
+  for (tmpl = streams; tmpl; tmpl = tmpl->next) {
+    GValue strval = G_VALUE_INIT;
+    const gchar *str = (const gchar *) tmpl->data;
+    g_value_init (&strval, G_TYPE_STRING);
+    g_value_set_string (&strval, str);
+    gst_value_list_append_and_take_value (&val, &strval);
+  }
+  gst_structure_id_take_value (struc, GST_QUARK (STREAMS), &val);
+  event = gst_event_new_custom (GST_EVENT_SELECT_STREAMS, struc);
+
+  return event;
+}
+
+/**
+ * gst_event_parse_select_streams:
+ * @event: The event to parse
+ * @streams: (out) (element-type gchar) (transfer full): the streams
+ *
+ * Parse the SELECT_STREAMS event and retrieve the contained streams.
+ */
+void
+gst_event_parse_select_streams (GstEvent * event, GList ** streams)
+{
+  GstStructure *structure;
+  GList *res = NULL;
+
+  g_return_if_fail (GST_IS_EVENT (event));
+  g_return_if_fail (GST_EVENT_TYPE (event) == GST_EVENT_SELECT_STREAMS);
+
+  structure = GST_EVENT_STRUCTURE (event);
+  if (G_LIKELY (streams)) {
+    const GValue *vlist =
+        gst_structure_id_get_value (structure, GST_QUARK (STREAMS));
+    guint i, sz = gst_value_list_get_size (vlist);
+    for (i = 0; i < sz; i++) {
+      const GValue *strv = gst_value_list_get_value (vlist, i);
+      res = g_list_append (res, g_value_dup_string (strv));
+    }
+    *streams = res;
+  }
+}
+
+
 /**
  * gst_event_new_eos:
  *
@@ -1507,6 +1580,44 @@ gst_event_parse_stream_start (GstEvent * event, const gchar ** stream_id)
     *stream_id = g_value_get_string (val);
 }
 
+/**
+ * gst_event_set_stream:
+ * @event: a stream-start event
+ * @stream: (transfer none): the stream object to set
+ *
+ * Set the @stream on the stream-start @event 
+ **/
+void
+gst_event_set_stream (GstEvent * event, GstStream * stream)
+{
+  g_return_if_fail (event != NULL);
+  g_return_if_fail (GST_EVENT_TYPE (event) == GST_EVENT_STREAM_START);
+  g_return_if_fail (gst_event_is_writable (event));
+
+  gst_structure_id_set (GST_EVENT_STRUCTURE (event),
+      GST_QUARK (STREAM), GST_TYPE_STREAM, stream, NULL);
+}
+
+/**
+ * gst_event_parse_stream:
+ * @event: a stream-start event
+ * @stream: (out) (transfer full): adress of variable to store the stream
+ *
+ * Parse a stream-start @event and extract the #GstStream from it.
+ **/
+void
+gst_event_parse_stream (GstEvent * event, GstStream ** stream)
+{
+  g_return_if_fail (event != NULL);
+  g_return_if_fail (GST_EVENT_TYPE (event) == GST_EVENT_STREAM_START);
+
+  if (stream) {
+    gst_structure_id_get (GST_EVENT_STRUCTURE (event),
+        GST_QUARK (STREAM), GST_TYPE_STREAM, stream, NULL);
+  }
+
+}
+
 /**
  * gst_event_set_stream_flags:
  * @event: a stream-start event
@@ -1595,6 +1706,52 @@ gst_event_parse_group_id (GstEvent * event, guint * group_id)
   return TRUE;
 }
 
+/**
+ * gst_event_new_stream_collection:
+ * @collection: Active collection for this data flow
+ *
+ * Create a new STREAM_COLLECTION event. The stream collection event can only
+ * travel downstream synchronized with the buffer flow.
+ *
+ * Source elements, demuxers and other elements that manage collections
+ * of streams and post #GstStreamCollection messages on the bus also send
+ * this event downstream on each pad involved in the collection, so that
+ * activation of a new collection can be tracked through the downstream
+ * data flow.
+ *
+ * Returns: (transfer full): the new STREAM_COLLECTION event.
+ */
+GstEvent *
+gst_event_new_stream_collection (GstStreamCollection * collection)
+{
+  GstStructure *s;
+
+  g_return_val_if_fail (collection != NULL, NULL);
+  g_return_val_if_fail (GST_IS_STREAM_COLLECTION (collection), NULL);
+
+  s = gst_structure_new_id (GST_QUARK (EVENT_STREAM_COLLECTION),
+      GST_QUARK (COLLECTION), GST_TYPE_STREAM_COLLECTION, collection, NULL);
+
+  return gst_event_new_custom (GST_EVENT_STREAM_COLLECTION, s);
+}
+
+void
+gst_event_parse_stream_collection (GstEvent * event,
+    GstStreamCollection ** collection)
+{
+  const GstStructure *structure;
+
+  g_return_if_fail (event != NULL);
+  g_return_if_fail (GST_EVENT_TYPE (event) == GST_EVENT_STREAM_COLLECTION);
+
+  structure = gst_event_get_structure (event);
+
+  if (collection) {
+    gst_structure_id_get (structure,
+        GST_QUARK (COLLECTION), GST_TYPE_STREAM_COLLECTION, collection, NULL);
+  }
+}
+
 /**
  * gst_event_new_toc:
  * @toc: (transfer none): #GstToc structure.

gst/gstevent.h

@@ -79,6 +79,7 @@ typedef enum {
  *                 from the pipeline and unblock all streaming threads.
  * @GST_EVENT_FLUSH_STOP: Stop a flush operation. This event resets the
  *                 running-time of the pipeline.
+ * @GST_EVENT_SELECT_STREAMS: A request to select one or more streams.
  * @GST_EVENT_STREAM_START: Event to mark the start of a new stream. Sent before any
  *                 other serialized event and only sent at the start of a new stream,
  *                 not after flushing seeks.
@@ -87,6 +88,7 @@ typedef enum {
  *                 segment events contains information for clipping buffers and
  *                 converting buffer timestamps to running-time and
  *                 stream-time.
+ * @GST_EVENT_STREAM_COLLECTION: A new #GstStreamCollection is available.
  * @GST_EVENT_TAG: A new set of metadata tags has been found in the stream.
  * @GST_EVENT_BUFFERSIZE: Notification of buffering requirements. Currently not
  *                 used yet.
@@ -94,7 +96,8 @@ typedef enum {
  *                          send messages that should be emitted in sync with
  *                          rendering.
  * @GST_EVENT_EOS: End-Of-Stream. No more data is to be expected to follow
- *                 without a SEGMENT event.
+ *                 without either a STREAM_START event, or a FLUSH_STOP and a SEGMENT
+ *                 event.
  * @GST_EVENT_SEGMENT_DONE: Marks the end of a segment playback.
  * @GST_EVENT_GAP: Marks a gap in the datastream.
  * @GST_EVENT_TOC: An event which indicates that a new table of contents (TOC)
@@ -144,6 +147,7 @@ typedef enum {
   GST_EVENT_STREAM_START          = GST_EVENT_MAKE_TYPE (40, FLAG(DOWNSTREAM) | FLAG(SERIALIZED) | FLAG(STICKY)),
   GST_EVENT_CAPS                  = GST_EVENT_MAKE_TYPE (50, FLAG(DOWNSTREAM) | FLAG(SERIALIZED) | FLAG(STICKY)),
   GST_EVENT_SEGMENT               = GST_EVENT_MAKE_TYPE (70, FLAG(DOWNSTREAM) | FLAG(SERIALIZED) | FLAG(STICKY)),
+  GST_EVENT_STREAM_COLLECTION     = GST_EVENT_MAKE_TYPE (75, FLAG(DOWNSTREAM) | FLAG(SERIALIZED) | FLAG(STICKY) | FLAG(STICKY_MULTI)),
   GST_EVENT_TAG                   = GST_EVENT_MAKE_TYPE (80, FLAG(DOWNSTREAM) | FLAG(SERIALIZED) | FLAG(STICKY) | FLAG(STICKY_MULTI)),
   GST_EVENT_BUFFERSIZE            = GST_EVENT_MAKE_TYPE (90, FLAG(DOWNSTREAM) | FLAG(SERIALIZED) | FLAG(STICKY)),
   GST_EVENT_SINK_MESSAGE          = GST_EVENT_MAKE_TYPE (100, FLAG(DOWNSTREAM) | FLAG(SERIALIZED) | FLAG(STICKY) | FLAG(STICKY_MULTI)),
@@ -163,6 +167,7 @@ typedef enum {
   GST_EVENT_STEP                  = GST_EVENT_MAKE_TYPE (230, FLAG(UPSTREAM)),
   GST_EVENT_RECONFIGURE           = GST_EVENT_MAKE_TYPE (240, FLAG(UPSTREAM)),
   GST_EVENT_TOC_SELECT            = GST_EVENT_MAKE_TYPE (250, FLAG(UPSTREAM)),
+  GST_EVENT_SELECT_STREAMS        = GST_EVENT_MAKE_TYPE (260, FLAG(UPSTREAM)),
 
   /* custom events start here */
   GST_EVENT_CUSTOM_UPSTREAM          = GST_EVENT_MAKE_TYPE (270, FLAG(UPSTREAM)),
@@ -174,6 +179,30 @@ typedef enum {
 } GstEventType;
 #undef FLAG
 
+/**
+ * GstStreamFlags:
+ * @GST_STREAM_FLAG_NONE: This stream has no special attributes
+ * @GST_STREAM_FLAG_SPARSE: This stream is a sparse stream (e.g. a subtitle
+ *    stream), data may flow only in irregular intervals with large gaps in
+ *    between.
+ * @GST_STREAM_FLAG_SELECT: This stream should be selected by default. This
+ *    flag may be used by demuxers to signal that a stream should be selected
+ *    by default in a playback scenario.
+ * @GST_STREAM_FLAG_UNSELECT: This stream should not be selected by default.
+ *    This flag may be used by demuxers to signal that a stream should not
+ *    be selected by default in a playback scenario, but only if explicitly
+ *    selected by the user (e.g. an audio track for the hard of hearing or
+ *    a director's commentary track).
+ *
+ * Since: 1.2
+ */
+typedef enum {
+  GST_STREAM_FLAG_NONE,
+  GST_STREAM_FLAG_SPARSE       = (1 << 0),
+  GST_STREAM_FLAG_SELECT       = (1 << 1),
+  GST_STREAM_FLAG_UNSELECT     = (1 << 2)
+} GstStreamFlags;
+
 #include <gst/gstminiobject.h>
 #include <gst/gstformat.h>
 #include <gst/gstobject.h>
@@ -355,29 +384,6 @@ typedef enum {
   GST_QOS_TYPE_THROTTLE        = 2
 } GstQOSType;
 
-/**
- * GstStreamFlags:
- * @GST_STREAM_FLAG_NONE: This stream has no special attributes
- * @GST_STREAM_FLAG_SPARSE: This stream is a sparse stream (e.g. a subtitle
- *    stream), data may flow only in irregular intervals with large gaps in
- *    between.
- * @GST_STREAM_FLAG_SELECT: This stream should be selected by default. This
- *    flag may be used by demuxers to signal that a stream should be selected
- *    by default in a playback scenario.
- * @GST_STREAM_FLAG_UNSELECT: This stream should not be selected by default.
- *    This flag may be used by demuxers to signal that a stream should not
- *    be selected by default in a playback scenario, but only if explicitly
- *    selected by the user (e.g. an audio track for the hard of hearing or
- *    a director's commentary track).
- *
- * Since: 1.2
- */
-typedef enum {
-  GST_STREAM_FLAG_NONE,
-  GST_STREAM_FLAG_SPARSE       = (1 << 0),
-  GST_STREAM_FLAG_SELECT       = (1 << 1),
-  GST_STREAM_FLAG_UNSELECT     = (1 << 2)
-} GstStreamFlags;
 
 /**
  * GstEvent:
@@ -467,6 +473,8 @@ void            gst_event_set_running_time_offset (GstEvent *event, gint64 offse
 /* Stream start event */
 GstEvent *      gst_event_new_stream_start      (const gchar *stream_id) G_GNUC_MALLOC;
 void            gst_event_parse_stream_start    (GstEvent *event, const gchar **stream_id);
+void            gst_event_set_stream		(GstEvent *event, GstStream *stream);
+void            gst_event_parse_stream		(GstEvent *event, GstStream **stream);
 
 void            gst_event_set_stream_flags      (GstEvent *event, GstStreamFlags flags);
 void            gst_event_parse_stream_flags    (GstEvent *event, GstStreamFlags *flags);
@@ -480,6 +488,14 @@ GstEvent *      gst_event_new_flush_start       (void) G_GNUC_MALLOC;
 GstEvent *      gst_event_new_flush_stop        (gboolean reset_time) G_GNUC_MALLOC;
 void            gst_event_parse_flush_stop      (GstEvent *event, gboolean *reset_time);
 
+/* Stream collection event */
+GstEvent *      gst_event_new_stream_collection   (GstStreamCollection *collection) G_GNUC_MALLOC;
+void            gst_event_parse_stream_collection (GstEvent *event, GstStreamCollection **collection);
+
+/* select streams event */
+GstEvent *      gst_event_new_select_streams    (GList *streams);
+void            gst_event_parse_select_streams  (GstEvent *event, GList **streams);
+
 /* EOS event */
 GstEvent *      gst_event_new_eos               (void) G_GNUC_MALLOC;
 

gst/gstmessage.c

@@ -52,6 +52,7 @@
 #include "gsttaglist.h"
 #include "gstutils.h"
 #include "gstquark.h"
+#include "gstvalue.h"
 
 
 typedef struct
@@ -106,6 +107,8 @@ static GstMessageQuarks message_quarks[] = {
   {GST_MESSAGE_DEVICE_ADDED, "device-added", 0},
   {GST_MESSAGE_DEVICE_REMOVED, "device-removed", 0},
   {GST_MESSAGE_PROPERTY_NOTIFY, "property-notify", 0},
+  {GST_MESSAGE_STREAM_COLLECTION, "stream-collection", 0},
+  {GST_MESSAGE_STREAMS_SELECTED, "streams-selected", 0},
   {0, NULL, 0}
 };
 
@@ -2522,3 +2525,198 @@ gst_message_parse_property_notify (GstMessage * message, GstObject ** object,
     *property_value =
         gst_structure_id_get_value (s, GST_QUARK (PROPERTY_VALUE));
 }
+
+/**
+ * gst_message_new_stream_collection:
+ * @src: The #GstObject that created the message
+ * @collection: (transfer none): The #GstStreamCollection
+ *
+ * Creates a new stream-collection message. The message is used to announce new
+ * #GstStreamCollection
+ *
+ * Returns: a newly allocated #GstMessage
+ *
+ * Since: 1.x
+ */
+GstMessage *
+gst_message_new_stream_collection (GstObject * src,
+    GstStreamCollection * collection)
+{
+  GstMessage *message;
+  GstStructure *structure;
+
+  g_return_val_if_fail (collection != NULL, NULL);
+  g_return_val_if_fail (GST_IS_STREAM_COLLECTION (collection), NULL);
+
+  structure =
+      gst_structure_new_id (GST_QUARK (MESSAGE_STREAM_COLLECTION),
+      GST_QUARK (COLLECTION), GST_TYPE_STREAM_COLLECTION, collection, NULL);
+  message =
+      gst_message_new_custom (GST_MESSAGE_STREAM_COLLECTION, src, structure);
+
+  return message;
+}
+
+/**
+ * gst_message_parse_stream_collection:
+ * @message: a #GstMessage of type %GST_MESSAGE_STREAM_COLLECTION
+ * @collection: (out) (allow-none) (transfer none): A location where to store a
+ *  pointer to the #GstStreamCollection, or %NULL
+ *
+ * Parses a stream-collection message. 
+ *
+ * Since: 1.x
+ */
+void
+gst_message_parse_stream_collection (GstMessage * message,
+    GstStreamCollection ** collection)
+{
+  g_return_if_fail (GST_IS_MESSAGE (message));
+  g_return_if_fail (GST_MESSAGE_TYPE (message) ==
+      GST_MESSAGE_STREAM_COLLECTION);
+
+  if (collection)
+    gst_structure_id_get (GST_MESSAGE_STRUCTURE (message),
+        GST_QUARK (COLLECTION), GST_TYPE_STREAM_COLLECTION, collection, NULL);
+}
+
+/**
+ * gst_message_new_streams_selected:
+ * @src: The #GstObject that created the message
+ * @collection: (transfer none): The #GstStreamCollection
+ *
+ * Creates a new steams-selected message. The message is used to announce
+ * that an array of streams has been selected. This is generally in response
+ * to a #GST_EVENT_SELECT_STREAMS event, or when an element (such as decodebin3)
+ * makes an initial selection of streams.
+ *
+ * The message also contains the #GstStreamCollection to which the various streams
+ * belong to.
+ *
+ * Users of gst_message_new_streams_selected() can add the selected streams with
+ * gst_message_streams_selected_add().
+ *
+ * Returns: a newly allocated #GstMessage
+ *
+ * Since: 1.x
+ */
+GstMessage *
+gst_message_new_streams_selected (GstObject * src,
+    GstStreamCollection * collection)
+{
+  GstMessage *message;
+  GstStructure *structure;
+  GValue val = G_VALUE_INIT;
+
+  g_return_val_if_fail (collection != NULL, NULL);
+  g_return_val_if_fail (GST_IS_STREAM_COLLECTION (collection), NULL);
+
+  structure =
+      gst_structure_new_id (GST_QUARK (MESSAGE_STREAMS_SELECTED),
+      GST_QUARK (COLLECTION), GST_TYPE_STREAM_COLLECTION, collection, NULL);
+  g_value_init (&val, GST_TYPE_ARRAY);
+  gst_structure_id_take_value (structure, GST_QUARK (STREAMS), &val);
+  message =
+      gst_message_new_custom (GST_MESSAGE_STREAMS_SELECTED, src, structure);
+
+  return message;
+}
+
+/**
+ * gst_message_streams_selected_get_size:
+ * @message: a #GstMessage of type %GST_MESSAGE_STREAMS_SELECTED
+ *
+ * Returns the number of streams contained in the @message.
+ *
+ * Returns: The number of streams contained within.
+ */
+guint
+gst_message_streams_selected_get_size (GstMessage * msg)
+{
+  const GValue *val;
+
+  g_return_val_if_fail (GST_IS_MESSAGE (msg), 0);
+  g_return_val_if_fail (GST_MESSAGE_TYPE (msg) == GST_MESSAGE_STREAMS_SELECTED,
+      0);
+
+  val =
+      gst_structure_id_get_value (GST_MESSAGE_STRUCTURE (msg),
+      GST_QUARK (STREAMS));
+  return gst_value_array_get_size (val);
+}
+
+/**
+ * gst_message_streams_selected_add:
+ * @message: a #GstMessage of type %GST_MESSAGE_STREAMS_SELECTED
+ * @stream: (transfer none): a #GstStream to add to @message
+ *
+ * Adds the @stream to the @message.
+ */
+void
+gst_message_streams_selected_add (GstMessage * msg, GstStream * stream)
+{
+  GValue *val;
+  GValue to_add = G_VALUE_INIT;
+
+  g_return_if_fail (GST_IS_MESSAGE (msg));
+  g_return_if_fail (GST_MESSAGE_TYPE (msg) == GST_MESSAGE_STREAMS_SELECTED);
+  g_return_if_fail (GST_IS_STREAM (stream));
+
+  val =
+      (GValue *) gst_structure_id_get_value (GST_MESSAGE_STRUCTURE (msg),
+      GST_QUARK (STREAMS));
+  g_value_init (&to_add, GST_TYPE_STREAM);
+  g_value_set_object (&to_add, stream);
+  gst_value_array_append_and_take_value (val, &to_add);
+}
+
+/**
+ * gst_message_streams_selected_get_stream:
+ * @message: a #GstMessage of type %GST_MESSAGE_STREAMS_SELECTED
+ * @idx: Index of the stream to retrieve
+ *
+ * Retrieves the #GstStream with index @index from the @message.
+ *
+ * Returns: (transfer full): A #GstStream
+ */
+GstStream *
+gst_message_streams_selected_get_stream (GstMessage * msg, guint idx)
+{
+  const GValue *streams, *val;
+
+  g_return_val_if_fail (GST_IS_MESSAGE (msg), NULL);
+  g_return_val_if_fail (GST_MESSAGE_TYPE (msg) == GST_MESSAGE_STREAMS_SELECTED,
+      NULL);
+
+  streams =
+      gst_structure_id_get_value (GST_MESSAGE_STRUCTURE (msg),
+      GST_QUARK (STREAMS));
+  val = gst_value_array_get_value (streams, idx);
+  if (val) {
+    return (GstStream *) g_value_dup_object (val);
+  }
+
+  return NULL;
+}
+
+/**
+ * gst_message_parse_streams_selected:
+ * @message: a #GstMessage of type %GST_MESSAGE_STREAMS_SELECTED
+ * @collection: (out) (allow-none) (transfer none): A location where to store a
+ *  pointer to the #GstStreamCollection, or %NULL
+ *
+ * Parses a streams-selected message. 
+ *
+ * Since: 1.x
+ */
+void
+gst_message_parse_streams_selected (GstMessage * message,
+    GstStreamCollection ** collection)
+{
+  g_return_if_fail (GST_IS_MESSAGE (message));
+  g_return_if_fail (GST_MESSAGE_TYPE (message) == GST_MESSAGE_STREAMS_SELECTED);
+
+  if (collection)
+    gst_structure_id_get (GST_MESSAGE_STRUCTURE (message),
+        GST_QUARK (COLLECTION), GST_TYPE_STREAM_COLLECTION, collection, NULL);
+}

gst/gstmessage.h

@@ -110,6 +110,10 @@ typedef struct _GstMessage GstMessage;
  *     from a #GstDeviceProvider (Since 1.4)
  * @GST_MESSAGE_PROPERTY_NOTIFY: Message indicating a #GObject property has
  *     changed (Since 1.10)
+ * @GST_MESSAGE_STREAM_COLLECTION: Message indicating a new #GstStreamCollection
+ *     is available.
+ * @GST_MESSAGE_STREAMS_SELECTED: Message indicating the active selection of
+ *     #GstStreams has changed.
  * @GST_MESSAGE_ANY: mask for all of the above messages.
  *
  * The different message types that are available.
@@ -159,6 +163,8 @@ typedef enum
   GST_MESSAGE_DEVICE_ADDED      = GST_MESSAGE_EXTENDED + 1,
   GST_MESSAGE_DEVICE_REMOVED    = GST_MESSAGE_EXTENDED + 2,
   GST_MESSAGE_PROPERTY_NOTIFY   = GST_MESSAGE_EXTENDED + 3,
+  GST_MESSAGE_STREAM_COLLECTION = GST_MESSAGE_EXTENDED + 4,
+  GST_MESSAGE_STREAMS_SELECTED  = GST_MESSAGE_EXTENDED + 5,
   GST_MESSAGE_ANY               = (gint) (0xffffffff)
 } GstMessageType;
 
@@ -170,6 +176,7 @@ typedef enum
 #include <gst/gstquery.h>
 #include <gst/gsttoc.h>
 #include <gst/gstdevice.h>
+#include <gst/gststreamcollection.h>
 
 GST_EXPORT GType _gst_message_type;
 
@@ -599,6 +606,17 @@ void            gst_message_parse_device_removed  (GstMessage * message, GstDevi
 GstMessage *    gst_message_new_property_notify   (GstObject * src, const gchar * property_name, GValue * val) G_GNUC_MALLOC;
 void            gst_message_parse_property_notify (GstMessage * message, GstObject ** object, const gchar ** property_name, const GValue ** property_value);
 
+/* STREAM_COLLECTION */
+GstMessage *    gst_message_new_stream_collection   (GstObject * src, GstStreamCollection * collection) G_GNUC_MALLOC;
+void            gst_message_parse_stream_collection (GstMessage *message, GstStreamCollection **collection);
+
+/* STREAMS_SELECTED */
+GstMessage *    gst_message_new_streams_selected (GstObject *src, GstStreamCollection *collection);
+void            gst_message_streams_selected_add (GstMessage *message, GstStream *stream);
+void            gst_message_parse_streams_selected (GstMessage * message, GstStreamCollection **collection);
+guint           gst_message_streams_selected_get_size (GstMessage * message);
+GstStream      *gst_message_streams_selected_get_stream (GstMessage *message, guint idx);
+
 #ifdef G_DEFINE_AUTOPTR_CLEANUP_FUNC
 G_DEFINE_AUTOPTR_CLEANUP_FUNC(GstMessage, gst_message_unref)
 #endif

gst/gstquark.c

@@ -71,7 +71,9 @@ static const gchar *_quark_strings[] = {
   "GstMessageStreamStart", "group-id", "uri-redirection",
   "GstMessageDeviceAdded", "GstMessageDeviceRemoved", "device",
   "uri-redirection-permanent", "GstMessagePropertyNotify", "property-name",
-  "property-value"
+  "property-value", "streams", "GstEventSelectStreams",
+  "GstMessageStreamCollection", "collection", "stream", "stream-collection",
+  "GstMessageStreamsSelected"
 };
 
 GQuark _priv_gst_quark_table[GST_QUARK_MAX];

gst/gstquark.h

@@ -205,7 +205,14 @@ typedef enum _GstQuarkId
   GST_QUARK_MESSAGE_PROPERTY_NOTIFY = 174,
   GST_QUARK_PROPERTY_NAME = 175,
   GST_QUARK_PROPERTY_VALUE = 176,
-  GST_QUARK_MAX = 177
+  GST_QUARK_STREAMS = 177,
+  GST_QUARK_EVENT_SELECT_STREAMS = 178,
+  GST_QUARK_MESSAGE_STREAM_COLLECTION = 179,
+  GST_QUARK_COLLECTION = 180,
+  GST_QUARK_STREAM = 181,
+  GST_QUARK_EVENT_STREAM_COLLECTION = 182,
+  GST_QUARK_MESSAGE_STREAMS_SELECTED = 183,
+  GST_QUARK_MAX = 184
 } GstQuarkId;
 
 extern GQuark _priv_gst_quark_table[GST_QUARK_MAX];

gst/gststreamcollection.c

@@ -0,0 +1,324 @@
+/* GStreamer
+ *
+ * Copyright (C) 2015 Centricular Ltd
+ *  @author: Edward Hervey <edward@centricular.com>
+ *  @author: Jan Schmidt <jan@centricular.com>
+ *
+ * gststreams.c: GstStreamCollection object and methods
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Library General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Library General Public License for more details.
+ *
+ * You should have received a copy of the GNU Library General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
+ * Boston, MA 02110-1301, USA.
+ *
+ * MT safe.
+ */
+
+/**
+ * SECTION:gststreamcollection
+ * @short_description: Base class for collection of streams
+ *
+ */
+
+#include "gst_private.h"
+
+#include "gstenumtypes.h"
+#include "gstevent.h"
+#include "gststreamcollection.h"
+
+GST_DEBUG_CATEGORY_STATIC (stream_collection_debug);
+#define GST_CAT_DEFAULT stream_collection_debug
+
+#define GST_STREAM_COLLECTION_GET_PRIVATE(obj)  \
+   (G_TYPE_INSTANCE_GET_PRIVATE ((obj), GST_TYPE_STREAM_COLLECTION, GstStreamCollectionPrivate))
+
+struct _GstStreamCollectionPrivate
+{
+  /* Maybe switch this to a GArray if performance is
+   * ever an issue? */
+  GQueue *streams;
+};
+
+/* stream signals and properties */
+enum
+{
+  SIG_STREAM_NOTIFY,
+  LAST_SIGNAL
+};
+
+enum
+{
+  PROP_0,
+  PROP_UPSTREAM_ID,
+  PROP_LAST
+};
+
+static guint gst_stream_collection_signals[LAST_SIGNAL] = { 0 };
+
+static void gst_stream_collection_dispose (GObject * object);
+
+static void gst_stream_collection_set_property (GObject * object, guint prop_id,
+    const GValue * value, GParamSpec * pspec);
+static void gst_stream_collection_get_property (GObject * object, guint prop_id,
+    GValue * value, GParamSpec * pspec);
+
+static void
+proxy_stream_notify_cb (GstStream * stream, GParamSpec * pspec,
+    GstStreamCollection * collection);
+
+#define _do_init				\
+{ \
+  GST_DEBUG_CATEGORY_INIT (stream_collection_debug, "streamcollection", GST_DEBUG_BOLD, \
+      "debugging info for the stream collection objects"); \
+  \
+}
+
+#define gst_stream_collection_parent_class parent_class
+G_DEFINE_TYPE_WITH_CODE (GstStreamCollection, gst_stream_collection,
+    GST_TYPE_OBJECT, _do_init);
+
+static void
+gst_stream_collection_class_init (GstStreamCollectionClass * klass)
+{
+  GObjectClass *gobject_class;
+
+  gobject_class = (GObjectClass *) klass;
+
+  g_type_class_add_private (klass, sizeof (GstStreamCollectionPrivate));
+
+  gobject_class->set_property = gst_stream_collection_set_property;
+  gobject_class->get_property = gst_stream_collection_get_property;
+
+  /**
+   * GstStream:upstream-id:
+   *
+   * stream-id
+   */
+  g_object_class_install_property (gobject_class, PROP_UPSTREAM_ID,
+      g_param_spec_string ("upstream-id", "Upstream ID",
+          "The stream ID of the parent stream",
+          NULL,
+          G_PARAM_READWRITE | G_PARAM_CONSTRUCT | G_PARAM_STATIC_STRINGS));
+
+  /**
+   * GstStream::stream-notify:
+   * @collection: a #GstStreamCollection
+   * @prop_stream: the #GstStream that originated the signal
+   * @prop: the property that changed
+   *
+   * The stream notify signal is used to be notified of property changes to
+   * streams within the collection.
+   */
+  gst_stream_collection_signals[SIG_STREAM_NOTIFY] =
+      g_signal_new ("stream-notify", G_TYPE_FROM_CLASS (klass),
+      G_SIGNAL_RUN_FIRST | G_SIGNAL_NO_RECURSE | G_SIGNAL_DETAILED |
+      G_SIGNAL_NO_HOOKS, G_STRUCT_OFFSET (GstStreamCollectionClass,
+          stream_notify), NULL, NULL, g_cclosure_marshal_generic, G_TYPE_NONE,
+      2, GST_TYPE_STREAM, G_TYPE_PARAM);
+
+  gobject_class->dispose = gst_stream_collection_dispose;
+}
+
+static void
+gst_stream_collection_init (GstStreamCollection * collection)
+{
+  collection->priv = GST_STREAM_COLLECTION_GET_PRIVATE (collection);
+  collection->priv->streams = g_queue_new ();
+}
+
+static void
+release_gst_stream (GstStream * stream, GstStreamCollection * collection)
+{
+  g_signal_handlers_disconnect_by_func (stream,
+      proxy_stream_notify_cb, collection);
+  gst_object_unref (stream);
+}
+
+static void
+gst_stream_collection_dispose (GObject * object)
+{
+  GstStreamCollection *collection = GST_STREAM_COLLECTION_CAST (object);
+
+  if (collection->upstream_id) {
+    g_free (collection->upstream_id);
+    collection->upstream_id = NULL;
+  }
+
+  if (collection->priv->streams) {
+    g_queue_foreach (collection->priv->streams,
+        (GFunc) release_gst_stream, collection);
+    g_queue_free (collection->priv->streams);
+    collection->priv->streams = NULL;
+  }
+
+  G_OBJECT_CLASS (parent_class)->dispose (object);
+}
+
+/**
+ * gst_stream_collection_new:
+ * @upstream_id: (allow-none): The stream id of the parent stream
+ *
+ * Create a new #GstStreamCollection.
+ *
+ * Returns: The new #GstStreamCollection.
+ */
+GstStreamCollection *
+gst_stream_collection_new (const gchar * upstream_id)
+{
+  return g_object_new (GST_TYPE_STREAM_COLLECTION, "upstream-id", upstream_id,
+      NULL);
+}
+
+static void
+gst_stream_collection_set_upstream_id (GstStreamCollection * collection,
+    const gchar * upstream_id)
+{
+  g_return_if_fail (collection->upstream_id == NULL);
+
+  /* Upstream ID should only be set once on construction, but let's
+   * not leak in case someone does something silly */
+  if (collection->upstream_id)
+    g_free (collection->upstream_id);
+
+  if (upstream_id)
+    collection->upstream_id = g_strdup (upstream_id);
+}
+
+/**
+ * gst_stream_collection_get_upstream_id:
+ * @collection: a #GstStreamCollection
+ *
+ * Returns the upstream id of the @collection.
+ *
+ * Returns: (transfer none): The upstream id
+ */
+const gchar *
+gst_stream_collection_get_upstream_id (GstStreamCollection * collection)
+{
+  const gchar *res;
+
+  res = collection->upstream_id;
+
+  return res;
+}
+
+static void
+gst_stream_collection_set_property (GObject * object, guint prop_id,
+    const GValue * value, GParamSpec * pspec)
+{
+  GstStreamCollection *collection;
+
+  collection = GST_STREAM_COLLECTION_CAST (object);
+
+  switch (prop_id) {
+    case PROP_UPSTREAM_ID:
+      gst_stream_collection_set_upstream_id (collection,
+          g_value_get_string (value));
+      break;
+    default:
+      G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
+      break;
+  }
+}
+
+static void
+gst_stream_collection_get_property (GObject * object, guint prop_id,
+    GValue * value, GParamSpec * pspec)
+{
+  GstStreamCollection *collection;
+
+  collection = GST_STREAM_COLLECTION_CAST (object);
+
+  switch (prop_id) {
+    case PROP_UPSTREAM_ID:
+      g_value_set_string (value,
+          gst_stream_collection_get_upstream_id (collection));
+      break;
+    default:
+      G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
+      break;
+  }
+}
+
+static void
+proxy_stream_notify_cb (GstStream * stream, GParamSpec * pspec,
+    GstStreamCollection * collection)
+{
+  GST_DEBUG_OBJECT (collection, "Stream %" GST_PTR_FORMAT " updated %s",
+      stream, pspec->name);
+  g_signal_emit (collection, gst_stream_collection_signals[SIG_STREAM_NOTIFY],
+      g_quark_from_string (pspec->name), stream, pspec);
+}
+
+/**
+ * gst_stream_collection_add_stream:
+ * @collection: a #GstStreamCollection
+ * @stream: (transfer full): the #GstStream to add
+ *
+ * Add the given @stream to the @collection.
+ *
+ * Returns: %TRUE if the @stream was properly added, else %FALSE
+ */
+gboolean
+gst_stream_collection_add_stream (GstStreamCollection * collection,
+    GstStream * stream)
+{
+  g_return_val_if_fail (GST_IS_STREAM_COLLECTION (collection), FALSE);
+  g_return_val_if_fail (GST_IS_STREAM (stream), FALSE);
+  g_return_val_if_fail (collection->priv->streams, FALSE);
+
+  GST_DEBUG_OBJECT (collection, "Adding stream %" GST_PTR_FORMAT, stream);
+
+  g_queue_push_tail (collection->priv->streams, stream);
+  g_signal_connect (stream, "notify", (GCallback) proxy_stream_notify_cb,
+      collection);
+
+  return TRUE;
+}
+
+/**
+ * gst_stream_collection_get_size:
+ * @collection: a #GstStreamCollection
+ *
+ * Get the number of streams this collection contains
+ *
+ * Returns: The number of streams that @collection contains
+ */
+guint
+gst_stream_collection_get_size (GstStreamCollection * collection)
+{
+  g_return_val_if_fail (GST_IS_STREAM_COLLECTION (collection), 0);
+  g_return_val_if_fail (collection->priv->streams, 0);
+
+  return g_queue_get_length (collection->priv->streams);
+}
+
+/**
+ * gst_stream_collection_get_stream:
+ * @collection: a #GstStreamCollection
+ * @index: Index of the stream to retrieve
+ *
+ * Retrieve the #GstStream with index @index from the collection.
+ *
+ * The caller should not modify the returned #GstStream
+ *
+ * Returns: (transfer none): A #GstStream
+ */
+GstStream *
+gst_stream_collection_get_stream (GstStreamCollection * collection, guint index)
+{
+  g_return_val_if_fail (GST_IS_STREAM_COLLECTION (collection), NULL);
+  g_return_val_if_fail (collection->priv->streams, NULL);
+
+  return g_queue_peek_nth (collection->priv->streams, index);
+}

gst/gststreamcollection.h

@@ -0,0 +1,106 @@
+/* GStreamer
+ * Copyright (C) 2015 Centricular Ltd
+ *  @author: Edward Hervey <edward@centricular.com>
+ *  @author: Jan Schmidt <jan@centricular.com>
+ *
+ * gststreams.h : Header for GstStreamCollection subsystem
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Library General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Library General Public License for more details.
+ *
+ * You should have received a copy of the GNU Library General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
+ * Boston, MA 02110-1301, USA.
+ */
+
+
+#ifndef __GST_STREAM_COLLECTION_H__
+#define __GST_STREAM_COLLECTION_H__
+
+#include <gst/gstobject.h>
+#include <gst/gststreams.h>
+
+G_BEGIN_DECLS
+
+#define GST_TYPE_STREAM_COLLECTION             (gst_stream_collection_get_type ())
+#define GST_IS_STREAM_COLLECTION(obj)          (G_TYPE_CHECK_INSTANCE_TYPE ((obj), GST_TYPE_STREAM_COLLECTION))
+#define GST_IS_STREAM_COLLECTION_CLASS(klass)  (G_TYPE_CHECK_CLASS_TYPE ((klass), GST_TYPE_STREAM_COLLECTION))
+#define GST_STREAM_COLLECTION_GET_CLASS(obj)   (G_TYPE_INSTANCE_GET_CLASS ((obj), GST_TYPE_STREAM_COLLECTION, GstStreamCollectionClass))
+#define GST_STREAM_COLLECTION(obj)             (G_TYPE_CHECK_INSTANCE_CAST ((obj), GST_TYPE_STREAM_COLLECTION, GstStreamCollection))
+#define GST_STREAM_COLLECTION_CLASS(klass)     (G_TYPE_CHECK_CLASS_CAST ((klass), GST_TYPE_STREAM_COLLECTION, GstStreamCollectionClass))
+#define GST_STREAM_COLLECTION_CAST(obj)        ((GstStreamCollection*)(obj))
+
+typedef struct _GstStreamCollection GstStreamCollection;
+typedef struct _GstStreamCollectionClass GstStreamCollectionClass;
+typedef struct _GstStreamCollectionPrivate GstStreamCollectionPrivate;
+/**
+ * GstStreamCollection:
+ *
+ * A collection of #GstStream that are available.
+ *
+ * A #GstStreamCollection will be provided by elements that can make those
+ * streams available. Applications can use the collection to show the user
+ * what streams are available by using %gst_stream_collection_get_stream()
+ *
+ * Once posted, a #GstStreamCollection is immutable. Updates are made by sending
+ * a new #GstStreamCollection message, which may or may not share some of
+ * the #GstStream objects from the collection it replaces. The receiver can check
+ * the sender of a stream collection message to know which collection is
+ * obsoleted.
+ *
+ * Several elements in a pipeline can provide #GstStreamCollection.
+ *
+ * Applications can activate streams from a collection by using the
+ * #GST_EVENT_SELECT_STREAMS event on a pipeline, bin or element.
+ *
+ */
+struct _GstStreamCollection {
+  GstObject object;
+
+  /*< private >*/
+  gchar *upstream_id;
+  GstStreamCollectionPrivate *priv;
+
+  gpointer _gst_reserved[GST_PADDING];
+};
+
+/**
+ * GstStreamCollectionClass:
+ * @parent_class: the parent class structure
+ * @stream_notify: default signal handler for the stream-notify signal
+ *
+ * GstStreamCollection class structure
+ */
+struct _GstStreamCollectionClass {
+  GstObjectClass parent_class;
+
+  /* signals */
+  void  (*stream_notify)      (GstStreamCollection *collection, GstStream *stream, GParamSpec * pspec);
+
+  /*< private >*/
+  gpointer _gst_reserved[GST_PADDING];
+};
+
+GType gst_stream_collection_get_type (void);
+
+GstStreamCollection *gst_stream_collection_new (const gchar *upstream_id);
+
+const gchar *gst_stream_collection_get_upstream_id (GstStreamCollection *collection);
+
+guint gst_stream_collection_get_size (GstStreamCollection *collection);
+GstStream *gst_stream_collection_get_stream (GstStreamCollection *collection, guint index);
+
+gboolean gst_stream_collection_add_stream (GstStreamCollection *collection,
+					   GstStream *stream);
+
+G_END_DECLS
+
+#endif /* __GST_STREAM_COLLECTION_H__ */

gst/gststreams.c

@@ -0,0 +1,518 @@
+/* GStreamer
+ *
+ * Copyright (C) 2015 Centricular Ltd
+ *  @author: Edward Hervey <edward@centricular.com>
+ *  @author: Jan Schmidt <jan@centricular.com>
+ *
+ * gststreams.c: GstStream and GstStreamCollection object and methods
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Library General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Library General Public License for more details.
+ *
+ * You should have received a copy of the GNU Library General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
+ * Boston, MA 02110-1301, USA.
+ *
+ * MT safe.
+ */
+
+/**
+ * SECTION:gststreams
+ * @short_description: Base class for stream objects
+ *
+ * A #GstStream is a high-level object defining a stream of data which is, or
+ * can be, present in a #GstPipeline.
+ *
+ * It is defined by a unique identifier, a "Stream ID". A #GstStream does not
+ * automatically imply the stream is present within a pipeline or element.
+ *
+ * Any element that can introduce new streams in a pipeline should create the
+ * appropriate #GstStream object, and can convey that object via the
+ * %GST_EVENT_STREAM_START event and/or the #GstStreamCollection.
+ *
+ * Elements that do not modify the nature of the stream can add extra information
+ * on it (such as enrich the #GstCaps, or #GstTagList). This is typically done
+ * by parsing elements.
+ */
+
+#include "gst_private.h"
+
+#include "gstenumtypes.h"
+#include "gstevent.h"
+#include "gststreams.h"
+
+GST_DEBUG_CATEGORY_STATIC (streams_debug);
+#define GST_CAT_DEFAULT streams_debug
+
+#define GST_STREAM_GET_PRIVATE(obj)  \
+   (G_TYPE_INSTANCE_GET_PRIVATE ((obj), GST_TYPE_STREAM, GstStreamPrivate))
+
+struct _GstStreamPrivate
+{
+  GstStreamFlags flags;
+  GstStreamType type;
+  GstTagList *tags;
+  GstCaps *caps;
+};
+
+/* stream signals and properties */
+enum
+{
+  LAST_SIGNAL
+};
+
+enum
+{
+  PROP_0,
+  PROP_STREAM_ID,
+  PROP_STREAM_FLAGS,
+  PROP_STREAM_TYPE,
+  PROP_TAGS,
+  PROP_CAPS,
+  PROP_LAST
+};
+
+static GParamSpec *gst_stream_pspecs[PROP_LAST] = { 0 };
+
+#if 0
+static guint gst_stream_signals[LAST_SIGNAL] = { 0 };
+#endif
+
+static void gst_stream_finalize (GObject * object);
+
+static void gst_stream_set_property (GObject * object, guint prop_id,
+    const GValue * value, GParamSpec * pspec);
+static void gst_stream_get_property (GObject * object, guint prop_id,
+    GValue * value, GParamSpec * pspec);
+
+#define _do_init				\
+{ \
+  GST_DEBUG_CATEGORY_INIT (streams_debug, "streams", GST_DEBUG_BOLD, \
+      "debugging info for the stream and stream collection objects"); \
+  \
+}
+
+#define gst_stream_parent_class parent_class
+G_DEFINE_TYPE_WITH_CODE (GstStream, gst_stream, GST_TYPE_OBJECT, _do_init);
+
+static void
+gst_stream_class_init (GstStreamClass * klass)
+{
+  GObjectClass *gobject_class;
+
+  gobject_class = (GObjectClass *) klass;
+
+  g_type_class_add_private (klass, sizeof (GstStreamPrivate));
+
+  gobject_class->set_property = gst_stream_set_property;
+  gobject_class->get_property = gst_stream_get_property;
+
+  /**
+   * GstStream:stream-id:
+   *
+   * The unique identifier of the #GstStream. Can only be set at construction
+   * time.
+   */
+  g_object_class_install_property (gobject_class, PROP_STREAM_ID,
+      g_param_spec_string ("stream-id", "Stream ID",
+          "The stream ID of the stream",
+          NULL,
+          G_PARAM_READWRITE | G_PARAM_CONSTRUCT_ONLY | G_PARAM_STATIC_STRINGS));
+
+  /**
+   * GstStream:flags:
+   *
+   * The #GstStreamFlags of the #GstStream. Can only be set at construction time.
+   **/
+  gst_stream_pspecs[PROP_STREAM_FLAGS] =
+      g_param_spec_flags ("stream-flags", "Stream Flags", "The stream flags",
+      GST_TYPE_STREAM_FLAGS, GST_STREAM_FLAG_NONE,
+      G_PARAM_READWRITE | G_PARAM_CONSTRUCT | G_PARAM_STATIC_STRINGS);
+  g_object_class_install_property (gobject_class, PROP_STREAM_FLAGS,
+      gst_stream_pspecs[PROP_STREAM_FLAGS]);
+
+  /**
+   * GstStream:stream-type:
+   *
+   * The #GstStreamType of the #GstStream. Can only be set at construction time.
+   **/
+  gst_stream_pspecs[PROP_STREAM_TYPE] =
+      g_param_spec_flags ("stream-type", "Stream Type", "The type of stream",
+      GST_TYPE_STREAM_TYPE, GST_STREAM_TYPE_UNKNOWN,
+      G_PARAM_READWRITE | G_PARAM_CONSTRUCT | G_PARAM_STATIC_STRINGS);
+  g_object_class_install_property (gobject_class, PROP_STREAM_TYPE,
+      gst_stream_pspecs[PROP_STREAM_TYPE]);
+
+  /**
+   * GstStream:caps:
+   *
+   * The #GstCaps of the #GstStream.
+   **/
+  gst_stream_pspecs[PROP_CAPS] =
+      g_param_spec_boxed ("caps", "Caps", "The caps of the stream",
+      GST_TYPE_CAPS, G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS);
+  g_object_class_install_property (gobject_class, PROP_CAPS,
+      gst_stream_pspecs[PROP_CAPS]);
+
+  /**
+   * GstStream:tags:
+   *
+   * The #GstTagList of the #GstStream.
+   **/
+  gst_stream_pspecs[PROP_TAGS] =
+      g_param_spec_boxed ("tags", "Tags", "The tags of the stream",
+      GST_TYPE_TAG_LIST, G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS);
+  g_object_class_install_property (gobject_class, PROP_TAGS,
+      gst_stream_pspecs[PROP_TAGS]);
+
+  gobject_class->finalize = gst_stream_finalize;
+}
+
+static void
+gst_stream_init (GstStream * stream)
+{
+  stream->priv = GST_STREAM_GET_PRIVATE (stream);
+  stream->priv->type = GST_STREAM_TYPE_UNKNOWN;
+}
+
+static void
+gst_stream_finalize (GObject * object)
+{
+  GstStream *stream = GST_STREAM_CAST (object);
+
+  gst_mini_object_replace ((GstMiniObject **) & stream->priv->tags,
+      (GstMiniObject *) NULL);
+  gst_caps_replace (&stream->priv->caps, NULL);
+  g_free ((gchar *) stream->stream_id);
+
+  G_OBJECT_CLASS (parent_class)->finalize (object);
+}
+
+/**
+ * gst_stream_new:
+ * @stream_id: (allow-none): the id for the new stream. If %NULL,
+ * a new one will be automatically generated
+ * @caps: (allow-none) (transfer none): the #GstCaps of the stream
+ * @type: the #GstStreamType of the stream
+ * @flags: the #GstStreamFlags of the stream
+ *
+ * Create a new #GstStream for the given @stream_id, @caps, @type
+ * and @flags
+ *
+ * Returns: The new #GstStream
+ */
+GstStream *
+gst_stream_new (const gchar * stream_id, GstCaps * caps, GstStreamType type,
+    GstStreamFlags flags)
+{
+  return g_object_new (GST_TYPE_STREAM, "stream-id", stream_id, "caps", caps,
+      "stream-type", type, "stream-flags", flags, NULL);
+}
+
+static void
+gst_stream_set_stream_id (GstStream * stream, const gchar * stream_id)
+{
+  GST_OBJECT_LOCK (stream);
+  g_assert (stream->stream_id == NULL);
+  if (stream_id)
+    stream->stream_id = g_strdup (stream_id);
+  else {
+    /* Create a randoom stream_id if NULL */
+    GST_FIXME_OBJECT (stream, "Creating random stream-id, consider "
+        "implementing a deterministic way of creating a stream-id");
+    stream->stream_id =
+        g_strdup_printf ("%08x%08x%08x%08x", g_random_int (), g_random_int (),
+        g_random_int (), g_random_int ());
+  }
+
+  GST_OBJECT_UNLOCK (stream);
+}
+
+/**
+ * gst_stream_get_stream_id:
+ * @stream: a #GstStream
+ *
+ * Returns the stream ID of @stream.
+ *
+ * Returns: (transfer none) (nullable): the stream ID of @stream. Only valid
+ * during the lifetime of @stream.
+ */
+const gchar *
+gst_stream_get_stream_id (GstStream * stream)
+{
+  return stream->stream_id;
+}
+
+/**
+ * gst_stream_set_stream_flags:
+ * @stream: a #GstStream
+ * @flags: the flags to set on @stream
+ *
+ * Set the @flags for the @stream.
+ */
+void
+gst_stream_set_stream_flags (GstStream * stream, GstStreamFlags flags)
+{
+  GST_OBJECT_LOCK (stream);
+  stream->priv->flags = flags;
+  GST_OBJECT_UNLOCK (stream);
+
+  g_object_notify_by_pspec (G_OBJECT (stream),
+      gst_stream_pspecs[PROP_STREAM_FLAGS]);
+}
+
+/**
+ * gst_stream_get_stream_flags:
+ * @stream: a #GstStream
+ *
+ * Retrieve the current stream flags for @stream
+ *
+ * Returns: The #GstStreamFlags for @stream
+ *
+ */
+GstStreamFlags
+gst_stream_get_stream_flags (GstStream * stream)
+{
+  GstStreamFlags res;
+
+  GST_OBJECT_LOCK (stream);
+  res = stream->priv->flags;
+  GST_OBJECT_UNLOCK (stream);
+
+  return res;
+}
+
+/**
+ * gst_stream_set_stream_type:
+ * @stream: a #GstStream
+ * @stream_type: the type to set on @stream
+ *
+ * Set the stream type of @stream
+ */
+void
+gst_stream_set_stream_type (GstStream * stream, GstStreamType stream_type)
+{
+  GST_OBJECT_LOCK (stream);
+  stream->priv->type = stream_type;
+  GST_OBJECT_UNLOCK (stream);
+
+  g_object_notify_by_pspec (G_OBJECT (stream),
+      gst_stream_pspecs[PROP_STREAM_TYPE]);
+}
+
+/**
+ * gst_stream_get_stream_type:
+ * @stream: a #GstStream
+ *
+ * Retrieve the stream type for @stream
+ *
+ * Returns: The #GstStreamType for @stream
+ *
+ */
+GstStreamType
+gst_stream_get_stream_type (GstStream * stream)
+{
+  GstStreamType res;
+
+  GST_OBJECT_LOCK (stream);
+  res = stream->priv->type;
+  GST_OBJECT_UNLOCK (stream);
+
+  return res;
+}
+
+/**
+ * gst_stream_set_tags:
+ * @stream: a #GstStream
+ * @tags: (transfer none) (allow-none): a #GstTagList
+ *
+ * Set the tags for the #GstStream
+ *
+ */
+void
+gst_stream_set_tags (GstStream * stream, GstTagList * tags)
+{
+  GST_OBJECT_LOCK (stream);
+  gst_mini_object_replace ((GstMiniObject **) & stream->priv->tags,
+      (GstMiniObject *) tags);
+  GST_OBJECT_UNLOCK (stream);
+  g_object_notify_by_pspec (G_OBJECT (stream), gst_stream_pspecs[PROP_TAGS]);
+}
+
+/**
+ * gst_stream_get_tags:
+ * @stream: a #GstStream
+ *
+ * Retrieve the tags for @stream, if any
+ *
+ * Returns: (transfer full) (nullable): The #GstTagList for @stream
+ *
+ */
+GstTagList *
+gst_stream_get_tags (GstStream * stream)
+{
+  GstTagList *res = NULL;
+
+  GST_OBJECT_LOCK (stream);
+  if (stream->priv->tags)
+    res = gst_tag_list_ref (stream->priv->tags);
+  GST_OBJECT_UNLOCK (stream);
+
+  return res;
+}
+
+/**
+ * gst_stream_set_caps:
+ * @stream: a #GstStream
+ * @caps: (transfer none) (allow-none): a #GstCaps
+ *
+ * Set the caps for the #GstStream
+ *
+ */
+void
+gst_stream_set_caps (GstStream * stream, GstCaps * caps)
+{
+  gboolean notify = FALSE;
+
+  GST_OBJECT_LOCK (stream);
+  if (stream->priv->caps == NULL || (caps
+          && !gst_caps_is_equal (stream->priv->caps, caps))) {
+    gst_caps_replace (&stream->priv->caps, caps);
+    notify = TRUE;
+  }
+  GST_OBJECT_UNLOCK (stream);
+
+  if (notify)
+    g_object_notify_by_pspec (G_OBJECT (stream), gst_stream_pspecs[PROP_CAPS]);
+}
+
+
+/**
+ * gst_stream_get_caps:
+ * @stream: a #GstStream
+ *
+ * Retrieve the caps for @stream, if any
+ *
+ * Returns: (transfer full) (nullable): The #GstCaps for @stream
+ *
+ */
+GstCaps *
+gst_stream_get_caps (GstStream * stream)
+{
+  GstCaps *res = NULL;
+
+  GST_OBJECT_LOCK (stream);
+  if (stream->priv->caps)
+    res = gst_caps_ref (stream->priv->caps);
+  GST_OBJECT_UNLOCK (stream);
+
+  return res;
+}
+
+static void
+gst_stream_set_property (GObject * object, guint prop_id,
+    const GValue * value, GParamSpec * pspec)
+{
+  GstStream *stream;
+
+  stream = GST_STREAM_CAST (object);
+
+  switch (prop_id) {
+    case PROP_STREAM_ID:
+      gst_stream_set_stream_id (stream, g_value_get_string (value));
+      break;
+    case PROP_STREAM_FLAGS:
+      GST_OBJECT_LOCK (stream);
+      stream->priv->flags = g_value_get_flags (value);
+      GST_OBJECT_UNLOCK (stream);
+      break;
+    case PROP_STREAM_TYPE:
+      GST_OBJECT_LOCK (stream);
+      stream->priv->type = g_value_get_flags (value);
+      GST_OBJECT_UNLOCK (stream);
+      break;
+    case PROP_TAGS:
+      GST_OBJECT_LOCK (stream);
+      gst_mini_object_replace ((GstMiniObject **) & stream->priv->tags,
+          (GstMiniObject *) g_value_get_boxed (value));
+      GST_OBJECT_UNLOCK (stream);
+      break;
+    case PROP_CAPS:
+      GST_OBJECT_LOCK (stream);
+      gst_mini_object_replace ((GstMiniObject **) & stream->priv->caps,
+          (GstMiniObject *) g_value_get_boxed (value));
+      GST_OBJECT_UNLOCK (stream);
+      break;
+    default:
+      G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
+      break;
+  }
+}
+
+static void
+gst_stream_get_property (GObject * object, guint prop_id,
+    GValue * value, GParamSpec * pspec)
+{
+  GstStream *stream;
+
+  stream = GST_STREAM_CAST (object);
+
+  switch (prop_id) {
+    case PROP_STREAM_ID:
+      g_value_set_string (value, gst_stream_get_stream_id (stream));
+      break;
+    case PROP_STREAM_FLAGS:
+      g_value_set_flags (value, gst_stream_get_stream_flags (stream));
+      break;
+    case PROP_STREAM_TYPE:
+      g_value_set_flags (value, gst_stream_get_stream_type (stream));
+      break;
+    case PROP_TAGS:
+      g_value_take_boxed (value, gst_stream_get_tags (stream));
+      break;
+    case PROP_CAPS:
+      g_value_take_boxed (value, gst_stream_get_caps (stream));
+      break;
+    default:
+      G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
+      break;
+  }
+}
+
+/**
+ * gst_stream_type_get_name:
+ * @stype: a #GstStreamType
+ *
+ * Get a descriptive string for a given #GstStreamType
+ *
+ * Returns: A string describing the stream type
+ */
+const gchar *
+gst_stream_type_get_name (GstStreamType stype)
+{
+  /* FIXME : Make this more flexible */
+  switch (stype) {
+    case GST_STREAM_TYPE_UNKNOWN:
+      return "unknown";
+    case GST_STREAM_TYPE_AUDIO:
+      return "audio";
+    case GST_STREAM_TYPE_VIDEO:
+      return "video";
+    case GST_STREAM_TYPE_CONTAINER:
+      return "container";
+    case GST_STREAM_TYPE_TEXT:
+      return "text";
+    default:
+      return NULL;
+  }
+
+  return NULL;
+}

gst/gststreams.h

@@ -0,0 +1,131 @@
+/* GStreamer
+ * Copyright (C) 2015 Centricular Ltd
+ *  @author: Edward Hervey <edward@centricular.com>
+ *  @author: Jan Schmidt <jan@centricular.com>
+ *
+ * gststreams.h : Header for GstStream subsystem
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Library General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Library General Public License for more details.
+ *
+ * You should have received a copy of the GNU Library General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
+ * Boston, MA 02110-1301, USA.
+ */
+
+
+#ifndef __GST_STREAMS_H__
+#define __GST_STREAMS_H__
+
+#include <gst/gstobject.h>
+#include <gst/gstevent.h>
+
+G_BEGIN_DECLS
+
+#define GST_TYPE_STREAM             (gst_stream_get_type ())
+#define GST_IS_STREAM(obj)          (G_TYPE_CHECK_INSTANCE_TYPE ((obj), GST_TYPE_STREAM))
+#define GST_IS_STREAM_CLASS(klass)  (G_TYPE_CHECK_CLASS_TYPE ((klass), GST_TYPE_STREAM))
+#define GST_STREAM_GET_CLASS(obj)   (G_TYPE_INSTANCE_GET_CLASS ((obj), GST_TYPE_STREAM, GstStreamClass))
+#define GST_STREAM(obj)             (G_TYPE_CHECK_INSTANCE_CAST ((obj), GST_TYPE_STREAM, GstStream))
+#define GST_STREAM_CLASS(klass)     (G_TYPE_CHECK_CLASS_CAST ((klass), GST_TYPE_STREAM, GstStreamClass))
+#define GST_STREAM_CAST(obj)        ((GstStream*)(obj))
+
+/**
+ * GstStreamType:
+ * @GST_STREAM_TYPE_UNKNOWN: The stream is of unknown (unclassified) type.
+ * @GST_STREAM_TYPE_AUDIO: The stream is of audio data
+ * @GST_STREAM_TYPE_VIDEO: The stream carries video data
+ * @GST_STREAM_TYPE_CONTAINER: The stream is a muxed container type
+ * @GST_STREAM_TYPE_TEXT: The stream contains subtitle / subpicture data.
+ *
+ * #GstStreamType describes a high level classification set for
+ * flows of data in #GstStream objects.
+ */
+typedef enum {
+  GST_STREAM_TYPE_UNKNOWN   = 1 << 0,
+  GST_STREAM_TYPE_AUDIO     = 1 << 1,
+  GST_STREAM_TYPE_VIDEO     = 1 << 2,
+  GST_STREAM_TYPE_CONTAINER = 1 << 3,
+  GST_STREAM_TYPE_TEXT      = 1 << 4
+} GstStreamType;
+
+
+typedef struct _GstStream GstStream;
+typedef struct _GstStreamClass GstStreamClass;
+typedef struct _GstStreamPrivate GstStreamPrivate;
+
+/**
+ * GstStream:
+ * @stream_id: The Stream Identifier for this #GstStream
+ * 
+ * A high-level object representing a single stream. It might be backed, or
+ * not, by an actual flow of data in a pipeline (#GstPad).
+ *
+ * A #GstStream does not care about data changes (such as decoding, encoding,
+ * parsing,...) as long as the underlying data flow corresponds to the same
+ * high-level flow (ex: a certain audio track).
+ *
+ * A #GstStream contains all the information pertinent to a stream, such as
+ * stream-id, tags, caps, type, ...
+ *
+ * Elements can subclass a #GstStream for internal usage (to contain information
+ * pertinent to streams of data).
+ */
+struct _GstStream {
+  GstObject object;
+
+  /*< public >*/
+  const gchar *stream_id;
+
+  /*< private >*/
+  GstStreamPrivate *priv;
+
+  gpointer _gst_reserved[GST_PADDING];
+};
+
+/**
+ * GstStreamClass:
+ * @parent_class: the parent class structure
+ *
+ * GstStream class structure
+ */
+struct _GstStreamClass {
+  GstObjectClass parent_class;
+  
+  /*< private >*/
+  gpointer _gst_reserved[GST_PADDING];
+};
+
+GType     gst_stream_get_type (void);
+
+GstStream *gst_stream_new            (const gchar *stream_id,
+				      GstCaps *caps,
+				      GstStreamType type,
+				      GstStreamFlags flags);
+
+const gchar *gst_stream_get_stream_id (GstStream *stream);
+
+void           gst_stream_set_stream_flags (GstStream *stream, GstStreamFlags flags);
+GstStreamFlags gst_stream_get_stream_flags (GstStream *stream);
+
+void          gst_stream_set_stream_type (GstStream *stream, GstStreamType stream_type);
+GstStreamType gst_stream_get_stream_type (GstStream *stream);
+
+void        gst_stream_set_tags (GstStream *stream, GstTagList *tags);
+GstTagList *gst_stream_get_tags (GstStream *stream);
+
+void     gst_stream_set_caps (GstStream *stream, GstCaps *caps);
+GstCaps *gst_stream_get_caps (GstStream *stream);
+
+const gchar *gst_stream_type_get_name (GstStreamType stype);
+G_END_DECLS
+
+#endif /* __GST_STREAMS_H__ */

gst/gstutils.c

@@ -3993,6 +3993,41 @@ gst_pad_get_stream_id (GstPad * pad)
   return ret;
 }
 
+/**
+ * gst_pad_get_stream:
+ * @pad: A source #GstPad
+ *
+ * Returns the current #GstStream for the @pad, or %NULL if none has been
+ * set yet, i.e. the pad has not received a stream-start event yet.
+ *
+ * This is a convenience wrapper around gst_pad_get_sticky_event() and
+ * gst_event_parse_stream().
+ *
+ * Returns: (nullable) (transfer full): the current #GstStream for @pad, or %NULL.
+ *     unref the returned stream when no longer needed.
+ *
+ * Since: 1.X
+ */
+GstStream *
+gst_pad_get_stream (GstPad * pad)
+{
+  GstStream *stream = NULL;
+  GstEvent *event;
+
+  g_return_val_if_fail (GST_IS_PAD (pad), NULL);
+
+  event = gst_pad_get_sticky_event (pad, GST_EVENT_STREAM_START, 0);
+  if (event != NULL) {
+    gst_event_parse_stream (event, &stream);
+    gst_event_unref (event);
+    GST_LOG_OBJECT (pad, "pad has stream object %p", stream);
+  } else {
+    GST_DEBUG_OBJECT (pad, "pad has not received a stream-start event yet");
+  }
+
+  return stream;
+}
+
 /**
  * gst_util_group_id_next:
  *

gst/gstutils.h

@@ -942,6 +942,7 @@ gchar *                 gst_pad_create_stream_id_printf        (GstPad * pad, Gs
 gchar *                 gst_pad_create_stream_id_printf_valist (GstPad * pad, GstElement * parent, const gchar *stream_id, va_list var_args) G_GNUC_PRINTF (3, 0) G_GNUC_MALLOC;
 
 gchar *                 gst_pad_get_stream_id           (GstPad * pad);
+GstStream *             gst_pad_get_stream              (GstPad * pad);
 
 /* bin functions */
 void                    gst_bin_add_many                (GstBin *bin, GstElement *element_1, ...) G_GNUC_NULL_TERMINATED;

tests/check/Makefile.am

@@ -137,6 +137,7 @@ check_PROGRAMS =				\
 	gst/gstsegment				\
 	gst/gstsystemclock			\
 	gst/gstclock				\
+	gst/gststream				\
 	gst/gststructure			\
 	gst/gsttag				\
 	gst/gsttracerrecord		 		\

tests/check/gst/.gitignore

@@ -38,6 +38,7 @@ gstprintf
 gstprotection
 gstregistry
 gstsegment
+gststream
 gststructure
 gstsystemclock
 gsttag

tests/check/gst/gstevent.c

@@ -52,6 +52,31 @@ GST_START_TEST (create_events)
     fail_unless (reset_time == TRUE);
     gst_event_unref (event);
   }
+  /* SELECT_STREAMS */
+  {
+    GList *streams = NULL;
+    GList *res = NULL;
+    GList *tmp;
+    streams = g_list_append (streams, (gpointer) "stream1");
+    streams = g_list_append (streams, (gpointer) "stream2");
+    event = gst_event_new_select_streams (streams);
+    fail_if (event == NULL);
+    fail_unless (GST_EVENT_TYPE (event) == GST_EVENT_SELECT_STREAMS);
+    fail_unless (GST_EVENT_IS_UPSTREAM (event));
+
+    gst_event_parse_select_streams (event, &res);
+    fail_if (res == NULL);
+    fail_unless_equals_int (g_list_length (res), 2);
+    tmp = res;
+    fail_unless_equals_string (tmp->data, "stream1");
+    tmp = tmp->next;
+    fail_unless_equals_string (tmp->data, "stream2");
+
+    gst_event_unref (event);
+
+    g_list_free (streams);
+    g_list_free_full (res, g_free);
+  }
   /* EOS */
   {
     event = gst_event_new_eos ();
@@ -219,6 +244,37 @@ GST_START_TEST (create_events)
     gst_event_unref (event);
   }
 
+  /* STREAM_COLLECTION */
+  {
+    GstStreamCollection *collection, *res = NULL;
+    GstStream *stream1, *stream2;
+    GstCaps *caps1, *caps2;
+
+    /* Create a collection of two streams */
+    caps1 = gst_caps_from_string ("some/caps");
+    caps2 = gst_caps_from_string ("some/other-string");
+
+    stream1 = gst_stream_new ("stream-1", caps1, GST_STREAM_TYPE_AUDIO, 0);
+    stream2 = gst_stream_new ("stream-2", caps2, GST_STREAM_TYPE_VIDEO, 0);
+
+    collection = gst_stream_collection_new ("something");
+    fail_unless (gst_stream_collection_add_stream (collection, stream1));
+    fail_unless (gst_stream_collection_add_stream (collection, stream2));
+
+    event = gst_event_new_stream_collection (collection);
+    fail_unless (event != NULL);
+
+    gst_event_parse_stream_collection (event, &res);
+    fail_unless (res != NULL);
+    fail_unless (res == collection);
+
+    gst_event_unref (event);
+    gst_object_unref (res);
+    gst_object_unref (collection);
+    gst_caps_unref (caps1);
+    gst_caps_unref (caps2);
+  }
+
   /* NAVIGATION */
   {
     structure = gst_structure_new ("application/x-gst-navigation", "event",

tests/check/gst/gstmessage.c

@@ -377,6 +377,90 @@ GST_START_TEST (test_parsing)
 
     gst_message_unref (message);
   }
+  /* GST_MESSAGE_STREAM_COLLECTION */
+  {
+    GstMessage *message;
+    GstStreamCollection *collection, *res = NULL;
+    GstStream *stream1, *stream2;
+    GstCaps *caps1, *caps2;
+
+    /* Create a collection of two streams */
+    caps1 = gst_caps_from_string ("some/caps");
+    caps2 = gst_caps_from_string ("some/other-string");
+
+    stream1 = gst_stream_new ("stream-1", caps1, GST_STREAM_TYPE_AUDIO, 0);
+    stream2 = gst_stream_new ("stream-2", caps2, GST_STREAM_TYPE_VIDEO, 0);
+
+    collection = gst_stream_collection_new ("something");
+    fail_unless (gst_stream_collection_add_stream (collection, stream1));
+    fail_unless (gst_stream_collection_add_stream (collection, stream2));
+
+    message = gst_message_new_stream_collection (NULL, collection);
+    fail_unless (message != NULL);
+
+    gst_message_parse_stream_collection (message, &res);
+    fail_unless (res != NULL);
+
+    gst_message_unref (message);
+    gst_object_unref (res);
+    gst_object_unref (collection);
+    gst_caps_unref (caps1);
+    gst_caps_unref (caps2);
+  }
+  /* GST_MESSAGE_STREAMS_SELECTED */
+  {
+    GstMessage *message;
+    GstStreamCollection *collection, *res = NULL;
+    GstStream *stream1, *stream2, *stream3;
+    GstCaps *caps1, *caps2;
+
+    /* Create a collection of two streams */
+    caps1 = gst_caps_from_string ("some/caps");
+    caps2 = gst_caps_from_string ("some/other-string");
+
+    stream1 = gst_stream_new ("stream-1", caps1, GST_STREAM_TYPE_AUDIO, 0);
+    stream2 = gst_stream_new ("stream-2", caps2, GST_STREAM_TYPE_VIDEO, 0);
+
+    collection = gst_stream_collection_new ("something");
+    fail_unless (gst_stream_collection_add_stream (collection, stream1));
+    fail_unless (gst_stream_collection_add_stream (collection, stream2));
+
+    message = gst_message_new_streams_selected (NULL, collection);
+    fail_unless (message != NULL);
+
+    gst_message_parse_streams_selected (message, &res);
+    fail_unless (res != NULL);
+
+    fail_unless (gst_message_streams_selected_get_size (message) == 0);
+    gst_object_unref (res);
+    gst_message_unref (message);
+
+    /* Once again, this time with a stream in it */
+    message = gst_message_new_streams_selected (NULL, collection);
+    fail_unless (message != NULL);
+
+    gst_message_streams_selected_add (message, stream1);
+
+    gst_message_parse_streams_selected (message, &res);
+    fail_unless (res != NULL);
+
+    /* There is only one stream ! */
+    fail_unless (gst_message_streams_selected_get_size (message) == 1);
+
+    stream3 = gst_message_streams_selected_get_stream (message, 0);
+    fail_unless (stream3 != NULL);
+    gst_object_unref (stream3);
+
+    /* Shoul fail */
+    ASSERT_CRITICAL (gst_message_streams_selected_get_stream (message, 1));
+
+    gst_object_unref (res);
+    gst_message_unref (message);
+
+    gst_object_unref (collection);
+    gst_caps_unref (caps1);
+    gst_caps_unref (caps2);
+  }
 }
 
 GST_END_TEST;

tests/check/gst/gststream.c

@@ -0,0 +1,225 @@
+/* GStreamer
+ * Copyright (C) <2015> Edward Hervey <edward@centricular.com>
+ *
+ * gststructure.c: Unit tests for GstStream and GstStreamCollection
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Library General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Library General Public License for more details.
+ *
+ * You should have received a copy of the GNU Library General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
+ * Boston, MA 02110-1301, USA.
+ */
+
+
+#include <gst/gst.h>
+#include <gst/check/gstcheck.h>
+
+GST_START_TEST (test_stream_creation)
+{
+  GstStream *stream;
+  GstCaps *caps;
+  GstCaps *caps2;
+  GstTagList *tags, *tags2;
+
+  caps = gst_caps_from_string ("some/caps");
+  stream = gst_stream_new ("stream-id", caps, GST_STREAM_TYPE_AUDIO, 0);
+  fail_unless (stream != NULL);
+
+  fail_unless_equals_string (gst_stream_get_stream_id (stream), "stream-id");
+  caps2 = gst_stream_get_caps (stream);
+  fail_unless (gst_caps_is_equal (caps, caps2));
+  gst_caps_unref (caps2);
+
+  fail_unless (gst_stream_get_stream_type (stream) == GST_STREAM_TYPE_AUDIO);
+
+  gst_caps_unref (caps);
+
+  tags = gst_tag_list_new (GST_TAG_ALBUM, "test-album", NULL);
+  g_object_set (stream, "tags", tags, NULL);
+  tags2 = gst_stream_get_tags (stream);
+  fail_unless (gst_tag_list_is_equal (tags, tags2));
+  gst_tag_list_unref (tags);
+  gst_tag_list_unref (tags2);
+
+  gst_object_unref (stream);
+}
+
+GST_END_TEST;
+
+GST_START_TEST (test_stream_event)
+{
+  GstEvent *event;
+  GstStream *stream, *stream2 = NULL;
+  GstCaps *caps;
+  GstCaps *caps2;
+
+  event = gst_event_new_stream_start ("here/we/go");
+  /* By default a stream-start event has no stream */
+  gst_event_parse_stream (event, &stream2);
+  fail_if (stream2 != NULL);
+
+  /* Create and set stream on event */
+  caps = gst_caps_from_string ("some/caps");
+  stream = gst_stream_new ("here/we/go", caps, GST_STREAM_TYPE_AUDIO, 0);
+  fail_unless (stream != NULL);
+  gst_event_set_stream (event, stream);
+
+  /* Parse and check it's the same */
+  gst_event_parse_stream (event, &stream2);
+  fail_unless (stream2 != NULL);
+  fail_unless_equals_string (gst_stream_get_stream_id (stream2), "here/we/go");
+  caps2 = gst_stream_get_caps (stream);
+  fail_unless (gst_caps_is_equal (caps, caps2));
+  fail_unless (gst_stream_get_stream_type (stream) == GST_STREAM_TYPE_AUDIO);
+  gst_caps_unref (caps2);
+
+  gst_event_unref (event);
+  gst_caps_unref (caps);
+  gst_object_unref (stream);
+  gst_object_unref (stream2);
+}
+
+GST_END_TEST;
+
+struct NotifyStats
+{
+  guint collection_notify;
+  guint collection_notify_caps;
+  guint collection_notify_tags;
+  guint collection_notify_type;
+  guint collection_notify_flags;
+
+  guint stream_notify;
+  guint stream_notify_caps;
+  guint stream_notify_tags;
+  guint stream_notify_type;
+  guint stream_notify_flags;
+
+  guint stream2_notify;
+  guint stream2_notify_caps;
+  guint stream2_notify_tags;
+  guint stream2_notify_type;
+  guint stream2_notify_flags;
+};
+
+static void
+stream_notify_cb (GstStreamCollection * collection, GstStream * stream,
+    GParamSpec * pspec, guint * val)
+{
+  GST_LOG ("Got stream-notify from %" GST_PTR_FORMAT " for %s from %"
+      GST_PTR_FORMAT, stream, pspec->name, collection);
+  (*val)++;
+}
+
+static void
+notify_cb (GstStream * stream, GParamSpec * pspec, guint * val)
+{
+  GST_LOG ("Got notify from %" GST_PTR_FORMAT " for %s", stream, pspec->name);
+  (*val)++;
+}
+
+GST_START_TEST (test_notifies)
+{
+  GstStreamCollection *collection;
+  GstStream *stream, *stream2 = NULL;
+  GstCaps *caps;
+  struct NotifyStats stats = { 0, };
+  GstTagList *tags;
+
+  collection = gst_stream_collection_new ("check-collection");
+  g_signal_connect (collection, "stream-notify", (GCallback) stream_notify_cb,
+      &stats.collection_notify);
+  g_signal_connect (collection, "stream-notify::stream-type",
+      (GCallback) stream_notify_cb, &stats.collection_notify_type);
+  g_signal_connect (collection, "stream-notify::stream-flags",
+      (GCallback) stream_notify_cb, &stats.collection_notify_flags);
+  g_signal_connect (collection, "stream-notify::caps",
+      (GCallback) stream_notify_cb, &stats.collection_notify_caps);
+  g_signal_connect (collection, "stream-notify::tags",
+      (GCallback) stream_notify_cb, &stats.collection_notify_tags);
+
+  caps = gst_caps_from_string ("some/audio-caps");
+  stream = gst_stream_new ("here/we/go", caps, GST_STREAM_TYPE_AUDIO, 0);
+  gst_caps_unref (caps);
+  g_signal_connect (stream, "notify", (GCallback) notify_cb,
+      &stats.stream_notify);
+  g_signal_connect (stream, "notify::stream-type", (GCallback) notify_cb,
+      &stats.stream_notify_type);
+  g_signal_connect (stream, "notify::stream-flags", (GCallback) notify_cb,
+      &stats.stream_notify_flags);
+  g_signal_connect (stream, "notify::caps", (GCallback) notify_cb,
+      &stats.stream_notify_caps);
+  g_signal_connect (stream, "notify::tags", (GCallback) notify_cb,
+      &stats.stream_notify_tags);
+  gst_stream_collection_add_stream (collection, stream);
+
+  caps = gst_caps_from_string ("some/video-caps");
+  stream2 = gst_stream_new ("here/we/go/again", caps, GST_STREAM_TYPE_VIDEO, 0);
+  gst_caps_unref (caps);
+  g_signal_connect (stream2, "notify", (GCallback) notify_cb,
+      &stats.stream2_notify);
+  g_signal_connect (stream2, "notify::stream-type", (GCallback) notify_cb,
+      &stats.stream2_notify_type);
+  g_signal_connect (stream2, "notify::stream-flags", (GCallback) notify_cb,
+      &stats.stream2_notify_flags);
+  g_signal_connect (stream2, "notify::caps", (GCallback) notify_cb,
+      &stats.stream2_notify_caps);
+  g_signal_connect (stream2, "notify::tags", (GCallback) notify_cb,
+      &stats.stream2_notify_tags);
+  gst_stream_collection_add_stream (collection, stream2);
+
+  caps = gst_caps_from_string ("some/new-video-caps");
+  gst_stream_set_caps (stream2, caps);
+  gst_caps_unref (caps);
+
+  fail_unless (stats.collection_notify == 1);
+  fail_unless (stats.collection_notify_caps == 1);
+  fail_unless (stats.stream_notify == 0);
+  fail_unless (stats.stream_notify_caps == 0);
+  fail_unless (stats.stream_notify_tags == 0);
+  fail_unless (stats.stream2_notify == 1);
+  fail_unless (stats.stream2_notify_caps == 1);
+  fail_unless (stats.stream2_notify_tags == 0);
+
+  tags = gst_tag_list_new (GST_TAG_ALBUM, "test-album", NULL);
+  gst_stream_set_tags (stream, tags);
+  gst_tag_list_unref (tags);
+
+  fail_unless (stats.collection_notify == 2);
+  fail_unless (stats.collection_notify_caps == 1);
+  fail_unless (stats.collection_notify_tags == 1);
+  fail_unless (stats.stream_notify == 1);
+  fail_unless (stats.stream_notify_caps == 0);
+  fail_unless (stats.stream_notify_tags == 1);
+  fail_unless (stats.stream2_notify == 1);
+  fail_unless (stats.stream2_notify_caps == 1);
+  fail_unless (stats.stream2_notify_tags == 0);
+
+  gst_object_unref (collection);
+}
+
+GST_END_TEST;
+
+static Suite *
+gst_streams_suite (void)
+{
+  Suite *s = suite_create ("GstStream");
+  TCase *tc_chain = tcase_create ("general");
+
+  suite_add_tcase (s, tc_chain);
+  tcase_add_test (tc_chain, test_stream_creation);
+  tcase_add_test (tc_chain, test_stream_event);
+  tcase_add_test (tc_chain, test_notifies);
+  return s;
+}
+
+GST_CHECK_MAIN (gst_streams);

tests/check/gst/gststream.h

@@ -0,0 +1,53 @@
+/* GStreamer
+ * Copyright (C) <2015> Edward Hervey <edward@centricular.com>
+ *
+ * gststructure.c: Unit tests for GstStream and GstStreamCollection
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Library General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Library General Public License for more details.
+ *
+ * You should have received a copy of the GNU Library General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
+ * Boston, MA 02110-1301, USA.
+ */
+
+
+#include <gst/gst.h>
+#include <gst/check/gstcheck.h>
+
+GST_START_TEST (test_stream_creation)
+{
+  GstStream *stream;
+  GstCaps *caps;
+
+  caps = gst_caps_from_string("some/caps");
+  stream = gst_stream_new ("upstream-id", caps, GST_STREAM_TYPE_AUDIO, 0);
+  fail_unless (stream != NULL);
+
+  fail_unless_equals_string (gst_stream_get_stream_id (stream), "upstream-id");
+
+  gst_object_unref (stream);
+}
+
+GST_END_TEST;
+
+static Suite *
+gst_streams_suite (void)
+{
+  Suite *s = suite_create ("GstStream");
+  TCase *tc_chain = tcase_create ("general");
+
+  suite_add_tcase (s, tc_chain);
+  tcase_add_test (tc_chain, test_stream_creation);
+  return s;
+}
+
+GST_CHECK_MAIN (gst_streams);

win32/common/libgstreamer.def

@@ -588,8 +588,10 @@ EXPORTS
 	gst_event_new_seek
 	gst_event_new_segment
 	gst_event_new_segment_done
+	gst_event_new_select_streams
 	gst_event_new_sink_message
 	gst_event_new_step
+	gst_event_new_stream_collection
 	gst_event_new_stream_start
 	gst_event_new_tag
 	gst_event_new_toc
@@ -605,8 +607,11 @@ EXPORTS
 	gst_event_parse_seek
 	gst_event_parse_segment
 	gst_event_parse_segment_done
+	gst_event_parse_select_streams
 	gst_event_parse_sink_message
 	gst_event_parse_step
+	gst_event_parse_stream
+	gst_event_parse_stream_collection
 	gst_event_parse_stream_flags
 	gst_event_parse_stream_start
 	gst_event_parse_tag
@@ -615,6 +620,7 @@ EXPORTS
 	gst_event_set_group_id
 	gst_event_set_running_time_offset
 	gst_event_set_seqnum
+	gst_event_set_stream
 	gst_event_set_stream_flags
 	gst_event_type_flags_get_type
 	gst_event_type_get_flags
@@ -726,8 +732,10 @@ EXPORTS
 	gst_message_new_state_dirty
 	gst_message_new_step_done
 	gst_message_new_step_start
+	gst_message_new_stream_collection
 	gst_message_new_stream_start
 	gst_message_new_stream_status
+	gst_message_new_streams_selected
 	gst_message_new_structure_change
 	gst_message_new_tag
 	gst_message_new_toc
@@ -757,7 +765,9 @@ EXPORTS
 	gst_message_parse_state_changed
 	gst_message_parse_step_done
 	gst_message_parse_step_start
+	gst_message_parse_stream_collection
 	gst_message_parse_stream_status
+	gst_message_parse_streams_selected
 	gst_message_parse_structure_change
 	gst_message_parse_tag
 	gst_message_parse_toc
@@ -768,6 +778,9 @@ EXPORTS
 	gst_message_set_qos_values
 	gst_message_set_seqnum
 	gst_message_set_stream_status_object
+	gst_message_streams_selected_add
+	gst_message_streams_selected_get_size
+	gst_message_streams_selected_get_stream
 	gst_message_type_get_name
 	gst_message_type_get_type
 	gst_message_type_to_quark
@@ -850,6 +863,7 @@ EXPORTS
 	gst_pad_get_peer
 	gst_pad_get_range
 	gst_pad_get_sticky_event
+	gst_pad_get_stream
 	gst_pad_get_stream_id
 	gst_pad_get_type
 	gst_pad_has_current_caps
@@ -1192,10 +1206,29 @@ EXPORTS
 	gst_static_pad_template_get
 	gst_static_pad_template_get_caps
 	gst_static_pad_template_get_type
+	gst_stream_collection_add_stream
+	gst_stream_collection_get_size
+	gst_stream_collection_get_stream
+	gst_stream_collection_get_type
+	gst_stream_collection_get_upstream_id
+	gst_stream_collection_new
 	gst_stream_error_get_type
 	gst_stream_error_quark
 	gst_stream_flags_get_type
+	gst_stream_get_caps
+	gst_stream_get_stream_flags
+	gst_stream_get_stream_id
+	gst_stream_get_stream_type
+	gst_stream_get_tags
+	gst_stream_get_type
+	gst_stream_new
+	gst_stream_set_caps
+	gst_stream_set_stream_flags
+	gst_stream_set_stream_type
+	gst_stream_set_tags
 	gst_stream_status_type_get_type
+	gst_stream_type_get_name
+	gst_stream_type_get_type
 	gst_structure_can_intersect
 	gst_structure_change_type_get_type
 	gst_structure_copy