mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2025-01-10 17:35:59 +00:00
+ updates to the mimetypes file, trying to get it in a form that could be changed to docbook eventually
Original commit message from CVS: + updates to the mimetypes file, trying to get it in a form that could be changed to docbook eventually
This commit is contained in:
Leif Johnson
parent
4481af601f
commit
5d3c78427c
1 changed files with 476 additions and 316 deletions
|
|
@ -1,301 +1,334 @@
|
||||||
Mimetypes in GStreamer
|
MIME types in GStreamer
|
||||||
======================
|
|
||||||
|
|
||||||
1) What is a mimetype
|
What is a MIME type ?
|
||||||
---------------------
|
=====================
|
||||||
A mimetype is a combination of two (short) strings (words), the content
|
|
||||||
type and the content subtype, that make up a pair that describes a file
|
A MIME type is a combination of two (short) strings (words)---the content type
|
||||||
content type. In multimedia, mime types are used to describe the media
|
and the content subtype. Content types are broad categories used for describing
|
||||||
streamtype . In GStreamer, obsiously, we use mimetypes in the same way.
|
almost all types of files: video, audio, text, and application are common
|
||||||
They are part of a GstCaps, that describes a media stream. Besides a
|
content types. The subtype further breaks the content type down into a more
|
||||||
mimetype, a GstCaps also contains stream properties (GstProps), which
|
specific type description, for example 'application/ogg', 'audio/raw',
|
||||||
are combinations of key/value pairs, and a name.
|
'video/mpeg', or 'text/plain'.
|
||||||
|
|
||||||
|
So the content type and subtype make up a pair that describes the type of
|
||||||
|
information contained in a file. In multimedia processing, MIME types are used
|
||||||
|
to describe the type of information carried by a media stream. In GStreamer, we
|
||||||
|
use MIME types in the same way, to identify the types of information that are
|
||||||
|
allowed to pass between GStreamer elements. The MIME type is part of a GstCaps
|
||||||
|
object that describes a media stream. Besides a MIME type, a GstCaps object also
|
||||||
|
contains a name and some stream properties (GstProps, which hold combinations of
|
||||||
|
key/value pairs).
|
||||||
|
|
||||||
|
An example of a MIME type is 'video/mpeg'. A corresponding GstCaps could be
|
||||||
|
created using code:
|
||||||
|
|
||||||
An example of a mimetype is 'video/mpeg'. A corresponding GstCaps could
|
|
||||||
be created using:
|
|
||||||
GstCaps *caps = gst_caps_new("video_mpeg_type",
|
GstCaps *caps = gst_caps_new("video_mpeg_type",
|
||||||
"video/mpeg",
|
"video/mpeg",
|
||||||
gst_props_new("width", GST_PROPS_INT(384),
|
gst_props_new("width", GST_PROPS_INT(384),
|
||||||
"height", GST_PROPS_INT(288),
|
"height", GST_PROPS_INT(288),
|
||||||
NULL));
|
NULL));
|
||||||
or using a macro:
|
or by using a macro:
|
||||||
|
|
||||||
GstCaps *caps = GST_CAPS_NEW("video_mpeg_type",
|
GstCaps *caps = GST_CAPS_NEW("video_mpeg_type",
|
||||||
"video/mpeg",
|
"video/mpeg",
|
||||||
"width", GST_PROPS_INT(384),
|
"width", GST_PROPS_INT(384),
|
||||||
"height", GST_PROPS_INT(288)
|
"height", GST_PROPS_INT(288));
|
||||||
);
|
|
||||||
|
|
||||||
Obviously, mimetypes and their corresponding properties are of major
|
Obviously, MIME types and their corresponding properties are of major importance
|
||||||
importance in GStreamer for uniquely identifying media streams.
|
in GStreamer for uniquely identifying media streams.
|
||||||
|
|
||||||
Official MIME media types are assigned by the IANA. Current
|
Official MIME media types are assigned by the IANA. Current assignments are at
|
||||||
assignments are at http://www.iana.org/assignments/media-types/.
|
http://www.iana.org/assignments/media-types/.
|
||||||
|
|
||||||
2) The problems
|
The problems
|
||||||
---------------
|
============
|
||||||
Some streams may have mimetypes or GstCaps that do not fully describe
|
|
||||||
the stream. In most cases, this is not a problem, though. For a stream
|
|
||||||
that contains Ogg/Vorbis data, we don't need to know the samplerate of
|
|
||||||
the raw audio stream, for example, since we can't play it back anyway.
|
|
||||||
The samplerate _is_ important for _raw_ audio, so a decoder would need
|
|
||||||
to retrieve the samplerate from the Ogg/Vorbis stream headers (that are
|
|
||||||
part of the bytestream) in order to pass it on in the GstCaps that
|
|
||||||
belongs to the decoded audio ('audio/raw').
|
|
||||||
However, other plugins *might* want to know such properties, even for
|
|
||||||
compressed streams. One such example is an AVI muxer, which does want
|
|
||||||
to know the samplerate of an audio stream, even when it is compressed.
|
|
||||||
|
|
||||||
Another problem is that many media types can be defined in multiple ways.
|
Some streams may have MIME types or GstCaps that do not fully describe the
|
||||||
For example, MJPEG video can be defined as video/jpeg, video/mjpeg,
|
stream. In most cases, this is not a problem, though. For example, if a stream
|
||||||
image/jpeg, video/avi with a compression of (fourcc) MJPG, etc. None of
|
contains Ogg/Vorbis data (which is of type 'application/ogg'), we don't need to
|
||||||
these is really official, since there isn't an official mimetype for
|
know the samplerate of the raw audio stream, since we can't play the encoded
|
||||||
encoded MJPEG video.
|
audio anyway. The samplerate is, however, important for raw audio, so a decoder
|
||||||
|
would need to retrieve the samplerate from the Ogg/Vorbis stream headers (the
|
||||||
|
headers are part of the bytestream) in order to pass it on in the GstCaps that
|
||||||
|
belongs to the decoded audio (which becomes a type like 'audio/raw'). However,
|
||||||
|
other plugins might want to know such properties, even for compressed streams.
|
||||||
|
One such example is an AVI muxer, which does want to know the samplerate of an
|
||||||
|
audio stream, even when it is compressed.
|
||||||
|
|
||||||
The main focus of this document is to propose a standardized set of
|
Another problem is that many media types can be defined in multiple ways. For
|
||||||
mimetypes and properties that will be used by the GStreamer plugins.
|
example, MJPEG video can be defined as 'video/jpeg', 'video/mjpeg',
|
||||||
|
'image/jpeg', 'video/avi' with a compression of (fourcc) MJPG, etc. None of
|
||||||
|
these is really official, since there isn't an official mimetype for encoded
|
||||||
|
MJPEG video.
|
||||||
|
|
||||||
3) Different types of streams
|
The main focus of this document is to propose a standardized set of MIME types
|
||||||
-----------------------------
|
and properties that will be used by the GStreamer plugins.
|
||||||
There are several types of media streams. The most important distinction
|
|
||||||
will be container formats, audio codecs and video codecs. Container
|
Different types of streams
|
||||||
formats are bytestreams that contain one or more substreams inside it,
|
==========================
|
||||||
and don't provide any direct media data itself. Examples are Quicktime,
|
|
||||||
AVI or MPEG System Stream. They mostly contain of a set of headers that
|
There are several types of media streams. The most important distinction will be
|
||||||
define the media stream(s) that is packed inside the container and the
|
container formats, audio codecs and video codecs. Container formats are
|
||||||
media data itself.
|
bytestreams that contain one or more substreams inside it, and don't provide any
|
||||||
Video codecs and audio codecs describe encoded audio or video data.
|
direct media data itself. Examples are Quicktime, AVI or MPEG System Stream.
|
||||||
Examples are MPEG-1 video, DivX video, MPEG-1 layer 3 (MP3) audio or
|
They mostly contain of a set of headers that define the media streams that are
|
||||||
Ogg/Vorbis audio. Actually, Ogg is a container format too (for Vorbis
|
packed inside the container, along with the media data itself.
|
||||||
audio), but these are usually used in conjunction with each other.
|
|
||||||
|
Video codecs and audio codecs describe encoded audio or video data. Examples are
|
||||||
|
MPEG-1 video, DivX video, MPEG-1 layer 3 (MP3) audio or Ogg/Vorbis audio.
|
||||||
|
Actually, Ogg is a container format too (for Vorbis audio), but these are
|
||||||
|
usually used in conjunction with each other.
|
||||||
|
|
||||||
|
Finally, there are the somewhat obvious (but not commonly encountered as files)
|
||||||
|
raw data formats.
|
||||||
|
|
||||||
|
Container formats
|
||||||
|
-----------------
|
||||||
|
|
||||||
3a) Container formats
|
|
||||||
---------------------
|
|
||||||
1 - AVI (Microsoft RIFF/AVI)
|
1 - AVI (Microsoft RIFF/AVI)
|
||||||
mimetype: video/avi
|
MIME type: video/avi
|
||||||
|
Properties:
|
||||||
|
Parser: avidemux
|
||||||
|
Formatter: avimux
|
||||||
|
|
||||||
2 - Quicktime (Apple)
|
2 - Quicktime (Apple)
|
||||||
mimetype: video/quicktime
|
MIME type: video/quicktime
|
||||||
|
Properties:
|
||||||
|
Parser: qtdemux
|
||||||
|
Formatter:
|
||||||
|
|
||||||
3 - MPEG (MPEG LA)
|
3 - MPEG (MPEG LA)
|
||||||
mimetype: video/mpeg
|
MIME type: video/mpeg
|
||||||
properties: 'systemstream' = TRUE (BOOLEAN)
|
Properties: 'systemstream' = TRUE (BOOLEAN)
|
||||||
|
Parser: mp1videoparse
|
||||||
|
Formatter:
|
||||||
|
|
||||||
4 - ASF (Microsoft)
|
4 - ASF (Microsoft)
|
||||||
mimetype: video/x-asf
|
MIME type: video/x-asf
|
||||||
|
Properties:
|
||||||
|
Parser: asfdemux
|
||||||
|
Formatter:
|
||||||
|
|
||||||
5 - WAV (PCM)
|
5 - WAV (PCM)
|
||||||
mimetype: audio/x-wav
|
MIME type: audio/x-wav
|
||||||
|
Properties:
|
||||||
|
Parser: wavparse
|
||||||
|
Formatter: wavenc
|
||||||
|
|
||||||
6 - RealMedia (Real)
|
6 - RealMedia (Real)
|
||||||
mimetype: video/x-pn-realvideo
|
MIME type: video/x-pn-realvideo
|
||||||
properties: 'systemstream' = TRUE (BOOLEAN)
|
Properties: 'systemstream' = TRUE (BOOLEAN)
|
||||||
|
Parser: rmdemux
|
||||||
|
Formatter:
|
||||||
|
|
||||||
7 - DV (Digital Video)
|
7 - DV (Digital Video)
|
||||||
mimetype: video/x-dv
|
MIME type: video/x-dv
|
||||||
properties: 'systemstream' = TRUE (BOOLEAN)
|
Properties: 'systemstream' = TRUE (BOOLEAN)
|
||||||
|
Parser: gst1394
|
||||||
|
Formatter:
|
||||||
|
|
||||||
8 - Ogg (Xiph)
|
8 - Ogg (Xiph)
|
||||||
mimetype: application/ogg
|
MIME type: application/ogg
|
||||||
|
Properties:
|
||||||
|
Parser: vorbisfile
|
||||||
|
Formatter: vorbisenc
|
||||||
|
|
||||||
9 - Matroska
|
9 - Matroska
|
||||||
mimetype: video/x-mkv
|
MIME type: video/x-mkv
|
||||||
|
Properties:
|
||||||
|
Parser:
|
||||||
|
Formatter:
|
||||||
|
|
||||||
10 - Shockwave (Macromedia)
|
10 - Shockwave (Macromedia)
|
||||||
mimetype: application/x-shockwave-flash
|
MIME type: application/x-shockwave-flash
|
||||||
|
Properties:
|
||||||
|
Parser: swfdec
|
||||||
|
Formatter:
|
||||||
|
|
||||||
11 - AU audio (Sun)
|
11 - AU audio (Sun)
|
||||||
mimetype: audio/x-au
|
MIME type: audio/x-au
|
||||||
|
Properties:
|
||||||
|
Parser: auparse
|
||||||
|
Formatter:
|
||||||
|
|
||||||
12 - Mod audio
|
12 - Mod audio
|
||||||
mimetype: audio/x-mod
|
MIME type: audio/x-mod
|
||||||
|
Properties:
|
||||||
|
Parser: modplug, mikmod
|
||||||
|
Formatter:
|
||||||
|
|
||||||
13 - FLX video (?)
|
13 - FLX video
|
||||||
mimetype: video/x-fli
|
MIME type: video/x-fli
|
||||||
|
Properties:
|
||||||
|
Parser: flxdec
|
||||||
|
Formatter:
|
||||||
|
|
||||||
14 - Monkeyaudio
|
14 - Monkeyaudio
|
||||||
mimetype: application/x-ape
|
MIME type: application/x-ape
|
||||||
|
Properties:
|
||||||
|
Parser:
|
||||||
|
Formatter:
|
||||||
|
|
||||||
15 - AIFF audio
|
15 - AIFF audio
|
||||||
mimetype: audio/x-aiff
|
MIME type: audio/x-aiff
|
||||||
|
Properties:
|
||||||
|
Parser:
|
||||||
|
Formatter:
|
||||||
|
|
||||||
16 - SID audio
|
16 - SID audio
|
||||||
mimetype: audio/x-sid
|
MIME type: audio/x-sid
|
||||||
|
Properties:
|
||||||
|
Parser:
|
||||||
|
Formatter:
|
||||||
|
|
||||||
Please note that we try to keep these mimetypes as similar as possible
|
Please note that we try to keep these MIME types as similar as possible to the
|
||||||
to what's used as standard mimetypes in Gnome (Gnome-VFS/Nautilus) and
|
MIME types used as standards in Gnome (Gnome-VFS/Nautilus) and KDE
|
||||||
KDE (Konqueror).
|
(Konqueror).
|
||||||
|
|
||||||
Current problems: there's a very thin line between audio codecs and
|
Also, there is a very thin line between audio codecs and audio containers
|
||||||
audio containers (take mp3 vs. sid, etc.) - this is just a per-case
|
(take mp3 vs. sid, etc.). This is just a per-case thing right now and needs to
|
||||||
thing right now and needs to be documented further.
|
be documented further.
|
||||||
|
|
||||||
|
Video codecs
|
||||||
|
------------
|
||||||
|
|
||||||
3b) Video codecs
|
|
||||||
For convenience, the fourcc codes used in the AVI container format will be
|
For convenience, the fourcc codes used in the AVI container format will be
|
||||||
listed along with the mimetype and optional properties.
|
listed along with the MIME type and optional properties.
|
||||||
|
|
||||||
Preface - (optional) properties for all video formats:
|
Optional properties for all video formats are the following:
|
||||||
'width' = X (INT)
|
|
||||||
'height' = X (INT)
|
|
||||||
'pixel_width' and 'pixel_height' = X (2xINT, together aspect ratio)
|
|
||||||
'framerate' = X (FLOAT)
|
|
||||||
|
|
||||||
1 - Raw Video (YUV/YCbCr)
|
width = 1 - MAXINT (INT)
|
||||||
mimetype: video/x-raw-yuv
|
height = 1 - MAXINT (INT)
|
||||||
properties: 'format' = 'XXXX' (fourcc)
|
pixel_width = 1 - MAXINT (INT, with pixel_height forms aspect ratio)
|
||||||
known fourccs: YUY2, I420, Y41P, YVYU, UYVY, etc.
|
pixel_height = 1 - MAXINT (INT, with pixel_width forms aspect ratio)
|
||||||
properties 'width' and 'height' are required
|
framerate = 0 - MAXFLOAT (FLOAT)
|
||||||
|
|
||||||
Note: some raw video formats have implicit alignment rules. We should
|
1 - MPEG-1, -2 and -4 video (ISO/LA MPEG)
|
||||||
discuss this more.
|
MIME type: video/mpeg
|
||||||
Note: some formats have multiple fourccs (e.g. IYUV/I420 or YUY2/YUYV).
|
Properties: systemstream = FALSE (BOOLEAN)
|
||||||
For each of these, we only use one (e.g. I420 and YUY2).
|
mpegversion = 1/2/4 (INT)
|
||||||
|
Known fourccs: MPEG, MPGI
|
||||||
|
Encoder: mpeg1enc, mpeg2enc
|
||||||
|
Decoder: mpeg1dec, mpeg2dec, mpeg2subt
|
||||||
|
|
||||||
Currently recognized formats:
|
2 - DivX 3.x, 4.x and 5.x video (divx.com)
|
||||||
YUY2: packed, Y-U-Y-V order, U/V hor 2x subsampled (YUV-4:2:2, 16 bpp)
|
MIME type: video/x-divx
|
||||||
YVYU: packed, Y-V-Y-U order, U/V hor 2x subsampled (YUV-4:2:2, 16 bpp)
|
Properties:
|
||||||
UYVY: packed, U-Y-V-Y order, U/V hor 2x subsampled (YUV-4:2:2, 16 bpp)
|
Optional properties: divxversion = 3/4/5 (INT)
|
||||||
Y41P: packed, UYVYUYVYYYYY order, U/V hor 4x subsampled (YUV-4:1:1, 12 bpp)
|
Known fourccs: DIV3, DIV4, DIV5, DIVX, DX50, DIVX, divx
|
||||||
IUY2: packed, U-Y-V order, not subsampled (YUV-1:1:1, 24 bpp)
|
Encoder:
|
||||||
|
Decoder: dvdreadsrc, dvdnavsrc
|
||||||
|
|
||||||
Y42B: planar, Y-U-V order, U/V hor 2x subsampled (YUV-4:2:2, 16 bpp)
|
3 - Microsoft MPEG 4.1, 4.2 and 4.3
|
||||||
YV12: planar, Y-V-U order, U/V hor+ver 2x subsampled (YUV-4:2:0, 12 bpp)
|
MIME type: video/x-msmpeg
|
||||||
I420: planar, Y-U-V order, U/V hor+ver 2x subsampled (YUV-4:2:0, 12 bpp)
|
Properties:
|
||||||
Y41B: planar, Y-U-V order, U/V hor 4x subsampled (YUV-4:1:1, 12bpp)
|
Optional properties: msmpegversion = 41/42/43 (INT)
|
||||||
YUV9: planar, Y-U-V order, U/V hor+ver 4x subsampled (YUV-4:1:0, 9bpp)
|
Known fourccs: MPG4, MP42, MP43
|
||||||
YVU9: planar, Y-V-U order, U/V hor+ver 4x subsampled (YUV-4:1:0, 9bpp)
|
Encoder: ffenc_msmpeg4, ffenc_msmpeg4v1, ffenc_msmpeg4v2
|
||||||
|
Decoder: ffdec_msmpeg4, ffdec_msmpeg4v1, ffdec_msmpeg4v2
|
||||||
|
|
||||||
Y800: one-plane (Y-only, YUV-4:0:0, 8bpp)
|
4 - Motion-JPEG (official and extended)
|
||||||
|
MIME type: video/x-jpeg
|
||||||
|
Properties:
|
||||||
|
Known fourccs: MJPG (YUY2 MJPEG), JPEG (any), PIXL (Pinnacle/Miro), VIXL
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
See http://www.fourcc.org/ for more information.
|
5 - Sorensen (Quicktime - SVQ1/SVQ3)
|
||||||
|
MIME types: video/x-svq
|
||||||
|
Properties: svqversion = 1/3 (INT)
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
Note: YUV-4:4:4 (both planar and packed, in multiple orders) are missing.
|
6 - H263 and related codecs
|
||||||
|
MIME type: video/x-h263
|
||||||
|
Properties:
|
||||||
|
Known fourccs: H263, i263, M263, x263, VDOW, VIVO
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
2) Raw Video (RGB)
|
7 - RealVideo (Real)
|
||||||
-------------------
|
MIME type: video/x-pn-realvideo
|
||||||
mimetype: video/x-raw-rgb
|
Properties: systemstream = FALSE (BOOLEAN)
|
||||||
properties: 'endianness' = 1234/4321 (INT) <- endianness
|
Known fourccs: RV10, RV20, RV30
|
||||||
'depth' = 15/16/24 (INT) <- bits per pixel (depth)
|
Encoder:
|
||||||
'bpp' = 16/24/32 (INT) <- bits per pixel (in memory)
|
Decoder: rmdemux
|
||||||
'red_mask' = bitmask (0x..) (INT) <- red pixel mask
|
|
||||||
'green_mask' = bitmask (0x..) (INT) <- green pixel mask
|
|
||||||
'blue_mask' = bitmask (0x..) (INT) <- blue pixel mask
|
|
||||||
properties 'width' and 'height' are required
|
|
||||||
|
|
||||||
'bpp' is the number of bits of memory used for each pixel. 'depth'
|
8 - Digital Video (DV)
|
||||||
is the color depth.
|
MIME type: video/x-dv
|
||||||
|
Properties: systemstream = FALSE (BOOLEAN)
|
||||||
|
Known fourccs: DVSD, dvsd
|
||||||
|
Encoder:
|
||||||
|
Decoder: dvdec
|
||||||
|
|
||||||
24 and 32 bit RGB should always be specified as big endian, since
|
9 - Windows Media Video 1 and 2 (WMV)
|
||||||
any little endian format can be transformed into big endian by
|
MIME type: video/x-wmv
|
||||||
rearranging the color masks. 15 and 16 bit formats should generally
|
Properties: wmvversion = 1/2 (INT)
|
||||||
have the same byte order as the cpu.
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
Color masks are interpreted by loading 'bpp' number of bits using
|
10 - XviD (xvid.org)
|
||||||
'endianness' rule, and masking and shifting by each color mask.
|
MIME type: video/x-xvid
|
||||||
Loading a 24-bit value cannot be done directly, but one can perform
|
Properties:
|
||||||
an equivalent operation.
|
Known fourccs: xvid, XVID
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
Examples:
|
11 - 3IVX (3ixv.org)
|
||||||
msb .. lsb
|
MIME type: video/x-3ivx
|
||||||
- memory: RRRRRRRR GGGGGGGG BBBBBBBB RRRRRRRR GGGGGGGG ...
|
Properties:
|
||||||
'bpp' = 24
|
Known fourccs: 3IV0, 3IV1, 3IV2
|
||||||
'depth' = 24
|
Encoder:
|
||||||
'endianness' = 4321 (G_BIG_ENDIAN)
|
Decoder:
|
||||||
'red_mask' = 0xff0000
|
|
||||||
'green_mask' = 0x00ff00
|
|
||||||
'blue_mask' = 0x0000ff
|
|
||||||
|
|
||||||
- memory: xRRRRRGG GGGBBBBB xRRRRRGG GGGBBBBB xRRRRRGG ...
|
12 - Ogg/Tarkin (Xiph)
|
||||||
'bpp' = 16
|
MIME type: video/x-tarkin
|
||||||
'depth' = 15
|
Properties:
|
||||||
'endianness' = 4321 (G_BIG_ENDIAN)
|
Encoder:
|
||||||
'red_mask' = 0x7c00
|
Decoder:
|
||||||
'green_mask' = 0x03e0
|
|
||||||
'blue_mask' = 0x003f
|
|
||||||
|
|
||||||
- memory: GGGBBBBB xRRRRRGG GGGBBBBB xRRRRRGG GGGBBBBB ...
|
13 - VP3
|
||||||
'bpp' = 16
|
MIME type: video/x-vp3
|
||||||
'depth' = 15
|
Properties:
|
||||||
'endianness' = 1234 (G_LITTLE_ENDIAN)
|
Encoder:
|
||||||
'red_mask' = 0x7c00
|
Decoder:
|
||||||
'green_mask' = 0x03e0
|
|
||||||
'blue_mask' = 0x003f
|
|
||||||
|
|
||||||
3 - MPEG-1, -2 and -4 video (ISO/LA MPEG)
|
14 - Ogg/Theora (Xiph, VP3-like)
|
||||||
mimetype: video/mpeg
|
MIME type: video/x-theora
|
||||||
properties: 'systemstream' = FALSE (BOOLEAN)
|
Properties:
|
||||||
'mpegversion' = 1/2/4 (INT)
|
Encoder:
|
||||||
known fourccs: MPEG, MPGI
|
Decoder:
|
||||||
|
|
||||||
4 - DivX 3.x, 4.x and 5.x video (divx.com)
|
15 - Huffyuv
|
||||||
mimetype: video/x-divx
|
MIME type: video/x-huffyuv
|
||||||
optional properties: 'divxversion' = 3/4/5 (INT)
|
Properties:
|
||||||
known fourccs: DIV3, DIV4, DIV5, DIVX, DX50, DIVX, divx
|
Known fourccs: HFYU
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
5 - Microsoft MPEG 4.1, 4.2 and 4.3
|
16 - FF Video 1 (FFMPEG)
|
||||||
mimetype: video/x-msmpeg
|
MIME type: video/x-ffv
|
||||||
optional properties: 'msmpegversion' = 41/42/43 (INT)
|
Properties: ffvversion = 1 (INT)
|
||||||
known fourccs: MPG4, MP42, MP43
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
6 - Motion-JPEG (official and extended)
|
17 - H264
|
||||||
mimetype: video/x-jpeg
|
MIME type: video/x-h264
|
||||||
known fourccs: MJPG (YUY2 MJPEG), JPEG (any), PIXL (Pinnacle/Miro), VIXL
|
Properties:
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
7 - Sorensen (Quicktime - SVQ1/SVQ3)
|
18 - Indeo 3 (Intel)
|
||||||
mimetypes: video/x-svq
|
MIME type: video/x-indeo
|
||||||
properties: 'svqversion' = 1/3 (INT)
|
Properties: indeoversion = 3 (INT)
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
8 - H263 and related codecs
|
19 - Portable Network Graphics (PNG)
|
||||||
mimetype: video/x-h263
|
MIME type: video/x-png
|
||||||
known fourccs: H263, i263, M263, x263, VDOW, VIVO
|
Properties:
|
||||||
|
Encoder:
|
||||||
9 - RealVideo (Real)
|
Decoder:
|
||||||
mimetype: video/x-pn-realvideo
|
|
||||||
properties: 'systemstream' = FALSE (BOOLEAN)
|
|
||||||
known fourccs: RV10, RV20, RV30
|
|
||||||
|
|
||||||
10 - Digital Video (DV)
|
|
||||||
mimetype: video/x-dv
|
|
||||||
properties: 'systemstream' = FALSE (BOOLEAN)
|
|
||||||
known fourccs: DVSD, dvsd
|
|
||||||
|
|
||||||
11 - Windows Media Video 1 and 2 (WMV)
|
|
||||||
mimetype: video/x-wmv
|
|
||||||
properties: 'wmvversion' = 1/2 (INT)
|
|
||||||
|
|
||||||
12 - XviD (xvid.org)
|
|
||||||
mimetype: video/x-xvid
|
|
||||||
known fourccs: xvid, XVID
|
|
||||||
|
|
||||||
13 - 3IVX (3ixv.org)
|
|
||||||
mimetype: video/x-3ivx
|
|
||||||
known fourccs: 3IV0, 3IV1, 3IV2
|
|
||||||
|
|
||||||
14 - Ogg/Tarkin (Xiph)
|
|
||||||
mimetype: video/x-tarkin
|
|
||||||
|
|
||||||
15 - VP3
|
|
||||||
mimetype: video/x-vp3
|
|
||||||
|
|
||||||
16 - Ogg/Theora (Xiph, VP3-like)
|
|
||||||
mimetype: video/x-theora
|
|
||||||
|
|
||||||
17 - Huffyuv
|
|
||||||
mimetype: video/x-huffyuv
|
|
||||||
known fourccs: HFYU
|
|
||||||
|
|
||||||
18 - FF Video 1 (FFMPEG)
|
|
||||||
mimetype: video/x-ffv
|
|
||||||
properties: 'ffvversion' = 1 (INT)
|
|
||||||
|
|
||||||
19 - H264
|
|
||||||
mimetype: video/x-h264
|
|
||||||
|
|
||||||
20 - Indeo 3 (Intel)
|
|
||||||
mimetype: video/x-indeo
|
|
||||||
properties: 'indeoversion' = 3 (INT)
|
|
||||||
|
|
||||||
21 - Portable Network Graphics (PNG)
|
|
||||||
mimetype: video/x-png
|
|
||||||
|
|
||||||
TODO: subsampling information for YUV?
|
TODO: subsampling information for YUV?
|
||||||
|
|
||||||
|
|
@ -303,121 +336,248 @@ TODO: colorspace identifications for MJPEG? How?
|
||||||
|
|
||||||
TODO: how to distinguish MJPEG-A/B (Quicktime) and lossless JPEG?
|
TODO: how to distinguish MJPEG-A/B (Quicktime) and lossless JPEG?
|
||||||
|
|
||||||
TODO: divx4/divx5/xvid/3ivx/mpeg-4 - how to make them overlap? (all
|
TODO: divx4/divx5/xvid/3ivx/mpeg-4 - how to make them overlap? (all
|
||||||
ISO MPEG-4 compatible)
|
ISO MPEG-4 compatible)
|
||||||
|
|
||||||
3c) Audio Codecs
|
Audio codecs
|
||||||
----------------
|
------------
|
||||||
for convenience, the two-byte hexcodes (as are being used for identification
|
|
||||||
in AVI files) are also given
|
|
||||||
|
|
||||||
Preface - (optional) properties for all audio formats:
|
For convenience, the two-byte hexcodes (as used for identification in AVI files)
|
||||||
'rate' = X (int) <- sampling rate
|
are also given.
|
||||||
'channels' = X (int) <- number of audio channels
|
|
||||||
|
|
||||||
1 - Raw Audio (integer format)
|
Properties for all audio formats include the following:
|
||||||
mimetype: audio/x-raw-int
|
|
||||||
properties: 'width' = X (INT) <- memory bits per sample
|
|
||||||
'depth' = X (INT) <- used bits per sample
|
|
||||||
'signed' = X (BOOLEAN)
|
|
||||||
'endianness' = 1234/4321 (INT)
|
|
||||||
|
|
||||||
2 - Raw Audio (floating point format)
|
rate = 1 - MAXINT (INT, sampling rate)
|
||||||
mimetype: audio/x-raw-float
|
channels = 1 - MAXINT (INT, number of audio channels)
|
||||||
properties: 'depth' = X (INT) <- 32=float, 64=double
|
|
||||||
'endianness' = 1234/4321 (INT) <- use G_BIG/LITTLE_ENDIAN!
|
|
||||||
'slope' = X (FLOAT, normally 1.0)
|
|
||||||
'intercept' = X (FLOAT, normally 0.0)
|
|
||||||
|
|
||||||
3 - Alaw Raw Audio
|
1 - Alaw Raw Audio
|
||||||
mimetype: audio/x-alaw
|
MIME type: audio/x-alaw
|
||||||
|
Properties:
|
||||||
|
Encoder: alawenc
|
||||||
|
Decoder: alawdec
|
||||||
|
|
||||||
4 - Mulaw Raw Audio
|
2 - Mulaw Raw Audio
|
||||||
mimetype: audio/x-mulaw
|
MIME type: audio/x-mulaw
|
||||||
|
Properties:
|
||||||
|
Encoder: mulawenc
|
||||||
|
Decoder: mulawdec
|
||||||
|
|
||||||
5 - MPEG-1 layer 1/2/3 audio
|
3 - MPEG-1 layer 1/2/3 audio
|
||||||
mimetype: audio/mpeg
|
MIME type: audio/mpeg
|
||||||
properties: 'mpegversion' = 1 (INT)
|
Properties: mpegversion = 1 (INT)
|
||||||
'layer' = 1/2/3 (INT)
|
layer = 1/2/3 (INT)
|
||||||
|
Encoder: lame
|
||||||
|
Decoder: mad
|
||||||
|
|
||||||
6 - Ogg/Vorbis
|
4 - Ogg/Vorbis
|
||||||
mimetype: audio/x-vorbis
|
MIME type: audio/x-vorbis
|
||||||
|
Encoder: vorbisenc
|
||||||
|
Decoder: vorbisfile
|
||||||
|
|
||||||
7 - Windows Media Audio 1 and 2 (WMA)
|
5 - Windows Media Audio 1 and 2 (WMA)
|
||||||
mimetype: audio/x-wma
|
MIME type: audio/x-wma
|
||||||
properties: 'wmaversion' = 1/2 (INT)
|
Properties: wmaversion = 1/2 (INT)
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
8 - AC3
|
6 - AC3
|
||||||
mimetype: audio/x-ac3
|
MIME type: audio/x-ac3
|
||||||
|
Properties:
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
9 - FLAC (Free Lossless Audio Codec)
|
7 - FLAC (Free Lossless Audio Codec)
|
||||||
mimetype: audio/x-flac
|
MIME type: audio/x-flac
|
||||||
|
Properties:
|
||||||
|
Encoder: flacenc
|
||||||
|
Decoder: flacdec
|
||||||
|
|
||||||
10 - MACE 3/6 (Quicktime audio)
|
8 - MACE 3/6 (Quicktime audio)
|
||||||
mimetype: audio/x-mace
|
MIME type: audio/x-mace
|
||||||
properties: 'maceversion' = 3/6 (INT)
|
Properties: maceversion = 3/6 (INT)
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
11 - MPEG-4 AAC
|
9 - MPEG-4 AAC
|
||||||
mimetype: audio/mpeg
|
MIME type: audio/mpeg
|
||||||
properties: 'mpegversion' = 4 (INT)
|
Properties: mpegversion = 4 (INT)
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
12 - (IMA) ADPCM (Quicktime/WAV/Microsoft/4XM)
|
10 - (IMA) ADPCM (Quicktime/WAV/Microsoft/4XM)
|
||||||
mimetype: audio/x-adpcm
|
MIME type: audio/x-adpcm
|
||||||
properties: 'layout' = "quicktime"/"wav"/"microsoft"/"4xm" (STRING)
|
Properties: layout = "quicktime"/"wav"/"microsoft"/"4xm" (STRING)
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
Note: the difference between each of these is the number of
|
Note: The difference between each of these four PCM formats is the number
|
||||||
samples packaed together per channel. For WAV, for
|
of samples packaed together per channel. For WAV, for example, each
|
||||||
example, each sample is 4 bit, and 8 samples are packed
|
sample is 4 bit, and 8 samples are packed together per channel in the
|
||||||
together per channel in the bytestream. For the others,
|
bytestream. For the others, refer to technical documentation. We
|
||||||
refer to technical documentation.
|
probably want to distinguish these differently, but I don't know how,
|
||||||
We probably want to distinguish these differently, but
|
yet.
|
||||||
I don't know how, yet.
|
|
||||||
|
|
||||||
13 - RealAudio (Real)
|
11 - RealAudio (Real)
|
||||||
mimetype: audio/x-pn-realaudio
|
MIME type: audio/x-pn-realaudio
|
||||||
properties: 'bitrate' = 14400/28800 (INT)
|
Properties: bitrate = 14400/28800 (INT)
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
14 - DV Audio
|
12 - DV Audio
|
||||||
mimetype: audio/x-dv
|
MIME type: audio/x-dv
|
||||||
|
Properties:
|
||||||
|
Encoder:
|
||||||
|
Decoder:
|
||||||
|
|
||||||
15 - GSM Audio
|
13 - GSM Audio
|
||||||
mimetype: audio/x-gsm
|
MIME type: audio/x-gsm
|
||||||
|
Properties:
|
||||||
|
Encoder: gsmenc
|
||||||
|
Decoder: gsmdec
|
||||||
|
|
||||||
16 - Speex audio
|
14 - Speex audio
|
||||||
mimetype: audio/x-speex
|
MIME type: audio/x-speex
|
||||||
|
Properties:
|
||||||
|
|
||||||
TODO: adpcm/dv needs confirmation from someone with knowledge...
|
TODO: adpcm/dv needs confirmation from someone with knowledge...
|
||||||
|
|
||||||
3d) Plugin Guidelines
|
Raw formats
|
||||||
---------------------
|
-----------
|
||||||
So, a short bit on what plugins should do. Above, I've stated that
|
|
||||||
audio properties like "channels" and "rate" or video properties like
|
|
||||||
"width" and "height" are all optional. This doesn't mean you can
|
|
||||||
just simply omit them and everything will still work!
|
|
||||||
|
|
||||||
An example is the best way to explain all this. AVI needs the width,
|
Raw formats contain unencoded, raw media information. These are rather rare from
|
||||||
height, rate and channels for the AVI header. So if these properties
|
an end user point of view since raw media files have historically been
|
||||||
are missing, avimux cannot work. On the other hand, MPEG doesn't have
|
prohibitively large ... hence the multitude of encoding formats.
|
||||||
such properties in its header and would thus need to parse the stream
|
|
||||||
in order to find them out; we don't want that either (a plugin does
|
|
||||||
one job). So normally, mpegdemux and avimux wouldn't allow transcoding.
|
|
||||||
To solve this problem, there are stream parser elements (such as
|
|
||||||
mpegaudioparse, ac3parse and mpeg1videoparse).
|
|
||||||
|
|
||||||
Conclusions to draw from here: a plugin gives info it can provide as
|
Raw video formats require the following common properties, in addition to
|
||||||
seen from its own task/job. If it can't, other elements might still
|
format-specific properties:
|
||||||
need it and a stream parser needs to be written if it doesn't already
|
|
||||||
exist.
|
|
||||||
|
|
||||||
On properties that can be described by one of these (properties such
|
width = 1 - MAXINT (INT)
|
||||||
as 'width', 'height', 'fps', etc.): they're forbidden and should be
|
height = 1 - MAXINT (INT)
|
||||||
handled using filtered caps.
|
|
||||||
|
|
||||||
4) Status of this document
|
1 - Raw Video (YUV/YCbCr)
|
||||||
---------------------------
|
MIME type: video/x-raw-yuv
|
||||||
Not all plugins strictly follow these guidelines yet, but these are the
|
Properties: 'format' = 'XXXX' (fourcc)
|
||||||
official types. Plugins not following these specs either use extensions
|
Known fourccs: YUY2, I420, Y41P, YVYU, UYVY, etc.
|
||||||
that should be documented, or are buggy (and should be fixed).
|
Properties:
|
||||||
|
|
||||||
Blame Ronald Bultje <rbultje@ronald.bitfreak.net> aka BBB for any mistakes
|
Some raw video formats have implicit alignment rules. We should discuss this
|
||||||
in this document.
|
more. Also, some formats have multiple fourccs (e.g. IYUV/I420 or
|
||||||
|
YUY2/YUYV). For each of these, we only use one (e.g. I420 and YUY2).
|
||||||
|
|
||||||
|
Currently recognized formats:
|
||||||
|
|
||||||
|
YUY2: packed, Y-U-Y-V order, U/V hor 2x subsampled (YUV-4:2:2, 16 bpp)
|
||||||
|
YVYU: packed, Y-V-Y-U order, U/V hor 2x subsampled (YUV-4:2:2, 16 bpp)
|
||||||
|
UYVY: packed, U-Y-V-Y order, U/V hor 2x subsampled (YUV-4:2:2, 16 bpp)
|
||||||
|
Y41P: packed, UYVYUYVYYYYY order, U/V hor 4x subsampled (YUV-4:1:1, 12 bpp)
|
||||||
|
IUY2: packed, U-Y-V order, not subsampled (YUV-1:1:1, 24 bpp)
|
||||||
|
|
||||||
|
Y42B: planar, Y-U-V order, U/V hor 2x subsampled (YUV-4:2:2, 16 bpp)
|
||||||
|
YV12: planar, Y-V-U order, U/V hor+ver 2x subsampled (YUV-4:2:0, 12 bpp)
|
||||||
|
I420: planar, Y-U-V order, U/V hor+ver 2x subsampled (YUV-4:2:0, 12 bpp)
|
||||||
|
Y41B: planar, Y-U-V order, U/V hor 4x subsampled (YUV-4:1:1, 12bpp)
|
||||||
|
YUV9: planar, Y-U-V order, U/V hor+ver 4x subsampled (YUV-4:1:0, 9bpp)
|
||||||
|
YVU9: planar, Y-V-U order, U/V hor+ver 4x subsampled (YUV-4:1:0, 9bpp)
|
||||||
|
|
||||||
|
Y800: one-plane (Y-only, YUV-4:0:0, 8bpp)
|
||||||
|
|
||||||
|
See http://www.fourcc.org/ for more information.
|
||||||
|
|
||||||
|
Note: YUV-4:4:4 (both planar and packed, in multiple orders) are missing.
|
||||||
|
|
||||||
|
2 - Raw video (RGB)
|
||||||
|
MIME type: video/x-raw-rgb
|
||||||
|
Properties: endianness = 1234/4321 (INT) <- use G_LITTLE/BIG_ENDIAN
|
||||||
|
depth = 15/16/24 (INT, color depth)
|
||||||
|
bpp = 16/24/32 (INT, bits used to store each pixel)
|
||||||
|
red_mask = bitmask (0x..) (INT)
|
||||||
|
green_mask = bitmask (0x..) (INT)
|
||||||
|
blue_mask = bitmask (0x..) (INT)
|
||||||
|
|
||||||
|
24 and 32 bit RGB should always be specified as big endian, since any little
|
||||||
|
endian format can be transformed into big endian by rearranging the color
|
||||||
|
masks. 15 and 16 bit formats should generally have the same byte order as
|
||||||
|
the CPU.
|
||||||
|
|
||||||
|
Color masks are interpreted by loading 'bpp' number of bits using the given
|
||||||
|
'endianness', and masking and shifting by each color mask. Loading a 24-bit
|
||||||
|
value cannot be done directly, but one can perform an equivalent operation.
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
msb .. lsb
|
||||||
|
- memory: RRRRRRRR GGGGGGGG BBBBBBBB RRRRRRRR GGGGGGGG ...
|
||||||
|
bpp = 24
|
||||||
|
depth = 24
|
||||||
|
endianness = 4321 (G_BIG_ENDIAN)
|
||||||
|
red_mask = 0xff0000
|
||||||
|
green_mask = 0x00ff00
|
||||||
|
blue_mask = 0x0000ff
|
||||||
|
|
||||||
|
- memory: xRRRRRGG GGGBBBBB xRRRRRGG GGGBBBBB xRRRRRGG ...
|
||||||
|
bpp = 16
|
||||||
|
depth = 15
|
||||||
|
endianness = 4321 (G_BIG_ENDIAN)
|
||||||
|
red_mask = 0x7c00
|
||||||
|
green_mask = 0x03e0
|
||||||
|
blue_mask = 0x003f
|
||||||
|
|
||||||
|
- memory: GGGBBBBB xRRRRRGG GGGBBBBB xRRRRRGG GGGBBBBB ...
|
||||||
|
bpp = 16
|
||||||
|
depth = 15
|
||||||
|
endianness = 1234 (G_LITTLE_ENDIAN)
|
||||||
|
red_mask = 0x7c00
|
||||||
|
green_mask = 0x03e0
|
||||||
|
blue_mask = 0x003f
|
||||||
|
|
||||||
|
The raw audio formats require the following common properties, in addition to
|
||||||
|
format-specific properties:
|
||||||
|
|
||||||
|
rate = 1 - MAXINT (INT, sampling rate)
|
||||||
|
channels = 1 - MAXINT (INT, number of audio channels)
|
||||||
|
buffer-frames = 1 - MAXINT (INT, number of frames per buffer)
|
||||||
|
endianness = 1234/4321 (INT) <- use G_BIG/LITTLE_ENDIAN
|
||||||
|
|
||||||
|
3 - Raw audio (integer format)
|
||||||
|
MIME type: audio/x-raw-int
|
||||||
|
properties: width = 8/16/32 (INT, bits used to store each sample)
|
||||||
|
depth = 8 - 32 (INT, bits actually used per sample)
|
||||||
|
signed = TRUE/FALSE (BOOLEAN)
|
||||||
|
|
||||||
|
4 - Raw audio (floating point format)
|
||||||
|
MIME type: audio/x-raw-float
|
||||||
|
Properties: width = 32/64 (INT)
|
||||||
|
|
||||||
|
Plugin Guidelines
|
||||||
|
=================
|
||||||
|
|
||||||
|
So, a short bit on what plugins should do. Above, I've stated that audio
|
||||||
|
properties like 'channels' and 'rate' or video properties like 'width' and
|
||||||
|
'height' are all optional. This doesn't mean you can just simply omit them and
|
||||||
|
everything will still work!
|
||||||
|
|
||||||
|
An example is the best way to explain all this. AVI needs the width, height,
|
||||||
|
rate and channels for the AVI header. So if these properties are missing, the
|
||||||
|
avimux element cannot properly create the AVI header. On the other hand, MPEG
|
||||||
|
doesn't have such properties in its header, so the mpegdemux element would need
|
||||||
|
to parse the separate streams in order to find them out. We don't want that
|
||||||
|
either, because a plugin only does one job. So normally, mpegdemux and avimux
|
||||||
|
wouldn't allow transcoding. To solve this problem, there are stream parser
|
||||||
|
elements (such as mpegaudioparse, ac3parse and mpeg1videoparse).
|
||||||
|
|
||||||
|
Conclusions to draw from here: a plugin gives info it can provide as seen from
|
||||||
|
its own task/job. If it can't, other elements might still need it and a stream
|
||||||
|
parser needs to be written if it doesn't already exist.
|
||||||
|
|
||||||
|
On properties that can be described by one of these (properties such as 'width',
|
||||||
|
'height', 'fps', etc.): they're forbidden and should be handled using filtered
|
||||||
|
caps.
|
||||||
|
|
||||||
|
Status of this document
|
||||||
|
=======================
|
||||||
|
|
||||||
|
Not all plugins strictly follow these guidelines yet, but these are the official
|
||||||
|
types. Plugins not following these specs either use extensions that should be
|
||||||
|
documented, or are buggy (and should be fixed).
|
||||||
|
|
||||||
|
Blame Ronald Bultje <rbultje@ronald.bitfreak.net> aka BBB for any mistakes in
|
||||||
|
this document.
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue