mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2024-07-03 13:16:04 +00:00
Code Issues 1 Projects Releases Wiki Activity
gstreamer/docs/random/mimetypes
Ronald S. Bultje 535ebb2f49 Some more additions to the list... We proposed to use different mimetypes for float/int/law audio, and for rgb/yuv...... 
Original commit message from CVS:
Some more additions to the list... We proposed to use different mimetypes for float/int/law audio, and for rgb/yuv... waiting for comments
2003-06-20 21:44:46 +00:00

327 lines
11 KiB
Plaintext
Raw Blame History

Mimetypes in GStreamer
======================
1) What is a mimetype
---------------------
A mimetype is a combination of two (short) strings (words), the content
type and the content subtype, that make up a pair that describes a file
content type. In multimedia, mime types are used to describe the media
streamtype . In GStreamer, obsiously, we use mimetypes in the same way.
They are part of a GstCaps, that describes a media stream. Besides a
mimetype, a GstCaps also contains stream properties (GstProps), which
are combinations of key/value pairs, and a name.
An example of a mimetype is 'video/mpeg'. A corresponding GstCaps could
be created using:
GstCaps *caps = gst_caps_new("video_mpeg_type",
"video/mpeg",
gst_props_new("width", GST_PROPS_INT(384),
"height", GST_PROPS_INT(288),
NULL));
or using a macro:
GstCaps *caps = GST_CAPS_NEW("video_mpeg_type",
"video/mpeg",
"width", GST_PROPS_INT(384),
"height", GST_PROPS_INT(288)
);
Obviously, mimetypes and their corresponding properties are of major
importance in GStreamer for uniquely identifying media streams.
2) The problems
---------------
Some streams may have mimetypes or GstCaps that do not fully describe
the stream. In most cases, this is not a problem, though. For a stream
that contains Ogg/Vorbis data, we don't need to know the samplerate of
the raw audio stream, for example, since we can't play it back anyway.
The samplerate _is_ important for _raw_ audio, so a decoder would need
to retrieve the samplerate from the Ogg/Vorbis stream headers (that are
part of the bytestream) in order to pass it on in the GstCaps that
belongs to the decoded audio ('audio/raw').
However, other plugins *might* want to know such properties, even for
compressed streams. One such example is an AVI muxer, which does want
to know the samplerate of an audio stream, even when it is compressed.
Another problem is that many media types can be defined in multiple ways.
For example, MJPEG video can be defined as video/jpeg, video/mjpeg,
image/jpeg, video/avi with a compression of (fourcc) MJPG, etc. None of
these is really official, since there isn't an official mimetype for
encoded MJPEG video.
The main focus of this document is to propose a standardized set of
mimetypes and properties that will be used by the GStreamer plugins.
3) Different types of streams
-----------------------------
There are several types of media streams. The most important distinction
will be container formats, audio codecs and video codecs. Container
formats are bytestreams that contain one or more substreams inside it,
and don't provide any direct media data itself. Examples are Quicktime,
AVI or MPEG (bytestream). They mostly contain of a set of headers that
define the media stream(s) that is packed inside the container and the
media data itself.
Video codecs and audio codecs describe encoded audio or video data.
Examples are MPEG-1 video, DivX video, MPEG-1 layer 3 (MP3) audio or
Ogg/Vorbis audio. Actually, Ogg is a container format too (for Vorbis
audio), but these are usually used in conjunction with each other.
3a) Container formats
1 - AVI (Microsoft RIFF/AVI)
mimetype: video/avi
2 - Quicktime (Apple)
mimetype: video/quicktime
3 - MPEG (MPEG LA)
mimetype: video/mpeg
properties: 'systemstream' = TRUE (BOOLEAN)
4 - ASF (Microsoft)
mimetype: video/x-ms-asf
5 - WAV (PCM)
mimetype: audio/x-wav
6 - RealMedia (Real)
mimetype: audio/x-pn-realaudio
7 - DV (Digital Video)
mimetype: video/dv
properties: 'systemstream' = TRUE (BOOLEAN)
8 - Ogg (Xiph)
mimetype: application/x-ogg
9 - Matroska
mimetype: video/x-mkv
10 - Shockwave (Macromedia)
mimetype: application/x-shockwave-flash
Please note that we try to keep these mimetypes as similar as possible
to what's used as standard mimetypes in Gnome (Gnome-VFS/Nautilus) and
KDE (Konqueror).
3b) Video codecs
For convenience, the fourcc codes used in the AVI container format will be
listed along with the mimetype and optional properties.
Preface - (optional) properties for all video formats:
'width' = X (INT)
'height' = X (INT)
'pixel_width' and 'pixel_height' = X (2xINT, together aspect ratio)
1 - Raw Video (YUV/YCbCr)
mimetype: video/x-raw-yuv
properties: 'format' = 'XXXX' (fourcc)
known fourccs: YUY2, I420, Y41P, YVYU, UYVY, etc.
Note: some raw video formats have implicit alignment rules. We should
discuss this more.
Note: some formats have multiple fourccs (e.g. IYUV/I420 or YUY2/YUYV).
For each of these, we only use one (e.g. I420 and YUY2).
Currently recognized formats:
YUY2: packed, Y-U-Y-V order, U/V hor 2x subsampled (YUV-4:2:2, 16 bpp)
YVYU: packed, Y-V-Y-U order, U/V hor 2x subsampled (YUV-4:2:2, 16 bpp)
UYVY: packed, U-Y-V-Y order, U/V hor 2x subsampled (YUV-4:2:2, 16 bpp)
Y42B: planar, Y-U-V order, U/V hor 2x subsampled (YUV-4:2:2, 16 bpp)
YV12: planar, Y-V-U order, U/V hor+ver 2x subsampled (YUV-4:2:0, 12 bpp)
I420: planar, Y-U-V order, U/V hor+ver 2x subsampled (YUV-4:2:0, 12 bpp)
Y41B: planar, Y-U-V order, U/V hor 4x subsampled (YUV-4:1:1, 12bpp)
YUV9: planar, Y-U-V order, U/V hor+ver 4x subsampled (YUV-4:1:0, 9bpp)
YVU9: planar, Y-V-U order, U/V hor+ver 4x subsampled (YUV-4:1:0, 9bpp)
See http://www.fourcc.org/ for more information.
Note: YUV-4:4:4 (both planar and packed, in multiple orders) are missing.
2 - Raw Video (RGB)
mimetype: video/x-raw-rgb
properties: 'endianness' = 1234/4321 (INT) <- endianness
'depth' = 15/16/24 (INT) <- bits per pixel (depth)
'bpp' = 16/24/32 (INT) <- bits per pixel (in memory)
'red_mask' = bitmask (0x..) (INT) <- red pixel mask
'green_mask' = bitmask (0x..) (INT) <- green pixel mask
'blue_mask' = bitmask (0x..) (INT) <- blue pixel mask
'bpp' is the number of bits of memory used for each pixel. 'depth'
is the color depth.
24 and 32 bit RGB should always be specified as big endian, since
any little endian format can be transformed into big endian by
rearranging the color masks. 15 and 16 bit formats should generally
have the same byte order as the cpu.
Color masks are interpreted by loading 'bpp' number of bits using
'endianness' rule, and masking and shifting by each color mask.
Loading a 24-bit value cannot be done directly, but one can perform
an equivalent operation.
Examples:
msb .. lsb
- memory: RRRRRRRR GGGGGGGG BBBBBBBB RRRRRRRR GGGGGGGG ...
'bpp' = 24
'depth' = 24
'endianness' = 4321 (G_BIG_ENDIAN)
'red_mask' = 0xff0000
'green_mask' = 0x00ff00
'blue_mask' = 0x0000ff
- memory: xRRRRRGG GGGBBBBB xRRRRRGG GGGBBBBB xRRRRRGG ...
'bpp' = 16
'depth' = 15
'endianness' = 4321 (G_BIG_ENDIAN)
'red_mask' = 0x7c00
'green_mask' = 0x03e0
'blue_mask' = 0x003f
- memory: GGGBBBBB xRRRRRGG GGGBBBBB xRRRRRGG GGGBBBBB ...
'bpp' = 16
'depth' = 15
'endianness' = 1234 (G_LITTLE_ENDIAN)
'red_mask' = 0x7c00
'green_mask' = 0x03e0
'blue_mask' = 0x003f
3 - MPEG-1, -2 and -4 video (ISO/LA MPEG)
mimetype: video/mpeg
properties: 'systemstream' = FALSE (BOOLEAN)
'mpegversion' = 1/2/4 (INT)
known fourccs: MPEG, MPGI
4 - DivX 3.x, 4.x and 5.x video
mimetype: video/divx
optional properties: 'divxversion' = 3/4/5 (INT)
known fourccs: DIV3, DIV4, DIV5, DIVX, DX50, DIVX, divx
5 - Microsoft MPEG 4.1, 4.2 and 4.3
mimetype: video/x-msmpeg
optional properties: 'mpegversion' = 41/42/43 (INT)
known fourccs: MPG4, MP42, MP43
6 - Motion-JPEG (official and extended)
mimetype: video/jpeg
known fourccs: MJPG (YUY2 MJPEG), JPEG (any), PIXL (Pinnacle/Miro), VIXL
7 - Sorensen (Quicktime - SVQ1/SVQ3)
mimetypes: video/x-svq
properties: 'svqversion' = 1/3 (INT)
8 - H263 and related codecs
mimetype: video/h263
known fourccs: H263, i263, M263, x263, VDOW, VIVO
9 - RealVideo (Real)
mimetype: video/realvideo
10 - Digital Video (DV)
mimetype: video/dv
properties: 'systemstream' = FALSE (BOOLEAN)
known fourccs: DVSD, dvsd
11 - Windows Media Video 1 and 2 (WMV)
mimetype: video/wmv
properties: 'wmvversion' = 1/2 (INT)
12 - XviD
mimetype: video/xvid
known fourccs: xvid, XVID
13 - 3IVX
mimetype: video/3ivx
known fourccs: 3IV0, 3IV1, 3IV2
14 - Ogg/Tarkin (Xiph)
mimetype: video/x-tarkin
15 - VP3
mimetype: video/vp3
16 - Ogg/Theora (Xiph, VP3-like)
mimetype: video/x-theora
17 - Huffyuv
mimetype: video/x-huffyuv
known fourccs: HFYU
18 - FF Video 1
mimetype: video/x-ffv
properties: 'ffvversion' = 1 (INT)
TODO: colorspace identifications for MJPEG? How? Move MPEG4-compatible codecs
over to video/mpeg with a subtype (xvid, divx3, divx4, divx5, iso, ...)?
interlacing?
3c) Audio Codecs
for convenience, the two-byte hexcodes (as are being used for identification
in AVI files) are also given
Preface - (optional) properties for all audio formats:
'rate' = X (int) <- sampling rate
'channels' = X (int) <- number of audio channels
1 - Raw Audio (integer format)
mimetype: audio/x-raw-int
properties: 'width' = X (INT) <- memory bits per sample
'depth' = X (INT) <- used bits per sample
'signedness' = X (BOOLEAN)
'endianness' = 1234/4321 <- endianness of audio stream
2 - Raw Audio (floating point format)
mimetype: audio/x-raw-float
properties: 'layout' = "gdouble" and "gfloat" (STRING)
'intercept' = X (FLOAT)
'slope' = X (FLOAT)
3 - Alaw Raw Audio
mimetype: audio/x-alaw
4 - Mulaw Raw Audio
mimetype: audio/x-mulaw
5 - MPEG-1 layer 1/2/3 audio
mimetype: audio/x-mp3
properties: 'layer' = 1/2/3 (INT)
6 - Ogg/Vorbis
mimetype: audio/vorbis
7 - Windows Media Audio 1 and 2 (WMA)
mimetype: audio/wma
properties: 'wmaversion' = 1/2 (INT)
8 - AC3
mimetype: audio/a52
9 - FLAC (Free Lossless Audio Codec)
mimetype: audio/x-flac
10 - MACE 3/6 (Quicktime audio)
mimetype: audio/x-mace
properties: 'maceversion' = 3/6 (INT)
11 - MPEG-4 AAC
mimetype: audio/x-mpeg-aac
properties: 'mpegversion' = 4 (INT)
'systemstream' = FALSE (BOOLEAN)
12 - IMA ADPCM Quicktime
mimetype: audio/x-ima-adpcm-quicktime
13 - IMA ADPCM WAV
mimetype: audio/x-ima-adpcm-wav
14 - ADPCM MS (Microsoft?)
mimetype: audio/x-adpcm-ms
4 - Status of this document
---------------------------
Not all plugins strictly follow these guidelines yet, but these are the
official types. Plugins not following these specs either use extensions
that should be documented, or are buggy (and should be fixed).
Blame Ronald Bultje <rbultje@ronald.bitfreak.net> aka BBB for any mistakes
in this document.
Reference in a new issue View git blame Copy permalink