mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2025-03-28 11:55:39 +00:00
Code Issues 1 Projects Releases Wiki Activity

todo: update planning and ideas

Browse source
This commit is contained in:
Stefan Sauer 2013-08-29 07:51:13 +02:00
parent 2a322ab168
commit 94607ec2d6
2 changed files with 196 additions and 205 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

213
mediainfo/README
View file

@ -2,210 +2,13 @@
The package contains a UI for browsing media files and showing details of the
files.
= Browsing =
- in the preview hook of the file browser we run discover and update the details
- if one has grillo installed we could use grillo sources in addition to local files
- there should be a "open url" menu entry in addition
= Usage =
The left side of the window has a file-browser. This has the keyborad focus.
Going through the files will show the media details immediately on the right
side. When staying on a file for a moment, the file will be played.
= Details =
- show container, video-tracks, audio-track, subtitle-trackes, metadata and
parsing messages
- auto-play after updating the details
have a menu command to turn auto-play off
- offer several commands in addition (see below)
example output from commandline tool:
> gst-discoverer -v /home/ensonic/Musik/xotox-hypnocat.mp3
Topology:
audio: audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)3, rate=(int)44100, channels=(int)2, parsed=(boolean)true
Codec:
audio/mpeg, mpegversion=(int)1, mpegaudioversion=(int)1, layer=(int)3, rate=(int)44100, channels=(int)2, parsed=(boolean)true
Additional info:
None
Channels: 2
Sample rate: 44100
Depth: 16
Bitrate: 128000
Max bitrate: 0
Tags:
taglist, audio-codec=(string)"MPEG\ 1\ Audio\,\ Layer\ 3\ \(MP3\)", has-crc=(boolean)false, channel-mode=(string)stereo, bitrate=(uint)128000;
Properties:
Duration: 0:04:00.039125000
Seekable: yes
== compare ==
- dup detail pane and allow to pick a new file
- show differing entries in red
- add a narrow vertical bar between the two details panes and mark blocks with a
delta red
- diffing one (broken) file against several known working files would be nice
- it would build ranges for the group of files (e.g. for the video width)
- it would highlight fields that the are different to any of the working files
== analyse ==
- play the file by using fakesinks and gather statistics:
- bitrate profiles for each a/mediainfo/v track
- media specific:
- audio: volume profile
- video: contrast profile
- show disconts as vertical bars
== structural view/hex view ==
- make parsing elements post structural messages (if enabled via property)
- message would contain:
- stream offset in bytes
- block identifier (e.g. fourcc)
- human readable description (if available)
- flags (e.g. if the block is used or skipped)
- indentation depth
- we would need a way to indicate that e.g. h264parse would be indented below
the containing container block
- a structured hex view would be nice
(somehow align the parse tree with the hexdump)
== unsorted ==
- show named audio channel configurations instead only numbers
e.g. "mono", "stereo", "5.1"
- handle chapters (once the patches are merged)
- tag lists
- if there is a "language-code" in the tags (or subtitles?) use flag icons
- deluge installs some under: /usr/share/pyshared/deluge/data/mediainfo/pixmaps/flags/
- famfamfam-flag-png: locale/usr/share/flags/countries/
- geo-tags: map-widget?, link to google-maps?
- artist: links to {last.fm,wikipedia}
- format dates nicely
- show level meters for audio next or below video drawable
== deep scan mode ==
- update fields when playing
- listen for duration messages on the bus
- get bit-rate over time
- specify window size, get min,max,avg bitrate for each window
- gst-mi can draw them as a graph
- get key-frame statistics (using gst-index)
- number of keyframes
- min,max,avg keyframe interval
- disconts
- raw data statistics
- audio: level, ...
- video: histogram, ...
= TODO for gstreamer =
- file/stream layout
- from every element we'd like to know what data is processes, what is pushed
further and some metadata about it:
layout {
gsize offset; // in bytes
gsize length; // in bytes
gboolean known; // or an enum: handled, skipped, unknown
gchar *name; // e.g. atom/chunk name
gchar *description; // long description or NULL
enum block_type type; // meta, audio, video, text, ...
};
- offset is not neccesarily easy to determine for later elements, not sure
if we can make it relative
- elements could emit messages with this info
- need a common way to enable it ("post-stream-layout" property)
- we would need a cairo custom widget to draw a table
- one row per element
- each row contains colored segments
= TODO for discoverer =
- add deep-scan mode (see above)
- add a mode property: quick-scan, deep-scan
(or just a boolean for deep-scan that is false by default)
- gst-discoverer will use "-d" for deep-scan/details
(-a for anayze or -s for scan are used otherwise already)
- in deep-scan, don't stop on handle_message::GST_MESSAGE_ASYNC_DONE
- in deep-scan we need pad-probes for encoded data pads (to get bitrates)
- uridecodebin_pad_added_cb() has raw pads :/
- look at uridecodebin_element_added_cb()
- the pads on uri-decodebin are sufficient to get keyframes and disconts
- qtdemux could return stsz table to get bitrate profile
http://wiki.multimedia.cx/index.php?title=QuickTime_container#stsz
- get duration per stream
- this would need individual queries on the demuxer src pads
- or duration as part of per stream tags
- errors/warnings about files/stream processing
- we'd like to know about fixable, unfixable issues in the file/stream
- many elements do this already (178 uses in 89 files)
- the pipeline and thus the bus is internal to discoverer, so it would be nice
if it could gather the messages and offer them
- having them globaly should be enough
- CBR/VBR type for each stream
- can we derive that from the bitrate tags (no min/max bitrate set)?
find . -name "*.c" -exec egrep -Hn "GST_TAG[A-Z_]*_BITRATE" {} \;
- formats
- all raw files are cbr
- some codecs are cbr only (some speech codecs)
- audio
- in an MP3 file, if there is no VBRI or Xing,
it probably is not a vbr stream
-
- lossy/lossless compression
- all raw formats are lossless (well rgb->yuv is lossy, but ...)
- some audio formats are lossless (flac, wavpack, ...)
- some video formats can be lossless (dirac,
= discoverer workflow =
== sync quick-scan ==
app disco
--- -----
| discover_uri() |
|--------------------->|
| done |
|<---------------------|
|
== async quick-scan ==
app disco
--- -----
| discover_uri_async() |
|--------------------->|
| ... |
| discover_uri_async() |
|--------------------->|
| start() |
|--------------------->|
| ::discovered |
|<---------------------|
| ... |
| ::discovered |
|<---------------------|
| ::finisheded |
|<---------------------|
|
== sync deep-scan ==
- same as "sync quick-scan", no intermediate result
== async deep-scan ==
- same as "async quick-scan", but each discovered signal is followed by an
"analyzed" signal with detailed information
- we could also just emit discovered twice.
app disco
--- -----
| discover_uri_async() |
|--------------------->|
| ... |
| discover_uri_async() |
|--------------------->|
| start() |
|--------------------->|
| ::discovered |
|<---------------------|
| ::analyzed |
|<---------------------|
| ... |
| ::discovered |
|<---------------------|
| ::analyzed |
|<---------------------|
| ::finisheded |
|<---------------------|
|
The information shown on the left side is enhanced for human readability. E.g.
codec and fomat names are turned into wikipedia links.
The UI has a compact mode for small screens (height < 600) where all streams
are in one notebook.

188
mediainfo/TODO
View file

@ -1,5 +1,193 @@
= Inspiration =
Simillar apps to take inspiration from:
http://www.fourcc.org/identifier/gspot_example.png
http://img.brothersoft.com/screenshots/softimage/g/gspot-190045-1.jpeg
http://www.headbands.com/gspot/v26x/index.htm
= browsing =
- if one has grillo installed we could use grillo sources in addition to local files
- there should be a "open url" menu entry in addition
- the tool should also accept a stream url when starting
- when a stream play we hide the file-browser pane
- we could also take playlists and use totem-pl-parser
- needs a list-view to show the playlist instead of the file-browser
= compare =
- dup detail pane and allow to pick a new file
- show differing entries in red
- add a narrow vertical bar between the two details panes and mark blocks with a
delta red
- diffing one (broken) file against several known working files would be nice
- it would build ranges for the group of files (e.g. for the video width)
- it would highlight fields that the are different to any of the working files
= analyse =
- play the file by using fakesinks and gather statistics:
- bitrate profiles for each a/mediainfo/v track
- media specific:
- audio: volume profile
- video: contrast profile
- show disconts as vertical bars
= structural view/hex view =
- make parsing elements post structural messages (if enabled via property)
- message would contain:
- stream offset in bytes
- block identifier (e.g. fourcc)
- human readable description (if available)
- flags (e.g. if the block is used or skipped)
- indentation depth
- we would need a way to indicate that e.g. h264parse would be indented below
the containing container block
- a structured hex view would be nice
(somehow align the parse tree with the hexdump)
= playback =
- have a menu command to turn auto-play off
- show level meters for audio next or below video drawable
- show a seek bar under the drawable / or seek by scrubbing on the drawable
= unsorted =
- handle subtitles
- show named audio channel configurations instead only numbers
e.g. "mono", "stereo", "5.1"
- handle chapters (toc)
- also use toc for seeking
- tag lists
- if there is a "language-code" in the tags (or subtitles?) use flag icons
- deluge installs some under: /usr/share/pyshared/deluge/data/mediainfo/pixmaps/flags/
- famfamfam-flag-png: locale/usr/share/flags/countries/
- geo-tags: map-widget?, link to google-maps?
- artist: links to {last.fm,wikipedia}
- format dates nicely
== deep scan mode ==
- update fields when playing
- listen for duration messages on the bus
- get bit-rate over time
- specify window size, get min,max,avg bitrate for each window
- gst-mi can draw them as a graph
- get key-frame statistics (using gst-index)
- number of keyframes
- min,max,avg keyframe interval
- disconts
- raw data statistics
- audio: level, ...
- video: histogram, ...
= TODO for gstreamer =
- file/stream layout
- from every element we'd like to know what data is processes, what is pushed
further and some metadata about it:
layout {
gsize offset; // in bytes
gsize length; // in bytes
gboolean known; // or an enum: handled, skipped, unknown
gchar *name; // e.g. atom/chunk name
gchar *description; // long description or NULL
enum block_type type; // meta, audio, video, text, ...
};
- offset is not neccesarily easy to determine for later elements, not sure
if we can make it relative
- elements could emit messages with this info
- need a common way to enable it ("post-stream-layout" property)
- we would need a cairo custom widget to draw a table
- one row per element
- each row contains colored segments
= TODO for discoverer =
- add deep-scan mode (see above)
- add a mode property: quick-scan, deep-scan
(or just a boolean for deep-scan that is false by default)
- gst-discoverer will use "-d" for deep-scan/details
(-a for anayze or -s for scan are used otherwise already)
- in deep-scan, don't stop on handle_message::GST_MESSAGE_ASYNC_DONE
- in deep-scan we need pad-probes for encoded data pads (to get bitrates)
- uridecodebin_pad_added_cb() has raw pads :/
- look at uridecodebin_element_added_cb()
- the pads on uri-decodebin are sufficient to get keyframes and disconts
- qtdemux could return stsz table to get bitrate profile
http://wiki.multimedia.cx/index.php?title=QuickTime_container#stsz
- get duration per stream
- this would need individual queries on the demuxer src pads
- or duration as part of per stream tags
- errors/warnings about files/stream processing
- we'd like to know about fixable, unfixable issues in the file/stream
- many elements do this already (178 uses in 89 files)
- the pipeline and thus the bus is internal to discoverer, so it would be nice
if it could gather the messages and offer them
- having them globaly should be enough
- CBR/VBR type for each stream
- can we derive that from the bitrate tags (no min/max bitrate set)?
find . -name "*.c" -exec egrep -Hn "GST_TAG[A-Z_]*_BITRATE" {} \;
- formats
- all raw files are cbr
- some codecs are cbr only (some speech codecs)
- audio
- in an MP3 file, if there is no VBRI or Xing,
it probably is not a vbr stream
-
- lossy/lossless compression
- all raw formats are lossless (well rgb->yuv is lossy, but ...)
- some audio formats are lossless (flac, wavpack, ...)
- some video formats can be lossless (dirac,
= discoverer workflow =
== sync quick-scan ==
app disco
--- -----
| discover_uri() |
|--------------------->|
| done |
|<---------------------|
|
== async quick-scan ==
app disco
--- -----
| discover_uri_async() |
|--------------------->|
| ... |
| discover_uri_async() |
|--------------------->|
| start() |
|--------------------->|
| ::discovered |
|<---------------------|
| ... |
| ::discovered |
|<---------------------|
| ::finisheded |
|<---------------------|
|
== sync deep-scan ==
- same as "sync quick-scan", no intermediate result
== async deep-scan ==
- same as "async quick-scan", but each discovered signal is followed by an
"analyzed" signal with detailed information
- we could also just emit discovered twice.
app disco
--- -----
| discover_uri_async() |
|--------------------->|
| ... |
| discover_uri_async() |
|--------------------->|
| start() |
|--------------------->|
| ::discovered |
|<---------------------|
| ::analyzed |
|<---------------------|
| ... |
| ::discovered |
|<---------------------|
| ::analyzed |
|<---------------------|
| ::finisheded |
|<---------------------|
|