gstreamer/gst/audiovisualizers
Edward Hervey f70a623418 Merge remote-tracking branch 'origin/master' into 0.11-premerge
Conflicts:
	docs/libs/Makefile.am
	ext/kate/gstkatetiger.c
	ext/opus/gstopusdec.c
	ext/xvid/gstxvidenc.c
	gst-libs/gst/basecamerabinsrc/Makefile.am
	gst-libs/gst/basecamerabinsrc/gstbasecamerasrc.c
	gst-libs/gst/basecamerabinsrc/gstbasecamerasrc.h
	gst-libs/gst/video/gstbasevideocodec.c
	gst-libs/gst/video/gstbasevideocodec.h
	gst-libs/gst/video/gstbasevideodecoder.c
	gst-libs/gst/video/gstbasevideoencoder.c
	gst/asfmux/gstasfmux.c
	gst/audiovisualizers/gstwavescope.c
	gst/camerabin2/gstcamerabin2.c
	gst/debugutils/gstcompare.c
	gst/frei0r/gstfrei0rmixer.c
	gst/mpegpsmux/mpegpsmux.c
	gst/mpegtsmux/mpegtsmux.c
	gst/mxf/mxfmux.c
	gst/videomeasure/gstvideomeasure_ssim.c
	gst/videoparsers/gsth264parse.c
	gst/videoparsers/gstmpeg4videoparse.c
2011-12-30 11:41:17 +01:00
..
gstbaseaudiovisualizer.c add parent to pad functions 2011-11-17 15:53:42 +01:00
gstbaseaudiovisualizer.h audiovisualizer: port to 0.11 2011-11-10 23:20:32 +02:00
gstdrawhelpers.h spacescope: add color-modes 2011-11-30 15:11:50 +01:00
gstspacescope.c Merge remote-tracking branch 'origin/master' into 0.11-premerge 2011-12-30 11:41:17 +01:00
gstspacescope.h spacespace: whitespace cleanups 2011-11-30 22:12:27 +01:00
gstspectrascope.c various: fix pad template ref leaks 2011-11-28 13:08:27 +00:00
gstspectrascope.h audiovisualizers: rename baseclass from basescope to baseaudiovisualizer 2011-06-06 15:25:14 +03:00
gstsynaescope.c various: fix pad template ref leaks 2011-11-28 13:08:27 +00:00
gstsynaescope.h audiovisualizers: rename baseclass from basescope to baseaudiovisualizer 2011-06-06 15:25:14 +03:00
gstwavescope.c Merge remote-tracking branch 'origin/master' into 0.11-premerge 2011-12-30 11:41:17 +01:00
gstwavescope.h wavescape: also add colormodes like in spacescope 2011-11-30 22:11:40 +01:00
Makefile.am Merge branch 'master' into 0.11 2011-11-23 11:08:39 +01:00
plugin.c controller: port to new controller location and api 2011-11-04 20:14:01 +01:00
README audiovisualizer: update README 2011-11-30 15:13:04 +01:00

A basclass for audiovisualizers. Takes care of re-fitting the audio-rate to
video-rate. It receives audio-data at the sampling-rate. It needs to render
video-frames at frame-rate. The rendering needs n audio samples (depends on
subclass). The baseclass takes care of that.

Some effects could be enhanced by running geometrictransform/effecttc elements
afterwards.

= Feedback =
* put 'Audio' to klass as well ?

= API =

* we have a couple of drawing helpers in gstdrawhelpers.h
  (would be nice if we could use cairo) 
  draw_point (x,y,color);
  draw_line (x1,x2,y1,y2,color);
* some more we could add:
  draw_hline (x1,x2,y,color);
  draw_vline (x,y1,y2,color);
  draw_rect (x1,x2,y1,y2,color);
  draw_box (x1,x2,y1,y2,color); // filled
* shading effects
  - would be nice to use a generic 3x3 matrix operation, we don't run inplace
    anyway
  - this way we could also blur the background

= Elements to port =
gst-plugin-ugly/gst/synaestesia -> synaescope
gst-plugin-bad/gst/smoothwave -> wavescope
gst-plugin-good/gst/monoscope -> blend into what we have in wavescope

= Elements to add =
spectrascope - done
spacescope - stereo wavescope
- left->x, right->y - done
- polar mapping
wavescope
- we could have a bouncing line as a base, like a quix:
  - two dots moving on a linear path and getting a new random dx,dy when hitting
    a border
  - the abs(dx/dy) - speed of movement - could be scaled by the sound level
  - we would need to rotate, stretch and clip the waveform drawing to fit the
    line
  - we could scratch the rotate part and just stretch/squeeze x and shift/clip y

xxxscope
- have a matrix of source and drawing-functions
  - sources: audio, spectrum, audio-low, audio-mid, audio-hi
  - drawing: waves (style, color), space (style,color)
- have the usual shade and move operations
- have a way to draw each operator in one or more color-channels
- we could calculate the sound-level (like in level element) and modulate
  colors/movements
  - for filtered low/mid/hi audio we could use different peak-falloffs

= TODO =
- element maker template
- test for baseclass

- we probably want a VisBin like the gnome video effects
- this way we can specify pipeline fragments
- VisBin can use a videomixer to switch effects based on time or song
- VisBin can e.g. control a text-overlay to render the title into the 
  visualisation for a while

= Test it =

GST_DEBUG="*:2,*scope*:4"

GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-inspect scopes

GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch audiotestsrc ! audioconvert ! wavescope ! colorspace ! ximagesink
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! audioconvert ! wavescope ! colorspace ! ximagesink
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! audioconvert ! wavescope style=lines shade-amount=0x00080402 ! edgetv ! vertigotv ! ximagesink

GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! audioconvert ! spacescope style=lines shade-amount=0x00080402 ! ximagesink
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! audioconvert ! spacescope style=lines shade-amount=0x00080402 ! vertigotv ! ximagesink

GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! audioconvert ! spectrascope ! colorspace ! ximagesink
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! audioconvert ! spectrascope shader=fade-and-move-up shade-amount=0x00040302 ! colorspace ! ximagesink

GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! tee name=t ! queue ! audioconvert ! synaesthesia ! ximagesink t. ! queue ! synaescope shade-amount=0x00040404 ! colorspace ! ximagesink