mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-11-03 16:09:39 +00:00
102 lines
4.8 KiB
Text
102 lines
4.8 KiB
Text
A basclass for audiovisualizers. Takes care of re-fitting the audio-rate to
|
|
video-rate. It receives audio-data at the sampling-rate. It needs to render
|
|
video-frames at frame-rate. The rendering needs n audio samples (depends on
|
|
subclass). The baseclass takes care of that.
|
|
|
|
Some effects could be enhanced by running geometrictransform/effecttc elements
|
|
afterwards.
|
|
|
|
= Feedback =
|
|
* put 'Audio' to klass as well ?
|
|
|
|
= API =
|
|
|
|
* we have a couple of drawing helpers in gstdrawhelpers.h
|
|
(would be nice if we could use cairo)
|
|
draw_point (x,y,color);
|
|
draw_line (x1,x2,y1,y2,color);
|
|
* some more we could add:
|
|
draw_hline (x1,x2,y,color);
|
|
draw_vline (x,y1,y2,color);
|
|
draw_rect (x1,x2,y1,y2,color);
|
|
draw_box (x1,x2,y1,y2,color); // filled
|
|
* shading effects
|
|
- would be nice to use a generic 3x3 matrix operation, we don't run inplace
|
|
anyway
|
|
- this way we could also blur the background
|
|
- we need to handle visualizer that don't draw with alpha:
|
|
- add API to set can_mix and if so enable shaders
|
|
- ev. do mixing ourself (needs extra buffer)
|
|
|
|
= Elements to port =
|
|
gst-plugins-ugly/gst/synaestesia -> synaescope
|
|
gst-plugins-bad/gst/smoothwave -> wavescope
|
|
gst-plugins-good/gst/monoscope -> blend into what we have in wavescope
|
|
gst-plugins-base/ext/libvisual - done
|
|
|
|
= Elements to add =
|
|
spectrascope - done
|
|
spacescope - stereo wavescope
|
|
- left->x, right->y - done
|
|
- polar mapping
|
|
wavescope
|
|
- we could have a bouncing line as a base, like a quix:
|
|
- two dots moving on a linear path and getting a new random dx,dy when hitting
|
|
a border
|
|
- the abs(dx/dy) - speed of movement - could be scaled by the sound level
|
|
- we would need to rotate, stretch and clip the waveform drawing to fit the
|
|
line
|
|
- we could scratch the rotate part and just stretch/squeeze x and shift/clip y
|
|
|
|
xxxscope
|
|
- have a matrix of source and drawing-functions
|
|
- sources: audio, spectrum, audio-low, audio-mid, audio-hi
|
|
- drawing: waves (style, color), space (style,color)
|
|
- have the usual shade and move operations
|
|
- have a way to draw each operator in one or more color-channels
|
|
- we could calculate the sound-level (like in level element) and modulate
|
|
colors/movements
|
|
- for filtered low/mid/hi audio we could use different peak-falloffs
|
|
|
|
= TODO =
|
|
- element maker template
|
|
- test for baseclass
|
|
|
|
- actors
|
|
- we use the wave, filtered waves, balance and fft so far
|
|
- we could have narrow filters over harmonic frequencies
|
|
- we could use loudness like determined in level-meter
|
|
|
|
- we probably want a VisBin like the gnome video effects
|
|
- this way we can specify pipeline fragments
|
|
- VisBin can use a videomixer to switch effects based on time or song
|
|
- VisBin can e.g. control a text-overlay to render the title into the
|
|
visualisation for a while
|
|
|
|
= Test it =
|
|
|
|
GST_DEBUG="*:2,*scope*:4"
|
|
|
|
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-inspect scopes
|
|
|
|
== 0.10 ==
|
|
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch audiotestsrc ! audioconvert ! wavescope ! colorspace ! ximagesink
|
|
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! audioconvert ! wavescope ! colorspace ! ximagesink
|
|
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! audioconvert ! wavescope style=lines shade-amount=0x00080402 ! edgetv ! vertigotv ! ximagesink
|
|
|
|
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! audioconvert ! spacescope style=lines shade-amount=0x00080402 ! ximagesink
|
|
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! audioconvert ! spacescope style=lines shade-amount=0x00080402 ! vertigotv ! ximagesink
|
|
|
|
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! audioconvert ! spectrascope ! colorspace ! ximagesink
|
|
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! audioconvert ! spectrascope shader=fade-and-move-up shade-amount=0x00040302 ! colorspace ! ximagesink
|
|
|
|
GST_PLUGIN_PATH=$GST_PLUGIN_PATH:$PWD gst-launch filesrc location=$HOME/Music/1.mp3 ! decodebin2 ! tee name=t ! queue ! audioconvert ! synaesthesia ! ximagesink t. ! queue ! synaescope shade-amount=0x00040404 ! colorspace ! ximagesink
|
|
|
|
== 0.11/1.0 ==
|
|
gst-launch-1.0 audiotestsrc ! audioconvert ! wavescope ! videoconvert ! ximagesink
|
|
|
|
gst-launch-1.0 filesrc location=$HOME/Music/1.mp3 ! decodebin ! audioconvert ! spectrascope ! videoconvert ! ximagesink
|
|
|
|
gst-launch-1.0 filesrc location=$HOME/Music/1.mp3 ! decodebin ! tee name=t ! queue ! audioconvert ! wavescope style=color-lines shade-amount=0x00080402 ! alpha alpha=0.5 ! videomixer name=m background=black ! videoconvert ! vertigotv ! ximagesink t. ! queue ! audioconvert ! spacescope style=color-lines shade-amount=0x00080402 ! alpha alpha=0.5 ! m. t. ! queue ! autoaudiosink
|
|
|
|
|