mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-23 08:46:40 +00:00
93 lines
2.2 KiB
Text
93 lines
2.2 KiB
Text
This document contains some ideas that can be used to implement
|
|
interactivity in media content.
|
|
|
|
Possible application: DVD navigation, flash,...
|
|
|
|
Requirements
|
|
------------
|
|
|
|
- capture mouse clicks, mouse position, movement occuring on
|
|
a video display plugin
|
|
- transport these events to the interested plugins
|
|
- allow for automation (ie, the technique should work without
|
|
a video plugin too)
|
|
- the core doesn't care
|
|
|
|
Capturing events
|
|
----------------
|
|
|
|
- the videosink element captures mouse events
|
|
- event is encapsulated into a generic data structure
|
|
describing the event (need to define a caps?)
|
|
- event is signalled to the app?.
|
|
- event is sent upstream?
|
|
|
|
* videosink has to add something to the main_loop to
|
|
be able to grab events
|
|
* thread issues?
|
|
* does the app need to know about the events?
|
|
|
|
- app captures mouse events
|
|
- no idea if that's possible
|
|
- app sends events upstream
|
|
|
|
Sending events to plugins
|
|
-------------------------
|
|
|
|
- are sent upstream using the event methods
|
|
* more generic
|
|
* less app control
|
|
|
|
- are sent to the appropriate plugin by the app
|
|
* app needs to know what plugins are interested,
|
|
less generic.
|
|
* more app control
|
|
|
|
automation will always work, the app can construct navigation
|
|
events and insert them into the pipeline.
|
|
|
|
What about flushing to minimize latency?
|
|
|
|
|
|
Defining an event
|
|
-----------------
|
|
|
|
some ideas:
|
|
|
|
GST_CAPS_NEW (
|
|
"videosink_event",
|
|
"application/x-gst-navigation"
|
|
"type", "click",
|
|
"x_pos", 30,
|
|
"y_pos", 40
|
|
)
|
|
|
|
GST_CAPS_NEW (
|
|
"videosink_event",
|
|
"application/x-gst-navigation"
|
|
"type", "move",
|
|
"x_pos", 30,
|
|
"y_pos", 40
|
|
)
|
|
|
|
...
|
|
|
|
do we need a library for this?
|
|
|
|
do we use custom events and use the mime type to detect the
|
|
type? do we creat a GST_EVENT_NAVIGATION?
|
|
|
|
can we encapsulate all events into a GstCaps? I would think so
|
|
|
|
Random thoughts
|
|
---------------
|
|
|
|
- we're basically defining an event model, I don't think there is
|
|
anything wrong with that.
|
|
- how is our coordinate system going to work? do
|
|
we use normalized values, 0-1000000 (or floats)
|
|
or real pixel values? real pixel values require scalers to adjust
|
|
the values (I don't think I like that)
|
|
|
|
|
|
|