docs: add draft for generic introduction of video acceleration APIs idea

2024-12-18 14:26:43 +00:00 · 2009-08-07 13:05:42 +02:00 · 2009-08-07 13:05:42 +02:00 · a80c81e3ab
commit a80c81e3ab
parent 6a5af299b0
1 changed files with 111 additions and 0 deletions
--- a/docs/design/draft-va.txt
+++ b/docs/design/draft-va.txt
@ -0,0 +1,111 @@
+Video Acceleration
+------------------
+
+Status:
+
+  DRAFT.
+  
+Purpose:
+
+  Provide an standarized generic way to introduce Video Acceleration APIs in
+  already available elements instead of duplicating those into specialized ones.
+  
+  Provide a mechanism for a light GstBuffer subclassing in order to be able
+  exchange VA related context and surfaces between elements.
+  
+  Provide some basic primitives to be used in the elements keeping
+  the implementation of those in the backends of the helper library.
+  
+  The motivation of this proposal is: 
+    - to avoid multiple duplicty of code
+    - to avoid the use of GstBuffer subclassing and be more close to GstBuffer2
+    - to avoid the overhead that's introduced with subclassing/GType checks 
+    - to permit multiple elements interact having a common standard
+
+Proposal:
+
+   video/x-raw-va
+   
+   Light subclassing embeding an structure in the data field of a standard
+   GstBuffer.
+   
+   struct {
+     context;
+     surface;
+     flags;
+     subpicture, x, y, w, h;
+     parent *
+     ...
+   } VASurface 
+        
+   GstVA helper library
+   --------------------
+   
+   Common API that it's implemented by a backend for each VA API. 
+   
+   +-------+      +---------------+
+   |       |----> | VDPAU backend |
+   | GstVA |      +---------------+
+   |       |      +---------------+
+   |       |----> | VAAPI backend |
+   +-------+      +---------------+
+   
+   gst_va_init () : Iterate in the backends and try to initialize those, stops
+   when a backend is able to be created.
+   
+   bool gst_va_available() : permit to query if there's VA available in runtime.
+   
+   GstBuffer * gst_va_buffer_new (vacontext, vasurface, vaflags, parent*, 
+     free_func*):
+   Create a new GstBuffer with the proper light subclass and store on it
+   the provided context, surface, flags...
+   - parent* and free_func* to implement a mechanism to return surfaces in 
+     the pool driven by parent.
+   
+   gst_va_buffer_get_context (GstBuffer *)
+   gst_va_buffer_get_surface (GstBuffer *)
+   gst_va_buffer_get_flags (GstBuffer *)
+   ....
+   Some public/private? accessors to be able recover fields from a VA GstBuffer
+   internal structure.
+   
+   Primitives:
+   
+   gst_va_render_display (GstBuffer * va, display, x, y, w, h, ...)
+   Put a surface in the screen at the specified position.
+
+   gst_va_render_pixmap (GstBuffer * va, pixmap *, w, h, ...)
+   Put a surface in a pixmap, to be used paired with GLX_texture_from_pixmap to
+   upload into a OpenGL texture.
+   
+   gst_va_render_memory (GstBuffer * va, planes *, strides *, w, h, ...) 
+   To pull back into main memory a VA surface.
+   
+   gst_va_buffer_associate_subpicture (GstBuffer * va, GstBuffer * subpicture,
+      x, y, w, h,...)
+   Take a reference of subpicture and keep it in va internal structure.
+  
+Use cases:
+
+   Instead of create a new specialized element for each API just change
+   a generic element once.
+   
+   To add support for a new API just have to create a new backend.
+
+   xvimagesink
+   -----------
+      
+   - In _setcaps negotiate accept VA kind of stream when gst_va_init and
+     gst_va_available tell us that we have hardware capable of it.
+   
+   - In _render when the buffers have the VA flag render display it with
+     gst_va_render_display()
+     
+   videomixer, dvdspu, textoverlay, ...
+   ------------------------------------
+
+   - In _setcaps negotiate accept VA kind of stream when gst_va_init and
+     gst_va_available tell us that we have hardware capable of it.
+   
+   - Use gst_va_buffer_associate_subpicture () where necessary.
+