Video Acceleration ------------------ Status: DRAFT. Purpose: Provide an standarized generic way to introduce Video Acceleration APIs in already available elements instead of duplicating those into specialized ones. Provide a mechanism for a light GstBuffer subclassing in order to be able exchange VA related context and surfaces between elements. Provide some basic primitives to be used in the elements keeping the implementation of those in the backends of the helper library. The motivation of this proposal is: - to avoid multiple duplicty of code - to avoid the use of GstBuffer subclassing and be more close to GstBuffer2 - to avoid the overhead that's introduced with subclassing/GType checks - to permit multiple elements interact having a common standard Proposal: video/x-raw-va Light subclassing embeding an structure in the data field of a standard GstBuffer. struct { context; surface; flags; subpicture, x, y, w, h; parent * ... } VASurface GstVA helper library -------------------- Common API that it's implemented by a backend for each VA API. +-------+ +---------------+ | |----> | VDPAU backend | | GstVA | +---------------+ | | +---------------+ | |----> | VAAPI backend | +-------+ +---------------+ gst_va_init () : Iterate in the backends and try to initialize those, stops when a backend is able to be created. bool gst_va_available() : permit to query if there's VA available in runtime. GstBuffer * gst_va_buffer_new (vacontext, vasurface, vaflags, parent*, free_func*): Create a new GstBuffer with the proper light subclass and store on it the provided context, surface, flags... - parent* and free_func* to implement a mechanism to return surfaces in the pool driven by parent. gst_va_buffer_get_context (GstBuffer *) gst_va_buffer_get_surface (GstBuffer *) gst_va_buffer_get_flags (GstBuffer *) .... Some public/private? accessors to be able recover fields from a VA GstBuffer internal structure. Primitives: gst_va_render_display (GstBuffer * va, display, x, y, w, h, ...) Put a surface in the screen at the specified position. gst_va_render_pixmap (GstBuffer * va, pixmap *, w, h, ...) Put a surface in a pixmap, to be used paired with GLX_texture_from_pixmap to upload into a OpenGL texture. gst_va_render_memory (GstBuffer * va, planes *, strides *, w, h, ...) To pull back into main memory a VA surface. gst_va_buffer_associate_subpicture (GstBuffer * va, GstBuffer * subpicture, x, y, w, h,...) Take a reference of subpicture and keep it in va internal structure. Use cases: Instead of create a new specialized element for each API just change a generic element once. To add support for a new API just have to create a new backend. xvimagesink ----------- - In _setcaps negotiate accept VA kind of stream when gst_va_init and gst_va_available tell us that we have hardware capable of it. - In _render when the buffers have the VA flag render display it with gst_va_render_display() videomixer, dvdspu, textoverlay, ... ------------------------------------ - In _setcaps negotiate accept VA kind of stream when gst_va_init and gst_va_available tell us that we have hardware capable of it. - Use gst_va_buffer_associate_subpicture () where necessary.