GstVideoContextInterface

GstVideoContextInterface — Interface to handle video library context

Synopsis

#define             GST_IS_VIDEO_CONTEXT                (obj)
                    GstVideoContext;
struct              GstVideoContextInterface;
GType               gst_video_context_iface_get_type    (void);
void                gst_video_context_set_context       (GstVideoContext *context,
                                                         const gchar *type,
                                                         const GValue *value);
void                gst_video_context_set_context_string
                                                        (GstVideoContext *context,
                                                         const gchar *type,
                                                         const gchar *value);
void                gst_video_context_set_context_pointer
                                                        (GstVideoContext *context,
                                                         const gchar *type,
                                                         gpointer value);
void                gst_video_context_set_context_object
                                                        (GstVideoContext *context,
                                                         const gchar *type,
                                                         GObject *value);
void                gst_video_context_prepare           (GstVideoContext *context,
                                                         const gchar **types);
gboolean            gst_video_context_message_parse_prepare
                                                        (GstMessage *message,
                                                         const gchar ***types,
                                                         GstVideoContext **ctx);
GstQuery *          gst_video_context_query_new         (const gchar **types);
gboolean            gst_video_context_run_query         (GstElement *element,
                                                         GstQuery *query);
const gchar **      gst_video_context_query_get_supported_types
                                                        (GstQuery *query);
void                gst_video_context_query_parse_value (GstQuery *query,
                                                         const gchar **type,
                                                         const GValue **value);
void                gst_video_context_query_set_value   (GstQuery *query,
                                                         const gchar *type,
                                                         GValue *value);
void                gst_video_context_query_set_string  (GstQuery *query,
                                                         const gchar *type,
                                                         const gchar *value);
void                gst_video_context_query_set_pointer (GstQuery *query,
                                                         const gchar *type,
                                                         gpointer value);
void                gst_video_context_query_set_object  (GstQuery *query,
                                                         const gchar *type,
                                                         GObject *value);

Description

The Video Context interface enable sharing video context (such as display name, X11 Display, VA-API Display, etc) between neighboor elements and the application.

Note

The GstVideoContext interface is unstable API and may change in future. One can define GST_USE_UNSTABLE_API to acknowledge and avoid this warning.

For Element

This interface shall be implement by group of elements that need to share a specific video context (like VDPAU, LibVA, OpenGL elements) or by video sink in order to let the application select appropriate display information (like the X11 display name) when those sink are auto-plugged.

Along with implementing the interface, elements will need to query neighboor elements or send message to the application when preparing the context (see gst_video_context_prepare()). They also need to reply to the neighboors element queries, so the context can be shared without the application help.

Elements that are guarantied to have both upstream and downstream neighboors element implementing the GstVideoContext (like the gloverlay element in gst-plugins-opengl) is not required to also implement the interface. Relying on neighboors query shall be sufficient (see gst_video_context_run_query()).

The query is an application query with a structure name set to "prepare-video-context" and an array of supported video context types set in the field named "types". This query shall be send downstream and upstream, iterating the pads in order to find neighboors regardless of a static (sink to src) or a dynamic (src to sink) activation. Element should used the helpers method gst_video_context_prepare() (or gst_video_context_run_query() if no GstVideoContext interface) to correctly execute the query . The result is set using the query helper functions, the structures fields name being "video-context-type" as string and "video-context" as a GValue.

If the query is not handled by any neighboor, the element should ask the application using the "prepare-video-context" message. The application may then use the interface to set the video context information. If no context was set, the element shall create one using default configuration. Elements with multiple src or sink pad shall implement proper locking to prevent the race of parallel queries being replied.

Well known video-context are: "x11-display-name" a string representing the X11 display to use, "x11-display" the X11 Display structure, "va-display", the VADisplay structure and more.

For Application

In the case there is no neighboor element with video context to share, the element will first turn toward the application, by sending a "prepare-video-context" message. This message is sent along with a list of supported display types. The application can optionally reply to this message by calling appropriate setter through the GstVideoContext interface. If the application supports more then one video context type, it should choose the first one to occure in the supported list. It's important to remember that the message is delivered from the streaming thread, and appropriate locking should be considered. If the application does not have a video context to share, the element will simply allocate one base on default settings. Usually, only applications using OpenGL base sink, or running on special X11 display need to share a video context.

Note

Applications sharing X11 Display structure should always initialize the X11 threading support using XInitThreads() as GStreamer will need to manipulate the display from a separeate threads.

Example using ClutterVideoGstVideoSink

This example is for user of ClutterGstVideoSink element, the ClutterGstPlayer object transparently handle this.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
#if CLUTTER_WINDOWING_X11
static GstBusSyncReply
on_sync_message (GstBus * bus, GstMessage * message, gpointer user_data)
{
  Display *display = user_data;
  GstVideoContext *context;
  const gchar **types;

  if (gst_video_context_message_parse_prepare (message, &types, &context)) {
    gint i;

    for (i = 0; types[i]; i++) {

      if (!strcmp(types[i], "x11-display")) {
        gst_video_context_set_context_pointer (context, "x11-display", display);
      }
      else if (!strcmp(types[i], "x11-display-name")) {
        gst_video_context_set_context_string (context, "x11-display-name",
            DisplayString (display));
      } else {
        continue;
      }

      gst_message_unref (message);
      return GST_BUS_DROP;
    }
  }

  return GST_BUS_PASS;
}
#endif

gint
main (gint argc, gchar **argv)
{
  GstBin *pipeline;
  GstBus *bus;

  ...

  bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));

  #if CLUTTER_WINDOWING_X11
  gst_bus_set_sync_handler (priv->bus, on_sync_message,
      clutter_x11_get_default_display ());
  #endif

  gst_object_unref (GST_OBJECT (priv->bus));

  ...
}

Details

GST_IS_VIDEO_CONTEXT()

#define GST_IS_VIDEO_CONTEXT(obj)           (GST_IMPLEMENTS_INTERFACE_CHECK_INSTANCE_TYPE ((obj), GST_TYPE_VIDEO_CONTEXT))

GstVideoContext

typedef struct _GstVideoContext GstVideoContext;

Opaque GstVideoContext data structure.


struct GstVideoContextInterface

struct GstVideoContextInterface {
  GTypeInterface parent;

  /* virtual functions */
  void (*set_context) (GstVideoContext * context,
                       const gchar * type,
                       const GValue * value);
};

GstVideoContextInterface interface.

GTypeInterface parent;

parent interface type.

set_context ()

vmethod to set video context.

gst_video_context_iface_get_type ()

GType               gst_video_context_iface_get_type    (void);

gst_video_context_set_context ()

void                gst_video_context_set_context       (GstVideoContext *context,
                                                         const gchar *type,
                                                         const GValue *value);

This is a wrapper for the set_context() virtual method. It is suggested to use one of the helpers to avoid having to manipulate GValue

context :

an element implementing GstVideoContext

type :

the type of display being set

value :

a GValue containing the context

gst_video_context_set_context_string ()

void                gst_video_context_set_context_string
                                                        (GstVideoContext *context,
                                                         const gchar *type,
                                                         const gchar *value);

This helper is commonly used for setting video context represented by a string like the X11 display name ("x11-display-name")/

context :

an element implementing GstVideoContext

type :

the type of display being set

string :

a string representing the video context

gst_video_context_set_context_pointer ()

void                gst_video_context_set_context_pointer
                                                        (GstVideoContext *context,
                                                         const gchar *type,
                                                         gpointer value);

This helper is used for setting video context using a pointer, typically to a structure like the X11 Display ("x11-display") or the VADisplay ("vaapi-display").

Note

Users of X11 Display should ensure that XInitThreads() was called before opening the display.

context :

an element implementing GstVideoContext

type :

the type of display being set

pointer :

a pointer to the video context

gst_video_context_set_context_object ()

void                gst_video_context_set_context_object
                                                        (GstVideoContext *context,
                                                         const gchar *type,
                                                         GObject *value);

This is for video context that are GObject, this helper allow taking benifit of the GObject refcounting. It is particularly handy for element to have refcounting as the order in which element will stop using the display is not defined.

context :

an element implementing GstVideoContext

type :

the type of display being set

object :

a GObject resenting the display

gst_video_context_prepare ()

void                gst_video_context_prepare           (GstVideoContext *context,
                                                         const gchar **types);

This method run "prepare-video-context" custom query dowstream, and upstream. If * the query has a reply, it sets the context value using gst_video_context_set_context(). Otherwise, it sends a "prepare-video-context" message to the application. The element can then continue video context initialization.

context :

an element implementing GstVideoContext interface

types :

an array of supported types, prefered first

gst_video_context_message_parse_prepare ()

gboolean            gst_video_context_message_parse_prepare
                                                        (GstMessage *message,
                                                         const gchar ***types,
                                                         GstVideoContext **ctx);

This helper shall be used by application to simply handling of the "prepare-video-context" message.

Rerturns: FALSE is the message was not valid "prepare-video-context" element message, otherwise TRUE with types and context set.

message :

a GstMessage

types :

return value for supported types

context :

return value for the element the implements GstVideoContext

gst_video_context_query_new ()

GstQuery *          gst_video_context_query_new         (const gchar **types);

Create a new custom GstQuery with structure name "prepare-video-context".

types :

a string array of video context types

gst_video_context_run_query ()

gboolean            gst_video_context_run_query         (GstElement *element,
                                                         GstQuery *query);

This helper runs the query on each downstream, then upstream pads in an element. This is called by gst_video_context_prepare(). This method is only used directly within elements that are required to have two neighboors elements with appropriate video context. This would be the case of specialized filters that only manipulate non-raw buffers (e.g. gldeinterlace). Those elements do not have to implement GstVideoContext interface.

element :

a GstElement

query :

a GstQuery

gst_video_context_query_get_supported_types ()

const gchar **      gst_video_context_query_get_supported_types
                                                        (GstQuery *query);

query :

a GstQuery

Returns :

An array of supported video context types

gst_video_context_query_parse_value ()

void                gst_video_context_query_parse_value (GstQuery *query,
                                                         const gchar **type,
                                                         const GValue **value);

Helper to extract the video context type and value from a GstQuery.

query :

a GstQuery

type :

return video context type

value :

return video context GValue

gst_video_context_query_set_value ()

void                gst_video_context_query_set_value   (GstQuery *query,
                                                         const gchar *type,
                                                         GValue *value);

Helper to set the video context as a GValue inside the GstQuery.

query :

a GstQuery

type :

the video context type

value :

a GValue set with video context

gst_video_context_query_set_string ()

void                gst_video_context_query_set_string  (GstQuery *query,
                                                         const gchar *type,
                                                         const gchar *value);

Helper to set the video context as a string inside the GstQuery.

query :

a GstQuery

type :

the video context type

value :

a string representing the video context

gst_video_context_query_set_pointer ()

void                gst_video_context_query_set_pointer (GstQuery *query,
                                                         const gchar *type,
                                                         gpointer value);

Helper to set the video context as a gpointer inside the GstQuery.

query :

a GstQuery

type :

the video context type

value :

a gpointer representing the video context

gst_video_context_query_set_object ()

void                gst_video_context_query_set_object  (GstQuery *query,
                                                         const gchar *type,
                                                         GObject *value);

Helper to set the video context as a GObject inside the GstQuery.

query :

a GstQuery

type :

the video context type

value :

a GObject representing the video context