Name String

    cl_intel_va_api_media_sharing

Contributors

    Ben Ashbaugh, Intel
    Jacek Danecki, Intel
    Krzysztof Laskowski, Intel

Contact

    Ben Ashbaugh, Intel (ben.ashbaugh 'at' intel.com)

Version

    Version 1, December 8, 2014

Number

    OpenCL Extension #36

Status

    Final Draft

Dependencies

    OpenCL 1.2 is required and a VA API implementation supporting 
    sharing of surfaces with OpenCL is required.  This extension is written 
    against revision 19 of the OpenCL 1.2 specification and against revision
    19 of the OpenCL 1.2 extension specification.

Overview

    The goal of this extension is to provide interoperability between OpenCL 
    and the cross-platform Video Acceleration API (VA API).  The extension 
    specifically enables sharing of VA API surfaces for media.  It uses 
    similar mechanisms and provides similar surface sharing capabilities for 
    VA API that are provided by the Khronos OpenCL 1.2 Media Surface Sharing 
    extension for DX9.

New Procedures and Functions

    cl_int clGetDeviceIDsFromVA_APIMediaAdapterINTEL   (
        cl_platform_id platform,
        cl_va_api_device_source_intel media_adapter_type  ,
        void *media_adapter,
        cl_va_api_device_set_intel media_adapter_set,
        cl_uint num_entries,
        cl_device_id *devices,
        cl_uint *num_devices)

    cl_mem clCreateFromVA_APIMediaSurfaceINTEL(
        cl_context context,
        cl_mem_flags flags,
        VASurfaceID *surface ,
        cl_uint plane,
        cl_int *errcode_ret)

    cl_int clEnqueueAcquireVA_APIMediaSurfacesINTEL(
        cl_command_queue command_queue,
        cl_uint num_objects,
        const cl_mem *mem_objects,
        cl_uint num_events_in_wait_list,
        const cl_event *event_wait_list,
        cl_event *event)

    cl_int clEnqueueReleaseVA_APIMediaSurfacesINTEL(
        cl_command_queue command_queue,
        cl_uint num_objects,
        const cl_mem *mem_objects,
        cl_uint num_events_in_wait_list,
        const cl_event *event_wait_list,
        cl_event *event)

New Tokens

    Accepted as a VA API device source in the <media_adapter_type> 
    parameter of clGetDeviceIDsFromVA_APIMediaAdapterINTEL:

        CL_VA_API_DISPLAY_INTEL                0x4094

    Accepted as a set of VA API devices in the <media_adapter_set> 
    parameter of clGetDeviceIDsFromVA_APIMediaAdapterINTEL:

        CL_PREFERRED_DEVICES_FOR_VA_API_INTEL  0x4095
        CL_ALL_DEVICES_FOR_VA_API_INTEL        0x4096

    Accepted as a property name in the <properties> parameter of
    clCreateContext and clCreateContextFromType:

        CL_CONTEXT_VA_API_DISPLAY_INTEL        0x4097

    Accepted as the property being queried in the <param_name> 
    parameter of clGetMemObjectInfo:

        CL_MEM_VA_API_MEDIA_SURFACE_INTEL      0x4098

    Accepted as the property being queried in the <param_name> 
    parameter of clGetImageInfo:

        CL_IMAGE_VA_API_PLANE_INTEL            0x4099

    Returned in the <param_value> parameter of clGetEventInfo when
    <param_name> is CL_EVENT_COMMAND_TYPE:

        CL_COMMAND_ACQUIRE_VA_API_MEDIA_SURFACES_INTEL 0x409A
        CL_COMMAND_RELEASE_VA_API_MEDIA_SURFACES_INTEL 0x409B

    Returned by clCreateContext and clCreateContextFromType if the VA API
    display specified for interoperability is not compatible with the 
    devices against which the context is to be created:

        CL_INVALID_VA_API_MEDIA_ADAPTER_INTEL  -1098

    Returned by clCreateFromVA_APIMediaSurfaceINTEL when <surface> is not a 
    VA API surface of the required type, by clGetMemObjectInfo when 
    <param_name> is CL_MEM_VA_API_MEDIA_SURFACE_INTEL when <memobj> was not 
    created from a VA API surface, and from clGetImageInfo when 
    <param_name> is CL_IMAGE_VA_API_PLANE_INTEL and <image> was not created 
    from a VA API surface:

        CL_INVALID_VA_API_MEDIA_SURFACE_INTEL  -1099

    Returned by clEnqueueAcquireVA_APIMediaSurfacesINTEL when any of <mem_objects> 
    are currently acquired by OpenCL:

        CL_VA_API_MEDIA_SURFACE_ALREADY_ACQUIRED_INTEL -1100

    Returned by clEnqueueReleaseVA_APIMediaSurfacesINTEL when any of <mem_objects> 
    are not currently acquired by OpenCL:
        
        CL_VA_API_MEDIA_SURFACE_NOT_ACQUIRED_INTEL -1101

Additions to Chapter 4 of the OpenCL 1.2 Specification

    In section 4.4, replace the description of <properties> under
    clCreateContext with:

   "<properties> specifies a list of context property names and their
    corresponding values.  Each property is followed immediately by the
    corresponding desired value.  The list is terminated with zero.  
    If a property is not specified in <properties>, then its default 
    value (listed in table 4.4) is used (it is said to be specified
    implicitly). If <properties> is NULL or empty (points to a list
    whose first value is zero), all attributes take on their default
    values."

    Add the following to table 4.5:

   "--------------------------------------------------------------------------------------
    cl_context_properties           Value type     Default value  Description
    ---------------------           ----------     -------------  -----------
    CL_CONTEXT_VA_API_DISPLAY_INTEL VADisplay*     NULL           Specifies a VA display to
                                                                  use for VA API interop.
    --------------------------------------------------------------------------------------"

    Add to the list of errors for clCreateContext:

   "* CL_INVALID_VA_API_MEDIA_ADAPTER_INTEL if the value of the property
      CL_CONTEXT_VA_API_DISPLAY_INTEL is non-NULL and does not specify a valid
      VA API display with which the cl_device_ids against which this context 
      is to be created may interoperate.

    * CL_INVALID_OPERATION if VA API interoperability is specified by 
      setting CL_CONTEXT_VA_API_DISPLAY_INTEL to a non-NULL value, and 
      interoperability with any other graphics API is also specified. "

    Add to the list of errors for clCreateContextFromType the same new
    errors described above for clCreateContext.

Additions to Chapter 5 of the OpenCL 1.2 Specification

    5.4.5 Memory Object Queries

    Add to the list of errors for clGetMemObjectInfo:

   "* CL_INVALID_VA_API_MEDIA_SURFACE_INTEL if <param_name> is 
      CL_MEM_VA_API_MEDIA_SURFACE_INTEL and <memobj> was not created from 
      a VA API surface using clCreateFromVA_APIMediaSurfaceINTEL."

    Extend table 5.11 to include the following entries:

    --------------------------------------------------------------------------
    cl_mem_info                 Return type      Info. returned in param_value
    -----------                 ---------------  -----------------------------
    CL_MEM_VA_API_MEDIA_        VASurfaceID*     Returns the <surface> argument 
    SURFACE_INTEL                                value specified when <memobj> 
                                                 was created.
    ---------------------------------------------------------------------------

    Add to the list of errors for clGetImageInfo:

   "* CL_INVALID_VA_API_MEDIA_SURFACE_INTEL if <param_name> is 
      CL_IMAGE_VA_API_PLANE_INTEL and <image> was not created from a VA API 
      surface using clCreateFromVA_APIMediaSurfaceINTEL."

    Extend table 5.9 to include the following entry:

    ------------------------------------------------------------------------
    cl_image_info               Return type     Info. returned in param_value
    -------------               -----------     -----------------------------
    CL_IMAGE_VA_API_PLANE_INTEL cl_uint         Returns the <plane> argument
                                                value specified when <image> 
                                                was created.
    ------------------------------------------------------------------------

    5.9 Event Objects

    Add to Table 5.18 in the "Info returned in <param_value>" column
    for cl_event_info CL_EVENT_COMMAND_TYPE:

      CL_COMMAND_ACQUIRE_VA_API_MEDIA_SURFACES_INTEL
      CL_COMMAND_RELEASE_VA_API_MEDIA_SURFACES_INTEL

Add new section 9.X to the OpenCL 1.2 extension specification:

    9.X Sharing VA API Media Surfaces with OpenCL

    This section discusses OpenCL functions that allow applications to use
    VA API media surfaces as OpenCL memory objects. This allows efficient 
    sharing of data between OpenCL and VA API. The OpenCL API may be 
    used to execute kernels that read and/or write memory objects that are 
    also VA API surfaces.  An OpenCL image object may be created from 
    a VA API surface. OpenCL memory objects may be created from VA API 
    surfaces if and only if the OpenCL context has been created from a VA 
    API display.

    9.X.1 Querying OpenCL Devices Corresponding to VA API Displays

    The OpenCL devices corresponding to a VA API display may be queried 
    using the function

        cl_int clGetDeviceIDsFromVA_APIMediaAdapterINTEL(
            cl_platform_id platform,
            cl_va_api_device_source_intel media_adapter_type,
            void *media_adapter,
            cl_va_api_device_set_intel media_adapter_set,
            cl_uint num_entries,
            cl_device_id *devices,
            cl_uint *num_devices)

    <platform> refers to the platform ID returned by clGetPlatformIDs.

    <media_adapter_type> specifies the type of <media_adapter>, and must be 
    one of the values shown in table 9.X.1.1.

    <media_adapter> specifies the media adapter whose corresponding OpenCL 
    devices are being queried.  The type of <media_adapter> must be as specified 
    in table 9.X.1.1.

    <media_adapter_set> specifies the set of devices to return and must be
    one of the values shown in table 9.X.1.2.

    <num_entries> is the number of cl_device_id entries that can be added to 
    <devices>. If <devices> is not NULL then <num_entries> must be greater than 
    zero.

    <devices> returns a list of OpenCL devices found. The cl_device_id values 
    returned in <devices> can be used to identify a specific OpenCL device. 
    If <devices> is NULL, this argument is ignored. The number of 
    OpenCL devices returned is the minimum of the value specified by 
    <num_entries> and the number of OpenCL devices corresponding to 
    <media_adapter>.

    <num_devices> returns the number of OpenCL devices available that 
    correspond to <media_adapter>.  If <num_devices> is NULL, this argument 
    is ignored.

    clGetDeviceIDsFromVA_APIMediaAdapterINTEL returns CL_SUCCESS if the 
    function is executed successfully.  Otherwise it may return:

      * CL_INVALID_PLATFORM if <platform> is not a valid platform, 

      * CL_INVALID_VALUE if <media_adapter_type> is not a valid value, 
        <media_adapter_set> is not a valid value, <num_entries> is equal 
        to zero and <devices> is not NULL, or if both <num_devices> 
        and <devices> are NULL.

      * CL_DEVICE_NOT_FOUND if no OpenCL devices that correspond to  
        <media_adapter> were found.

    --------------------------------------------------------------------
    cl_va_api_device_source_intel      Type of <media_adapter> 
    -----------------------------      -----------------------
    CL_VA_API_DISPLAY_INTEL            VADisplay*
    --------------------------------------------------------------------
    Table 9.X.1.1: Types used to specify the object whose corresponding 
    OpenCL devices are being queried by 
    clGetDeviceIDsFromVA_APIMediaAdapterINTEL.

    --------------------------------------------------------------------
    cl_va_api_device_set_intel            Devices returned in <devices>
    --------------------------            -----------------------------
    CL_PREFERRED_DEVICES_FOR_VA_API_INTEL The OpenCL devices associated with
                                          the specified media adapter.

    CL_ALL_DEVICES_FOR_VA_API_INTEL       All OpenCL devices which may 
                                          interoperate with the specified
                                          media adapter.  Performance of
                                          sharing data on these devices may 
                                          be considerably less than on the 
                                          preferred devices.
    --------------------------------------------------------------------
    Table 9.X.1.2: Types used to specify the set of devices returned from
    clGetDeviceIDsFromVA_APIMediaAdapterINTEL.

    9.X.2 Lifetime of Shared Resources 

    An OpenCL memory object created from a VA API surface remains 
    valid as long as the corresponding VA API surface has not been 
    deleted.  If the VA API surface is deleted through the VA API,
    subsequent use of the OpenCL memory object will result in undefined 
    behavior, including but not limited to possible OpenCL errors, data 
    corruption, and program termination.

    9.X.4 Sharing VA API Media Surfaces as OpenCL Image Objects

    The function

        cl_mem  clCreateFromVA_APIMediaSurfaceINTEL(
            cl_context context,
            cl_mem_flags flags,
            VASurfaceID *surface,
            cl_uint plane,
            cl_int *errcode_ret)

    creates an OpenCL 2D image object from a VA API media surface or a 
    plane of a VA API media surface.

    <context> is a valid OpenCL context created from a VA API display.

    <flags> is a bit-field that is used to specify usage information.
    Refer to table 5.3 for a description of <flags>. Only 
    CL_MEM_READ_ONLY, CL_MEM_WRITE_ONLY and CL_MEM_READ_WRITE values
    specified in table 5.3 can be used.

    <surface> is a pointer to the VA API surface to share.

    <plane> is the plane of <surface> to share, for planar formats.  
    For non-planar formats, <plane> must be 0.

    <errcode_ret> will return an appropriate error code. If
    <errcode_ret> is NULL, no error code is returned.

    clCreateFromVA_APIMediaSurfaceINTEL    returns a valid non-zero OpenCL 2D 
    image object and <errcode_ret> is set to CL_SUCCESS if the OpenCL 2D 
    image object is created successfully.  Otherwise, it returns a NULL value 
    with one of the following error values returned in <errcode_ret>:

      * CL_INVALID_CONTEXT if <context> is not a valid context.

      * CL_INVALID_VALUE if values specified in <flags> are not valid
        or if <plane> is not a valid plane of <resource>.

      * CL_INVALID_VA_API_MEDIA_SURFACE_INTEL if <surface> is not a VA API
        surface, if a cl_mem from <surface> and <plane> has already been 
        created, or if <context> was not created against the same VA API 
        display from which <surface> was created.

      * CL_INVALID_IMAGE_FORMAT_DESCRIPTOR if the format of <surface> is 
        not listed in table 9.X.4.1.

      * CL_INVALID_OPERATION if there are no devices in <context> that
        support VA API sharing.

      * CL_OUT_OF_RESOURCES if there is a failure to allocate resources
        required by the OpenCL implementation on the device.

      * CL_OUT_OF_HOST_MEMORY if there is a failure to allocate resources
        required by the OpenCL implementation on the host.

    The width and height of the returned OpenCL 2D image object are 
    determined by the width and height of the <plane> of <surface>.  The 
    channel type and order of the returned OpenCL 2D image object is 
    determined by the format and <plane> of <surface> in table 9.X.4.1.
    
    ------------------------------------------------------------------
    VA_FOURCC                         cl_channel_order  cl_channel_type    
    ------------------------------   ----------------  ---------------
    VA_FOURCC_NV12,Plane0            CL_R              CL_UNORM_INT8
    VA_FOURCC_NV12,Plane1            CL_RG             CL_UNORM_INT8
    ------------------------------------------------------------------
    Table 9.X.4.1: List of VA API FourCC codes and corresponding OpenCL 
    Image Formats.  

    For the NV12 surface format, plane 0 corresponds to the luminance (Y)
    channel data and plane 1 corresponds to the chrominance (UV) channel 
    data.

    9.X.5 Querying Properties of Memory Objects Created From VA API Surfaces

    Properties of media objects created from VA API surfaces may be queried 
    using clGetMemObjectInfo and clGetImageInfo with <param_name> 
    CL_MEM_VA_API_MEDIA_SURFACE_INTEL and CL_IMAGE_VA_API_PLANE_INTEL as 
    described in section 5.3.7 and 5.4.5.

    9.X.6 Sharing Memory Objects Created From VA API Surfaces Between VA API
          and OpenCL

    The function

        cl_int  clEnqueueAcquireVA_APIMediaSurfacesINTEL(
            cl_command_queue command_queue,
            cl_uint num_objects,
            const cl_mem *mem_objects,
            cl_uint num_events_in_wait_list,
            const cl_event *event_wait_list,
            cl_event *event)

    is used to acquire OpenCL memory objects that have been created from
    VA API surfaces.  The VA API surfaces are acquired by the OpenCL context 
    associated with <command_queue> and can therefore be used by all command-
    queues associated with the OpenCL context.

    OpenCL memory objects created from VA API surfaces must be acquired before 
    they can be used by any OpenCL commands queued to a command-queue.  If an 
    OpenCL memory object created from a VA API surface is used while it is not 
    acquired by OpenCL, the call attempting to use that OpenCL memory object 
    will return CL_VA_API_MEDIA_SURFACE_NOT_ACQUIRED_INTEL.

    If CL_CONTEXT_INTEROP_USER_SYNC is not specified as CL_TRUE during context
    creation, clEnqueueAcquireVA_APIMediaSurfacesINTEL provides the 
    synchronization guarantee that any VA API calls made before 
    clEnqueueAcquireVA_APIMediaSurfacesINTEL is called will complete executing 
    before <event> reports completion and before the execution of any subsequent 
    OpenCL work issued in <command_queue> begins.  If the context was created
    with properties specifying CL_CONTEXT_INTEROP_USER_SYNC as CL_TRUE, the user
    is responsible for guaranteeing that any VA API calls involving the interop
    device(s) used in the OpenCL context made before 
    clEnqueueAcquireVA_APIMediaSurfacesINTEL is called have completed before 
    calling clEnqueueAcquireVA_APIMediaSurfacesINTEL.

    <command_queue> is a valid command-queue.

    <num_objects> is the number of memory objects to be acquired in
    <mem_objects>.

    <mem_objects> is a pointer to a list of OpenCL memory objects that 
    were created from VA API surfaces.

    <event_wait_list> and <num_events_in_wait_list> specify events that
    need to complete before this particular command can be executed. If
    <event_wait_list> is NULL, then this particular command does not
    wait on any event to complete. If <event_wait_list> is NULL,
    <num_events_in_wait_list> must be 0. If <event_wait_list> is not
    NULL, the list of events pointed to by <event_wait_list> must be
    valid and <num_events_in_wait_list> must be greater than 0. The
    events specified in <event_wait_list> act as synchronization points.

    <event> returns an event object that identifies this particular 
    command and can be used to query or queue a wait for this
    particular command to complete. <event> can be NULL in which case it
    will not be possible for the application to query the status of this
    command or queue a wait for this command to complete.

    clEnqueueAcquireVA_APIMediaSurfacesINTEL returns CL_SUCCESS if the 
    function is executed successfully. If <num_objects> is 0 and 
    <mem_objects> is NULL then the function does nothing and returns 
    CL_SUCCESS. Otherwise it returns one of the following errors:

      * CL_INVALID_VALUE if <num_objects> is zero and <mem_objects> is
        not a NULL value or if <num_objects> > 0 and <mem_objects> is
        NULL.

      * CL_INVALID_MEM_OBJECT if memory objects in <mem_objects> are not
        valid OpenCL memory objects or if memory objects in <mem_objects> 
        have not been created from VA API surfaces.

      * CL_INVALID_COMMAND_QUEUE if <command_queue> is not a valid
        command-queue.

      * CL_INVALID_CONTEXT if the context associated with <command_queue> 
        was not created from a VA API display.

      * CL_VA_API_MEDIA_SURFACE_ALREADY_ACQUIRED_INTEL if memory objects 
        in <mem_objects> have previously been acquired using 
        clEnqueueAcquireVA_APIMediaSurfacesINTEL but have not been released 
        using clEnqueueReleaseVA_APIMediaSurfacesINTEL.

      * CL_INVALID_EVENT_WAIT_LIST if <event_wait_list> is NULL and
        <num_events_in_wait_list> > 0, or <event_wait_list> is not NULL
        and <num_events_in_wait_list> is 0, or if event objects in
        <event_wait_list> are not valid events.

      * CL_OUT_OF_RESOURCES if there is a failure to allocate resources
        required by the OpenCL implementation on the device.

      * CL_OUT_OF_HOST_MEMORY if there is a failure to allocate resources
        required by the OpenCL implementation on the host.

    The function

        cl_int clEnqueueReleaseVA_APIMediaSurfacesINTEL(
            cl_command_queue command_queue,
            cl_uint num_objects,
            const cl_mem *mem_objects,
            cl_uint num_events_in_wait_list,
            const cl_event *event_wait_list,
            cl_event *event)

    is used to release OpenCL memory objects that have been created from
    VA API surfaces. The VA API surfaces are released by the OpenCL context 
    associated with <command_queue>.

    OpenCL memory objects created from VA API surfaces which have been 
    acquired by OpenCL must be released by OpenCL before they may be 
    accessed by VA API.  Accessing a VA API surface while its corresponding 
    OpenCL memory object is acquired is in error and will result in 
    undefined behavior, including but not limited to possible OpenCL errors, 
    data corruption, and program termination.

    If CL_CONTEXT_INTEROP_USER_SYNC is not specified as CL_TRUE during
    context creation, clEnqueueReleaseVA_APIMediaSurfacesINTEL provides the 
    synchronization guarantee that any VA API calls made after
    clEnqueueReleaseVA_APIMediaSurfacesINTEL is called will not start 
    executing until after all events in <event_wait_list> are complete and 
    all work already submitted to <command_queue> completes execution.  If 
    the context was created with properties specifying 
    CL_CONTEXT_INTEROP_USER_SYNC specified as CL_TRUE, the user is 
    responsible for guaranteeing that any VA API calls involving the
    interop device(s) used in the OpenCL context made after 
    clEnqueueReleaseVA_APIMediaSurfacesINTEL is called will not start 
    executing until after the event returned by 
    clEnqueueReleaseVA_APIMediaSurfacesINTEL reports completion.

    <num_objects> is the number of memory objects to be released in
    <mem_objects>.

    <mem_objects> is a pointer to a list of OpenCL memory objects that 
    were created from VA API surfaces.

    <event_wait_list> and <num_events_in_wait_list> specify events that
    need to complete before this particular command can be executed. If
    <event_wait_list> is NULL, then this particular command does not
    wait on any event to complete.  If <event_wait_list> is NULL,
    <num_events_in_wait_list> must be 0. If <event_wait_list> is not
    NULL, the list of events pointed to by <event_wait_list> must be
    valid and <num_events_in_wait_list> must be greater than 0. The
    events specified in <event_wait_list> act as synchronization points.

    <event> returns an event object that identifies this particular 
    command and can be used to query or queue a wait for this
    particular command to complete. <event> can be NULL in which case it
    will not be possible for the application to query the status of this
    command or queue a wait for this command to complete.

    clEnqueueReleaseVA_APIMediaSurfacesINTEL returns CL_SUCCESS if the 
    function is executed successfully. If <num_objects> is 0 and 
    <mem_objects> is NULL the function does nothing and returns 
    CL_SUCCESS.  Otherwise it returns one of the following errors:

      * CL_INVALID_VALUE if <num_objects> is zero and <mem_objects> is
        not a NULL value or if <num_objects> > 0 and <mem_objects> is
        NULL.

      * CL_INVALID_MEM_OBJECT if memory objects in <mem_objects> are not
        valid OpenCL memory objects or if memory objects in <mem_objects> 
        have not been created from VA API surfaces.

      * CL_INVALID_COMMAND_QUEUE if <command_queue> is not a valid
        command-queue.

      * CL_INVALID_CONTEXT if the context associated with <command_queue> 
        was not created from a VA API display.

      * CL_VA_API_MEDIA_SURFACE_NOT_ACQUIRED_INTEL if memory objects in
        <mem_objects> have not previously been acquired using
        clEnqueueAcquireVA_APIMediaSurfacesINTEL, or have already been 
        released using clEnqueueReleaseVA_APIMediaSurfacesINTEL since the 
        last time that they were acquired.

      * CL_INVALID_EVENT_WAIT_LIST if <event_wait_list> is NULL and
        <num_events_in_wait_list> > 0, or <event_wait_list> is not NULL
        and <num_events_in_wait_list> is 0, or if event objects in
        <event_wait_list> are not valid events.

      * CL_OUT_OF_RESOURCES if there is a failure to allocate resources
        required by the OpenCL implementation on the device.

      * CL_OUT_OF_HOST_MEMORY if there is a failure to allocate resources
        required by the OpenCL implementation on the host.

Issues

    None

Revision History

    Version 1 (2014/12/08): First public revision.