blob: e2f45742a5fc43a342fa59a0c3575220bccbc596 [file] [log] [blame]
Name String
cl_intel_va_api_media_sharing
Contributors
Ben Ashbaugh, Intel
Jacek Danecki, Intel
Krzysztof Laskowski, Intel
Contact
Ben Ashbaugh, Intel (ben.ashbaugh 'at' intel.com)
Version
Version 1, December 8, 2014
Number
OpenCL Extension #36
Status
Final Draft
Dependencies
OpenCL 1.2 is required and a VA API implementation supporting
sharing of surfaces with OpenCL is required. This extension is written
against revision 19 of the OpenCL 1.2 specification and against revision
19 of the OpenCL 1.2 extension specification.
Overview
The goal of this extension is to provide interoperability between OpenCL
and the cross-platform Video Acceleration API (VA API). The extension
specifically enables sharing of VA API surfaces for media. It uses
similar mechanisms and provides similar surface sharing capabilities for
VA API that are provided by the Khronos OpenCL 1.2 Media Surface Sharing
extension for DX9.
New Procedures and Functions
cl_int clGetDeviceIDsFromVA_APIMediaAdapterINTEL (
cl_platform_id platform,
cl_va_api_device_source_intel media_adapter_type ,
void *media_adapter,
cl_va_api_device_set_intel media_adapter_set,
cl_uint num_entries,
cl_device_id *devices,
cl_uint *num_devices)
cl_mem clCreateFromVA_APIMediaSurfaceINTEL(
cl_context context,
cl_mem_flags flags,
VASurfaceID *surface ,
cl_uint plane,
cl_int *errcode_ret)
cl_int clEnqueueAcquireVA_APIMediaSurfacesINTEL(
cl_command_queue command_queue,
cl_uint num_objects,
const cl_mem *mem_objects,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event)
cl_int clEnqueueReleaseVA_APIMediaSurfacesINTEL(
cl_command_queue command_queue,
cl_uint num_objects,
const cl_mem *mem_objects,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event)
New Tokens
Accepted as a VA API device source in the <media_adapter_type>
parameter of clGetDeviceIDsFromVA_APIMediaAdapterINTEL:
CL_VA_API_DISPLAY_INTEL 0x4094
Accepted as a set of VA API devices in the <media_adapter_set>
parameter of clGetDeviceIDsFromVA_APIMediaAdapterINTEL:
CL_PREFERRED_DEVICES_FOR_VA_API_INTEL 0x4095
CL_ALL_DEVICES_FOR_VA_API_INTEL 0x4096
Accepted as a property name in the <properties> parameter of
clCreateContext and clCreateContextFromType:
CL_CONTEXT_VA_API_DISPLAY_INTEL 0x4097
Accepted as the property being queried in the <param_name>
parameter of clGetMemObjectInfo:
CL_MEM_VA_API_MEDIA_SURFACE_INTEL 0x4098
Accepted as the property being queried in the <param_name>
parameter of clGetImageInfo:
CL_IMAGE_VA_API_PLANE_INTEL 0x4099
Returned in the <param_value> parameter of clGetEventInfo when
<param_name> is CL_EVENT_COMMAND_TYPE:
CL_COMMAND_ACQUIRE_VA_API_MEDIA_SURFACES_INTEL 0x409A
CL_COMMAND_RELEASE_VA_API_MEDIA_SURFACES_INTEL 0x409B
Returned by clCreateContext and clCreateContextFromType if the VA API
display specified for interoperability is not compatible with the
devices against which the context is to be created:
CL_INVALID_VA_API_MEDIA_ADAPTER_INTEL -1098
Returned by clCreateFromVA_APIMediaSurfaceINTEL when <surface> is not a
VA API surface of the required type, by clGetMemObjectInfo when
<param_name> is CL_MEM_VA_API_MEDIA_SURFACE_INTEL when <memobj> was not
created from a VA API surface, and from clGetImageInfo when
<param_name> is CL_IMAGE_VA_API_PLANE_INTEL and <image> was not created
from a VA API surface:
CL_INVALID_VA_API_MEDIA_SURFACE_INTEL -1099
Returned by clEnqueueAcquireVA_APIMediaSurfacesINTEL when any of <mem_objects>
are currently acquired by OpenCL:
CL_VA_API_MEDIA_SURFACE_ALREADY_ACQUIRED_INTEL -1100
Returned by clEnqueueReleaseVA_APIMediaSurfacesINTEL when any of <mem_objects>
are not currently acquired by OpenCL:
CL_VA_API_MEDIA_SURFACE_NOT_ACQUIRED_INTEL -1101
Additions to Chapter 4 of the OpenCL 1.2 Specification
In section 4.4, replace the description of <properties> under
clCreateContext with:
"<properties> specifies a list of context property names and their
corresponding values. Each property is followed immediately by the
corresponding desired value. The list is terminated with zero.
If a property is not specified in <properties>, then its default
value (listed in table 4.4) is used (it is said to be specified
implicitly). If <properties> is NULL or empty (points to a list
whose first value is zero), all attributes take on their default
values."
Add the following to table 4.5:
"--------------------------------------------------------------------------------------
cl_context_properties Value type Default value Description
--------------------- ---------- ------------- -----------
CL_CONTEXT_VA_API_DISPLAY_INTEL VADisplay* NULL Specifies a VA display to
use for VA API interop.
--------------------------------------------------------------------------------------"
Add to the list of errors for clCreateContext:
"* CL_INVALID_VA_API_MEDIA_ADAPTER_INTEL if the value of the property
CL_CONTEXT_VA_API_DISPLAY_INTEL is non-NULL and does not specify a valid
VA API display with which the cl_device_ids against which this context
is to be created may interoperate.
* CL_INVALID_OPERATION if VA API interoperability is specified by
setting CL_CONTEXT_VA_API_DISPLAY_INTEL to a non-NULL value, and
interoperability with any other graphics API is also specified. "
Add to the list of errors for clCreateContextFromType the same new
errors described above for clCreateContext.
Additions to Chapter 5 of the OpenCL 1.2 Specification
5.4.5 Memory Object Queries
Add to the list of errors for clGetMemObjectInfo:
"* CL_INVALID_VA_API_MEDIA_SURFACE_INTEL if <param_name> is
CL_MEM_VA_API_MEDIA_SURFACE_INTEL and <memobj> was not created from
a VA API surface using clCreateFromVA_APIMediaSurfaceINTEL."
Extend table 5.11 to include the following entries:
--------------------------------------------------------------------------
cl_mem_info Return type Info. returned in param_value
----------- --------------- -----------------------------
CL_MEM_VA_API_MEDIA_ VASurfaceID* Returns the <surface> argument
SURFACE_INTEL value specified when <memobj>
was created.
---------------------------------------------------------------------------
Add to the list of errors for clGetImageInfo:
"* CL_INVALID_VA_API_MEDIA_SURFACE_INTEL if <param_name> is
CL_IMAGE_VA_API_PLANE_INTEL and <image> was not created from a VA API
surface using clCreateFromVA_APIMediaSurfaceINTEL."
Extend table 5.9 to include the following entry:
------------------------------------------------------------------------
cl_image_info Return type Info. returned in param_value
------------- ----------- -----------------------------
CL_IMAGE_VA_API_PLANE_INTEL cl_uint Returns the <plane> argument
value specified when <image>
was created.
------------------------------------------------------------------------
5.9 Event Objects
Add to Table 5.18 in the "Info returned in <param_value>" column
for cl_event_info CL_EVENT_COMMAND_TYPE:
CL_COMMAND_ACQUIRE_VA_API_MEDIA_SURFACES_INTEL
CL_COMMAND_RELEASE_VA_API_MEDIA_SURFACES_INTEL
Add new section 9.X to the OpenCL 1.2 extension specification:
9.X Sharing VA API Media Surfaces with OpenCL
This section discusses OpenCL functions that allow applications to use
VA API media surfaces as OpenCL memory objects. This allows efficient
sharing of data between OpenCL and VA API. The OpenCL API may be
used to execute kernels that read and/or write memory objects that are
also VA API surfaces. An OpenCL image object may be created from
a VA API surface. OpenCL memory objects may be created from VA API
surfaces if and only if the OpenCL context has been created from a VA
API display.
9.X.1 Querying OpenCL Devices Corresponding to VA API Displays
The OpenCL devices corresponding to a VA API display may be queried
using the function
cl_int clGetDeviceIDsFromVA_APIMediaAdapterINTEL(
cl_platform_id platform,
cl_va_api_device_source_intel media_adapter_type,
void *media_adapter,
cl_va_api_device_set_intel media_adapter_set,
cl_uint num_entries,
cl_device_id *devices,
cl_uint *num_devices)
<platform> refers to the platform ID returned by clGetPlatformIDs.
<media_adapter_type> specifies the type of <media_adapter>, and must be
one of the values shown in table 9.X.1.1.
<media_adapter> specifies the media adapter whose corresponding OpenCL
devices are being queried. The type of <media_adapter> must be as specified
in table 9.X.1.1.
<media_adapter_set> specifies the set of devices to return and must be
one of the values shown in table 9.X.1.2.
<num_entries> is the number of cl_device_id entries that can be added to
<devices>. If <devices> is not NULL then <num_entries> must be greater than
zero.
<devices> returns a list of OpenCL devices found. The cl_device_id values
returned in <devices> can be used to identify a specific OpenCL device.
If <devices> is NULL, this argument is ignored. The number of
OpenCL devices returned is the minimum of the value specified by
<num_entries> and the number of OpenCL devices corresponding to
<media_adapter>.
<num_devices> returns the number of OpenCL devices available that
correspond to <media_adapter>. If <num_devices> is NULL, this argument
is ignored.
clGetDeviceIDsFromVA_APIMediaAdapterINTEL returns CL_SUCCESS if the
function is executed successfully. Otherwise it may return:
* CL_INVALID_PLATFORM if <platform> is not a valid platform,
* CL_INVALID_VALUE if <media_adapter_type> is not a valid value,
<media_adapter_set> is not a valid value, <num_entries> is equal
to zero and <devices> is not NULL, or if both <num_devices>
and <devices> are NULL.
* CL_DEVICE_NOT_FOUND if no OpenCL devices that correspond to
<media_adapter> were found.
--------------------------------------------------------------------
cl_va_api_device_source_intel Type of <media_adapter>
----------------------------- -----------------------
CL_VA_API_DISPLAY_INTEL VADisplay*
--------------------------------------------------------------------
Table 9.X.1.1: Types used to specify the object whose corresponding
OpenCL devices are being queried by
clGetDeviceIDsFromVA_APIMediaAdapterINTEL.
--------------------------------------------------------------------
cl_va_api_device_set_intel Devices returned in <devices>
-------------------------- -----------------------------
CL_PREFERRED_DEVICES_FOR_VA_API_INTEL The OpenCL devices associated with
the specified media adapter.
CL_ALL_DEVICES_FOR_VA_API_INTEL All OpenCL devices which may
interoperate with the specified
media adapter. Performance of
sharing data on these devices may
be considerably less than on the
preferred devices.
--------------------------------------------------------------------
Table 9.X.1.2: Types used to specify the set of devices returned from
clGetDeviceIDsFromVA_APIMediaAdapterINTEL.
9.X.2 Lifetime of Shared Resources
An OpenCL memory object created from a VA API surface remains
valid as long as the corresponding VA API surface has not been
deleted. If the VA API surface is deleted through the VA API,
subsequent use of the OpenCL memory object will result in undefined
behavior, including but not limited to possible OpenCL errors, data
corruption, and program termination.
9.X.4 Sharing VA API Media Surfaces as OpenCL Image Objects
The function
cl_mem clCreateFromVA_APIMediaSurfaceINTEL(
cl_context context,
cl_mem_flags flags,
VASurfaceID *surface,
cl_uint plane,
cl_int *errcode_ret)
creates an OpenCL 2D image object from a VA API media surface or a
plane of a VA API media surface.
<context> is a valid OpenCL context created from a VA API display.
<flags> is a bit-field that is used to specify usage information.
Refer to table 5.3 for a description of <flags>. Only
CL_MEM_READ_ONLY, CL_MEM_WRITE_ONLY and CL_MEM_READ_WRITE values
specified in table 5.3 can be used.
<surface> is a pointer to the VA API surface to share.
<plane> is the plane of <surface> to share, for planar formats.
For non-planar formats, <plane> must be 0.
<errcode_ret> will return an appropriate error code. If
<errcode_ret> is NULL, no error code is returned.
clCreateFromVA_APIMediaSurfaceINTEL returns a valid non-zero OpenCL 2D
image object and <errcode_ret> is set to CL_SUCCESS if the OpenCL 2D
image object is created successfully. Otherwise, it returns a NULL value
with one of the following error values returned in <errcode_ret>:
* CL_INVALID_CONTEXT if <context> is not a valid context.
* CL_INVALID_VALUE if values specified in <flags> are not valid
or if <plane> is not a valid plane of <resource>.
* CL_INVALID_VA_API_MEDIA_SURFACE_INTEL if <surface> is not a VA API
surface, if a cl_mem from <surface> and <plane> has already been
created, or if <context> was not created against the same VA API
display from which <surface> was created.
* CL_INVALID_IMAGE_FORMAT_DESCRIPTOR if the format of <surface> is
not listed in table 9.X.4.1.
* CL_INVALID_OPERATION if there are no devices in <context> that
support VA API sharing.
* CL_OUT_OF_RESOURCES if there is a failure to allocate resources
required by the OpenCL implementation on the device.
* CL_OUT_OF_HOST_MEMORY if there is a failure to allocate resources
required by the OpenCL implementation on the host.
The width and height of the returned OpenCL 2D image object are
determined by the width and height of the <plane> of <surface>. The
channel type and order of the returned OpenCL 2D image object is
determined by the format and <plane> of <surface> in table 9.X.4.1.
------------------------------------------------------------------
VA_FOURCC cl_channel_order cl_channel_type
------------------------------ ---------------- ---------------
VA_FOURCC_NV12,Plane0 CL_R CL_UNORM_INT8
VA_FOURCC_NV12,Plane1 CL_RG CL_UNORM_INT8
------------------------------------------------------------------
Table 9.X.4.1: List of VA API FourCC codes and corresponding OpenCL
Image Formats.
For the NV12 surface format, plane 0 corresponds to the luminance (Y)
channel data and plane 1 corresponds to the chrominance (UV) channel
data.
9.X.5 Querying Properties of Memory Objects Created From VA API Surfaces
Properties of media objects created from VA API surfaces may be queried
using clGetMemObjectInfo and clGetImageInfo with <param_name>
CL_MEM_VA_API_MEDIA_SURFACE_INTEL and CL_IMAGE_VA_API_PLANE_INTEL as
described in section 5.3.7 and 5.4.5.
9.X.6 Sharing Memory Objects Created From VA API Surfaces Between VA API
and OpenCL
The function
cl_int clEnqueueAcquireVA_APIMediaSurfacesINTEL(
cl_command_queue command_queue,
cl_uint num_objects,
const cl_mem *mem_objects,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event)
is used to acquire OpenCL memory objects that have been created from
VA API surfaces. The VA API surfaces are acquired by the OpenCL context
associated with <command_queue> and can therefore be used by all command-
queues associated with the OpenCL context.
OpenCL memory objects created from VA API surfaces must be acquired before
they can be used by any OpenCL commands queued to a command-queue. If an
OpenCL memory object created from a VA API surface is used while it is not
acquired by OpenCL, the call attempting to use that OpenCL memory object
will return CL_VA_API_MEDIA_SURFACE_NOT_ACQUIRED_INTEL.
If CL_CONTEXT_INTEROP_USER_SYNC is not specified as CL_TRUE during context
creation, clEnqueueAcquireVA_APIMediaSurfacesINTEL provides the
synchronization guarantee that any VA API calls made before
clEnqueueAcquireVA_APIMediaSurfacesINTEL is called will complete executing
before <event> reports completion and before the execution of any subsequent
OpenCL work issued in <command_queue> begins. If the context was created
with properties specifying CL_CONTEXT_INTEROP_USER_SYNC as CL_TRUE, the user
is responsible for guaranteeing that any VA API calls involving the interop
device(s) used in the OpenCL context made before
clEnqueueAcquireVA_APIMediaSurfacesINTEL is called have completed before
calling clEnqueueAcquireVA_APIMediaSurfacesINTEL.
<command_queue> is a valid command-queue.
<num_objects> is the number of memory objects to be acquired in
<mem_objects>.
<mem_objects> is a pointer to a list of OpenCL memory objects that
were created from VA API surfaces.
<event_wait_list> and <num_events_in_wait_list> specify events that
need to complete before this particular command can be executed. If
<event_wait_list> is NULL, then this particular command does not
wait on any event to complete. If <event_wait_list> is NULL,
<num_events_in_wait_list> must be 0. If <event_wait_list> is not
NULL, the list of events pointed to by <event_wait_list> must be
valid and <num_events_in_wait_list> must be greater than 0. The
events specified in <event_wait_list> act as synchronization points.
<event> returns an event object that identifies this particular
command and can be used to query or queue a wait for this
particular command to complete. <event> can be NULL in which case it
will not be possible for the application to query the status of this
command or queue a wait for this command to complete.
clEnqueueAcquireVA_APIMediaSurfacesINTEL returns CL_SUCCESS if the
function is executed successfully. If <num_objects> is 0 and
<mem_objects> is NULL then the function does nothing and returns
CL_SUCCESS. Otherwise it returns one of the following errors:
* CL_INVALID_VALUE if <num_objects> is zero and <mem_objects> is
not a NULL value or if <num_objects> > 0 and <mem_objects> is
NULL.
* CL_INVALID_MEM_OBJECT if memory objects in <mem_objects> are not
valid OpenCL memory objects or if memory objects in <mem_objects>
have not been created from VA API surfaces.
* CL_INVALID_COMMAND_QUEUE if <command_queue> is not a valid
command-queue.
* CL_INVALID_CONTEXT if the context associated with <command_queue>
was not created from a VA API display.
* CL_VA_API_MEDIA_SURFACE_ALREADY_ACQUIRED_INTEL if memory objects
in <mem_objects> have previously been acquired using
clEnqueueAcquireVA_APIMediaSurfacesINTEL but have not been released
using clEnqueueReleaseVA_APIMediaSurfacesINTEL.
* CL_INVALID_EVENT_WAIT_LIST if <event_wait_list> is NULL and
<num_events_in_wait_list> > 0, or <event_wait_list> is not NULL
and <num_events_in_wait_list> is 0, or if event objects in
<event_wait_list> are not valid events.
* CL_OUT_OF_RESOURCES if there is a failure to allocate resources
required by the OpenCL implementation on the device.
* CL_OUT_OF_HOST_MEMORY if there is a failure to allocate resources
required by the OpenCL implementation on the host.
The function
cl_int clEnqueueReleaseVA_APIMediaSurfacesINTEL(
cl_command_queue command_queue,
cl_uint num_objects,
const cl_mem *mem_objects,
cl_uint num_events_in_wait_list,
const cl_event *event_wait_list,
cl_event *event)
is used to release OpenCL memory objects that have been created from
VA API surfaces. The VA API surfaces are released by the OpenCL context
associated with <command_queue>.
OpenCL memory objects created from VA API surfaces which have been
acquired by OpenCL must be released by OpenCL before they may be
accessed by VA API. Accessing a VA API surface while its corresponding
OpenCL memory object is acquired is in error and will result in
undefined behavior, including but not limited to possible OpenCL errors,
data corruption, and program termination.
If CL_CONTEXT_INTEROP_USER_SYNC is not specified as CL_TRUE during
context creation, clEnqueueReleaseVA_APIMediaSurfacesINTEL provides the
synchronization guarantee that any VA API calls made after
clEnqueueReleaseVA_APIMediaSurfacesINTEL is called will not start
executing until after all events in <event_wait_list> are complete and
all work already submitted to <command_queue> completes execution. If
the context was created with properties specifying
CL_CONTEXT_INTEROP_USER_SYNC specified as CL_TRUE, the user is
responsible for guaranteeing that any VA API calls involving the
interop device(s) used in the OpenCL context made after
clEnqueueReleaseVA_APIMediaSurfacesINTEL is called will not start
executing until after the event returned by
clEnqueueReleaseVA_APIMediaSurfacesINTEL reports completion.
<num_objects> is the number of memory objects to be released in
<mem_objects>.
<mem_objects> is a pointer to a list of OpenCL memory objects that
were created from VA API surfaces.
<event_wait_list> and <num_events_in_wait_list> specify events that
need to complete before this particular command can be executed. If
<event_wait_list> is NULL, then this particular command does not
wait on any event to complete. If <event_wait_list> is NULL,
<num_events_in_wait_list> must be 0. If <event_wait_list> is not
NULL, the list of events pointed to by <event_wait_list> must be
valid and <num_events_in_wait_list> must be greater than 0. The
events specified in <event_wait_list> act as synchronization points.
<event> returns an event object that identifies this particular
command and can be used to query or queue a wait for this
particular command to complete. <event> can be NULL in which case it
will not be possible for the application to query the status of this
command or queue a wait for this command to complete.
clEnqueueReleaseVA_APIMediaSurfacesINTEL returns CL_SUCCESS if the
function is executed successfully. If <num_objects> is 0 and
<mem_objects> is NULL the function does nothing and returns
CL_SUCCESS. Otherwise it returns one of the following errors:
* CL_INVALID_VALUE if <num_objects> is zero and <mem_objects> is
not a NULL value or if <num_objects> > 0 and <mem_objects> is
NULL.
* CL_INVALID_MEM_OBJECT if memory objects in <mem_objects> are not
valid OpenCL memory objects or if memory objects in <mem_objects>
have not been created from VA API surfaces.
* CL_INVALID_COMMAND_QUEUE if <command_queue> is not a valid
command-queue.
* CL_INVALID_CONTEXT if the context associated with <command_queue>
was not created from a VA API display.
* CL_VA_API_MEDIA_SURFACE_NOT_ACQUIRED_INTEL if memory objects in
<mem_objects> have not previously been acquired using
clEnqueueAcquireVA_APIMediaSurfacesINTEL, or have already been
released using clEnqueueReleaseVA_APIMediaSurfacesINTEL since the
last time that they were acquired.
* CL_INVALID_EVENT_WAIT_LIST if <event_wait_list> is NULL and
<num_events_in_wait_list> > 0, or <event_wait_list> is not NULL
and <num_events_in_wait_list> is 0, or if event objects in
<event_wait_list> are not valid events.
* CL_OUT_OF_RESOURCES if there is a failure to allocate resources
required by the OpenCL implementation on the device.
* CL_OUT_OF_HOST_MEMORY if there is a failure to allocate resources
required by the OpenCL implementation on the host.
Issues
None
Revision History
Version 1 (2014/12/08): First public revision.