blob: 9108d34cab31e27bc2564bdea51da2a241aea3e6 [file] [log] [blame]
Name String
cl_intel_planar_yuv
Contributors
Dan Petre, Intel
Krzysztof Laskowski, Intel
Bartosz Sochacki, Intel
Ben Ashbaugh, Intel
Biju George, Intel
Contact
Krzysztof Laskowski, Intel (krzysztof.laskowski 'at' intel.com)
Version
Version 1, November 22, 2016
Number
OpenCL Extension #49
Status
Final Draft
Extension Type
OpenCL platform extension
Dependencies
OpenCL 1.2 is required. The concept of using clCreateImage to create a 2D image from
another 2D image object is based on OpenCL 2.0. This extension is written against
revision 29 of the OpenCL 2.0 specification and revision 30 of the OpenCL C 2.0
specification.
Overview
The purpose of this extension is to provide OpenCL support for the Planar YUV (YCbCr)
image formats. NV12 format must be supported; support for other Planar YUV formats
that may be defined in this extension is optional.
The extension introduces two new cl_mem_flags:
- CL_MEM_NO_ACCESS_INTEL which should be used together with image formats for which
device does not support reading from or writing to at OpenCL kernels level, but are
still useful in other use cases.
- CL_MEM_ACCESS_FLAGS_UNRESTRICTED_INTEL which may be used to relax the memory access
rights specified in cl_mem_flags at memory object creation time and allow to access
and modify the contents of the underlying data storage in unrestricted way e.g. by
creating another memory object from that memory object or using dedicated device
mechanisms.
New Procedures and Functions
None
New Tokens
Accepted as <cl_mem_flags>:
CL_MEM_NO_ACCESS_INTEL (1 << 24)
CL_MEM_ACCESS_FLAGS_UNRESTRICTED_INTEL (1 << 25)
Accepted as <image_channel_order> of <cl_image_format>:
CL_NV12_INTEL 0x410E
Accepted as arguments passed to clGetDeviceInfo:
CL_DEVICE_PLANAR_YUV_MAX_WIDTH_INTEL 0x417E
CL_DEVICE_PLANAR_YUV_MAX_HEIGHT_INTEL 0x417F
New Types
None
Additions to Chapter 4 of the OpenCL 2.0 Specification:
In section 4.2 "Querying Devices" modify the description of function clGetDeviceInfo:
Table 4.3 must be extended to include the following enumeration constants:
+---------------------------------------+-------------+---------------------------------+
| cl_device_info | Return Type | Description |
+---------------------------------------+-------------+---------------------------------+
| CL_DEVICE_PLANAR_YUV_MAX_WIDTH_INTEL | size_t | Max width of Planar YUV image |
| | | in pixels. |
+---------------------------------------+-------------+---------------------------------+
| CL_DEVICE_PLANAR_YUV_MAX_HEIGHT_INTEL | size_t | Max height of Planar YUV image |
| | | in pixels. |
+---------------------------------------+-------------+---------------------------------+
Additions to Chapter 5 of the OpenCL 2.0 Specification:
In section 5.2.1 "Creating Buffer Objects", extend table 5.3 "List of supported
<cl_mem_flags> values" with:
+-------------------------+-------------------------------------------------------------+
| cl_mem_flags | Description |
+-------------------------+-------------------------------------------------------------+
| | This flag specifies that the device will not read or write |
| CL_MEM_NO_ACCESS_INTEL | to the memory object. |
| | CL_MEM_READ_WRITE or |
| | CL_MEM_WRITE_ONLY or |
| | CL_MEM_READ_ONLY and |
| | CL_MEM_NO_ACCESS_INTEL |
| | are mutually exclusive. |
+-------------------------+-------------------------------------------------------------+
| | This flag indicates that the host and device access flags |
| CL_MEM_ACCESS_FLAGS_ | used together with this flag do not strictly prohibit |
| UNRESTRICTED_INTEL | reading or modifying the contents of this memory object. |
| | Memory objects created from this memory object may |
| | re-specify the host and device access capabilities of the |
| | created memory object with new access capabilities, and any |
| | mechanisms provided by the implementation which explicitly |
| | support certain operations on memory objects of this type |
| | are allowed to access this memory object without any |
| | restrictions. |
+-------------------------+-------------------------------------------------------------+
In section 5.3.1 "Creating Image Objects", in the table that lists the image types
change the CL_MEM_OBJECT_2D entry description to:
+-----------------------+-------------------------------------------+
| Image Type | Size of buffer that host_ptr points to |
+-----------------------+-------------------------------------------+
| | >= image_row_pitch * image_height + |
| CL_MEM_OBJECT_IMAGE2D | image_row_pitch * image_height / 2 |
| | for images with CL_NV12_INTEL |
| | image_channel_order. |
+-----------------------+-------------------------------------------+
Extend the description of clCreateImage with:
"<flags> is a bit-field [...].If <image_channel_order> of <image_format> is a Planar
YUV format then <flags> must include CL_MEM_HOST_NO_ACCESS.
[...]
If CL_MEM_ACCESS_FLAGS_UNRESTRICTED_INTEL is specified in the memory access
qualifier values associated with <mem_object> then <flags> can specify any host and
device access capabilities regardless of the memory access qualifier values
associated with <mem_object>."
"A 2D image can be created from another 2D image object [...].
Creating a 2D image from a Planar YUV image object allows users to create a new
image object that shares the Planar YUV image object's data store but represents
only the specified plane. The restrictions are:
- all the values specified in image_desc except for mem_object must match the image
descriptor information associated with mem_object, with exception where mem_object
is a Planar YUV image object then image_width and image_height are ignored and
derived from the mem_object and image_depth specifies the index of the target
plane the image will be created against and must be one of the following:
+---------------------+-------+-----------------------+
| image_channel_order | plane | image_depth specified |
| of mem_object | | in image_desc |
+---------------------+-------+-----------------------+
| CL_NV12_INTEL | Y | 0 |
| CL_NV12_INTEL | UV | 1 |
+----------------------+-------+----------------------+
The derived values of image_width and image_height can be later queried using
clGetImageInfo.
- the channel data type specified in image_format must match the channel data type
associated with mem_object with exception to the following list of supported
combinations:
+---------------------+---------------------------+
| image_channel_order | image_channel_data_type |
| of mem_object | specified in image_format |
+---------------------+---------------------------+
| CL_NV12_INTEL | CL_UNORM_INT8 |
| CL_NV12_INTEL | CL_UNSIGNED_INT8 |
+---------------------+---------------------------+
- the channel order specified in image_format must match the channel order
associated with mem_object with exception to the following list of supported
combinations:
+---------------------------+-------------------------+
| image_channel_order | image_channel_order of |
| specified in image_format | mem_object |
+---------------------------+-------------------------+
| CL_sBGRA | CL_BGRA |
| CL_BGRA | CL_sBGRA |
| CL_sRGBA | CL_RGBA |
| CL_RGBA | CL_sRGBA |
| CL_sRGB | CL_RGB |
| CL_RGB | CL_sRGB |
| CL_sRGBx | CL_RGBx |
| CL_RGBx | CL_sRGBx |
| CL_DEPTH | CL_R |
+---------------------------+-------------------------+
If mem_object is a Planar YUV image object the channel order specified in image
format must be one of the following:
+---------------------------+-------------------------+-------+------------------+
| image_channel_order | image_channel_order of | plane | channel mappings |
| specified in image_format | mem_object | | |
+---------------------------+-------------------------+-------+------------------+
| CL_R | CL_NV12_INTEL | Y | R = Y |
| CL_RG | CL_NV12_INTEL | UV | R = U, G = V |
+---------------------------+-------------------------+-------+------------------+
NOTE:
Concurrent reading from or writing to both a Planar YUV image object and an image
object created from the Planar YUV image object is undefined.
Reading from or writing to an image created from a Planar YUV image and then reading
from or writing to the Planar YUV image in a kernel even if appropriate
synchronization operations (such as a barrier) are performed between the reads or
writes is undefined. Similarly, reading from and writing to the Planar YUV image and
then reading from or writing to the image created from the Planar YUV image with
appropriate synchronization between the reads or writes is undefined."
Modify the section about the returned error values:
"clCreateImage returns a valid non-zero image object created and <errcode_ret> is
set to CL_SUCCESS if the image object is created successfully. Otherwise, it returns
a NULL value with one of the following error values returned in <errcode_ret>:
[...]
* CL_INVALID_VALUE if an image is being created from another memory object (buffer
or image) under one of the following circumstances: 1) <mem_object> was created
with CL_MEM_WRITE_ONLY and <flags> specifies CL_MEM_READ_WRITE or
CL_MEM_READ_ONLY, 2) <mem_object> was created with CL_MEM_READ_ONLY and <flags>
specifies CL_MEM_READ_WRITE or CL_MEM_WRITE_ONLY, 3) <mem_object> was created with
CL_MEM_NO_ACCESS_INTEL and <flags> specifies CL_MEM_READ_ONLY, CL_MEM_WRITE_ONLY
or CL_MEM_READ_WRITE, 4) <flags> specifies CL_MEM_USE_HOST_PTR or
CL_MEM_ALLOC_HOST_PTR or CL_MEM_COPY_HOST_PTR. However, restrictions 1), 2) and 3)
described above do not apply if <mem_object> was created with <flag>
CL_MEM_ACCESS_FLAGS_UNRESTRICTED_INTEL.
* CL_INVALID_VALUE if an image is being created from another memory object (buffer
or image) and <mem_object> was created with CL_MEM_HOST_WRITE_ONLY and <flags>
specifies CL_MEM_HOST_READ_ONLY, or if <mem_object> was created with
CL_MEM_HOST_READ_ONLY and <flags> specifies CL_MEM_HOST_WRITE_ONLY, or if
<mem_object> was created with CL_MEM_HOST_NO_ACCESS and flags specifies
CL_MEM_HOST_READ_ONLY or CL_MEM_HOST_WRITE_ONLY. However, these restrictions do
not apply if <mem_object> was created with <flag>
CL_MEM_ACCESS_FLAGS_UNRESTRICTED_INTEL."
Modify the section about the memory layout of a 2D image:
"For a 2D RGB image, the image data specified by <host_ptr> is stored as a linear
sequence of adjacent scanlines. Each scanline is a linear sequence of image
elements. For 2D Planar YUV images see section 5.3.1.3 on the "Memory Layout for
Planar YUV Images"."
Extend table 5.6 "List of supported Image Channel Order Values" with:
+------------------------------------------------------------------+
| Enum values that can be specified in channel_order |
+------------------------------------------------------------------+
| CL_NV12_INTEL. Images created with this <image_channel_order> |
| value are Planar YUV images. CL_NV12_INTEL <image_channel_order> |
| can only be used if <image_channel_data_type> is CL_UNORM_INT8 |
| and <image_type> is CL_MEM_OBJECT_IMAGE2D. Number of channels in |
| a CL_NV12_INTEL image equals 3 and the element size refers to |
| the Y plane and equals 1. |
+------------------------------------------------------------------+
In section 5.3.1.2 "Image Descriptor", extend the description of <image_width>,
<image_height> and <image_depth>:
"<image_width> is the width of the image in pixels. [...] If this is a CL_NV12_INTEL
image, <image_width> must be a multiple of 4 and less than or equal to
CL_DEVICE_PLANAR_YUV_MAX_WIDTH_INTEL.
<image_height> is the height of the image in pixels. [...] If this is a
CL_NV12_INTEL image, <image_height> must be a multiple of 4 and less than or equal
to CL_DEVICE_PLANAR_YUV_MAX_HEIGHT_INTEL.
<image_depth> is the depth of the image in pixels. [...] If this is a CL_NV12_INTEL
image, <image_depth> must be 1.
Add section 5.3.1.3 "Memory Layout for Planar YUV Images":
"In Planar YUV formats the Y, U and V components can all be stored as separate planes
or the U and V components can be stored combined as one plane. There are various
flavors of Planar YUV formats, differing in the number of planes, order, layout and
the subsampling methods used for the U and V components. See below for explanation
of the memory layout of CL_NV12_INTEL format.
CL_NV12_INTEL image is laid out in memory in the form of two planes, Y (luma) plane
and an interleaved UV (chroma) plane:
<---- WIDTH ---->
+------------------------+ ^
|YYYYYYYYYYYYYYYYYYYY^^^^| |
|YYYYYYYYYYYYYYYYYYYY^^^^| H
|YYYYYYYYYYYYYYYYYYYY^^^^| E
|YYYYYYYYYYYYYYYYYYYY^^^^| I Luma plane (Y)
|YYYYYYYYYYYYYYYYYYYY^^^^| G
|YYYYYYYYYYYYYYYYYYYY^^^^| H
|YYYYYYYYYYYYYYYYYYYY^^^^| T
|YYYYYYYYYYYYYYYYYYYY^^^^| |
+------------------------+ v
|UVUVUVUVUVUVUVUVUVUV^^^^|
|UVUVUVUVUVUVUVUVUVUV^^^^| Chroma plane (UV)
|UVUVUVUVUVUVUVUVUVUV^^^^|
|UVUVUVUVUVUVUVUVUVUV^^^^|
+------------------------+
<---- ROW PITCH --->
The luma plane contains 8 bit Y samples in case of CL_NV12_INTEL format, one for
each pixel:
+-----+-----+-----+-----+--
| Y00 | Y01 | Y02 | Y03 | ...
+-----+-----+-----+-----+--
| Y10 | Y11 | Y12 | Y13 | ...
+-----+-----+-----+-----+--
| Y20 | Y21 | Y22 | Y23 | ...
+-----+-----+-----+-----+--
| ... | ... | ... | ... |
Sample -> 0 1 2 3
Offset
The chroma plane contains interleaved 8 bit UV 2x2 samples in case of CL_NV12_INTEL
format. The chroma components are sampled only once for every other pixel and for
every other row of pixels:
+-----+-----+-----+-----+--
| U00 | V00 | U02 | V02 | ...
+-----+-----+-----+-----+--
| U20 | V20 | U22 | V22 | ...
+-----+-----+-----+-----+--
| ... | ... | ... | ... |
Sample -> 0 1 2 3
Offset
Using the above notation we can represent pixels like this:
+-----+-----+-----+-----+--
| P00 | P01 | P02 | P03 | ...
+-----+-----+-----+-----+--
| P10 | P11 | P12 | P13 | ...
+-----+-----+-----+-----+--
| P20 | P21 | P22 | P23 | ...
+-----+-----+-----+-----+--
| ... | ... | ... | ... |
where:
P00 = Y00U00V00 P01 = Y01U00V00
P10 = Y10U00V00 P11 = Y11U00V00
...
P20 = Y20U20V20 P21 = Y21U20V20
...
P30 = Y30U20V20 P31 = Y31U20V20
etc.
The luma plane is followed immediately by the chroma plane.
Both the luma and the chroma planes have the same <image_row_pitch>.
The luma plane height is <image_height>.
The chroma plane height is (<image_height> / 2).
The luma plane width is <image_width>.
The chroma plane width is (<image_width> / 2)."
In section 5.3.2 "Querying List of Supported Image Formats", modify the description
of <flags>:
"<flags> is a bit-field that is used to specify allocation and usage information
about the image memory object being queried and is described in table 5.3. To get a
list of supported image formats that can be read from or written to by a kernel,
<flags> must be set to CL_MEM_READ_WRITE (get a list of images that can be read from
and written to by different kernel instances when correctly ordered by event
dependencies), CL_MEM_READ_ONLY (list of images that can be read from by a kernel)
or CL_MEM_WRITE_ONLY (list of images that can be written to by a kernel). To get a
list of supported image formats that can be both read from and written to by the
same kernel instance, flags must be set to CL_MEM_KERNEL_READ_AND_WRITE. To get a
list of images that cannot be read from nor written to by a kernel, flags must be
set to CL_MEM_NO_ACCESS_INTEL. Please see section 5.3.2.2 for clarification."
In section 5.3.2.1 "Minimum List of Supported Image Formats" add a description:
"For 2D image objects, the mandated minimum list of image formats that are not
required to be read from nor written to by a kernel and that must be supported by
all devices that support cl_intel_planar_yuv extension is described in table 5.8.c.
--------------------------------------------------
num_channels channel_order channel_data_type
------------ ------------- -----------------
3 CL_NV12_INTEL CL_UNORM_INT8
--------------------------------------------------
Table 5.8.c "Min. list of supported image formats - kernel no-access 2D Images""
Additions to Chapter 6 of the OpenCL 2.0 Specification:
Extend the section 6.13.14.1.1 "Determining the border color or value":
"If the image channel order is CL_R, CL_RG, CL_RGB, CL_LUMINANCE, or a CL_NV12_INTEL
format's channel order, the border color is (0.0f, 0.0f, 0.0f, 1.0f)."
Extend the paragraph at the beginning of section 6.13.14.2 "Built-in Image Read
Functions":
"The following built-in function calls to read images with a sampler are supported.
Note that reading from a CL_NV12_INTEL image object is only supported by read_imagef
calls that take integer coordinates."
Add a note below the table in section 6.13.14.7 "Mapping image channels to color
values returned by read_image and color values passed to write_image to image
channels":
"NOTE: The mapping of CL_NV12_INTEL image channels to the appropriate components in
the float4 vector data type is ( V, Y, U, 1.0 )."
In section 6.13.14.5 "Built-in Image Query Functions", in table 6.26 "Built-in Image
Query Functions" extend the list of valid values returned by get_image_channel_order
built-in function with CLK_NV12_INTEL.
Appendix A
Sample host code:
cl_image_format image_format;
image_format.image_channel_order = CL_NV12_INTEL;
image_format.image_channel_data_type = CL_UNORM_INT8;
cl_image_desc image_desc;
image_desc.image_type = CL_MEM_OBJECT_IMAGE2D;
image_desc.image_width = width;
image_desc.image_height = height;
image_desc.image_array_size = 0;
image_desc.image_row_pitch = 0;
image_desc.image_slice_pitch = 0;
image_desc.num_mip_levels = 0;
image_desc.num_samples = 0;
image_desc.mem_object = NULL;
// create a CL_NV12_IMAGE
cl_mem nv12Img = clCreateImage(context,
CL_MEM_READ_ONLY | CL_MEM_HOST_NO_ACCESS |
CL_MEM_ACCESS_FLAGS_UNRESTRICTED_INTEL,
image_format, image_desc,
host_ptr, errcode_ret);
// image_width & image_height are ignored for plane extraction
image_desc.image_width = 0;
image_desc.image_height = 0;
// set mem_object to the full NV12 image
image_desc.mem_object = nv12Img;
// get access to the Y plane (CL_R)
image_desc.image_depth = 0;
// set proper image_format for the Y plane
image_format.image_channel_order = CL_R;
image_format.image_channel_data_type = CL_UNORM_INT8;
cl_mem nv12YplaneImg = clCreateImage(context, CL_MEM_READ_WRITE,
image_format, image_desc,
NULL, errcode_ret);
// get access to the UV plane (CL_RG)
image_desc.image_depth = 1;
// set proper image_format for the UV plane
image_format.image_channel_order = CL_RG;
image_format.image_channel_data_type = CL_UNORM_INT8;
cl_mem nv12UVplaneImg = clCreateImage(context, CL_MEM_READ_WRITE,
image_format, image_desc,
NULL, errcode_ret);
// NOT SUPPORTED: transfer the whole NV12 image to the device
// status = clEnqueueWriteImage(queue, nv12Img, true, origin, region,
row_pitch, slice_pitch,
ptr, 0, NULL, NULL);
// write Y plane of NV12 image
status = clEnqueueWriteImage(queue, nv12YplaneImg, true,
origin, region, row_pitch, slice_pitch,
ptr, 0, NULL, NULL);
// write UV plane of NV12 image
status = clEnqueueWriteImage(queue, nv12YplaneImg, true,
origin, region, row_pitch, slice_pitch,
ptr + uvPlaneOffset, 0, NULL, NULL);
// NOT SUPPORTED: read the whole NV12 image back
// status = clEnqueueReadImage(queue, nv12Img, true,
origin, region, row_pitch, slice_pitch,
ptr, 0, NULL, NULL);
// read Y plane of NV12 image
status = clEnqueueReadImage(queue, nv12UVplaneImg, true,
origin, region, row_pitch, slice_pitch,
ptr, 0, NULL, NULL);
// read UV plane of NV12 image
status = clEnqueueReadImage(queue, nv12UVplaneImg, true,
origin, region, row_pitch, slice_pitch,
ptr + uvPlaneOffset, 0, NULL, NULL);
Appendix B
Sample kernel code:
// do something with a whole NV12 image
kernel void DoSomethingWithNV12
(
...
read_write image2d_t nv12Img,
...
)
{
...
// sample the CL_NV12_INTEL image - supported if CL_NV12_INTEL format is
// available with CL_MEM_READ_ONLY or CL_MEM_READ_WRITE access flags
// based on clGetSupportedImageFormats query.
float4 p = read_imagef(nv12Img, sampler, coord);
...
// write to the CL_NV12_INTEL image - supported if CL_NV12_INTEL format is
// available with CL_MEM_WRITE_ONLY or CL_MEM_READ_WRITE access flags
// based on clGetSupportedImageFormats query.
write_imagef(nv12Img, coord, p);
...
}
// do something with planes of an NV12 image
kernel void DoSomethingWithNV12Planes
(
...
read_write image2d_t nv12ImgYPlane,
read_write image2d_t nv12ImgUVPlane,
...
)
{
...
// sample the Y & UV planes
float4 py = read_imagef(nv12ImgYPlane, sampler, coord);
float4 puv = read_imagef(nv12ImgUVPlane, sampler, coord);
...
// write to Y & UV planes
write_imagef(nv12ImgYPlane, coord, py);
write_imagef(nv12ImgUVPlane, coord, puv);
...
}
Revision History
Version 1, 2016/22/11 (K.Laskowski) - Initial version.