blob: 916e305bdc638aa840d7294cf70b622adb710bb4 [file] [log] [blame]
Name
QCOM_texture_foveated
Name Strings
GL_QCOM_texture_foveated
Contributors
Tate Hornbeck
Jonathan Wicks
Robert VanReenen
Matthew Hoffman
Jeff Leger
Contact
Jeff Leger - jleger 'at' qti.qualcomm.com
Status
Complete
Version
Last Modified Date: Oct 06, 2017
Revision: #5
Number
OpenGL ES Extension #293
Dependencies
OpenGL ES 2.0 is required. This extension is written against OpenGL ES 3.2.
This interacts with QCOM_framebuffer_foveated.
Overview
Foveated rendering is a technique that aims to reduce fragment processing
workload and bandwidth by reducing the average resolution of a render target.
Perceived image quality is kept high by leaving the focal point of
rendering at full resolution.
It exists in two major forms:
- Static foveated (lens matched) rendering: where the gaze point is
fixed with a large fovea region and designed to match up with the lens
characteristics.
- Eye-tracked foveated rendering: where the gaze point is continuously
tracked by a sensor to allow a smaller fovea region (further reducing
average resolution)
Traditionally foveated rendering involves breaking a render target's area
into smaller regions such as bins, tiles, viewports, or layers which are
rendered to individually. Each of these regions has the geometry projected
or scaled differently so that the net resolution of these layers is less
than the original render target's resolution. When these regions are mapped
back to the original render target, they create a rendered result with
decreased quality as pixels get further from the focal point.
Foveated rendering is currently achieved by large modifications to an
applications render pipelines to manually implement the required geometry
amplifications, blits, and projection changes. This presents a large
implementation cost to an application developer and is generally
inefficient as it can not make use of a platforms unique hardware features
or optimized software paths. This extension aims to address these problems
by exposing foveated rendering in an explicit and vendor neutral way, and by
providing an interface with minimal changes to how an application specifies
its render targets.
New Tokens
Accepted as a value for <pname> for the TexParameter{if} and
TexParameter{if}v commands and for the <pname> parameter of
GetTexParameter{if}v:
TEXTURE_FOVEATED_FEATURE_BITS_QCOM 0x8BFB
TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM 0x8BFC
Accepted as the <pname> parameter of GetTexParameter{if}v:
TEXTURE_FOVEATED_FEATURE_QUERY_QCOM 0x8BFD
TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM 0x8BFE
Accepted as a value to <param> for the TexParameter{if} and
to <params> for the TexParameter{if}v commands with a <pname> of
TEXTURE_FOVEATED_FEATURE_BITS_QCOM; returned as possible values for
<params> when GetTexParameter{if}v is queried with a <pname> of
TEXTURE_FOVEATED_FEATURE_BITS_QCOM:
FOVEATION_ENABLE_BIT_QCOM 0x1
FOVEATION_SCALED_BIN_METHOD_BIT_QCOM 0x2
Returned by CheckFramebufferStatus:
FRAMEBUFFER_INCOMPLETE_FOVEATION_QCOM 0x8BFF
Add new rows to Table 8.19 (Texture parameters and their values):
Name | Type | Legal Values
------------------------------------------------------------
TEXTURE_FOVEATED_FEATURE_BITS_QCOM | uint | 0,
FOVEATION_ENABLE_BIT_QCOM,
FOVEATION_ENABLE_BIT_QCOM |
FOVEATION_SCALED_BIN_METHOD_BIT_QCOM)
TEXTURE_FOVEATED_FEATURE_QUERY_QCOM | uint | 0,
FOVEATION_ENABLE_BIT_QCOM,
FOVEATION_ENABLE_BIT_QCOM |
FOVEATION_SCALED_BIN_METHOD_BIT_QCOM)
TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM | uint | Any integer greater than 0
TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM | float | Any float between 0.0 and 1.0
Add new rows to Table 21.10 Textures (state per texture object)
Get value | Type | Get Command | Initial Value | Description | Sec
------------------------------------------------------------------
TEXTURE_FOVEATED_FEATURE_BITS_QCOM | Z+ | GetTexParameter{if}v | 0 | Foveation State | 8.19
TEXTURE_FOVEATED_FEATURE_QUERY_QCOM | Z+ | GetTexParameter{if}v | see sec 8.19 | Supported foveation state | 8.19
TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM | Z+ | GetTexParameter{if}v | see sec 8.19 | Number of supported focal points per texture layer | 8.19
TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM | R[0.0,1.0] | GetTexParameter{if}v | 0.0 | Minimum pixel density allowed | 8.19
New Procedures and Functions
void TextureFoveationParametersQCOM(uint texture,
uint layer,
uint focalPoint,
float focalX,
float focalY,
float gainX,
float gainY,
float foveaArea);
Additions to the end of section 8.19 of the OpenGL ES 3.2 Specification
TEXTURE_FOVEATED_FEATURE_QUERY_QCOM is a texture property that can only be
queried via GetTexParameter{if}v. This will return the implementation's
supported foveation methods.
glGetTexParameteriv(GL_TEXTURE_2D,
GL_TEXTURE_FOVEATED_FEATURE_QUERY_QCOM,
&query);
if ((query & FOVEATION_ENABLE_BIT_QCOM == FOVEATION_ENABLE_BIT_QCOM) &&
(query & FOVEATION_SCALED_BIN_METHOD_BIT_QCOM ==
FOVEATION_SCALED_BIN_METHOD_BIT_QCOM))
{
// Implementation supports scaled bin method of foveation
}
TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM is a texture property that
can only be queried GetTexParameter{if}v. This will return the number of
focal points per texture layer that the implementation supports. This must
be greater than 0.
TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM defines the minimum pixel density
that can be used for scaled bin foveated rendering.
TEXTURE_FOVEATED_FEATURE_BITS_QCOM can be used to enable foveation for a
texture render target.
An explanation of each of the features is below:
FOVEATION_ENABLE_BIT_QCOM: Is used to enable foveated rendering, if
this bit is not specified foveated rendering will not be used.
FOVEATION_SCALED_BIN_METHOD_BIT_QCOM: Requests that the implementation
perform foveated rendering by dividing the texture render target into a
grid of subregions. Each subregions will be greater than or equal to one pixel
and less than or equal to the full size of the texture. Then rendering
the geometry to each of these regions with a different projection or scale.
Then, finally upscaling the subregion to the native texture resolution.
Quality in the scaled bin method is defined as a minimum pixel density
which is the ratio of the resolution rendered compared to the native
texture.
glTexParameteri(GL_TEXTURE_2D,
GL_TEXTURE_FOVEATED_FEATURE_BITS_QCOM,
GL_FOVEATION_ENABLE_BIT_QCOM |
GL_FOVEATION_SCALED_BIN_METHOD_BIT_QCOM);
This will set a texture as foveated so all subsequent rendering to
this texture will be foveated as long as the FOVEATION_ENABLE_BIT_QCOM
is set. Foveation is a texture property that is only applicable for
rendering operations, it does not affect traditional texture functions
like TexImage2D or TexSubImage2D.
The command
void TextureFoveationParametersQCOM(uint texture,
uint layer,
uint focalPoint,
float focalX,
float focalY,
float gainX,
float gainY,
float foveaArea);
is used to control the falloff of the foveated rendering of 'focalPoint'
for layer 'layer' in the texture object 'texture'. Multiple focal points
per layer are provided to enable foveated rendering when different regions
of a texture represent different views, such as with double wide
rendering. TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM focal points are
supported per texture so values of 0 to
TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM minus one are valid for the
'focalPoint' input and specify which focal point's data to update for the
layer.
'focalX' and 'focalY' is used to specify the x and y coordinate
of the focal point of the foveated texture in normalized device
coordinates. 'gainX' and 'gainY' are used to control how quickly the
quality falls off as you get further away from the focal point in each
axis. The larger these values are the faster the quality degrades.
'foveaArea' is used to control the minimum size of the fovea region, the
area before the quality starts to fall off. These parameters should be
modified to match the lens characteristics.
For the scaled bin method, these parameters define the minimum pixel
density allowed for a given focal point at the location (px,py) on a
texture layer in NDC as:
min_pixel_density=0.;
for(int i=0;i<focalPointsPerLayer;++i){
focal_point_density = 1./max((focalX[i]-px)^2*gainX[i]^2+
(focalY[i]-py)^2*gainY[i]^2-foveaArea[i],1.);
min_pixel_density=max(min_pixel_density,focal_point_density);
min_pixel_desnsity=max(min_pixel_desnsity,
TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM);
}
While this function is continuous, it is worth noting that an
implementation is allowed to decimate to a fixed number of supported
quality levels, and it is allowed to group pixels into larger regions of
constant quality level, as long as the implementation at least provides
the quality level given in the above equation.
Future supported foveation methods could have different definitions of
quality.
The default values for each of the focal points in a layer is:
focalX = focalY = 0.0;
gainX = gainY = 0.0;
foveaArea = 0.0;
Which requires the entire render target to be rendered at full quality.
By specifying these constraints an application can fully constrain its
render quality while leaving the implementation enough flexibility to
render efficiently.
Additions to Chapter 9.4 (Framebuffer Completeness) of the OpenGL ES 3.2 Specification
More than one color attachment is foveated.
{ FRAMEBUFFER_INCOMPLETE_FOVEATION_QCOM }
Depth or stencil attachments are foveated textures.
{ FRAMEBUFFER_INCOMPLETE_FOVEATION_QCOM }
The framebuffer has been configured for foveation via QCOM_framebuffer_foveated
and any color attachment is a foveated texture.
{ FRAMEBUFFER_INCOMPLETE_FOVEATION_QCOM }
Errors
INVALID_VALUE is generated by TextureFoveationParametersQCOM if 'texture'
is not a valid texture object.
INVALID_OPERATION is generated by TextureFoveationParametersQCOM if
'texture' has not been set as foveated. i.e. 'texture's parameter
TEXTURE_FOVEATED_FEATURE_BITS_QCOM does not contain
FOVEATION_ENABLE_BIT_QCOM.
INVALID_VALUE is generated by TextureFoveationParametersQCOM if
'focalPoint' is larger than TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM
minus one.
INVALID_ENUM is generated by TexParameter{if} or TexParamter{if}v
if <pname> is TEXTURE_FOVEATED_FEATURE_QUERY_QCOM or
TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM.
INVALID_ENUM is generated by TexParameter{if} or TexParamter{if}v
if <pname> is TEXTURE_FOVEATED_FEATURE_BITS_QCOM and <param> has
other bits set besides the legal values listed in table 8.19.
INVALID_OPERATION is generated by TexParameter{if} or TexParamter{if}v
if <pname> is TEXTURE_FOVEATED_FEATURE_BITS_QCOM and <param> does not have
FOVEATION_ENABLE_BIT_QCOM bit set, but the texture's parameter
TEXTURE_FOVEATED_FEATURE_BITS_QCOM already contains FOVEATION_ENABLE_BIT_QCOM.
i.e. Once foveation has been enabled for a texture, it cannot be disabled.
INVALID_OPERATION is generated by TexParameter{if} or TexParamter{if}v
if <pname> is TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM and <param> is a
float less than 0.0 or greater than 1.0.
INVALID_OPERATION is generated by TexParameter{if} or TexParamter{if}v if
<pname> is GL_TEXTURE_FOVEATED_FEATURE_BITS_QCOM and <param> contains
FOVEATION_ENABLE_BIT_QCOM, but the query of TEXTURE_FOVEATED_FEATURE_QUERY_QCOM
of <target> does not contain FOVEATION_ENABLE_BIT_QCOM.
Issues
1. Are texture arrays supported?
Texture arrays are supported to enable stereoscopic foveated
rendering which is a main use case of this extension. When a texture
array is used as a foveated render target, each slice has its own set
of foveation parameters.
2. How is foveation performed?
How foveated rendering is performed to the texture render target is implementation
defined. However, if 'FOVEATION_SCALED_BIN_METHOD_BIT_QCOM' is set the
implementation must perform foveation by dividing the render target into a
grid of subregions. Then rendering the geometry to each of these regions
with a different projection or scale. And finally upscaling the subregion
to the native full resolution render target.
When that bit is not set the implementation can use any algorithm it
wants for foveated rendering as long as it meets the application
requested features.
3. How are MRTs handled?
Only one color attachment may be foveated, all other color attachments
will inherit the foveated color attachment's foveation state.
4. Effect on screenspace shader built-ins
When using the scaled bin method, a number of screenspace built-ins
can produce unexpected results.
gl_FragCoord will be scaled to match the relative location in a
foveated texture. This means the absolute value of gl_FragCoord
will not be correct in lower resolution areas, but the value relative
to the full resolution will be consistent.
interpolateAtOffset, gl_PointSize, gl_SamplePosition, dFdx, dFdy,
glLineWidth will have no corrective scaling applied and thus could
have unexpected results.
5. How is depth/stencil handled?
Foveation cannot be enabled for depth or stencil texture attachments. However,
they will inherit foveation from a foveated color attachment attached to the
same framebuffer. In this case the depth and/or stencil attachments should
be discarded or invalidated after rendering, as the upscaled depth contents
are unlikely to be useful and may cause undesired rendering artifacts when used.
6. Does foveation have any effect on BlitFramebuffer.
No, there is no option to foveate a BlitFramebuffer blit. You can BlitFramebuffer
from a fbo with a foveated color attachment.
7. Rendering to a foveated texture multiple times per flush
The application must be careful to fully clear, or discard/invalidate, any
foveated attachments before rendering. Failure to do so would cause
unresolves of foveated content which may be undesirable (e.g. cases where
the foveation parameters or focal point has changed between resolves).
To prevent this the implementation may disable foveation for any rendering
to a foveated attachment that requires unresolves. Texture state related
to foveation, like TEXTURE_FOVEATED_FEATURE_BITS_QCOM and
TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM will not be affected.
8. Interactions with QCOM_framebuffer_foveated
It is illegal to mix usage of these extensions. If a framebuffer has been
configured for foveation via QCOM_framebuffer_foveated, no attached textures
can be configured for foveation via QCOM_texture_foveated. The framebuffer
will be incomplete for this situation.
9. Implementation forced down non-tiled path
Certain feature sets may force an implementation to perform non tiled rendering.
The implemenation may implicitly disable foveation for these cases. Some
potential examples include tessellation, geometry shaders, or compute.
Examples:
(1) Setup a foveated texture
// Allocate a texture
GLuint foveatedTexture;
glGenTextures(1, &foveatedTexture);
glBindTexture(GL_TEXTURE_2D, foveatedTexture);
...
// Set texture as foveated
glTexParameteri(GL_TEXTURE_2D,
GL_TEXTURE_FOVEATED_FEATURE_BITS_QCOM,
GL_FOVEATION_ENABLE_BIT_QCOM | GL_FOVEATION_SCALED_BIN_METHOD_BIT_QCOM);
...
// Rendering to foveatedTexture
...
// Set foveation parameters on texture
glTextureFoveationParametersQCOM(foveatedTexture, 0, 0, focalX, focalY, gainX, gainY, 0.0f);
glFlush();
(2) Setting parameters for a multiview stereo texture array
float focalX1 = 0.0f; // Gaze of left eye
float focalY1 = 0.0f; // Gaze of left eye
float focalX2 = 0.0f; // Gaze of right eye
float focalY2 = 0.0f; // Gaze of right eye
float gainX = 4.0f; // Weak foveation
float gainY = 4.0f; // Weak foveation
glTextureFoveationParametersQCOM(foveatedTextureArray,
0,
0,
focalX1,
focalY1,
gainX,
gainY,
2.0f);
glTextureFoveationParametersQCOM(foveatedTextureArray,
1,
0,
focalX2,
focalY2,
gainX,
gainY,
2.0f);
(3) Setting parameters for a double wide stereo texture
float focalX1 = -0.5f; // Gaze of left eye
float focalY1 = 0.0f; // Gaze of left eye
float focalX2 = 0.5f; // Gaze of right eye
float focalY2 = 0.0f; // Gaze of right eye
float gainX = 8.0f; // Strong foveation
float gainY = 8.0f; // Strong foveation
glTextureFoveationParametersQCOM(foveatedTexture,
0,
0,
focalX1,
focalY1,
gainX * 2.0f,
gainY,
8.0f);
glTextureFoveationParametersQCOM(foveatedTexture,
0,
1,
focalX2,
focalY2,
gainX * 2.0f,
gainY,
8.0f);
Revision History
Rev. Date Author Changes
---- -------- -------- ----------------------------------------------
1 08/22/17 tateh Initial spec
2 09/20/17 tateh Clarified screenspace shader issues. Added
way to query number of supported focal points.
3 09/25/17 tateh Add max scale factor texture parameter.
4 10/3/17 tateh Add Table 21.10 modifications and issue #9
5 10/6/17 tateh Changed max scale factor to min pixel density
6 01/8/17 tateh Removed depth/stencil discard and invalidate
language.