| Name |
| |
| QCOM_texture_foveated |
| |
| Name Strings |
| |
| GL_QCOM_texture_foveated |
| |
| Contributors |
| |
| Tate Hornbeck |
| Jonathan Wicks |
| Robert VanReenen |
| Matthew Hoffman |
| Jeff Leger |
| |
| Contact |
| |
| Jeff Leger - jleger 'at' qti.qualcomm.com |
| |
| Status |
| |
| Complete |
| |
| Version |
| |
| Last Modified Date: Oct 06, 2017 |
| Revision: #5 |
| |
| Number |
| |
| OpenGL ES Extension #293 |
| |
| Dependencies |
| |
| OpenGL ES 2.0 is required. This extension is written against OpenGL ES 3.2. |
| |
| This interacts with QCOM_framebuffer_foveated. |
| |
| Overview |
| |
| Foveated rendering is a technique that aims to reduce fragment processing |
| workload and bandwidth by reducing the average resolution of a render target. |
| Perceived image quality is kept high by leaving the focal point of |
| rendering at full resolution. |
| |
| It exists in two major forms: |
| |
| - Static foveated (lens matched) rendering: where the gaze point is |
| fixed with a large fovea region and designed to match up with the lens |
| characteristics. |
| - Eye-tracked foveated rendering: where the gaze point is continuously |
| tracked by a sensor to allow a smaller fovea region (further reducing |
| average resolution) |
| |
| Traditionally foveated rendering involves breaking a render target's area |
| into smaller regions such as bins, tiles, viewports, or layers which are |
| rendered to individually. Each of these regions has the geometry projected |
| or scaled differently so that the net resolution of these layers is less |
| than the original render target's resolution. When these regions are mapped |
| back to the original render target, they create a rendered result with |
| decreased quality as pixels get further from the focal point. |
| |
| Foveated rendering is currently achieved by large modifications to an |
| applications render pipelines to manually implement the required geometry |
| amplifications, blits, and projection changes. This presents a large |
| implementation cost to an application developer and is generally |
| inefficient as it can not make use of a platforms unique hardware features |
| or optimized software paths. This extension aims to address these problems |
| by exposing foveated rendering in an explicit and vendor neutral way, and by |
| providing an interface with minimal changes to how an application specifies |
| its render targets. |
| |
| New Tokens |
| |
| Accepted as a value for <pname> for the TexParameter{if} and |
| TexParameter{if}v commands and for the <pname> parameter of |
| GetTexParameter{if}v: |
| |
| TEXTURE_FOVEATED_FEATURE_BITS_QCOM 0x8BFB |
| TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM 0x8BFC |
| |
| Accepted as the <pname> parameter of GetTexParameter{if}v: |
| |
| TEXTURE_FOVEATED_FEATURE_QUERY_QCOM 0x8BFD |
| TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM 0x8BFE |
| |
| Accepted as a value to <param> for the TexParameter{if} and |
| to <params> for the TexParameter{if}v commands with a <pname> of |
| TEXTURE_FOVEATED_FEATURE_BITS_QCOM; returned as possible values for |
| <params> when GetTexParameter{if}v is queried with a <pname> of |
| TEXTURE_FOVEATED_FEATURE_BITS_QCOM: |
| |
| FOVEATION_ENABLE_BIT_QCOM 0x1 |
| FOVEATION_SCALED_BIN_METHOD_BIT_QCOM 0x2 |
| |
| Returned by CheckFramebufferStatus: |
| |
| FRAMEBUFFER_INCOMPLETE_FOVEATION_QCOM 0x8BFF |
| |
| Add new rows to Table 8.19 (Texture parameters and their values): |
| |
| Name | Type | Legal Values |
| ------------------------------------------------------------ |
| TEXTURE_FOVEATED_FEATURE_BITS_QCOM | uint | 0, |
| FOVEATION_ENABLE_BIT_QCOM, |
| FOVEATION_ENABLE_BIT_QCOM | |
| FOVEATION_SCALED_BIN_METHOD_BIT_QCOM) |
| |
| TEXTURE_FOVEATED_FEATURE_QUERY_QCOM | uint | 0, |
| FOVEATION_ENABLE_BIT_QCOM, |
| FOVEATION_ENABLE_BIT_QCOM | |
| FOVEATION_SCALED_BIN_METHOD_BIT_QCOM) |
| |
| TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM | uint | Any integer greater than 0 |
| |
| TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM | float | Any float between 0.0 and 1.0 |
| |
| Add new rows to Table 21.10 Textures (state per texture object) |
| |
| Get value | Type | Get Command | Initial Value | Description | Sec |
| ------------------------------------------------------------------ |
| TEXTURE_FOVEATED_FEATURE_BITS_QCOM | Z+ | GetTexParameter{if}v | 0 | Foveation State | 8.19 |
| |
| TEXTURE_FOVEATED_FEATURE_QUERY_QCOM | Z+ | GetTexParameter{if}v | see sec 8.19 | Supported foveation state | 8.19 |
| |
| TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM | Z+ | GetTexParameter{if}v | see sec 8.19 | Number of supported focal points per texture layer | 8.19 |
| |
| TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM | R[0.0,1.0] | GetTexParameter{if}v | 0.0 | Minimum pixel density allowed | 8.19 |
| |
| New Procedures and Functions |
| |
| void TextureFoveationParametersQCOM(uint texture, |
| uint layer, |
| uint focalPoint, |
| float focalX, |
| float focalY, |
| float gainX, |
| float gainY, |
| float foveaArea); |
| |
| Additions to the end of section 8.19 of the OpenGL ES 3.2 Specification |
| |
| TEXTURE_FOVEATED_FEATURE_QUERY_QCOM is a texture property that can only be |
| queried via GetTexParameter{if}v. This will return the implementation's |
| supported foveation methods. |
| |
| glGetTexParameteriv(GL_TEXTURE_2D, |
| GL_TEXTURE_FOVEATED_FEATURE_QUERY_QCOM, |
| &query); |
| |
| if ((query & FOVEATION_ENABLE_BIT_QCOM == FOVEATION_ENABLE_BIT_QCOM) && |
| (query & FOVEATION_SCALED_BIN_METHOD_BIT_QCOM == |
| FOVEATION_SCALED_BIN_METHOD_BIT_QCOM)) |
| { |
| // Implementation supports scaled bin method of foveation |
| } |
| |
| TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM is a texture property that |
| can only be queried GetTexParameter{if}v. This will return the number of |
| focal points per texture layer that the implementation supports. This must |
| be greater than 0. |
| |
| TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM defines the minimum pixel density |
| that can be used for scaled bin foveated rendering. |
| |
| TEXTURE_FOVEATED_FEATURE_BITS_QCOM can be used to enable foveation for a |
| texture render target. |
| |
| An explanation of each of the features is below: |
| |
| FOVEATION_ENABLE_BIT_QCOM: Is used to enable foveated rendering, if |
| this bit is not specified foveated rendering will not be used. |
| |
| FOVEATION_SCALED_BIN_METHOD_BIT_QCOM: Requests that the implementation |
| perform foveated rendering by dividing the texture render target into a |
| grid of subregions. Each subregions will be greater than or equal to one pixel |
| and less than or equal to the full size of the texture. Then rendering |
| the geometry to each of these regions with a different projection or scale. |
| Then, finally upscaling the subregion to the native texture resolution. |
| Quality in the scaled bin method is defined as a minimum pixel density |
| which is the ratio of the resolution rendered compared to the native |
| texture. |
| |
| glTexParameteri(GL_TEXTURE_2D, |
| GL_TEXTURE_FOVEATED_FEATURE_BITS_QCOM, |
| GL_FOVEATION_ENABLE_BIT_QCOM | |
| GL_FOVEATION_SCALED_BIN_METHOD_BIT_QCOM); |
| |
| This will set a texture as foveated so all subsequent rendering to |
| this texture will be foveated as long as the FOVEATION_ENABLE_BIT_QCOM |
| is set. Foveation is a texture property that is only applicable for |
| rendering operations, it does not affect traditional texture functions |
| like TexImage2D or TexSubImage2D. |
| |
| The command |
| |
| void TextureFoveationParametersQCOM(uint texture, |
| uint layer, |
| uint focalPoint, |
| float focalX, |
| float focalY, |
| float gainX, |
| float gainY, |
| float foveaArea); |
| |
| is used to control the falloff of the foveated rendering of 'focalPoint' |
| for layer 'layer' in the texture object 'texture'. Multiple focal points |
| per layer are provided to enable foveated rendering when different regions |
| of a texture represent different views, such as with double wide |
| rendering. TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM focal points are |
| supported per texture so values of 0 to |
| TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM minus one are valid for the |
| 'focalPoint' input and specify which focal point's data to update for the |
| layer. |
| |
| 'focalX' and 'focalY' is used to specify the x and y coordinate |
| of the focal point of the foveated texture in normalized device |
| coordinates. 'gainX' and 'gainY' are used to control how quickly the |
| quality falls off as you get further away from the focal point in each |
| axis. The larger these values are the faster the quality degrades. |
| 'foveaArea' is used to control the minimum size of the fovea region, the |
| area before the quality starts to fall off. These parameters should be |
| modified to match the lens characteristics. |
| |
| For the scaled bin method, these parameters define the minimum pixel |
| density allowed for a given focal point at the location (px,py) on a |
| texture layer in NDC as: |
| |
| min_pixel_density=0.; |
| for(int i=0;i<focalPointsPerLayer;++i){ |
| focal_point_density = 1./max((focalX[i]-px)^2*gainX[i]^2+ |
| (focalY[i]-py)^2*gainY[i]^2-foveaArea[i],1.); |
| min_pixel_density=max(min_pixel_density,focal_point_density); |
| min_pixel_desnsity=max(min_pixel_desnsity, |
| TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM); |
| } |
| |
| While this function is continuous, it is worth noting that an |
| implementation is allowed to decimate to a fixed number of supported |
| quality levels, and it is allowed to group pixels into larger regions of |
| constant quality level, as long as the implementation at least provides |
| the quality level given in the above equation. |
| |
| Future supported foveation methods could have different definitions of |
| quality. |
| |
| The default values for each of the focal points in a layer is: |
| |
| focalX = focalY = 0.0; |
| gainX = gainY = 0.0; |
| foveaArea = 0.0; |
| |
| Which requires the entire render target to be rendered at full quality. |
| |
| By specifying these constraints an application can fully constrain its |
| render quality while leaving the implementation enough flexibility to |
| render efficiently. |
| |
| Additions to Chapter 9.4 (Framebuffer Completeness) of the OpenGL ES 3.2 Specification |
| |
| More than one color attachment is foveated. |
| |
| { FRAMEBUFFER_INCOMPLETE_FOVEATION_QCOM } |
| |
| Depth or stencil attachments are foveated textures. |
| |
| { FRAMEBUFFER_INCOMPLETE_FOVEATION_QCOM } |
| |
| The framebuffer has been configured for foveation via QCOM_framebuffer_foveated |
| and any color attachment is a foveated texture. |
| |
| { FRAMEBUFFER_INCOMPLETE_FOVEATION_QCOM } |
| |
| Errors |
| |
| INVALID_VALUE is generated by TextureFoveationParametersQCOM if 'texture' |
| is not a valid texture object. |
| |
| INVALID_OPERATION is generated by TextureFoveationParametersQCOM if |
| 'texture' has not been set as foveated. i.e. 'texture's parameter |
| TEXTURE_FOVEATED_FEATURE_BITS_QCOM does not contain |
| FOVEATION_ENABLE_BIT_QCOM. |
| |
| INVALID_VALUE is generated by TextureFoveationParametersQCOM if |
| 'focalPoint' is larger than TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM |
| minus one. |
| |
| INVALID_ENUM is generated by TexParameter{if} or TexParamter{if}v |
| if <pname> is TEXTURE_FOVEATED_FEATURE_QUERY_QCOM or |
| TEXTURE_FOVEATED_NUM_FOCAL_POINTS_QUERY_QCOM. |
| |
| INVALID_ENUM is generated by TexParameter{if} or TexParamter{if}v |
| if <pname> is TEXTURE_FOVEATED_FEATURE_BITS_QCOM and <param> has |
| other bits set besides the legal values listed in table 8.19. |
| |
| INVALID_OPERATION is generated by TexParameter{if} or TexParamter{if}v |
| if <pname> is TEXTURE_FOVEATED_FEATURE_BITS_QCOM and <param> does not have |
| FOVEATION_ENABLE_BIT_QCOM bit set, but the texture's parameter |
| TEXTURE_FOVEATED_FEATURE_BITS_QCOM already contains FOVEATION_ENABLE_BIT_QCOM. |
| i.e. Once foveation has been enabled for a texture, it cannot be disabled. |
| |
| INVALID_OPERATION is generated by TexParameter{if} or TexParamter{if}v |
| if <pname> is TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM and <param> is a |
| float less than 0.0 or greater than 1.0. |
| |
| INVALID_OPERATION is generated by TexParameter{if} or TexParamter{if}v if |
| <pname> is GL_TEXTURE_FOVEATED_FEATURE_BITS_QCOM and <param> contains |
| FOVEATION_ENABLE_BIT_QCOM, but the query of TEXTURE_FOVEATED_FEATURE_QUERY_QCOM |
| of <target> does not contain FOVEATION_ENABLE_BIT_QCOM. |
| |
| Issues |
| |
| 1. Are texture arrays supported? |
| |
| Texture arrays are supported to enable stereoscopic foveated |
| rendering which is a main use case of this extension. When a texture |
| array is used as a foveated render target, each slice has its own set |
| of foveation parameters. |
| |
| 2. How is foveation performed? |
| |
| How foveated rendering is performed to the texture render target is implementation |
| defined. However, if 'FOVEATION_SCALED_BIN_METHOD_BIT_QCOM' is set the |
| implementation must perform foveation by dividing the render target into a |
| grid of subregions. Then rendering the geometry to each of these regions |
| with a different projection or scale. And finally upscaling the subregion |
| to the native full resolution render target. |
| |
| When that bit is not set the implementation can use any algorithm it |
| wants for foveated rendering as long as it meets the application |
| requested features. |
| |
| 3. How are MRTs handled? |
| |
| Only one color attachment may be foveated, all other color attachments |
| will inherit the foveated color attachment's foveation state. |
| |
| 4. Effect on screenspace shader built-ins |
| |
| When using the scaled bin method, a number of screenspace built-ins |
| can produce unexpected results. |
| |
| gl_FragCoord will be scaled to match the relative location in a |
| foveated texture. This means the absolute value of gl_FragCoord |
| will not be correct in lower resolution areas, but the value relative |
| to the full resolution will be consistent. |
| |
| interpolateAtOffset, gl_PointSize, gl_SamplePosition, dFdx, dFdy, |
| glLineWidth will have no corrective scaling applied and thus could |
| have unexpected results. |
| |
| 5. How is depth/stencil handled? |
| |
| Foveation cannot be enabled for depth or stencil texture attachments. However, |
| they will inherit foveation from a foveated color attachment attached to the |
| same framebuffer. In this case the depth and/or stencil attachments should |
| be discarded or invalidated after rendering, as the upscaled depth contents |
| are unlikely to be useful and may cause undesired rendering artifacts when used. |
| |
| 6. Does foveation have any effect on BlitFramebuffer. |
| |
| No, there is no option to foveate a BlitFramebuffer blit. You can BlitFramebuffer |
| from an fbo with a foveated color attachment. |
| |
| 7. Rendering to a foveated texture multiple times per flush |
| |
| The application must be careful to fully clear, or discard/invalidate, any |
| foveated attachments before rendering. Failure to do so would cause |
| unresolves of foveated content which may be undesirable (e.g. cases where |
| the foveation parameters or focal point has changed between resolves). |
| To prevent this the implementation may disable foveation for any rendering |
| to a foveated attachment that requires unresolves. Texture state related |
| to foveation, like TEXTURE_FOVEATED_FEATURE_BITS_QCOM and |
| TEXTURE_FOVEATED_MIN_PIXEL_DENSITY_QCOM will not be affected. |
| |
| 8. Interactions with QCOM_framebuffer_foveated |
| |
| It is illegal to mix usage of these extensions. If a framebuffer has been |
| configured for foveation via QCOM_framebuffer_foveated, no attached textures |
| can be configured for foveation via QCOM_texture_foveated. The framebuffer |
| will be incomplete for this situation. |
| |
| 9. Implementation forced down non-tiled path |
| |
| Certain feature sets may force an implementation to perform non tiled rendering. |
| The implemenation may implicitly disable foveation for these cases. Some |
| potential examples include tessellation, geometry shaders, or compute. |
| |
| Examples: |
| |
| (1) Setup a foveated texture |
| |
| // Allocate a texture |
| GLuint foveatedTexture; |
| glGenTextures(1, &foveatedTexture); |
| glBindTexture(GL_TEXTURE_2D, foveatedTexture); |
| ... |
| // Set texture as foveated |
| glTexParameteri(GL_TEXTURE_2D, |
| GL_TEXTURE_FOVEATED_FEATURE_BITS_QCOM, |
| GL_FOVEATION_ENABLE_BIT_QCOM | GL_FOVEATION_SCALED_BIN_METHOD_BIT_QCOM); |
| ... |
| // Rendering to foveatedTexture |
| ... |
| // Set foveation parameters on texture |
| glTextureFoveationParametersQCOM(foveatedTexture, 0, 0, focalX, focalY, gainX, gainY, 0.0f); |
| glFlush(); |
| |
| (2) Setting parameters for a multiview stereo texture array |
| |
| float focalX1 = 0.0f; // Gaze of left eye |
| float focalY1 = 0.0f; // Gaze of left eye |
| float focalX2 = 0.0f; // Gaze of right eye |
| float focalY2 = 0.0f; // Gaze of right eye |
| float gainX = 4.0f; // Weak foveation |
| float gainY = 4.0f; // Weak foveation |
| |
| glTextureFoveationParametersQCOM(foveatedTextureArray, |
| 0, |
| 0, |
| focalX1, |
| focalY1, |
| gainX, |
| gainY, |
| 2.0f); |
| glTextureFoveationParametersQCOM(foveatedTextureArray, |
| 1, |
| 0, |
| focalX2, |
| focalY2, |
| gainX, |
| gainY, |
| 2.0f); |
| |
| (3) Setting parameters for a double wide stereo texture |
| |
| float focalX1 = -0.5f; // Gaze of left eye |
| float focalY1 = 0.0f; // Gaze of left eye |
| float focalX2 = 0.5f; // Gaze of right eye |
| float focalY2 = 0.0f; // Gaze of right eye |
| float gainX = 8.0f; // Strong foveation |
| float gainY = 8.0f; // Strong foveation |
| |
| glTextureFoveationParametersQCOM(foveatedTexture, |
| 0, |
| 0, |
| focalX1, |
| focalY1, |
| gainX * 2.0f, |
| gainY, |
| 8.0f); |
| glTextureFoveationParametersQCOM(foveatedTexture, |
| 0, |
| 1, |
| focalX2, |
| focalY2, |
| gainX * 2.0f, |
| gainY, |
| 8.0f); |
| |
| Revision History |
| |
| Rev. Date Author Changes |
| ---- -------- -------- ---------------------------------------------- |
| 1 08/22/17 tateh Initial spec |
| 2 09/20/17 tateh Clarified screenspace shader issues. Added |
| way to query number of supported focal points. |
| 3 09/25/17 tateh Add max scale factor texture parameter. |
| 4 10/3/17 tateh Add Table 21.10 modifications and issue #9 |
| 5 10/6/17 tateh Changed max scale factor to min pixel density |
| 6 01/8/17 tateh Removed depth/stencil discard and invalidate |
| language. |