| Name |
| |
| NV_vdpau_interop |
| |
| Name Strings |
| |
| GL_NV_vdpau_interop |
| |
| Contributors |
| |
| Stephen Warren, NVIDIA |
| James Jones, NVIDIA |
| |
| Contact |
| |
| Stephen Warren, NVIDIA (swarren 'at' nvidia.com) |
| |
| Status |
| |
| Complete. |
| |
| Version |
| |
| 11 (2 Jan 2014) |
| |
| Number |
| |
| 396 |
| |
| Dependencies |
| |
| This extension is written against the OpenGL 3.2 Specification |
| but can apply to OpenGL 1.1 and up. |
| |
| OpenGL 1.1 is required. |
| GL_EXT_framebuffer_object affects the definition of this extension. |
| GL_ARB_texture_rectangle affects the definition of this extension. |
| GL_ARB_texture_non_power_of_two affects the definition of this |
| extension. |
| |
| Overview |
| |
| This extension allows VDPAU video and output surfaces to be used |
| for texturing and rendering. |
| |
| This allows the GL to process and display the content of video |
| streams decoded using VDPAU. |
| |
| Alternatively, the GL may modify VDPAU surfaces in-place, and VDPAU |
| may then process and/or display those surfaces itself. |
| |
| This allows the GL to be used to combine application user-interface |
| elements with decoded video, implement custom video-processing |
| algorithms, etc. |
| |
| IP Status |
| |
| There are no known IP issues. |
| |
| New Types |
| |
| typedef GLintptr GLvdpauSurfaceNV; |
| |
| New Procedures and Functions |
| |
| void VDPAUInitNV (const void *vdpDevice, |
| const void *getProcAddress); |
| |
| void VDPAUFiniNV (void); |
| |
| vdpauSurfaceNV VDPAURegisterVideoSurfaceNV (const void *vdpSurface, |
| enum target, |
| sizei numTextureNames, |
| const uint *textureNames); |
| |
| vdpauSurfaceNV VDPAURegisterOutputSurfaceNV (const void *vdpSurface, |
| enum target, |
| sizei numTextureNames, |
| const uint *textureNames); |
| |
| boolean VDPAUIsSurfaceNV (vdpauSurfaceNV surface); |
| |
| void VDPAUUnregisterSurfaceNV (vdpauSurfaceNV surface); |
| |
| void VDPAUGetSurfaceivNV (vdpauSurfaceNV surface, |
| enum pname, |
| sizei bufSize, |
| sizei *length, |
| int *values); |
| |
| void VDPAUSurfaceAccessNV (vdpauSurfaceNV surface, |
| enum access); |
| |
| void VDPAUMapSurfacesNV (sizei numSurfaces, |
| const vdpauSurfaceNV *surfaces); |
| |
| void VDPAUUnmapSurfacesNV (sizei numSurface, |
| const vdpauSurfaceNV *surfaces); |
| |
| New Tokens |
| |
| Accepted as the <pname> parameter of VDPAUGetSurfaceivNV: |
| |
| SURFACE_STATE_NV 0x86EB |
| |
| Returned in <values> for VDPAUGetSurfaceivNV <pname> |
| SURFACE_STATE_NV: |
| |
| SURFACE_REGISTERED_NV 0x86FD |
| SURFACE_MAPPED_NV 0x8700 |
| |
| Accepted as the <pname> parameter of VDPAUSurfaceAccessNV: |
| |
| WRITE_DISCARD_NV 0x88BE |
| |
| Additions to Chapter 2 of the OpenGL 3.2 (unabridged) Specification |
| (OpenGL Operation) |
| |
| Add to Table 2.2, GL data types: |
| |
| GL Type Minimum Description |
| Bit Width |
| ------- --------- ---------------------------------------------- |
| vdpauSurfaceNV <ptrbits> VDPAU surface object handle (see section 3.8.3) |
| |
| Additions to Chapter 3 of the OpenGL 3.2 (unabridged) Specification |
| (Rasterization) |
| |
| Insert a new section following "Alternate Texture Image |
| Specification Commands" (Section 3.8.2). Renumber existing section |
| 3.8.3 "Compressed Texture Images" and all following 3.8.* sections. |
| |
| 3.8.3, Alternate Texture Image Specification Using VDPAU Surfaces |
| ----------------------------------------------------------------- |
| |
| Texture images may also be specified using image data taken |
| directly from the framebuffer, in the form of VDPAU surfaces. |
| |
| VDPAU surfaces are logically ascribed an interop state. States |
| and transitions are: |
| |
| VDPAUUnegisterSurfaceNV/VDPAUFiniNV |
| /----------------------------------------\ |
| | | |
| v VDPAURegister*SurfaceNV | |
| Unknown/unregistered ----------------------> Registered |
| ^ | ^ |
| | VDPAUMapSurfacesNV | | VDPAUUnmapSurfacesNV |
| | v | |
| \------------------------------------- Mapped |
| VDPAUFiniNV |
| |
| In order to use such "interop" functionality, the command |
| |
| void VDPAUInitNV (const void *vdpDevice, |
| const void *getProcAddress); |
| |
| must be used to inform the GL which VDPAU device to interact with. |
| <vdpDevice> must be an extant VdpDevice handle previously created |
| using VDPAU. <getProcAddress> must be the VdpGetProcAddress |
| generated during VdpDevice creation. |
| |
| When interop functionality is no longer required, the GL may be |
| informed using the command |
| |
| void VDPAUFiniNV (void); |
| |
| This operation will succeed even if surfaces exist in the mapped |
| or registered states; such surfaces will internally be unmapped and |
| unregistered before interop state is destroyed. |
| |
| The command |
| |
| vdpauSurfaceNV VDPAURegisterVideoSurfaceNV (const void *vdpSurface, |
| enum target, |
| sizei numTextureNames, |
| const uint *textureNames); |
| |
| defines a set of two-dimensional textures, where the image data may |
| be taken from the VdpVideoSurface <vdpSurface>. <target> must be |
| one of TEXTURE_2D or TEXTURE_RECTANGLE. <numTextureNames> |
| determines how many textures are defined. <textureNames> contains |
| the names of the textures that are defined. The surface is |
| transitioned into the registered state. |
| |
| Legal values for <numTextureNames> are derived from the |
| VdpChromaType of <vdpSurface>, as defined in table 3.8.3.1. |
| |
| Internal |
| VdpChromaType numTextureNames Index Size Format Content |
| ------------- --------------- ----- ---- -------- ------------------- |
| VDP_CHROMA_TYPE_420 4 0 w x h/2 R8 Top-field luma |
| 1 w x h/2 R8 Bottom-field luma |
| 2 w/2 x h/4 R8G8 Top-field chroma |
| 3 w/2 x h/4 R8G8 Bottom-field chroma |
| VDP_CHROMA_TYPE_422 4 0 w x h/2 R8 Top-field luma |
| 1 w x h/2 R8 Bottom-field luma |
| 2 w/2 x h/2 R8G8 Top-field chroma |
| 3 w/2 x h/2 R8G8 Bottom-field chroma |
| |
| Table 3.8.3.1: Supported VdpChromaType values, and derived values |
| of <numTextureNames>, and texture parameters for each texture. |
| |
| VDPAURegisterVideoSurfaceNV's return value is a handle used by |
| various other commands detailed below. |
| |
| The command |
| |
| vdpauSurfaceNV VDPAURegisterOutputSurfaceNV (const void *vdpSurface, |
| enum target, |
| sizei numTextureNames, |
| const uint *textureNames); |
| |
| defines a set of two-dimensional textures, where the image data may |
| be taken from the VdpOutputSurface vdpSurface. <target> must be one |
| of TEXTURE_2D or TEXTURE_RECTANGLE. <numTextureNames> determines |
| how many textures are defined. <textureNames> contains the names of |
| the textures that are defined. The surface is transitioned into the |
| registered state. |
| |
| Legal values for <numTextureNames> are derived from the |
| VdpRGBAFormat of <vdpSurface>, as defined in table 3.8.3.2. |
| |
| Internal |
| VdpRGBAFormat numTextureNames Index Size Format Content |
| ------------- --------------- ----- ---- -------- -------------- |
| VDP_RGBA_FORMAT_B8G8R8A8 1 0 w x h ARGB8 Entire surface |
| VDP_RGBA_FORMAT_R10G10B10A2 1 0 w x h A2BGR10 Entire surface |
| |
| Table 3.8.3.2: Supported VdpRGBAFormat values, and derived values |
| of <numTextureNames>, and texture parameters for each texture. |
| |
| In all cases, textures defined by VDPAU surfaces will be y- |
| inverted; applications are expected to use y-inverted texture co- |
| ordinates when using such textures. |
| |
| VDPAURegisterOutputSurfaceNV's return value is a handle to be used |
| as the surface parameter to various other commands detailed below. |
| |
| The command |
| |
| void VDPAUUnregisterSurfaceNV (GLvdpauSurfaceNV surface); |
| |
| may be used to revert the effect of VDPAURegisterVideoSurfaceNV or |
| VDPAURegisterOutputSurfaceNV; namely, any <textureName>s that were |
| passed into the register function will be returned to their default |
| state. The surface will be transitioned into the unregistered state. |
| This operation will succeed even if the surface is in the mapped |
| state; such surfaces will internally be unmapped before being |
| unregistered. |
| |
| The command |
| |
| void VDPAUSurfaceAccessNV (GLvdpauSurfaceNV surface, |
| enum access); |
| |
| informs the GL whether the application expects only to read from the |
| VDPAU surface using the GL (<access> set to READ_ONLY), only to |
| write to the VDPAU surface using the GL (<access> set to |
| WRITE_DISCARD_NV), or both (<access> set to READ_WRITE). In |
| implementations that need to copy image data during |
| VDPAUMapSurfacesNV or VDPAUUnmapSurfacesNV, this flag may allow the |
| GL to skip some of those copy operations to improve performance. |
| |
| An <access> value of READ_ONLY ensures that any writes to the |
| surface made by VDPAU prior to VDPAUMapSurfacesNV will be seen by |
| the GL when reading the surface after VDPAUMapSurfacesNV. If the |
| GL writes to the surface, results are undefined, possibly including |
| program termination. |
| |
| An <access> value of READ_WRITE ensures that any writes to the |
| surface made by VDPAU prior to VDPAUMapSurfacesNV will be seen by |
| the GL when reading the surface after VDPAUMapSurfacesNV. Equally, |
| any writes to the surface made by the GL prior to |
| VDPAUUnmapSurfacesNV will be seen by VDPAU after |
| VDPAUUnmapSurfacesNV. |
| |
| An <access> value of WRITE_DISCARD_NV ensures that any writes to the |
| surface made by the GL prior to VDPAUUnmapSurfacesNV will be seen |
| by VDPAU after VDPAUUnmapSurfacesNV. If the GL reads from the |
| surface prior to writing those particular pixels itself, the |
| graphical results are undefined, but should not include program |
| termination. |
| |
| VDPAUSurfaceAccessNV sets the access flag for the VDPAU surface. |
| the GL reads the access flag only during VDPAUMapSurfacesNV. |
| VDPAUUnmapSurfacesNV uses the value of the access flag that was |
| set when the previous VDPAUMapSurfacesNV was called. |
| VDPAUSurfaceAccessNV itself has no effect on the image data of any |
| surface, and in particular does not initiate any copy operations. |
| |
| Calling VDPAUSurfaceAccessNV when the surface is in a mapped state |
| is illegal, and will generate an error. |
| |
| The command |
| |
| void VDPAUMapSurfacesNV (sizei numSurfaces, |
| vdpauSurfaceNV *surfaces); |
| |
| moves a set of VDPAU surfaces from the registered state into the |
| mapped state. |
| |
| The command |
| |
| void VDPAUUnmapSurfacesNV (sizei numSurface, |
| vdpauSurfaceNV* surfaces); |
| |
| moves a set of VDPAU surfaces from the mapped state into the |
| registered state. |
| |
| Using the GL to texture from, or write to, a surface in any state |
| other than mapped will yield undefined results, potentially |
| including program termination. |
| |
| Using any other API to access a VDPAU surface while that surface is |
| in the mapped state in the GL will yield undefined results, |
| potentially including program termination. |
| |
| Texture image data for VDPAU surfaces in the mapped state is |
| defined as being identical to the VDPAU surface content. |
| |
| While a VDPAU surface is in the mapped or registered states, the |
| textures associated with that surface are immutable. In other |
| words, operations that affect texture state, or attempt to replace |
| the source of the texture image data (e.g. TexImage2D) will return |
| an error. Rendering to the texture is allowed. |
| |
| Additions to Chapter 4 of the OpenGL 3.2 (unabridged) Specification |
| (Per-Fragment Operations and the Frame Buffer) |
| |
| None |
| |
| Additions to Chapter 5 of the OpenGL 3.2 (unabridged) Specification |
| (Special Functions) |
| |
| None |
| |
| Additions to Chapter 6 of the OpenGL 3.2 (unabridged) Specification |
| (State and State Requests) |
| |
| Add a new subsection to "Querying GL state" (section |
| 6.1) describing VDPAU surface object queries. |
| |
| "6.1.X VDPAU Surface Object Queries |
| |
| Properties of VDPAU surface objects may be queried using the command |
| |
| void VDPAUGetSurfaceivNV (vdpauSurfaceNV surface, |
| enum pname, |
| sizei bufSize, |
| sizei *length, |
| int *values); |
| |
| The value or values being queried are returned in the parameters |
| <length> and <values>. |
| |
| On success, VDPAUGetSurfaceivNV replaces up to <bufSize> integers |
| in <values> with the corresponding property values of the object |
| being queried. The actual number of integers replaced is returned |
| in *<length>. If <length> is NULL, no length is returned. |
| |
| If <pname> is SURFACE_STATE_NV, a single value representing the state |
| of the VDPAU surface object (SURFACE_REGISTERED_NV, |
| SURFACE_MAPPED_NV) is placed in <values>. |
| |
| If <surface> is not the name of a VDPAU surface object, an |
| INVALID_VALUE error is generated. If <pname> is not one of the |
| values described above, an INVALID_ENUM error is generated. If an |
| error occurs, nothing will be written to <values> or <length>. |
| |
| The command |
| |
| boolean VDPAUIsSurfaceNV (vdpauSurfaceNV surface); |
| |
| returns TRUE if <surface> is the name of a VDPAU surface object. If |
| <surface> is not the name of a VDPAU surface object, or if an error |
| condition occurs, IsSurface returns FALSE (note that zero is not |
| the name of a VDPAU surface object). |
| |
| Additions to the AGL/GLX/WGL Specifications |
| |
| None |
| |
| Additions to the OpenGL Shading Language |
| |
| None |
| |
| GLX Protocol |
| |
| VDPAU implementations currently only support direct-rendering. |
| Consequently, no GLX protocol is currently defined for this |
| extension. |
| |
| Dependencies on GL_ARB_texture_rectangle |
| |
| If GL_ARB_texture_rectangle is not supported, TEXTURE_RECTANGLE may |
| not be used as target for VDPAURegisterVideoSurfaceNV. |
| |
| Dependencies on GL_ARB_texture_non_power_of_two |
| |
| If GL_ARB_texture_non_power_of_two is not supported, only VDPAU |
| surfaces with power-of-two size may be used with target TEXTURE_2D. |
| |
| Errors |
| |
| INVALID_VALUE is generated if the <vdpDevice> or |
| <getProcAddressOpaque> parameters to VDPAUInitNV is NULL. |
| |
| INVALID_OPERATION is generated if VDPAUInitNV is called on a given |
| GL context, where VDPAUInitNV has been called, and VDPAUFiniNV has |
| not been called since. |
| |
| INVALID_OPERATION is generated by VDPAUInitNV, |
| VDPAURegisterVideoSurfaceNV, or VDPAURegisterOutputSurfaceNV if the |
| VDPAU driver refuses the request for some reason. |
| |
| INVALID_OPERATION is generated if VDPAUFiniNV, |
| VDPAURegisterVideoSurfaceNV, VDPAURegisterOutputSurfaceNV, |
| VDPAUIsSurfaceNV, VDPAUGetSurfaceivNV, VDPAUSurfaceAccessNV, |
| VDPAUMapSurfacesNV, or VDPAUUnmapSurfacesNV are called on a given |
| GL context, where VDPAUInitNV has not been called, or VDPAUFiniNV |
| has been called since. |
| |
| INVALID_ENUM is generated if the <target> parameter of |
| VDPAURegisterVideoSurfaceNV or VDPAURegisterOutputSurfaceNV is not |
| TEXTURE_2D or TEXTURE_RECTANGLE. |
| |
| INVALID_VALUE is generated if the <numTextureNames> parameter of |
| VDPAURegisterVideoSurfaceNV or VDPAURegisterOutputSurfaceNV does |
| not match the required value; see tables 3.8.3.1 and 3.8.3.2. |
| |
| INVALID_OPERATION is generated if any texture named by an entry |
| within the <textureNames> parameter of VDPAURegisterVideoSurfaceNV |
| or VDPAURegisterOutputSurfaceNV is marked as immutable. |
| |
| INVALID_VALUE is generated if the VDPAU surface named by the |
| <vdpSurface> parameter of VDPAURegisterVideoSurfaceNV or |
| VDPAUUnregisterOutputSurfaceNV does not have a supported format; |
| see tables 3.8.3.1 and 3.8.3.2. |
| |
| INVALID_VALUE is generated if the <surface> parameter of |
| VDPAUGetSurfaceivNV, VDPAUSurfaceAccessNV, VDPAUMapSurfacesNV, |
| VDPAUUnmapSurfacesNV is not the name of a VDPAU surface object. |
| |
| INVALID_VALUE is generated if the <surface> parameter of |
| VDPAUUnregisterSurfaceNV is neither zero nor the name of a VDPAU |
| surface object. |
| |
| INVALID_VALUE is generated if the <bufSize> parameter of |
| VDPAUGetSurfaceivNV is too small to return the results via the |
| <values> paramter. |
| |
| INVALID_ENUM is generated if the <pname> parameter of |
| VDPAUGetSurfaceivNV is not SURFACE_STATE_NV. |
| |
| INVALID_VALUE is generated if the <access> parameter of |
| VDPAUSurfaceAccessNV is none of READ_ONLY, WRITE_ONLY, nor |
| READ_WRITE. |
| |
| INVALID_OPERATION is generated if the state of the surface(s) named |
| by the <surface> parameter of VDPAUSurfaceAccessNV, or named by any |
| entry within the <surfaces> parameter of VDPAUMapSurfacesNV is |
| SURFACE_MAPPED_NV. |
| |
| INVALID_OPERATION is generated if the state of any of the surfaces |
| named by entries within the <surfaces> parameter of |
| VDPAUUnmapSurfacesNV is not SURFACE_MAPPED_NV. |
| |
| New State |
| |
| Table 6.X. VDPAU Surface Objects. |
| |
| Get value Type Get command Initial value Description Section |
| ------------------ ---- ----------- ---------------------------- --------------- ------- |
| SURFACE_STATE_NV Z VDPAUGetSurfaceivNV GL_SURFACE_REGISTERED_NV Surface state 3.8.3 |
| |
| New Implementation State |
| |
| None |
| |
| Issues |
| |
| 1. Should YUV surfaces be exposed as a frame Y and frame UV |
| texture, or two field Y and two field UV textures? |
| |
| RESOLVED: YUV surfaces will be exposed as separate fields |
| |
| Exposing the surface as separate fields allows applications to |
| directly implement de-interlacing algorithms other than weave |
| without having to undo the weaving of fields together. Put |
| another way: |
| |
| Frames: |
| |
| Pros: |
| |
| * Simple applications can directly access the entire frame, |
| without having to manually implement weave de-interlacing. |
| This also applies for progressive content. |
| |
| However, simple applications probably won't want to interop |
| on a VdpVideoSurface, but will rather rely on VDPAU's |
| VdpVideoMixer for post-processing, and instead interop on a |
| VdpOutputSurface. |
| |
| Cons: |
| |
| * In order to implement bob de-interlacing, vertical |
| interpolation between lines of a single field is required. |
| The GL's texturing does not allow interpolation to skip |
| lines in a texture. Consequently, the two fields must be |
| manually separated by the application using a shader and |
| some texture co-ordinate calculation prior to bobbing. This |
| forces a copy operation, which would reduce performance. |
| |
| * In NVIDIA's implementation, VDPAU surfaces are stored as |
| separate fields. Consequently, VDPAUMapSurfacesNV would have |
| to perform a copy operation, which would reduce performance. |
| |
| Fields: |
| |
| Pros: |
| |
| * Advanced de-interlacing and post-processing algorithms get |
| direct access to individual fields. |
| |
| * In NVIDIA's implementation, no copying is required during |
| VDPAUMapSurfacesNV. |
| |
| Cons: |
| |
| * Simple applications that simply want to weave, or for |
| progressive content, must manually combine the fields. |
| However, this operation is relatively simple to code, and |
| sample code will be provided. |
| |
| 2. Should Map/Unmap functions be removed, such that the GL |
| implements those operations transparently at appropriate times? |
| |
| RESOLVED: No, map/unmap functions will be kept. |
| |
| The map/unmap functions provide a convenient place to perform |
| synchronization between VDPAU and the GL command streams, and |
| any format-converting copies that may be required by the GL or |
| VDPAU implementation. Without explicit application-controlled |
| APIs, it would potentially be difficult to trigger those |
| operations at the appropriate time. It would also be less |
| obvious to software developers when such operations were |
| occurring, which may prevent successful investigation of |
| interop performance. |
| |
| 3. Should SurfaceAccess function be removed, and instead an |
| "access" parameter added to the Map function? |
| |
| RESOLVED: No, a separate function will be kept. |
| |
| The value of this flag typically will not change on a per-frame |
| basis, so specifying it for each map call was deemed redundant. |
| For an advanced application that needs to change the flag often, |
| the overhead of using a separate function should not be too |
| significant. |
| |
| Revision History |
| |
| 1. 23 Dec 2009 - Stephen W |
| Initial version |
| |
| 2. 5 Jan 2010 - Stephen W |
| -Renamed VDPAUSetSurfaceAccessNV to VDPAUSurfaceAccessNV thus |
| avoiding the word Set. |
| -Fixed/wrote "Dependencies", "Dependencies on ..." sections. |
| -Initial version of "Additions to Chapter 3". |
| -Fill in answers for issues 1-3. |
| |
| 3. 7 Jan 2010 - Stephen W |
| -Dropped the GL prefix from type names. |
| -Refer to "the GL" not "GL". |
| -Removed XXX re: PROXY_TEXTURE_*; those targets don't seem |
| useful for this extension. |
| -Use <> around function parameter names in text. |
| -Reword description of legal values for <numTextureNames>. |
| Number, title, and reformat related tables. |
| Rename "format" column to "internal format". |
| -Describe VDPAUSurfaceAccessNV before Map/Ummap for more |
| logical ordering. |
| -Significant text rewrite to more explicitly describe states, |
| state transitions, legal access to surfaces by various APIs |
| during specific states, etc. |
| -Explicitly state that registered/mapped textures are |
| immutable. |
| -Re-used VBO's READ/WRITE/READ_WRITE enums instead of inventing |
| new enums. |
| -Added explicit type GLvdpauSurfaceNV instead of using GLvoid*. |
| -Added VDPAUIsSurface, VDPAUGetSurfaceiv, and associated |
| tokens. |
| -Documented state, state queries, and errors. |
| |
| 4. 13 Jan 2010 - Stephen W |
| - Added token WRITE_DISCARD_NV. Stated semantics of each legal |
| value of VDPAUSurfaceAccessNV's <access> parameter. |
| |
| 5. 13 Jan 2010 - Stephen W |
| - Removed some XXXs. Marked spec as complete. |
| |
| 6. 18 Jan 2010 - Stephen W |
| - Added missing NV suffix to VDPAUIsSurfaceNV, |
| VDPAUGetSurfaceivNV |
| |
| 7. 18 Jan 2010 |
| - Made various function parameters const. |
| |
| 8. 24 Feb 2010 |
| - Fixed typo in table 3.8.3.1; the heights of luma surfaces |
| were incorrectly specified. |
| |
| 9. 29 May 2010 |
| - Fix minor typos and formatting issues. |
| |
| 10. 16 Jun 2010 |
| - Added "New Types" section |
| |
| 11. 2 Jan 2014 |
| - Fixed return type of VDPAUIsSurfaceNV and remove it from the |
| list of commands that generate INVALID_VALUE. |
| |