Name OVR_multiview Name Strings GL_OVR_multiview Contact Cass Everitt, Oculus (cass.everitt 'at' oculus.com) Contributors John Carmack, Oculus Tom Forsyth, Oculus Maurice Ribble, Qualcomm James Dolan, NVIDIA Corporation Mark Kilgard, NVIDIA Corporation Michael Songy, NVIDIA Corporation Yury Uralsky, NVIDIA Corporation Jesse Hall, Google Timothy Lottes, Epic Jan-Harald Fredriksen, ARM Jonas Gustavsson, Sony Mobile Sam Holmes, Qualcomm Nigel Williams, Qualcomm Status Incomplete. Version Last Modified Date: July 1, 2016 Author Revision: 0.6 Number OpenGL Extension #478 OpenGL ES Extension #241 Dependencies OpenGL 3.0 or OpenGL ES 3.0 is required. Overview The method of stereo rendering supported in OpenGL is currently achieved by rendering to the two eye buffers sequentially. This typically incurs double the application and driver overhead, despite the fact that the command streams and render states are almost identical. This extension seeks to address the inefficiency of sequential multiview rendering by adding a means to render to multiple elements of a 2D texture array simultaneously. In multiview rendering, draw calls are instanced into each corresponding element of the texture array. The vertex program uses a new ViewID variable to compute per-view values, typically the vertex position and view-dependent variables like reflection. The formulation of this extension is high level in order to allow implementation freedom. On existing hardware, applications and drivers can realize the benefits of a single scene traversal, even if all GPU work is fully duplicated per-view. But future support could enable simultaneous rendering via multi-GPU, tile-based architectures could sort geometry into tiles for multiple views in a single pass, and the implementation could even choose to interleave at the fragment level for better texture cache utilization and more coherent fragment shader branching. The most obvious use case in this model is to support two simultaneous views: one view for each eye. However, we also anticipate a usage where two views are rendered per eye, where one has a wide field of view and the other has a narrow one. The nature of wide field of view planar projection is that the sample density can become unacceptably low in the view direction. By rendering two inset eye views per eye, we can get the required sample density in the center of projection without wasting samples, memory, and time by oversampling in the periphery. New Tokens Accepted by the parameter of GetFramebufferAttachmentParameteriv: FRAMEBUFFER_ATTACHMENT_TEXTURE_NUM_VIEWS_OVR 0x9630 FRAMEBUFFER_ATTACHMENT_TEXTURE_BASE_VIEW_INDEX_OVR 0x9632 Accepted by the parameter of GetIntegerv: MAX_VIEWS_OVR 0x9631 Returned by CheckFramebufferStatus: FRAMEBUFFER_INCOMPLETE_VIEW_TARGETS_OVR 0x9633 New Procedures and Functions void FramebufferTextureMultiviewOVR( enum target, enum attachment, uint texture, int level, int baseViewIndex, sizei numViews ); Additions to Chapter X of the OpenGL 3.0 Specification Additions to Chapter 4 of the OpenGL ES 3.0 Specification The command FramebufferTextureMultiviewOVR( enum target, enum attachment, uint texture, int level, int baseViewIndex, sizei numViews ); operates similarly to FramebufferTextureLayer, except that baseViewIndex and numViews selects a range of texture array elements that will be targeted when rendering. The command View( uint id ); does not exist in the GL, but is used here to describe the multi-view functionality in this section. The effect of this hypothetical function is to set the value of the shader built-in input uint gl_ViewID_OVR. When multi-view rendering is enabled, drawing commands have the same effect as: for( int i = 0; i < numViews; i++ ) { FramebufferTextureLayer( target, attachment, texture, level, baseViewIndex + i ); View( i ); } The result is that every drawing command is broadcast into every active view. The shader uses gl_ViewID_OVR to compute view dependent outputs. The number of views, as specified by numViews, must be the same for all framebuffer attachments points where the value of FRAMEBUFFER_ATTACHMENT_- OBJECT_TYPE is not NONE or the framebuffer is incomplete. The following is added to the conditions for Whole Framebuffer Completeness: The number of views is the same for all populated attachments. { FRAMEBUFFER_INCOMPLETE_VIEW_TARGETS_OVR } In this mode there are several restrictions: - in vertex shader gl_Position is the only output that can depend on ViewID - no transform feedback - no tessellation control or evaluation shaders - no geometry shader - no timer query - occlusion query results must be between max and sum of per-view queries, inclusive Modifications to The OpenGL ES Shading Language Specification, Version 3.00.04 Including the following line in a shader can be used to control the language features described in this extension: #extension GL_OVR_multiview In section 4.3.8.1 "Input Layout Qualifiers": Insert a paragraph before the final one ("Fragment shaders cannot ..."): Vertex shaders also allow the following layout qualifier on "in" only (not with variable declarations) layout-qualifier-id num_views = integer-constant to indicate that the shader will only be used with the given number of views, as described in section 4.4 ("Framebuffer Objects") of the OpenGL ES Specification. If this qualifier is not declared, the behavior is as if it had been set to 1. Errors INVALID_FRAMEBUFFER_OPERATION is generated by commands that read from the framebuffer such as BlitFramebuffer, ReadPixels, CopyTexImage*, and CopyTexSubImage*, if the number of views in the current read framebuffer is greater than 1. INVALID_VALUE is generated by FramebufferTextureMultiviewOVR if numViews is less than 1, if numViews is more than MAX_VIEWS_OVR or if (baseViewIndex + numViews) exceeds GL_MAX_ARRAY_TEXTURE_LAYERS. INVALID_OPERATION is generated if a rendering command is issued and the the number of views in the current draw framebuffer is not equal to the number of views declared in the currently bound program. Issues (1) Should geometry shaders be allowed in multiview mode? Resolved: Not in this extension. By disallowing it, we hope to enable more implementations to be available sooner, and there are complex issues that arise when a GS is used to target gl_Layer explicitly. (2) Should there be separate scissor and viewport per view? Resolved: No, while there might be some uses for such support, it adds unnecessary implementation complexity. In the case of inset rendering, there will be a need to adjust the scissor per view. We will defer that issue for now by forcing all views to use the same scissor and viewport. (3) Why not just use geometry shaders? Resolved: GS could be used to achieve stereo rendering by broadcasting each primitive to each view. The problem with this approach is that it requires the GS's very general mechanism with known performance implications to solve a problem that does not require that solution and perhaps more frequently than not, will not be the most efficient means of implementation. (4) Why use texture arrays instead of separate FBOs? Resolved: Use of arrays does imply that we use a minimum version of GL and ES 3.0. On the other hand, it has some nice simplifying properties. In particular, the format and resolution of each view is known to be the same and only one FBO is bound, just like with normal rendering. It has some potentially limiting interactions with GS use, but on the whole, the implementation simplifications are considered worth the implied limitations. (5) How does this extension interact with occlusion queries, timer queries, etc? Resolved: The bias will be toward relaxed rules to allow implementation freedom. For example, occlusion queries should not return fewer than the max samples returned from any view, but returning the sum may also be fine. Simply reporting the result from view 0 is not sufficient. (6) Is ViewID visible at every pipeline stage? Resolved: To make integration simple for app developers, the intent is for ViewID to be visible as a built-in at each programmable pipeline stage. (7) Are view-dependent parameters exposed explicitly? Resolved: No. This is implicit in the dependence of a parameter on ViewID. In this extension, however, only gl_Position is allowed to depend on ViewID in the vertex shader. If a shader violates this restriction it results in undefined behavior or may produce an error. Later extensions may relax that restriction. (8) Should the parameters that affect view-dependent position be driver visible or otherwise restricted? Resolved: No. Letting the app index via ViewID makes it a lot harder for the driver to know much about the details of the transform inputs that result in view-dependent outputs, but no such support exists in the API today either. Adding that support would be complex, and thus not in the spirit of an extension that can be implemented broadly with low risk. (9) Should there be a per-view UBO instead of exposing ViewID in the shader? Resolved: No. Exposing the ViewID variable is the smallest change. It does imply dynamic branching, predication, or indexing based on the view id, however, implementations could compile separate versions of the shader with view id translated to literals if that would improve performance. Extra API and machinery would be required to sequence one or more UBOs to different objects per view. (10) Should there be DSA style entry points? Resolved: Almost certainly in GL 4.5 and beyond. Less clear that it should be required for ES 3.0. (11) Should tessellation shaders be allowed in multiview mode? Resolved: Not in this extension. By disallowing it, we hope to enable more implementations to be available sooner. (12) Does the number of views need to be declared in the shader? Resolved: Yes. This enables the implementation to specialize the compilation for the declared number of views, for example by loop-unrolling. It could also serves as a notification that the shader outputs must be 'broadcast' to a set of views - even in cases where the shader does not reference from ViewID. (13) What should happen if the number of views declared in the shader does not match num_views? Resolved: Such a mismatch can only be detected at draw call time, so it would have to either generate a draw call time error, or just result in undefined results for any view > num_views. To avoid undefined results, the draw call time error is preferred. (14) How should read operations on FBOs with multiview texture attachments be handled? Resolved: Generating INVALID_OPERATION when the target is GL_READ_FRAMEBUFFER for FramebufferTextureMultiviewOVR to disallow attaching a multiview texture to the current read framebuffer is not sufficient to prevent a read operation on a FBO with a multiview texture attachment. The multiview texture may still be attached to the current write framebuffer and then bound as the read framebuffer. Instead, an INVALID_FRAMEBUFFER_OPERATION error should be generated for any read operations that occur whenever a multiview texture is attached to the current read framebuffer. The multiview texture needs to be instead attached as a 2D texture array with the level explicitly specified to be read. This conforms to the expected behavior for read operations on FBOs with multiview texture attachments to be consistent with FramebufferTextureMultisampleMultiviewOVR. Revision History Rev. Date Author Changes ---- -------- -------- ----------------------------------------- 0.1 10/17/14 cass Initial draft 0.2 11/10/14 cass Changes to use texture array instead of separate FBOs 0.3 02/05/15 cass Only gl_Position can be view dependent in the vertex shader. 0.4 02/11/15 cass Switch to view instead of layer, as these are distinct now 0.5 04/15/15 cass Clean up pass before publishing 0.6 07/01/16 nigelw Modify errors to conform to multisample multiview spec changes