DE102019103336A1 - METHOD FOR EFFICIENTLY GROUPING CACHE REQUIREMENTS FOR DATA PATH SCHEDULING - Google Patents

Abstract

In a ray tracker, a workload streaming cache groups ray requests for coherent successive bounding volume hierarchy traversal operations by sending shared data along an attached data path to all ray requests in the group at the same time or at about the same time. Grouping the requests provides good performance with a smaller number of cache lines.

Description

Cross-reference to related patents and applications

• • US 16/101,066 mit dem Titel „Method for Continued Bounding Volume Hierarchy Traversal on Intersection without Shader Intervention“;
• • US 16/101,247 mit dem Titel „A Robust, Efficient Multiprocessor-Coprocessor Interface“;
• • US 16/101,180 mit dem Titel „Query-Specific Behavioral Modification of Tree Traversal“;
• • US 16/101,148 mit dem Titel „Conservative Watertight Ray Triangle Intersection“;
• • US 16/101,196 mit dem Titel „Method for Handling Out-of-Order Opaque and Alpha Ray/Primitive Intersections“; und
• • US 16/101,232 mit dem Titel „Method for Forward Progress and Programmable Timeouts of Tree Traversal Mechanisms in Hardware“.

This application relates to the following commonly assigned U.S. patents and patent applications, the entire contents of which are incorporated by reference: U.S. Application No. 14 / 563,872 entitled "Short Stack Traversal of Tree Data Structures", filed on December 8, 2014; U.S. Patent No. 9,582,607 entitled “Block-Based Bounding Volume Hierarchy”; U.S. Patent No. 9,552,664 entitled "Relative Encoding For A Block-Based Bounding Volume Hierarchy"; U.S. Patent No. 9,569,559 entitled "Beam Tracing", filed on March 18, 2015; U.S. Patent No. 10,025,879 with the title "Tree Data Structures Based on a Plurality of Local Coordinate Systems"; U.S. Application No. 14 / 737,343 entitled "Block-Based Lossless Compression of Geometry Data", filed on June 11, 2015; and the following U.S. applications filed hereby concurrently:

• • US 16 / 101,066 entitled “Method for Continued Bounding Volume Hierarchy Traversal on Intersection without Shader Intervention”;
• • US 16 / 101,247 with the title "A Robust, Efficient Multiprocessor-Coprocessor Interface";
• • US 16 / 101,180 entitled "Query-Specific Behavioral Modification of Tree Traversal";
• • US 16 / 101,148 entitled “Conservative Watertight Ray Triangle Intersection”;
• • US 16 / 101,196 with the title "Method for Handling Out-of-Order Opaque and Alpha Ray / Primitive Intersections"; and
• • US 16 / 101,232 with the title "Method for Forward Progress and Programmable Timeouts of Tree Traversal Mechanisms in Hardware".

area

The present technology relates to computer graphics and in particular to ray tracers. In particular, the technique relates to hardware acceleration of computer graphics processing, including but not limited to ray tracing. More specifically, the non-limiting example technique herein relates to a hardware-based traversal coprocessor that efficiently traverses an acceleration data structure, e.g. B. for real-time beam tracking. More specifically, the technology herein provides an improved hardware-based scheduling cache for ray tracing bounding volume hierarchy traversal. The technique herein has the advantage that beam requests and associated tests are scheduled in accordance with the same address grouping to shorten the time that data must reside in a data RAM so that other data is streamed across the cache can be. This enables a reduced cache size. Smaller area with high performance enables more area to be better used for data path improvements and other improvements in performance and / or storage management.

Background & abstract

If you look around in the visual scene in front of you, you will find that some of the most interesting visual effects you see are created by rays of light that interact with surfaces. Because light is the only thing we see. We don't see objects - we see the light that is reflected or refracted by the objects. Most of the objects we can see reflect light (the color of an object is determined by which parts of the light the object reflects and which parts it absorbs). Shiny surfaces such as B. metallic surfaces, shiny surfaces, ceramics, liquid surfaces and a variety of others (including the corneas of the human eyes) act as mirrors that reflect light in a specular manner. For example, a shiny metal surface reflects light at the same angle as it hits the surface. An object can also cast shadows by preventing light from hitting other surfaces that are behind the object relative to a light source. If you look around, you will find that the number and type of reflections and the number, type and length of shadows depend on many factors, including the number and type of lights in the scene. A single point light source such as B. a single distant light bulb creates single reflections and hard shadows. Area light sources such as B. Windows or panel lights produce other types of reflection highlights and softer shadows. Multiple light sources typically produce multiple reflections and more complex shadows (e.g., three separate point light sources create three shadows, which may overlap depending on the position of the light sources relative to an object).

If you move your head while looking at the scene, you will notice that the reflections change in position and shape (the shadows do the same). By changing your point of view, you are changing the different angles of the light rays that your eyes recognize. This happens instantly - you move your head and the visual scene changes instantly.

The simple act of drinking a cup of tea is a complex visual experience. The various shiny surfaces of the shiny ceramic cup on the table in front of you reflect every light in the room, and the cup casts a shadow for every light. The moving surface of the tea in the cup is self-reflecting. You can see small reflected images of the light sources on the surface of the tea, and even smaller reflections on the part of the surface of the tea where the liquid hits the walls of the cup. The cup walls also cast shadows on the surface of the liquid in the cup. Lifting the cup to your mouth causes these reflections and shadows to shift and shimmer when your position changes and the surface of the liquid is moved by movement.

We take this complexity of reflections and shadows for granted. Our brains are adept at deciphering the positions, sizes, and shapes of shadows and reflections and using them as visual cues. In part, we recognize the position of objects, how we distinguish one object from another, and how we learn what objects are made of. Different object surfaces reflect differently. Specular (mirror-like) reflection from hard metal creates images of reflected objects, while diffuse reflection on rough surfaces is responsible for color and illuminates objects more softly. Shadows can be soft and diffuse or hard and clear depending on the type of lighting, and the length and direction of the shadows depends on the angle of the light rays in relation to the object and our eyes.

Beginner artists typically don't try to show reflections or shadows. They tend to draw flat scenes that have no shadows and no reflections or highlights. The same was true for computer graphics of the past.

Real-time computer graphics has evolved enormously in the past 30 years. With the development of powerful graphics processors (GPUs) in the 1980s that provide 3D hardware graphics pipelines, it became possible to generate 3D graphics displays based on texture-mapped polygon primitives in real time in response to user input. Such real-time graphics processors were based on a technique called scan conversion rasterization, which is a means of determining visibility from a single point or perspective. Using this method, three-dimensional objects are modeled from surfaces that are made up of geometric primitives, typically polygons such as z. B. triangles. The scanning signal conversion process creates and projects primitive polygon vertices onto a view plane and fills the points within the edges of the primitive. See e.g. B. Foley, Van Dam, Hughes et al, Computer Graphics: Principles and Practice (2nd Edition Addison-Wesley 1995 & 3rd Edition Addison-Wesley 2014).

Hardware has long been used to determine how to shade and texture map each polygon area, and to rasterize the shaded texture-mapped polygon areas for display. Typical three-dimensional scenes are often constructed from millions of polygons. Fast modern GPU hardware can efficiently process millions of graphics primitives for each display frame (every 1/30 or 1/60 second) in real time in response to user input. The resulting graphics displays have been used in a variety of real-time graphical user interfaces, including but not limited to augmented reality, virtual reality, video games, and medical imaging. Traditionally, such interactive graphics hardware has not been able to accurately model and display reflections and shadows.

Some have built other techniques on this basic scanning conversion method to enable real-time graphics systems to achieve some level of realism in rendering shadows and reflections. For example, texture mapping has sometimes been used to simulate reflections and shadows in a 3D scene. One way to accomplish this is to transform, project, and rasterize objects from different perspectives, write the rasterized results in texture maps, and sample the texture maps to allow reflection mapping, environment mapping, and shadow mapping. Although these techniques have proven useful and moderately successful, they do not work well in all situations. For example, so-called “environment mapping” can often be required if the environment is infinitely far from the object. In addition, an environment-mapped object may typically not be able to reflect itself. See e.g. B. http://developer.download.nvidia.com/CgTutorial/cg_tutorial_chapter07.html. These limitations result from the fact that conventional computer graphics hardware - although fast enough for excellent polygon rendering - does not perform the light visualization necessary for accurate and realistic reflections and shadows. Some have compared raster / text approximations of reflections and shadows as the visual equivalent of AM radio.

There is another graphic technique that performs physically realistic visibility determinations for reflection and shadow formation. It is referred to as "ray tracing". Beam tracing was developed in the late 1960s and improved in the 1980s. See e.g. B. Apple, "Some Techniques for Shading Machine Renderings of Solids" (SJCC 1968) pp. 27-45; Whitted, "An Improved Illumination Model for Shaded Display" pp. 343-349 Communications of the ACM, Volume 23, Issue 6 (June 1980); and Kajiya, "The Rendering Equation", Computer Graphics (SIGGRAPH 1986 Proceedings, Volume 20, pages 143-150). Since then, beam tracing has been used in non-real-time graphics applications such as B. Design and film making used. Anyone who has seen “Finding Dorie” (2016) or other Pixar animated films has seen the result of the ray tracing method for computer graphics - namely realistic shadows and reflections. See e.g. B. Hery et al., Towards Bidirectional Path Tracing at Pixar (2016) ,

Ray tracing is a basic principle used in a variety of rendering algorithms, including path tracing and Metropolis Light Transport, for example. In an example algorithm, ray tracing simulates the physics of light by modeling the light transport through the scene to calculate all global effects (including, for example, reflections from shiny surfaces) using ray optics. In such beam tracking applications, one can try to track each of many hundreds or thousands of light rays as they travel through the three-dimensional scene from potentially multiple light sources to the viewpoint. Such rays are often tracked through the scene relative to the eye and tested against a database of all geometries in the scene. The rays can be traced forward from light sources to the eye, or backward from the eye to the light sources, or they can be traced to see if paths starting on the virtual camera and starting on the eye have a clear line of sight. Testing either determines the closest intersection (to determine what is visible to the eye) or tracks rays from the surface of an object to a light source to determine if there is anything in between that is the transmission of light to that point in space would block. Because the rays are similar to the rays of light in reality, they provide a number of realistic effects that are not possible with the raster-based real-time 3D graphics technology that has been implemented in the past thirty years. Because each illuminating beam is evaluated by every light source within the scene as it traverses every object in the scene, the resulting images can look as if they were photographed in reality. Accordingly, these beam tracing methods have long been used in professional graphics applications such as. B. Design and film used where they now dominate over raster-based rendering.

In general, the biggest challenge with ray tracing was speed. Beam tracking requires that the graphics system calculate and analyze for each frame each of millions of light rays that strike (and possibly reflect on) every surface of the scene. In the past, it was impossible to do this enormous amount of computational complexity in real time.

One reason modern GPU 3D graphics pipelines render shaded, texture-mapped surfaces so quickly is that they use coherence efficiently. With conventional scanning signal conversion, it is assumed that everything is viewed through a common window in a common image plane and projected down to a single point of view. Each triangle or other primitive is sent through the graphics pipeline and covers a number of pixels. All related calculations can be shared across all pixels rendered from this triangle. Rectangular pixel tiles that correspond to coherent lines of sight that pass through the window can thus correspond to groups of threads that run in step in the same streaming processor. All pixels that fall between the edges of the triangle are assumed to be the same material that runs the same shader (in German "shader") and retrieves neighboring groups of texels from the same textures. In contrast, in ray tracing, rays can start or end at a common point (a light source or a virtual camera lens), but as they spread through the scene and interact with different materials, they quickly diverge. For example, each ray searches for the closest object. Caching and sharing of results can be done, but since each beam can potentially hit different objects, the type of coherence that GPUs have traditionally used in conjunction with texture-depicted shaded triangles is absent (e.g., a shared one View angle, window and image plane are not available for ray tracing). This makes beam tracing much more computationally intensive than other graphics methods - and therefore much more difficult to perform interactively.

A lot of research has been done to make the process of tracking rays more efficient and timely. See e.g. B. Glassner, An Introduction to Ray Tracing (Academic Press Inc., 1989). Because every ray in ray tracing is naturally assessed independently of the rest, beam tracking has been described as "uncomfortably parallel". See e.g. B. Akenine-Möller et al., Real Time Rendering in Section 9.8.2, page 412 (Third Ed. CRC Press 2008). As mentioned above, beam tracing effectively involves testing each beam against all objects and surfaces in the scene. Optimization called "acceleration data structure" and associated processes enable the graphics system to use a "divide and conquer" method across the acceleration data structure to determine which surfaces the beam hits and which surfaces the beam does not. Each beam traverses the acceleration data structure in an individualistic way. This means that dedicating more processors to beam tracing enables an almost linear increase in performance. With the increasing parallelism of graphics processing systems, some have begun to imagine the possibility that ray tracing could be done in real time. For example, an early special hardware system for interactive beam tracing was developed at the Saarland University in the mid-2000s, which offered a certain programmability for the use of geometry, vertex and lighting shaders. See Woop et al., "RPU: A Programmable Ray Processing Unit for Real Time Ray Tracing" (ACM 2005). As another example, Advanced Rendering Technology developed “RenderDrive” based on an array of AR250 / 350 render processors derived from ARM1 and expanded with customer-specific pipelines for beam / triangle intersection and SIMD vector and texture mathematics , but without traversing logic with a fixed function. See e.g. B. http://www.graphicshardware.org/previous/ www_2001 / presentations / Hot3D_Daniel_Hall.pdf

In 2010, NVIDIA then used the high degree of parallelism of NVIDIA GPUs and other highly parallel architectures to develop the OptiX ™ ray tracing engine. See Parker et al., "OptiX: A General Purpose Ray Tracing Engine" (ACM Transactions on Graphics, Volume 29, No. 4, Article 66, July 2010). In addition to improvements to APIs (application programming interfaces), one of the advances in OptiX ™ has been the improvement of the acceleration data structures used to find an intersection between a beam and the scene geometry. Such acceleration data structures are typically spatial or object hierarchies that are used by the ray tracing traversal algorithm to efficiently search for primitives that potentially intersect a given ray. OptiX ™ provides a number of different types of acceleration structures that the application can choose from. Each acceleration structure in the node diagram can be a different type, which enables combinations of high-quality static structures with dynamically updated structures.

The OptiX ™ programmable ray tracing pipeline made significant progress, but was still not in itself able to provide real-time interactive responses to user input on relatively inexpensive computer platforms for complex 3D scenes. Since then, NVIDIA has developed hardware acceleration capabilities for beam tracing. See e.g. B. US 9,582,607 ; US 9,569,559 ; US 20160070820 ; and US 20160070767 ,

Given the great potential of a truly interactive, real-time ray tracing graphics processing system to render high quality images of any complexity in response to, for example, user input, further work is possible and desirable.

list of figures

• 1 illustrates an example of a non-limiting ray tracing graphics system.
• 2A shows an example of a specular object.
• 2 B shows the sample object within a bounding volume.
• 2C shows a volumetric example subdivision of the bounding volume of 2 B ,
• 2D . 2E and 2F show examples of further levels of volumetric subdivision of the limitation volume to create a limitation volume hierarchy (BVH).
• 2G shows an example part of the object, which consists of primitive surfaces, in this case triangles.
• 3A-3C show exemplary simplified ray tracing tests to determine whether the ray traverses a confinement volume containing geometry and whether the ray intersects geometry.
• 4 illustrates an example ray tracing flowchart.
• 5A-5C show examples of different scenarios for beam-primitive intersections.
• 6A and 6B show an example of how texture mapping can affect ray-primitive intersection results.
• 7A and 7B illustrate ray instance transformations.
• 8A illustrates a non-limiting example limitation volume hierarchy (BVH).
• 8B shows an example acceleration data structure in the form of a graph or tree.
• 9 shows a non-limiting, simplified example traversal coprocessor having a tree traversal unit (TTU).
• 10 illustrates a flow diagram of a non-limiting example ray tracing shading pipeline.
• 11B and 11B illustrate more detailed ray tracing pipelines.
• 12 Figure 3 shows a non-limiting example of data maintained by the traversal coprocessor L0 cache;
• 13 Fig. 4 is a flow diagram of example operations performed by the traversal coprocessor L0 cache;
• 14 illustrates an example flow diagram for generating an image.
• 15 illustrates an example parallel processing unit (PPU).
• 16 illustrates an example storage partition unit.
• 17 illustrates an example general processing cluster (GPC) within the parallel processing unit of FIG 15 ,
• eighteen is a conceptual diagram of one using the GPC of FIG 17 implemented graphics processing pipeline.
• 19 and 20 illustrate an example streaming multiprocessor.
• 21 is a conceptual diagram of one using PPUs from 15 implemented processing system.
• 22 expanded 21 to show additional connected devices.

Detailed description of non-limiting embodiments

The technology herein provides hardware capabilities that accelerate ray tracing to the point that games and other real-time interactive computer graphics provide beam tracing performance, initially providing high quality shadow and reflection effects, and ultimately global lighting. In practice, this means beam tracing by up to an order of magnitude or more compared to what would be possible in software on the same graphics render system.

In particular, the non-limiting example technique provides dedicated hardware for accelerating beam tracing. In non-limiting embodiments, a hardware coprocessor (referred to herein as a "traversing coprocessor" or in some embodiments a "tree traversing unit" or "TTU") speeds up certain processes that support interactive beam tracing, including beam confinement volume intersection tests, beam primitive intersection tests, and beam "instance" transformations.

In some non-limiting embodiments, the traversal coprocessor queries an acceleration data structure for processes that run on potentially massively parallel streaming multiprocessors (SMs). The traversal coprocessor traverses the acceleration data structure to find information about how a given beam interacts with an object that describes or represents the acceleration data structure. The traversing coprocessors can be called up for beam tracking. B. Fixed function units that perform an operation once between logical pipeline stages that run different types of threads (e.g., vertex threads and pixel threads).

In some non-limiting embodiments, the acceleration data structure includes a hierarchy of delimitation volumes (delimitation volume hierarchy or BVH) that encapsulates delimitation volume subdivisions that become smaller and smaller recursively. The largest boundary volume can be referred to as the "root node". The smallest subdivisions of such a hierarchy of delimitation volumes ("leaf nodes") contain elements. The elements could be primitives (e.g. polygons such as triangles) that define surfaces of the object. Or an element could be an area that contains a whole new level of the world that exists as an element since it has not been added to the BVH (think of the collar pendant on the cat from "Men in Black", which is a whole Miniature galaxy contained therein). If the element includes primitives, the traversal coprocessor tests rays against the primitives to determine which object surfaces intersect the rays and which object surfaces are visible along the ray.

The traversal coprocessor tests each beam against a wide range of confinement volumes and can discard any confinement volumes that do not intersect with that beam (one Undergo culling). Starting at a root node that delimits everything in the scene, the traversing coprocessor tests each beam against smaller (potentially overlapping) child delimitation volumes, which in turn delimit the subsequent branches of the BVH. The beam follows the child pointers for the boundary volumes that the beam strikes to other nodes until the leaves or end nodes (volumes) of the BVH are reached. Once the traversal coprocessor traverses the acceleration data structure to reach an end or "leaf" node that contains a geometric primitive, it performs an accelerated beam-primitive intersection test that determines whether the beam is that primitive (and thus intersects the object surface defined by this primitive). The beam primitive test can provide additional information about primitives that the beam intersects, which can be used to determine the material properties of the surface that are required for shading and visualization. Recursive traversal through the acceleration data structure allows the traversal coprocessor to discover all of the object primitives that the beam intersects, or the closest (from the point of view perspective) primitive that the beam intersects (which in some cases is the only primitive, that is visible from the viewpoint along the beam).

The traversal coprocessor also accelerates the transformation of each ray from world space to object space to keep the primitive bounding box encapsulations becoming finer and to reduce the duplication of these primitives throughout the scene. Objects that are replicated many times in the scene in various positions, orientations, and scales can act as instance nodes in the scene that map a transformation to a bounding box and leaf nodes in the World Space BVH that apply to the World Space ray can be transformed to an object coordinate space and a pointer to an object-space BVH can be displayed. This prevents the object-space BVH data from being replicated multiple times in the world space, which saves memory and associated memory access. Instance transformation increases efficiency by transforming the beam into the object space instead of having to transform the geometry or the bounding volume hierarchy into the world space (beam space), and is also compatible with additional, conventional screening processes that graphics processing to visualize the primitives.

The non-limiting embodiments disclosed herein thus provide a traversal coprocessor, a new subunit of one or a group of streaming multiprocessors SM of a 3D graphics processing pipeline. To understand where the traversal coprocessor fits into the big picture, it can be helpful to understand some basics of the algorithm used by most or all modern ray trackers. Note, however, that the technique herein provides a generic ability to determine what is the closest visible thing from a given point along a specified direction to a thread running in a GPU, or whether anything is between two points. A common use for such a capability will be in processes that are beginning to track rays from points that have already been scanned onto triangles using conventional scanning signal conversion techniques. The disclosed technique can, but does not necessarily have to, replace and substitute the scan signal conversion technique, and can often supplement it, and can be used in conjunction with scan signal conversion techniques to enhance images with photo-realistic reflections, shadows, and other effects.

Ray tracing techniques

Generally, ray tracing is a rendering technique that uses rays to determine the visibility of various elements in the scene. Ray tracing can be used to determine if anything is visible along a ray (for example, testing for obscurations between a shadowed point on a geometric primitive and a point on a light source) and can also be used to evaluate reflections (for example performing traversal to determine the closest visible surface along a line of sight so that software running on a streaming processor can evaluate a material shading function according to what has been hit - which in turn adds one or more additional rays into the Scene according to the material properties of the object that has been cut) to determine the light that comes back to the eye along the beam. In classic whitted-style ray tracing, rays are shot into the scene from the viewpoint through the pixel grid, but other path traversals are also possible. The closest object is typically found for each beam. This intersection can then be determined to be illuminated or in the shade by shooting a beam from it to each light source in the scene and determining if there are any objects in between. Opaque objects block the light, while transparent objects dim it. Other rays can be created from an intersection. For example, if the cutting surface is shiny or reflective rays are generated in the reflection direction. The beam can take on the color of the first cut object, the intersection of which has again been tested for shadows. This reflection process is repeated recursively until a recursion limit is reached or the potential contribution of subsequent impacts falls below a threshold. Rays can also be generated in the direction of refraction for transparent solid objects and can be evaluated recursively. See Akenine-Möller et al., Cited above. The ray tracing technique thus enables a graphics system to develop physically correct reflections and shadows that are not subject to the limitations and artifacts of scanning signal conversion techniques.

Traversierungs coprocessor

The basic task that the traversal coprocessor performs is to test a beam against all primitives (usually triangles in one embodiment) in the scene and, depending on the application, either the closest hit (corresponding to the distance measured along the beam) or simple report the first (not necessarily closest) hit. The natural algorithm would be an O (n) brute force search. However, by preprocessing the scene geometry and building a suitable acceleration data structure in advance, it is possible to reduce the complexity to O (log n) on average. For ray tracing, the time to find the closest (or any shadow) intersection for a ray is typically on the order of O (log n) for n objects using an acceleration data structure. For example, bounding volume hierarchies (BVHs) of the type commonly used for modern beam tracking acceleration data structures typically have an O (log n) search behavior.

Bounding volume hierarchies

The most commonly used acceleration data structure by modern beam trackers is a bounding volume hierarchy (BVH), which has nested, axis-aligned bounding boxes (AABBs). The leaf nodes of the BVH contain the primitives (e.g. triangles) that are to be tested for intersections. The BVH is most often represented by a graph or tree structure data representation. In such cases, the traversing coprocessor can be referred to as a "tree traversing unit" or "TTU".

In a BVH, ray tracing amounts correspond to a tree search, in which each node in the tree visited by the ray has a bounding volume for each subsequent branch or leaf and the ray only visits the subsequent branches or leaves whose corresponding limiting volume it intersects. In this way, only a small number of primitives need to be explicitly tested for intersections, namely those that are in leaf nodes that are intersected by the beam. In the non-limiting embodiment, the traversal coprocessor speeds both tree traversal (including beam volume tests) and beam primitive tests. As part of the traversal, the traversal coprocessor can also handle "instance transformations" - transforming a beam from world-space coordinates into the coordinate system of an instanced mesh (object-space), e.g. B. to avoid the computational complexity of transforming the primitive vertices into space. This can be done in a MIMD (Multiple Instruction, Multiple Data) manner, which means that the beams are treated independently once they are in the traversing coprocessor.

Non-limiting example of real-time interactive ray tracing system

1 illustrates an example of a real-time interactive ray tracing graphics system 100 for generating images using three-dimensional (3D) data of a scene or one or more objects. The system 100 contains an input device 110 , one or more processors 120 , one or more graphics processing units (GPUs) 130 , Storage 140 and one or more displays 150 , This in 1 The system shown may take any form factor, including but not limited to a PC, smartphone, or other smart device, video game system, portable virtual or augmented reality system, cloud-based computer system, vehicle-mounted graphics system, one system on a chip (SoC; system-on-a-chip), etc.

The processor 120 can be a multi-core central processing unit (CPU) operable to interact with an application in real time in response to the input device 110 execute, and their output images for display on the display 150 contains. the display 50 can be any type of display, such as B. a stationary display, a display attached to the head such. B. a display glasses or - safety glasses, other types of portable displays, a hand-held display, a vehicle-mounted display, etc. For example, the processor 120 run an application based on input from the input device 110 (e.g., a joystick, an inertial sensor, an ambient light sensor, etc.) and the GPU 130 Instruct images showing application progress for display on the display 150 to create.

Based on the execution of the application on the processor 120 the processor can provide instructions for the GPU 130 output to images using in memory 140 generate saved 3D data. The GPU 130 contains specialized hardware to accelerate the generation of images in real time. For example, the GPU 130 able to process information for thousands or millions of graphics primitives (polygons) in real time because of the GPU's ability to perform repetitive and highly parallel specialized computing tasks such as B. Perform polygon scan signal conversion much faster than conventional software controlled CPUs. Different from the processor 120 , which can have multiple cores with a lot of cache memory that can handle a few software threads at the same time, the GPU 130 for example hundreds or thousands of processing cores or "streaming multiprocessors" (SMs) 132 included that run in parallel.

In one embodiment, the GPU contains 130 a plurality of programmable streaming multiprocessors (SMs) 132 and a hardware-based graphics pipeline with a graphics primitive engine 134 and a raster engine 136 , These components of the GPU 130 are configured to perform real-time image rendering using a technique called "scan conversion" to render three-dimensional scenes on a two-dimensional display 150 display. During rasterization, geometric building blocks (e.g. dots, lines, triangles, quadrilaterals, mesh networks, etc.) of a 3D scene are mapped onto pixels of the display (often via an image buffer memory).

The GPU 130 converts the geometric building blocks (ie polygon primitives such as triangles) of the 3D model into pixels of the 2D image and assigns an initial color value to each pixel. The graphics pipeline can apply shading, transparency, texture, and / or color effects to portions of the image by defining or adjusting the color values of the pixels. The final pixel values can be anti-aliased, filtered and displayed 150 can be fed for display. Many advances in software and hardware over the years have improved subjective image quality by using rasterization techniques with frame rates that are suitable for real-time graphics (ie 30 to 60 frames per second) at high screen resolutions such as. B. 4096 x 2160 pixels or more on one or more displays 150 required are.

Traversal coprocessor Extension of the architecture

So that the GPU 130 Can perform beam tracing in real time in an efficient manner, is the GPU with a traversing coprocessor 138 equipped with one or more SMs 132 is coupled. The traversal coprocessor 138 contains hardware components that are configured to perform operations commonly used in ray tracing algorithms. A target of the traversal coprocessor 138 is to speed up the operations used in ray tracing to the point where real-time graphics applications (e.g. games) bring the performance of ray tracing to high quality shadows, reflections and global lighting. The traversal coprocessor 138 includes query-specific traversal hardware in some embodiments 139 , which enables a query-specific programming of the behavior of the traversing coprocessor in order to e.g. B. Increase the flexibility and responsiveness of beam tracking operations to dynamic changes and the like in a scene being rendered. As explained in more detail below, the result of the traversal coprocessor 138 together with or as an alternative to other graphics related operations performed in the GPU 130 be carried out, used.

In the example architecture shown, the new hardware component "traversing coprocessor" 138 used to speed up certain tasks, including but not limited to ray tracing. Ray tracing refers to throwing a ray into a scene and determining if and where that ray intersects the scene's geometry. This basic ray tracing visibility test is the fundamental principle that underpins a variety of rendering algorithms and techniques in computer graphics. For example, beam tracing can be used together with or as an alternative to screening and Z-buffering for scanning the scene geometry. It can also be used as an alternative to (or in combination with) environmental imaging and shadow texturing to achieve more realistic reflection, refraction, and shadow effects than can be achieved using texturing techniques or other raster “hacks”. In order to overcome image quality restrictions that can be achieved with screening, the system can 100 also generate whole images or parts of images using ray tracing techniques. Beam tracing can also be used as the basic principle to transport light in physically based rendering algorithms such as B. Path Tracking, Photon Mapping, Metropolis Light Transport and precisely simulate other light transport algorithms.

In particular, SMs 132 and the traversal coprocessor 138 work together to throw rays into a 3D model and determine if and where that ray intersects the geometry of the model. Beam tracing directly simulates how light moves through a virtual environment or scene. The results of the beam intersections together with the surface structure, the direction of view and / or the lighting conditions are used to determine pixel color values. By SMs 132 in collaboration with the traversal coprocessor 138 ray tracing performed enables computer-generated images to capture shadows, reflections, and refractions in a way that is indistinguishable from real-world photos or videos. Since beam tracing techniques are even more computationally intensive than screening due to the large number of beams to be traced, the traversing coprocessor is 138 able to accelerate some of the more compute-intensive aspects of this process in hardware.

In the non-limiting example technique herein, the traversal coprocessor speeds up 138 both beam box tests and beam primitive tests. As part of the traversal, it can also handle at least one level of the instance transformation and transform a beam of world-space coordinates into the coordinate system of an instanced mesh. In the non-limiting embodiment, the traversal coprocessor accomplishes 138 all in a MIMD manner, which means that rays are treated independently once within the traversal coprocessor.

In non-limiting embodiments, the traversal coprocessor operates 138 as a servant (coprocessor) of the SMs (streaming multiprocessors) 132 , In other words, the traversal coprocessor 138 in non-limiting embodiments, does not work independently, but follows the commands of the SMs 132 to perform certain computationally intensive tasks related to ray tracing much more efficiently than SMs 132 could do it yourself.

In the examples shown, the traversal coprocessor receives 138 Commands via instructions from the SM 132 and writes results back to an SM register file. The traversing coprocessor can be used for many common applications (e.g. opaque triangles with a maximum of one instance level) 138 the ray tracing query without further interaction with the SM 132 serve. More complex queries (e.g. with alpha-tested triangles, primitives other than triangles or multiple levels of instantiation) may require multiple tours. In addition to tracking rays, the traversing coprocessor is 138 able to perform more general spatial queries where an AABB or the expanded volume between two AABBs (what we call a "beam") takes the place of the beam. This is the traversal coprocessor 138 Although particularly suitable for accelerating beam tracing tasks, it can also be used for tasks other than beam tracing.

In addition to the traversal coprocessor 138 provides the to support the system 100 of 1 Non-limiting example technique used additional improvements in accelerated beam tracing for a number of units, as well as substantial performance dedicated to BVH construction. The BVH construction need not be hardware accelerated (although it may be in some non-limiting embodiments), but instead could be implemented using highly optimized software routines that are based on SMs 132 and / or the CPU 120 and / or other development systems are running, e.g. B. during the development of an application. The following illustration describes, among other things, the software-visible behavior of the traversing coprocessor 138 , Interfaces to surrounding units (SMs 132 and the storage subsystem) and additional features that are part of a complete beam tracking solution, such as. B. certain extensions of the group of SMs 132 and the memory cache system.

Traversing an acceleration data structure

A good way to speed up beam tracing is to use an acceleration data structure. The acceleration data structure represents the 3D model of an object or scene in a manner that helps to quickly decide which part of the object a particular ray is likely to intersect and large parts of the scene that the ray will not intersect quickly to discard. A bounding volume hierarchy data structure (BVH data structure) is a type of acceleration data structure that can help reduce the number of intersection points to be tested. The BVH data structure represents a scene or an object with a boundary volume and divides the boundary volume into ever smaller boundary volumes that end in leaf nodes that contain geometric primitives. The bounding volumes are hierarchical, meaning that the top level includes the level below, this level includes the next level below, and so on. In one embodiment, leaf nodes can be others Potentially overlap leaf nodes in the bounding volume hierarchy.

To illustrate how a bounding volume hierarchy works, show 2A-2G a teapot that is recursively subdivided into increasingly smaller hierarchical bounding volumes. 2A shows a teapot object, and 2 B shows a limit volume 202 (in this case a box, cube or rectangular cuboid) that encloses the entire teapot. The limit volume 202 , which can be defined efficiently by its vertices, provides an indication of the spatial position of the object and is typically dimensioned such that it is only slightly larger than the object.

The first stage of the acceleration structure design captures the bounding boxes of the referenced geometry. This is achieved by performing a bounding box procedure for each geometric primitive in an object, which procedure uses a conservative, axis-aligned bounding box for its input primitive, e.g. B. the in 2 B shown box 202 returns. Using these bounding boxes as elementary primitives for the acceleration structures provides the abstraction necessary to trace rays against any user-defined geometry (including multiple geometry types within a single structure). There in 2 B the limit volume 202 is larger than the teapot and contains it completely, a beam that does not cut the confinement volume cannot cut the teapot, although a beam that cuts the confinement volume may or may not cut the teapot. Because the limit volume 202 Easily defined by the x, y, z coordinates of its vertices in 3D space and a ray is defined by its x, y, z coordinates in 3D space, the ray confinement volume test is to determine whether a ray the limit volume 202 cuts, uncomplicated (although a transformation can be used to adapt to different coordinate systems, as explained below).

2C shows the limitation volume 202 divided into smaller, contained limitation volumes. While the subdivision scheme shown here for illustration is a so-called octal subdivision or an "oct tree", with each volume divided into eight smaller volumes of uniform size, many other spatial hierarchies and subdivision schemes are known, such as e.g. B. a binary tree, a quaternary tree, a kd tree, a binary space divider tree (BSP tree) and a bounding volume hierarchy tree (BVH tree). See e.g. B. U.S. Patent 9,582,607.

Each of the in 2C The subdivided limitation volumes shown can be further subdivided. 2D shows one of the divided volumes 204 of 2C which is further subdivided to create additional subdivided, encapsulated bounding volumes. As in 2D shown, some of the partitioned containment volumes contain portions of the teapot and others do not. Volumes that do not contain any part of the teapot are not further subdivided, since the further subdivisions do not provide any further spatial information about the teapot. Limiting volumes that have already been subdivided and that contain at least part of the teapot can be subdivided even more recursively - such as the emergence of each of a sequence of ever smaller cats from the hats of Dr. Seuss' The Cat In The Hat Comes Back (1958). The parts of the space within the bounding volume 202 that contain geometry are recursively subdivided to the traversal coprocessor 138 to allow the volumetric divisions to be used to efficiently find out where the geometry is with respect to any given ray. Note that while spatial or active volume subdivision is possible, many implementations will create the hierarchical structure that defines volumes and subvolumes in advance. In such cases, the builder can often build the hierarchy from individual triangles and not from the entire scene. Building means that you don't have to determine whether a subdivided volume contains anything, because by definition it contains what is below it in a hierarchy of volumetric subdivisions.

2E shows another such subdivision of the bounding volume 204 into another smaller contained containment volume 206 , which in this example only contains the spout of the teapot plus another surface on the wall of the teapot, and 2F shows an additional subdivision of the limitation volume 206 into an even smaller subdivision included 208 that encapsulates the end of the spout of the teapot. Depending on the way in which the BVH is constructed, the limitation volume may vary 208 can be subdivided as required - and the traversing coprocessor 138 enables the system 100 of 1 to efficiently traverse the BVH down to any subdivision level. The number and configurations of the recursive subdivisions depend on the complexity and configuration of the 3D object being modeled, as well as other factors such as B. desired resolution, the distance of the object from the point of view, etc.

At some level of subdivision (which is different for different parts of the BVH The traversing coprocessor 138 on geometry that forms the encapsulated object just modeled. Analogous to a tree, the successive volumetric subdivisions are trunk, branches, branches and twigs, and the geometry is finally shown on the entire tips of the tree, namely the leaves. In this case it shows 2G the surface of the spout of the teapot, which is defined by a sample mesh of geometric primitives. The geometric primitives shown are triangles, but other geometric primitives such as e.g. For example, quadrilaterals, lines, rectangles, quadrics, spots, or other geometric primitives known to those skilled in the art (in one embodiment, such other types of primitives can be expressed as triangles or converted to triangles). The geometric primitives in the mesh represent the shape of the 3D surface of the object being modeled. The example shown here is a mesh, but the limited geometry may include a discontinuous geometry, such as B. Particles that may not be connected. In the non-limiting embodiments, the traversing coprocessor speeds up 138 also beam intersection tests with this geometry to quickly determine which triangles are hit by a given beam. Determining ray-primitive intersections involves comparing the spatial xyz coordinates of the vertices of each primitive with the xyz coordinates of the ray to determine whether the ray and the surface defining the primitive occupy the same space. The beam-primitive intersection test can be computationally intensive because there may be many triangles to test. For example, in the 2G mesh network the spout of the teapot alone is composed of over a hundred triangles - although in some implementations it can be more efficient to further subdivide volumetrically and thereby the number of triangles in such a "leaf knot" to something like 16 or less limit.

As discussed above, ray tracing procedures determine which geometric primitives of a scene are intersected by a ray. However, due to the large number of primitives in a 3D scene, it may not be efficient or feasible to test each geometric primitive for an intersection. Acceleration data structures such as B. BVH enable a quick determination of which boundary volumes can be ignored, which boundary volumes can contain cut geometric primitives and which cut geometric primitives are important for visualization and which are not.

Beam-intersection testing

3A-3C illustrate ray tracing based on the confinement volume 208 of 2G is applied, which is a triangular mesh 320 contains. 3A shows a ray 302 in a virtual space with limited volumes 310 and 315 , To determine if the beam 302 one or more triangles in the mesh 320 intersects, each triangle could be directly against the beam 302 getting tested. To speed up the process (since the object could contain many thousands of triangles), the beam 302 first against the limitation volumes 310 and 315 tested. If the beam 302 does not intersect a bounding volume, it does not intersect any triangles within the bounding volume, and all triangles within the bounding volume can be ignored for the purposes of this beam. There in 3A the beam 302 the limit volume 310 missed, the triangles of the mesh 320 are not tested for intersections within this limitation volume. Although the limit volume 315 from the beam 302 is cut, contains the limiting volume 315 no geometry, so no further testing is required.

On the other hand, if a beam such. B. the in 3B shown beam 304 a limit volume 310 intersects that contains geometry, then the beam may or may not intersect the geometry within the confinement volume, so further testing must be performed on the geometry itself to find possible intersections. Because the rays 304 . 306 in 3B and 3C a limit volume 310 that contains geometry, further tests must be performed to determine whether any (and which) of the primitives are cut within the bounding volume. In 3B further testing of the intersections with the primitives would indicate that the beam 304 the limitation volume 310 runs through, but does not intersect any of the primitives included in the bounding volume (alternatively, as mentioned above, the bounding volume could 310 be further volumetrically subdivided so that a bounding volume intersection test could be used to show that the beam does not intersect geometry or, in particular, which primitives the beam may intersect).

3C shows a situation in which the limiting volume 310 from the beam 306 is cut and contains geometry that the beam 306 cuts. The traversal coprocessor 138 tests the intersections between the beam 306 and the individual primitives to determine which primitives the beam intersects.

Ray tracing operations

4 Figure 12 is a flow diagram summarizing example beam tracking operations performed by the traversal coprocessor 138 in collaboration with SM (s) 132 as described above. The operations in 4 are from the traversal coprocessor 138 in collaboration with his interaction with an SM 132 carried out. The traversal coprocessor 138 can thus identify a beam from the SM 132 and receive the traversal status that enumerates one or more nodes in one or more BVHs that the beam must traverse. The traversal coprocessor 138 determines which boundary volumes of a BVH data structure the beam intersects (the "beam complete" test 512 ), and then whether the beam intersects one or more primitives in the cut boundary volumes and which triangles are cut (the “beam primitive test” 520). In non-limiting embodiments, "complets" (compressed treelets ("threefold")) specify root or inner nodes (ie volumes) of the bounding volume hierarchy with children that are other completes or leaf nodes of a single type per complet.

First, the traversal coprocessor checks 138 the traversal status of the beam. If a stack the traversal coprocessor 138 maintains for the beam, is empty, the traversing is complete. If there is an entry at the top of the stack, the traversal coprocessor sends 138 a request to the storage subsystem to read this node. The traversal coprocessor 138 then performs a bounding box test 512 to determine whether a bounding volume of a BVH data structure is intersected by a particular beam that the SM 132 specified (step 512 . 514 ). If the bounding box test determines that the bounding volume is not cut by the beam (“No” in step 514 ), it is not necessary to carry out further tests for visualization, and the traversing coprocessor 138 this result can be sent to the requesting SM 132 hand back. Because if a beam misses a limiting volume (as in 3A in terms of the limitation volume 310 ), then the beam misses all other smaller confinement volumes within the confinement volume just tested and any primitives that contain the confinement volume.

This results from the traversing coprocessor 138 Limiting box test carried out that the limiting volume is cut by the jet ("Yes" in step 514 ), then the traversal coprocessor determines whether the bounding volume can be divided into smaller bounding volumes (step 518 ). In one embodiment, the traversal coprocessor performs 138 not necessarily any division by yourself. Rather, each node in the BVH has one or more children (each child being a leaf or a branch in the BVH). For each child there is a volume limit and a pointer that leads to a branch or leaf node. If a beam knots using the traversal coprocessor 138 processed, it tests itself against the limited volumes of the children of the node. The beam only pushes batch entries for those branches or leaves on whose stack the representing limiting volumes have been hit. When a beam retrieves a node in the embodiment, it does not test against the knot's bounding volume - it tests against the knot's child's bounding volumes. The traversal coprocessor 138 pushes nodes whose boundary volumes are hit by a beam onto the beam traversal stack in an order determined by the beam configuration. For example, it is possible to push nodes onto the traversal stack in the order in which the nodes appear in memory, in the order in which they appear along the length of the beam, or in another order. If there are further subdivisions of the limitation volume ("Yes" in step 518 ), then these further divisions of the bounding volume are accessed and the bounding box test is performed for each of the resulting divided bounding volumes to determine which divided bounding volumes are intersected by the beam and which are not. In this recursive process, some of the confinement volumes can be tested 514 are eliminated, while other delimitation volumes can result in more and more subdivisions by the traversal coprocessor 138 recursively by applying the steps 512 - 518 be tested for intersections.

Once the traversal coprocessor 138 has determined that the boundary volumes intersected by the beam are leaf nodes ("No" in step 518 ), the traversal coprocessor performs a primitive (e.g. triangle) intersection test 520 to determine whether the beam intersects primitives in the cut boundary volumes and which primitives the beam intersects. The traversal coprocessor 138 thus traverses cut subsequent branch nodes by depth until leaf nodes are reached. The traversal coprocessor 138 processes the leaf nodes. If the leaf nodes are primitive areas, the traversal coprocessor tests 138 them against the beam. If the leaf nodes are instance nodes, the traversal coprocessor will apply 138 the instance transformation. If the leaf nodes are element areas, the traversal coprocessor returns 138 them to the requesting SM 132 back. In the non-limiting embodiments, the SM 132 the traversal coprocessor 138 command to perform different types of beam primitive intersection tests and report different results depending on the operations that come from an application (or a software stack on which the application is running) and are forwarded from the SM to the TTU. For example, the SM 132 the traversal coprocessor 138 command to report the closest visible primitive that has been detected by the intersection test or to report all primitives that the beam intersects, regardless of whether it is the closest visible primitive. The SM 132 can use these different results for different types of visualization. Once the traversal coprocessor 138 Once the leaf nodes have been processed, there may be further branch nodes (which were previously pushed onto the stack of the beam) for testing.

Multiple intersections

As in 3C shown, in particular any given ray can intersect multiple primitives within a confinement volume. Whether the beam intersection is important for visualization within a given primitive depends on the properties and position of this primitive as well as on the visualization procedures that the SM 132 performs. For example, primitives can be opaque, transparent, or partially transparent (ie, translucent). Opaque primitives block the passage of a beam through the primitive because the eye cannot see through the opaque surface of the primitive. Transparent primitives let the beam pass (because the eye can see through the transparent primitive), but the situation can be more complex. For example, transparent primitives can have specular properties that cause part of the beam to be reflected (think reflection on a windowpane) and the rest of the beam to pass through. Other transparent primitives are used to create a surface on which a texture is mapped. For example, every single leaf of a tree can be modeled by a transparent primitive onto which an image of the leaf is mapped texture.

5A-5C illustrate some of these scenarios using an example of three triangles, which are assumed to be in the same confinement volume and are each intersected by a ray. 5A illustrates a ray directed at these three triangles, the first triangle the ray hits with respect to the viewpoint being opaque. Since the “front” triangle (as seen from the eye in the direction of the beam) is opaque, this triangle blocks the beam so that the beam does not reach the other triangles, although it intersects them spatially. In this example, the triangles "behind" the opaque triangle can be ignored (discarded) from the point of view after the intersection of the opaque triangle has been identified, since the "front" opaque triangle detects the other triangles from the user's view along the beam covered. Culling is done in 5A-5C indicated by dashed lines. In this case, the traversal coprocessor 138 possibly only the identification of the first opaque triangle on the SM 132 Report.

5B illustrates a ray directed at the same three triangles, but the closest visible triangle is now partially transparent instead of opaque. Since the closest visible intersected triangle is at least partially transparent, the beam can pass through it to hit the opaque triangle behind it. In this case, the opaque triangle is visible through the partially transparent triangle, but blocks the user's view of the third triangle along the beam. Here the traversing coprocessor can 138 the identification of the two front triangles on the SM 132 report, but not the third, separated triangle, although the beam intersects this third triangle spatially. The order of discovery can be important here. In the case of an alpha and opaque triangle, if the opaque has been found first, the traversal coprocessor returns 138 the opaque triangle on the SM 132 with a traversal status that will resume testing on the alpha triangle. Although there is a meaning here that the alpha means transparent, it really means: “Take me to the SM 132 back and let the SM decide how to deal with it ”. For example, an alpha triangle can be trimmed according to a texture or function so that parts of the triangle are cut away (ie absent, not transparent). The traversal coprocessor 138 do not know how the SM 132 will treat the alpha triangles (ie, it will treat transparent triangles no differently than cropped triangles). Thus, alpha triangles may or may not block or color the light coming from points beyond it along the beam, and in embodiments require intervention by the SM 132 to handle / determine these things.

5C illustrates a scenario in which the first two triangles that the beam hits are partially transparent. Because the first and the second cut triangle at least partially are transparent, the beam passes through the first and second triangles to meet the third opaque triangle, which also intersects. Since the third triangle cut is opaque, it blocks the beam and the beam does not hit any other triangles behind the third triangle, even if they can be spatially cut by it. In this case, the traversal coprocessor 138 all three triangles to the SM 132 report, but does not have to report any further triangles behind the opaque triangle, since the opaque triangle prevents the beam from reaching these additional triangles.

In some modes, the SM 132 however, may know the identities of all the triangles the ray intersects, regardless of whether they are opaque or transparent. In these modes, the traversal coprocessor can 138 simply perform the intersection test and return the identities of all the triangles that the beam intersects spatially (in such modes, the traversing coprocessor is used for all three in 5A-5C scenarios shown return the same intersection results) and it to the SM 132 enable them to be sorted - or in some cases the traversal coprocessor 138 order further tests to be carried out on these triangles.

As explained in more detail below, the traversing coprocessor can 138 If a ray intersects an opaque triangle, in some operations it may be programmed to reduce the length of the ray being tested to the position of the intersection with an opaque triangle so that it does not report triangles "behind" the intersected triangle. If a partially transparent triangle is determined to be cut by a beam, the traversal coprocessor returns 138 return a more complete list of triangles encountered by the beam and the requesting SM for visualization purposes 132 may perform further processing to determine, for example, whether the beam is blocked, transmitted, or partially transmitted and partially reflected, based on any texture or other properties of the triangle. In embodiments, the traversal coprocessor 138 does not have access to triangle texture properties and therefore does not attempt to determine visualization related to these properties.

Textures or other surface modifications

For example, show 6A and 6B a transparent triangle 610 with a texture 615 of a leaf attached to the triangle. One could imagine a triangle made of plexiglass with a sticker on a sheet. As in 6A shown, the beam intersects 620 the transparent triangle 610 at a point that is outside of the applied texture 615 lies. Because the beam 620 the triangle outside the applied texture 615 cuts, the texture blocks the beam 620 not, and the beam traverses the transparent triangle 610 unhindered. This is as you can see through the parts of the plexiglass triangle that are not covered by the sheet sticker. Note that in one embodiment of the SM 132 performs the visibility determination because the traversal coprocessor 138 does not necessarily have access to information about the sheet sticker. The traversal coprocessor 138 helps the SM 132 by returning to the SM the identification of the triangle the ray intersects, along with information about the properties of that triangle.

In 6B the ray cuts 630 the transparent triangle where the texture 615 is applied. The SM 132 determined based on whether the texture 615 the beam 630 block or the beam 630 is let through whether subsequent traversing by the traversing coprocessor 138 is necessary or not. If the beam 630 through the texture 615 is blocked, other triangles are behind the transparent triangle 610 that are otherwise from the beam 630 would have been cut, hidden by the texture and do not contribute to visibility along the beam. In the non-limiting embodiments contained herein, the traversal co-processor has 138 does not have access to texture information and therefore does not attempt to speed up this determination. The traversal coprocessor 138 can, for example, all intersection points between the beam and the various triangles within the object to the requesting SM 132 return, and the SM can then use the graphics primitive engine 134 use to perform further ray tracing visualization determinations. In other embodiments, the traversal coprocessor could 138 Accelerate some or all of these tests by working with the texture map and other parts of the 3D graphics pipeline within the graphics primitive engine 134 interacts to make the necessary visualization determinations.

coordinate transformations

2A-3C affect only one object, namely a teapot. Just as the room you are in contains several objects, most 3D scenes contain many objects. For example, a 3D scene with a teapot may also contain a cup, saucer, milk jug, spoon, sugar bowl, etc., all of which are on a table. With 3D graphics, each of these objects is typically modeled independently. The graphics system 100 then uses instructions from the processor 120 to all Place models together in the desired positions, orientations and sizes for the purpose of visualization in the common scene (just as you set and arrange the table for serving tea). This means that the SM 132 the traversal processor 138 command to analyze the same beam for multiple objects in the scene. However, the fact that each of these objects is transformed in position, orientation, and size when placed in the common scene is handled by the traversal coprocessor 138 considered and accelerated. In non-limiting exemplary embodiments, the transformation from world space to object space is stored together with a world space boundary box in the world space BVH. The traversal coprocessor 138 accelerates the transformation process by transforming the ray from the world (scene) space into the object space, around which in 4 perform the tests shown. Since the transformation of the geometry from the object space into the world (scene) space is computationally intensive, this transformation becomes especially the graphics pipeline graphics primitive engine 134 and / or raster engine 136 left to do as part of the screening. The traversal coprocessor 138 instead converts a given beam from the world space into the coordinate system of each object defined by an acceleration data structure and performs its tests in the object space.

7A and 7B illustrate how the traversal coprocessor 138 transformed the same beam into three different object spaces. 7A shows three objects on a table: a cup, a teapot and a jug. These three objects and a table form a scene that exists in the world space. A ray, which is also defined in world space, starts from the point of view and intersects each of the three objects.

7B shows each of the three objects as defined in object spaces. Each of these three objects is defined by a respective model that exists in a respective object space. The traversal coprocessor 138 transforms the beam into the object space of each object before performing the intersection tests for that object in non-limiting embodiments. This "instance transformation" saves the computing effort for the transformation of the geometry of each object and the associated volumetric subdivisions of the acceleration data structure from the object space into the world space for the purposes of the traversal coprocessor 138 when performing intersection tests.

The requesting SM 132 keeps track of which objects are in front of which other objects in relation to each individual ray, and clarifies visibility in cases where one object hides another object, casts a shadow on another object and / or reflects light on another object. The requesting SM 132 can the traversal processor 138 use to speed up each of these tests.

Sample Baurn-BVH acceleration data structure

8A and 8B show a recursively subdivided boundary volume of a 3D scene ( 8A ) and a corresponding tree data structure ( 8B ) to which the traversal coprocessor 138 can be accessed and used for hardware accelerated operations performed by the traversal coprocessor. The subdivision of the limitation volumes can be represented in a hierarchical tree data structure 2 B shown large bounding volumes are represented by a parent node of the tree and the smaller bounding volumes are represented by child nodes of the tree, which are encompassed by the parent nodes. The smallest boundary volumes are represented as leaf nodes in the tree and identify one or more geometric primitives that are contained in these smallest boundary volumes.

The tree data structure can be in memory outside of the traversal coprocessor 138 saved and based on queries that the SMs 132 to the traversal coprocessor 138 output, be read out. The tree data structure contains a large number of nodes which are arranged in a hierarchy. The root nodes N1 the tree structure corresponds to the limitation volume N1 that includes all triangles 01-08. The root node N1 can be the vertices of the bounding volume N1 and identify child nodes of the root node.

In 8A is the limit volume N1 in limited volumes N2 and N3 divided. The child knot N2 and N3 the tree structure of 8B correspond to those in 8A shown limiting volumes N2 and N3 and represent them. The child nodes N2 and N3 identify the vertices of the respective bounding volumes in the tree data structure N2 and N3 in the room. Each of the limitation volumes N2 and N3 is further subdivided in this particular example. The limit volume N2 is in containment volumes N4 and N5 divided. The limit volume N3 is in containment volumes N6 and N7 divided. The limit volume N7 contains two delimitation volumes N8 and N9 , The limit volume N8 contains the triangles O7 and O8 , and the limit volume N9 contains sheet boundary volumes N10 and N11 than his child limit volumes. The sheet limit volume N10 contains a primitive Area (e.g. triangle area) O10 , and the sheet limit volume N11 contains an element area O9 , Respective child knot N4 . N5 . N6 . N8 . N10 and N11 the tree structure of 8B correspond to the limitation volumes N4 . N5 . N6 . N8 . N10 and N11 of 8A in the room and represent them.

The tree in 8B is only three to six levels deep, so the volumes N4 . N5 . N6 . N8 . N10 and N11 Form "leaf nodes" - that is, nodes in the tree that have no child nodes. 8A shows that each of the leaf node boundary volumes N4 . N5 . N6 and N8 contains two triangles of geometry in the scene. For example, the volumetric subdivision contains N4 the triangles O1 & O2 ; the volumetric division N5 contains the triangles O3 & O4 ; the volumetric division N6 contains the triangles O5 & O6 ; and the volumetric division N8 contains the triangles O7 & O8 , In the 8B The tree structure shown represents these leaf nodes N4 . N5 . N6 and N7 represent them by corresponding triangles to the triangles 01 - 08 assigned to the scene geometry. In order to access this scene geometry, the traversing coprocessor traverses 138 the tree data structure of 8B down to the leaf nodes. In general, different parts of the tree can and will have different depths and contain different numbers of triangles. Leaf nodes that are assigned to volumetric subdivisions that do not contain any geometry do not have to be explicitly represented in the tree data structure (ie the tree is “trimmed”).

In some embodiments, the at N7 rooted subtree represent a set of bounding volumes or BVH that is defined in a different coordinate space than the bounding volumes that define the node N1 - N3 correspond. If the limit volume N7 is in a different coordinate space than its parent bounding volume N3 , can be an instance node N7 ' which provides the beam transformation necessary to complete the N7 rooted subtree to traverse the rest of the tree with the N7 connect rooted subtree. The instance node N7 ' connects the limitation volume or the BVH according to the nodes N1 - N3 with the limiting volumes or BVH corresponding to the nodes N7 etc. by defining the transformation from coordinate space of N1-N3 (e.g. world space) in the coordinate space of N7 etc. (e.g. object area).

The internal structure and operation of the traversal coprocessor 138

9 shows a simplified example block diagram of the traversal coprocessor 138 with hardware configured to perform accelerated traversal operations as described above (an even more detailed implementation of this traversal coprocessor 138 is described below). Since the in 9 traversal coprocessor shown 138 for traversing tree-based acceleration data structures such as B. in 8A . 8B shown, it can also be used as a "tree traversing unit" or "TTU" 700 be referred to (the reference symbol 700 refers to the more detailed, non-limiting implementation of the in 1 shown traversal coprocessor 138 ). For example, tree traversal operations may include determining whether a beam intersects bounding volumes and / or primitives of a tree data structure (e.g., a BVH tree), which tests may include transforming the beam into an object space.

The TTU 700 includes dedicated hardware to determine whether a beam intersects boundary volumes and dedicated hardware to determine whether a beam intersects primitives of the tree data structure. In some embodiments, the TTU 700 traverse a bounding volume hierarchy by depth using short stack traversal with intersection testing of supported leaf node primitives and return of alpha primitives and unsupported leaf node primitives (elements) in the middle of the traversal. The intersection of the primitives is discussed with respect to triangles, but other geometric primitives can also be used.

In particular, the TTU contains 700 an intersection management block 722 , a beam management block 730 and a batch management block 740 , Each of these blocks (and all other blocks in 9 ) can form dedicated hardware that is implemented by logic gates, registers, hardware lookup tables embedded in hardware, or other combinatorial logic, etc.

The beam management block 730 is responsible for managing information about and performing operations related to a beam from the beam management block of an SM 132 is specified. The batch management block 740 works in conjunction with traversal logic 712 to manage information about the traversal of a BVH acceleration data structure and to perform related operations. The traversal logic 712 is obtained from results of a beam complete test block 710 directed, the intersection between that of the beam management block 730 displayed beam and volumetric subdivisions represented by the BVH, using instance transformations as required. The beam complete test block 710 reads additional information regarding the BVH a LO complete cache 752 , part of the TTU 700 is from memory 140 out. The results of the beam complete test block 710 inform the traversal logic 712 whether further recursive traversing is required. The batch management block 740 maintains stacks for keeping track of status information when traversing logic 712 passes from one level of the BVH to another, with the batch management block pushing elements onto the stack if the traversing logic traverses deeper into the BVH and removes elements from the stack if the traversing logic traverses upwards in the BVH. The batch management block 740 is able to meet the requesting SM 132 to provide status information (e.g. interim or final results) at any point in time when the SM requests it.

The intersection management block 722 maintains information about and performs intersection-ray-primitive intersection operations using instance transformations as needed. The beam primitive test block 720 reads information related to geometry on an as needed basis through an L0 primitive cache 754 , part of the TTU 700 is from memory 140 out. The intersection management block 722 is informed about the results of intersection tests that the beam primitive test and transformation block 720 performs. Thus the beam primitive test and transformation block leads 720 Intersection results to the intersection management block 722 to, the geometry hit and intersections at the requesting SM 132 reports.

A batch manager 740 checks the traversal status to determine what type of data to read and what data path (complete or primitive) it will consume. The intersection points for the boundary volumes are in the TTU's beam complete test path 700 determines which one or more beam complete test blocks 710 and one or more traversal logic blocks 712 contains. A complete specifies the root or inner node of a bounding volume. A complete can therefore define one or more limiting volumes for the beam complete test. The TTU beam complete test path 700 identifies which boundary volumes are intersected by the beam. Boundary volumes cut by the beam must be processed further to determine whether the primitives associated with the cut boundary volumes are cut. The intersections for the primitives are determined in the beam-primitive test path, the one or more beam-primitive test and transformation blocks 720 and one or more intersection management blocks 722 contains.

The TTU 700 receives queries from one or more SMs 132 to perform tree traversal operations. The query can request whether a beam intersects bounding volumes and / or primitives in a BVH data structure. The query can identify a ray (e.g., origin, direction, and length of the ray) and a BVH data structure and traversal status (e.g., short stack) that contains one or more entries that are on nodes in one or more bounding volume hierarchies that the beam is to visit. The query can also include information about how the beam should handle certain types of intersection points during traversal. The beam information can be in the beam management block 730 get saved. The saved beam information (e.g. beam length) can be updated based on the results of the beam primitive test.

The TTU 700 can request that the BVH data structure identified in the query comes from memory outside the TTU 700 is read out. Parts of the BVH data structure that have been read out can be stored in level zero (L0) cache 750 within the TTU 700 be cached so that the information is available for other time-coherent TTU operations, thereby accessing the memory 140 be reduced. Parts of the BVH data structure that are required for the beam complete test can be stored in a LO complete cache 752 and portions of the BVH data structure needed for the beam primitive test can be stored in an L0 primitive cache 754.

After the complete information required for a requested traversal step in the complete cache 752 the beam complete test block determines 710 the boundary volumes cut by the beam. When performing this test, the beam can be transformed from the coordinate space of the bounding volume hierarchy into a coordinate space defined relative to a complete. The beam is tested against the bounding boxes associated with the child nodes of the complete. In the non-limiting embodiment, the beam is not tested against the complete bounding box because ( 1 ) the TTU 700 previously tested the beam against a similar bounding box when testing the parent bounding box child who referred to this complet, and ( 2 ) A purpose of the complete bounding box is to define a local coordinate system within which the child bounding boxes can be expressed in a compressed form. If the beam intersects any of the child bounding boxes, the results are passed to the traversal logic to determine the order in which the appropriate child pointers are then pushed onto the traversal stack (further testing likely requires the traversal logic 712 traversed down to the next level of BVH). These steps are repeated recursively until cut leaf nodes of the BVH are found.

The beam complete test block 710 can the traversal logic 612 Feed beam complete intersection points. The traversal logic builds using the results of the beam complete test 712 Batch entries sent to the batch management block 740 are to be forwarded. The batch entries can be internal nodes (ie a node that contains one or more child nodes) by the beam complete test block 710 must be further tested for beam intersection points, and / or indicate triangles identified in a cut leaf node by the beam primitive test and transformation block 720 must be tested for beam intersections. The beam complete test block 710 can repeat the traversal on internal nodes identified in the stack to determine all leaf nodes in the BVH that the beam is cutting. The exact tests that the beam complete test block 710 in the non-limiting embodiment are determined by mode bits, beam operations (see below), and hit culling, and the TTU 700 can send intermediate and final results to the SM 132 hand back.

The cut leaf nodes identify primitives that may or may not be cut by the beam. An option for the TTU 700 is, e.g. B. a range of geometries identified in the cut leaf nodes the SM 132 to be fed for further processing. For example, the SM 132 based on the information that the TTU 700 as a result of traversing the BVH TTU provides to determine whether the identified primitives are cut by the beam. To the SM 132 relieve of this processing and thereby using the hardware of the TTU 700 to speed up the batch management block 740 Requirements for the beam primitive and transformation block 720 issue a beam primitive test for the primitives within cut leaf nodes that the beam complete test block 710 the TTU has identified. In some embodiments, the SM 132 a request for the beam primitive test to test a certain range of primitives on the transform block 720 output regardless of how this geometry area was identified.

After ensuring that the primitive data required for a requested beam primitive test is in the primitive cache 754 are available, the beam primitive and transformation block 710 using those in the beam management block 730 determine stored beam information primitives that are intersected by the beam. The beam primitive test block 720 carries out the identification of primitives determined to be cut by the beam to the intersection management block 722 to.

The intersection management block 722 can send the results of the beam primitive test to the SM 132 hand back. The results of the beam primitive test can include identifiers of cut primitives, the removal of intersections from the beam origin, and other information about properties of the cut primitives. In some embodiments, the intersection management block 722 an existing beam primitive test based on previous intersection results from the beam primitive and transformation block 710 modify (e.g. by modifying the length of the beam).

The intersection management block 722 can also keep an eye on different types of primitives. For example, the various types of triangles include opaque triangles that block a beam when cut and alpha triangles that may or may not block the beam when cut or may require additional treatment by the SM. Whether or not a beam is blocked by a transparent triangle can be determined, for example, by the texture (s) imaged on the triangle, the area of the triangle occupied by the texture (see 6A and 6B ) and how the texture modifies the triangle. For example, transparency (e.g. stained glass) in some embodiments requires that the SM 132 Keeps an eye on hits from transparent objects so that they can be sorted and shaded in beam parametric order, and typically doesn't really block the beam. In the meantime, alpha “pruning” enables the shape of the primitive to be trimmed based on the shape of a texture mapped onto the primitive - for example, cutting a leaf shape from a triangle. (Note that in raster graphics, transparency is often referred to as "alpha blending" and cropping as "alpha test"). In other embodiments, the TTU 700 transparent hits for later treatment by the SM 132 queue into memory and handle trimmed triangles directly by sending requests to the texture unit. Each triangle can contain an identifier to indicate the triangle type. The intersection management block 722 is configured to maintain a result queue to keep track of the different types of intersected triangles. For example, the result queue can have one or more sliced-opaque- Store triangle identifiers in one queue and one or more transparent triangle identifiers in another queue.

For opaque triangles, the beam intersection in the TTU 700 fully determined because the area of the opaque triangle prevents the beam from passing the surface of the triangle. For transparent triangles, in some embodiments, beam intersections cannot be completely in the TTU 700 be determined because the TTU 700 performs the intersection test based on the geometry of the triangle and may not have access to the texture of the triangle and / or the area of the triangle occupied by the texture (in other embodiments, the texture mapping block of the graphics pipeline can provide the TTU with texture information). To fully determine whether the triangle is intersected, you can use information about transparent triangles that the beam primitive and transform block 710 determined as cut, to the SM 132 are sent so that the SM can make the full determination of whether the triangle affects visibility along the beam.

The SM 132 can clarify whether the beam intersects a texture assigned to the transparent triangle or not and / or whether the beam is blocked by the texture. The SM 132 In some cases, based on this determination, a modified query can be sent to the TTU 700 send (e.g. shortening the beam if the beam is blocked by the texture).

In one embodiment, the TTU 700 configured to include all triangles determined to intersect the beam for further processing to the SM 132 return. Since the return of each triangle intersection to the SM 132 for further processing with regard to interface and thread synchronization, the TTU can 700 configured to hide triangles that are intersected, but are proven to be hidden without any functional impact on the resulting scene. Since the TTU 700 For example, with triangle type information (e.g. whether a triangle is opaque or transparent), the TTU can 700 use the triangle type information to determine intersected triangles that are covered by another intersecting opaque triangle along the ray and therefore do not need to be included in the results as they will not affect visibility along the ray. As above with reference to 5A-5C discussed, if the TTU 700 knows that a triangle along the beam is hidden by an opaque triangle, the hidden triangle is hidden from the results without affecting the visualization of the resulting scene.

The intersection management block 722 may contain a result queue for storing hits that map a triangle ID and information about the point at which the ray hits the triangle. If a beam is determined to intersect an opaque triangle, the identity of the triangle and the distance of the intersection from the beam origin can be stored in the result queue. If the beam is determined to be intersecting another opaque triangle, the other opaque triangle can be omitted from the result if the distance of the intersection from the beam origin is greater than the distance of the intersected opaque triangle already stored in the result queue. If the distance of the intersection from the beam origin is less than the distance of the cut opaque triangle already stored in the result queue, the other cut opaque triangle can replace the opaque triangle stored in the result queue. After all triangles of a query have been tested, the information about opaque triangles and the intersection information stored in the result queue can be sent to the SM 132 be sent.

In some embodiments, the intersection management block 722 as soon as an intersection with an opaque triangle has been identified, that in the beam management block 730 Shorten the stored ray so that bounding volumes (which may contain triangles) behind the intersected opaque triangle (along the ray) are not identified as intersecting the ray.

The intersection management block 722 can store information about cut transparent triangles in a separate queue. The stored information about cut transparent triangles can be sent to the SM 132 are sent so that the SM clarifies whether the beam intersects a texture assigned to the triangle or not and / or whether the texture blocks the beam. The SM can send the results of this determination to the TTU 700 return and / or modify the query based on this determination (e.g. shorten the beam if the beam is blocked by the texture).

Sample ray tracing shading pipeline

10 shows an example ray tracing shading pipeline 900 by the SM 132 carried out and by the TTU 700 can be accelerated. The ray tracing shading pipeline 900 starts with an SM 132 a beam generation 910 calls and a corresponding ray tracing request to the TTU 700 outputs. The ray tracing requirement identifies you single beam thrown into the scene and demands the TTU 700 on, after intersections with one also by the SM 132 search specified acceleration data structure. The TTU 700 traversed ( 10 block 920 ) the acceleration data structure to determine intersection points or potential intersection points between the beam and the volumetric divisions and associated triangles that the acceleration data structure represents. Potential intersection points can be identified by finding boundary volumes in the acceleration data structure that are intersected by the beam. Offspring from uncut cut volumes do not need to be checked.

The beam primitive test block performs for triangles within cut boundary volumes 720 the TTU 700 an intersection process 930 to determine if the beam intersects the primitives. The TTU 700 gives intersection information to the SM 132 back, in response to the intersection determination, an “any hit” shading operation 940 can perform. For example, the SM 132 perform a texture lookup after a cut primitive (or have it carried out by other hardware) and decide, based on the appropriate Texel value, how to shade a pixel visualizing the beam. The SM 132 keeps an eye on these results since the TTU 700 may return multiple intersections with different geometry in the scene in any order.

Alternatively, primitives that the TTU 700 determined as cut, processed to determine 950 whether it is a miss 960 or as a closest hit 970 to be shaded. The SM 132 can for example the TTU 700 instruct to report a closest hit in the specified geometry, or it can instruct the TTU to report all hits in the specified geometry. For example, it can be the business of SM 132 be based on implemented environment lookups (e.g. approximating the appearance of a reflective surface using a pre-calculated texture image) to implement a “miss” shading operation for a primitive that the TTU 700 determined as cut, such as. B. in 6A & 6B shown. The SM 132 may perform a closest hit shading operation to determine the closest cut primitive based on material ratings and texture lookups in response to closest hit messages that the TTU 700 delivered for a certain object geometry.

The more detailed flowchart of a ray tracing pipeline from 11A shows the data flow and the interaction between components for a representative application: tracing rays against a scene containing geometric primitives with instance transformations handled in hardware. In one non-limiting embodiment, the beam tracking pipeline is from 11A essentially software-defined (which in exemplary embodiments means that they are defined by the SMs 132 is determined), but uses hardware acceleration by the TTU to a high degree 700 , Key components include the SM 132 (and the rest of the computing pipeline), the TTU 700 (which serves as a coprocessor to the SM) as well as the L1 cache and the downstream storage system from which the TTU retrieves BVH and triangle data.

In the 11A The pipeline shown shows that the limit volume hierarchy generation 1002 can be done in advance by a development system. It also shows that beam generation and distribution 1004 from the SM 132 or other software in the embodiment, such as shading (which may include lighting and texturing). The sample pipeline contains a "top-level" BVH tree traversal 1006 , a beam transformation 1014 , a bottom-level BVH tree traversal 1018 and a ray / triangle (or other primitive) intersection 1026 , each from the TTU 700 be performed. These do not have to be carried out in the order shown, since handshaking between the TTU 700 and the SM 132 determines what the TTU 700 does and in what order.

The SM 132 puts the TTU 700 one or more beams at a time. Every ray that the SM 132 the TTU 700 for traversal may include geometric parameters of the beam, traversal status, and beam flag, mode flag, and beam operation information of the beam. In one embodiment, a ray operation (RayOp) provides or includes an auxiliary arithmetic or logic test to suppress, override, and / or allow an intersection to be saved. The traversal stack can also be from the SM 132 used to send certain status information to the TTU 700 to be transmitted for use in traversal. A new beam query can be started with an explicit traversal stack. For some queries, however, a small number of batch initializers can be provided to start the new query of a given type, such as: traversing from a complet; Intersection of a ray with an area of triangles; Intersection of a ray with a range of triangles, followed by traversing from a complete; Vertex fetch from a triangle buffer for a given triangle, etc. In some embodiments, the use of improves Batch initializers perform instead of explicit batch initialization because batch initializers use fewer streaming processor registers and reduce the number of parameters that must be transferred from the streaming processor to the TTU.

In the embodiment, a set of mode flags (flags) that the SM 132 with every query (e.g. beam) submitted, at least partially control how the TTU 700 processes the query when the query intersects the bounding volume of a particular type or intersects a primitive of a particular primitive type. The mode flags that the SM 132 the TTU 700 provides the SM and / or the application with the ability, for. B. by a RayOp to specify an auxiliary arithmetic or logic test to suppress, override and / or allow storage of an intersection. For example, the mode flags may allow the traversal behavior to be changed in accordance with aspects such as a depth (or distance) associated with each bounding volume and / or primitive, the size of a bounding volume or primitive with respect to a distance from Origin or the ray, certain instances of an object, etc. This ability can be used by applications to dynamically and / or selectively enable / disable sets of objects for intersection testing against certain sets or groups of queries, for example different versions of models to use when an application status changes (for example, when opening or closing doors), or to provide different versions of a model that are selected as a function of the length of the beam to implement some form of geometric level of detail, or certain Sentences of obje to allow certain classes of rays to make certain layers visible or invisible in certain views.

In addition to the set of mode flags that can be specified separately for the beam complete intersection and for the beam primitive intersection, the beam data structure can specify other RayOp test related parameters such as: B. beam flags, beam parameters and a RayOp test. The beam flags can be obtained from the TTU 700 can be used to control various aspects of traversal behavior, backface pulling, and handling of the different child node types, depending on the pass / fail status of an optional RayOp test. RayOp tests therefore add to the capabilities of the TTU 700 Add flexibility, at the cost of some complexity. The TTU 700 reserves a "beam slot" for each active beam it processes and can store the beam flags, mode flags and / or the RayOp information in the corresponding beam slot buffer within the TTU during traversal.

In the in 11A shown example leads the TTU 700 a top-level tree traversal 1006 and a bottom-level tree traversal 1018 by. In the exemplary embodiment, the two-level traversing (two-level traversing) enables BVH fast beam tracking reactions to dynamic scene changes.

The beam transformation 1014 offers the appropriate transition from top-level tree traversing 1006 to the bottom-level tree traversal 1018 by moving the beam that can be used in the top-level traversing in a first coordinate space (e.g. world space) into another coordinate space (e.g. object space) of the BVH of the bottom-level Traversal is transformed. An example BVH traversal technique using two-level traversal is described in previous literature, see e.g. B. Woop, "A Ray Tracing Hardware Architecture for Dynamic Scenes", Saarland University, 2004 However, embodiments are not limited to this.

In some embodiments, the top level traversal (in world space) is performed in a BVH, which can be dynamically recalculated in response to changes in the scene (e.g. by the SM 132 ), and bottom-level traversal is performed in a BVH of bounding volumes that remain static or essentially static, even if changes occur in the scene. The limitation volumes in the BVH, for the bottom-level tree traversal 1018 (in the object space) can be used to include more detailed information about the scene geometry than the respective limitation volumes that are used in the top-level tree traversal 1006 are used, whereby the modification of the bottom-level traversing BVH in response to scene changes is avoided or at least reduced. This helps speed up beam tracking of dynamic scenes.

Example of top-level tree traversal

The top-level tree traversal 1006 by the TTU 700 receives complets from the L1 cache 1012 and delivers an instance to the beam transformation 1014 for transformation or a miss / end issue 1013 to the SM 132 for processing the closest hit shader 1015 by the SM (this block can also work recursively on the basis of non-leaf nodes / no hit conditions). In the top-level tree traversal 1006 calls a next complete fetch step 1008 the next in step 1010 Complet to be tested at beam intersections from the memory and / or the cache hierarchy, and the beam boundary volume intersection test is performed on the boundary volumes in the retrieved complete.

As described above, an instance node connects a BVH to another BVH that is in a different coordinate system. If a child of the cut boundary volume is an instance node, the ray transform is 1014 able to find a suitable transformation matrix from the L1 cache 1016 read. The TTU 700 transforms the beam into the coordinate system of the child BVH using the corresponding transformation matrix. The U.S. Patent Application No. 14 / 697,480 , which has already been incorporated by reference, describes transformation nodes that connect a first set of nodes in a tree to a second set of nodes, the first and second sets of nodes being in different coordinate systems. The instance nodes in exemplary embodiments can be the transformation node in the U.S. Application No. 14 / 697,480 be similar to. In an alternative, non-instancing mode of the TTU 700 , in 11B shown, the TTU does not carry out a “botton” level tree traversal, and non-instanced tree BVH traversings are carried out by blocks 1008 . 1010 performed, e.g. B. using only one stack. The TTU 700 can, based on what it reads from the BVH and / or the query type, between the instanced operations of 11A and the non-instantiated operations of 11B switch. For example, a particular type of query can limit the TTU to only use the uninstanced operations. Any such intersected instance nodes would be returned to the SM in such a query.

In some non-limiting embodiments, the bound volume intersection test is in step 1010 performed on each bound volume in the retrieved complet before the next complet is retrieved. Other embodiments may use other techniques, such as e.g. B. Traversing the top-level traversing BVH in a way based on depth. That already incorporated by reference U.S. Patent No. 9,582,607 describes one or more complete structures and contents that can be used in exemplary embodiments. The U.S. Patent No. 9,582,607 also describes an example traversal of complets.

If a confinement volume is determined to be intersected by the beam, the child confinement volumes (or references thereto) of the intersected confinement volume are monitored for subsequent intersection testing with the beam and for traversal. In embodiments, one or more stack data structures are used to keep an eye on child boundary volumes that are subsequently tested for intersection with the beam. In some embodiments, a small-sized traversal stack can be used to complete by operating the top-level tree traversal 1006 to be traversed, and to keep an eye on primitives to be tested for intersections, and a larger local batch data structure can be used to track traversal status in the bottom-level tree traversal 1018 to keep in mind.

Example of bottom-level tree traversal

Bottom-level tree traversal 1018 calls a next complete fetch step 1022 the next, in step 1024 Complet to be tested on beam intersections from the memory and / or the cache hierarchy 1020 and the beam confinement volume intersection testing is performed on the confinement volumes in the retrieved complete. As mentioned above, the bottom-level tree traversal can contain completes with boundary volumes in a different coordinate system than the boundary volumes traversed in the tree traversal of the upper level. The bottom-level tree traversal also receives completes from the L1 cache and can be based on non-leaf / no-hit conditions and also with the top-level tree traversal 1006 work recursively or iteratively in yourself based on miss / end detection. Intersection points of the beam with the boundary volumes in the BVH of the lower level can be determined, whereby the beam is transformed into the coordinate system of the read complete from the lower level. The leaf boundary volumes found in the tree traversal of the lower level as cut by the beam then become the beam / triangle intersection 1026 fed.

The leaf editions of the bottom-level tree traversal 1018 become the ray / triangle intersection 1026 (which has both access to the LO cache and the ability to triangle through the L1 cache 1028 read). The L0 complete and triangle caches can have small read-only caches within the TTU 700 his. The ray / triangle intersection 1026 can also leaf editions from the top-level tree traversing 1006 received when certain leaf nodes are reached without traversing an instanced BVH.

After all primitives in the primitive area have been processed, the intersection manager checks the status of the result queue and prepares packets for the batch manager and / or Beam management unit are to be sent to update the attributes and traversing status of the beam, set up the next traversal step of the beam and / or the beam (if necessary) to the SM 132 return. If the result queue contains opaque or alpha intersections found during primitive region processing, then the intersection manager signals the ray manager the parametric length (t) of the closest opaque intersection in the result queue to be the tmax of Record the beam to shorten the beam. To update the traversal status to establish the next traversal step of the beam, the intersection manager signals the batch manager whether there is an opaque intersection from the primitive area in the result queue, whether there are one or more alpha intersections in the result queue, whether that Result queue is full, whether additional alpha intersections have been found in the primitive area that have not been returned to the SM and that do not exist in the result queue, and the index of the next alpha primitive in the primitive area for the to testing ray after the SM has consumed the contents of the result queue (the index of the next primitive in the area after the alpha primitive with the highest storage order from the current primitive area in the result queue).

If the batch manager 740 the package from the intersection manager 722 receives, checks the batch manager 740 the package to determine the next action required to complete the traversal step and start the next one. If the package from the intersection manager 722 indicates that an opaque intersection has been found in the primitive area, and the beam mode bits indicate that the beam should stop traversing as soon as any intersection is found, the batch manager reports 740 return the beam and its result queue to the SM with a traversal status indicating that the traversal is complete (a done flag and / or an empty top-level and bottom-level stack). If the package from the intersection manager 722 indicates that there are opaque or alpha intersections in the result queue and that there are remaining alpha intersections in the primitive area not present in the result queue that the ray hit during primitive area processing and that are not already on the batch manager gives back to the SM 740 return the beam and result queue to the SM, modifying the traversal status to set the cull opaque bit to prevent further processing of opaque primitives in the primitive area and to the first alpha primitive after the highest alpha -Primitive intersection of the primitive area advanced starting index of the primitive area is returned to the SM in the result queue of the beam. If the package from the intersection manager 722 indicating that no opaque or alpha intersections were found when the beam processed the primitive area, the batch manager removes 740 the upper part of the stack entry (corresponding to the finished primitive area) from the active traversal stack. If the package from the batch manager 740 indicates or that there are either opaque intersections in the result queue and the ray mode bits do not indicate that the ray should end traversing once any intersection has been found and / or alpha intersections exist in the result queue but The batch manager does not remove any remaining alpha intersections in the primitive that are not in the result queue and that have not already been returned to the SM 740 the top of the batch entry (corresponding to the finished primitive area) from the active traversal stack and modifies the contents of the result queue to indicate that all intersections in the result queue come from a primitive area whose processing has been completed.

If the active batch is the bottom batch and the bottom batch is empty, the batch management unit sets 740 the active stack on top of the stack. If the top batch is the active batch and the active batch is empty, the batch manager gives 740 return the beam and its result queue to the SM with a traversal status indicating that the traversal is complete (a done flag and / or an empty top-level and bottom-level stack). If the active batch contains one or more batch entries, the batch manager checks 740 the top stack entry and starts the next traversing step. Testing primitives and / or primitive areas for intersections with a beam and returning the results to the SM 132 are described in the pending U.S. Application No. 16 / 101,148 titled "Conservative Watertight Ray Triangle Intersection", the U.S. Application No. 16 / 101,066 with the title "Method for Continued Bounding Volume Hierarchy Traversal on Intersection without Shader Intervention" and the U.S. Application No. 16 / 101,066 entitled "Method for Handling Out-of-Order Opaque and Alpha Ray / Primitive Intersections", which are hereby incorporated by reference in their entirety.

Although the above disclosure is embedded in the specific context of computer graphics and visualization, beam tracing and the disclosed traversal coprocessor could be used for a variety of applications beyond graphics and visualization. Non-limiting examples include sound propagation for realistic sound synthesis, simulation of sonar systems, design of optical elements and systems, particle transport simulation (e.g. for medical physics or experimental high-energy physics), general wave propagation simulation, comparison with LIDAR data for purposes e.g. B. robot or vehicle location and others. For some of these applications, OptiX ™ has been used in the past.

Streaming scheduling cache

In a beam tracker, each beam traverses the acceleration data structure (e.g. BVH) in an individualistic way. It may seem at first glance that the rays shot into a given scene are relatively coherent, since they all start from the same point of view. However, this may only be the case at the beginning of a typical ray tracing process. In the further course of the beam tracing procedure, the coherence with traversing and shading diverges. It usually takes several steps to determine which objects are visible, which light reaches which objects, and which surfaces reflect where. As this divergence spreads in the beams that are being processed, the execution cast on banks of the single-instruction multithreaded engines used by some beam trackers decreases very quickly. In such SIMT beam tracing architectures, only a small portion of the SIMT processors will be active at any given time, while the rest of the processors will be idle. In some embodiments, a streaming multiprocessor executes on a warp, so as the divergence of threads increases, the number of active threads decreases, but the number of warps processed on different streaming multiprocessors does not necessarily decrease until the very end. Accordingly, improvements are possible.

Beam tracing often involves performing a beam intersection query against a pre-built bounded volume hierarchy (BVH). Individual rays in these queries often take similar paths through the hierarchy. Grouping their coherent execution can have performance and performance advantages. Previous implementations outside the tree traversal unit (TTU) generally did not group this implementation and duplicate BVH nodes in register files instead of working directly from the cache using a single data instance.

Streaming cache

The present non-limiting embodiments provide a traversal coprocessor cache that is configured to be configurable, small, and efficient in streaming workload. It is also a cache that does not have to return to the requesting client, but plans to execute along an attached data path using request metadata and data resident in the cache itself. In this way, the cache is not a typical cache, but also acts as a scheduler that uses resident data to appropriately and efficiently group operation / execution of beam requests.

In non-limiting exemplary embodiments, the acceleration data structure is stored in compressed form in memory. As a beam travels through the hierarchy that defines the acceleration data structure, the TTU reads 700 the required data from memory via a cache line of an L1 cache. The TTU 700 then tests the beam against the data that has been read out and made available in the cache line.

To make this process easier, the TTU has 700 via its own internal small but efficient streaming cache 750 , here called "L0 cache"("L zero cache" or "level zero cache"). In the in 9 The non-limiting example shown is the LO cache within the TTU 700 itself. This TTU-LO cache 750 is supported by a larger, more powerful storage system that has an additional L1 cache ("level one cache") and possibly additional cache levels such as B. level 2 cache, etc., which ultimately access main memory 140 enable (see 1 ). In the non-limiting embodiment, the LO cache 750 only from the TTU 700 used and is intended only for this. This LO cache 750 pulls data for use by the TTU 700 and also plans to use this data against the rays that want to test against it. The cache 750 implicitly performs its scheduling function by the order in which it data travels down the data path to the other parts of the TTU 700 streams.

Background - basic cache concepts

As one of ordinary skill in the art knows, conventional "cache memory" has been used in high speed computer architectures for many years. The basic idea behind a cache memory is to place a small, easily accessible memory close to a high-speed processor, typically on the same silicon. The processor issues requests for data from main memory by sending them through the cache. The cache reads the requested data from the main memory and stores the read data in a small local memory, which the processor can access more quickly.

A typical cache reads out data that an executing process needs to run and stores and maintains it. Reading into a typical cache memory is initiated by a process that calls for this data from the main memory. But instead of simply returning the read data to the process, the cache also maintains a copy of the data in local memory close to the process that uses the data. If the process needs the same data again (which can often be the case due to a phenomenon known as "localized execution"), the cache can provide it quickly without having to read it out of main memory again. When the executing process no longer needs the data (e.g. because it has advanced to another part of the process), the data can be thrown out of the cache to make room for other data that the process now needs.

Household chefs will be familiar with the concept of a cache, since a typical household refrigerator is a kind of food cache. There are huge amounts of different ingredients in the refrigeration department of your local grocery store, but having to go all the way to the store every time you need an ingredient would be very annoying. Instead, the household chef occasionally brings the ingredients from the supermarket that are likely to be needed in the next few days and stores them in a household refrigerator. The household refrigerator is just a few steps away from the sink and stove, so the ingredients it contains are easily accessible to the cook. The cook must periodically replenish the contents of the household refrigerator. And the cook may sometimes have to run to the store to get special ingredients that the household fridge doesn't contain.

A typical advantage of a cache is the reduced latency - the time it takes to read data from memory. It is usually much faster for a process to extract data from a local cache than to read it from memory. Because many processes tend to reuse the same data over and over again ("referral locality"), it can be very efficient to maintain a local copy of data that the process can access much faster than if it were reading it from shared memory.

The traversal processor cache also acts as a scheduler

If the storage latency of the TTU 700 (ie the time it takes to read data from the storage system) the only limitation would be a possible solution, a large dedicated L0 cache for the TTU 700 before the L1 cache that the TTU 700 with the SMs 132 shared. However, for reasons of efficiency and to save hardware space, a better design provides TTU task scheduling.

In the non-limiting exemplary embodiments, the TTU builds 700 to the storage system provided and is supported by it, which contains a larger L1 cache, that of one or more SMs 132 , the TTU 700 and texture mapping units is shared. In addition, the TTU 700 however, through their own very small and highly efficient streaming L0 cache 750 , which temporarily stores data read from the L1 cache and also plans the beam execution via its data path.

Operations beam scheduling

To provide high efficiency, the L0 cache provides 750 of the non-limiting embodiment, beam execution scheduling via the data path into the cache itself is ready. The cache performs in non-limiting exemplary embodiments 750 through its beam execution scheduling based on the order in which it meets data requests. In particular, the cache persists 750 keeps track of which rays are waiting for the same data to be returned from the storage system, and then, as soon as it reads the data and stores it in a cache line, approximately simultaneously meets the requirements of all those rays that are on the same Waiting for data.

The cache 750 thus dictates the TTU's execution of any particular collection of currently activated beams currently waiting for the same data 700 time coherence by essentially forcing the TTU to execute all of these beams at approximately the same time. Since all rays in the group execute at approximately the same time and each takes approximately the same time to execute, the cache is bundled 750 the beams effectively to be coherent in time by operating them at about the same time. These focused rays continue to iterate each time the recursive traversal of the Repeat the acceleration data structure in a time-coherent manner as long as the beams continue to request the same data for each iteration.

The bundling of rays through the cache 750 in a time-coherent manner by providing them with the data they are waiting for at about the same time, effectively planning the TTU's next successive data requests 700 for these rays so that they also occur approximately simultaneously, which means that the cache can fulfill these successive data requests approximately simultaneously with the same new data read from the storage system.

An advantage of the L0 cache 750 grouping of beams in this manner is that the resulting set of beam requests made on the same data take approximately the same traversal path through the hierarchical data structure and therefore likely for each of several successive iterations, the same data from the L0 cache approximately simultaneously 750 request - even if each individual beam request is not formally coordinated with any other beam request. Due to the fact that the L0 cache 750 via the data path to the TTU blocks 710 . 712 . 740 plans, the L0 cache effectively plans its own future memory system requirements on behalf of the beams it bundles to minimize latency while maintaining acceptable performance with a relatively small cache data RAM with a relatively small number of cache lines to provide. This bundling also has the effect of improving the reference locality in the L1 cache and any downstream caches.

When beams in the bundle begin to diverge requesting different traversal data, the cache stops 750 at the same time with other rays in the bundle to operate them. The divergence occurs when the size of the bounding boxes in the BVH decreases at lower levels. What may have been tiny, ignorable differences in origin or direction at an early stage now means that previously bundled beams miss or hit these smaller boundary boxes differently.

Hierarchical data structure traversing - How beam operations are activated

In the non-limiting embodiment, the TTU-L0 cache 750 completes feeds into a data path, at the end of which there is a batch management block 740 located. The batch management block 740 in the non-limiting example traversal coprocessor architecture determines which operation is performed next for a given beam. Thus the batch management block determines 740 whether recursion is required within the acceleration data structure (e.g. to further divide a limit volume for further successive beam-complete tests into its children) or whether a leaf node in the acceleration data structure has been reached (in this case the batch management unit becomes one Plan the beam-triangle test from the beam-triangle test block 720 is to be carried out). If the batch management block 740 determines that additional recursion is required along the bounding box path, then the batch management block 740 initiate a further request to the L0 cache 750 in order to read out the next complete in the hierarchical data structure, which is for a next successive iteration of the beam complete test by the block 710 is required.

The complete cache 752 is therefore linear in the data path. It is part of an execute-request-execute-request loop which, since it is within the data path, has information about which completes have been read and are available, and which completes lead to cache failures that cause additional memory accesses to the Storage 140 require.

By grouping the requirements so that many beam complete tests that are tested against the same complete data are scheduled to be performed more or less concurrently, beams become “coherent” - which means that they are are grouped to do their beam complete tests against the same complete data - stay grouped for additional tests. By continuing to group these beams as a bundle of coherent beams, the number of redundant memory accesses in order to read out the same complete data again and again is considerably reduced, and therefore the TTU works 700 much more efficient. In other words, the beams that take more or less the same traversal path through the acceleration data structure tend to be grouped for the purpose of performing the beam complete test - not only for the current test execution, but also for other successive test executions because this bundle of " coherent “rays continue along the traversal of the acceleration data structure. This significantly increases the efficiency of any request made to memory by using it across a number of different beams, and significantly reduces hardware power consumption.

Around the beam complete test blocks 710 The complete cache should continuously supply new data 752 ideally in everyone Cycle deliver new complete data. Expecting a more distant L1 cache to offer such high responsiveness may be unrealistic. On the contrary, the more distant L1 cache could end up being overloaded and becoming a bottleneck for the beam complete tests that the TTU 700 can perform at extremely high rates. In the non-limiting embodiment, the L1 cache is handled by the TTU 700 , one or more SMs 132 and one or more texture mapping units. The L0 cache 750 within the TTU 700 thus offers a buffering effect, so that memory access takes place via the TTU or is loaded onto the TTU's own L0 cache, and those memory accesses do not interfere with memory accesses which the SMs 132 and may perform texture mapping units through the shared L1 cache. The L0 cache 750 within the TTU 700 thus enables bandwidth amplification without significant additional complexity of the storage system. As a result of the L0 cache being hit rather than going to the L1 cache or even main memory, the TTU's L0 cache 750 has the effect of reducing latency and increasing bandwidth.

Great advantages result from the ability of the L0 cache structure 750 to group the beam execution based on the data that the grouped beams need to traverse the acceleration data structure. The SM 132 who the TTU 700 Submitting rays for complete testing may have no idea in a general case that these rays are "coherent". The beams can coexist in three-dimensional space, but typically traverse the acceleration data structure completely independently of one another. So whether certain rays are coherent does not only depend on the spatial positions of the rays (which of the SM 132 itself or through other auxiliary operations such as B. can even determine artificial intelligence), but also from the detailed determined acceleration data structure that traverses the beams. Since the SM 132 , who requests the beam complete tests, does not necessarily have direct access to the detailed acceleration data structure (would still have the time to analyze the acceleration data structure, even if he had access), the SM relies on the TTU 700 to accelerate the data structure traversal for rays that the SM 132 always submitted by the TTU for testing. In the non-limiting embodiment, the TTU's own L0 cache 750 700 an additional level of intelligence ready, based on independently traversing beams requesting the same complete data at approximately the same time, discovers that these beams are consistently traversing the acceleration data structure. By initially grouping these coherent beams so that they perform beam complete tests at approximately the same time, these coherent beams are again given the opportunity to be grouped for successive tests when the beams traverse the acceleration data structure. The TTU-L0 cache 750 thus does not rely on any given grouping of beams as coherent (although it is of a natural order of supply of beams by the requesting SM 132 Uses, which is simply based on the spatial proximity of the beams as presented by the SM for testing), but instead observes that based on the data that the beams need for testing, while traversing the acceleration data structure, these beams are the same parts of the Traverse acceleration data structure and can be grouped for efficiency.

Beam activation

In general, everything the TTU 700 in the non-limiting example implementation performed on behalf of a ray. The L0 cache thus carries 750 generally perform tasks on behalf of rays, as opposed to performing other memory read tasks. So all requests from the TTU-L0 cache 750 to the storage system including the L1 cache on behalf of a beam. In the non-limiting embodiment, the parameters that determine a beam include, for example, origin, direction, tmin and tmax. The SM 132 in his request to the TTU 700 explicitly set and not loaded from memory. What directly for use by the TTU 700 loads are completes, triangles and instance nodes. The complete system in the non-limiting embodiment represents bounding boxes and associated pointers. The complete is in a compressed form in memory 140 saved, and the cache 750 reads them from the storage system in this compressed format.

In non-limiting embodiments, it is the batch manager 740 that updates the TTU stack as beams traverse the acceleration data structure recursively down and then up again. It's the batch manager 740 which, for example, determines whether a particular beam requires further recursive traversal through the acceleration data structure, or whether the beam now requires, for example, a beam-triangle test using the other data path. In the non-limiting embodiment, the batch manager 740 in turn choose which beam to activate next, which is essentially a random choice with respect to any particular time coherency of data requirements of that particular ray. A randomly selected beam is thus selected to go along the beam complete data path, and it is this randomly selected beam that makes a new data request to the L0 cache 750 initiated after a complete that this randomly selected beam needs for its own beam complete test. It is the L0 cache 750 who detects any way to group this randomly selected beam with other beams that need the same data to store the TTU's memory accesses 700 to make it more efficient. Accordingly, the L0 cache 750 dictate a coherent beams that traverse similar paths through the acceleration data structure to a group execution order, although the formal beam execution scheduling process that the TTU 700 via the batch management unit 740 and actually complete the scheduling based on what may be a random selection process, select beams for activation.

In the exemplary embodiments, each beam starts its traversal of the acceleration data structure from a point at which it is started by an SM 132 has requested. The L0 cache 750 finds ways to put rays behind certain blocks of memory. Since more beams need the same block of memory while a pending request is being met, the L0 cache shifts 750 these new requirements in a pending requirements table. As soon as a certain cache line has been read out of the main memory and is valid, all rays which are waiting for this data can then be derived into the data pipe.

Because any of the L0 cache 750 processed request to memory is initiated on behalf of a specific beam, some readout takes place from higher-level memory in the L0 cache, also with respect to a specific requesting beam. For example, the first ray that requests a certain complete of memory becomes a request through the L0 cache 750 Initiate in order to read this complete from higher-level memory. In the non-limiting embodiment, this request on behalf of a specific beam is carried out by the batch management unit 740 initiated in potential collaboration with the complete scheduler based on the beams available for activation.

The batch management unit 740 looks for a beam for activation, and this selected beam activates and initiates a request for a complete via the L0 cache 750 , In some cases, the batch manager determines 740 possibly that the request on behalf of this particular beam is not for a complete, but instead the other data path is to be used to carry out a beam-triangle test and / or an instance transformation.

The batch management unit 740 will repeat this same process independently for the next beam that is available for activation. This next beam is activated independently and independently generates its own memory request via the L0 cache 750 , In the non-limiting embodiment, the L0 cache uses 750 however, its internal status information to determine whether the cache has already sent a request for the same data to higher-level storage. If the L0 cache 750 has already read the same data and this data in a cache line of the data RAM 1208 (see below) are available, the L0 cache can of course the requesting process within the TTU 700 provide. In this case, however, assume that the L0 cache 750 has made a request for the same data on behalf of another beam and has not yet received the data from the higher-level memory, so that the data are not yet available and are valid in a cache line. At this point the L0 cache registers 750 the new request and effectively links or groups the beam that makes the new request with the beam that made the previous request since both beams request the same data.

The L0 cache 750 can link or group any number of beams in this way. As soon as the data has been read and is valid in a cache line, the L0 cache can 750 supply all of these grouped beams with the data at the same or approximately the same time. This scheduling offers higher levels of efficiency, even if, in the non-limiting embodiment, the only rays ever specific data from the L0 cache 750 received, are those rays that have requested this particular data.

Thus, in non-limiting embodiments, a beam will come into the cache that designates a complete against which the beam is to test. A complete is typically read out on a single cache line. In response to the beam complete test 710 created the TTU 700 a request to memory 140 , the complete in the L0 complete cache 752 read out so that it is for the beam complete test block 710 is available for this beam. It turns out that other beams may want to test against exactly the same complete data.

To enable this shared use of the read data and unnecessary To minimize additional memory reads, the non-limiting embodiment groups rays using an entry of a pending address table (PAT) 1206 (see below). If this complete data is in the L0 complete cache 752 the complete cache plans to be read out 752 essentially the beam complete test for all beams that have been waiting for this complete data by addressing the memory access requirements of the beam complete test block 710 all served at the same time or at about the same time. These operations could work simultaneously in parallel in some example implementations, but there is one in the 9 The example shown may only have a limited number (e.g. two) beam complete tests 710 , so that some scheduled beam complete tests may be performed simultaneously, while others may be scheduled to be performed sequentially (e.g., one after the other), but still in a timely manner (e.g., one after the other). This scheduling is done as part of the data path (ie simply by having the cache 750 provides the data for these now grouped beams in a time-coherent manner) rather than intervention by an SM 132 , an explicit beam scheduler or other return to a client is required.

TTU 700 can provide a complete scheduler, but can do that from the cache 750 performed scheduling for this complete scheduler and simply optimizes the performance of the cache in relation to whatever collection of rays the complete scheduler plans to execute. In particular, the cache 750 To dictate time coherency to the execution of beams that need the same data simply by serving their individual independent data requests at about the same time and then proceeding from traversal iteration to traversal iteration to increase the efficiency of the cache in serving Increase data requirements for all rays that the TTU 700 currently being treated. The goal is not to speed up memory performance or the execution associated with any particular beam, but to group the execution of beams that traverse the same parts of the acceleration data structure so that the streaming cache 750 can serve all of these beams with the same data that the cache receives from the storage system (e.g., an LI cache) to reduce latency while minimizing the size of the streaming cache.

TTU queries provide initial groupings of beam operations

The L0 cache 750 can be effective in grouping beams in this way by taking advantage of the time coherency of traversing certain parts of the acceleration data structure of multiple beams that the SMs 132 the TTU 700 submit. This time coherence begins with the query itself. A query to the TTU 700 from an SM 132 will typically consist of a number of rays (e.g. 32 different rays in some embodiments). This means that the TTU 700 starts from a certain number of beams at the same point in the acceleration data structure (e.g. the root node of the limitation volume hierarchy tree). These initial rays are “primary rays”, ie they typically start from a standpoint position and are shot into the three-dimensional scene in often similar directions. These rays will often take similar initial paths through the hierarchical data structure because they emanate from the same viewpoint position within the scene and will be shot into the scene in similar directions. They are not guaranteed to take similar traversal paths, but they will likely do so.

The TTU-L0 cache maintains and uses the initial beam operations grouping

Since the SM 132 these beams are initially grouped to simultaneously enter the TTU 700 to be shot, the TTU sends the complete request for these beams to the L0 cache one after the other 750 , The L0 cache 750 therefore, if these complete requests come from these different beams more or less than a group, this grouping is retained when memory values are read from the higher-level memory, and this grouping is maintained from iteration to iteration as the beams continue to traverse the acceleration data structure.

When the rays begin to diverge (that is, they no longer take the same path through the data hierarchy because they encounter different objects or other parts of the hierarchy), then that from the L0 cache 750 tried grouping less effectively. Such a divergence leads to singletons ("single pieces"), in which there is only one beam in a group, which is waiting for a certain complete to be read from the higher storage levels. For primary beam traversal, the singleton ratio is relatively low. When modeling reflections in which the primary rays have been reflected from different surfaces of the scene and from the different objects in the scene in random directions through the Scene, the traversal becomes much less coherent and the ratio of singletons to grouped rays increases. But even if this happens because all rays still traverse the same BVH, all rays still start traversing at the top of the tree and therefore require at least the same initial completion. So the L0 cache 750 Even in the case of traversing secondary or reflected rays, benefit from a certain coherence between grouped rays if they traverse near the top of the hierarchical data structure of the BVH. As these secondary beams begin to traverse down into the hierarchical data structure, coherence begins to be lost and the singletons ratio becomes much higher.

The TTU-L0 cache discovers beam operations that follow the same or similar traversal paths through a bounding volume hierarchy

In the non-limiting embodiment, it is the grouping in the L0 cache 750 itself, which promotes coherence, not some explicit scheduling by the batch manager 740 or even through SMs 132 , If the SM queries start with some coherence, the L0 cache tends 750 to recognize and maintain coherence and use it as a way to increase memory access efficiency. The L0 cache 750 is, however, able to achieve coherence independently of that in SM queries 132 discover the coherence contained. For example, the L0 cache 750 able to discover data coherence for rays that all access the top of the hierarchical data structure, even if the paths that these rays will eventually take through the data structure are very divergent and incoherent.

In the non-limiting embodiment, each SM leads 132 a variety of threads, such as B. 32 Threads, each thread managing a specific ray. If the SM 132 32 Threads executes includes its query to the TTU 700 a stack of 32 Rays. As soon as this 32 Rays within the TTU 700 they are processed independently of each other until it is time to send the results back to the requesting SM from the TTU 132 return. The TTU 700 can process these beams in any order and with any staggering, the main time constraint being that the TTU reports the SMs requesting 132 sends beam traversal results in the same grouping as the original requests.

In the non-limiting embodiment, there are multiple SMs 132 , and in some embodiments serves a single TTU 700 several (e.g. two) SMs 132 , In the non-limiting embodiment, the TTU processes 700 Beam requirements from these two different SMs 132 , These SMs 132 do not coordinate their respective beam requirements in the form of time coherence, but the TTU-L0 cache can 750 discover that from an SM 132 fired beams follow the same traversal path as beams fired by another SM, and can group beam requests from different SMs to make reading data for all of these beams more efficient over a series of data structure traversal iterations. The L0 cache 750 Thus, in the non-limiting embodiment, is able to achieve coherence between beams in different queries from different SMs 132 to discover and serve these queries more efficiently, based on the fact that the L0 cache 750 Exploits coherence between these different beams - coherence which the SMs 132 itself not recognized or the TTU 700 have communicated. In the non-limiting embodiment, there can be hundreds of beams that are active at the same time, and it is up to the TTU to operate all of these beams as efficiently as possible. The L0 cache 750 thus acts effectively as a scheduler by promoting coherence as a result of the grouping that the L0 cache performs.

Non-limiting example grouping analogy

As a rough analogy, think of a cafeteria food service row that is barely staffed with food service staff. The food serving range is designed to serve different dishes from different serving stations: meat is served from a meat serving station, fish is served from a fish serving station, and vegetarian dishes are served from a vegetarian serving station. But the boss brought the bad news to the server: two other employees reported sick and only one server now has to serve himself from all three stations. Since servers are scarce, it is not possible to assign a different server to each serving station. Worse, any other food selection requires that the individual food server go to another serving station and pick up other serving utensils.

If the guests are served in the order of "first come, first served", the server will keep storing and picking up various serving utensils and switching between the serving stations. Suppose the first guest orders meat and the server is at the vegetarian station. The server must put down the vegetarian serving utensil to the meat station go, pick up the meat serving utensil and serve meat to the guest. The next guest in the row may also want meat that the server can serve very efficiently, since he is already at the meat station. But let's say the next guest wants fish. The server must put down the meat serving utensil, go to the fish station, pick up the fish serving utensil and fulfill the guest's request for fish. In the meantime, the serving line is getting longer and longer, with hungry guests eager to get their dishes.

The server then gets a good idea. He calls to the guests and asks them to form three different rows: one for meat, a second for fish and a third for vegetarians. The server serves meat to the first ten guests in the meat line in quick succession; then he goes to the fish station to serve the first ten guests waiting for fish; and then he goes to the vegetarian station to serve vegetarian dishes to the guests waiting there. By grouping the guests based on the food they are waiting for, the server has increased their overall efficiency and effectively planned when the guests will be served (serving meat eaters, then seafood eaters, then vegetarians, etc.). Individual guests in the front row may wait longer to receive their individual dishes than if everyone were served in the "first come, first served" order, but the overall effect is that all guests average faster and more efficiently from one less hectic food servers are served.

Continuing the analogy above, it is assumed that all guests are entitled to a reference. With this knowledge, the server modifies his strategy. As a first aid, the server decides to serve all the vegetarians in the row as soon as the tray with vegetarian dishes comes out of the kitchen, and only changes the station after all the vegetarians have been served. He then serves all of the seafood eaters before serving any meat eater. Since the vegetarians were all initially served at about the same time, they will all appear for their reference at about the same time. This means that based on the time when the guests in the cafeteria express their desires for reference, the server will be able to follow a similar strategy, namely to serve all vegetarians from the vegetarian station, then to serve all fish eaters their reference from the fish serving station and finally to serve all meat eaters their reference from the meat serving station - which minimizes the number of times the server needs to be changed from one serving station to another, although the server does not has explicit control over when any guest comes for reference. The server will be happy to accommodate guests who initially ordered fish, but now want a vegetarian lookup, but based on their initial wishes, the server has effectively dictated a time grouping to the guests, so that they will of course appear for a lookup in a coherent manner - which increases the efficiency of the server to serve the entire dish including one or more lookups - and in the case of the TTU 700 with a large acceleration data structure - is further increased.

More detailed example of the structure & implementation of the streaming cache 750

12 shows a non-limiting embodiment of an implementation of the cache memory 750 , In this example, the structure of the cache includes 750 a pending requirements table (PRT) 1202 , a label structure (TAG) 1204 , a pending address table (PAT) 1206 , a data RAM (DAT) 1208 and a logic circuit 1210 ,

The logic circuit 1210 assigns every request to the cache 750 an entry in the PRT table 1202 to. Each PRT table 1202 entry holds metadata on request and has a pointer to a single PAT 1206 entry. In the non-limiting embodiment, each request received by the L0 cache 750 an entry is submitted in the pending requirements table 1202 , The pending requirements table 1202 is therefore used to distinguish clear requirements. The PRT table 1202 entry remains valid until the cache fulfills the request by sending the data for that request to the attached data path.

All cache requirements 750 check the labels (TAG) 1204 to a previous valid entry at the same address and in the same context. The TAG 1204 can be multi-way, quantity-associative or directly mapped, as the expert recognizes. As in 12 shown, the TAG 1204 for example, an entry corresponding to each row of data in the data RAM 1208 have, the label the main memory address ("ADDR") of the data block stored in the cache line, an offset or pointer ("OFF") in the data RAM 1208 , which indicates which of the data lines stores the data block read from this memory address and contains a valid (“V”) bit, which indicates whether the data in the corresponding cache line is valid. In the non-limiting embodiment, the cache is 750 a read-only memory, so no memory mark in the TAG 1204 is necessary that indicate a need would write modified data back to main memory, but in other embodiments such a memory tag and writeability could be provided.

Hit and failure detection

1. (1) Datentreffer ohne vorherige Anforderungen an die gleichen im Cache residenten Daten (Ausgang „Nein“ zum Entscheidungsblock 1256),
2. (2) Datentreffer mit vorheriger bzw. vorherigen Anforderung(en) an die gleichen im Cache residenten Daten (Ausgang „Ja“ zum Entscheidungsblock 1256),
3. (3) Datenfehlschlag mit vorheriger bzw. vorherigen Anforderung(en) an die gleichen im Cache residenten Daten (Ausgang „Ja“ zum Entscheidungsblock 1258), oder
4. (4) Datenfehlschlag ohne vorherige Anforderungen an die gleichen im Cache residenten Daten (Ausgang „Nein“ zum Entscheidungsblock 1258).

As in 13 shown, depending on the status, is the result of a TAG-1204 check of a new address request to the cache 750 either:

1. (1) Data hit without previous requests for the same data residing in the cache (exit "No" to the decision block 1256 )
2. (2) Data hit with previous or previous request (s) for the same data residing in the cache (output "Yes" to the decision block 1256 )
3. (3) Data failure with previous or previous request (s) for the same data residing in the cache ("Yes" output to the decision block 1258 ), or
4. (4) Data failure without previous requests for the same data residing in the cache (exit "No" to the decision block 1258 ).

A data “hit” here means that valid data in response to the request is already in the data RAM 1208 are resident. A data "failure" means that valid data in response to the request is not already in the data RAM 1208 are resident and must be read from higher memory levels. A "hit" or "failure" is typically obtained by comparing the memory address of the incoming request with that in the TAG 1204 stored memory addresses determined.

In the cases 1 and 4 a TAG-1204 entry is assigned (see below) (blocks 1262 . 1266 ), which may throw out a previous label entry. A label entry remains valid until it is either removed or the associated PAT 1206 entry and data RAM 1208 entry are invalidated. That is, a label entry need not be valid for the length of the request that created it. Some example implementations could link the TAG-1204 and PAT-1206 entries together. In this case, the label entry remains valid while the PAT entry is valid and / or the data RAM is valid.

A PAT-1206 entry is only in those cases 1 and 4 assigned above (blocks 1260 . 1264 ). The PAT-1206 entry contains all of the information required to request data from other levels of the cache hierarchy and non-unique data along the attached data path to the rest of the TTU 700 to send.

In the embodiment, the pending address table (PAT) creates 1206 the unique groups for the unique addresses requested by the other storage levels. There is an entry in the pending address table for each unique address 1206 for every request for that address within the cache 750 is active at any time. The TAG 1204 maps an address to a current entry in the pending address table 1206 to. With TAG-1204 access it is possible to cache the same without prior request 750 resident address to have a data hit of something in the data RAM 1208 is resident. This type of request allows both a PRT-1202 entry and a PAT-1206 entry to be assigned.

When assigned, the TAG-1204 entry is pointered to the PAT 1206 -Entry (blocks 1262 . 1266 ) updated so that subsequent requirements of this hit (TAG test result cases 2 and 3 above) can be assigned to the same group. Some implementations could use the TAG-1204 and PAT- 1206 - Link entries together. Multiple PRT-1202 entries can refer to the same PAT-1206 entry. The PAT-1206 entry remains valid until all assigned PRT-1202 entries related to it have been scheduled.

Groups of in the PRT 1202 Tracked requests associated with the same PAT-1206 entry are referred to herein as "PAT groups". Data requests to other levels of the cache are made based on the contents of the PAT-1206 entry. The returning data is stored in the data RAM (DAT) 1208 saved. The DAT 1208 uses an assignment-on-return policy. In one embodiment, if there are no available entries in the data RAM 1208 there, the answer ignored (dropped), and the cache 750 makes a new request to higher-level storage. To ensure progress, data is dropped only a few times before any new requests are withheld until the data comes back and finds a valid entry (see below).

As explained above, another type of request is a data hit where a previous request for the same data is already resident in the cache. This means that a previous request has already assigned a PAT-1206 entry. In this case, the cache 750 don't make an additional requirement, just allow the current request to be piggybacked from the previous request. In this case, the PRT-1202 entry corresponding to this last request is written to provide a pointer to the PAT-1206 entry that was previously requested.

A third option for the label is a data failure with the previous request for the same data already in the cache. This means that the previous request failed and the cache 750 must send the request to memory to read the data from memory. In this case, the current request simply piggybacks that previous request to memory. A new memory request is not made and the PRT-1202 entry is simply assigned to the previous request.

In the event of a data failure without a prior request, the request that fails will place a request on the storage hierarchy. It assigns a PAT-1206 entry and a PRT-1202 entry. Finally, the data is read from the memory and in the data RAM 1208 saved. When this happens, the PAT-1206 entry is activated, which has been waiting for the data to be read from the memory. If this PAT-1206 entry is activated, all PRT-1202 entries that are assigned to this PAT entry are also activated. This information is then kept within the TTU 700 sent along the data path. This is an example of how the TTU-L0 cache 750 the beams are grouped for execution - simply by the TTU-L0 cache 750 the various requirements at about the same time along the data path, for example to the beam complete tests 710 lets forward.

Scheduling beam processing through the data path

As soon as he is in DAT 1208 if resident, the TAG-1204 entry, if still associated with the PAT-1206 entry for the data, is updated to point to that data RAM entry. Subsequent requests can then meet this data entry.

If the data for a PAT 1206 are resident in the data RAM (either have just been written or already exist in the event of a hit), this PAT-1206 entry is activated. After winning arbitration among other activated PAT-1206 entries, the entry is activated and sends the resident data (plus PRT metadata) from the data RAM 1208 (Block 1272 ), empties the associated PRT-1202 entries and removes the assignment of the PAT entries. A subsequent request that could have met this PAT group (which has now been served) will instead start a new PAT group. In this way, all entries in a PAT group are planned together.

In some implementations, multiple data paths are attached to the cache. In this case, each path can be supplied independently of different PAT groups or of elements of the same PAT group that are distributed over the data paths.

Return Assignment & Capacity Error Handling

One of the other aspects of this TTU-L0 cache 750 is that the data RAM it contains 1208 works on the principle of allocation-on-return policy. This allows the data RAM 1208 be kept very small. For example, in an operating mode 100 labels 1204 be pending, all using only eight rows of data from data RAM 1208 is sufficient. When data comes back from memory, all of these rows of data are locked correctly because there are many pending requests trying to read data from the cache, and a backup copy of these pending requests is made. The cache 750 will drop the request and create a whole new request to the L1 cache. Therefore, if the allocation-on-return mechanism fails, the L0 cache tries 750 just the request again. In the non-limiting embodiment, there is a limit on the number of times that any given request can be repeated before the request enters a critical mode in which no new requests are allowed until the previous requests are met. This prevents new requests from coming in and older requests from starving.

As for the L1 cache, the repeated request is a completely separate and new request. From the perspective of the L0 cache 750 has the L0 the data dropped because there was no place to store it. The L0 cache 750 repeats the request and sends a new request to the L1 cache. However, this new request is identical to the original request from the point of view of the L1 cache and can be met in the same way if the data is still resident in the L1 cache. The only difference is that in the non-limiting embodiment, the L0 cache 750 records that it has previously requested the same data and that it is retrying based on unavailability of space in the data RAM 1208 is about them save as soon as the L1 cache has provided them. If, in the non-limiting embodiment, this process is repeated too many times (in one embodiment, “too many times” means that it is executed twice in total), the TTU-L0 cache prevents 750 that new requests come in to prevent space in the data RAM 1208 is filled with new requirements if older requirements remain unfulfilled. This enables the older requirements to make progress.

For example, suppose it is 64 pending L0 cache requirements 750 gives. Assume that all of these requests request the same address. The TTU-L0 cache 750 only makes a corresponding request to the L1 cache to all 64 To meet requirements for the same data. When this data returns from the L1 cache and into the data RAM 1208 of the L0 cache 750 there are written 64 Requests that the L0 cache plans to execute based on the return of this data. The rate at which the non-limiting embodiment schedules beam tests for execution can be one per cycle. It can therefore 64 Cycles take around 64 L0 cache requests 750 to empty. Other implementations are possible.

Now suppose there was just behind 64 Requests for the same additional data 64 Requests, each request requesting a different address. Suppose the L1 cache starts on this one by one 64 respond to new requirements. After the L1 cache has responded to a certain number of requests (e.g., seven new requests), the data RAM is 1208 in the TTU-L0 cache 750 completely filled - since the data RAM must keep the first value as long as it has the 64 served original requests, all using the same address. Since the L1 cache continues to deliver data in response to the new requests, the TTU-L0 cache becomes 750 determine that the data RAM 1208 is completely filled with valid data that is waiting to be used and cannot yet be overwritten.

In the non-limiting embodiment, there is data RAM 1208 no space for the eighth response for new data returned from the L1 cache, so the L0 cache drops the response and makes a new request to the L1 cache for the same address. In this case the back pressure is in the L0 cache 750 generated by one or typically a small number of data valid conditions in which the L0 cache continues to deliver the same data to a relatively large number of beam requests, each of which requested the same data, but is the TTU 700 unable to make all of these requests at once, and instead relies on the L0 cache to keep the data while the TTU is working on the various large number of requests that use this data. In this case, the data valid cache lines that provide data together for a large number of requests remain locked longer than would be ideal - which binds the cache line, the TTU 700 Give time to complete the various requests that use this data.

Reconfigurable streaming cache

In one non-limiting embodiment, data RAM 1208 has N rows, each containing M bytes of data, for a total data RAM size of N x M bytes. The TAG 1204 and the PAT table 1206 each contain X entries, and the PRT 1202 has Y entries. This means that at any given time up to Y requests from the L0 cache 750 can be served, but at any given time, there can be a maximum of only X requests to the L1 cache pending. In this implementation, there is a maximum of N read data blocks in the data RAM at any given time 1208 resident.

In the non-limiting embodiment, the sizes are PRT 1202 , DAY 1204 , PAT 1206 and DAT 1208 all independently configurable. Even sizes down to individual entries are allowed. For example, a single entry TAG 1204 only group consecutive requests to the same address. Or a single entry DAT 1208 would only allow to act on a single line of data at any time. In non-limiting embodiments, each of these parameters can be configured completely independently. For example, it would be possible to use the L0 cache 750 to configure it to have a single label entry 64 Data RAM entries and 64 Has PRT entries. The size ratio also plays a role, with the PRT / PAT ratio potentially being determined by the mean PAT group size.

The ability to cache 750 dimensioning very small while still maintaining acceptable performance enables good space savings. By including a scheduler in the cache design, requests with similar execution requirements can be grouped, which can improve performance.

In one embodiment, TTU 700 for example handle hundreds of rays at the same time, and yet the streaming cache could 750 be very small (e.g. 8 cache lines). The streaming cache 750 Keeping small, saves chip area and power consumption without increasing latency by taking advantage of inherent coherence properties of independent parallel beams that come from the same point of view and traverse the same parts of the BVH acceleration data structure.

Alternative, non-limiting example implementations

While it may be possible in some non-limiting example implementations, for the TTU to schedule beam complete tests 700 to use the same cache as that used for reading out other data by SMs 132 , Texture maps and the like is used, offers a separate dedicated L0 complete cache 752 within the TTU 700 reduced latency and is therefore helpful to increase the speed at which the TTU 700 the beam complete test 710 accelerated.

For example, it might be possible to make such a request using the L1 cache. The L1 cache would provide the requested information, some in the order of the requests received (if the data was already available in the L1 cache) and some not in the order (e.g. in the case of a cache failure reading out) the store 140 ) Requires. With such a design, the return of the data from the L1 cache would be in response to any request made. This would require the return of data on one row of data for each row for each request that is made. In contrast, the non-limiting exemplary embodiment enables return data from the complete cache to be returned 752 only once for each group of rays that request and need the data to perform the beam complete test against those rays. This saves both electricity and access time significantly.

Allocate cache lines on return

Additional features of this sample L0 TTU cache 750 are remarkable. With traditional caches of earlier systems, failures cause cache lines to be allocated. In contrast, in the non-limiting embodiment, there is a table of requests that are assigned on failure. Here, cache lines are not allocated on failure, but instead are allocated on return from the L1 cache.

When a line of data comes back and now data in data RAM 1208 are available, the PAT entry 1206 activated at this time. There may be several PAT-1206 entries that are activated and ready for use. In the non-limiting embodiment, the TTU-L0 cache chooses 750 between such activated and ready-to-use entries in the PAT table 1206 using a series method. In the non-limiting embodiment, there is no elder type and first return policy. Instead, the TTU-L0 cache serves 750 TTU requirements using an in-line algorithm that also takes into account inter-group operation - because when the L0 cache starts serving a particular PAT-1206 entry, it will provide all the different processes that require that particular address, before selecting another PAT entry to deliver to the TTU. In the non-limiting embodiment, each read from the L1 cache relates to only a single PAT-1206 entry.

In the non-limiting embodiment, there can be two variants of how the PAT entries are implemented. In a sample implementation, it is possible to create a PAT 1206 link with a TAG-1204 line. In another example implementation, a PAT-1206 entry is not linked to a TAG-1204 line. In the event that the PAT-1206 entry is linked to a TAG-1204 line, the label is kept valid as long as the PAT-1206 entry is valid. This creates the potential for back pressure on the label side. This will happen when a label is full of lines that are for PAT-1206 entries. In this case, each PAT-1206 entry is really a unique address. This is because, in the non-limiting embodiment, each unique address can have only a single PAT-1206 entry. All cached requests for this address are grouped.

In another, non-limiting example implementation, the PAT-1206 entries are separated from the TAG-1204 entries. In such an implementation, the TAG-1204 entry need not be valid for the period of time that a PAT-1206 entry is valid. In this case it is possible that the L0 cache 750 makes a request to the L1 cache that results in a fetch from the L1 cache, and then an intermediate request takes place. Because the label 1204 a label with the new request can not be assigned to the pending request that is still pending in the storage hierarchy. This means that it is the PAT 1206 used to track pending requests and to track whether or not they have been met, and the TAG structure 1204 is not used for such persecution. In this situation, however, there will be no grouping between new and pending requests - that is, those in the cache 750 incoming new request is not grouped with the request that has already been submitted to the L1 cache and is waiting for a response from the L1 cache. This results in two or more PAT-1206 entries for the same address, and grouping occurs when they are cached, provided the PRT-1202 entries are associated with each group. In this embodiment, it is possible to have data returning from the L1 cache that is identical to data for the same address as a unique PAT-1206 entry. Each PAT-1206 entry will have a single piece of data that will come back or already in the data cache 750 If there was a data hit, the resident is on the label 1204 has been accessed for the first time. In this case, each PAT-1206 entry corresponds to data to which the TTU 700 will access.

In the non-limiting embodiment, the L0 cache is 750 a read-only cache, and the system assumes that the data will not change between successive read accesses to the L0 cache. This is a reasonable assumption since the data to be read out are read from an acceleration data structure which is not necessarily changed dynamically during execution, but which is stored in advance and which is easily accessed for traversing by certain rays. If the data was subject to change, the L0 cache would have to 750 explicitly invalidate the data to avoid inconsistencies between successive reads from the same memory address.

In some non-limiting embodiments, it may be possible to exploit the coherence between different beam requests using a read path other than a cache. For example, in some embodiments it may be possible to use the L0 complete cache 752 to be replaced by a buffer that meets the requirements for higher storage levels such as B. provides the L1 cache without linking it to previous beam requests. In such an implementation, the ability to take advantage of redundant requirements for coherent beams could be realized with a single TAG 1204 implementation (which the design discussed above actually allows by programming the usable size and depth of the data RAM). In this reduced example, the only grouping available is that relating to the immediately previous request - either the last data read is the same data that was just requested, or it is different data. This example implementation would work less like a cache and more like a buffer, but would still be able to take advantage of beam data coherence by successively grouping beam requests using the same data read from higher levels of memory.

It is also possible to have an L0 cache 750 which does not directly use this grouping aspect discussed above. Such a design would reduce memory traffic to enable more efficient memory reading, but would not further optimize TTU 700 operation by taking advantage of beam scheduling capabilities based on beams that require the same data that is likely to be read Follow the same or similar traversal paths through a bounding volume hierarchy. In such an implementation, the grouping that would be performed would essentially be an in-order servicing of requests.

Example ray tracing imaging pipeline

The beam tracing and other capabilities described above can be used in a variety of ways. For example, not only can they be used to render a scene using ray tracing, but they can also be used in combination with scanning signal conversion techniques such as. B. in the context of the scanning signal conversion of geometric building blocks (ie polygon primitives such as triangles) of a 3D model to generate an image for display (e.g. on the in 1 shown display 150 ) are implemented. 14 10 illustrates an example flowchart for processing primitives to provide image pixel values of an image, according to one embodiment.

How 14 shows, an image of a 3D model can be generated in response to receiving a user input (step 1652 ). The user input can be a request to display an image or an image sequence, such as e.g. B. An input operation that is performed while interacting with an application (e.g., a game application). In response to user input, the system performs scan signal conversion and rasterization of geometric primitives of the 3D model of a scene using a conventional GPU 3D graphics pipeline (step 1654 ). For example, scanning signal conversion and rasterization of geometric primitives may include processing primitives of the 3D model to obtain image pixel values using conventional techniques such as B. lighting, transformations, texture mapping, screening and the like determine how they are known to those skilled in the art and hereinafter in connection with eighteen be discussed. The generated pixel data can be written into an image buffer.

In step 1656 one or more beams from one or more points on the rasterized primitives can be tracked using TTU hardware acceleration. The beams can be tracked according to one or more of the beam tracking capabilities disclosed in this application. Based on the results of the ray tracing, the pixel values stored in the buffer can be modified (step 1658 ). For example, modifying the pixel values may improve image quality in some applications, for example by using more realistic reflections and / or shadows. An image is displayed using the modified pixel values stored in the buffer (step 1660 ).

In one example, scan signal conversion and rasterization of geometric primitives can be done using the with respect to 15-18 . 19 . 20 . 21 and / or 22 processing system described, and beam tracking can be performed by the SMs 132 using the regarding 9 described TTU architecture can be implemented to add further visualization features (e.g. mirror reflection, shadows, etc.). 14 is just one non-limiting example - the SMs 132 could use the described TTU without texture processing or other conventional 3D graphics processing itself to generate images, or the SMs could use texture processing and other conventional 3D graphics processing without the described TTU to generate images. The SMs can also implement any desired imaging or other functionality in software, depending on the application, to provide any desired programmable functionality that is not tied to the hardware acceleration features provided by texture mapping hardware, tree traversing hardware, or other graphics pipeline hardware to be provided.

Example parallel processing architecture with beam tracing

The TTU structure described above can be implemented in or in connection with a non-limiting example parallel processing system architecture, as described below in relation to FIG 15-22 is described. Such a parallel processing architecture can be used, for example, to implement the GPU 130 of 1 be used.

For example parallel processing architecture

15 illustrates a non-limiting example parallel processing unit (PPU) 1700 , In one embodiment, the PPU 1700 a multi-thread processor implemented on one or more integrated circuits. The PPU 1700 is a latency hiding architecture that is designed to process many threads in parallel. A thread (ie, a thread of execution) is an instantiation of a set of instructions that are configured to be issued by the PPU 1700 to be executed. In one embodiment, the PPU 1700 configured to implement a graphics render pipeline for processing three-dimensional (3D) graphics data to generate two-dimensional (2D) image data for display on a display device such as B. a liquid crystal display device (LCD device), a device with organic light-emitting diodes (OLED), a device with transparent light-emitting diodes (TOLED), a field emission display (FED), a field sequence display, a projection display, one on Head mounted display or any other desired display. In other embodiments, the PPU 1700 be used to carry out universal calculations. Although an example parallel processor is provided herein for purposes of illustration, it should be understood that this processor is provided for illustrative purposes only and that any processor may be used to supplement and / or replace it.

For example, one or more PPUs 1700 Configured to accelerate thousands of high performance computing (HPC), data center and machine learning applications. The PPU 1700 can be configured to accelerate numerous deep learning systems and applications, including autonomous vehicle platforms, deep learning, high-precision speech, image and text recognition systems, intelligent video analytics, molecular simulations, drug discovery, disease diagnosis, weather forecast, big data Analytics, astronomy, molecular dynamics simulation, financial modeling, robotics, factory automation, real-time language translation, online search optimization and personalized user recommendations and the like.

The PPU 1700 can be in a desktop computer, a laptop computer, a tablet computer, servers, supercomputers, a smartphone (e.g. a wireless, portable device), a personal digital assistant (PDA), a digital camera, a vehicle, a attached to the head display, a portable electronic device and the like. In one embodiment the PPU 1700 formed on a single semiconductor substrate. In another embodiment, the PPU 1700 together with one or more other devices, such as. B. additional PPUs 1700 , the store 1704 , a CPU for reduced instruction set computers (RISC), a memory management unit (MMU), a digital-to-analog converter (DAC), and the like are contained in a SoC (system-on-a-chip, system on a chip).

In one embodiment, the PPU 1700 be included on a graphics card that has one or more storage devices 1704 contains. The graphics card can be configured to interface with a PCIe slot on a motherboard of a desktop computer. In another embodiment, the PPU 1700 an integrated graphics processing unit (iGPU) or a parallel processor, which is contained in the chipset of the motherboard.

As in 15 shown contains the PPU 1700 an input / output (I / O) unit 1705, a preprocessing unit 1715 , a scheduler unit 1720 , a work distribution unit 1725 , a hub 1730 , a switching matrix (XBar, crossbar) 1770 , one or more general processing clusters (GPCs) 1750 and one or more partition units 1780 , The PPU 1700 Can be connected to a host processor or other PPUs via one or more high-speed NVLink 1710 connections 1700 be connected. The PPU 1700 can over a connection 1702 connected to a host processor or other peripheral devices. The PPU 1700 can also be connected to local storage that has a number of storage devices 1704 includes. In one embodiment, the local memory may include a number of DRAM (Dynamic Random Access Memory) devices. The DRAM devices can be configured as a HBM (high bandwidth memory) subsystem, with multiple DRAM dies (wafer pieces) stacked in each device.

The NVLink 1710 connection enables systems to have one or more PPUs 1700 Scaling and integrating in combination with one or more CPUs supports cache coherence between the PPUs 1700 and CPUs as well as CPU mastering. Data and / or commands can be sent from the NVLink 1710 about the hub 1730 to / from other units of the PPU 1700 are transmitted such. B. one or more copy engines, a video encoder, a video decoder, a power management unit, etc. (not explicitly shown). The NVLink 1710 is used in conjunction with 21 described in more detail.

I / O unit 1705 is configured to operate over the connection 1702 Communication (ie commands, data, etc.) from a host processor (not shown) sends and receives. The I / O unit 1705 can connect directly to the host processor 1702 or via one or more intermediate devices such. B. communicate a memory bridge. In one embodiment, the I / O unit 1705 about the connection 1702 communicate with one or more other processors, e.g. B. one or more of the PPUs 1700 , In one embodiment, the I / O unit implements 1705 a PCIe (Peripheral Component Interconnect Express) interface for communication via a PCIe bus, and the connection 1702 is a PCIe bus. In alternative embodiments, the I / O unit 1705 implement other types of known interfaces for communication with external devices.

I / O unit 1705 decodes packets over the link 1702 were received. In one embodiment, the packets represent commands configured to do the PPU 1700 have various operations performed. The I / O unit 1705 transmits the decoded instructions to various other units of the PPU 1700 how the commands can specify. For example, some commands can be sent to the preprocessing unit 1715 be transmitted. Other commands can be sent to the hub 1730 or other units of the PPU 1700 are transmitted such. B. one or more copy engines, a video encoder, a video decoder, a power management unit, etc. (not explicitly shown). In other words, I / O unit 1705 is configured to communicate between and among the various logic units of the PPU 1700 forwards.

In one embodiment, a program executed by the host processor encodes an instruction stream in a buffer that the PPU 1700 Supplies work (workload) for processing. A workload can contain multiple statements, as well as data to be processed by these statements. The buffer is an area in memory that both the host processor and the PPU access 1700 access (ie read / write). For example, the I / O unit 1705 Be configured to meet storage requests over the connection 1702 are transferred, accesses the buffer in system memory associated with the connection 1702 connected is. In one embodiment, the host processor writes the instruction stream to the buffer and then sends a pointer to the beginning of the instruction stream to the PPU 1700 , The preprocessing unit 1715 receives pointers to one or more instruction streams. The preprocessing unit 1715 manages one or more streams, reads commands from the streams and forwards commands to the various units of the PPU 1700 further.

The preprocessing unit 1715 is with a scheduler unit 1720 coupled which the different GPCs 1750 configured to process tasks defined by the one or more streams. The scheduler unit 1720 is configured to track status information about the various tasks performed by the scheduler unit 1720 to get managed. The status can indicate which GPC 1750 a task is assigned, whether the task is active or inactive, a priority level assigned to the task, and so on. The scheduler unit 1720 manages the execution of a variety of tasks on the one or more GPCs 1750 ,

The scheduler unit 1720 is with a work distribution unit 1725 coupled, which is configured to perform tasks on the GPCs 1750 dispatches. The work distribution unit 1725 can track a number of scheduled tasks by the scheduler unit 1720 were received. In one embodiment, the work distribution unit manages 1725 for each of the GPCs 1750 a pool of upcoming tasks and a pool of active tasks. The pool of pending tasks can include a number of slots (e.g., 32 slots) that contain tasks performed by a particular GPC 1750 are to be processed. The pool of active tasks can include a number of slots (e.g., 4 slots) for tasks currently being performed by the GPCs 1750 be actively processed. If a GPC 1750 When a task finishes executing, that task is removed from the pool of active tasks for the GPC 1750 removed and one of the other tasks is selected from the pool of pending tasks and is to be executed on the GPC 1750 planned. If an active task on the GPC 1750 was idle, e.g. B. while waiting for resolution of a data dependency, the active task from the GPC 1750 removed and returned to the pool of pending tasks while another task is selected in the pool of pending tasks and for execution on the GPC 1750 is scheduled.

The work distribution unit 1725 communicates via the XBar 1770 with one or more GPCs 1750 , The XBar 1770 is a connection network that many units of the PPU 1700 with other units of the PPU 1700 coupled. For example, the XBar 1770 be configured to be the work distribution unit 1725 with a certain GPC 1750 coupled. Although not explicitly shown, one or more other units of the PPU 1700 also about the hub 1770 with the XBar 1770 be connected.

The tasks are handled by the scheduler unit 1720 managed and managed by the work distribution unit 1725 to a GPC 1750 handled. The GPC 1750 is configured to process the task and produce results. The results can be from other tasks within the GPC 1750 used up via the XBar 1770 to another GPC 1750 forwarded or in memory 1704 get saved. The results can be found on the partition units 1780 that have a memory interface for reading and writing data to and from memory 1704 implement in memory 1704 to be written. The results can be viewed via the NVLink 1710 to another PPU 1704 or CPU are transferred. In one embodiment, the PPU includes 1700 a number U of partition units 1780 that is the number of separate and separate storage devices 1704 corresponds to that with the PPU 1700 are coupled. A partition unit 1780 is used in conjunction with 16 described in more detail.

In one embodiment, a host processor (e.g., the processor 120 of 1 ) a driver kernel that implements an application programming interface (API) that enables one or more applications running on the host processor to perform operations on the PPU 1700 to plan. In one embodiment, multiple computing applications are run simultaneously by the PPU 1700 executed, and the PPU 1700 offers isolation, quality of service (QoS) and independent address spaces for the multiple computing applications. An application can generate instructions (ie, API calls) that cause the driver kernel to perform one or more tasks for execution by the PPU 1700 to generate. The driver kernel issues tasks to one or more streams that are generated by the PPU 1700 to be edited. Each task can include one or more groups of related threads, referred to herein as a warp. In one embodiment, a warp includes 32 related threads that can run in parallel. Cooperating threads can refer to a variety of threads, including instructions on how to perform the task and that data can be shared across shared storage. Threads and cooperating threads are associated with 19 described in more detail.

Example memory partition unit

The MMU 1890 provides an interface between the GPC 1750 and the partition unit 1780 ready. The MMU 1890 can enable translation of virtual addresses into physical addresses, memory protection and arbitration of memory requests. In one embodiment, the MMU provides 1890 one or more translation buffers (TLBs, Translation Lookaside Buffers) ready to translate virtual addresses into physical addresses in memory 1704 perform.

16 illustrates a storage partition unit 1780 the PPU 1700 of 15 according to one embodiment. As in 16 shown contains the storage partition unit 1780 a raster operation (ROP) unit 1850 , a level 2 cache (L2 cache) 1860 and a memory interface 1870 , The storage interface 1870 is with memory 1704 coupled. The storage interface 1870 can implement 32, 64, 128, 1024 bit data buses or the like for high speed data transmission. In one embodiment, the PPU 1700 U memory interfaces 1870 , a storage interface 1870 per pair of partition units 1780 , where each pair of partition units 1780 with an appropriate storage device 1704 connected is. For example, the PPU 1700 with up to Y storage devices 1704 be connected, such as B. High bandwidth memory stack or dual data rate graphics, version 5, synchronous dynamic random access memory, or other types of persistent memory.

In one embodiment, the memory interface implements 1870 an HBM2 memory interface and corresponds to U half a U. In one embodiment, the HBM2 memory stacks are on the same physical chassis as the PPU 1700 , which leads to significant performance and space savings compared to conventional GDDR5-SDRAM systems. In one embodiment, each HBM2 stack contains and is four memory dies Y equal 4 , with the HBM2 stack two 128-bit channels per die for a total of 8 channels and a data bus width of 1024 Bits.

In one embodiment, the memory supports 1704 Single error correcting and double error detecting (SECDED) error correction code (ECC) to protect data. ECC offers greater reliability for computing applications that are sensitive to data corruption. Reliability is particularly important in large cluster computing environments where PPUs are used 1700 process very large data sets and / or applications run over longer periods of time.

In one embodiment, the PPU implements 1700 a multi-level storage hierarchy. In one embodiment, the storage partition unit supports 1780 a unified memory to a single unified virtual address space for memory of the CPU and PPU 1700 to provide, which enables the sharing of data between virtual storage systems. In one embodiment, the frequency of accesses by a PPU 1700 traces memory that resides on other processors to ensure that pages of memory are in the physical memory of that PPU 1700 that access the pages more often. In one embodiment, the NVLink supports 1710 Address translation services that it the PPU 1700 enable direct access to page tables of a CPU and full access to CPU memory by the PPU 1700 enable.

In one embodiment, copy engines transfer data between multiple PPUs 1700 or between PPUs 1700 and CPUs. The copy engines can generate page faults for addresses that are not entered in the page tables. The storage partition unit 1780 can then serve the page faults and enter the addresses in the page table, whereupon the copy engine can carry out the transfer. In a conventional system, memory for multiple copy engine operations is pinned (ie, non-pageable) between multiple processors, which significantly reduces available memory. In the event of hardware page failure, addresses can be passed to the copy engines without worrying about whether the memory pages are resident and the copy operation is transparent.

Data from memory 1704 or other system memory can be from the storage partition unit 1780 accessed and in the L2 cache 1860 , the on-chip (on the chip) is located and between the different GPCs 1750 shared, saved. As shown, each storage partition unit contains 1780 part of the L2 cache 1860 of a corresponding storage device 1704 assigned. Subordinate caches can then be in different units within the GPCs 1750 be implemented. For example, each of the SMs 1840 implement a level one cache (L1 cache). The L1 cache is private memory that a specific SM 1840 is permanently assigned. Data from the L2 cache 1860 can be accessed and in each of the L1 caches for processing in the functional units of the SMs 1840 get saved. The L2 cache 1860 is with the storage interface 1870 and the XBar 1770 coupled.

The ROP unit 1850 performs graphic raster operations related to pixel color, such as B. color compression, pixel blending and the like. The ROP unit 1850 also performs depth testing in conjunction with the raster engine 1825 through and receives a depth for a scan position associated with a pixel fragment from the raster engine culling engine 1825 , The depth is tested against a corresponding depth in a depth buffer for a scan position assigned to the fragment. If the fragment passes the depth test for the scan position, the ROP updates 1850 the depth buffer and sends a result of the depth test to the raster engine 1825 , Note that the number of partition units 1780 on the number of GPCs 1750 can deviate and hence every ROP unit 1850 with each of the GPCs 1750 can be coupled. The ROP unit 1850 keeps track of those from the various GPCs 1750 received packets and determines which GPC 1750 one from the ROP unit 1850 result generated by the XBar 1770 is directed. Although the ROP unit 1850 in 16 within the storage partition unit 1780 is included, the ROP unit 1850 in another embodiment outside the storage partition unit 1780 are located. For example, the ROP unit 1850 in the GPC 1750 or another unit.

Sample Allgemeinverarbeitungsduster

17 illustrates a GPC 1750 the PPU 1700 of 15 according to one embodiment. As in 17 shown, each contains GPC 1750 a number of hardware units for processing tasks. In one embodiment, each contains GPC 1750 a pipeline manager 1810 , a pre-raster operation unit (PROP) 1815 , a raster engine 1825 , a work distribution switching matrix (WDX) 1880 , a memory management unit (MMU) 1890 and one or more data processing clusters (DPCs) 1820 , Note that the GPC 1750 of 17 instead of or in addition to the in 17 units shown may contain other hardware units.

In one embodiment, the operation of the GPC 1750 through the pipeline manager 1810 controlled. The pipeline manager 1810 manages the configuration of one or more DPCs 1820 for processing tasks that the GPC 1750 are assigned. In one embodiment, the pipeline manager 1810 at least one of the one or more DPCs 1820 Configure to implement at least part of a graphics rendering pipeline.

Everyone in the GPC 1750 DPC included 1820 contains an M-Pipe-Controller (MPC) 1830 , a primitive engine 1835 , one or more SMs 1840 , one or more texture units 1842 and one or more TTUs 700 , The SM 1840 can be similar to the SM described above 132 be structured. The MPC 1830 controls the operation of the DPC 1820 and directs that from the pipeline manager 1810 received packets to the corresponding units in the DPC 1820 further. For example, packages associated with a vertex can be sent to the primitive engine 1835 that are configured to take vertex attributes associated with the vertex from memory 1704 get. In contrast, packets that are assigned to a shader program can be sent to the SM 1840 be transmitted.

When configured for universal parallel calculation, a simpler configuration can be used compared to graphics processing. In particular, the in 15 bypassed shown graphics processing units with a fixed function, which creates a much simpler programming model. In the universal parallel calculation configuration, the work distribution unit points 1725 Thread blocks directly to the DPCs 1820 and distribute them on it. The threads in a block run the same program, using a unique thread ID in the calculation to ensure that each thread produces unique results, with the SM 1840 The shared memory / L1 cache is used to run the program and perform calculations 1970 to communicate between threads and the LSU 1954 to go through the shared memory / L 1 cache 1970 and the storage partition unit 1780 read and write global memory. When configured for universal parallel calculation, the SM 1840 also write commands with which the scheduler unit 1720 new work on the DPCs 1820 can start. The TTU 700 can be used to speed up spatial queries in the context of universal parallel calculation.

Graphics rendering pipeline

A DPC 1820 can be configured to run a vertex shader program on the programmable streaming multiprocessor (SM) 1840 executes what certain shading operations with the TTU 700 can accelerate. The pipeline manager 1810 can also be configured to be from the work distribution unit 1725 packets received to the corresponding logic units within the GPC 1750 forwards. For example, some packages of hardware units with a fixed function in the PROP 1815 and / or the raster engine 1825 are forwarded while other packets are sent to the DPCs 1820 for processing by the primitive engine 1835 or the SM 1840 can be forwarded. In one embodiment, the pipeline manager 1810 at least one of the one or more DPCs 1820 Configure to implement a neural network model and / or compute pipeline.

The PROP unit 1815 is configured to be by the raster engine 1825 and the DPCs 1820 forwards generated data to a raster operation (ROP) unit in connection with 16 is described in more detail. The PROP unit 1815 can also be configured to perform color mixing optimizations, organize pixel data, perform address translations, and the like.

The raster engine 1825 contains a number of fixed-function hardware units that are configured to perform various raster operations. In one embodiment, the raster engine includes 1825 a setup engine, a coarse screen engine, a culling engine, a clipping engine, a fine screen engine and a tile coalescing engine. The setup engine receives transformed vertices and generates plane equations associated with the geometric primitive defined by the vertices. The level equations are transferred to the coarse grid engine to generate coverage information (e.g., an x, y coverage mask for a tile) for the primitive. The output of the coarse screen engine is transferred to the culling engine, where fragments associated with the primitive that a z-test does not pass are subjected to culling (reading) and transferred to a clipping engine, where fragments, that are outside of a stump are subjected to clipping. Those fragments that survive clipping and culling can be passed to the fine grid engine to generate attributes for the pixel fragments based on the level equations generated by the setup engine. The output of the raster engine 1825 includes fragments that are to be processed, for example, by a fragment shader operating within a DPC 1820 is implemented.

Specifically, the PPU 1700 configured to receive commands that specify shader programs for processing graphic data. Graphic data can be defined as a set of primitives, such as: B. points, lines, triangles, quadrilaterals, triangular strips and the like. Typically, a primitive contains data that specifies a number of vertices or vertices for the primitive (e.g. in a model space coordinate system), as well as attributes that are assigned to each vertex of the primitive. The PPU 1700 can be configured to process the graphics primitives to generate an image buffer (ie, pixel data for each pixel of the display), for example using the TTU 700 as a hardware acceleration resource.

An application writes model data for a scene (ie, a collection of vertices and attributes) into a memory, such as. B. a system memory or memory 1704 , The model data define each of the objects that can be visible on a display. The model data can also define one or more BVHs as described above. The application then makes an API call to the driver kernel, which requests that the model data be rendered and displayed. The driver kernel reads the model data and writes commands to the one or more streams to perform operations to process the model data. The commands can refer to various shader programs that are on the SMs 1840 the PPU 1700 are to be implemented, including one or more vertex shaders, hull shaders, domain shaders, geometry shaders, pixel shaders, beam generation shaders, beam intersection shaders, beam hit shaders and beam miss shaders (these correspond to those by DirectX Raytracing (DXR) API defined shaders and ignore any distinction between "closest hit" and "any hit"shaders; see https://devblogs.nvidia.com/introduction-nvidia-rtx-directx-raytracing/). For example, one or more SMs 1840 be configured to run a vertex shader program that processes a number of vertices defined by the model data. In one embodiment, the different SMs 1840 configured to run different shader programs at the same time. For example, a first subset of SMs 1840 configured to run a vertex shader program while a second subset of SMs 1840 can be configured to run a pixel shader program. The first subset of SMs 1840 processes vertex data into processed vertex data and writes the processed vertex data into the L2 cache 1860 and / or the memory 1704 (please refer 16 ). After the processed vertex data has been rasterized (ie transformed from three-dimensional data into two-dimensional data in the screen space) to generate fragment data, the second subset of SMs 1840 a pixel shader to generate processed fragment data, which is then mixed with other processed fragment data and placed in the frame buffer in memory 1704 to be written. The vertex shader program and the pixel shader program can be executed simultaneously, with different data of the same scene being processed according to a pipeline procedure until all model data for the scene have been rendered in the image buffer. The content of the image buffer is then transferred to a display controller for display on a display device.

eighteen is a conceptual diagram of one using the PPU 1700 of 15 implemented graphics processing pipeline 2000 , The graphics processing pipeline 2000 is an abstract flowchart of the processing steps implemented to generate 2D computer images from 3D geometry data. As is known, pipeline architectures can perform long latency operations more efficiently by dividing the operation into a plurality of stages, with the output of each stage coupled to the input of the next stage. Thus, the graphics processing pipeline receives 2000 input data 2001 by one level to the next level of the graphics processing pipeline 2000 transferred to output data 2002 to create. In one embodiment, the graphics processing pipeline 2000 represent a graphics processing pipeline defined by the OpenGL® API. Optionally, the graphics processing pipeline 2000 implemented in the context of the functionality and architecture of the previous figures and / or any subsequent figure (s). As above with reference to 14 discussed, ray tracing can be used to improve the image quality generated by rasterizing geometric primitives. In some embodiments, the ray tracing operations and TTU structures disclosed in this application can point to one or more states of the graphics processing pipeline 2000 applied to improve subjective image quality.

As in eighteen shown, the graphics processing pipeline 2000 a pipeline architecture comprising a number of stages. The levels include, but are not limited to, a data build level 2010 , a vertex shading level 2020 , a primitive advanced level 2030 , a geometry shading level 2040 , a Viewport scaling, culling and clipping (VSCC) level 550 , a grid level 2060 , a fragment shading level 2070 and a raster operations stage 2080 , The input data 2001 In one embodiment, include instructions that configure the processing units to match the stages of the graphics processing pipeline 2000 and geometric primitives (e.g. dots, lines, triangles, quadrilaterals, triangular stripes or fans, etc.) to be processed by the levels. The output data 2002 may include pixel data (ie, color data) copied to an image buffer or other type of surface data structure in memory.

The data development level 2010 receives the input data 2001 that specify vertex data for higher order surfaces, primitives, or the like. The data development level 2010 collects the vertex data in temporary memory or a queue, such as By receiving a command from the host processor that contains a pointer to a buffer in memory and reading the vertex data from the buffer. The vertex data is then sent to the vertex shading stage for processing 2020 transfer.

The vertex shading level 2020 processes vertex data by performing a series of operations (ie, a vertex shader or program) once for each of the vertices. Vertices can e.g. B. can be specified as a 4-coordinate vector (ie <x, y, z, w>) which is assigned to one or more vertex attributes (e.g. color, texture coordinates, surface normal, etc.). The vertex shading level 2020 can separate vertex attributes such as B. handle position, color, texture coordinates and the like. In other words, the vertex shading stage leads 2020 Operations on the vertex coordinates or other vertex attributes associated with a vertex. Such operations usually include lighting operations (ie, modifying color attributes for a vertex) and transformation operations (ie, modifying the coordinate space for a vertex). For example, vertices can be specified with the help of coordinates in an object coordinate space, which are transformed by multiplying the coordinates with a matrix, which converts the coordinates from the object coordinate space into a world space or an NCD space (space with standardized device coordinates ) translated. The vertex shading level 2020 generates transformed vertex data that is at the primitive level 2030 be transmitted.

The primitive advanced level 2030 collects vertices by the vertex shading level 2020 and groups the vertices into geometric primitives for processing by the geometry shading level 2040 , For example, the primitive advanced level 2030 be configured to always have three consecutive vertices as a geometric primitive (ie a triangle) for transmission to the geometry shading stage 2040 grouped. In some embodiments, certain vertices can be reused for successive geometric primitives (e.g., two successive triangles in a triangle strip can share two vertices). The primitive advanced level 2030 transfers geometric primitives (ie a collection of related vertices) to the geometry shading stage 2040 ,

The geometry shading level 2040 processes geometric primitives by performing a series of operations (that is, a geometry shader or program) on the geometric primitives. Tessellation operations can generate one or more geometric primitives from each geometric primitive. In other words, the geometry shading level 2040 each geometric primitive into a finer mesh of two or more geometric primitives for processing by the rest of the graphics processing pipeline 2000 divide. The geometry shading level 2040 transfers geometric primitives to the Viewport SCC level 550 ,

In one embodiment, the graphics processing pipeline 2000 work within a streaming multiprocessor, and the vertex shading stage 2020 who have favourited Primitive Advanced Level 2030 who have favourited Geometry Shading Level 2040 who have favourited Fragment Shading Level 2070 and / or associated hardware or software can perform processing operations sequentially. In one embodiment, once the sequential processing operations are completed, the Viewport SCC stage 550 use the data. In one embodiment, one or more stages of the graphics processing pipeline 2000 processed primitive data are written to a cache (e.g. L1 cache, a vertex cache, etc.). In this case, the Viewport SCC level 550 access the data in the cache in one embodiment. In one embodiment, the Viewport are SCC level 550 and the screening level 2060 implemented as circuits with a fixed function.

The Viewport SCC level 550 performs the scaling, culling and clipping of the geometric primitives. An abstract camera position is assigned to each surface that is rendered. The camera position represents the location of a viewer looking at the scene and defines a stump of observation that encloses the objects of the scene. The viewing stump can include a viewing level, a back level, and four clipping levels. Any geometric primitive that is completely outside of the stump can be culled (ie discarded) because the geometric primitive does not contribute to the final rendered scene. Any geometric primitive that is partially inside the viewing stump and partially outside the viewing stump can be cut off (ie, converted to a new geometric primitive that is enclosed within the viewing stump). In addition, geometric primitives can be scaled based on the depth of the observation stump. All potentially visible geometric primitives are then added to the screening level 2060 transfer.

The screening level 2060 converts the geometric 3D primitives into 2D fragments (e.g. usable for display etc.). The screening level 2060 can be configured to use the vertices of the geometric primitives to create a set of plane equations from which various attributes can be interpolated. The screening level 2060 can also compute a cover mask for a plurality of pixels, indicating whether one or more sample positions for the pixel cut off the geometric primitive. In one embodiment, Z testing can also be performed to determine whether the geometric primitive is hidden by other geometric primitives that have already been rasterized. The screening level 2060 generates fragment data (ie, interpolated vertex attributes associated with a particular scan position for each covered pixel) that is at the fragment shading level 2070 be transmitted.

The fragment shading level 2070 processes fragment data by performing a series of operations (e.g., a fragment shader or program) on each of the fragments. The fragment shading level 2070 can generate pixel data (ie, color values) for the fragment, e.g. B. by performing lighting operations or scanning texture maps using interpolated texture coordinates for the fragment. The fragment shading level 2070 generates pixel data related to the raster operations stage 2080 be transmitted.

The raster operations level 2080 can perform various operations on the pixel data, such as B. Alpha tests, template tests and mixing the pixel data with other pixel data corresponding to other fragments associated with the pixel. If the raster operations stage 2080 the processing of the pixel data (ie the output data 2002 ) has completed, the pixel data can be transferred to a rendering target such as B. an image buffer, a color buffer or the like.

Note that one or more additional stages are in addition to or instead of one or more of the stages described above in the graphics processing pipeline 2000 can be included. Different implementations of the abstract graphics processing pipeline can implement different stages. In addition, in some embodiments, one or more of the levels described above (such as the geometry shading level 2040 ) be excluded from the graphics processing pipeline. Other types of graphics processing pipelines are considered to be within the scope of the present disclosure. In addition, each of the stages of the graphics processing pipeline 2000 of one or more dedicated hardware units within a graphics processor such as B. the PPU 200 be implemented. Other stages of the graphics processing pipeline 2000 can by programmable hardware units such. B. the SM 1840 the PPU 1700 be implemented.

The graphics processing pipeline 2000 can be implemented via an application that is run by a host processor, such as B. a CPU 120 , In one embodiment, a device driver can implement an application programming interface (API) that defines various functions that can be used by an application to generate graphical data for display. The device driver is a software program that contains a variety of instructions that guide the operation of the PPU 1700 Taxes. The API offers an abstraction to a programmer that enables a programmer to use specialized graphics hardware such as B. the PPU 1700 to use to generate the graphical data without the programmer having the specific instruction set for the PPU 1700 must use. The application can include an API call to the device driver for the PPU 1700 is forwarded. The device driver interprets the API call and performs various operations to respond to the API call. In some cases, the device driver can perform operations by executing instructions on the CPU. In other cases, the device driver can at least partially perform operations by performing operations on the PPU 1700 via an input / output interface between the CPU and the PPU 1700 starts. In one embodiment, the device driver is configured to be the graphics processing pipeline 2000 using the hardware of the PPU 1700 implemented.

Within the PPU 1700 Different programs can run to the different stages of the graphics processing pipeline 2000 to implement. For example, the device driver can have a kernel on the PPU 1700 start to the vertex shading level 2020 on an SM 1840 (or several SMs 1840 ) to carry out. The device driver (or that of the PPU 1800 running initial kernel) may also be other kernels on the PPU 1800 start to other stages of the graphics processing pipeline 2000 execute such. B. the geometry shading level 2040 and the fragment shading level 2070 , It can also use some of the stages of the graphics processing pipeline 2000 be implemented on fixed unit hardware such as B. a rasterizer or a data assembler in the PPU 1700 is implemented. Note that results from a kernel can be processed by one or more intermediate fixed-function hardware units before being processed by a subsequent kernel on an SM 1840 are processed.

Example streaming multiprocessor

The SM 1840 includes a programmable streaming processor that is configured to process tasks represented by a number of threads. Every SM 1840 is multi-threaded and configured so that it can execute a large number of threads (e.g. 32 threads) from a specific group of threads at the same time. In one embodiment, the SM implements 1840 a SIMD (single instruction, multiple data) architecture in which each thread in a group of threads (ie, a warp) is configured to process a different record based on the same instruction set. All threads in the thread group follow the same instructions. In another embodiment, the SM implements 1840 a SIMT (single-instruction, multiple-thread) architecture in which each thread in a group of threads is configured to process a different record based on the same instruction set, but with individual threads in the group of threads during the Execution may differ from each other. In one embodiment, a program counter, a call stack and an execution status are maintained for each warp, which enables simultaneity between warps and serial execution within warps if threads within the warps differ from one another. In a further embodiment, a program counter, a call stack and an execution status are maintained for each individual thread, which enables the same simultaneity between all threads, within and between warps. If execution status is maintained for each thread, threads that execute the same instructions can be converged and executed in parallel for maximum efficiency.

19 illustrates the streaming multiprocessor 1840 of 17 according to one embodiment. As in 19 shown, the SM contains 1840 an instruction cache 1905 , one or more scheduler units 1910 , a register file 1920 , one or more processing computing cores 1950 , one or more special functional units (SFUs) 1952 , one or more load / store units (LSUs) 1954 , a connection network 1980 and a shared memory / L1 cache 1970 ,

As described above, the work distribution unit manufactures 1725 Tasks for execution on the GPCs 1750 the PPU 1700 from. The tasks are a specific DPC 1820 within a GPC 1750 assigned, and if the task is assigned to a shader program, the task can be assigned to an SM 1840 be assigned to. The scheduler unit 1910 receives the tasks from the work distribution unit 1725 and manages instruction scheduling for one or more thread blocks that the SM 1840 assigned. The scheduler unit 1910 schedules thread blocks for execution as warps of parallel threads, with at least one warp assigned to each thread block. In one embodiment, each warp 32 Threads out. The scheduler unit 1910 can manage a variety of different thread blocks that assign warps to the different thread blocks and then, during each clock cycle, direct instructions from the plurality of different cooperative groups to the different functional units (ie, cores) 1950 , SFUs 1952 and LSUs 1954 ) dispatch.

Cooperative Groups is a programming model for organizing groups of communicating threads that allows developers to express the granularity with which threads communicate, which enables richer, more efficient parallel decompositions to be expressed. Cooperative start APIs support synchronization between thread blocks for the execution of parallel algorithms. Conventional programming models offer a single, simple construct for synchronizing cooperating threads: a barrier across all threads of a thread block (ie the function syncthreads ()). However, programmers often want to define groups of threads with smaller granularity than thread blocks and synchronize them within the defined groups to provide more performance, design flexibility and software reuse in the form of common group-wide functional interfaces.

Cooperative groups allow programmers to explicitly define groups of threads on sub-block (i.e., as small as a single thread) and multiblock granularities and perform collective operations such as B. Synchronize the threads in a cooperative group. The programming model supports clean composition across software boundaries, so that libraries and utility functions can synchronize safely in their local context without having to make assumptions about convergence. Cooperative groups primitives enable new patterns of cooperative parallelism, including producer-consumer parallelism, opportunistic parallelism, and global synchronization across an entire network of thread blocks.

A handling unit 1915 is configured so that it can send instructions to one or more of the functional units. In the embodiment, the scheduler unit contains 1910 two handling units 1915 that allow two different instructions to be sent from the same warp during each clock cycle. In alternative embodiments, each scheduler unit can 1910 a single handling unit 1915 or additional handling units 1915 contain.

Every SM 1840 contains a register file 1920 that have a set of registers for the functional units of the SM 1840 provides. In one embodiment, the register file is 1920 divided between the individual functional units so that each functional unit has a specific part of the register file 1920 is permanently assigned. In another embodiment, the register file is 1920 split between the different warps created by the SM 1840 be carried out. The register file 1920 provides temporary storage for operands that are connected to the data paths of the functional units. 21 illustrates an example configuration of the register files of the SM 1840 ,

Every SM 1840 includes L processing cores 1950 , In one embodiment, the SM includes 1840 a large number (e.g. 128, etc.) of separate processing cores 1950 , Every calculation core 1950 may include a full pipeline, single precision, double precision, and / or mixed precision processing unit that includes a floating point arithmetic logic unit and an integer arithmetic logic unit. In one embodiment, the floating point arithmetic logic units implement the IEEE 754-2008 standard for floating point arithmetic. In one embodiment, the computing cores include 1950 64 single-precision (32-bit) floating point cores, 64 integer computing cores, 32 double-precision (64-bit) floating point computing cores and 8 tensor computing cores.

Tensor cores are configured to perform matrix operations, and in one embodiment, one or more tensor cores are in the cores 1950 contain. In particular, the tensor cores are configured so that they can perform deep leaming matrix arithmetic, such as. B. Convolutional operations for training and inferencing neural networks. In one embodiment, each tensor core operates on a 4x4 matrix and performs a matrix multiplication and accumulation operation D = A × B + C, where A, B, C and D are 4x4 matrices.

In one embodiment, the matrix multiplication inputs A and B are 16-bit floating point matrices, while the accumulation matrices C and D can be 16-bit floating point or 32-bit floating point matrices. Tensor cores work on 16-bit floating point input data with 32-bit floating point accumulation. The 16-bit floating point multiplication requires 64 operations and results in a high-precision product that is then accumulated using 32-bit floating point addition with the other intermediates for 4x4x4 matrix multiplication.

In practice, tensor cores are used to perform much larger two-dimensional or higher-dimensional matrix operations composed of these smaller elements. An API, such as B. CUDA 9 C ++ API, provides specialized matrix load, matrix multiplication and accumulation as well as matrix storage operations to efficiently use tensor cores from a CUDA C ++ program. At the CUDA level, the interface at warp level requires 16x16 size dies, all of them 32 Spanning warp threads.

Every SM 1840 also contains M SFUs 1952 that perform special functions (e.g. attribute evaluation, reciprocal square root and the like). In one embodiment, the SFUs 1952 include a tree traversing unit configured to traverse a hierarchical tree data structure. In one embodiment, the SFUs 1952 include a texture unit configured to perform texture mapping filtering operations. In one embodiment, the texture units are configured to take texture maps (e.g., a 2D array of texels) from memory 1704 load and sample the texture maps to generate sampled texture values for use in shader programs that the SM 1840 be carried out. In one embodiment, the texture maps are in shared memory / L1 cache 1970 saved. The texture units implement texture operations such as B. Filter operations using mip maps (ie texture maps with different levels of detail). In one embodiment, each contains SM 340 two texture units.

Every SM 1840 also includes N LSUs 1954, the load and store operations between the shared memory / L 1 cache 1970 and the register file 1920 to implement. Every SM 1840 contains a connection network 1980 that each of the functional units with the register file 1920 and the LSU 1954 with the register file 1920 , Shared memory / L1 cache 1970 combines. In one embodiment, the connection network is 1980 a switching matrix that can be configured to have any of the functional units with any of the registers in the register file 1920 connects and the LSUs 1954 with the register file and storage locations in the shared memory / L1 cache 1970 combines.

The shared memory / L1 cache 1970 is an array of on-chip memory that stores data and communication between the SM 1840 and the primitive engine 1835 as well as between threads in SM 1840 allows. In one embodiment, the shared memory comprises / L1 cache 1970 128KB storage capacity and is in the path of the SM 1840 to the partition unit 1780 , With the shared memory / L1 cache 1970 read and write accesses can be buffered. One or more of the shared memory / L1 cache 1970 , L2 cache 1860 and memory 1704 are backup storage.

The combination of data cache and shared memory functionality in a single block of memory provides the best overall performance for both types of memory access. The capacity is usable as a cache of programs that do not use shared memory. For example, if shared storage is configured to use half of the capacity, texture and load / store operations can use the remaining capacity. Integration in the shared memory / L 1 cache 1970 enables shared memory / L 1 cache 1970 to act as a high-throughput line for streaming data while providing high bandwidth and low latency access to frequently reused data.

21 illustrates an example architecture for the SM 1840 , As in 19 shown, the SM 1840 with one or more texture units 1842 and / or one or more TTUs 700 be coupled. As a compromise between performance and area, a non-limiting embodiment can have a single texture unit 1842 and / or a single TTU 700 per group of SMs 1840 included (e.g. see 19 ). The TTU 700 can with the SMs 1840 communicate via a TTU input / output block in the memory input / output and with an L1 cache via a dedicated read interface. In one embodiment, the TTU reads 700 only from the main memory and does not write to the main memory.

More detailed example TTU architecture

As discussed above, the TTU 700 a coprocessor to the SM 1840 his. Like a texture processor, it is accessed through a set of SM instructions, accesses memory as a read-only client of the L1 cache, and returns results to the SM register file. Unlike some texture processors, it makes the amount of data necessary for a typical query in the TTU 700 must be swapped in and out, difficult in some embodiments to specify all source and destination registers in a single instruction (and since most of this data is per-thread unique, there is no TTU analog of texture heads and scanners). As a result, the TTU 700 programmed via a multi-instruction sequence in some embodiments. In some implementations, this sequence can be designed as a single “macro instruction”.

• - Ein oder mehrere BVHs, wobei jede BVH zum Beispiel ein Baum von achsenausgerichteten Begrenzungskästen ist, die in einem komprimierten Format gespeichert sind, das den Speicherverkehr im Vergleich zu einer unkomprimierten Darstellung stark reduziert. Jeder Knoten in der BVH wird als eine Complet-Struktur gespeichert, wobei Größe und Ausrichtung in manchen Implementierungen an die einer L1-Cache-Zeile angepasst sind. Kind-Complets eines gegebenen Elternteils werden vorzugsweise zusammenhängend in Speicher gespeichert, und Kind-Zeiger werden in komprimierter Form gespeichert.
• - Null oder mehr Instanzknoten, die eine Möglichkeit bieten, ein Blatt einer BVH mit der Wurzel einer anderen zu verbinden. Ein Instanzknoten kann eine Datenstruktur sein, die ebenfalls ausgerichtet ist. Diese Struktur kann einen Zeiger auf die Sub-BVH, Flags, die das Backface-Culling-Verhalten in der Sub-BVH beeinflussen, und eine Matrix enthalten, die den ersten drei Zeilen einer beliebigen Transformationsmatrix (in homogenen Koordinaten) von dem Koordinatensystem der Top-Level-BVH (gewöhnlich „Welt-Raum“) zu dem der Sub-BVH (gewöhnlich „Objekt-Raum“) entspricht. Die letzte Zeile der Matrix in manchen Ausführungsformen ist in manchen Implementierungen implizit (0, 0, 0, 1).
• - Null oder mehr Dreieck- oder andere Primitiv-Puffer, die zum Beispiel Dreiecke enthalten, die entweder als ein Triplett von Koordinaten pro Vertex oder in einem verlustfrei komprimierten Format gespeichert sind, das von der TTU 700 verstanden wird. Zusätzlich kann pro Dreieck oder anderem Primitiv ein Alpha-Bit bereitgestellt werden, das Dreiecke anzeigt, die besondere Behandlung durch die Software erfordern, um zu bestimmen, ob das Dreieck tatsächlich von einem gegebenen Strahl geschnitten wird. Dreieck-Puffer können in Blöcken organisiert sein. Es kann auch ein Erzwinge-kein-Culling-Funktionsbit pro Dreieck geben. Wenn dieses Bit gesetzt ist, zeigt es an, dass beide Seiten des Dreiecks in Bezug auf Culling als nach vorne oder nach hinten weisend zu behandeln sind, d. h. das Dreieck ist nicht auszusondern, weil der Strahl die „Rückseite“ statt der „Vorderseite“ schneidet. Der einfachste Anwendungsfall dafür ist ein einzelnes Dreieck, das ein Blatt darstellt, wobei wir das Blatt noch sehen können, wenn der Strahl es auf der Rückseite trifft.

And just like texture units 1842 can the TTU 700 in some implementations rely on certain read-only data structures in memory that are prefilled with software. These include:

• One or more BVHs, each BVH being, for example, a tree of axis-oriented bounding boxes that are stored in a compressed format that greatly reduces storage traffic compared to an uncompressed representation. Each node in the BVH is saved as a complete structure, with size and orientation in some Implementations that are adapted to an L1 cache line. Kind completes of a given parent are preferably stored contiguously in memory, and child pointers are stored in compressed form.
• - Zero or more instance nodes that offer a possibility to connect one leaf of a BVH to the root of another. An instance node can be a data structure that is also aligned. This structure can include a pointer to the sub-BVH, flags that affect the back-face culling behavior in the sub-BVH, and a matrix that represents the first three rows of any transformation matrix (in homogeneous coordinates) from the coordinate system of the top -Level BVH (usually "world space") to which the sub-BVH (usually "object space") corresponds. The last line of the matrix in some embodiments is implicit (0, 0, 0, 1) in some implementations.
• Zero or more triangle or other primitive buffers containing, for example, triangles stored either as a triplet of coordinates per vertex or in a losslessly compressed format developed by the TTU 700 is understood. In addition, an alpha bit can be provided per triangle or other primitive indicating triangles that require special software treatment to determine whether the triangle is actually intersected by a given ray. Triangle buffers can be organized in blocks. There can also be one force-no-culling function bit per triangle. When set, this bit indicates that both sides of the triangle should be treated as facing forward or backward in terms of culling, ie the triangle should not be discarded because the beam intersects the "back" instead of the "front" , The simplest use case for this is a single triangle that represents a leaf, and we can still see the leaf when the beam hits it on the back.

The TTU 700 is stateless in some embodiments, which means that no architectural status is maintained in the TTU between queries. At the same time, it often makes sense for software running on the SM 1840 running, requesting continuation of an earlier query, meaning the relevant status from the TTU 700 to be written in registers and then returned to the TTU in registers (often on site) to continue. This status can take the form of a traversal stack that tracks the progress in traversing the BVH.

• • Traversierung startend von einem Complet
• • Schnittpunkt eines Strahls mit einem Bereich von Dreiecken
• • Schnittpunkt eines Strahls mit einem Bereich von Dreiecken, gefolgt von einer Traversierung, die von einem Complet startet
• • Vertex-Abruf aus einem Dreieck-Puffer für ein gegebenes Dreieck
• • Optionale Unterstützung für Instanztransformationen vor dem „Traversierung startend von einem Complet“ und „Schnittpunkt eines Strahls mit einem Bereich von Dreiecken“.

A small number of batch initializers can also be provided to start a new query of a given type, for example:

• • Traversing starting from a complete
• • Intersection of a ray with a range of triangles
• • Intersection of a ray with a range of triangles, followed by traversing that starts from a complete
• • Vertex retrieval from a triangle buffer for a given triangle
• • Optional support for instance transformations before "traversing starting from a complete" and "intersection of a ray with a range of triangles".

Vertex retrieval is a simple query that can be specified with request data that consists of a batch initializer and nothing else. Other types of queries may require specification of a ray or bundle of rays, along with the stack or batch initializer and various ray flags that describe details of the query. A ray is given by its three-coordinate origin, the three-coordinate direction and minimum and maximum values for the t parameter along the ray. A beam of rays is additionally given by a second origin and a second direction.

• ■ Ein Kind-Complet (d. h. ein interner Knoten)

Standardmäßig setzt die TTU 700 die Traversierung fort, indem sie in die Kind-Complets absteigt.

• ■ Ein Dreieck-Bereich, der einem zusammenhängenden Satz von Dreiecken innerhalb eines Dreieck-Puffers entspricht.
  1. (1) Standardmäßig werden Dreieck-Bereiche, auf die ein Strahl trifft, von der TTU 700 nativ behandelt, indem die Dreiecke auf Schnittpunkte getestet und der Strahl entsprechend verkürzt wird. Wenn die Traversierung abgeschlossen ist und ein Dreieck getroffen worden ist, wird standardmäßig die Dreieck-ID zusammen mit dem t-Wert und baryzentrischen Koordinaten des Schnittpunkts an den SM 1840 zurückgegeben. Dies ist der Treffertyp „Dreieck“.
  2. (2) Standardmäßig werden geschnittene Dreiecke mit dem Alpha-Bit-Set an den SM 1840 zurückgegeben, auch wenn die Traversierung noch nicht abgeschlossen ist. Der zurückgegebene Traversierungsstapel enthält den Status, der erforderlich ist, um die Traversierung fortzusetzen, wenn die Software bestimmt, dass das Dreieck tatsächlich transparent war.
  3. (3) Dreieck-Schnittpunkte in manchen Ausführungsformen werden für Strahlenbündel nicht unterstützt, so dass getroffene Dreieck-Bereiche standardmäßig an den SM 1840 als Treffertyp „TriRange“ oder „DreiBereich“ zurückgegeben werden, der einen Zeiger auf den ersten Dreieck-Block, der den Bereich überlappt, Parameter, die den Bereich spezifizieren, und den t-Wert des Schnittpunkts mit dem Blatt-Begrenzungskasten enthält.
• ■ Ein Element-Bereich, bestehend aus einem Index (abgeleitet von einer vom Benutzer bereitgestellten „Element-Bereich-Basis“, die im Complet gespeichert ist) und einer Zählung von Elementen.

Different ray flags can be used to control different aspects of traversal behavior, backface culling and handling of the different child node types, depending on the pass / fail status of an optional RayOp test. RayOps add flexibility to the capabilities of the TTU. In some embodiments, the RayOps section introduces two beam flag versions that can be dynamically selected based on a specified operation on data transmitted with the beam and data stored in the complete. To explore such flags, it is first helpful to understand the different types of child nodes that are allowed within a BVH, as well as the different types of hits that the TTU 700 can return to the SM. Example node types are:

• ■ A child complete (ie an internal node)

The TTU sets by default 700 the traversal continues by descending into the child completes.

• ■ A triangle area that corresponds to a contiguous set of triangles within a triangle buffer.
  1. (1) By default, triangle areas that a beam hits are identified by the TTU 700 treated natively by testing the triangles for intersections and shortening the beam accordingly. When traversing is complete and a triangle has been hit, by default the triangle ID is sent to the SM along with the t-value and barycentric coordinates of the intersection 1840 returned. This is the "triangle" hit type.
  2. (2) By default, cut triangles are sent to the SM using the alpha bit set 1840 returned, even if the traversal has not yet been completed. The returned traversal batch contains the status required to continue traversing when the software determines that the triangle was actually transparent.
  3. (3) Triangle intersections in some embodiments are not supported for beams, so triangle areas hit by default on the SM 1840 returned as the hit type "TriRange" or "ThreeRange", which contains a pointer to the first triangle block that overlaps the area, parameters that specify the area, and the t-value of the intersection with the leaf boundary box.
• ■ An element area, consisting of an index (derived from a user-provided "element area base", which is stored in the complete) and a count of elements.

• ■ Ein Instanzknoten.

By default, element areas are sent to the SM as the "ItemRange" or "Element Area" type 1840 returned consisting of z. An index, a count, and the t-value of the intersection with the leaf bounding box.

• ■ An instance node.

The TTU 700 can, in some embodiments, edit an instantiation level natively by transforming the beam into the coordinate system of the instance BVH. Additional levels of instantiation (or any other level of instantiation, depending on the strategy) can be handled in the software. For this purpose, the "InstanceNode" or "InstanceNode" hit type is provided, which consists of a pointer to the instance node and the t-value of the intersection with the leaf delimitation box. In other implementations, the hardware can process two, three, or more levels of instantiation.

In addition to the node-specific hit types, a generic "NodeRef" hit type is provided, which consists of a pointer to the parent element itself, an ID indicating which child has been cut and the t-value of the point of intersection with the bounding box Child exists.

A hit type "error" can be provided for cases in which the query or the BVH was improperly formed or if the traversal encountered problems during the traversal.

In the event that the ray or the ray bundle misses the entire geometry in the scene, a “no” hit type can be provided.

How the TTU deals with each of the four possible node types is determined by a set of node-specific mode flags that are set as part of the query for a particular ray. The "default" behavior mentioned above corresponds to the case where the mode flags are set to all zeros.

Alternative values for the flags make it possible to cull all nodes of a certain type, return nodes of a certain type to the SM as hit type NodeRef ("node reference") or return triangle areas or instance nodes to the SM using their corresponding hit types, instead of them natively within the TTU 700 to process.

Additional mode flags can be provided to control the treatment of alpha triangles.

Example computing system

Systems with multiple GPUs and CPUs are used in a variety of industries because developers have more parallelism in applications such as B. Reveal and use artificial intelligence. Powerful GPU-accelerated systems with tens of thousands of computing nodes are used in data centers, research facilities and supercomputers to solve ever bigger problems. As the number of processing devices within the high-performance systems increases, the communication and data transmission mechanisms have to be scaled to support the increased band transfer between the processing devices.

21 is a conceptual diagram of one using the PPU 1700 of 15 implemented processing system 2000 according to one Embodiment. The example system 2000 may be configured to implement one or more of the methods disclosed in this application. The processing system 2000 contains a CPU 1930 , a switch 1910 and several PPUs each 1700 and corresponding memory 1704 , The NVLink 1710 provides high speed communication links between each of the PPUs 1700 ready. Although in 21 a certain number of connections via NVLink 1710 and connection 1702 shown is the number of connections to each PPU 1700 and the CPU 1930 vary. The switch 1910 forms an interface between the connection 1702 and the CPU 1930 , The PPUs 1700 , the memory 1704 and the NVLinks 1710 can reside on a single semiconductor platform around a parallel processing module 1925 to build. In one embodiment, the switch supports 1910 two or more protocols for interfacing between different different connections and / or links.

In another embodiment (not shown), the NVLink provides 1710 one or more high speed communication links between each of the PPUs 1700 and the CPU 1930 ready, and the switch 1910 forms interfaces between the connection 1702 and each of the PPUs 1700 , The PPUs 1700 who have favourited Memory 1704 and the connection 1702 can reside on a single semiconductor platform around a parallel processing module 1925 to build. In yet another embodiment (not shown) the connection 1702 one or more communication links between each of the PPUs 1700 and the CPU 1930 ready, and the switch 1910 forms interfaces between each of the PPUs 1700 using the NVLink 1710 to establish one or more high-speed communication links between the PPUs 1700 provide. In yet another embodiment (not shown), the NVLink provides 1710 one or more high speed communication links between the PPUs 1700 and the CPU 1930 via the switch 1910 ready. In yet another embodiment (not shown) the connection 1702 one or more communication links between each of the PPUs 1700 ready right away. One or more of the high-speed communication links NVLink 1710 can be a physical NVLink connection or an on-chip or on-die connection using the same protocol as the NVLink 1710 be implemented.

In the context of the present description, a single semiconductor platform can refer to a single unitary semiconductor-based integrated circuit that is fabricated on a die or chip. It should be noted that the term single semiconductor platform can also refer to multichip modules with increased connectivity, which simulate on-chip operation and make significant improvements compared to conventional bus implementation. Of course, the various circuits or devices can also be arranged separately or in different combinations of semiconductor platforms as desired by the user. Alternatively, the parallel processing module 1925 can be implemented as a circuit board substrate and any of the PPUs 1700 and / or storage 1704 be designed as a packed device. In one embodiment, the CPU is located 1930 , the switch 1910 and the parallel processing module 1925 on a single semiconductor platform.

In one embodiment, the signal transmission rate of each NVLink is 1710 20 to 25 gigabits / second and contains each PPU 1700 six NVLink 1710 Interfaces (as in 22 shown are five NVLink 1710 Interfaces for every PPU 1700 contain). Any NVLink 1710 offers a data transfer rate of 25 gigabytes / second in each direction, with six links 1700 Deliver gigabytes / second. The NVLinks 1710 may be used only for PPU-to-PPU communication, as in 21 shown, or for any combination of PPU-to-PPU and PPU-to-CPU if the CPU 1930 also one or more NVLink 1710 Contains interfaces.

In one embodiment, the NVLink enables 1710 direct load / memory / atomic access from the CPU 1930 to the store 1704 any PPU 1700 , In one embodiment, the NVLink supports 1710 Coherence operations so that from the memories 1704 read data in the cache hierarchy of the CPU 1930 can be saved, which is the cache access latency for the CPU 1930 reduced. In one embodiment, the NVLink includes 1710 Support for address translation services (ATS), so the PPU 1700 directly on page tables within the CPU 1930 can access. One or more of the NVLinks 1710 can also be configured to operate in a low consumption mode.

22 illustrates an example system 1965 in which the different architecture and / or functionality of the various previous embodiments can be implemented. The example system 1965 can be configured to use one or more of the methods disclosed in this application 250 implemented.

As shown, is a system 1965 provided the at least one central processing unit 1930 contains that with a communication bus 1975 connected is. The communication bus 1975 can be implemented using any suitable protocol, e.g. B. PCI ( Peripheral Component Interconnect), PCI-Express, AGP (Accelerated Graphics Port), HyperTransport or any other bus or point-to-point communication protocol (s). The system 1965 also contains a main memory 1940 , Control logic (software) and data are in main memory 1940 stored, which can be designed as a random access memory (RAM).

The system 1965 also contains input devices 1960 , the parallel processing system 1925 and display devices 1945 , ie a conventional CRT (cathode ray tube), LCD (liquid crystal display), LED (light emitting diodes), plasma display or the like. User input can be from the input devices 1960 are received, e.g. B. a keyboard, a mouse, a touchpad, a microphone and the like. Each of the aforementioned modules and / or devices can even reside on a single semiconductor platform around the system 1965 to build. Alternatively, the different modules can also be arranged separately or in different combinations of semiconductor platforms according to the wishes of the user.

Furthermore, the system 1965 for communication purposes via a network interface 1935 be coupled to a network (e.g. a telecommunications network, local area network (LAN), wireless network, wide area network (WAN) such as the Internet, peer-to-peer network, cable network or the like).

The system 1965 may also include secondary storage (not shown). The secondary storage contains, for example, a hard disk drive and / or a removable storage drive, which represents a floppy disk drive, a magnetic tape drive, a CD drive, a DVD drive, a recording device or a USB flash memory. The removable storage drive reads and / or writes from or to a removable storage unit in a known manner.

Computer programs or computer control logic algorithms can be in main memory 1940 and / or the secondary storage can be stored. Such computer programs enable the system 1965 to perform various functions when they are performed. The memory 1940 , secondary storage and / or any other storage are possible examples of computer readable media.

The architecture and / or functionality of the various previous figures can be implemented in the context of a general computer system, a board system, a game console system for entertainment purposes, an application specific system and / or any other desired system. The system 1965 can take the form of, for example, a desktop computer, a laptop computer, a tablet computer, servers, supercomputers, a smartphone (e.g. a wireless, portable device), a personal digital assistant (PDA), a digital camera, a vehicle, a head-mounted display, a portable electronic device, a cellular phone, a television, a workstation, a game console, an embedded system, and / or other type of logic.

Machine learning

Deep neural networks (DNNs) that are based on processors such. B. the PPU 1700 have been used for various applications, from self-driving automobiles to faster drug development, from automatic image capture in online image databases to intelligent real-time voice translation in video chat applications. Deep learning is a technique that models the neural learning process of the human brain, learns continuously, becomes increasingly intelligent and, over time, provides more accurate results. A child is first taught by an adult to correctly identify and classify different forms in order to be able to identify forms without tutoring. Similarly, a deep learning or neural learning system needs to be trained in object recognition and classification so that it becomes smarter and more efficient to identify basic objects, hidden objects, etc., and to assign context to the objects.

At the simplest level, neurons in the human brain look at various inputs that are received, each of these inputs are assigned levels of importance, and outputs are forwarded to other neurons to respond to. An artificial neuron or perceptron is the most basic model of a neural network. In one example, a perceptron may receive one or more inputs that represent various features of an object for which the perceptron is trained to recognize and classify them, and each of these features is assigned a specific weight based on the importance of that feature for the definition of the shape of an object is based.

A DNN model contains several layers of many connected perceptrons (e.g. nodes) that can be trained with enormous amounts of input data in order to solve complex problems quickly and with high accuracy. In one example, a first layer of the DNN model breaks down an input image of an automobile into different parts and searches for basic patterns such as e.g. B. lines and angles. The the second layer composes the lines to create patterns of higher levels such B. Search wheels, windshields and mirrors. The next layer identifies the vehicle type, and the last few layers create a label for the input image that identifies the model of a particular vehicle brand.

Once the DNN is trained, the DNN can be used and used to identify and classify objects or patterns in a process known as a conclusion. Examples of a conclusion (the process by which a DNN extracts useful information from a given input) include identifying handwritten numbers on ATM checks, identifying pictures of friends in photos, providing movie recommendations to over fifty million users , identifying and classifying different types of automobiles, pedestrians and road hazards in driverless vehicles or translating human speech in real time.

During training, data flows through the DNN in a forward propagation phase until a prediction is generated that indicates a label corresponding to the input. If the neural network does not correctly label the input, errors between the correct label and the predicted label are analyzed and the weights for each feature are adjusted during a backward propagation phase until the DNN correctly labels the input and other inputs in a training record. Training complex neural networks requires enormous amounts of parallel computing power, including floating point multiplications and additions, by the PPU 1700 get supported. Conclusion is less computationally intensive than training because it is a latency-sensitive process in which a trained neural network is applied to new inputs that it has not seen before to classify images, translate speech, and generally infer new information.

Neural networks rely heavily on mathematical matrix operations, and complex multilayer networks require enormous amounts of floating point power and bandwidth for efficiency and speed. With thousands of processing cores optimized for mathematical matrix operations and delivering tens to hundreds of TFLOPS performance, the PPU is 1700 a computing platform capable of delivering the performance required for deep neural network-based artificial intelligence and machine learning applications.

All of the above patents and publications are incorporated by reference as if they were expressly stated.

While the invention has been described in connection with what is currently considered to be the most practical and preferred embodiments, it will be appreciated that the invention is not to be limited to the disclosed embodiments, but on the contrary is intended to cover various modifications and equivalent arrangements that are spiritually and Scope of protection of the appended claims are included.

QUOTES INCLUDE IN THE DESCRIPTION

This list of documents listed by the applicant has been generated automatically and is only included for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• US 14563872 [0001]
• US 9582607 [0001, 0017, 0107]
• US 9552664 [0001]
• US 9569559 [0001, 0017]
• US 10025879 [0001]
• US 14737343 [0001]
• US 16101066 [0001,0113]
• US 16101247 [0001]
• US 16101180 [0001]
• US 16101148 [0001,0113]
• US 16101196 [0001]
• US 16101232 [0001]
• US 20160070820 [0017]
• US 20160070767 [0017]
• US 14697480 [0106]

Non-patent literature cited

• Hery et al., Towards Bidirectional Path Tracing at Pixar (2016) [0011]

Claims (27)

Streaming cache for use by a ray tracker, the streaming cache comprising: at least one cache line; a hit / miss detection circuit that determines whether memory access requests from beam operations are hits or failures, the circuit initiating a read of data from memory into the at least one cache line in response to detected failures; and a data path that provides time coherent data to groups of beam operations requesting the same cache line. A method of using a cache to schedule execution of requests, the method comprising: Determining whether a first request uses data used by a second request; if it is determined that the first request uses the same data used by the second request, grouping the first and second requests; and when the data becomes available in the cache, delivering the data in response to the first and second requests at approximately the same time, thereby scheduling the grouped first and second requests for execution. Procedure according to Claim 2 , the first and second requirements being requirements to determine intersection points between beams and volume limit boxes. Streaming cache for use by a ray tracker, the streaming cache comprising: a hit / miss detection circuit that determines whether memory access requests to the cache of beam operations are hits or failures, the circuit initiating readout of data from memory in response to detected failures; and a data path which provides the data read out from the memory in a coherent manner in time in response to detected failures to groups of beam operations which request the same read data. Streaming cache after Claim 4 , wherein the data path plans successive beam operations that traverse the same part of a bounding volume hierarchy by time coherently delivering the data read from memory in response to detected failures to groups of beam operations requesting the same read data to traverse the same portion of a bounding volume hierarchy to be carried out. Streaming cache after Claim 4 or 5 which further includes a pending address table and a pending request table, the pending address table containing addresses of data read out of memory by the circuit in response to detected failures, the pending requests- Table has multiple entries referencing the same pending address table entry, the multiple pending requests table entries corresponding to one of the groups of ray operations. Streaming cache according to one of the Claims 4 to 6 wherein the beam operations include traversing a bounding volume hierarchy. Streaming cache according to one of the Claims 4 to 7 , the memory including a cache memory that supports the streaming cache. Streaming cache according to one of the Claims 4 to 8th , the circuit including a label structure. Streaming cache after Claim 9 , where the label structure is linked to a pending address table. Streaming cache after Claim 9 , where the label structure is not linked to a pending address table. Streaming cache according to one of the Claims 4 to 11 wherein the circuit is designed to drop responses from memory when there is no space to store the responses in cache lines. Streaming cache after Claim 12 , the circuit being further designed not to accept new requests if too many responses are dropped from memory. Streaming cache for use by a ray tracker, the streaming cache comprising: at least one cache line; a hit / miss detection circuit that determines whether memory access requests from beam operations are hits or failures, the circuit initiating a read of data from memory into the at least one cache line in response to detected failures; and a data path that provides data to groups of independent beam operations requesting the same cache line at the same time or approximately the same time, thereby scheduling the independent beam operations. Cache streaming method, comprising: Determining whether beam access memory access requests are hits or failures in a cache memory; Requesting data from main memory in response to detected failures; and temporally coherent streaming of data provided by main memory via a data path based on requests in response to detected failures to groups of beam operations that require the same read data. Cache streaming procedure after Claim 15 , whereby the time-coherent streaming of data via the data path thereby plans the time-coherent execution of successive beam operations that traverse the same part of a limitation volume hierarchy. Cache streaming procedure after Claim 15 or 16 , further comprising storing addresses of data requested from main memory in a pending address table in response to detected cache misses. Cache streaming procedure after Claim 17 further comprising storing multiple entries referencing the same pending address table entry in a pending requests table, the multiple pending requests table entries corresponding to one of the groups of ray operations. Cache streaming method according to one of the Claims 15 to eighteen , which further includes traversing a bounding volume hierarchy with beam operations. Cache streaming method according to one of the Claims 15 to 19 , which further includes supporting the cache memory with the main memory. Cache streaming method according to one of the Claims 15 to 20 which further comprises maintaining a tag structure for the cache memory. Cache streaming procedure Claim 21 , which further includes linking the label structure to a pending address table. Cache streaming procedure after Claim 21 or 22 , which further includes not linking the label structure to a pending address table. Cache streaming method according to one of the Claims 15 to 23 , further comprising dropping a response from main memory when there is no space to cache the response. Cache streaming after Claim 24 , which further includes not accepting new beam processing requirements if too many responses are dropped from main memory. Process comprising: Determining that multiple independent beams request common data; and Providing data to the plurality of independent beams that have been determined to request the common data, thereby synchronizing the independent beams to process the common data. Procedure according to Claim 26 , wherein the determining and delivering is performed by a streaming cache in response to memory read requests.

Images