KR20200018207A - Systems and methods for rendering reflections - Google Patents

Abstract

Embodiments of the present invention provide systems for rendering reflections and methods thereof. To add reflections to a pixel in an image, ray marching is used to attempt to find a ray intersection for primary reflections. When using rasterization to render a scene, objects outside the viewport are culled. As such, ray marching may fail in various situations. For example, during ray marching, a ray emits the viewport without intersecting any other objects of the scene. In such a situation where ray marching fails, the ray can be re-cast as a ray-traced ray. The ray-traced ray is cast into the full three-dimensional (3D) scene with all objects present (i.e., objects are not culled). Ray tracing is then used to attempt to find a ray intersection, i.e., for a primary reflection. The embodiments can be used in real-time or near-real-time applications such as video games.

Description

Systems and Methods of Rendering Reflections {SYSTEMS AND METHODS FOR RENDERING REFLECTIONS}

FIELD The present disclosure relates generally to computer graphics, and more particularly to systems and methods of rendering reflections.

Computer-generated images are often generated by examining geometric models in view space and modeled objects in view space. The geometric model of the objects may have arbitrary resolution but generally each object is placed in view space and has a finite number of polygons, such as triangles, and / or having a color, color pattern, or texture on their surface. Expressed as alpha values representing the transparency of the polygon. The image is typically output (ie stored, displayed, transmitted or otherwise processed) as a pixel array.

One common feature of computer generated images is the inclusion of reflections in the output image. Foaming the reflections provides more visual and realistic results. In certain applications, such as animated films, ray tracing techniques may be used to provide ray tracing reflections that provide good results. However, ray tracing is computationally expensive. Complex scenes can take hours or even days to render a single frame of a video sequence using ray tracing reflections even when using very powerful computers or render farms.

In other applications, such as video games, computation speed is a priority. Frames in video games are rendered very quickly when the user interacts with the video game, ie in real time or in near real time. As such, ray tracing techniques for reflections (which may take hours or days to render single frames) are not suitable for video games. For this reason, various other techniques have been developed for rendering reflections in video games in real time or near real time. However, these other techniques suffer from poor quality, especially compared to the reflections obtained from full ray tracing.

As such, there is a need in the art for a system and method of rendering reflections that overcome the disadvantages and limitations of existing approaches.

One embodiment provides a method, computer readable storage medium and device for generating reflection information for a pixel in an image. The method includes: determining, by one or more processors, an illuminance value of a surface of an object in a pixel; Determining, by one or more processors, the number of rays to occur for the pixel based on the illuminance value; Selecting a ray direction for the ray by one or more processors for each ray in the number of rays; For each ray in the number of rays, performing, by one or more processors, ray propagation of the ray based on the ray direction of the ray; For each ray for which ray progression is successful, storing color information of the object intersected by the ray found through the ray progression by one or more processors; For each ray for which ray propagation has failed, casting the ray through ray tracing by one or more processors and storing color information of the object intersected by the ray found through ray tracing; And generate reflection information for the pixel based on the color information of the object crossed by the rays found through the ray propagation by the one or more processors and the color information of the objects crossed by the rays found through the ray tracing. It includes a step.

One embodiment provides a method, computer readable storage medium and device for generating reflection information in an image. The method includes: determining, by one or more processors, a first roughness value of a surface of an object at a first pixel of an image; Determining, by one or more processors, the number of rays occurring for the first pixel based on the first illuminance value; Identifying, by one or more processors, a second pixel within a threshold radius of the first pixel in the image; Determining, by one or more processors, a second roughness value of the surface of the object at the second pixel; Determining, by one or more processors, that the difference between the first illuminance value and the second illuminance value is equal to or less than an illuminance threshold; Determining, by one or more processors, color information of the object crossed by the reflected light beam corresponding to the second pixel; And generating, by one or more processors, reflection information for the first pixel based on the color information of the object intersected by the reflection ray corresponding to the second pixel; The first pixel is included in the first set of pixels and the second pixel is included in the second set of pixels, and the color information of the objects crossed by the reflected rays corresponding to the pixels in the second set of pixels is Color information of the objects determined through ray tracing and / or ray propagation and intersected by the reflected rays corresponding to the pixels in the first set of pixels are reflected rays corresponding to the pixels in the second set of pixels. Based on the color information of the objects intersected by them.

One embodiment provides a method, computer readable storage medium, and device for generating reflective information. The method includes: determining, with respect to the first frame, the location of the intersection of the object based on the reflected light beam for the first pixel, wherein the location of the intersection of the object corresponds to the shape of the ellipse. step; For a first frame, determining a position in a reflective area of an intersection of the object based on the reflected light beam for the first pixel; Projecting a first line relative to the first frame from a position in the reflective region of the intersection of the object by the reflected light beam relative to the first pixel towards the first position of the camera; Determining a position of the microfacet portion of the first line on the surface of the first object relative to the first frame; For a second frame, determining a location of the intersection of the object based on the reflected light beam for the second pixel, the second frame following the first frame; For a second frame, determining a position in the reflective area of the intersection of the object based on the reflected light beam for the second pixel; Projecting, for a second frame, a second line from a position in the reflective region of the intersection of the object with the reflected light beam relative to the second pixel towards the second position of the camera; Determining, with respect to the second frame, the location of the intersection of the second line on the surface of the second object; Determining that the surface roughness of the position of the intersection of the second line on the surface of the second object is within a critical roughness of the surface roughness of the position of the intersection of the first line on the surface of the first object; And generating reflection information for the second pixel based on the reflection information for the first pixel.

1 is a block diagram of a computer system for rendering images in accordance with aspects of the present disclosure.
2 is a block diagram illustrating the interaction of a process and a buffer according to one embodiment.
3 is a block diagram of a scene to be rendered according to one embodiment.
4A is a block diagram illustrating rendering a scene using rasterization, in accordance with an embodiment.
4B is an example of a rasterized image of a scene according to one embodiment.
5 is an example image illustrating different reflection types according to one embodiment.
6 is a block diagram illustrating specular reflections in accordance with an embodiment.
7 is a block diagram illustrating glossy reflections in accordance with an embodiment.
8A is a block diagram illustrating light propagation success in accordance with an embodiment.
8B illustrates an image in screen space according to one embodiment.
9A is a block diagram illustrating light propagation failure, according to one embodiment.
9B illustrates an image in screen space of the scene shown in FIG. 9A, according to one embodiment.
10A is a graphical representation of a portion of z-buffer data according to one embodiment.
10B-10D are graphical representations of portions of z-buffer data according to one embodiment.
11 is a flow diagram of method steps for performing ray propagation in accordance with an embodiment.
12 is a block diagram illustrating performing ray tracing on rays that failed to propagate rays in accordance with one embodiment.
13 is a flow diagram of method steps for rendering reflections in accordance with an embodiment.
14A is a block diagram illustrating a 3D (three-dimensional) scene according to one embodiment.
FIG. 14B is a block diagram illustrating the 3D scene in FIG. 14A, where some rays are reused from neighboring pixels according to one embodiment. FIG.
15 is a flow diagram of method steps for reusing ray information of adjacent pixels in accordance with an embodiment.
16A is an example of reflection, according to one embodiment.
16B illustrates a line from the projected point to the virtual position of the update of the eye / camera, in accordance with one embodiment.
17 is a block diagram illustrating two points of subsequent frames having different surface roughness values, in accordance with an embodiment.
18 is a flow diagram of method steps for performing temporal filtering according to an embodiment.
19 illustrates examples of drawn reflections.
20 is a block diagram illustrating elongated reflections, according to one embodiment.
21 is an example of elongated reflection, according to one embodiment.
22 is a plot of pre-computed lengths of short axis of ellipses representing elongated reflections, according to one embodiment.
FIG. 23 is a plot of a pre-computed length of the major axis of an ellipse representing elongated reflections, according to one embodiment.
24 is an example of applying an elliptical filter to glossy reflections according to one embodiment.
25 is a flow diagram of applying an elliptical filter in accordance with an embodiment.

The following detailed description is not intended to limit the disclosure or its application and uses, but is merely illustrative. Moreover, it is not limited by any representation or implied theory presented in the prior art, background, summary, brief description of the drawings, or the following detailed description.

Embodiments of the present disclosure provide systems and methods for rendering reflections. The disclosed embodiments can be performed in real time or near real time and are therefore suitable for applications, such as video games. As described in more detail herein, the disclosed embodiments utilize rasterization to render primary gazes (ie, from a virtual camera into screen space) of a 3D (three-dimensional) virtual scene. Ray progression is used to find the ray intersection for the primary reflections to add reflections for each pixel in screen space. Objects outside the viewport are curled when rendering the scene using rasterization. Thereby, ray progression may fail in various situations, such as when the ray progresses, the ray exits the viewport without intersecting any other object in the scene. In this situation where beam progression fails, the beam may be recast as a ray traced beam. Ray traced rays are cast to a full 3D scene where all objects are present (ie, the object is not curled). Ray tracing is then used to find out about the ray intersection, that is, primary reflection. The disclosed embodiments can be used in real time or near real time applications to realize visually satisfactory reflections that may in some cases be indistinguishable from fully ray traced reflections.

Returning to the drawings, FIG. 1 is a block diagram of a computer system 100 for rendering images in accordance with aspects of the present disclosure. Computer system 100 may be used, for example, to render images of a video game. Computer system 100 is shown to include a console 102 coupled to a display 104 and input / output (I / O) devices 106. The console 102 is shown to include a processor 110, a program code store 112, a temporary data store 114, and a graphics processor 116. The console 102 may be a handheld video game device, a video game console (eg, special purpose computing device) for operating video games, a general purpose laptop or desktop computer, or any suitable computing system, such as a mobile phone or It may also be a tablet computer. Although shown as one processor in FIG. 1, the processor 110 may include one or more processors having one or more processing cores. Similarly, although shown as one processor in FIG. 1, graphics processor 116 may include one or more processors having one or more processing cores.

Program code storage 112 includes ROM (read only memory), RAM (random access memory), DRAM (dynamic random access memory), SRAM (static random access memory), hard disk, other magnetic storage, optical storage, It may be a combination or variation of these storage device types. In some embodiments, a portion of the program code is programmable ROM (eg, ROM, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)). Portions of the program code may be stored on a removable medium, such as disk 120 (eg, CD-ROM, DVD-ROM, etc.) or stored on a cartridge, memory chip, or the like. Or may be obtained via a network or other electronic channel as needed. In some implementations, the program code can be viewed as being implemented in a non-transitory computer readable storage medium.

Temporary data storage 114 is available for storing variables and other game and processor data. In some embodiments, temporary data store 114 is RAM and stores data generated during play of a video game, the portions of which are frame buffers, depth buffers, polygon lists, texture stores, and / or video game presentations. It may be reserved for other data required or available for rendering the images as part of.

In one embodiment, I / O devices 106 are devices that a user interacts with in order to play a video game or otherwise interact with the console 102. I / O device 106 is any device for interacting with console 102 including, but not limited to, a video game controller, joystick, keyboard, mouse, keypad, VR (virtual reality) headset or device, and the like. It may also include.

Display 104 may be any type of display device including a television, computer monitor, laptop screen, mobile device screen, tablet screen, and the like. In some embodiments, I / O devices 106 and display 104 include a common device, eg, a touch screen device. Still further, in some embodiments, one or more I / O devices 106 and display 104 are integrated into console 102.

In various embodiments, the console 102 is more likely because the particular game sequence presented on the display 104 by the video game depends on the results of the game command processing and these game commands likewise rely on subsequent user inputs as well. (And processor 110 and graphics processor 116) are configured to process the inputs quickly and render the responsive image sequence in real time or near real time.

Various other components may be included in console 102, but are omitted for clarity. One example includes a console device configured to connect the console 102 to a network such as the Internet.

2 is a block diagram illustrating the interaction of a process and a buffer according to one embodiment. As shown in FIG. 2, the processor 110 executes program code and program data. In response to executing the program code, the processor 110 outputs rendering instructions to the graphics processor 116. Graphics processor 116 then reads data from polygon buffer 150 and interacts with pixel buffer (s) 160 to form an image sequence of one or more images output to the display. Alternatively, instead of sending rendering instructions to graphics processor 116 or in addition to sending rendering instructions to graphics processor 116, processor 110 may interact directly with polygon buffer 150. For example, processor 110 may determine which objects should appear in the view and provide polygon or other mathematical representations of these objects to polygon buffer 150 for subsequent processing by graphics processor 116.

In one example implementation, processor 110 issues high level graphics commands to graphics processor 116. In some implementations, these high level graphics commands may be as specified by the OpenGL specification or as specified by the graphics processor manufacturer.

In one implementation of the image rendering process, graphics processor 116 reads polygon data from polygon buffer 150 for the polygon, processes the polygon and updates pixel buffer (s) 160 accordingly, and then Move to the next polygon until the polygons are processed or until at least all of the polygons in the view that need to be processed and / or are processed. As such, even if the polygons may be read in place and are a finite set, the renderer processes the stream of polygons where the number of polygons is known or determinable. With regard to memory efficiency and speed, it may be desirable in some implementations in which polygons are processed as a stream (as opposed to random access or other ordering), so that the high speed memory used for the polygons being processed contains all of the images. Not required for polygons.

In some embodiments, the processor 110 may load the polygon data into the polygon buffer 150 in sort order (if not possible with nested polygons), but more generally the polygon is in an unordered order. Is stored in the polygon buffer 150. These examples use polygon as the image elements being processed, but it should be understood that the apparatuses and methods described herein may also be used for image elements other than polygons.

3 is a block diagram of a scene 300 to be rendered according to one embodiment. The scene 300 includes a plurality of 3D (three-dimensional) objects 302, 302A-302B. Each object 302 may consist of a set of polygons, such as triangles. Camera 306 is configured to capture an image of scene 300. The projection of scene 300 is captured by camera 306 and represented by screen space 308. The view of the scene 300 captured by the camera 306 is represented by the viewport 304. As shown, some of the objects, such as object 302A of scene 300, may be external to viewport 304. As shown, some of the objects, such as object 302B, may be partially outside of viewport 304.

In one embodiment, the image of the scene 300 displayed on the display device corresponds to the screen space 308. The view of the scene 300 that the camera 306 can see (ie, the image represented by the screen space 308) changes as the camera 306 moves in 3D space with respect to the objects 302 in the scene. Can be. Also, the objects 302 can move in the scene 300.

4A is a block diagram illustrating rendering scene 400 using rasterization, in accordance with an embodiment. Similar to the image shown in FIG. 3, scene 400 includes a plurality of objects. Camera 406 is configured to capture an image of scene 400 represented in screen space 408. Camera 406 observes scene 400 through viewport 410.

The disclosed embodiments include rasterizing objects in scene 400 to generate an image in screen space 408. Rasterization seeks to render the pixels as visible directly from the camera 406. Rasterization can provide good performance when the renderer (eg, processor 110 and / or graphics processor 116) does not need any global information about scene 400.

One rasterization algorithm takes a 3D scene 400 described as objects comprising polygons and renders the scene on a 2D surface, typically a computer monitor, in scene space 408. The polygons themselves are represented as a set of triangles. Each triangle is represented by three vertices in 3D space. At a very basic level, the rasterizer takes a stream of vertices, converts them into corresponding 2D points in screen space 408, and fills the transformed 2D triangles as needed.

In general, rasterization involves curling one or more objects or partial objects. The frustum curl removes any objects outside of viewport 410, such as object 402A. Viewport curl removes portions of objects that partially overlap with an edge of viewport 410, such as portion of object 402B. The back curl removes the back portion 404 of the objects that cannot be seen by the camera 406. In some embodiments, a depth test may be performed to remove portions 406 of objects that are obscured by other objects in pixels in screen space 408.

When rasterization is complete, an image of the scene in screen space 408 is generated. In some embodiments, pixel density of screen space 408 can also result in loss of information about objects in scene 400.

4B is an example of a rasterized image of a scene according to one embodiment. The image shown in FIG. 4B represents an image of the screen space 408 of the scene 400 captured by the camera 406 of FIG. 4A.

For each pixel 412 in screen space 408, the processor accesses data corresponding to the position of the object within the pixel, the color of the object, the orientation of the object, and / or surface texture information (eg, roughness values), and the like. do. The result of rasterization is therefore a 2D image with relatively limited information of the actual 3D scene. Adding reflections to rasterized images can be a difficult problem to solve because the information needed for high quality reflections may be outside of the viewport 410 and / or behind a partially curled object and thus these objects This is because the information about is no longer available to the renderer.

As discussed, limited information is available for the renderer when reflections are added to the image shown in FIG. 4B. Since various objects and parts of the objects are curled to produce a rasterized image, the data on the objects and parts of the objects that were curled are no longer available and cannot be used to add reflections. For example, data corresponding to object 402A (ie, an object that was curled by frustum curl) is no longer available to add reflection to uncurled objects, such as object 414.

5 is an example image illustrating different reflection types according to one embodiment. In some embodiments, the reflections can be classified as mirror reflection or glossy reflection. The lower right portion 502 of the image represents mirror reflections, and the upper left portion 504 of the image represents glossy reflections.

The reflection can be mirror reflective or glossy based on the roughness (or smoothness) of the surface forming the reflection. As used herein, mirror reflection refers to mirror type reflection from a smooth surface; Glossy reflection arises from parallel rays of light that reflect from a surface comprising a plurality of microfacets, which reflect or bounce off the microfacets in various directions towards the viewer or camera.

6 is a block diagram illustrating specular reflections in accordance with an embodiment. Surface 600 has a flat and smooth surface. Incident parallel light ray 602 reflects from surface 600 as reflected light rays 604. Since the surface 600 has a smooth surface, the reflected rays 604 are also parallel, like the incident rays 602. The result is mirror type reflections from the surface.

7 is a block diagram illustrating glycine reflections in accordance with an embodiment. Surface 700 is not smooth and includes a plurality of microfacets 706 (eg, reflective surfaces with random orientations). Incident parallel light ray 702 reflects from surface 700 as reflected light rays 704. Because the surface 700 is not smooth, the reflected rays 704 reflect from the microfacets 706 in a plurality of directions towards the viewer or camera 708. Thus, glossy reflections have a blurred effect when the viewer or camera 708 sees.

As disclosed herein, reflections may be formed on objects from adjacent objects. In one embodiment, the technique for determining reflection of one object onto another object is referred to as ray marching. As described below, the ray propagation may result in the ray propagation success or the ray propagation failure.

8A is a block diagram illustrating light propagation success in accordance with an embodiment. The image in screen space 808 is shown in FIG. 8B. For each pixel of the image in screen space 808, the processor determines whether an object is shown in the pixel. Objects that are visible to a given pixel are shown as line of sight 814 in FIG. 8A. Some pixels, such as pixel 804, may show a background (eg, sky), and thus may not show any object. In some embodiments, ray propagation is omitted for any background pixels, such as pixel 804.

In one embodiment, ray progression is performed for each pixel of the image in screen space 808 where the object is displayed, such as pixel 802. As discussed above, for an object shown in pixel 802, the processor may determine the position of the object, the color of the object, the orientation of the object, and / or surface texture information (eg, surface roughness values) in pixel 802. Access to data corresponding to the " Based on the data for this object, one or more light rays may proceed to determine the color of the reflections to be shown at pixel 802.

In some implementations, for smooth surfaces, one ray travels from the pixel 802. As discussed, the smooth surface result is mirror reflections, where the reflected light forms parallel rays. Thus only one ray of light is needed to determine reflection information for smooth surfaces. However, for a rougher surface on which reflected light forms light rays traveling in various directions, multiple light rays are used to determine the reflection information. In some implementations, the number of rays propagated for the pixel increases for a rougher surface. In some implementations, the number of propagating rays may be limited to a ray threshold per frame, such as four rays. In the example shown in FIGS. 8A and 8B, one ray is shown for clarity, but multiple rays may travel from pixel 802 in several directions (ie, for glossy reflections).

As shown in FIGS. 8A and 8B, the point 810 on the object in the scene corresponds to the pixel 802. Based on the surface orientation on the object at point 810, the direction of the primary reflected light ray 806 can be determined. Ray 806 is cast in that direction based on the surface orientation on the object at point 810. In one implementation, ray 806 "goes" pixel by pixel through the image of screen space 808 to determine whether ray 806 intersects another object of the image in screen space 808. The intersection can be determined by examining the depth value (so-called "z value") of the object at the given pixel, compared to the depth (z value) of the ray 806 at the given pixel. As shown, ray 806 intersects another object at point 812, which is 4 pixels away from the example image shown. Ray progression of ray 806 results in "ray progression success" because other objects in screen space 808 are intersected by ray 806. Color information from point 812 may be stored in a buffer to compute the color of the reflection displayed at pixel 802. As described, multiple light rays can be cast from point 810 in several directions based on surface roughness at point 810. Color information of the object intersected by the ray propagated rays may be stored for each ray resulting in ray propagation success and aggregated to form final reflection information at pixel 802.

However, some advanced rays may not intersect any objects of the image in screen space 808 before reaching the edge of the viewport, referred to as "ray progress failure". 9A is a block diagram illustrating light propagation failure, according to one embodiment. An image in the screen space 908 of the scene is shown in FIG. 9B.

As shown, at pixel 902, a point 910 on the object is displayed. Based on the surface orientation (and / or surface roughness) at point 910, one or more light rays travel from point 910. One ray 906 is shown for clarity. The direction of light ray 906 is based on the surface orientation at point 910. Ray 906 travels pixel by pixel through screen space 908. However, ray 906 reaches the edge of viewport 912 without intersecting with any other objects. This is considered to be a ray progression failure. When a ray propagation failure occurs, no color information for any reflection is obtained for point 910 based on the ray propagated ray 906. The number and / or percentage of ray propagation failures occurring in an image may depend on the complexity of the scene, the arrangement of the objects in the scene, and the surface orientations of the objects in the pixels of the scene space image.

One embodiment for performing ray propagation is screen space in pixels until another object intersects (ie, ray propagation success, FIG. 8B) or reaches an edge of the viewport (ie, ray propagation failure, FIG. 9B). It involves traversing the image. Another embodiment for performing optimized ray propagation is described in FIGS. 10A-10D and 11.

In some embodiments, for an image in screen space, a z-buffer can be generated. The z-buffer stores the depth value of the nearest object (relative to the camera position) of the scene for each pixel in the image in screen space. If no objects are shown in screen space for a given pixel, the z buffer can store a null value for that pixel.

In addition, N additional coarse representations of the z buffer may be computed. The number N is configurable. For example, N may be five. In each coarser level representation of the z-buffer the pixels can be reduced to power groups of two (ie 2 pixels, 4 pixels, 8 pixels, 16 pixels, etc.) or reduced to a finer level. In one implementation, downscaling a set of z-values to one z-value is performed by setting the downscaled z-value to the minimum z-value of the set of downscaled z-values.

One embodiment for performing optimized ray propagation is described herein. As described, the processor examines the full size (original) z-buffer to determine whether the light beam cast from the origin pixel intersects another object in the neighboring pixel in the light direction. Otherwise, the processor examines the corresponding pixel position at the next coarser Z buffer level. Check for intersection by examining the next "pixel" in the ray direction at the next coarser z-buffer level. In other words, advancing one "pixel" in the coarser z-buffer corresponds to advancing or "jumping" over multiple pixels of the original image because the coarser z-buffer represents a plurality of pixels. If no intersection is found at the next pixel of the coarser z-buffer, the next coarse z-buffer is selected and examined. If the coarsest z-buffer (e.g., the Nth z-buffer) is already selected, the normal ray advances pixel by pixel to search for the intersection (in the coarsest z-buffer).

If an intersection is found in one of the coarser z-buffers, the processor performs a "level up" and examines the pixel position of the intersection in the next finer z-buffer and the ray travels in the light direction in the next finer z-buffer. do. In this way, once the intersection is found, the processor upscales to the next finer z buffer and continues with the ray propagation. Each time the processor steps in the radial direction (at the proper level of the z-buffer) without operating anything, the processor downscales to the next coarser z-buffer, potentially making larger areas on the screen potentially larger areas of the screen. Skip to As soon as there is an intersection, the processor upscales to the next finer z-buffer to refine the data and then back to ray progression. When the processor reaches an intersection with something at the finest level, the process is completed when an intersection point is found.

10A is a graphical representation of a portion of a z-buffer according to one embodiment. In FIG. 10A, depth values (ie, z values) are shown on the vertical axis, pixels with objects closer to the camera are represented by larger histogram bars, and pixels with objects further away from the camera are smaller histograms. It is shown by a bar. For pixel location 1002, ray 1004 may travel in a particular direction based on the surface orientation at pixel location 1002. The histogram bars shown in FIG. 10A represent z-values of the pixels in the direction of ray 1004 in screen space.

In order to perform optimized ray propagation, the z-buffer of the depth values of the objects in screen space is accessed by the processor. The graphical representation of the downscaled z-buffer is shown in FIG. 10B. Referring to FIGS. 10A and 10B, the z-values 1010A and 1010B in FIG. 10A have been downscaled to the z-value 1010C in FIG. 10B. Similarly, the z-values 1012A and 1012B in FIG. 10A have been downscaled to the z-value 1012C in FIG. 10B. Since no intersection was found at the next pixel in FIG. 10A, the processor examines the next coarser z-buffer as shown in FIG. 10B. In FIG. 10B, the processor again determines whether ray 1004 intersects the z-buffer of the neighboring pixel. Otherwise, the processor examines the next coarser z-buffer as shown in FIG. 10C.

Further graphical representation of the downscaled z-buffer is shown in FIG. 10C. Referring to FIGS. 10B and 10C, the z-values 1010C and 1012C in FIG. 10B have been downscaled to the z-value 1020 in FIG. 10C. Again, ray 1004 is tested to determine whether ray 1004 intersects the z-buffer of neighboring pixels in the further downscaled z-buffer. In one embodiment, this process is repeated until the ray intersects the neighboring downscaled z value or until it reaches the edge of the viewport. In one embodiment, once the roughest z-buffer is reached, the light beam may be cast "pixel by pixel" in the most downscaled z-buffer space to determine whether an intersection is found.

 A further graphical representation of the collapsing z-buffer is shown in FIG. 10D. As shown in FIG. 10D, the ray intersects another z-buffer value in the downscaled z-buffer. This indicates the ray progression success.

At this stage the processor selects the next finer z-buffer ("lower level") and identifies the next finer z-buffered pixel. The ray propagation is then done on the next finer level to search for intersections. In this way, the ray propagation can efficiently move the empty space by taking larger steps using coarser z-buffer data.

11 is a flow diagram of method steps for performing ray propagation in accordance with an embodiment. In some implementations, 2D images of the scene can be generated using screen space rasterization. For each pixel of the 2D image a z-value corresponding to the depth of the object shown in the pixel may be stored in the z-buffer.

In step 1102, the processor receives z-buffer data corresponding to depth values of the objects in the image. The z-buffer data may comprise a full z-buffer or part of a z-buffer. In step 1104, the processor receives N scaled versions of z-buffer data. Each downscaled version of z-buffer data may combine depth information from two or more pixels. In some implementations, the downscaled version of the z-buffer data takes the minimum depth value for the set of z-values to be downscaled.

For a given pixel on which ray propagation is performed, at step 1106, the processor determines the ray direction for the ray propagation. In step 1108, the processor determines that the next pixel position of the z-buffer data in the ray direction in the ray direction (at the current level initially at the full-size buffer level) is different based on the z value at the next pixel position. Determine whether or not to intersect with. If no intersection is found, the processor determines in step 1110 whether the current level is the coarse z-buffer level. If yes, the method returns to step 1108 where the processor beam advances to the next pixel at the current level. If at step 1110 the processor determines that the current level is not the coarse z-buffer level, then at step 1112 the processor looks for the pixel position in the next coarse z-buffer corresponding to the pixel (ie, "raising the level"). . The method then returns to step 1108, where the processor beam proceeds to the next pixel at the current level (which was only downscaled to the next coarser level).

 If at step 1108 the processor finds an intersection at the next pixel location, at step 1114 the processor determines whether the current level is the original (full size) z-buffer level. If yes, at step 1118, the processor determines the intersection information for the ray based on the z-buffer information (ie, ray propagation success) for the intersection.

If in step 1114 the processor determines that the current level is not the original (full size) z-buffer level, in step 1116 the processor looks for the pixel position in the next finer z-buffer that corresponds to the pixel (ie, "lowering the level"). "). The method then returns to step 1108, where the processor beam proceeds to the next pixel at the current level (which was upscaled to the next finer level).

As such, the method of FIG. 11 provides an optimized ray propagation technique, where large empty spaces can pass faster than irradiating screen space pixel by pixel.

However, as described above in FIGS. 9A and 9B, in some cases, the light propagation results in a light propagation failure. One embodiment of the present disclosure provides a ray traced ray with ray propagation failure.

12 is a block diagram illustrating performing ray tracing on rays that failed to propagate rays in accordance with one embodiment. As shown, a given pixel of the screen space image corresponds to point 1202 of the 3D scene. Point 1202 is located on the object that occurs in the example of FIG. 12 to be the bottom surface of the scene. Light ray 1204 is a light ray traveling from point 1202; Because ray 1204 reaches the edge of viewport 1212 without intersecting with any other objects, ray 1204 results in a ray propagation failure.

According to embodiments of the present invention, ray 1208 may be a ray traced starting at point 1206, the point at which ray propagation reached the edge of viewport 1212. In some cases, ray traced ray 1208 intersects an object, such as object 1210. Color information from point 1214 where ray traced ray 1208 intersects object 1210 may be stored in a buffer to compute the color of the reflection at the pixel associated with point 1202. When ray traced ray 1208 reaches the bounding box of the scene (not shown), ray traced ray 1208 is discarded and any color information for the pixel associated with point 1202 based on ray 1208. Can not even get.

13 is a flow diagram of method steps for rendering reflections in accordance with an embodiment. The method of FIG. 13 provides a hybrid ray propagation and ray tracing technique, where the ray propagation is first used to find possible intersections in screen space. Ray tracing is used when ray propagation fails.

In step 1302, the processor selects a pixel in screen space. The processor of FIG. 13 may be one or a combination of processor 110 and graphics processor 116 of FIG. 1. In one embodiment, screen space rasterization may be used to generate a 2D image of the scene in screen space. The method in FIG. 13 is used to add reflections to the pixels of the screen space image. The pixel selected in step 1302 may be any pixel of the screen space image.

In step 1304, the processor determines whether the pixel contains an object. In some cases, a pixel may include a background (eg, a sky) and thus not include any object of the scene. If at step 1304 the processor determines that the pixel does not contain an object, the reflections for the pixel are not calculated, the method proceeds to step 1306, where the processor proceeds to any further in screen space to process for reflections. It is determined whether there are more pixels. If yes, the method returns to step 1302, where another pixel is selected. In one implementation, each pixel is processed sequentially. However, in other implementations, each pixel of the image may be processed in parallel by, for example, a graphics processor (eg, a GPU).

If at step 1304 the processor determines that the pixel contains an object, the method proceeds to step 1308. In step 1308, the processor determines an illuminance value of the surface of the object in the pixel. In some embodiments, rasterizing a 3D scene into a 2D image results in certain information about an object associated with a pixel of the 2D image. Examples of such information include surface roughness and surface orientation. Surface roughness may be expressed as a roughness value. For example, the illuminance value may be a decimal value between 0.0 and 1.0.

In step 1310, the processor determines the number of light rays that occur for the pixel based on the illuminance value. As described above, only one ray is generated for smooth surfaces. More light rays can be generated for the rougher surfaces. In some implementations, the number of generated rays is limited to a threshold limit (eg four rays per frame).

In step 1312, the processor selects a beam direction for each beam generated. The ray direction of each ray is based on the surface orientation of the object in the pixel. For smooth surfaces (ie, one ray is generated), the direction of the ray can be determined based on simple mathematical reflections from the surface of the object.

When multiple rays are generated, the ray generation algorithm can determine the direction of each ray generated. In some implementations, a quasi-Monte Carlo technique may be used, ie deterministic random number generation for generated rays. For example, for a given frame, four rays are determined to be generated for the pixel based on the illuminance value. The quasi-Monte Carlo technique can be used to determine the directions for the four rays. Each of the four rays has a direction that falls within the reachable direction distribution based on the surface orientation.

 Then, in subsequent frames containing pixels corresponding to the same surface of the same object, the quasi-Monte Carlo technique is again used to determine the direction of the rays to occur, but selected directions for the rays in the previous frame (s). Select directions that do not overlap with. This process can be repeated for subsequent frames, thereby selecting different ray directions for different frames with the same surface of the same object shown. By selecting different ray directions in different frames, different reflection information is computed for different frames. Since the reflections may look different frame by frame for the same point, this results in jittering the reflections. As described in more detail below, different reflection information over several frames may be aggregated over a series of frames. In this way, less noise results can be realized because the light rays generated for the different frames have different directions (and hence other reflection information) that can be aggregated together to form the reflection information.

Once the ray direction is determined for one or more rays (step 1312), in step 1314, the processor performs ray propagation for each of the one or more rays. In one embodiment, the ray progression includes irradiating each ray pixel by pixel to search for intersections. In another embodiment, an optimized ray propagation method can be used as described in FIG. 11.

In step 1316, the processor determines whether the ray progression was success or failure for each ray. For a given ray, if ray progression is successful, at step 1318, the processor stores the color value information of the intersection where the ray found through the ray progression. If the ray propagation fails for a given ray, in step 1320 the processor performs ray tracing on the ray. As described, the ray propagation fails when the ray reaches the edge of the viewport without interacting with any object. Ray traced rays can begin at the viewport edge, because the processor knows there are no other intersections in the viewport (ie, the ray propagation failed).

In step 1322, for each ray traced ray, the processor determines whether the ray traced ray intersects an object in the 3D scene. As described, ray tracing can include retrieving intersections with all objects in the scene as well as objects within the viewport.

For each ray traced ray that does not intersect any object (eg, reaches the edge of the bounding box of the scene), the processor discards the ray at step 1324. Thus color information for reflections is not obtained for the light ray.

For each ray traced ray that intersects the object, at step 1326, the processor stores the color value information of the intersection where the ray found through the ray progression.

In step 1328, the processor aggregates the color values of the one or more rays that intersect the object in the scene. Color values can be obtained through ray propagation or ray tracing. In one embodiment, the color values are simply averaged together to determine the color value for reflection in the pixel.

After step 1328, the method proceeds to step 1306, where the processor determines whether there are more pixels in the screen space to process the reflection as described above.

In one embodiment, FIG. 13 is used to generate primary reflections for each pixel in the screen space where the object is located. In one embodiment, the secondary reflection can be generated using cube maps. In another embodiment, the secondary reflection can also be generated using the method of FIG. 13.

As noted above, embodiments of the present disclosure provide a system and method in which reflection is created by performing ray tracing if a ray progression fails to proceed. Another embodiment described below provides a system and method for reusing ray propagation results of adjacent pixels for a given pixel.

14A is a block diagram illustrating a 3D scene according to one embodiment. You can rasterize 3D scenes into 2D screen space. Three pixels of 2D screen space are shown for reference. Pixel 1402A corresponds to the first point on the surface of the object in the scene (ie, the object is a bottom), pixel 1402B corresponds to the second point on the surface of the object in the scene, and pixel 1402C is the scene Corresponds to the third point on the surface of the object in. Assume that the method described in FIG. 13 is performed to determine reflection information for the pixel 1402B. Further, based on the surface roughness of the object in pixel 1402B, assume that three light rays should be generated to determine the reflection information. Using the method shown in FIG. 13, three light beam directions can be determined, the light beams can proceed individually, and if the light propagation fails, the light beams are recast through light tracing, as described.

However, some embodiments of the present disclosure may avoid generating three new rays for new rays, for example pixel 1402B, and in some cases, adjacent pixels to determine reflection information. Information from can be reused. In such embodiments, if adjacent pixels (within the critical radius) have similar surface roughness (within the critical illuminance) and have similar surface orientations (within the critical orientation), then light propagation and / or from these adjacent pixels The ray traced information can be reused in determining reflective color information for a given pixel. In some embodiments, there may be an additional constraint that the object color information to be reused must be within the ray distribution reachable from a given pixel.

FIG. 14B is a block diagram illustrating a 3D scene in FIG. 14A, with some rays being reused from neighboring pixels according to one embodiment. FIG. As described, when attempting to determine reflection information for a given pixel (eg, pixel 1402B), some reflection information from adjacent pixels from previous frames may be reused. In the example shown, pixels 1402A and 1402C are within a critical radius of pixel 1402B. The critical radius may be configurable. In one implementation, the threshold radius is about 6-8 pixels from the center pixel to be irradiated. When computing reflection information for pixel 1402A (either in the previous frame or the current frame), the light beam was cast from pixel 1402A and identified as intersecting the object at point 1406A. This intersection can be found using ray propagation or ray tracing according to the method of FIG. 13. In the example shown in FIG. 14B, when the object of the intersection is in the viewport, the intersection was found through ray tracing. Similarly, when computing reflection information for pixel 1402C (either in the previous frame or the current frame), the ray was cast from pixel 1402C and identified as intersecting the object at point 1406C. As disclosed herein, instead of generating multiple rays of light from the pixel 1402B to determine reflection information, some embodiments may reuse color information from points 1406A and 1406C if certain conditions are met. have.

In some embodiments, the first condition is that a difference between the surface roughness of the object associated with the pixel (ie, 1402A or 1402C) that is a source of potentially reusable information and the surface roughness of the object associated with the pixel 1402B is within the threshold roughness. Is there. In some embodiments, the threshold illuminance is configurable.

In some embodiments, the second condition is that a difference between the surface orientation of the object associated with the pixel (ie, 1402A or 1402C) that is a source of potentially reusable information and the surface orientation of the object associated with the pixel 1402B is within the critical orientation. Is there. In some embodiments, the critical orientation is configurable.

In some embodiments, a third condition is that potentially reusable information must be reachable from pixel 1402B within a particular ray distribution 1408. In some embodiments, the set of potential ray directions of the primary reflected ray depends on the angle and surface roughness from the object in the pixel to the camera. An exemplary ray distribution 1408 is shown in FIG. 14B. In one implementation, ray distribution 1408 may be computed by the GGX shading model. A line may be drawn into an object associated with pixel 1402B from sources of potentially reusable information (ie, from points 1406A, 1406C). As shown in FIG. 14B, the lines for both sources of potentially reusable information (ie, from points 1406A, 1406C) are in ray distribution 1408, thus satisfying a third condition.

In various embodiments, one, two, or all of the first, second, and third conditions can be used to determine whether data can be reused in this manner.

In one embodiment, as described above, the reflection information determined by the reused neighboring pixels was computed in the previous frame. In other embodiments, the reused information may be from the same frame, but from the pixels for which the reflection information has already been computed.

15 is a flow diagram of method steps for reusing ray information of adjacent pixels in accordance with an embodiment. In step 1502, for the first pixel, the processor determines the number of rays to occur. The processor of FIG. 15 may be one or a combination of processor 110 and graphics processor 116 of FIG. 1. As mentioned above, in some embodiments, the number of rays generated is based on the surface roughness of the object associated with the first pixel. In step 1504, the processor identifies neighboring pixels within a threshold radius of the first pixel. In one embodiment, the critical radius is configurable. In one embodiment, each neighboring pixel within the critical radius is examined to find reusable reflection information. In another embodiment, neighboring pixels within the critical radius are examined until sufficient re-availability information is located, and no additional neighboring pixels are examined at this point.

In step 1506, for a given neighboring pixel, the processor determines whether the object associated with the neighboring pixel has a surface roughness similar to the object associated with the first pixel. Otherwise, reflection information from neighboring pixels is not reused. If yes, the method proceeds to step 1508.

In step 1508, for a given neighboring pixel, the processor determines whether the object associated with the neighboring pixel has a similar surface orientation as the object associated with the first pixel. Otherwise, reflection information from neighboring pixels is not reused. If yes, the method proceeds to step 1510.

In step 1510, for a given neighboring pixel, the processor determines whether potentially reusable information is reachable from the object associated with the first pixel in the ray distribution. Otherwise, reflection information from neighboring pixels is not reused. If yes, the method proceeds to step 1512.

In step 1512, the processor reuses reflection information from neighboring pixels.

Although steps 1506, 1508, 1510 are shown in a particular order, in other embodiments steps 1506, 1508, 1510 may be performed in any order. Also, in some embodiments, one or more of steps 1506, 1508, 1510 may be optional and may be omitted.

In one embodiment, in the first frame, reflections for all other pixels (or some set of pixels) of the image are generated using the method of FIG. 13. For example, if the image is represented in a black and white checkerboard pattern, the method of FIG. 13 applies only to the pixels marked with white checkers (ie, all other pixels). For pixels represented by black checkers, the ray reuse technique of FIG. 15 may be used to generate reflection information for these pixels. In the next frame, the operation is reversed, the method of FIG. 13 is performed to generate reflection information for the pixels represented by the black checkers and the ray reuse technique of FIG. 15 reflects the pixels for the pixels represented by the white checkers. It is done to generate the information. In this way, there is a full resolution of reflection information for a frame for each frame, but only half of many rays are needed.

In some implementations, the reflections can be added to each pixel of the image using the method of FIG. 13 and with or without the ray reuse of FIG. 15. In some cases, reflection results over the entire frame may exhibit noise, particularly when the scene includes glossy reflections. As such, some embodiments may smooth noise by performing temporal filtering.

16A is an example of reflection, according to one embodiment. A given pixel of screen space may correspond to point 1602 on the object. The reflected light rays may be generated from the pixels associated with point 1602 and may intersect other objects at point 1604. Color information from point 1604 can be used to add reflection information to the pixel associated with point 1602. The projected point 1606 may also be computed for the point 1604 of the virtual reflection area for the virtual position 1608A of the eye / camera for a given frame.

In a subsequent frame, as shown in FIG. 16B, the virtual position 1608B of the camera / eye has moved to another position. A given pixel of screen space in the updated virtual position 1608B of the camera / eye may correspond to point 1610 on the object. The reflected light rays may be generated from the pixels associated with point 1610 and may intersect other objects at point 1614. The projected point 1616 can also be computed relative to the point 1614 of the virtual reflection area for the updated virtual position 1608B of the eye / camera for a given frame. In FIG. 16B, a line from the projected point 1616 to the updated virtual position 1608B of the eye / camera passes through point 1610.

In some embodiments, the processor may be configured to determine whether point 1602 of the previous frame (FIG. 16A) has a surface roughness similar to point 1610 in the subsequent frame (FIG. 16B). If the surface roughness values of points 1602 and 1610 are within a threshold difference, the reflection results for the two frames can be blurred together, which is called temporal filtering. This is because both points 1602 and 1610 correspond to reflections representing the same portion of the object. Temporal filtering may be repeated for each pixel in frame units of the image. In some implementations the results more smoothly represent low noise reflections. However, in some cases, the processor may determine that there is a low certainty that the reflection results for the two frames may be blurred together. For example, if the surface roughness values of points 1602 and 1610 are different enough to exceed the threshold, the reflection results for the two frames cannot be blurred together. Also, if another object has moved in front of the original reflection intersection (eg, point 1604 of FIG. 16A), the reflection results for the two frames cannot be blurred together.

17 is a block diagram illustrating two points of subsequent frames having different surface roughness values, in accordance with an embodiment. Assume that the surface (ie, bottom) of the object 1700 has a checkerboard pattern with alternating regions of rough surfaces 1710 and smooth surfaces 1720. In the first frame, a line from point 1730 in the virtual reflection area to the first virtual position 1708A of the eye / camera passes through point 1702 corresponding to rough surface 1710. In the second frame, a line from point 1730 in the virtual reflection area to the second virtual position 1708B of the eye / camera passes through point 1704 corresponding to smooth surface 1720. In this case, the reflection results for the two frames cannot be blurred together because the surface roughness values of points 1702 and 1704 are not within the critical difference from each other.

In the examples shown in FIGS. 16A-16B, a single reflected ray is shown. Such an implementation may correspond to a smooth surface resulting in mirror reflection. On a rough surface where multiple rays are generated to produce glossy reflections, the average position of the intersection of the object by the multiple rays can be computed, and the process described above can be calculated by the multiple rays of the average position of the intersection of the object. Can be repeated on the basis of

18 is a flow diagram of method steps for performing temporal filtering according to an embodiment. For the first frame, at step 1802, the processor determines the location of the intersection of the object by the reflected ray or set of reflected rays. The processor of FIG. 18 may be one or a combination of processor 110 and graphics processor 116 of FIG. 1. In various embodiments, the location of the intersection can be a single location from a single reflected ray or can be an average location of the intersection based on multiple ray reflections intersecting the object. In other embodiments, multiple positions of the intersection may be computed, such as when multiple reflected rays intersect the object.

In step 1804, the processor determines the location of the intersection in the reflection area. In step 1806, the processor projects a line from the location of the intersection in the reflection area towards the location of the eye / camera. In step 1808, the processor determines the location of the intersection of the projected lines on the surface of the object for which reflection is shown.

For subsequent frames, the processor determines the location of the intersection of the object by the reflected ray or set of reflected rays (step 1810), determines the location of the intersection in the reflection area (step 1812), and positions the intersection in the reflection area. Project a line from the camera to the location of the eye / camera (step 1814), and determine the location of the intersection of the projected lines on the surface of the object for which reflection is shown (step 1816). Steps 1810, 1812, 1814, 1816 are similar for steps 1802, 1804, 1806, 1808, but for subsequent frames, respectively.

In step 1818, the processor determines whether the surface roughness of the position of the intersection of the projected lines on the surface of the object relative to the first frame is within a threshold difference in the surface roughness of the position of the intersection of the projected lines on the surface of the object relative to the next frame. Determine whether or not. If the surface roughness of the intersection positions of the projected line for the first frame and the subsequent frame are within a threshold difference, the reflection information can be blurred together to form smoother reflections.

In some embodiments, additional criteria are examined before determining that the reflection information can be blurred together. For example, the color information of the position of the intersection of the object by the reflected ray or the set of reflected rays from the first frame and the subsequent frame can be for example whether it is the same object being reflected in both frames or if the object is Can be compared to determine if the color has changed. The process described in FIG. 18 refers to a single location of the intersection. The process can be repeated for the locations of the various intersections.

As described above, some reflections may be mirror reflections and some reflections may be glossy reflections. Some glossy reflections may also be stretched reflections.

19 illustrates examples 1900 of elongated reflection. 20 is a block diagram illustrating elongated reflections, according to one embodiment. Rough surface 2000 includes a plurality of microfacets 2002 that result in glossy reflections. Based on the grazing angle of the eye / camera 2004 relative to the light source 2006 (or source of reflection information), the glossy reflection can be unfolded. The shape of the elongated reflection generally takes the form of an ellipse as shown in FIG. Reflection data in the ellipse may be blurred together by a blurring kernel having the shape and size of the ellipse to generate reflection information for the pixels in the ellipse. In some embodiments, the blurring kernel can set weights for how much a given piece of reflection information contributes to providing overall results. In some implementations, reflection information closer to the center of the ellipse may be weighted more heavily (ie, have a greater impact) than reflection information closer to the edge of the ellipse.

21 is an example of elongated reflection, according to one embodiment. The reflection seen at point 2102 from object 2104 may take the form of an ellipse 2106. Ellipse 2106 has long axis 2108 and short axis 2110. The lengths of the long axis 2108 and the short axis 2110 may be precomputed and stored in the lookup table based on the surface roughness and the reflection angle. In one implementation, the ellipse axis is for a unit length vector (ie, a length of 1). As such, the size of the ellipse can be scaled linearly based on average beam length and projection, as described below. FIG. 22 is a plot of a pre-computed length of a minor axis of an ellipse representing drawn reflections based on surface roughness and reflection angle, according to one embodiment. FIG. 23 is a plot of a pre-computed length of an axis of ellipse representing elongated reflections based on surface roughness and reflection angle, according to one embodiment.

24 is an example of applying an elliptical filter to glossy reflections according to one embodiment. As shown, the reflection at point 2402 takes an elliptic shape based on the surface roughness and the reflection angle. Reflection information of an adjacent object is shown as an ellipse 2404 on the object. The location for the reflection information in the reflection area can be computed and placed in the ellipse 2406. Ellipse 2406 is projected into screen space 2408 and scaled as ellipse 2410. Reflection information in ellipse 2410 in screen space may be blurred together using a blurring kernel to arrive at the final reflection information.

25 is a flow diagram of applying an elliptical filter in accordance with an embodiment. In step 2502, for a given image, the processor determines the surface roughness of the object shown in the image. The processor of FIG. 25 may be one or a combination of processor 110 and graphics processor 116 of FIG. 1. Surface roughness may indicate a rough surface that results in glossy reflections.

In step 2504, the processor determines a reflection angle between the camera / eye position and the reflected object. In step 2506, the processor performs a lookup on the table to determine the shape of the ellipse based on the surface roughness and the reflection angle. Ellipses have a long axis and a short axis.

In step 2508, the processor determines the location of the reflection information on the object that is being reflected. The position of the reflection information takes the form of an ellipse. In step 2510, the processor determines a location for the reflection information in the reflection area. The position of the reflection information in the reflection area also takes the form of an ellipse.

In step 2512, the processor projects and scales the position of the reflection information in the reflection area into screen space. In step 2514, the processor applies a blurring kernel to the scaled projection of the ellipse in screen space to arrive at the final reflection information. In some embodiments, the blurring kernel can set weights for how much a given piece of reflection information contributes to providing overall results. In some implementations, reflection information closer to the center of the ellipse may be weighted more heavily (ie, have a greater impact) than reflection information closer to the edge of the ellipse. In another embodiment, more rays may be selected at the center of the ellipse when selecting the light beam directions over the series of frames. As such, each ray may have the same weight but the weight is implicit because more rays are clustered in the center of the ellipse (so-called "important sampling").

Another embodiment may generate a plurality of points within an ellipse used during the blurring stage. Temporal filtering (FIG. 18) may be applied to each of these points to look up each of these points in the previous frame. The selected number of highest weighted points (eg 4-5 points) is used for blurring. This can improve the quality of time filtering.

All citations, including publications, patent applications, and patents cited herein are hereby incorporated by reference to the same extent as if each document was indicated to be cited individually and as a reference. It is described.

The use of the terms “a” and “an” and “the” and “at least one” and similar indicators in the context of describing the present invention (particularly in the context of the following claims) is otherwise indicated herein or expressly indicated by the context. Unless negated, it means both singular and plural. The use of the term "at least one" (eg, "at least one of A and B") followed by a list of one or more items is as listed unless otherwise indicated herein or explicitly disclaimed by context. One item (A or B) selected from or any combination of two or more items of the listed items (A and B). The terms "comprising", "having", "containing" and "comprising" are to be interpreted as open terms (ie, meaning "including but not limited to") unless otherwise specified. Enumeration of a range of values herein is intended to serve as a shorthand way of individually referring to each individual value within a range, unless otherwise indicated herein, each individual value being as individually cited herein. As incorporated herein.

All of the methods described herein may be performed in any suitable order unless otherwise indicated herein or otherwise explicitly negated in the context. The use of any and all embodiments or exemplary language (eg, “such as”) provided herein is intended to better represent the invention unless otherwise claimed and does not limit the scope of the invention. . No language in the specification should be construed as indicating an element which is not claimed as essential to the practice of the invention.

Preferred embodiments of the invention are described herein. Modifications of these preferred embodiments may become apparent to those skilled in the art upon reading the foregoing description. The inventors expect skilled artisans to employ such variations as appropriate, and the inventors intend for the invention to be practiced otherwise than as specifically described herein. Accordingly, the present invention includes all modifications and equivalents of the subject matter listed in the claims appended hereto as permitted by applicable law. In addition, any combination of the foregoing elements is included in the present invention in all possible variations, unless expressly contradicted by context or otherwise indicated herein.

Claims (23)

A computer program stored on a computer readable medium,
The computer program, when executed by one or more processors, causes the computer to:
Determining a roughness value of the surface of the object in the pixel;
Determining the number of rays occurring for the pixel based on the illuminance value;
For each ray in the number of rays, selecting a ray direction for the ray;
For each ray in the number of rays, performing ray marching of the ray based on the ray direction of the ray;
For each ray for which ray progression is successful, storing color information of the object intersected by the ray found through the ray progression;
For each ray for which ray propagation has failed, casting a ray through ray tracing and storing color information of the object intersected by the ray found through ray tracing; And
Generating reflection information for the pixel based on the color information of the objects crossed by the rays found through ray propagation and the color information of the objects crossed by the rays found through ray tracing.
And generate reflection information for the pixel in the image by means of a computer program. The method of claim 1,
The number of light rays is at least two light rays, and generating reflection information for the pixel includes aggregating the color information of the objects crossed by the at least two light rays. Computer program stored in. The method of claim 2,
Aggregating color information of the objects intersected by the at least two light rays includes computing an average color value of the color information of the objects intersected by the at least two light rays. Computer program stored on media. The method of claim 1,
Performing ray propagation for a first ray irradiates depth values on a pixel-by-pixel basis in the image in the direction of the first ray corresponding to the first ray to locate the intersection of the first ray with another object in the image. A computer program stored on a computer readable medium, the computer program storing the computer program. The method of claim 1,
Performing ray propagation with respect to the first ray,
Receiving data corresponding to depth values of objects in the scene;
Receiving the downsampled data corresponding to depth values of objects in the scene, the downsampled data comprising a coarser representation of data corresponding to depth values of objects in the scene. Receiving; And
Attempting to locate an intersection of the first ray and another object in the image based on the downsampled data. The method of claim 1,
And if the ray propagates against an edge of a viewport corresponding to the image without intersecting the ray with any other objects in the image, the ray propagation fails for the ray. The method of claim 1,
Casting the ray through the ray tracing comprises casting a ray originating at a point on a viewport corresponding to the image. The method of claim 1,
And the image is a 2D (two-dimensional) rasterized image of a 3D (three-dimensional) scene that includes a plurality of objects. A computer program stored on a computer readable medium,
When the computer program is executed by one or more processors, it causes the computer to:
Determining a first roughness value of the surface of the object at the first pixel;
Determining the number of rays occurring for the first pixel based on the first illuminance value;
Identifying a second pixel within a threshold radius of the first pixel in the image;
Determining a second roughness value of the surface of the object at the second pixel;
Determining that a difference between the first illuminance value and the second illuminance value is less than an illuminance threshold;
Determining color information of the object intersected by the reflected light beam corresponding to the second pixel; And
Generating reflection information for the first pixel based on the color information of the object intersected by the reflection ray corresponding to the second pixel
To generate reflection information for the first pixel in the image,
The first pixel is included in a first set of pixels and the second pixel is included in a second set of pixels,
Color information of objects intersected by reflected rays corresponding to pixels in the second set of pixels is determined via ray tracing or ray propagation,
Color information of the objects intersected by the reflected rays corresponding to the pixels in the first set of pixels is the color of the objects intersected by the reflected rays corresponding to the pixels in the second set of pixels. Determined based on information,
And the first set of pixels and the second set of pixels are arranged in a checkerboard pattern. The method of claim 9,
Running the computer program also causes the computer to:
Determine a first orientation value of the surface of the object at the first pixel;
Determine a second orientation value of the surface of the object at the second pixel;
Determine that the difference between the first and second orientation values is less than an orientation threshold,
Generating the reflection information for the first pixel based on the color information of the object intersected by the reflected light rays corresponding to the second pixel may result in a difference between the first orientation value and the second orientation value. A computer program stored on a computer readable medium based on being below an orientation threshold. The method of claim 10,
Running the computer program also causes the computer to:
Determine a location of the object intersected by the reflected light beam corresponding to the second pixel; And
Determine whether the position of the object intersected by the reflected light beam corresponding to the second pixel is reachable from the first pixel in the light beam distribution,
Generating the reflection information for the first pixel based on the color information of the object intersected by the reflection ray corresponding to the second pixel is crossed by the reflection ray corresponding to the second pixel. And based on determining that the location of the object is reachable from the first pixel within the ray distribution. The method of claim 11,
And wherein the shape of the ray distribution is based on the first illuminance value of the surface of the object at the first pixel. The method of claim 9,
Running the computer program also causes the computer to:
Determine a location of the object intersected by the reflected light beam corresponding to the second pixel; And
Determine whether the position of the object intersected by the reflected light beam corresponding to the second pixel is reachable from the first pixel in the light beam distribution,
Generating the reflection information for the first pixel based on the color information of the object intersected by the reflection ray corresponding to the second pixel is crossed by the reflection ray corresponding to the second pixel. And based on determining that the location of the object is reachable from the first pixel within the ray distribution. The method of claim 9,
And the reflected ray corresponding to the second pixel is ray propagated to intersect the object corresponding to the color information. The method of claim 9,
And the reflected light beam corresponding to the second pixel is ray traced to intersect the object corresponding to the color information. The method of claim 9,
And the first set of pixels and the second set of pixels are arranged in a checkerboard pattern. A computer program stored on a computer readable medium,
When the computer program is executed by one or more processors, it causes the computer to:
With respect to the first frame, determining the position of the intersection of the object based on the reflected light beam for the first pixel, wherein the position of the intersection of the object corresponds to the shape of the ellipse. ;
Determining, with respect to the first frame, a position in the reflective region of the intersection of the object based on the reflected light beam for the first pixel;
Projecting a first line relative to the first frame from a position in the reflective region of the intersection of the object with the reflected light beam relative to the first pixel towards the first position of the camera;
Determining, relative to the first frame, the location of the intersection of the first line on the surface of the first object;
For a second frame, determining the location of the intersection of the object based on the reflected light beam for the second pixel, wherein the second frame is subsequent to the first frame;
Determining, with respect to the second frame, a position in the reflective area of the intersection of the object based on the reflected light beam for the second pixel;
Projecting a second line relative to the second frame from a position in the reflective region of the intersection of the object by the reflected light beam relative to the second pixel towards the second position of the camera;
Determining, with respect to the second frame, the location of the intersection of the second line on the surface of the second object;
Determining that the surface roughness of the position of the intersection of the second line on the surface of the second object is within a critical roughness of the surface roughness of the position of the intersection of the first line on the surface of the first object; And
Generating reflection information for the second pixel based on the reflection information for the first pixel
Computer program, stored in a computer readable medium, for generating reflection information by means of a computer. The method of claim 17,
And the first object and the second object comprise the same object. The method of claim 17,
Generating the reflection information for the second pixel includes blurring the reflection information for the first pixel with the reflection information determined for the second pixel. The method of claim 17,
And the position of the intersection of the object based on the reflected light beam relative to the first pixel comprises an aggregated position based on a plurality of reflected light beams. The method of claim 17,
Running the computer program also causes the computer to:
Determine to generate a plurality of reflected light rays for the first pixel based on the surface roughness of the object in the first pixel; And
Perform a lookup in a database to determine the shape of the ellipse based on the surface roughness of the object at the first pixel and the first position of the camera,
And the position of the intersection of the object based on the plurality of reflected rays relative to the first pixel is based on the shape of the ellipse. The method of claim 21,
Performing a lookup on the database comprises acquiring data for the long axis of the ellipse and acquiring data for the short axis of the ellipse. The method of claim 17,
An image is a computer program stored on a computer readable medium, wherein the image is a 2D (two-dimensional) rasterized image of a 3D (three-dimensional) scene including a plurality of objects.

Images