KR102558739B1 - 3d graphic rendering method and apparatus - Google Patents

Abstract

A 3D rendering method and a 3D rendering apparatus for rendering a 3D scene are disclosed. A 3D rendering apparatus according to an embodiment may determine shading points on a 3D scene where shading is to be performed, and perform shading on the shading points. The 3D rendering device may determine shading information for a 3D scene based on a shading result of a shading point.

Description

3D rendering method and apparatus {3D GRAPHIC RENDERING METHOD AND APPARATUS}

The description below relates to an image processing technique for performing 3D rendering.

In 3D computer graphics, a graphics pipeline or a rendering pipeline represents a step-by-step method for expressing a 3D image as a 2D raster image. Here, a raster is a method of expressing image information in a computer, and indicates that an image is composed of pixels in a two-dimensional array and image information is expressed with pixels at regular intervals. The graphics pipeline includes a vertex shader that provides a special effect of a 3D object by performing mathematical operations on vertex information of the 3D object and a pixel shader that calculates the color of each pixel. The vertex shader may move a 3D object to a special position based on vertex information, change a texture, change a color, or the like. Pixel shaders can read colors from textures, apply light, or apply complex phenomena such as shadows, reflected light, and transparency.

A 3D rendering method according to an embodiment includes determining a shading point on a 3D scene where shading is to be performed; performing shading on the shading points; and determining shading information for the 3D scene based on a shading result of the shading point.

In the 3D rendering method according to an embodiment, the determining of the shading point may include determining some of vertices of a 3D object in the 3D scene as the shading point.

In the 3D rendering method according to an embodiment, in the determining of the shading point, vertices of a 3D object in the 3D scene and additional points within the 3D object may be determined as the shading point.

In the 3D rendering method according to an embodiment, the determining of the shading point may include determining some of vertices of a 3D object in the 3D scene and an additional point in the 3D object as the shading point.

In the 3D rendering method according to an embodiment, in the determining of the shading point, a point in the 3D scene that is not a location of vertices of a 3D object may be determined as the shading point.

In the 3D rendering method according to an embodiment, the determining of the shading information may include determining shading values of vertices adjacent to the shading points by interpolating shading values of a plurality of shading points.

In the 3D rendering method according to an embodiment, the determining of the shading point may include determining a shading point on which the shading is to be performed based on at least one of spatial characteristics and temporal characteristics of the 3D scene.

In the 3D rendering method according to an embodiment, the determining of the shading point may include grouping vertices of the 3D scene into vertex groups; and determining one or more shading points for each vertex group.

In the 3D rendering method according to an embodiment, the determining of the shading point may further include determining the shading level based on at least one of a motion of a virtual light source, a motion of a virtual camera, a motion of a 3D object, a brightness difference between adjacent vertices, and a vertex position.

A 3D rendering device according to an embodiment includes at least one processor; and at least one memory storing instructions to be executed by the processor, wherein the instructions, when executed by the processor, cause the processor to perform an operation of determining a shading point on a 3D scene where shading is to be performed; performing shading on the shading point; and determining shading information for the 3D scene based on a shading result of the shading point.

1 is a diagram showing an example of a 3D scene and a vertex structure of the 3D scene.
2 is a flowchart illustrating an operation of a 3D rendering method according to an exemplary embodiment.
3 is a flowchart illustrating an operation of determining a shading point according to an exemplary embodiment in more detail.
4 is a diagram for explaining an example of determining a shading point according to an exemplary embodiment.
5 is a diagram for explaining an example of determining a shading point according to a shading level according to an exemplary embodiment.
6 is a diagram for explaining an example of determining a shading value of another vertex based on a shading result of a shading point according to an exemplary embodiment.
7 is a diagram for explaining an example of determining a pixel value of a rendered image based on shading values of shading points according to an exemplary embodiment.
8A and 8B are diagrams for explaining a relationship between a direct virtual light source and an indirect virtual light source according to an exemplary embodiment.
9 is a diagram illustrating a configuration of a 3D rendering device according to an embodiment.
10 is a diagram showing the configuration of a 3D rendering device according to another embodiment.

Specific structural or functional descriptions below are merely exemplified for the purpose of describing the embodiments, and the scope of the patent application should not be construed as being limited to the content described herein. Various modifications and variations can be made from these descriptions by those skilled in the art. Reference herein to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment, and references to “one embodiment” or “an embodiment” are not to be construed as all referring to the same embodiment.

Terms such as first or second may be used to distinguish various elements, but elements should not be construed as being limited by the first or second term. In addition, terms used in the examples are only used to describe a specific example, and are not intended to limit the example. Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this specification, terms such as "comprise" or "having" are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but it should be understood that the presence or addition of one or more other features or numbers, steps, operations, components, parts, or combinations thereof is not excluded in advance.

Embodiments to be described below may be applied to generating a rendering result image by rendering a 3D scene. An operation of rendering a 3D scene includes a shading process of determining a color of a 3D object based on light emitted from a virtual light source providing an illumination effect to the 3D scene.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted.

1 is a diagram showing an example of a 3D scene and a vertex structure of the 3D scene. Referring to FIG. 1 , reference numeral 110 denotes a 3D scene to be rendered, and reference numeral 140 denotes a vertex structure of the 3D scene 110 .

In vertex-based shading, after shading is performed on vertices of a 3D object, color values of pixels are determined by interpolating shading values of the vertices. The complexity of the vertex structure is due to the geometric complexity of each region, and the shading complexity may be due to the complexity of shading processing or the application of special effects such as specular reflection and diffused reflection. Therefore, in vertex-based shading, the geometric complexity of each region of the 3D scene 110 may not match the shading complexity. For example, when shading is performed based on vertices, a deterioration in image quality may occur in an area 120 having a small number of vertices because shading complexity is high but geometric complexity is low. On the other hand, waste of shading calculations may occur in the region 130 having a large number of vertices although the shading complexity is low but the geometric complexity is high.

The 3D rendering method and apparatus to be described below can adaptively determine shading points on which shading is to be performed based on spatial or temporal characteristics of the 3D scene 110 and perform shading on the determined shading points, thereby performing shading more quickly without deteriorating the quality of the resulting image. For example, the 3D rendering method and apparatus may reduce the number of shading processes by reducing the density of shading points in an area where the number of vertices is greater than the shading complexity, such as area 130, and accordingly, the rendering process speed may be increased.

2 is a flowchart illustrating an operation of a 3D rendering method according to an exemplary embodiment. The 3D rendering method may be performed by a 3D rendering device (eg, the 3D rendering device shown in FIGS. 9 to 10).

Referring to FIG. 2 , in step 210, the 3D rendering device determines a shading point on a 3D scene where shading is to be performed. The 3D rendering device may determine any point in the 3D scene as a shading point. For example, the 3D rendering apparatus may determine not only points where vertices of a 3D object included in a 3D scene are located but also points where vertices are not located as shading points where shading is to be performed. The shading point may be determined based on one or more of spatial and temporal characteristics of the 3D scene. A point where shading is performed may vary according to spatial or temporal characteristics of a 3D scene.

For example, the 3D rendering apparatus may determine a shading point based on information about a virtual light source providing a lighting effect, information about a virtual camera determining a viewing point of a 3D object, information about a 3D object appearing in a 3D scene, or shading information of a previous image frame. Here, the information on the virtual light source may include information about the position, color, brightness, direction, moving speed of the virtual light source, a distance between the virtual light source and the 3D object, or an angle between the virtual light source and the 3D object. Information about the virtual camera may include information about the position, direction, and moving speed of the virtual camera, or an angle between the virtual camera and the 3D object. Information on the 3D object may include information on the shape, color, or material of the 3D object. The shading information of the previous image frame may include shading values for vertices used in the previous image frame.

According to an embodiment, the 3D rendering apparatus may group vertices into a plurality of vertex groups and determine shading points on which shading is to be performed for each vertex group. According to an embodiment, if it is determined that fine shading is necessary, the 3D rendering device may designate additional points in the 3D scene as shading points in addition to the vertices included in the vertex group to increase the number of shading points. If it is determined that coarse shading is necessary, the 3D rendering apparatus determines only some of the vertices included in the vertex group as shading points or determines one or more points in the 3D scene regardless of vertices as shading points. Alternatively, in some cases, the 3D rendering apparatus may determine a point that is not a vertex in the 3D scene together with some of vertices of a 3D object in the 3D scene as a shading point. The 3D rendering apparatus may perform shading more efficiently by determining a shading point based on spatial or temporal characteristics of the 3D scene instead of performing shading on all vertices or pixels of 3D objects included in the 3D scene.

An operation of determining the shading point by the 3D rendering device will be described in more detail below with reference to FIG. 3 .

In step 220, the 3D rendering device performs shading on the shading points determined in step 210. The 3D rendering device may perform shading at the shading point to determine a color value that is a shading value of the shading point. The shading process may be based on an illumination effect by one or more virtual light sources. Here, the lighting effect is based on characteristics (eg, color and direction) of light emitted from the virtual light source and characteristics (eg, color and material) of the 3D object, and may include a shadow effect by occlusion. The virtual light source may include a direct virtual light source that directly emits light to the 3D object and an indirect light source that emits light in a region where light emitted from the direct virtual light source is reflected, diffracted, or refracted. The direct virtual light source and the indirect virtual light source will be described in more detail below with reference to FIGS. 8A and 8B .

In step 230, the 3D rendering device determines shading information for the 3D scene based on the shading result in step 220. A 3D rendering apparatus may use interpolation to determine overall shading information of a 3D scene. According to an embodiment, the 3D rendering apparatus may determine a shading value of a vertex adjacent to a shading point by interpolating shading values of a plurality of shading points. Here, a barycentric interpolation technique that performs interpolation using the distance between a vertex and each shading point as a weight may be used, but the scope of the embodiment is not limited to the above interpolation technique, and various interpolation techniques may be used.

The 3D rendering device may store shading values of shading points on which shading is performed and shading values of vertices determined through interpolation. Shading values of shading points and shading values of vertices may be stored, for example, in a texture buffer of a graphics processing unit (GPU), and the stored shading values may be used in a shading process of a next image frame. In addition, information about positions and attributes of shading points and vertices may be stored in a texture buffer. By using the stored shading values, the 3D rendering apparatus can reduce the amount of computation required for shading calculations and can reduce the occurrence of a flickering phenomenon.

According to an embodiment, the 3D rendering apparatus may adaptively update shading values of vertices in successive image frames of a 3D scene. Accordingly, shading values of vertices corresponding to each other between image frames may be updated at different time intervals. The process of updating the shading value includes a process of determining the color value of the vertex by performing shading. For example, the 3D rendering device may update color values of vertices that are close to the screen by performing shading every image frame, and update color values of vertices that are far from the screen or existing in a shadow area at intervals of a specific number of image frames instead of updating color values every image frame.

The 3D rendering device may generate a rendering result image based on shading information of the 3D scene. The 3D rendering apparatus may determine color values of pixels constituting a rendering result image based on shading values of shading points and shading values of vertices determined through interpolation. For example, the 3D rendering device may determine a pixel value of a current pixel by interpolating shading values of a shading point and a vertex adjacent to the current pixel. Such an interpolation process may be repeatedly performed for each pixel constituting the rendering result image, and a rendering result image may be generated as a result of the repetition.

3 is a flowchart illustrating an operation of determining a shading point according to an exemplary embodiment in more detail.

Referring to FIG. 3 , in step 310, the 3D rendering device groups vertices of a 3D scene into a plurality of vertex groups. The 3D rendering apparatus may group vertices based on vertex positions, vertex properties (eg, normal, color, etc.), a shading result of a previous image frame, or whether vertices are included in the same 3D object. For example, the 3D rendering device may set vertices estimated to have similar properties among vertices as one vertex group.

According to an embodiment, the 3D rendering device may group vertices that are located adjacent to each other and have similar properties, such as normal, depth, or color. The 3D rendering apparatus may determine vertices estimated to have similar properties based on a preliminary prediction or a shading result of a previous image frame. For example, the 3D rendering device may group vertices having similar colors in the previous image frame.

According to another embodiment, the 3D rendering device may group vertices constituting the same 3D object or vertices having similar shape characteristics. According to another embodiment, the 3D rendering apparatus may group vertices that are spaced apart but whose colors similarly change over time based on color changes of the vertices over time.

The 3D rendering device may determine hierarchical structure information about shading points for each vertex group. The hierarchical structure information defines a hierarchical structure of shading points on which shading is to be performed according to the shading level determined in step 320 . For example, in the hierarchical structure information, which vertex or which point is to be determined as a shading point may be defined according to a shading level. As the shading level increases, the number of shading points determined in the vertex group decreases, and as the shading level decreases, the number of shading points determined in the vertex group may increase.

In step 320, the 3D rendering device determines a shading level for each vertex group based on hierarchical structure information about shading points. One or more shading points on which shading is to be performed in a vertex group may be determined according to the shading level. In this case, some of the vertices included in the vertex group may be determined as shading points, or other points in the 3D scene other than the points at which the vertices are located may be determined as shading points.

The 3D rendering apparatus may determine the shading level based on temporal characteristics of the 3D scene, such as movements of a virtual light source, virtual camera, and 3D object over time, or spatial characteristics of the 3D scene, such as a vertex position and a brightness difference between adjacent vertices. Different shading levels may be determined for each vertex group according to temporal characteristics or spatial characteristics of the 3D scene.

According to an embodiment, when a moving speed of a virtual light source, a virtual camera, or a 3D object in the temporal domain is faster than a threshold value, the 3D rendering device may reduce the number of shading points by increasing the shading level. Through this, the time required for shading processing may be reduced. Also, the 3D rendering device may increase the overall shading level when fast rendering processing is required. When the moving speed of the virtual light source, virtual camera, or 3D object is slower than the threshold value, the 3D rendering apparatus may increase the number of shading points by lowering a shading level for more detailed expression.

According to an embodiment, the shading level may be adaptively determined according to temporal characteristics of a 3D scene. For example, the shading level may be adjusted upward as the moving speed of the virtual camera or the 3D object increases, and the shading level may be adjusted downward as the moving speed of the virtual camera or 3D object decreases.

According to an embodiment, the 3D rendering apparatus estimates a change in brightness for each area over time using color information of a previous image frame, adjusts the shading level down in an area where the brightness change is large, and adjusts the shading level up in an area where the brightness change is small.

According to an embodiment, when a vertex is located in a central region (or region of interest) of a screen in a spatial domain or an region in which a brightness difference between adjacent vertices is large, the 3D rendering device may adjust the shading level downward for more detailed expression. When a vertex is located in a peripheral area of the screen or an area with a small difference in brightness between adjacent vertices, the 3D rendering device may increase the shading level in order to speed up the rendering process by reducing the number of shading points. Here, the 3D rendering apparatus may estimate the brightness difference between adjacent vertices in the current image frame using shading information of the previous image frame.

According to an embodiment, the 3D rendering apparatus may set different shading levels between a focused area and the remaining areas, or set different shading levels between an area to which a blurring effect is applied and the remaining areas. For example, in a device using a lens such as a head mounted display (HMD), the size and density of pixels visible to the user may be different. In this case, the 3D rendering device may set a relatively high density of shading points (decrease the shading level) in the central area to be focused, and set a relatively low density of shading points (adjust the shading level upward) in the outer area.

According to an embodiment, the 3D rendering apparatus may determine an optimal shading level for each vertex group from the highest shading level, which is the most approximate shading level, to the lower shading level. For example, the 3D rendering apparatus may compare a shading result according to a first shading level at the top level with a shading result according to a second shading level lower than the first shading level based on hierarchical structure information about shading points, and calculate a difference in color values according to a difference in shading level based on the comparison result. Since the second shading level is lower than the first shading level, more shading points are determined in the second shading level than in the first shading level. When the difference between the calculated color values is smaller than the threshold value, the 3D rendering device may determine the first shading level, which is the current shading level, as the shading level of the current vertex group. When the difference between color values is greater than or equal to a threshold value, the 3D rendering apparatus may calculate a difference between color values according to the second shading level and the third shading level based on a third shading level lower than the second shading level, and determine whether the calculated difference is smaller than the threshold value. When the difference is smaller than the threshold value, the 3D rendering device may determine the second shading level as the shading level of the current vertex group. The 3D rendering device may repeatedly perform the above process to determine a shading level at which a difference between color values is smaller than a threshold value as a final shading level applied to the current vertex group.

An operation in which the 3D rendering device determines the optimal shading level to be applied to the vertex group, starting from the highest shading level, may be performed basically every time an image frame is rendered, or may be performed when the camera viewpoint, position of a light source, or position of an object changes. However, the scope of the embodiment is not limited thereto, and when the 3D rendering apparatus determines that the change in the camera viewpoint, the position of the light source, or the position of the object between successive image frames is not large, the process of determining the shading level in the current image frame may be omitted and the shading level determined in the previous image frame may be used.

In step 330, the 3D rendering device determines one or more shading points for each vertex group according to the shading level. The 3D rendering apparatus may determine a shading point on which a shading operation is actually performed according to a shading level based on hierarchical structure information for each vertex group.

According to an embodiment, a shading point may be determined according to one of the following (1) to (6) for each vertex group based on the shading level. (1) All vertices belonging to the current vertex group are determined as shading points (2) All vertices and additional non-vertex points are determined as shading points (3) Some vertices among all vertices belonging to the current vertex group are determined as shading points (4) Some vertices and additional non-vertex points are determined as shading points (5) One or more non-vertex points are determined as shading points (6) Shading for the current vertex group Points not determined. For example, in an entire 3D scene, all vertices belonging to a vertex group may be determined as shading points in an area corresponding to one vertex group according to (1), and one or more points other than vertices belonging to another vertex group may be determined as shading points in an area corresponding to another vertex group according to (5).

4 is a diagram for explaining an example of determining a shading point according to an exemplary embodiment. Referring to FIG. 4 , reference numeral 410 denotes a 3D object including a plurality of vertices 422 to 434, and reference numeral 440 denotes shading points 422, 424, 430, 434, 450, 452, 454, and 456 determined for the 3D object 410.

In vertex-based shading, shading is performed on each of the vertices 422 to 434 of the 3D object 410, but the 3D rendering device can adaptively determine the point where the shading operation is to be performed based on the temporal or spatial characteristics of the 3D scene. For example, as in the embodiment shown in FIG. 4 , the 3D rendering device may determine some of the vertices 422, 424, 430, and 434 of the vertices 422 to 434 of the 3D object 410 and points 450, 452, 454, and 456 that require shading as shading points to perform shading. After performing shading on the shading points 422, 424, 430, 434, 450, 452, 454, and 456, the 3D rendering device may determine shading values of vertices 426, 428, and 432 on which shading is not performed based on the shading result. For example, the 3D rendering device may determine the shading value of the vertex 432 by interpolating the shading values of the shading points 430 , 452 , and 454 .

5 is a diagram for explaining an example of determining a shading point according to a shading level according to an exemplary embodiment.

Referring to FIG. 5 , reference numeral 510 denotes a 3D object composed of a plurality of vertices 515 . The 3D rendering apparatus may generate hierarchical structure information for the vertices 515 of the 3D object 510 and determine shading points according to shading levels based on the hierarchical structure information. A shading point where shading is to be performed in the area of the 3D object 510 may be determined according to the shading level. As the shading level decreases, the number of shading points on which shading is performed may increase.

According to an embodiment, in the first shading level 520 , only some of the vertices 522 among the vertices 515 of the 3D object 510 may be determined as shading points. Accordingly, vertices 522 determined as shading points and vertices 525 not determined as shading points may exist in the first shading level 520 . The 3D rendering device may determine shading values by performing shading on vertices 522 determined as shading points, and interpolating shading values of vertices 522 to determine shading values of vertices 525 not determined as shading points. The 3D rendering apparatus may perform rendering more quickly by performing shading operations on only some vertices 522 instead of all vertices.

According to an embodiment, the 3D rendering apparatus may perform Delaunay triangulation on shading points for which shading values are determined to divide an area into a plurality of sub-regions having a triangular shape, and determine shading values of vertices for which shading values are not determined through interpolation based on the shading values of the shading points. According to another embodiment, the 3D rendering apparatus may determine shading points on an atlas on which a mesh of a 3D object is spread, perform shading on the shading points, and then determine shading values of vertices whose shading values have not been determined through interpolation based on shading values of the shading points.

In the second shading level 530, not all vertices 515 of the 3D object 510 are determined as shading points, but points 535 that are not vertices may be determined as shading points instead. The 3D rendering device may perform shading on points 535 determined as shading points, and determine shading values of vertices 532 that are not determined as shading points through interpolation of shading result values.

In the third shading level 540, similarly to the first shading level 520, among the vertices 515 of the 3D object 510, vertices 542 determined as shading points and vertices 545 not determined as shading points may exist. Also, points 546 that are not vertices within the area of the 3D object 510 may be additionally determined as shading points. The positions of the points 546 may be determined, for example, as random positions within the area of the 3D object 510 or based on probability values according to distribution of brightness values within the area of the 3D object 510 . As the brightness value increases, a probability value determined as a point to be determined as a shading point may increase. The 3D rendering device may perform shading on the vertices 542 and 546 determined as shading points, and interpolate the shading result values to determine shading values of vertices 545 not determined as shading points.

In the fourth shading level 550, all vertices 515 of the 3D object 510 are determined as shading points, and shading may be performed on each vertex 515.

In the fifth shading level 560, points 565 other than the vertices 515 of the 3D object 510 on the 3D object 510 as well as the vertices 515 of the 3D object 510 may be additionally determined as shading points. The 3D rendering apparatus may determine additional shading points 565 based on temporal or spatial characteristics of a 3D scene in which the 3D object 510 is expressed. For example, when it is necessary to implement a more detailed shading effect (for example, when a lighting effect is complex), the 3D rendering device may additionally designate points 565 where shading is to be performed within the area of the 3D object 510. The 3D rendering device may perform shading on the vertices 515 and additionally determined points 565 .

6 is a diagram for explaining an example of determining a shading value of another vertex based on a shading result of a shading point according to an exemplary embodiment.

In the embodiment shown in FIG. 6 , it is assumed that the 3D object 610 includes vertices 612 to 626 . In addition, it is assumed that the 3D rendering device determines (630) some of the vertices 614, 616, 622, 624 and points 632, 634, and 636 on the 3D object 610 as shading points, among the vertices 612 to 626. The 3D rendering device performs shading on the shading points 614, 616, 622, 624, 632, 634, and 636, and performs shading based on shading values (eg, color values) of the shading points 614, 616, 622, 624, 632, 634, and 636 determined by performing the shading. Shading values of the vertices 612, 618, 620, and 626 that are not performed may be determined. For example, in the case of the vertex 626, the 3D rendering device may interpolate the shading values of the shading points 622, 624, 634, and 636 adjacent to the vertex 626 to determine the shading value of the vertex 626 (640). The 3D rendering device may determine shading values for the remaining vertices 612, 618, and 620 on which shading is not performed through a similar process. The shading values of the shading points 614, 616, 622, 624, 632, 634, and 636 and the shading values of the vertices 612, 618, 620, and 626 determined through interpolation are stored, and the stored shading values can be used in a shading process for the next image frame.

7 is a diagram for explaining an example of determining a pixel value of a rendered image based on shading values of shading points according to an exemplary embodiment.

In the embodiment of FIG. 7 , it is assumed that shading values are determined 710 at vertices 715 , 720 , and 725 and points 730 and 735 on the 3D object. The 3D rendering apparatus may determine color values of pixels by performing color interpolation for each pixel based on the shading values of the vertices 715 , 720 , and 725 and the shading values of the points 730 and 735 . For example, the 3D rendering device may determine the color value of the pixel 720 by interpolating a shading value of a vertex 725 adjacent to the pixel 720 and a color value of points 730 and 735 . The 3D rendering apparatus may repeatedly perform a similar process for the remaining pixels to determine color values of pixels constituting the rendering result image.

8A and 8B are diagrams for explaining a relationship between a direct virtual light source and an indirect virtual light source according to an exemplary embodiment. The direct virtual light source and the indirect virtual light source represent virtual light sources that give lighting effects to 3D objects in computer graphics. As described above, the direct virtual light source is a virtual light source that directly emits light to a 3D object, and the indirect virtual light source is a virtual light source that emits light in a region where light emitted from the direct virtual light source is reflected, diffracted, or refracted.

FIG. 8A shows 3D objects 820 and 830 constituting a 3D model and a direct virtual light source 810 . Here, only one direct virtual light source 810 is shown for convenience of description, but a plurality of direct virtual light sources may exist in the 3D model. The direct virtual light source 810 directly emits light of the 3D object 820 . A bright area and a dark area may be primarily determined in a virtual space in which the 3D object 820 is rendered by a positional relationship between the direct virtual light source 810 and the 3D object 820 . Light directly emitted from the virtual light source 810 may be reflected, refracted, or diffracted by the 3D object 820 . For example, the light 840 directly output from the virtual light source 810 may be reflected on the 3D object 820 and then reflected again on another 3D object 830, for example, a wall surface surrounding the area around the object 820. The 3D objects 820 and 830 are rendered from the viewpoint of the virtual camera 815, and a rendering result image may be provided to the user.

The 3D rendering apparatus may generate a detailed rendering result image by applying a direct light effect by light output from the direct virtual light source 810 as well as a lighting effect by light output from an indirect virtual light source. The 3D rendering apparatus may implement a more realistic lighting effect by appropriately arranging direct virtual light sources as well as indirect virtual light sources in a 3D space.

Referring to FIG. 8B , an indirect virtual light source 855 located in an area where light 840 directly output from a virtual light source 810 in FIG. 8A is reflected by a 3D object 820 and indirect virtual light sources 850 and 860 located in areas where light 840 is reflected by another 3D object 830 are illustrated. During the rendering process of the 3D objects 820 and 830 , lighting effects by not only the direct virtual light source 810 but also the indirect virtual light sources 850 , 855 and 860 may be applied to the 3D objects 820 and 830 . The indirect virtual light sources 850 , 855 , and 860 are affected not only by the direct virtual light source 810 but also by characteristics of regions where the indirect virtual light sources 850 , 855 , and 860 are located. For example, the lighting effect of the indirect virtual light sources 850 , 855 , and 860 may be affected by the color or material of the surface of the 3D object on which the indirect virtual light sources 850 , 855 , and 860 are located.

9 is a diagram illustrating a configuration of a 3D rendering device according to an embodiment. Referring to FIG. 9 , a 3D rendering device 900 includes a vertex shader 910, a pixel shader 920, and a shading point shader 930.

The vertex shader 910 may perform vertex transform on vertices displayed in a 3D scene based on vertex attribute information such as spatial position, color, normal vector, and texture of the vertex. The process of transforming a vertex may include one or more of a process of moving a vertex position, a process of transforming a normal vector of a vertex, a process of calculating a light effect on a vertex, a process of calculating a color of a vertex, and a process of generating and transforming texture coordinates.

The shading point shader 930 may perform operations related to the shading process among the operations described above with reference to FIGS. 1 to 8B. For example, the shading point shader 930 may group vertices appearing in a 3D scene into a plurality of vertex groups and determine a shading level for each vertex group. The shading point shader 930 may determine shading points to perform shading for each vertex group according to the determined shading level, and perform shading on the determined shading points. The shading point shader 930 may interpolate shading values of shading points on which shading is performed to determine shading values of vertices where shading is not performed, thereby determining shading information for a 3D image.

The shading point shader 930 may perform color shading in units of vertices, and such color shading may be processed separately from each rendering process of the vertex shader 910 and the pixel shader 920 . According to an embodiment, the shading process performed by the shading point shader 930 may be performed as a separate process from pipelines of the vertex shader 910 and the pixel shader 920. Accordingly, shading values for shading points and vertices may be stored. The stored shading values may be used to determine shading information of the next image frame, and through this, correlation between image frames may be increased, thereby reducing a flickering phenomenon and reducing an amount of computation.

According to an embodiment, the shading point shader 930 may determine a shading level for each vertex group based on a shading result. The shading point shader 930 may determine an optimal shading level to be applied to each vertex group based on a difference between color values according to shading levels. For example, the shading point shader 930 compares a result of performing shading based on the current shading level with a result of performing shading based on a shading level lower than the current shading level, calculates a difference in color values according to a level difference, and adjusts the current shading level when the difference is greater than or equal to a threshold value. The shading point shader 930 repeatedly performs the above process based on the adjusted shading level, and determines a shading level at which a difference in color values according to a difference in shading level is smaller than a threshold value as an optimal shading level to be applied to a vertex group.

When shading information for the 3D scene is determined, the shading point shader 930 may transmit the shading information to the pixel shader 920, and the pixel shader 920 may perform color interpolation for each pixel based on the shading information. The pixel shader 920 may perform color interpolation using color values of vertices constituting a polygon (e.g., a triangle) to which a pixel belongs and a color value of a shading point where shading is performed, thereby determining the color value of each pixel included in the rendering result image. In addition, the pixel shader 920 may express the texture of the virtual 3D object by performing a texture mapping process of applying a texture on the 3D object.

According to another embodiment, the 3D rendering device 900 may further include an indirect virtual light source sampler 940 that samples an indirect virtual light source. The shading point shader 930 may perform shading using not only lighting information from a direct virtual light source but also lighting information from an indirect virtual light source, and the corresponding indirect virtual light source may be sampled by the indirect virtual light source sampler 940. The process of sampling the indirect virtual light source may include a process of arranging the indirect virtual light source in a 3D space. The indirect virtual light source sampler 940 may control an indirect lighting effect applied to a 3D object by adjusting the number of indirect virtual light sources disposed in a region where the indirect virtual light sources are disposed or in a 3D space.

According to an embodiment, the indirect virtual light source sampler 940 may sample an indirect virtual light source from an image resulting from light view rendering or sample an indirect virtual light source by using a ray tracing technique for light. For example, the indirect virtual light source sampler 940 may generate a probability map proportional to brightness based on a brightness distribution of an image resulting from light viewpoint rendering, and may arrange an indirect virtual light source at a position determined through the probability map. As another example, the indirect virtual light source sampler 940 may determine a location where light collides with a 3D object through a ray tracing technique, and may place an indirect virtual light source at the determined location.

10 is a diagram showing the configuration of a 3D rendering device according to another embodiment. Referring to FIG. 10 , an image processing device 1010 includes one or more processors 1020 and one or more memories 1030 .

The processor 1020 performs one or more operations described above through FIGS. 1 to 9 . For example, the processor 1020 may determine a shading point on which shading is to be performed on the 3D scene, and perform shading on the determined shading point. Here, the processor 1020 may group vertices of the 3D scene into vertex groups and determine a shading point based on a shading level determined for each vertex group. The processor 1020 may determine shading information for a 3D scene based on a shading result of a shading point, and generate a rendering result image based on the shading information.

Such a processor 1020 may be implemented as an array of logic gates, but it may be understood by those skilled in the art that the processor 1020 may be implemented as other types of hardware. Also, the processor 1020 may include one or more graphics processing units for rendering 3D objects. Shading information for vertices and shading points may be stored in a texture buffer of a graphics processing unit, and the stored shading information may be used in a shading process for a next image frame.

The memory 1030 may store instructions for performing one or more operations described above through FIGS. 1 to 9 or may store data and results obtained while the 3D rendering device 1010 is operated. In some embodiments, memory 1030 may include non-transitory computer-readable media, such as high-speed random access memory and/or non-volatile computer-readable storage media (e.g., one or more disk storage devices, flash memory devices, or other non-volatile solid state memory devices).

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general purpose or special purpose computers, such as, for example, a processor, controller, arithmetic logic unit (ALU), digital signal processor, microcomputer, field programmable gate array (FPGA), programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will recognize that the processing device may include a plurality of processing elements and/or multiple types of processing elements. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired, or may independently or collectively direct a processing device. Software and/or data may be permanently or temporarily embodied in any tangible machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, to be interpreted by, or to provide instructions or data to, a processing device. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. Computer readable media may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on a computer readable medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions such as ROM, RAM, and flash memory. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, even if the described techniques are performed in an order different from the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or replaced or substituted by other components or equivalents, appropriate results can be achieved.

Claims (25)

In the 3D rendering method of rendering a 3D scene,
determining shading points on the 3D scene where shading is to be performed;
performing shading on the shading point; and
Determining shading information for the 3D scene based on a shading result of the shading point
including,
The step of determining the shading point,
increasing the number of shading points of the first region of the 3D scene when the geometric complexity based on the vertices is lower than the shading complexity of the first region of the 3D scene; and
Reducing the number of shading points of the second region when the geometric complexity based on vertices is higher than the shading complexity of the second region of the 3D scene.
3D rendering method. According to claim 1,
The step of determining the shading point,
The 3D rendering method of determining some of the vertices of the 3D object in the 3D scene as the shading point. According to claim 1,
The step of determining the shading point,
The 3D rendering method of determining vertices of a 3D object in the 3D scene and an additional point in the 3D object as the shading point. According to claim 1,
The shading point,
The 3D rendering method including only some of the vertices of the 3D object in the 3D scene. According to claim 1,
The shading point,
A 3D rendering method comprising a point that is not a vertex of a 3D object in the 3D scene. According to claim 5,
The shading point,
Further comprising some of the vertices of the 3D object, 3D rendering method. According to claim 5,
The shading point,
Further comprising all vertices of the 3D object, 3D rendering method. According to claim 1,
The step of determining the shading point,
The 3D rendering method of determining some of the vertices of a 3D object in the 3D scene and an additional point in the 3D object as the shading point. According to claim 1,
The step of determining the shading point,
The 3D rendering method of determining a shading point on which the shading is to be performed based on at least one of spatial and temporal characteristics of the 3D scene. According to claim 9,
The step of determining the shading point,
The 3D rendering method of determining the shading point based on at least one of information about a virtual light source, information about a virtual camera, information about a 3D object included in the 3D scene, and a shading result of a previous image frame. According to claim 10,
The information on the virtual light source includes information on at least one of a position, color, brightness, direction, angle, and moving speed of the virtual light source;
The information on the virtual camera includes information on at least one of a position, direction, angle, and moving speed of the virtual camera;
The information on the 3D object includes information on at least one of a shape, color, and material of the 3D object. According to claim 1,
The step of determining the shading point,
grouping vertices of the 3D scene into vertex groups; and
Determining one or more shading points for each vertex group
3D rendering method including. According to claim 12,
Grouping the vertices into vertex groups,
Grouping the vertices into the vertex groups based on at least one of a position of the vertices, a normal of the vertices, a shading result of a previous image frame, and whether or not the vertices are included in the same 3D object. 3D rendering method. According to claim 12,
The step of determining the shading point,
Determining the shading level based on at least one of a motion of a virtual light source, a motion of a virtual camera, a motion of a 3D object, a difference in brightness between adjacent vertices, and a position of a vertex.
A 3D rendering method further comprising a. According to claim 14,
The step of determining the shading point,
The 3D rendering method of determining the shading point on which shading is to be performed for each of the vertex groups according to the shading level. According to claim 12,
Grouping the vertices of the 3D scene into vertex groups,
Determining hierarchical structure information for the shading point for each vertex group
3D rendering method including. According to claim 1,
In the step of determining the shading information,
Determining shading values of vertices adjacent to the shading points by interpolating shading values of a plurality of shading points
3D rendering method including. According to claim 17,
In the step of determining the shading information,
Determining color values of pixels included in a rendering result image of the 3D scene by interpolating the shading values of the shading points and the determined shading values of the vertices
3D rendering method including. According to claim 17,
In the step of determining the shading information,
storing the shading values of the shading points and the shading values of the vertices in a texture buffer;
3D rendering method including. According to claim 1,
The step of performing the shading,
determining a color value of the shading point based on a lighting effect by at least one of a direct virtual light source and an indirect virtual light source;
The lighting effect includes a shadow effect by occlusion, 3D rendering method. A non-transitory computer readable storage medium storing instructions that cause computing hardware to execute the method of any one of claims 1 to 20. at least one processor; and
at least one memory storing instructions to be executed by the processor;
The instructions, when executed by the processor, cause the processor to:
determining shading points on the 3D scene where shading is to be performed;
performing shading on the shading point; and
Determining shading information for the 3D scene based on a shading result of the shading point
configured to run
The operation of determining the shading point,
increasing the number of shading points of the first region of the 3D scene when the geometric complexity based on the vertices is lower than the shading complexity of the first region of the 3D scene; and
When the geometric complexity based on vertices is higher than the shading complexity of the second area of the 3D scene, reducing the number of shading points in the second area
3D rendering device.
The method of claim 22,
The operation of determining the shading point,
and determining a shading value of at least one of vertices adjacent to the shading points and other shading points by interpolating shading values of a plurality of shading points. The method of claim 22,
The operation of determining the shading point,
grouping vertices of the 3D scene into vertex groups; and
Determining one or more shading points for each of the vertex groups based on a shading level
3D rendering device including a. The method of claim 22,
The instructions cause the processor to:
An operation of determining whether to update a shading value of the current vertex in a next image frame based on at least one of a distance between the current vertex and the screen and whether or not the current vertex exists in a shadow area.
A 3D rendering device configured to further execute

Images