JP2022136548A - Image processing system and method thereof and program - Google Patents

Abstract

To render a virtual object that is a permeation object at a high speed while reducing discomfort in its appearance.SOLUTION: An image processing system comprises attribute determination means for determining the permeation attribute of a virtual object, setting means for setting a virtual viewpoint to observe the virtual object, relations determination means for determining the physical relationship among each of a plurality of polygons representing the virtual object and the virtual viewpoint, determination means for determining if a surface of the polygon is to be rendered for each polygon of the plurality of polygons based on the physical relationship among the permeation attribute of the virtual object, the virtual viewpoint, and the polygon, and rendering means for rendering the surface of the polygon that is determined to be rendered by the determination means.SELECTED DRAWING: Figure 2

Description

The present invention relates to technology for displaying virtual objects.

2. Description of the Related Art In recent years, image display using three-dimensional computer graphics (CG) technology has been performed for various purposes. Three-dimensional CG technology is required to display CG images at high speed, as well as to provide expressions closer to reality. In order to approximate the appearance of the real thing, it is necessary to faithfully calculate all information such as light, materials, reflections and context. However, in many cases, there is a trade-off between the process of approximating the appearance of the real object and the speed of rendering.

In addition, there are several methods of drawing a solid body such as glass that transmits light (hereinafter referred to as "transmissive object") using CG technology. As a method of drawing a transparent object, for example, there is a method of rendering the entire scene, and when semi-transparent objects overlap, the method of rendering sequentially from the innermost value. Thus, realistic rendering requires computational resources on the one hand. Patent Document 1 proposes a method called depth peeling that requires less computational resources, but the current situation is that a high-performance GPU is still required.

As a rendering method for transparent objects that does not require a relatively large amount of computational resources, there is a method in which the context of transparent objects is determined on an object-by-object basis and the mixture of drawing colors due to transparency is calculated. In this case, the calculation of the front-back relationship of the surfaces within a single object is omitted. Therefore, if multiple surfaces can be viewed overlapping in a single object such as a cylinder or sphere, the front-back relationship of the surfaces will not be the same as the real object at the time of rendering. may be rendered in a different order.

U.S. Pat. No. 8,217,934

In the above-described method of calculating the mixture of rendering colors due to transparency, the context of a single object is not taken into account, and the appearance of the object may appear strange. In this case, by performing back surface culling for omitting the drawing of the back surface of the transparent object, it is expected that the unnatural appearance due to an unexpected pattern can be reduced.

However, if all the objects to be drawn are subjected to back culling, although it contributes to speeding up the processing, there are cases where the appearance is broken. For example, when the virtual viewpoint is inside the cube CG, if the virtual viewpoint is directed to the outside of the cube, all the faces of the cube become the back faces from the inside of the cube, so rendering is not performed. By unconditionally performing culling on the entire surface in this way, an unexpected side effect may occur.

In order to solve the above problems, according to one aspect of the present invention, an image processing apparatus includes attribute determination means for determining a transmission attribute of a virtual object, setting means for setting a virtual viewpoint for observing the virtual object, and relationship determination means for determining a positional relationship between each of a plurality of polygons representing a virtual object and the virtual viewpoint; and, for each polygon of the plurality of polygons, a transparency attribute of the virtual object and a relationship between the virtual viewpoint and the polygon. and a drawing means for drawing the face of the polygon determined by the determining means to be drawn.

According to the present invention, in drawing a virtual object that is a transmissive object, high-speed drawing can be realized while reducing the sense of incongruity in appearance.

FIG. 10 is a schematic diagram of how a translucent object looks when culling is not performed and when culling is performed; 2 is a block diagram showing a functional configuration example of an image processing apparatus according to the first embodiment; FIG. 4 is a flow chart showing an example of a processing procedure in the first embodiment; 4 is a flow chart showing an example of a processing procedure in the first embodiment; It is a flow chart showing an example of a processing procedure in a modification. 2 is a block diagram showing an example of the hardware configuration of an image processing device; FIG.

Preferred embodiments of the present invention will be described in detail below with reference to the drawings. The configurations described in the following embodiments are representative examples, and the scope of the present invention is not limited to those specific configurations.

(Embodiment 1)
In the present embodiment, when rendering a CG model of a transmissive object, by applying back face culling, light processing and appropriate CG rendering are achieved.

FIG. 1 shows an example of rendering a transmissive object that is different from the appearance of the real thing. FIG. 1 shows a CG model 101 of a transparent object drawn with culling and a CG model 102 of a transparent object drawn without culling. In the CG model 102, the bottom surface of the cylinder, which is actually the back surface, is rendered to look the same as the top surface, which is different from the actual appearance. The model 101 shows that by performing back face culling on a transmissive object, it is possible to reduce a sense of incongruity in appearance.

FIG. 2 is a block diagram showing a functional configuration example of an image processing apparatus that implements the above. A device 201 is an image processing device that implements the first embodiment.

A model data acquisition unit 203 acquires model data from the CG model database 202 . A transparent model judgment unit 204 judges the transparent attribute of the acquired model data to check whether it is a transparent object (translucent). In the present embodiment, rendering of the model data of the transmissive object implements back face culling in which polygons that have a positional relationship facing the back face with respect to the virtual viewpoint are not rendered.

The polygon surface direction calculation unit 205 acquires the surface direction of the polygon surface in order to perform back surface culling. The procedure for calculating the surface direction of the polygon face is determined, for example, by the arrangement of polygon vertices. The rotation direction of the order of the vertices is taken, and if it is "clockwise", it is set to "front", and if it is "counterclockwise", it is set to "back". The normal direction of the polygon surface can be calculated by calculating the outer product of two vectors according to the order of the vertices.

A viewpoint setting unit 207 sets the position, posture, and direction of a virtual viewpoint for observing model data. This viewpoint setting may be obtained from a numerical value designated by the user on the UI, or from an external device such as a camera or HMD (head mounted display) connected to the image processing apparatus 201 . The line-of-sight direction acquisition unit 208 acquires the line-of-sight direction of the virtual viewpoint.

Based on the direction of the polygon surface acquired from the polygon surface direction calculation unit 205 and the line-of-sight direction acquired from the line-of-sight direction acquisition unit 208, the drawing determination unit 206 determines the normal direction of the polygon surface and the line-of-sight direction. A relation determination is performed to determine the CG drawing method. A drawing processing unit 209 executes CG drawing processing based on the determination result of the image determination unit 206 . The display unit 210 displays the data of the model drawn by the drawing processing unit 209 on the external device.

Next, the hardware configuration of the image processing apparatus 201 will be described. FIG. 6 is a block diagram showing the hardware configuration of the image processing apparatus 201 according to this embodiment. In the figure, a CPU 6200 centrally controls each device connected via a bus. The CPU 6200 also reads and executes processing steps and programs stored in a read-only memory (ROM) 6300 . An operating system (OS), processing programs, device drivers, and the like according to this embodiment are stored in a ROM 6300, temporarily stored in a random access memory (RAM) 6400, and executed by a CPU 6200 as appropriate.

Also, the input I/F 6500 receives an input signal from an external device (imaging device) or the like in a format that can be processed by the image processing device 201 . Also, the output I/F 6600 outputs an output signal in a processable format to an external device (display device) or the like.

Next, a processing procedure of this embodiment realized by the above apparatus will be described. 3 and 4 are flowcharts for explaining the procedure of rendering processing according to this embodiment. FIG. 3 shows processing performed by the model data acquisition unit 203 and the transparent model judgment unit 204 when model data is read, and judges whether or not the acquired model data is a transparent model.

In step S3100, the model data acquisition unit 203 acquires model data from the CG model database 202. FIG. In step S3200, transmission model determination section 204 determines whether the acquired model is a CG model of a transmission object. Whether or not it is a CG model of a transmissive object is determined by, for example, whether the acquired model has translucent material information. If the acquired model data has translucent material information, the process moves to step S3300. In step S3300, the transparent model determination unit 204 turns on the back surface culling flag of the model data, and shifts to CG rendering processing. The user may be able to set whether or not the processing of the flag is to be determined by the transparent model determination unit 204 .

In FIG. 4, model data is rendered based on the back face culling flag set by the transparent model determination unit 204 and the virtual viewpoint direction. This processing will be described.

In step S4100, the viewpoint direction acquisition unit 208 acquires the direction of the virtual viewpoint. The direction of the virtual viewpoint is calculated, for example, from the direction of the virtual viewpoint coordinate system with the position and orientation of the virtual viewpoint in the world coordinate system as the origin. In step S4200, it is checked whether the back face culling flag of the model data is ON. If the back face culling flag is ON, the process proceeds to step S4300; otherwise, the process proceeds to step S4700. In step S4300, the polygon surface direction calculation unit 205 calculates the polygon surface direction. The normal direction of the polygon surface is calculated by cross product calculation of two vectors according to the order of vertices.

In step S4400, the rendering determination unit 206 calculates the inner product of the virtual viewpoint direction acquired by the viewpoint direction acquisition unit 208 and the polygon surface direction. If the inner product is greater than 0, the process proceeds to step S4500; otherwise, the process proceeds to step S4600. In step S4500, the rendering determination unit 206 determines that the polygon faces the back side of the virtual viewpoint, excludes it from rendering, and proceeds to step S4700.

In step S4600, a model determined to be opaque by the transparent model determination unit 204 or a polygon determined by the rendering determination unit 206 not to face the back side is to be rendered. In step S4700, it is checked whether or not all polygons have been determined to be drawing objects. If there are unprocessed polygons, the process proceeds to step S4200, and if all polygons have been processed, the process proceeds to step S4800.

As described above, in CG rendering of transparent objects, it is possible to realize high-speed rendering while reducing the sense of incongruity in appearance.

(Modification 1)
In the first embodiment, whether or not the virtual object is translucent is determined based on whether or not the model data is set as a transmissive object, but other determination methods may be used. For example, a method of thresholding the transmissivity of a virtual object to determine whether it is translucent can be considered.

FIG. 5 is a flow chart showing the processing procedure of the modification. Since the steps up to S3200 are the same as those in the first embodiment, description thereof is omitted. In step S5100, the transmittance of the model determined to be a transparent model is obtained. If the transmittance is greater than the preset threshold, the process proceeds to S3300.

(Other embodiments)
In the first embodiment, it is determined whether or not to draw the back surface (whether to turn on the back surface culling flag) depending on whether or not the back surface is translucent, but another determination condition may be added. For example, it may be the ratio of the virtual object to the visible area (display area). If the ratio is determined to be small (less than the threshold value) by the ratio determination, the effect on appearance is small, so the back face culling flag may be turned off (all polygon faces are drawn). This ratio may be determined using the number of pixels of the virtual object. Alternatively, the shape of the virtual object may be determined, and the back face culling flag may be turned off for an object with a complicated shape. In this shape determination, when the surface area is large with respect to the volume, it may be determined that the shape is complicated.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

201 image processing device 202 CG model database 203 model data acquisition unit 204 transmission model determination unit 205 polygon surface direction calculation unit 206 rendering determination unit 207 viewpoint setting unit 208 line of sight direction acquisition unit 209 rendering processing unit 210 display unit

Claims (16)

attribute determining means for determining a transmission attribute of a virtual object;
setting means for setting a virtual viewpoint for observing the virtual object;
relationship determining means for determining a positional relationship between each of a plurality of polygons representing the virtual object and the virtual viewpoint;
determining means for determining, for each polygon of the plurality of polygons, whether to draw the surface of the polygon based on the transparency attribute of the virtual object and the positional relationship between the virtual viewpoint and the polygon;
and drawing means for drawing the face of the polygon determined to be drawn by the determining means. The judging means judges not to draw the face of the polygon when it is judged that the transparency attribute of the virtual object is translucent and the polygon is positioned behind the virtual object with respect to the virtual viewpoint. 2. The image processing apparatus according to claim 1, wherein: The relationship determination means determines whether the polygon is located behind the virtual object with respect to the virtual viewpoint, based on the line-of-sight direction of the virtual viewpoint and the normal direction of the surface of the polygon. 3. The image processing apparatus according to claim 2. The relationship determination means determines that the polygon is positioned behind the virtual object with respect to the virtual viewpoint based on the inner product of the vector representing the line-of-sight direction of the virtual viewpoint and the vector representing the normal direction of the surface of the polygon. 4. The image processing apparatus according to claim 3, wherein determination is made as to whether the image is positioned. 5. The image processing apparatus according to claim 4, wherein, if the value of the inner product is positive, the relationship determining means determines that the polygon is positioned behind the virtual object with respect to the virtual viewpoint. 6. The relationship determining means according to claim 4, wherein said relationship determining means obtains a vector representing the normal direction of the face of said polygon based on the outer product of two vectors obtained by connecting vertices of said polygon in order. image processing device. The relationship determining means determines the positional relationship between each of the plurality of polygons representing the virtual object and the virtual viewpoint only when the attribute determining means determines that the virtual object has a translucent attribute. 2. The image processing apparatus according to claim 1, wherein: 2. The image processing apparatus according to claim 1, wherein, when it is determined that the transparency attribute of the virtual object is opaque, the determining means determines to render the surface of the polygon. An attribute indicating whether or not the virtual object is translucent is set in the virtual object,
2. The image processing apparatus according to claim 1, wherein said attribute determination means determines a transparency attribute of said virtual object based on said set attribute. Transmittance is set for the virtual object,
2. The image processing apparatus according to claim 1, wherein said attribute determining means determines the transparency attribute of said virtual object by comparing said transmittance with a threshold value. further comprising ratio determination means for determining the ratio of the virtual object to the display area;
2. The image processing apparatus according to claim 1, wherein, when said ratio is determined to be less than a threshold, said determination means determines to render all polygon surfaces of said virtual object. 12. The image processing apparatus according to claim 11, wherein said ratio determination means determines said ratio using the number of pixels of said virtual object. further comprising shape determining means for determining the shape of the virtual object;
2. The image processing apparatus according to claim 1, wherein, when the virtual object is determined to have a complicated shape, the determination means determines to draw all polygon surfaces of the virtual object. 14. The image processing apparatus according to claim 13, wherein said shape determining means determines the shape of said virtual object using the volume and surface area of said virtual object. an attribute determination step of determining a transmission attribute of the virtual object;
a setting step of setting a virtual viewpoint for observing the virtual object;
a relationship determination step of determining a positional relationship between each of a plurality of polygons representing the virtual object and the virtual viewpoint;
a determination step of determining, for each polygon of the plurality of polygons, whether the surface of the polygon is to be drawn, based on the transparency attribute of the virtual object and the positional relationship between the virtual viewpoint and the polygon;
and a drawing step of drawing the face of the polygon determined to be drawn in the determining step. A program that causes a computer to function as each means of the image processing apparatus according to any one of claims 1 to 14.

Images