JP3058769B2 - 3D image generation method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional image generating method for generating an image having a perspective from a three-dimensional image obtained by a computer.

[0002]

2. Description of the Related Art Computer graphics are widely used in all fields for generating moving images and still images. Computer graphics can generate not only a flat image but also a three-dimensional image based on perspective and display it on a display.

[0003] Here, an image optically projected by a conventional general camera will be described. FIG. 2 is an explanatory diagram of an image forming operation by a pinhole camera. In the figure, it is assumed that, for example, three objects A, B, and C are located at an appropriate distance from the pinhole 3, respectively. In this case, the images of these objects are formed on the image plane 4 by the pinhole 3. As long as the pinhole 3 has a sufficiently small area, any of these images can be accurately focused. Normally, the original image obtained by computer graphics is in focus regardless of its position in the depth direction, similarly to the image obtained by the pinhole camera as shown in FIG.

On the other hand, FIG. 3 is an explanatory view of the image forming operation by a lens. As shown in the figure, when three objects A, B, and C are located in front of the lens 5, these images are formed on the image plane 4 by the lens 5. In this case, for example, object B
When the lens 5 is in focus, a clear image of the object B appears on the image plane 4. However, the object A and the object C are in a state where the image is blurred because the image forming position is ahead or behind the image plane 4. The plane that is in focus and parallel to the imaging plane 4, that is, the plane on which the object B exists, is called a focal plane 6.

FIG. 4 is a view for explaining an image of a pinhole camera, and FIG. 5 is a view for explaining an image by a lens. As shown in FIG. 4, in the case of using a pinhole camera, all images are clearly displayed even though the objects A, B, and C are arranged at different positions in the depth direction. However, as shown in FIG. 6, in the case of using a lens, the object B existing in the focal plane is accurately focused, and a clear image is displayed.
The objects A and C arranged before and after the object are projected out of focus.

[0006]

By the way, comparing the figures of FIGS. 4 and 5, it can be seen that the image shown in FIG. 4 is shown on the display and the image shown in FIG. 5 is shown on the display. The effect obtained when viewed as a three-dimensional image differs from that in the case. In other words, the image shown in FIG. 5 is closer to a state where a deep object is viewed with the naked eye. That is, it can be said that the image shown in FIG. 5 has a higher depth perception effect. Therefore, conventionally, it has been attempted to generate such an image having a high depth perception effect using computer graphics. One of the methods is to generate a plurality of images obtained by shifting the pinhole slightly on a plane perpendicular to the depth direction while moving the pinhole camera toward the object of interest, changing the way of movement, and thus There is a method of obtaining a target image by taking an average of pixel values for each pixel of a plurality of obtained images.

However, in such a method, a huge amount of calculation must be performed to generate a plurality of images first, and there is a problem that a long processing time is required.
In addition to the above, there is a method of uniformly blurring an entire image. According to this method, each pixel of an image is subjected to a smoothing process using the same weighting filter, and the effect of blur is uniformly applied regardless of the degree of depth.

FIG. 6 shows an example of an image generated by such a conventional method. As shown in the figure, the conventional method using a weighting filter has a problem in that the object B in the focal plane is also affected by the smoothing of the weighting filter, causing blurring. Also, the effect of blurring of objects behind, which should not be visible behind the object in front,
There was also a problem that it reached the object in the foreground. The present invention has been made in view of the above points, and it is an object of the present invention to provide a three-dimensional image generation method that executes a process for obtaining an image close to nature with an enhanced depth perception effect.

[0009]

SUMMARY OF THE INVENTION The present invention is directed to
Includes the distance in the depth direction as depth information for each pixel
Generate a three-dimensional image to add blur from a three-dimensional image
Setting a predetermined reference position in the depth direction
And from the depth information for each pixel,
Determine the separation distance from the reference position, and then,
Each pixel has a blurring effect on other surrounding pixels
Based on each of the above separation distances to assume
Set the radius as the blur radius, then add a new pixel value
Set any pixel as a target pixel to be set, and
For a predetermined number of each peripheral pixel centered on the pixel, the corresponding
Set weighting value based on each blur radius
To create a weighting filter,
Is larger than that of the pixel of interest,
Remove the side pixels from the weighting target, and
The blur radius is the distance from the peripheral pixel to the pixel of interest.
If it is smaller than the separation, the surrounding pixels are removed from the weighting target,
The target image is calculated by the arithmetic processing using the weighting filter.
It is characterized by setting the elementary pixel value .

[0010]

According to this method, a three-dimensional image generated including depth information is prepared. Then, the reference position is set for each pixel.
Is determined, and blurring is performed based on this distance.
The blur radius R to determine the range affected by
You. Next, in order to determine a pixel value that determines the color and density of the pixel of interest , a weighting value based on the blur radius of surrounding pixels is calculated.
Get the weighting filter to set . In this weighting filter , a peripheral pixel having a larger separation distance than the target pixel or a peripheral pixel having a distance to the target pixel larger than the blur radius is used.
Are excluded because they do not affect the target pixel. If the surrounding pixels give the effect of blurring to the pixel of interest,
And the weighting value is set. According to this method, pairs in each pixel is given the effect of blurring corresponding to the separation distance. Since blur processing is performed on one image,
The time required for arithmetic processing is short. Further, a three-dimensional image close to nature can be obtained without blurring the image of the object behind the image of the object at the focal position .

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 is an explanatory diagram showing an embodiment of a three-dimensional image generation method according to the present invention. An outline of the method of the present invention is shown in the figure. First, the object arranged three-dimensionally
An in-focus original image 2 having a shape as viewed from the viewpoint 10 is generated from 0 by projective transformation. In this figure, for the sake of simplicity, a rectangular plate-shaped object is arranged at positions Z1, Z2, and Z3 in the depth direction. At this time, hidden surface elimination is performed by the Z buffer algorithm as shown in the figure. Further, through generation of the original image, depth information Z indicating a position in the depth direction corresponding to each pixel of the original image is stored in the Z buffer. That is, the depth information is Z1, Z2,
It becomes a value such as Z3.

Here, in the method of the present invention, the original image 2
And the depth information Z, the blur radius R of each pixel of the original image is obtained in advance. Assuming that the depth information of the focal plane that is in focus is Z2, the blur radius R is a radius indicating a range affected by the blur, which is almost proportional to the difference between the depth information of each pixel and the depth information of the focal plane. Is set to a value corresponding to
When obtaining the generated image 12, the original image is smoothed using the weighting filter 11 obtained based on the depth information and the blur radius of the target pixel and its surrounding pixels, and the pixel value of the target pixel is determined. Note that the weighting filter 11 will be described with reference to FIG.

FIG. 7 shows an apparatus block diagram for implementing the method of the present invention. In order to carry out the method of the present invention, an arithmetic processing circuit having a configuration as shown in the figure is prepared. The device shown is
It comprises a CPU (Central Processing Unit) 21 and a memory 23 connected thereto via a bus line 22. The memory 23 is provided with an original image data buffer 24, a Z buffer 25, an R buffer 26, and a generated image buffer 27. The original image data buffer 24 stores an image to be processed. In the Z buffer 25, the depth information Z described above with reference to FIG. 1 is stored in association with each pixel constituting the image. Similarly, the R buffer 26 stores the blur radius R in association with each pixel. The generated image data buffer 27 stores the generated image that has been finally subjected to the blurring processing.

FIG. 8 shows a schematic operation flowchart of the method of the present invention. The present invention is implemented by executing the processing from step S1 to step S4 as shown in the figure. Step S1 is a blur radius generation process for obtaining a blur radius for each pixel forming an image based on the depth information. Step S2 is a step for determining whether or not the processing has been completed for all pixels. Step S3 is processing for generating a weighting filter necessary for smoothing processing for obtaining a pixel value for each target pixel. Step S4 is a part for executing a smoothing process for each target pixel by actually using the weighting filter.

Hereinafter, the details of each operation of the above-mentioned flowchart will be described. FIG. 9 is a diagram for explaining a method of calculating a blur radius according to the present invention. In this figure, Z is depth information, R (Z) is a blur radius, C is a constant constant related to coordinate transformation when generating original image data, and F is a base.
Depth information of the position of the focal plane becomes quasi, Z-F represents the distance from the focal plane. Specifically, the object 1 in the figure
2 is the position of the focal plane, and its depth information Z2 is equal to F here. Also, for example, object 1-
Since the depth information of No. 3 is Z3, the distance from the focal plane is Z3-F.

Here, the point 8-2 on the focal plane object 1-2 is
Does not need to cause blur on the target image, but points 8-3 and 8-1 on other objects 1-3 and 1-1 need to cause a certain blur. In the present invention, the degree of the blur is determined according to the distance from the focal plane. The coefficient is a constant C. Therefore, the blur radius R (Z) is the product of C and the absolute value of ZF.
In this embodiment, as shown in the lower part of the figure, the blur radius R (Z) is focused only when the absolute value of ZF is equal to or less than the limit value D in order to suppress the effect of blur from spreading infinitely. In this case, the blur radius R (Z) is set to a constant C.
And the limit value D is expressed by a product CD.

Next, the weighting filter will be described.
FIG. 10 is an explanatory diagram of the matrix shape of the weighting filter. The weighting filter is, for example, 7 × 7 in this embodiment.
It consists of matrix-like operators. Here, in the figure, the target pixel 13 is displayed as fc, and the peripheral pixels 14 are displayed as fe. The target pixel 13 is a pixel to be processed in the image. Further, the peripheral pixel 14 refers to a pixel in a certain range to be considered in determining the pixel value of the target pixel 13. That is, in this weighting filter, each of the forty-eight peripheral pixels 14 included in the matrix includes the target pixel 1
For each of the three, the degree of blurring is quantified. The weight values thus obtained are added and averaged, and the pixel value of the pixel of interest 13 is determined.

In order to generate such a weighting filter, first, an operation table shown in FIGS. 11 and 12 is required. FIG. 11 is an explanatory diagram of a calculation table of the distance De to the target pixel. This calculation table is a table for calculating the distance De from each peripheral pixel to the target pixel 13. As shown in the figure, 1 dot, 2 dots, 3 dots
If the distance De is set to 1, 2, and 3 for pixels separated by dots, the distance to the pixel of interest 13 can be obtained by calculation in the vertical and horizontal diagonal directions.
If the result is tabulated and set as an operation table, the matrix of the weighting filter shown in FIG.
The distance to 3 can be determined immediately and compared to the blur radius.

FIG. 12 is an explanatory diagram of an operation table of the function f (R (Z)). The function f (R (Z)) indicates each weight value on the matrix of the weight filter. FIG.
The calculation table of No. 2 is used when obtaining a weight value from the blur radius R. As shown in the figure, this value is obtained by counting the number of pixels that fall within the circle indicated by the blur radius R of the peripheral pixel 14, and a function f (R) corresponding to the blur radius R (Z).
(Z)) is represented by the reciprocal of the number of pixels falling within a circle indicated by the blur radius. Therefore, the blur radius is 1 or less,
That is, if the pixel does not affect the peripheral pixels, the value of the function f (R (Z)) becomes 1, and the value of the function f (R (Z)) decreases as the blur radius increases. If the blur radius R is equal to or larger than the calculation table shown in the figure, it deviates from the weighting filter, so the value of the function f (R (Z)) is set to a constant value.

FIG. 13 is a diagram illustrating a weighting filter according to the present invention and its application processing. The weighting filter 11 as shown in the figure is generated by using the above-described operation table. For example, in this example, the weight value of the peripheral pixel 14-1 is 0, the weight value of the peripheral pixel 14-2 is 1/21, and the weight value of the peripheral pixel 14-3 is 1/37. Here, the weight value becomes 0 because
If the pixel in the depth direction is farther than the pixel of interest,
Alternatively, the blur radius R (Z) is the distance De to the pixel of interest.
Is smaller than. That is, the weighting value is forcibly set to 0 for a pixel that is farther away from the target pixel if a blurring effect is applied to the image in the foreground. When the pixel of interest 13 does not fall within the circle of the blur radius, the weighting value is set to 0 so that the blur of the pixel does not affect the pixel of interest 13.
I am trying to be.

The other pixels, that is, peripheral pixels that have a certain blurring effect on the target pixel 13 are shown in FIG.
The value of the function f (R (Z)) described using is directly used as the weighting value. In the example of the weighting filter shown in the figure, the values of all the pixels constituting the matrix are not entered, but in practice, the minimum value of the function f (R (Z)) is calculated from 0 or 1 for all the pixels. Any weight value up to 1/45 will be set. The weighting filter obtained as described above is sequentially applied to all the pixels constituting the image, and the application processing is executed for each pixel. In the application processing, as shown in the equation in the figure, all the products of the original pixel values and the weight values in each matrix are added, and this is divided by the sum total of the weight values. Thus, a new pixel value of the target pixel 13 is obtained.

In the case of a color image, the pixel values have, for example, three color components of red, green and blue. The pixel value of each color component indicates, for example, an 8-bit density level, and a smoothing process using the above-mentioned weighting filter is performed on each pixel value. By the above processing,
The effect of blurring of the image in the foreground from the viewpoint extends to the image in the rear, and the degree of the effect is almost proportional to the distance from the focal plane. Also, in the rear image, only the portion appearing on the image is blurred, and the image in front is not affected by the blur at all, and the image close to the focal plane is not blurred.

The present invention is not limited to the above embodiment. The values of the depth information and the blur radius used in the above embodiment may be formally obtained by any other expression other than the above, and the configuration of the weighting filter is not limited to the 7 × 7 matrix but may be the required synthesis. Any size can be selected according to the quality of the image.

[0024]

According to the three-dimensional image generation method of the present invention described above, a three-dimensional image generated including depth information is used in accordance with the distance from a reference position using the depth information. Obtain a blur radius indicating the range of the effect of the blur, and further calculate a weighting value for determining a pixel value of the pixel of interest with respect to peripheral pixels included in a certain area centered on the pixel of interest in the image, Since a matrix weighting filter is obtained and the image is smoothed, the calculation time can be sufficiently reduced. Ma
In addition, by setting the blur radius in accordance with the separation distance, it is possible to obtain an image closer to nature with a larger blur as the distance from the reference position increases. In addition, the peripheral image to be weighted
Since the element is selected in advance, the smoothing process is performed on the image in front so as not to be affected by the blur of the image behind,
The effect of unnatural blur can be prevented, and a more natural three-dimensional image can be generated.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of a three-dimensional image generation method according to the present invention.

FIG. 2 is an explanatory diagram of an image forming operation by a pinhole camera.

FIG. 3 is an explanatory diagram of an image forming operation by a lens.

FIG. 4 is an explanatory diagram of an image of a pinhole camera.

FIG. 5 is an explanatory diagram of an image by a lens.

FIG. 6 is an explanatory diagram of a conventional generated image example.

FIG. 7 is an apparatus block diagram for implementing the method of the present invention.

FIG. 8 is a schematic operational flowchart of the method of the present invention.

FIG. 9 is an explanatory diagram of a blur radius calculation method according to the present invention.

FIG. 10 is an explanatory diagram of a matrix shape of a weighting filter.

FIG. 11 is an explanatory diagram of a calculation table of a distance De to a target pixel.

FIG. 12 is an explanatory diagram of a calculation table of a function f (R (Z)).

FIG. 13 is a diagram illustrating a weighting filter according to the present invention and its application processing.

[Explanation of symbols]

 1-1 to 1-3 Object 2 Original image 10 Viewpoint 11 Weighting filter 13 Target pixel 12 Generated image Z Depth information R Blur radius

Claims (1)

(57) [Claims] The distance in the depth direction away from the viewpoint is determined for each image.
Adds blur from 3D images containing depth information for each element
A method for generating a three-dimensional image to be given , comprising: setting a predetermined reference position in the depth direction;
From each of the depth information for each pixel,
Of each of the pixels , and then, for each of the pixels, a blur with respect to other surrounding pixels.
Each of the above separation distances is assumed to
Set the radius based on the separation as the blur radius, and then focus on one of the pixels to set a new pixel value
Pixels and a predetermined number of each periphery around the pixel of interest
For the pixel, it is based on each corresponding blur radius
Create weighting filters by setting weighting values
At the same time, the separation distance of the peripheral pixel is larger than that of the pixel of interest.
If it is larger, the peripheral pixel is removed from the weighting target, and the blur radius of the peripheral pixel is
If the distance to the pixel of interest is smaller than
From the target image, and perform the arithmetic processing using the weighting filter.
3D image generation characterized by setting elementary pixel values
Method.