JP2001291116A - Device and method for generating three-dimensional image and program providing medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a texture mapping technique capable of generating a high quality three-dimensional image. SOLUTION: Two camera photographing viewpoint images adjacent to a virtual viewpoint are selected, weighting is performed about a parameter the two images have, e.g. RGB values based on an angle formed between a virtual viewpoint vector and a photographing viewpoint vector, and the texture image of the virtual viewpoint is generated based on the two camera photographing viewpoint images. When the angle formed between the vector V1 of a photographing viewpoint 1 and the vector E of the virtual viewpoint is defined as θ1 and an angle formed between the vector V2 of a photographing viewpoint 2 and the virtual viewpoint, image synthesis is calculated as α=θ1/(θ1+θ2), and RGB values at the virtual viewpoint is a sum of RGB values at the viewpoint 1×(1-α)+RGB values of the viewpoint 2×α.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a three-dimensional image generating apparatus, a three-dimensional image generating method, and a program providing medium, which relate to a technique for pasting a texture image into a three-dimensional shape model. More specifically, the present invention relates to a three-dimensional image generating apparatus and a three-dimensional image generating method capable of generating a realistic three-dimensional image with less unnaturalness by improving a pasting process of a texture image of a virtual viewpoint. .

[0002]

2. Description of the Related Art The three-dimensional shape of an object is measured, for example, by projecting light onto an object to be measured, measuring the time until the light is reflected and returning, or applying a slit-shaped pattern light to the object to be measured. A stereo image method is often used to examine the shape of the pattern light projected on the measurement object, or to use two or more cameras to find the corresponding points between the images to determine parallax and measure the distance, etc. Are known.

A realistic three-dimensional image is generated by pasting an image reflecting the actual color of the measurement target to a three-dimensional model as distance data of the measurement target measured by such various methods. . The actual image to be measured is called a texture image, and attaching the texture image to three-dimensional data is called texture mapping.

Generally, an image to be pasted on the surface of a three-dimensional model is a two-dimensional bitmap image or an image file image. 3 created in 3D graphics technology
By attaching a texture image composed of a two-dimensional bitmap image or an image file image to the surface of the three-dimensional shape of the object obtained by the D-shape model or the 3D shape measuring device, the object is displayed three-dimensionally. For example, a natural three-dimensional wall can be expressed by attaching an image such as a brick, or by attaching a picture of the world map to a sphere,
A three-dimensional earth comes to be expressed.

FIG. 1 is a diagram for explaining a general texture mapping technique. A three-dimensional shape model 101 shown in FIG. 1 is a three-dimensional shape model 101 based on distance data acquired by, for example, the above-described stereo image method, and a two-dimensional image (texture) 102 is a visual image captured from a certain viewpoint. A two-dimensional image (texture) 102 as an actual image reflecting various color data and the like. Texture mapping is performed by attaching a two-dimensional image (texture) 102 to a three-dimensional shape model 101, and a realistic three-dimensional expression is possible by this texture mapping. Note that the wire frame display shown in the figure shows a plane area where a texture obtained from the three-dimensional shape model 101 is to be pasted.

Until now, the 3D measurement of an object obtained by a 3D measuring apparatus using a stereo image method, laser beam projection, etc.
Various studies have been made on a technique of attaching a texture image to a shape. When pasting texture images from multiple viewpoints to respective areas of the measurement target, if multiple texture images are captured under different light source conditions, the difference between the light source conditions at the time of shooting of each texture image may cause a difference between the images. And the generated image may have an unnatural hue if only the paste processing of the texture image is performed. In order to solve such a situation, a technique has been developed for performing high-quality texture mapping by estimating a light source situation at the time of capturing a texture image and correcting a color tone difference between the images based on the estimation. (For example, Imari Sato, Yoichi Sato, Katsushi Ikeuchi: Measurement of real light source environment by omnidirectional stereo and superimposition of virtual object on real image based on it, IEICE Transactions D-
II, Vol. J81-D-II, No. 5, pp86
1-871, 1998).

[0007]

However, in general, when photographing an outdoor scenery or a building using a digital camera, a DV cam, or the like, it is not easy to estimate the light source of the illumination light. ) Is difficult to perform high quality texture mapping. In order to avoid the unnaturalness caused by using a plurality of images, there is a method of performing texture mapping using only one texture image as shown in FIG. 2, for example. FIG. 2 shows a configuration in which texture mapping is performed by attaching an image of a corresponding area of one two-dimensional image (texture image) 202 to a wire frame of a three-dimensional shape model 201.

However, in such a method using one image, there is a problem that the image quality of the texture is deteriorated due to the change of the viewpoint. For example, when the image of the front surface 203 and the image of the end portion 204 of the cup shown in FIG. 2 are both acquired from one two-dimensional image (texture) 202, a difference occurs between the image quality of the front surface of the cup and the image quality of the end portion. Is reduced.

[0009] As a method of solving such a decrease in image quality, there is a method of selecting a texture image of a shooting viewpoint closest to a virtual viewpoint and pasting the texture image on a three-dimensional shape model. This method will be described with reference to FIG. In FIG. 3, when a texture image of the measurement target 301 is captured from a plurality of viewpoints (imaging viewpoints 1 to n) and a virtual viewpoint is set,
The photographing viewpoint closest to the virtual viewpoint is selected, and the texture image is attached to the three-dimensional shape model of the measurement target 301. In the example shown in FIG. 3, for example, the RGB value of the virtual viewpoint is either the shooting viewpoint 1 or the shooting viewpoint 2, but the angle between the shooting viewpoint 1 and the virtual viewpoint is θ1, and the angle between the shooting viewpoint 2 and the virtual viewpoint is As θ2, if θ1 <θ2, the RGB value of the imaging viewpoint 1 is selected, and if θ1> θ2, the RGB value of the imaging viewpoint 2 is selected. According to such a method, the image quality is improved. However, in this method, it is difficult to maintain the continuity of light and shade between images, and the image becomes unnatural due to a change in viewpoint.

[0010] When a plurality of texture images can be used for one surface, VDTM (View-dependent text) representing a realistic three-dimensional object is obtained by selectively synthesizing the texture images as the viewpoint moves.
A method called "true mapping" has been developed (Paul E. Debevec, Camillo J. Taylor and Jitendra).
Malik: "Modeling and Rendering Architecture from Ph
otographs: A hybrid geometry-and image-based approa
ch, "ACM SHINNRAPH '96 Proceedings, pp.11-20,199
6.) This method is different from the conventional CG method that performs texture mapping using one texture image.
It is possible to reproduce the change in appearance due to the movement of the viewpoint of a part that cannot be reproduced as a three-dimensional shape. However, this method has a problem that the amount of image data to be held to increase the image quality is enormous, and also has a limitation that the light source conditions of a plurality of texture images must be constant.

In order to reduce the amount of image data, a method of transmitting and mapping only the texture data of the visible portion every time the viewpoint changes (S. Horbelt, F. Jordan TE
brahimi: "View-Dependent Texture Coding for Transmi
ssion of Virtual Environment. "Proceedings of the 1
998 IEEE International Symposium on Circuits andSy
stems. Vol. 5, No. 6, pp. 498-501, 1998.). However, in this method, data transmission is performed every time the viewpoint is changed on the user side.
There is a disadvantage that data access efficiency is low.

Further, interpolation between images is performed using correspondence between images taken at the time of reconstruction of a three-dimensional shape (Shenchang
Eric Chen, Lance Williams: "View Interpolation for
Image Synthesis, "ACM SHIGGRAPH '93 Proceedings.p
p. 279-288, 1993., SSteven M. Seitz and Charles R.
Dyer: "View Morphing," ACM SHIGGRAPH '96 Proceeding
s, pp. 21-30, 1996.), a method for reducing the amount of image data has been proposed. However, with this approach,
There is a problem of deterioration of image quality and unnaturalness depending on how to take an image at the time of shooting.

On the other hand, a method of estimating the light source condition at the time of capturing a texture image (Imari Sato, Yoichi Sato, Katsushi Ikeuchi: "
Estimation of Real Illumination Environment Based on Soft Shadow of Real Object, IPSJ Research Report, 98-CVIM-11
0, 1998), a method of correcting a color difference between images has been studied. However, there is a restriction that a certain object in the image must be moved for the correction process. For example, it is not possible to estimate the light source of an image taken outdoors.

The three-dimensional image generating apparatus and the three-dimensional image generating method of the present invention have been made in view of the above-mentioned drawbacks of the various prior arts. By using two images taken from the viewpoint closest to the virtual viewpoint, a texture image of the virtual viewpoint is created using a gray value correction method using variance and variance to generate a realistic three-dimensional image without unnaturalness. The purpose is to make it possible.

[0015]

SUMMARY OF THE INVENTION The present invention has been made in consideration of the above problems, and a first aspect of the present invention is to generate a three-dimensional image by pasting a texture image on a three-dimensional shape model. In the three-dimensional image generation device, two camera-captured texture images having the closest viewpoint to the virtual viewpoint for the three-dimensional image are selected from the plurality of camera-capture viewpoint images, and the virtual viewpoint direction and the two camera-captured images are selected. Based on the angle between each of the two photographing viewpoint directions of the texture image, a process of synthesizing the two images by weighting the image parameters of the two camera photographed texture images is performed to calculate image parameters at the virtual viewpoint. A three-dimensional image generating apparatus having a configuration for executing texture mapping based on the parameters; .

Further, in one embodiment of the three-dimensional image generating apparatus according to the present invention, the processing of calculating the parameters of the image at the virtual viewpoint is performed by photographing two camera photographed texture images having the photographing viewpoint directions closest to the virtual viewpoint. When the viewpoints are imaging viewpoint 1 and imaging viewpoint 2, the angle between the vector V1 of the imaging viewpoint 1 and the vector E of the virtual viewpoint is θ1, and the angle between the vector V2 of the imaging viewpoint 2 and the vector E of the virtual viewpoint is Is defined as θ2, α = θ1 / (θ1 + θ
2) is obtained, and the image parameter at the virtual viewpoint = the image parameter of the photographing viewpoint 1 × (1−α) + the image parameter of the photographing viewpoint 2 × α.

Further, in one embodiment of the three-dimensional image generating apparatus according to the present invention, the image parameters include one of an RGB value and a gray value.

Further, in one embodiment of the three-dimensional image generating apparatus of the present invention, an image area used as a texture image in the plurality of camera viewpoint images is
A normalization process of the gray-scale average value and the variance value of the image is executed, and the image after the normalization process is subjected to a synthesis process of the two images by weighting the image parameters to execute a calculation process of the image parameter at the virtual viewpoint. It is characterized by having a configuration.

According to a second aspect of the present invention, there is provided a three-dimensional image generating method for generating a three-dimensional image by pasting a texture image on a three-dimensional shape model. An image selecting step of selecting two camera-photographed texture images having the closest photographing viewpoint directions to the virtual viewpoint with respect to the angle between the virtual viewpoint direction and each of the two photographing viewpoint directions of the two camera-photographed texture images. A combining process step of performing a combining process of the two images by weighting the image parameters of the two camera-photographed texture images, and calculating image parameters at the virtual viewpoint, and performing texture mapping based on the parameters. Texture mapping step; and In the image generation process.

Further, in one embodiment of the three-dimensional image generation method of the present invention, the processing of calculating the parameters of the image at the virtual viewpoint is performed by photographing two camera photographed texture images having the photographing viewpoint directions closest to the virtual viewpoint. When the viewpoints are imaging viewpoint 1 and imaging viewpoint 2, the angle between the vector V1 of the imaging viewpoint 1 and the vector E of the virtual viewpoint is θ1, and the angle between the vector V2 of the imaging viewpoint 2 and the vector E of the virtual viewpoint is Is defined as θ2, α = θ1 / (θ1 + θ
2) is obtained, and the image parameter at the virtual viewpoint = the image parameter of the imaging viewpoint 1 × (1−α) + the image parameter of the imaging viewpoint 2 × α.

Further, in one embodiment of the three-dimensional image generating method according to the present invention, the image parameters include one of an RGB value and a gray value.

Further, in one embodiment of the three-dimensional image generating method of the present invention, the image area used as a texture image in the plurality of camera viewpoint images is
The image processing apparatus further includes a normalization processing step of performing a normalization processing of an average value and a variance value of the image, and the synthesis processing step executes synthesis processing of the two images by weighting image parameters on the image after the normalization processing. It is characterized by doing.

Furthermore, a third aspect of the present invention is a computer-based three-dimensional image generation process for generating a three-dimensional image by pasting a texture image on a three-dimensional model.
A program providing medium tangibly providing a computer program to be executed on a system, wherein the computer program has a shooting viewpoint direction closest to a virtual viewpoint for a three-dimensional image from a plurality of camera shooting viewpoint images. An image selection step of selecting two camera-photographed texture images; and an angle between the virtual viewpoint direction and each of the two camera-view texture directions of the two camera-photographed texture images. A combination processing step of performing a combination processing of the two images by weighting the image parameters; and a texture mapping step of calculating an image parameter in the virtual viewpoint and performing a texture mapping based on the parameter. On the program distribution medium

The program providing medium according to the third aspect of the present invention is, for example, a medium for providing a computer program in a computer-readable format to a general-purpose computer system capable of executing various program codes. . The form of the medium is not particularly limited, such as a storage medium such as a CD, an FD, and an MO, and a transmission medium such as a network.

Such a program providing medium defines a structural or functional cooperative relationship between the computer program and the providing medium for realizing a predetermined computer program function on a computer system. Things. In other words, by installing the computer program into the computer system via the providing medium, a cooperative operation is exerted on the computer system, and the same operation and effect as the other aspects of the present invention can be obtained. You can do it.

Still other objects, features and advantages of the present invention are:
It will become apparent from the following more detailed description based on the embodiments of the present invention and the accompanying drawings.

[0027]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A three-dimensional image generating apparatus and a three-dimensional image generating method according to the present invention will be described in detail below.
First, the outline of the texture image mapping executed in the three-dimensional image generation device of the present invention will be described with reference to FIG.

In the texture image mapping executed in the three-dimensional image generating apparatus of the present invention, the object 401 to be measured is used.
Using multiple texture images with different viewpoints for
The two images closest to the virtual viewpoint are selected, and the selected two
Based on the images, texture mapping is performed using a new texture image obtained by a synthesis process in which the angle between the virtual viewpoint and the actual shooting viewpoint is weighted.

For example, it is assumed that image display is executed using RGB values as parameters for determining the color value of an image. When the virtual viewpoint is located between the shooting viewpoint 1 and the shooting viewpoint 2, the RGB values at the virtual viewpoint are calculated by the following equations. In the following equation, α is α1, where θ1 is the angle between the vector V1 of the shooting viewpoint 1 and the vector E of the virtual viewpoint, and θ2 is the angle between the vector V2 of the shooting viewpoint 2 and the vector E of the virtual viewpoint. = Θ1 / (θ1 +
θ2).

[0030]

## EQU1 ## RGB value at virtual viewpoint = RGB value of photographing viewpoint 1 × (1−α) + RGB value of photographing viewpoint 2 × α (Equation 1)

As can be understood from the above equation, the RGB values, which are the image parameters of the texture image of the virtual viewpoint, are obtained by synthesizing images from two photographing viewpoints close to the virtual viewpoint. When the virtual viewpoint direction matches one shooting viewpoint direction, the image of the virtual viewpoint is defined as the image of the matching shooting viewpoint, and the combining processing of the two images is not performed.

Since the texture image of the virtual viewpoint obtained by the above equation is generated by a synthesis process using the image of the photographing viewpoint, the difference in the illumination conditions of the n images photographed at the viewpoints 1 to n is absorbed. And continuity between images is maintained. As a result, high-resolution texture image mapping becomes possible, and even for a part that cannot be reproduced as a three-dimensional shape, a change in appearance due to a viewpoint change can be reproduced, and a more realistic three-dimensional display becomes possible. .

In particular, there is an effect that an image change when the viewpoint is changed becomes smooth. That is, when there are n camera viewpoint images and these are V1 to Vn, for example, a certain virtual viewpoint image is a camera photographing viewpoint image V1, V2.
When the viewpoint is shifted, the image is switched to the synthesized image based on the camera photographing viewpoint images V2 and V3.
And the combined image of V4 and the combined image of V4 and V5. Thus, if you change the viewpoint continuously,
As the corresponding texture image, the image that was continuously used for the previous texture image is continuously used, and the usage ratio also changes continuously as the viewpoint moves, so discontinuity between images Is reduced, and the observer does not feel discomfort.

Next, a process of generating a three-dimensional image by performing texture mapping by a synthesizing process using a plurality of photographing viewpoint images in the three-dimensional image generating apparatus of the present invention will be described with reference to FIG.

Step S501 is an initialization step for setting the number of images of the photographing viewpoint images. Step S
Numeral 501 indicates that the number of photographing viewpoint images = n. In step S502, it is determined whether or not the virtual viewpoint vector E has changed. When it is determined that the virtual viewpoint vector E has changed, in steps S503 to S506, a vector comparison is performed between the camera photographing viewpoint vectors V1 to Vn based on the camera viewpoints i = 1 to n and the virtual viewpoint vector E, The photographing viewpoint vector closest to the virtual viewpoint vector E is selected. Specifically, the inner product of each camera shooting viewpoint vector Vi and the virtual viewpoint vector E is calculated,
The values are compared with each other.

When the two photographing viewpoint vectors closest to the virtual viewpoint are selected, in the next step S507, the average value and the variance of the image gradation distribution are reduced in order to suppress the dispersion of the gradation distribution between images in the three-dimensional image. Perform value normalization. FIG. 6 shows this processing flow.

In step S601 in FIG. 6, a texture mapping image is input. That is, texture images of camera viewpoints 1 to n are input. Step S602
Then, a partial image region to be used as a texture is cut out for images 1 to n. This partial area corresponds to, for example, FIG.
Are images including the image of the area selected as the image used for synthesizing the virtual viewpoint in steps S503 to S606 shown in FIG.

Next, in step S603, the average value and the variance of the gray level of the partial image area extracted in step S602 are calculated. Next, in step S604, normalization processing of the grayscale average value and variance value of each partial image area is executed to correct the image used for the texture mapping processing. Note that the normalization processing based on the calculation of the grayscale average value and the variance of the partial image area does not necessarily need to be performed, for example, when it is determined that there is almost no grayscale variation in each camera viewpoint image. .

Thereafter, a texture image at the virtual viewpoint is generated in step S508 by a synthesis process from the two camera viewpoint images selected in step S505. The method of synthesizing the image is executed by the weighting of (Equation 1) described above. In this way, a texture image of the virtual viewpoint is generated and mapped to generate a three-dimensional image. The two camera viewpoint images for which the synthesizing process is performed in step S508 are the camera viewpoint images that have been subjected to the normalization processing of the grayscale average value and the variance value in step S507. It will be suppressed. When the virtual viewpoint and one photographing viewpoint match, only one photographing viewpoint image is used, and the combining process from the two images is not executed.

In the above-described image synthesizing process, the synthesizing process has been described as an example in which two images are color images and images having RGB values.
Even for parameters other than B, it is possible to execute the image parameters of the virtual viewpoint image by weighting the parameters of the two closest camera viewpoint images in the same manner as in the above equation 1. The weighting of these parameters is executed based on the angle between the virtual viewpoint vector E and the camera viewpoint vectors of the two closest camera viewpoint images.

By the above-described image processing, a realistic three-dimensional image having a more realistic feeling is generated. In the texture mapping, for example, a three-dimensional image is generated by generating the above-described composite image for each wire frame region shown in FIG. 1 and executing a process of pasting the composite image for each wire frame region. Adjustment of the consistency between the texture images attached to the plurality of wire frame regions is realized by, for example, a factor decomposition method. In this method, as shown in FIG. 7, camera viewpoint images (t = 1 to n) are photographed from a plurality of different viewpoints for an object to be photographed, and one camera viewpoint image, for example, t = Some feature points are acquired based on one camera viewpoint image. Further, the position corresponding to the acquired feature point is determined in each camera viewpoint image, and a measurement matrix (Measurement Matrix) is determined.
trix) is calculated, UΣV 'is factorized, and a process of generating a shape S according to the movement M of the camera is executed.

FIG. 7 shows a method and procedure for restoring the three-dimensional shape of an object and the movement of a camera using a plurality of images taken from different viewpoints.

First, an object is photographed while changing the viewpoint of the camera. F ft (x, y) | t
= 1,. . . . , F} and the first image f1
P feature points are extracted from (x, y) by the variance evaluation value in the window and the like, and are traced over F images. The feature point coordinates (Xfp, Yfp) on each frame obtained by the feature point tracking can be represented by a matrix W (herein referred to as a measurement matrix). Then, by applying a linear projection model, the measurement matrix W can be decomposed into two orthogonal matrices U and V ′ and a diagonal matrix に よ っ て by singular value decomposition (SVD). Therefore, the orthogonal matrices U and V ′ include the camera motion information M and the target three-dimensional information S, respectively. Further, by using the constraint condition of the unit vector (i, j, k) representing the posture of the camera, a matrix A for uniquely determining M and S is obtained. Therefore, the camera movement M and the three-dimensional shape S of the object are uniquely determined.

For details on the method of restoring the three-dimensional shape of an object and camera motion using the above-described factorization method, see, for example, a paper by Kanade et al. Transaction D-II, Vol. J
76-D-II, no. 8, pp. 1497-1505
(1993.3).

FIG. 8 shows an example in which texture mapping is performed in the three-dimensional image generating apparatus of the present invention by using a synthesizing process using two camera viewpoint images close to a virtual viewpoint. In this example, 16 camera viewpoint images are used, and by the above-described method for each virtual viewpoint, a combining process is performed by weighting the angle between the virtual viewpoint vector and the two closest camera viewpoint vectors, thereby obtaining a virtual viewpoint. The mapping is performed by calculating image parameters, for example, RGB values. With this method, high-quality texture images were mapped at any viewpoint.

FIG. 9 shows a comparison between the three-dimensional image (c) obtained by the three-dimensional image generation processing of the present invention and the three-dimensional images (a) and (b) obtained by the conventional method. . The conventional method (a) is a three-dimensional image obtained by performing texture mapping using one texture image. This example is a display result when mapping to a three-dimensional image using the image of frame 1 shown in FIG. 8A and viewing the result from a certain viewpoint. In the mode (a), even if the viewpoint is changed, the texture is not changed. Therefore, the texture when the viewpoint is changed causes unnaturalness, and the quality of the texture of the window is particularly deteriorated.

FIG. 9B shows a view from a certain viewpoint by a so-called mapping method in which an image of a shooting viewpoint closest to the virtual viewpoint is selected using texture images from 16 different viewpoints shown in FIG. An image is displayed. In the example of (b), the texture is also changed in accordance with the change of the viewpoint, but a sharp change may occur in the grayscale value between the images when the texture is changed, which causes a sense of discomfort to the observer. This is due to the difference in lighting conditions at the time of photographing the image from each camera photographing viewpoint, and similarly to FIG.

FIG. 9C shows an image generated by the three-dimensional image generation method of the present invention, in which a viewpoint image is generated by synthesizing two photographed images close to the viewpoint. As the viewpoint changes, the texture image also changes, and the image at that time is, for example, a camera viewpoint image V1 to V1.
When Vn is set, a certain virtual viewpoint image is synthesized by the camera shooting viewpoint images V1 and V2. When the viewpoint is shifted, an image synthesized by the camera shooting viewpoint images V2 and V3, and further, a synthesized image of V3 and V4, V4 and This follows the composite image of V5. As described above, when the viewpoint is continuously changed, the discontinuity between images is reduced because the corresponding texture image is continuously used as the image used for the previous texture image. It does not cause any discomfort to the observer. Further, if the normalization processing of the grayscale average value and the variance value described above with reference to FIG. 6 is executed, the difference between the images is further eliminated.

As described above, according to the three-dimensional image generating apparatus and the three-dimensional image generating method of the present invention, higher image quality can be obtained.
In addition, a three-dimensional image in which the continuity of the image when the viewpoint moves is improved. The three-dimensional image generation method of the present invention is applied to, for example,
By applying to L (Virtual Reality Modeling Language), a more realistic three-dimensional image can be displayed on a browser.

The present invention has been described in detail with reference to the specific embodiments. However, it is obvious that those skilled in the art can modify or substitute the embodiment without departing from the spirit of the present invention. That is, the present invention has been disclosed by way of example, and should not be construed as limiting. In order to determine the gist of the present invention, the claims described at the beginning should be considered.

[0051]

As described above, according to the three-dimensional image generation apparatus and the three-dimensional image generation method of the present invention, texture mapping is performed using a synthesis process using two camera viewpoint images close to a virtual viewpoint. With this configuration, it is possible to generate a three-dimensional image with high image quality and improved continuity of images when moving the viewpoint.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a general texture mapping technique.

FIG. 2 is a diagram illustrating a conventional texture mapping method using one image.

FIG. 3 is a diagram illustrating a conventional mapping method based on texture image selection using a plurality of images.

FIG. 4 is a diagram illustrating a texture mapping method using two images applied in the three-dimensional image generation device of the present invention.

FIG. 5 is a diagram illustrating a processing flow of a texture mapping method using two images applied in the three-dimensional image generation device of the present invention.

FIG. 6 is a diagram illustrating a normalization processing flow of an average value and a variance value of an image in texture mapping using two images applied in the three-dimensional image generation device of the present invention.

FIG. 7 is a diagram for explaining a shape restoring method in the three-dimensional shape measuring apparatus of the present invention.

FIG. 8 is a diagram illustrating a processing example of a three-dimensional image generated by the three-dimensional shape measuring apparatus of the present invention.

FIG. 9 is a diagram comparing a three-dimensional image generated by the three-dimensional shape measuring apparatus of the present invention with a three-dimensional image generated by a conventional method.

[Explanation of symbols]

 Reference Signs List 101 3D shape model 102 2D image (texture) 201 3D shape model 202 2D image (texture) 301 Measurement target 401 Measurement target

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Hiroyuki Sato 3-1-1, Honcho, Yonezawa-shi, Yamagata Fukushima, Room 201 F-term (reference) 5B050 BA09 DA04 DA07 EA13 EA19 EA26 5B080 GA22

Claims (9)

[Claims] 1. A three-dimensional image generating apparatus for generating a three-dimensional image by pasting a texture image to a three-dimensional shape model, comprising: a camera viewpoint direction closest to a virtual viewpoint for the three-dimensional image from a plurality of camera viewpoint images; The two camera-captured texture images are selected based on an angle between the virtual viewpoint direction and each of the two camera-captured texture images. A three-dimensional image generation apparatus, comprising: a configuration for executing image synthesis processing of both images by weighting parameters to calculate image parameters at the virtual viewpoint, and executing texture mapping based on the parameters. 2. A process for calculating parameters of an image at a virtual viewpoint, comprising: setting a shooting viewpoint 1 and a shooting viewpoint 2 of two camera shooting texture images having a shooting viewpoint direction closest to the virtual viewpoint; The angle between the vector V1 of the viewpoint 1 and the vector E of the virtual viewpoint is θ1, and the angle between the vector V2 of the imaging viewpoint 2 and the vector E of the virtual viewpoint is θ.
2, the configuration is such that α = θ1 / (θ1 + θ2) is obtained, and the image parameter at the virtual viewpoint = the image parameter of the imaging viewpoint 1 × (1−α) + the image parameter of the imaging viewpoint 2 × α. The three-dimensional image generation device according to claim 1. 3. The three-dimensional image generating apparatus according to claim 1, wherein said image parameter includes one of an RGB value and a gray value. 4. A three-dimensional image generating apparatus, comprising: executing a normalization process of an average value and a variance value of an image with respect to an image region used as a texture image in the plurality of camera viewpoint images; 2. The image processing apparatus according to claim 1, wherein a combination of the two images is performed by weighting the image parameters, and a calculation process of the image parameters at the virtual viewpoint is performed.
3. The three-dimensional image generation device according to 1. 5. A three-dimensional image generating method for generating a three-dimensional image by pasting a texture image on a three-dimensional shape model, comprising: a plurality of camera viewpoint images; An image selecting step of selecting two camera-photographed texture images having: a virtual viewpoint direction; and an angle between each of the two camera-viewpoint directions of the two camera-photographed texture images. A combination processing step of performing a combination processing of both images by weighting image parameters possessed by the image, and a texture mapping step of calculating an image parameter in the virtual viewpoint and performing a texture mapping based on the parameter. Characteristic three-dimensional image generation method. 6. A process for calculating parameters of an image at the virtual viewpoint, wherein the photographing viewpoints of two camera photographing texture images having the photographing viewpoint directions closest to the virtual viewpoint are photographing viewpoint 1 and photographing viewpoint 2. The angle between the vector V1 of the viewpoint 1 and the vector E of the virtual viewpoint is θ1, and the angle between the vector V2 of the imaging viewpoint 2 and the vector E of the virtual viewpoint is θ.
6. A value of α = θ1 / (θ1 + θ2) is calculated as 2, and an image parameter at the virtual viewpoint = image parameter of shooting viewpoint 1 × (1−α) + image parameter of shooting viewpoint 2 × α. 3. The method for generating a three-dimensional image according to item 1. 7. The method according to claim 5, wherein the image parameters include one of an RGB value and a gray value. 8. A three-dimensional image generation method, further comprising: a normalization processing step of executing a normalization process of an average value and a variance value of an image for an image area used as a texture image in the plurality of camera viewpoint images. The three-dimensional image generation method according to claim 5, further comprising: performing a synthesis process on the image after the normalization process by weighting image parameters. 9. A program providing medium tangibly providing a computer program for causing a computer system to execute a three-dimensional image generation process of generating a three-dimensional image by pasting a texture image on a three-dimensional shape model. An image selection step of selecting, from a plurality of camera shooting viewpoint images, two camera shot texture images having shooting viewpoint directions closest to a virtual viewpoint for a three-dimensional image; and the virtual viewpoint direction; A combining process step of performing a combining process of the two camera captured texture images by weighting image parameters of the two camera captured texture images based on an angle formed between each of the two captured captured image directions and the two capturing viewpoint directions; Calculate the image parameters at the virtual viewpoint, and Program providing medium characterized by having a texture mapping step of performing a texture mapping based on.