JPH05181980A - Method and device for three-dimensional shape restoration - Google Patents

Abstract

PURPOSE:To obtain the three-dimensional shape restoring method which can accurately restore even sudden variation of the surface of an object body when image data for restoring the three-dimensional shape of the object body are generated by using shadow information regarding two-dimensional brightness image data obtained by picking up an image of the object body. CONSTITUTION:A function block which carries out the three-dimensional shape restoring method detects the edge of two-dimensional brightness image data I(x, y) (16) and calculates parameters (k) and (m) indicating the smoothness of the surface of the object body from the edge (17). Those parameters are used to determine reflection map parameters (p) and (q) as the shadow information for accurately restoring the three-dimensional shape (27 and 29) and the reflection map parameters (p) and (q) are used to generate image data for restoring the three-dimensional shape of the object body (30).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional shape restoring method and apparatus for generating image data for restoring a three-dimensional shape of a target object from two-dimensional luminance image data of a target object having a three-dimensional shape. In particular, the 2D luminance image data of a target object having a 3D shape is used in a short time and accurately by using its shadow information.
The present invention relates to a three-dimensional shape restoration method and apparatus for generating image data for restoring a three-dimensional shape of a target object.

[0002]

2. Description of the Related Art For example, in factory automation (FA), industrial robots, etc., in order to recognize the hand gripping state of a robot, two-dimensional image data picked up by an image pickup means such as a CCD camera is obtained.
Attempts have been made to restore the original gripped state of the robot hand by performing image processing such as feature extraction on three-dimensional image data.

Two problems are encountered when generating image data for restoring the three-dimensional shape from the two-dimensional image data of a target object having such a three-dimensional shape. The first problem is how to accurately generate the image data for restoring the three-dimensional shape of the target object from the two-dimensional image data lacking the depth information. The second problem is how to quickly generate image data for reconstructing a three-dimensional shape from a very large amount of two-dimensional image data. This problem is particularly problematic for industrial robots, etc. from the viewpoint of using the three-dimensional shape restoration result in real time. Regarding this problem, a method using a high-speed information processing device capable of processing a large amount of image data at high speed, for example, a digital signal processor (DSP) is known.

In an attempt to solve the former problem, mathematically, the image data represented by a two-dimensional function f (x, y) indicating the brightness at the parameters x and y of the orthogonal two-dimensional coordinate system is logarithmically transformed. Methods are known (eg C
Q Publishing, "Interface", October 1988 issue,
Pp. 172-182). However, this method is difficult to apply from the viewpoint of real-time processing because the image processing is complicated and takes a long processing time. Further, there is a problem that it is difficult to obtain image data that accurately reconstructs a target object having a three-dimensional shape.

As another attempt, a method (“Shape from Shading”) of generating image data showing the three-dimensional shape of a target object from shadow information has been tried. The method of generating three-dimensional shape image data from this shadow information is 3
The surface can be calculated by differentiating the dimensional shape, and when the surface is converted using a reflection map, it becomes the shadow information, and the direction of the surface of the target object is obtained from the shadow information.
Furthermore, the surface thus obtained is integrated to restore the three-dimensional shape. Considering a normal vector in the direction perpendicular to the surface of the target object, illuminating the surface of the target object with illumination light at a certain angle, and observing the reflected light at another angle different from the illumination light, specifically, , Which is a map used when imaged by a video camera or the like, and shows a map in which the lighting conditions, the state of the surface of the target object, and the position of the observer are used as parameters. That is, as shown in FIG.
In, the inclination of the shape of the target object is calculated from the two-dimensional luminance image data, and at this time, appropriate reflection map parameters: p, q (or mapping parameters: f, g) used as shadow information are estimated, and the reflection map parameters: p,
3 of the original target object is obtained by integrating q in the integrating means 62.
Generate an image that restores the three-dimensional shape. As a specific device configuration, as shown in FIG. 11, a video camera 71 for picking up a target object, an analog / digital converter 72 for converting the picked-up image data into a digital value, and the converted two-dimensional luminance image data are provided. Video RAM 73 for storing,
The device comprises a work station 75 for performing image processing, which will be described later, a CRT 76 for displaying the result, and a keyboard 77 for inputting a command desired by the user.

In general, it is not possible to explicitly calculate the surface of the shape of the target object from the shadow information. Therefore, a three-dimensional shape restoration method has been attempted, assuming that the surface of the shape is smooth and assuming the continuity of the surface for regularization. When restoring the three-dimensional shape of the target object,
The above-mentioned illumination conditions, the state of the surface of the target object, and the position of the observer are problems. On the other hand, when these illumination conditions, the state of the surface of the target object, and the position of the observer are determined, the brightness can be determined by the inclination of the surface. δx, δ in the x, y, z coordinate system
When the change in the z direction is tilted by pδx in the x direction and qδy is tilted in the y direction with respect to the change in y, the brightness R (p, q is defined as parameters p and q represented by Expression 20 described later. ) Can be asked. The map used for this is called a reflection map. The two-dimensional luminance data of the target object captured by the video camera 71 or the like is I (x, y).
And the luminance data on the reflection map is R (p (x,
y), q (x, y)), I (x, y) = R (p (x, y), q (x, y)) = Rs (f, g) : P, q or mapping parameters: f, g may be determined. The relationship between the reflection map parameters: p, q and the mapping parameters: f, g will be described later.

Mapping parameters: f and g will be described. 3 and 4 show the coordinate system by the stereoscopic projection method. Considering a plane contacting a sphere with a radius of 1, the extension of the line connecting the pole at the lower end of the sphere and the intersection of the plane with the sphere Mapping position of a point intersecting with the above plane: f, g
, The position of the point where the extension of the line connecting the center of the sphere and a point on the sphere intersects the plane is the reflection map parameter: p, q
It is represented by. Then, these parameters f, g, p, q
The following relational expression holds between and. f ² + q ² = 2 ² f = 2p / [1+ (1 + p ² + q ² ) ^1/2 ] g = 2q / [1+ (1 + p ² + q ² ) ^1/2 ] p = 4f / (4-f ² -q ^{2) q = 4g / (4} -f 2 -q 2) coordinates in two-dimensional space (x, luminance with respect to y) I
For (x, y), if the brightness for the mapping parameter: f, g is R _s (f, g), the brightness Rs is Rs (f, g) = R (p (x, y) q (x , Y)). Therefore, it becomes possible to express the inclinations of all the surfaces of the three-dimensional shape of the target object using the mapping parameters: f and g. Of course, the reflection map parameters:
It is also possible to express the inclinations of all the three-dimensional shapes of the target object using p and q.

The method of generating the image data in which the three-dimensional shape of the target object is restored from the above-mentioned shadow information will be summarized using mathematical expressions. The two-dimensional luminance image data obtained when the target object is imaged by the video camera 71 is the luminance data I at an arbitrary position (x, y) in the orthogonal two-dimensional coordinate system.
It is represented as (x, y). The degree of change e _s of the curvature of the three-dimensional surface of the target object can be represented by using mapping parameters f and g as shown in Expression 1.

[Equation 1] However, fx and gx are mapping parameters in the x direction, and fy and gy are mapping parameters in the y direction. When capturing an image of a target object, the captured brightness information is greatly affected by the brightness of the lighting or the like. Equation 2 shows the fidelity e _i to the brightness of the two-dimensional image data of the target object.

[Equation 2] However, I (x, y) is the luminance data on the x, y plane, and Rs (f, g) is the mapping parameter f, g.
Is the brightness for.

Since both Equations 1 and 2 are general continuous equations, if they are rewritten as discrete equations suitable for digital signal processing, the rate of change of the curvature of the surface of the three-dimensional shape of the target object will be discrete. The degree e _s is given by the following formula.

[Equation 3] The discrete fidelity to the brightness of the two-dimensional image of the target object e _i is given by the following equation.

[Equation 4]

Here, the energy parameter e indicating the energy as the discriminant value for determining the mapping parameters f and g for accurately restoring the three-dimensional shape of the target object is expressed by the following formula. e = S _ij + λ · r _ij However, S _ij is a value for the mapping parameter shown in Expression 3, r _ij is a parameter indicating the fidelity shown in Expression 4, and λ is an evaluation coefficient. In this formula,
Mapping parameters: f, g that minimize the energy parameter e, for example, 0 or at least almost 0, may be obtained. In order to minimize the value of this energy parameter e, as shown in Equation 5 and Equation 6, the energy parameter e is partially differentiated by the mapping parameters: f _kl, g _kl , and their minimum value (0 or almost 0)
You should ask.

[Equation 5]

[Equation 6]

In equations (5) and (6), the average mapping parameters for the mapping parameters (f, g) around the center positions i, j in which the bar is added to the heads of the mapping parameters (f, g) are shown in equations (7) and (8). expressed.

[Equation 7]

[Equation 8]

In general, equations 5 and 6 are expressed by the following equations.

[Equation 9]

[Equation 10]

With reference to FIGS. 12 and 13, a conventional three-dimensional shape restoration method and a three-dimensional shape restoration apparatus using shadow information about two-dimensional luminance image data of a target object will be described. This three-dimensional shape restoring device has a color video camera 81 having a CCD, a color television 82 and a video tape recording / reproducing device (VTR) 83, an image processing device 90 connected to them, and a personal computer 84. .. The image processing device 90 includes a color video camera 81, a color television 82 or a VTR.
The two-dimensional image data from 83 is input to generate image data for restoring the three-dimensional shape of the target object. The image data is output to the personal computer 84, and the three-dimensional shape of the target object is displayed on the CRT display device or the like from the personal computer 84. Color video
Color two-dimensional luminance image data obtained by imaging the target object is output from the camera 81 or the like to the image processing device 90. However, color is not considered in this example.

An image processing apparatus 90 for generating image data for restoring the three-dimensional shape of a target object from the two-dimensional image data.
Is an image input unit 901 as an interface for inputting color two-dimensional luminance image data from the color video camera 81 and the like, and an A / D conversion unit 9 for converting the input two-dimensional image data from an analog signal to a digital signal.
02, an image data storage unit 903 for storing two-dimensional image data converted into a digital signal, and an image processing device 90
Control unit 905 that controls the overall control of the image data, an image data calculation unit 906 that performs digital image calculation processing of two-dimensional digital image data, and whether the calculation result of the de-image data calculation unit 906 is appropriate as three-dimensional restored image data. An image to be output to an external device such as a personal computer 84, which is a restored image data determination unit 907 that determines whether or not, and a three-dimensional shape image data image that is restored in the image data calculation unit 906 and temporarily stored in the image data storage unit 903. And a data output unit 904.

The image processing device 9 of this three-dimensional shape restoration device
0 generates image data showing the three-dimensional shape of the target object from the shadow information about the two-dimensional image data according to the functional block illustrated in FIG. The operation of the image processing device 90 in the three-dimensional shape restoration device shown in FIG. 12 will be described with reference to the functional blocks shown in FIG. 13 and the above mathematical expressions. Image data input means 10
5 inputs, for example, two-dimensional luminance image data I (x, y) of the target object imaged by the video camera 81. The mapping parameter generation means 101 generates mapping parameters: f and g. The average mapping parameter calculation means 102 calculates the average of the mapping parameters f and g generated by the mapping parameter generation means 101 according to equations 7 and 8. Further, the average mapping parameter calculation means 102 uses the first
Calculate the deviation of the mapping parameters shown in Section. The brightness calculation means 104 calculates the brightness Rs for the mapping parameters f and g. The brightness differential calculating means 103
Mapping parameters: Partial differentiation of the brightness Rs with respect to f and g according to Expression 5 and Expression 6 to calculate partial differential values of brightness, ∂Rs / ∂f, and ∂Rs / ∂g.

The subtracting means 106 subtracts the brightness Rs (f, g) for the mapping parameters f, g from the two-dimensional brightness image data I (x, y). That is, the subtraction means 1
06 calculates the brightness deviation ΔI = I (x, y) −Rs (f, g). The multiplication means 107 has a luminance deviation ΔI = I (x,
y) −Rs (f, g) to the luminance partial differentiation result ∂Rs / ∂f,
Multiply by ∂Rs / ∂g. Further, the multiplication means 108 multiplies the multiplication result of the multiplication means 107 by the evaluation coefficient 2λ. As a result, the second term on the right side of Expressions 5 and 6 is calculated. The subtraction unit 109 is the average mapping parameter calculation unit 10
From the result shown in the first term on the right side of Expressions 5 and 6 calculated in 2, the result shown in the second term on the right side of Expressions 5 and 6 which is the calculation result in the multiplication means 108 is subtracted. As a result, the output of the subtracting means 109 becomes the results shown in Expression 5 and Expression 6. The determination means 110 determines whether or not the calculation result of the subtraction means 109 is 0 or almost 0, which indicates a value suitable for restoring the three-dimensional shape of the target object. If the judgment means 1
When it is determined in 10 that an appropriate result is obtained, the reflection map processing means 113 calculates the parameters p and q based on the relational expression between the mapping parameters f and g and the parameters p and q. The integrating means 114 integrates the obtained parameters p and q and converts them into image data for restoring the three-dimensional shape of the target object. Subtraction means 10
When the result of 9 is not almost 0 and the judgment result of the judgment means 110 is negative, the delay means 111 delays for a predetermined time, and then the mapping parameter generation means 101 updates the mapping parameters: f and g again. Then, the above calculation and determination are repeated.

[0017]

In the three-dimensional shape restoration method and apparatus shown in FIGS. 12 and 13,
The determination means 110 has a feedback loop that returns to the mapping parameter generation means 101 via the delay means 111, and a feedback loop that returns from the determination means 110 directly to the mapping parameter generation means 101. When it is determined that the result is not sufficient to reconstruct the 3D shape of the target object, the mapping parameters:
Since f and g are updated and the convergence calculation is repeated, there is a problem that the three-dimensional shape restoration processing time of the target object becomes long. As a result, we have encountered the problem of not being able to satisfy real-time performance. In order to satisfy the real-time property, a high-priced device such as a DSP that operates at a considerably high speed is required, which has the aspect of causing new problems in terms of price. The three-dimensional shape restoration method is premised on the assumption that the surface of the target object is smooth to perform the arithmetic processing. Therefore, when attempting to restore a target object that has a shape such that the shape of the target object changes abruptly, the restored shape is not restored as the original surface with a sudden change, and a smooth surface is We have encountered a problem that the shape is restored as it has and the accuracy of the three-dimensional shape restoration of the target object is lacking.

Further, in the conventional three-dimensional shape restoring method, as shown in FIGS. 14 and 15, when generating image data for restoring the three-dimensional shape of the target object from the shadow information, the occluded contour portion The slope of the normal is given as an initial value, and that value is fixed to obtain the slope of the surface of the target object. However, especially in the case of a natural image, it is difficult to accurately obtain the normal line of the occlusion contour, and the initial value itself contains an error, so that the image data for restoring the three-dimensional shape of the target object can be accurately obtained. I am running into the problem of being difficult to obtain.

Therefore, it is an object of the present invention to provide a three-dimensional shape restoring method and a three-dimensional shape restoring apparatus that shorten the three-dimensional shape restoring processing time of the above-mentioned target object and satisfy the real-time property. Further, the present invention provides a three-dimensional shape restoring method and a three-dimensional shape restoring apparatus capable of accurately generating image data for restoring the three-dimensional shape even when the shape surface of the target object is not necessarily smooth. The purpose is to Further, according to the present invention, when the image data for reconstructing the three-dimensional shape of the target object is generated from the shadow information, even if the normal vector given as the initial value includes some error, the influence of the error is removed, An object of the present invention is to provide a three-dimensional shape restoring method and a three-dimensional shape restoring apparatus that can accurately restore the three-dimensional shape of a target object.

[0020]

In order to solve the above problems, according to the present invention, an image for reconstructing the three-dimensional shape of the target object using shadow information about two-dimensional luminance image data of the target object is captured. A three-dimensional shape restoration method for generating data, wherein the smoothness of the surface of the shape of the target object is detected from the two-dimensional luminance image data, and the restoration state of the shape of the target object is determined using this smoothness. , 3 of the target object by using the shadow information that shows the appropriate determination result
A first three-dimensional shape restoration method for generating image data for restoring a three-dimensional shape is provided. According to the present invention, there is provided an apparatus for performing the above three-dimensional shape restoration method. This first
The three-dimensional shape restoration device of, the means for providing the two-dimensional luminance image data of the target object, the means for calculating the reflection map parameter indicating the shadow of the target object for the two-dimensional luminance image data, and the adjacent reflection map parameters. Means for calculating the square of the difference, a means for calculating the brightness for the reflection map parameter, a means for calculating a differential result for the reflection map parameter with respect to the reflection map parameter, and the object from the two-dimensional brightness image data. A means for calculating the smoothness of the shape of the object, a difference between the two-dimensional luminance image and the luminance with respect to the reflection map parameter, and the product of the difference and the multiplication result of the luminance with respect to the reflection map parameter. The result obtained by multiplying the square of the difference between adjacent reflection map parameters by the parameter indicating the smoothness is added. And calculating the discriminant value, determining whether the discriminant value is minimum, and determining the discriminant value by the determining means, when the discriminant value is determined to be the minimum, the three-dimensional shape of the target object is calculated from the reflection map parameter at that time. And means for generating image data showing the shape. A mapping parameter may be used instead of the reflection map parameter.

Further, according to the present invention, there is provided a three-dimensional shape restoring method for generating image data for restoring the three-dimensional shape of the target object by using the shadow information about the two-dimensional luminance image data of the target object. , Compensating for the error of the initial value of the reflection map parameter, which is the shadow information, and determining the restoration state of the shape of the target object, and using the shadow information whose determination result shows an appropriate value, the three-dimensional shape of the target object. A second three-dimensional shape restoration method for generating image data for restoring the image is provided. According to the present invention, there is provided an apparatus for performing the above three-dimensional shape restoration method. The second three-dimensional shape restoration device provides means for providing two-dimensional luminance image data of a target object, means for calculating a reflection map parameter indicating a shadow of the target object for the two-dimensional luminance image data, and a reflection map. Means for setting an initial value of the parameter, means for correcting the initial value of the reflection map parameter using the reflection map parameter calculated above, means for calculating a square of a difference between adjacent reflection map parameters, and a reflection map Means for calculating the brightness for the parameter, means for calculating a differential result for the reflection map parameter with respect to the brightness for the reflection map parameter, and a difference between the two-dimensional brightness image and the brightness for the reflection map parameter, and the difference This multiplication result obtained by multiplying the above-mentioned reflection map parameter by the luminance differentiation result and the above reflection map parameter A means for calculating a discriminant value by adding to the square of the difference, a means for determining whether or not the discriminant value is the minimum, and, when the discriminant value is determined by the determining means to be the minimum, the above-mentioned reflection map parameter is used to calculate the discriminant value. And means for generating image data showing the three-dimensional shape of the target object.

Further, according to the present invention, there is provided a three-dimensional shape restoring method for generating image data for restoring the three-dimensional shape of the target object using the shadow information about the two-dimensional luminance image data of the target object. , The continuity of the target object is added to the discriminant for restoring the three-dimensional shape of the target object to determine the restored state of the shape of the target object, and the shadow information indicating an appropriate value is obtained as the result of this determination. A third three-dimensional shape restoration method is provided that is used to generate image data that restores the three-dimensional shape of the target object. According to the present invention, there is provided an apparatus for performing the above three-dimensional shape restoration method. This apparatus comprises means for providing two-dimensional luminance image data of a target object, means for calculating a reflection map parameter indicating a shadow of the target object for the two-dimensional luminance image data,
Regarding the means for calculating the value indicating the continuity of the surface of the target object, the means for calculating the square of the difference between adjacent reflection map parameters, the means for calculating the brightness for the reflection map parameters, and the brightness for the reflection map parameters. Means for calculating a differential result with respect to the reflection map parameter, and a difference between the two-dimensional luminance image and the luminance with respect to the reflection map parameter, and the multiplication result obtained by multiplying the difference by the differential result with respect to the reflection map parameter, A means for calculating a discriminant value by adding the square of the difference between the reflection map parameters and a value indicating the continuity of the surface of the target object; a means for determining whether the discriminant value is minimum; When the discriminating value is judged to be the minimum by the means, the image data showing the three-dimensional shape of the target object is generated from the reflection map parameter at that time. And a stage.

[0023]

In the first three-dimensional shape restoration method of the present invention, the brightness for the reflection map parameter (or mapping parameter) indicating the shadow information is obtained, and based on the brightness and the original input value obtained by imaging the target object. The smoothness of the object and the inclination of the surface are defined, and the inclination of the surface is restored to a three-dimensional shape. Using the smoothness of the shape of the target object as a parameter, a smooth portion and a non-smooth portion are identified to generate image data for restoring the three-dimensional shape of the target object. The first three-dimensional shape restoration device of the present invention implements the three-dimensional shape restoration method.

In the second three-dimensional shape restoring method, even if there is an error in the initial value of the reflection map parameter, the error is eliminated by the convergence calculation. As a result, image data that accurately restores the three-dimensional shape of the target object can be generated using the initial reflection map parameter that includes some error.
The second three-dimensional shape restoration device of the present invention implements the three-dimensional shape restoration method.

In the third three-dimensional shape restoring method of the present invention, the continuity of the surface of the target object is added to the term for calculating the discriminant value that determines the reflection map parameter for accurately restoring the three-dimensional shape of the target object. Add the parameters shown. As a result, image data for restoring an accurate two-dimensional shape in consideration of the continuity of the surface of the target object can be obtained.

[0026]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the three-dimensional shape restoring method and the three-dimensional shape restoring apparatus of the present invention will be described. FIG. 1 is a block diagram showing the configuration of a first embodiment of a three-dimensional shape restoration device of the present invention,
FIG. 2 is a functional block diagram of the first embodiment of the present invention. The three-dimensional shape restoration device 1A captures an image of a target object and displays the color 2
It has a color video camera 5 having a CCD for generating three-dimensional image data, a color television 6 for displaying two-dimensional luminance image data from the color video camera 5, a VTR 7, an image processing device 2A, and a personal computer 15. The color video camera 5, the color television 6, and / or the VTR 7 functions as the two-dimensional luminance image data output means 4 for outputting the two-dimensional image data of the target object. Any one of these is sufficient. Alternatively, instead of these exemplified devices, a unit for outputting the two-dimensional luminance image data stored in the memory in advance may be used. In the present invention, since color is not directly related, color will not be referred to in the following description.

The image processing apparatus 2A includes an image input unit 8, A /
It has a D conversion unit 9, an image data storage unit 10, an image data output unit 14, a control unit 11, an image data calculation unit 12, a restored image data judgment unit 13, and a smoothness parameter calculation processing unit 3A. This circuit configuration is similar to that of the image processing apparatus 90 shown in FIG. 12, and when described in comparison with FIGS. 1 and 12, the image input unit 8 shown in FIG. Corresponding to the input unit 901, the A / D conversion unit 9 corresponds to the A / D conversion unit 902, the image data storage unit 10 corresponds to the image data storage unit 903, and the image data output unit 14 corresponds to the image data output unit 904. Corresponding to the control unit 1
1 corresponds to the control unit 905, the image data calculation unit 12 corresponds to the image data calculation unit 906, and the restored image data determination unit 13 corresponds to the restored image data determination unit 907. The 3D shape restoration device 1A shown in FIG. 1 is provided with a smoothness parameter calculation processing section 3A which does not exist in the 3D shape restoration device shown in FIG. Image processing device 2A
The three-dimensional image data reconstructed in (1) is output to the personal computer 15, and the three-dimensional shape of the target object is displayed on the CRT display device, for example. Alternatively, for example, the grasped state of the hand of the industrial robot is recognized from the restored three-dimensional shape image data.

FIG. 2 is a three-dimensional shape restoration device 1 shown in FIG.
3 is a functional block diagram showing processing contents of an image processing apparatus 2A in A. FIG. In this embodiment, the discriminants shown in Formula 11 are used instead of the discriminants corresponding to Formulas 5 and 6. The details will be described later. The image processing device 2A shown in FIG.
Mapping parameter generation means 19, mapping parameter difference square calculation means 20, mapping parameters:
Luminance calculation means 22 for calculating the luminance Rs (f, g) for f and g, and the luminance R calculated by the luminance calculation means 22.
Luminance differentiating means 21, a subtracting means 23, a multiplying means 24, a multiplying means 25, a subtracting means 26, a judging means 27, a delay means 28, which partially differentiates s with mapping parameters: f and g.
It has a reflection map processing means 29 and an integrating means 30. The image processing apparatus 2A further includes the edge detection means 16,
It has smoothness parameter determining means 17 and evaluation coefficient correcting means 18.

The components in the image processing apparatus 2A shown in FIG. 1 are made to correspond to the means in the functional block shown in FIG. The image input unit 8 corresponds to the two-dimensional image data output unit 31, and the control unit 11 and the image data calculation unit 12 cooperate to create a mapping parameter generation unit 19, a mapping parameter difference square calculation unit 20, a brightness calculation unit 22, It corresponds to the luminance differential operation means 21, the subtraction means 23, the multiplication means 24, the multiplication means 25, the subtraction means 26, and the delay means 28. Further, the control unit 11 and the image data calculation unit 12 correspond to the reflection map processing unit 29 and the integrating unit 30. The restored image data determination unit 13 corresponds to the determination means 27. The means in the functional block shown in FIG. 2 described above corresponds to the means described with reference to FIG. 13 as the prior art. However, as will be described later, the mapping parameter difference square calculation means 20 is different from the average mapping parameter calculation means 102. The A / D conversion unit 9 and the image data storage unit 10 shown in FIG. 1 are not represented by the functional blocks shown in FIG.

The smoothness parameter calculation processing section 3A added in this embodiment corresponds to the edge detecting means 16, the parameter determining means 17, and the evaluation coefficient correcting means 18. It should be noted that the loop for performing negative feedback from the determination means 110 shown in FIG. 13 to the mapping parameter generation means 101 does not exist in the functional block shown in FIG. That is, in the three-dimensional shape restoration processing block shown in FIG. 2, there is no negative feedback loop from the determination means 27 to the mapping parameter generation means 19, and the iterative calculation in this loop is not performed.

The operation of the three-dimensional shape restoring device shown in FIGS. 1 and 2 will be described in detail below. Regarding the target object, for example, the two-dimensional luminance image data I (x, y) captured by the color video camera 5 is input to the two-dimensional image data output means 31, converted into a digital value by the A / D conversion unit 9, and the image is obtained. It is stored in the data storage unit 10. Hereinafter, the two-dimensional luminance image data I (x, y) is digital signal processed. The mapping parameter generating means 19 is a mapping parameter in the reflection map coordinate system shown in FIGS. 3 and 4:
Calculate f and g. The mapping parameter difference square calculation means 20 calculates the average of the mapping parameters: f and g shown in Expressions 7 and 8. Further, the average mapping parameter calculation means 20 uses the difference 2 between the mapping parameters f and g shown in the first term and the second term on the right side of Expression 11.
Multiplication, _{i.e., (f i + 1, j} -f i, j) 2/4 + (f i, j + 1 -f i, j) 2/4 (g i + 1, j -g i, j) 2 _{/ 4 + (g i, j} + 1 -g i, j) to calculate the ^2/4. The brightness calculation means 22 calculates the brightness Rs (f, g) for the mapping parameters: f, g. The brightness Rs (f, g) is represented by the following formula. R _s (f, g) = R (p (x, y), q (x, y)) The brightness differential calculating means 21 is a partial differential ∂Rs / ∂f, ∂Rs of the mapping parameter: f, g for the brightness Rs. / ∂g
Ask for.

The edge detecting means 16 has the two-dimensional luminance image data I output from the two-dimensional image data output means 31.
The (x, y) is differentiated with respect to the two-dimensional direction, that is, the x direction and the y direction, and the edge of the luminance is detected. The smoothness parameter determining means 17 determines the parameter k _{i, j} indicating the smoothness in the x direction and the parameter m _{i, j} indicating the smoothness in the y direction for the differential value in the edge detecting means 16. Parameter k _{i, j} indicating smoothness in the x direction
X is calculated by the edge detecting means 16.
If the differential value in the direction is large, it is considered that the smoothness is small, and the parameter k _{i, j} is made small, and if the differential value in the x direction is small, the parameter k _{i, j} is considered to be smooth and the parameter k _{i, j} is made large. y
The method for determining the parameter m _{i, j} indicating the smoothness of the direction is the same as above.

Subtracting means 2 using the above calculation results
3, the multiplying means 24, the multiplying means 25, the subtracting means 26, and the multiplying means 32 perform the calculation shown in Expression 11. Subtraction means 2
3 is the luminance deviation ΔI = I (x, y) −Rs (fi _{, j} , g
_{i, j} ) is calculated. The multiplication means 24 has a luminance deviation ΔI = I
(X, y) -Rs (f _{i, j} , g _{i, j} ) is multiplied by ∂Rs / ∂f and ∂Rs / ∂g calculated by the brightness differential calculating means 21. The multiplication means 25 adds the evaluation coefficient λ to the result of the multiplication means 24.
Ride. As a result, the value of the third term on the right side of Expression 11 is obtained. The multiplying means 32 multiplies the smoothing parameter difference square calculation result calculated by the mapping parameter difference square calculating means 20 by the smoothness parameters k _{i, j} and m _{i, j} calculated by the smoothness parameter determining means 17. As a result, the second term and the second term on the right side of Expression 11 are
The calculation result of the term is obtained. The adding means 26 is the multiplying means 25.
And the calculation result from the multiplication means 32 are added. As a result, the result shown by the discriminant 11 indicating the energy e is obtained. The discriminant 11 corresponds to the equations 5 and 6, and image data for reconstructing the three-dimensional shape of the target object by using the mapping parameters f and g when the result shown by the discriminant 11 is 0 or nearly 0. Means that is specified.

The judging means 27 judges whether or not the output result of the adding means 26 shown in the discriminant equation 11 shows the minimum value. Formula 1
When it is judged that the result shown by 1 is the minimum value, the judging means 27 drives the reflection map processing means 29 and the integrating means 30 and the reflection map processing means 29 uses the mapping parameters: f and g as in the conventional case. The reflection map parameters p and q are calculated based on the above-mentioned relational expression, and the integrating means 30
The reflection map parameters: p and q calculated in 3 are integrated to generate image data that restores the desired three-dimensional shape of the target object.

When the judging means 27 has not judged that the output result of the adding means 26 expressed by the equation 11 is still the final value, the delay means 28 delays for a predetermined time, for example,
After the time for one frame of the video signal, the mapping parameter generating means 19 sets the mapping parameters:
f and g are updated and the above-mentioned processing is performed.

As shown in FIG. 2, when the smoothness parameter determining means 17 calculates smoothness parameters k _{i, j} and m _{i, j} indicating smoothness, these smoothness parameters are used to an assessment factor λ shown in discriminant 11 corrects, omitted while performing multiplication processing and outputs the correction evaluation coefficient λ in multiplication means 25, smoothness parameter k _i in multiplication means _{32, j} and m _i, the multiplication of _j You may.

As described above, in this embodiment,
By introducing the smoothness parameter parameters k _{i, j} and m _{i, j} , the two-dimensional shape of the target object can be correctly restored even when the surface of the three-dimensional shape of the target object changes abruptly. Also, as described above, it should be noted that in the configuration shown in FIG. 2, there is no loop corresponding to the negative feedback loop from the determination means 110 shown in FIG. 13 to the mapping parameter generation means 101. That is, in the present embodiment, the brightness in the x direction and the y direction of the two-dimensional image of a certain object is differentiated, the smoothness in the x direction and the y direction is determined based on the differentiated value, and the result is used appropriately. By discriminating in (1), convergence can be speeded up and it is possible to avoid a long calculation time due to iterative calculation.

A second embodiment of the three-dimensional shape restoring method and the three-dimensional shape restoring apparatus of the present invention will be described. This embodiment is caused by the error of the initial value, which is described with reference to FIGS. 14 and 15, when the inclination of the normal line of the occluding contour is given as the initial value when restoring the three-dimensional shape from the shadow information. The inaccuracy of the three-dimensional shape restoration of the target object is improved. FIG. 5 is the second
6 is a functional block diagram of the three-dimensional shape restoring device shown in FIG. 5, and FIG. 7 shows a three-dimensional shape restoring method as a second embodiment. It is a flowchart which shows a process.

The three-dimensional shape restoration device 1B shown in FIG. 5 has almost the same structure as the three-dimensional shape restoration device 1A shown in FIG. 1, and the corresponding components are designated by the same reference numerals. The three-dimensional shape restoration device 1B shown in FIG. 5 is an image processing device 2B in place of the image processing device 2A shown in FIG. 1, but the difference from the image processing device 2A is that the smoothness in the image processing device 2A. The point is that a mapping parameter initial error component calculator 3B is provided in place of the parameter calculator 3A. The functional block shown in FIG. 6 also corresponds to the functional block shown in FIG. 2, but in FIG. 6, the initial value setting means 36, the subtracting means 37, the initial value error defining means 35, and the mapping parameter difference The difference is that the square calculation means 34, the multiplication means 38, and the addition means 33 are provided.

The outline of the processing contents of the three-dimensional shape restoration method of the second embodiment will be described with reference to the flowchart shown in FIG. Step 1: As illustrated and described in FIGS. 3 and 4,
The rotation angle of the normal line of the two-dimensional shadow image with respect to the rotation direction of the optical axis in the coordinate system based on the optical axis connecting the light source and the two-dimensional shadow image is defined as the first angle, and the ray axis and the normal line are defined. The angle formed by is defined as the second angle. Step 2: Based on these first and second angles, it is determined whether the target image information can be obtained. Step 3: When it is determined that the target image information can be obtained, image data for restoring the three-dimensional shape of the target object from the two-dimensional shadow image is generated, and the process ends. Step 4: When it is determined that the target image information cannot be obtained, the first angle is updated, and the process returns to step 2.

In the present embodiment, when the three-dimensional shape restored image data is generated from the two-dimensional shadow information, the term (parameter) defining the error: t _{i, j} is added and iteratively calculated to consider the error. Restore the three-dimensional shape. That is,
Letting the parameter t _{i, j} that defines the error, the quantity S _{i, j} that defines the smoothness of the surface at the point (i, j), and the fidelity of the shadow be r _{i, j} , they are respectively expressed by the following equations. It

[Equation 12] However, F _{i, j} is the initial value of the mapping parameter f, and G _{i, j} is the initial value of the mapping parameter g.

[Equation 13]

[Equation 14]

The evaluation parameter e to be minimized to restore the three-dimensional shape is expressed by the following equation 15.

[Equation 15] However, λ ₂ is an evaluation coefficient. Since the minimum value of the evaluation parameter e gives the image data for reconstructing the desired three-dimensional shape of the target object, the value of Expression 15 is the minimum value, 0 or almost 0.
The condition that becomes. Therefore, the following Equation 16 and Equation 17 obtained by partially differentiating Equation 15 with mapping parameters: f and g
Is repeatedly calculated.

[Equation 16]

[Equation 17] However, the average mapping parameter in Expression 16 and Expression 17 is expressed by the following expression.

[Equation 18]

[Formula 19]

According to the above calculation, even if a slight error is included in the inclination of the normal line given as the initial value, it is possible to restore the correct three-dimensional shape while removing the influence of the error included in the initial value. become. In the functional block shown in FIG. 6, the initial value setting means 36 sets the initial values F _{i, j} and G _{i, j} of the mapping parameters shown in Expression 12.
The average mapping parameter calculation means 20B performs the average mapping parameter calculation processing shown in Expression 18 and Expression 19. The initial value error defining means 35 performs the calculation shown in Expression 12. Mapping parameter difference square calculation means 34
Performs the calculation shown in Expression 13. The subtraction means 23 calculates the brightness deviation shown in Expression 14. The multiplication means 38 calculates the square and multiplies it by the first evaluation coefficient λ1. The multiplying means 24, the adding means 33, the multiplying means 25, and the initial value setting means 36 perform the calculations shown in Expressions 17 and 18. The judging means 27 judges the output of the initial value setting means 36. The processing after the determination by the determination means 27 is the same as that in the first embodiment described above.

In the present embodiment, as shown in Expression 12, the mapping parameter: f, g from the mapping parameter generating means 19 and the initial value F _{i, j} , G _{i, j} of the mapping parameter from the initial value setting means 36. By using the initial value error definition means 35 for calculating the difference between the initial value error definition means 35 and the discriminant of the expression 15, the initial values F _{i, j} , G _i, G _{i, of the} mapping parameters f, g given from the initial value setting means 36 are used _{. Even when j} has a slight inclination in the normal line, it is possible to restore the three-dimensional shape while removing the influence of the error. Therefore, in the present embodiment, in particular, even if the inclination of the normal line given as the initial value includes an error as in a natural image,
It becomes possible to restore the correct three-dimensional shape while removing the influence of the error.

A third embodiment of the three-dimensional shape restoring method and the three-dimensional shape restoring apparatus of the present invention will be described. FIG. 8 is the third
It is a figure which shows the coordinate system used for description of Example, and FIG. 9 is a block diagram of the three-dimensional shape restoration apparatus of 3rd Example. As shown in FIG. 8, there is a target object having a three-dimensional shape in the x, y, z coordinate system. If the function representing the surface of the target object is Z = z (x, y,), the slope p in the x direction of the surface of the target object is p, and the slope in the y direction is q, these slopes are expressed by Equation 20. It

[Equation 20] The heights of points 1 and 2 on the surface of the target object shown in FIG. 8 are expressed by the following equations from the definition equation 20 of p and q above.

[Equation 21]

If point 1 and point 2 are the same point, the following formula is established if the plane is continuous even if it passes through any closed loop.

[Equation 22] Therefore, the following formula is established.

[Equation 23] Using the Stokes theorem, it is expressed by the following equation.

[Equation 24] Since the symbol S in Equation 24 is an arbitrary closed loop region,
The following formula is established.

[Equation 25] On the contrary, if Expression 25 holds, it is clear that Expression 23 holds. Thus, the sufficient condition that the three-dimensional plane is continuous results in the expression 25 being satisfied.

If the two-dimensional luminance image data I (x.y) at the point (i, j) where the target object is imaged and the luminance data R (p _{i, j} , q _{i, j} ) in the reflection map are given, the target object As described above, the problem of obtaining the image data for reconstructing the three-dimensional shape of the above results in the problem of obtaining the reflection map parameters (p _{i, j} , q _{i, j} ) so as to satisfy the expression 26.

[Equation 26]

However, the reflection map parameter (p _{i, j} , q _{i, j} ) cannot be solved only by the equation 26. Therefore, as described above, it is necessary to make an assumption that the inclination of the surface of the target object changes smoothly. Furthermore, it is necessary to consider the continuity of the surface of the target object considered as described above. Here, the constraint parameter S _{i, j} for the change in the inclination of the shape of the target object, the constraint parameter C _{i, j} for the surface continuity, and the fidelity r _{i, j} for the two-dimensional luminance image data are expressed by the following equation 27. Represent

[Equation 27] The energy e at this time is represented by the following formula.

[Equation 28]

As described above, the condition for minimizing this energy e is to obtain the reflection map parameters (p, q) representing the image data for reconstructing the three-dimensional shape of the target object.
Result in seeking. Therefore, as mentioned above,
The energy e is partially differentiated with mapping parameters: f and g. The arithmetic expressions are shown in Expression 29 and Expression 30.

[Equation 29]

[Equation 30] here,

[Equation 31] Putting it another way, if the calculation shown in the following equation is repeated, the energy e
The reflection map parameter (p, q) that minimizes can be obtained.

[Equation 32]

[Expression 33] The average reflection map parameters p and q with a bar at the head in Expressions 32 and 33 are expressed by the following expressions.

[Equation 34]

[Equation 35]

After the reflection map parameters (p, q) that minimize the energy e are obtained, the values are integrated to obtain image data showing the three-dimensional shape of the target object. As described above, in the third embodiment, the energy function e shown as a discriminant for obtaining an appropriate reflection parameter (p, q) is discriminated by adding an additional term that satisfies the continuity of the surface of the target object to discriminate. The three-dimensional shape of the target object can be accurately restored.

FIG. 9 is a functional block diagram for performing the three-dimensional shape restoring method of the third embodiment described above. The entire functional block includes reflection map parameter generation means 41, average reflection map parameter calculation means 42, multiplication means 43, reflection map parameter difference / sum calculation means 44, multiplication means 45, brightness differentiating means 46, brightness calculation means 47, Energy calculation means 48, discrimination means 49, subtraction means 50, multiplication means 51,
It has an addition means 52, an addition means 53, a multiplication means 54, a switch means 55, a delay means 56, and an integration means 57.
Also in the third embodiment, as described with reference to FIGS. 1 and 5 as the first and second embodiments, there is the configuration of the three-dimensional shape restoring device, but the basic difference is In place of the image data preprocessing unit 3A and the error amount computing unit 3B described above, only the part that adds an additional term that satisfies the continuity of the target object to the energy function e as the discriminant described above is different. The configuration of the original shape restoring device is omitted.

Each of the functional blocks shown in FIG. 9 performs the processing described below. Reflection map parameter generation means 4
1 inputs initial reflection map parameters: p ₀ , q ₀ to generate reflection map parameters: p, q. The average reflection map parameter calculation means 42 calculates the average of the reflection map parameters: p and q according to the equations 34 and 35. The multiplying means 43 multiplies the average reflection map parameter calculated by the average reflection map parameter calculating means 42 by the evaluation coefficient λ ₁ . As a result, the calculation of the first term in [] on the right side of Expressions 32 and 33 is performed. The reflection map parameter difference / sum calculation means 44 uses [] on the right side of the equations 32 and 33.
The partial operation of the second term is performed. Multiplying means 45 multiplies an assessment factor lambda _2/2 of the calculation result of the reflection map parameter difference-sum operation unit 44 _performs calculation of the second term in the right side of Formula 32 and Formula 33 []. The brightness calculation means 47 calculates the brightness Rs (p, q) for the reflection map parameters: p, q. The brightness differential calculating means 46 is a brightness calculating means 47.
The brightness Rs (p, q) calculated in step S3 is used as a reflection map parameter:
Partial differentiation is performed with p and q to calculate ∂R / ∂p and ∂R / ∂q. The subtraction means 50 uses [] on the right side of Expressions 32 and 33.
The luminance deviation ΔI = two-dimensional luminance image data I (x, y) −luminance Rs shown in the third term of the above is calculated. The multiplication means 51 is
This brightness deviation ΔI = I (x, y) −Rs is added to the brightness partial differentiation result ∂R / ∂p, ∂R / ∂q from the brightness differential calculating means 46
Multiply by to obtain the value of the third term in [] on the right side of Expressions 32 and 33. The adding means 52 adds the result from the multiplying means 51 to the result from the multiplying means 45. The addition means 53 adds the multiplication result of the multiplication means 43 to the addition result of the addition means 52. As a result, the values shown in [] on the right side of Expressions 32 and 33 are calculated. The multiplication means 54 multiplies the addition result of the addition means 53 by the evaluation coefficient 1 / (λ ₁ + λ ₂ ). As a result, the calculation based on the equations 32 and 33 is performed.

The energy calculating means 48 calculates the energy e defined in equation 28. The determination means 49 determines whether or not the energy e calculated by the energy calculation means 48 is the minimum value. When it is determined that the energy e is the minimum value, the result is output to the integrating means 57 via the switch means 55, and the target object is calculated based on the reflection map parameters: p and q obtained by the above calculation. The image data for restoring the three-dimensional shape of is generated. Judgment means 49
When it is determined that the energy e does not indicate the minimum value, the calculation result of the multiplication means 54 is delayed by the delay means 56 for a predetermined time, and again the reflection map parameter: p, q in the reflection map parameter generation means 41. Is updated, and the above-mentioned calculation is repeated. Delay means 56
The delay time at is a delay time corresponding to one frame of the video signal.

The mutual relation of the functional blocks of the first to third embodiments will be touched upon. The functional block diagram shown in FIG. 9 is also similar to the functional block shown in FIG. 2 or 6 described above. However, in FIG. 9, FIG.
In place of the mapping parameters: f and g shown in (1), reflection map parameters: p and q are used for the expression. For example, the correspondence between the means shown in FIG. 9 and the means shown in FIG. 6 is as follows. Means in FIG . 9 Means in FIG. 6 Reflection map parameter generation means 41 Mapping parameter generation means 19 Average reflection map parameter calculation means 42 Average mapping parameter calculation means 20B Reflection map parameter difference / sum calculation means Mapping parameter difference square calculation 44 Means 34 Brightness differential calculation means 46 Brightness differential calculation means 21 Brightness calculation means 47 Brightness calculation means 22 Subtraction means 50 Subtraction means 23 Multiplication means 51 Multiplication means 24 Addition means 52, 53 Addition means 33 Multiplication means 54 Multiplication means 25 Judgment means 49 Judgment means 27 Delay means 56 Delay means 28 Integrator means 57 Integrator means 30 Since the expression forms are different and the arithmetic processing contents do not necessarily match, a perfect correspondence relationship is not established, but when considered as a large functional block, a certain correspondence relationship Is in The functional block shown in FIG. 2 and the functional block shown in FIG. Similarly, FIG.
There is also a correspondence relationship between the functional block shown in FIG. 9 and the functional block shown in FIG.

The above-described first to third embodiments are (a) a three-dimensional shape restoration method capable of accurately restoring the three-dimensional shape of the target object even when the shape of the target object changes abruptly. And its device, (b) mapping parameters: f, g
3D shape restoration method and apparatus capable of accurately restoring the 3D shape of a target object without being affected by the error even if there is an error in the initial value of, and (c) energy e used as a discriminant The three-dimensional shape restoration method and apparatus that can more accurately restore the three-dimensional shape of the target object by providing additional terms that satisfy the continuity of the surface of the object have been described individually. The three-dimensional shape restoring method and the three-dimensional shape restoring apparatus of the present invention can take various embodiments by appropriately combining these first to third embodiments. In these embodiments, since the common processing means is present as described above as an example of the correspondence relationship, the common processing means naturally processes the calculation in common.

[0056]

As described above, according to the present invention, it is possible to generate image data that accurately restores a three-dimensional shape of a target object even when there is a rapid change in the three-dimensional shape. You can Further, according to the present invention, even if the initial value of the mapping parameter includes an error, it is possible to generate image data that restores the three-dimensional shape of the target object while correcting the error, and set the initial value of the mapping parameter. It has the advantage that it does not require much labor. Further, according to the present invention, it is possible to generate the image data that restores the highly realistic three-dimensional shape of the target object in consideration of the continuity of the target object. The three-dimensional shape restoration method and the three-dimensional shape restoration device of the present invention can generate image data for restoring the three-dimensional shape of a target object in a short time, and provide three-dimensional shape restored image data satisfying real-time property. it can.
Further, according to the three-dimensional shape restoration method of the present invention, a high-speed signal processing device such as a DSP is not necessarily required, and it is possible to provide image data in which the signal processing device restores the three-dimensional shape of the target object at low cost.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a first embodiment of a three-dimensional shape restoration device of the present invention.

FIG. 2 is a diagram showing, as functional blocks, processing contents of a first embodiment of a three-dimensional shape restoring method of the present invention and an operation of the three-dimensional shape restoring device shown in FIG.

FIG. 3 is a cross-sectional view of a coordinate system of mapping parameters showing shadow information applied in the method for restoring a three-dimensional shape using shadow information of the present invention.

FIG. 4 is a perspective view of a coordinate system of mapping parameters showing shadow information applied in the method for restoring a three-dimensional shape using shadow information of the present invention.

FIG. 5 is a configuration diagram of a second embodiment of a three-dimensional shape restoration device of the present invention.

FIG. 6 is a diagram showing, as functional blocks, processing contents of a second embodiment of a three-dimensional shape restoring method of the present invention and operation of the three-dimensional shape restoring device shown in FIG.

FIG. 7 is a flowchart showing the processing of the three-dimensional shape restoration method of the second embodiment.

FIG. 8 is a configuration diagram of a second embodiment of the three-dimensional shape restoration device of the present invention.

FIG. 9 is a diagram showing, as functional blocks, processing contents of a second embodiment of a three-dimensional shape restoring method of the present invention and operation of the three-dimensional shape restoring device shown in FIG.

FIG. 10 is a block configuration diagram for generating image data for restoring a three-dimensional shape of a target object from conventional shadow information.

11 is a configuration diagram of a conventional three-dimensional shape restoration device that realizes the three-dimensional shape restoration method of FIG.

FIG. 12 is a configuration diagram of a conventional three-dimensional shape restoration device.

FIG. 13 is a functional block diagram of a conventional three-dimensional shape restoration method and three-dimensional shape restoration device.

FIG. 14 is a plan view showing a normal line of a shielding contour portion when a three-dimensional shape is restored from conventional shadow information.

FIG. 15 is a perspective view of the shielding contour shown in FIG.

[Explanation of symbols]

1A, 1B ··· Three-dimensional shape restoration device 2A, 2B · · Image processing device 3A · · Smoothness parameter arithmetic processing unit 3B · · Mapping parameter error amount arithmetic unit 4 · · 2D luminance image data output means 5 · · Color Video camera 6. Color TV
7 ... VTR 8 ... Image input unit 9 ... A / D conversion unit 10 ... Image data storage unit 11 ... Control unit 12 ... Image data calculation unit 13 ... Restored image data determination unit 14 ... Image data Output unit 15 Personal computer 16 Edge detection unit 17 Smoothness parameter determination unit 18 Evaluation coefficient correction unit 19 Mapping parameter generation unit 20 Mapping parameter difference square calculation unit 20B Average Mapping parameter calculating means 21. Brightness differential calculating means 22. Brightness calculating means 23. Subtracting means 24. Multiplying means 25. Multiplying means 26 .. Addition means 27 .. Judging means 28 .. Delaying means 29 .. Reflection map processing means 30-Integration means 31-Two-dimensional image data output means 32-Multiplication means 33-Adding means 34- Squared difference calculation means 35 for the difference in the wrapping parameters 35 Initial value error definition means 36 Initial value setting means 37 Subtraction means 38 Multiplication means 41 Reflection map parameter generation means 42 Average reflection map parameter calculation Means 43. Multiplication means 44. Reflection map parameter difference / sum calculation means 45. Multiplication means 46. Brightness differential calculation means 47. Brightness calculation means 48. Energy calculation means 49. Judgment means 50. Means 51. Multiplication means 52. Addition means 53. Addition means 54. Multiplication means 56 .. Delay means 57 .. Integration means

[Equation 11]

Claims (6)

[Claims] 1. A three-dimensional shape restoration method for generating image data for restoring a three-dimensional shape of a target object by using shadow information about two-dimensional luminance image data obtained by capturing an image of the target object. The smoothness of the surface of the shape of the target object is detected from the image data, the restoration state of the shape of the target object is discriminated using this smoothness, and the discrimination result shows an appropriate value. A three-dimensional shape restoring method for generating image data for restoring a three-dimensional shape of an object. 2. A means for providing two-dimensional luminance image data of a target object, a means for calculating a reflection map parameter indicating a shadow of the target object for the two-dimensional luminance image data, and a difference between adjacent reflection map parameters. Means for calculating the square, means for calculating the brightness for the reflection map parameter, means for calculating a differential result for the reflection map parameter with respect to the reflection map parameter, and shape of the target object from the two-dimensional brightness image data. Means for calculating the smoothness of the two-dimensional luminance image and the difference between the two-dimensional luminance image and the luminance with respect to the reflection map parameter, the difference is multiplied by the differential result of the luminance with respect to the reflection map parameter, and the multiplication result and the adjacent reflection Discriminant value by adding the square of the difference of map parameters and the result of multiplying the parameter indicating the smoothness Image data showing the three-dimensional shape of the target object from the reflection map parameter at that time when the judgment value is judged to be the minimum by the calculating means, the judgment means for judging whether the judgment value is the minimum, and the judgment means. And a three-dimensional shape restoration device having a means for generating. 3. A three-dimensional shape restoration method for generating image data for restoring the three-dimensional shape of the target object using the shadow information about the two-dimensional luminance image data obtained by imaging the target object, which is shadow information. Compensate the error of the initial value of the reflection map parameter, determine the restoration state of the shape of the target object,
A three-dimensional shape restoration method for generating image data for restoring the three-dimensional shape of the target object by using the shadow information indicating the determination result having an appropriate value. 4. A means for providing two-dimensional luminance image data of a target object, a means for calculating a reflection map parameter indicating a shadow of the target object with respect to the two-dimensional luminance image data, and an initial value of the reflection map parameter. Means, a means for correcting the initial value of this reflection map parameter using the reflection map parameter calculated above, a means for calculating the square of the difference between adjacent reflection map parameters, and a luminance for the reflection map parameter. Means, a means for calculating a differentiation result of the luminance with respect to the reflection map parameter with respect to the reflection map parameter, and a difference between the two-dimensional luminance image and the luminance with respect to the reflection map parameter, and the difference with respect to the luminance for the reflection map parameter. Multiply this differential result by adding the squared difference of the reflection map parameters When the determination value is determined to be the minimum by the means for calculating another value, the determination means for determining whether the determination value is the minimum, and the determination means, the three-dimensional shape of the target object is calculated from the reflection map parameter at that time. A three-dimensional shape restoration device having means for generating image data shown. 5. A three-dimensional shape restoration method for generating image data for restoring a three-dimensional shape of the target object using shadow information about two-dimensional luminance image data obtained by capturing the image of the target object. The continuity of the target object is added to the discriminant for restoring the three-dimensional shape to determine the restored state of the shape of the target object, and the target object is determined by using the shadow information in which the determination result shows an appropriate value. 3D shape restoration method for generating image data for restoring 3D shape of 6. A means for providing two-dimensional luminance image data of a target object, a means for calculating a reflection map parameter indicating a shadow of the target object for the two-dimensional luminance image data, and continuity of the surface of the target object. Is calculated, a means for calculating the square of the difference between adjacent reflection map parameters, a means for calculating the brightness for the reflection map parameter, and a differential result for the reflection map parameter with respect to the brightness for the reflection map parameter. A means for calculating, a difference between the two-dimensional luminance image and the luminance with respect to the reflection map parameter is calculated, and the difference is multiplied by a differential result of the luminance with respect to the reflection map parameter. The multiplication result is squared with a difference between the reflection map parameter. And a means for calculating a discriminant value by adding a value indicating the continuity of the surface of the target object, A three-dimensional shape having means for determining whether or not the determination value is determined by the determination means, and means for generating image data indicating the three-dimensional shape of the target object from the reflection map parameter at that time. Restoration device.