JPH06102027A - Three-dimensional shape recovering method - Google Patents

Abstract

PURPOSE:To allow accurate recovery of the three-dimensional shape of an object by determining the direction where the variation of feature amount of image is high, moving the viewpoint in that direction, and then determining a correcponding point accurately. CONSTITUTION:An object is imaged, at an initial position thereof, by means of a television camera and a distributing condition in the direction where the variation of luminance is highest (direction of gradient angle) is determined for all pixels in order to prepare a histogram and then the moving direction of viewpoint is set in the direction corresponding to the peak value of the histogram. When the axis of a columnar object is set in the direction of x-axis, for example, the camera is moved along x-axis or y-axis and the object is imaged before and after movement thereof. In this regard, luminance varies in the direction of y-axis but does not vary in the direction of x-axis because of the columnar structure. Consequently, a corresponding point can be obtained by moving the viewpoint in the direction of y-axis. The camera is moved in the direction thus determined and the objected is imaged and then the image is compared with the image at initial position thus operating the depth of the corresponding point. This method allows accurate recovery of the three-dimensional shape of an object.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional image suitable for obtaining a three-dimensional image of an object from an image obtained by photographing a predetermined object at different positions by using, for example, a television camera. The present invention relates to a shape restoration method.

[0002]

2. Description of the Related Art There is known a method in which a predetermined object is photographed at different positions (viewpoints), an image obtained at that time is processed, and a three-dimensional shape of the object is restored. The object is photographed at the position A, the object moves in a predetermined plane from the position A, and the object is photographed at the position B moved by a predetermined distance. At this time, the optical axis of the lens of the TV camera that shoots the object is
It is perpendicular to the moving plane. At each position, the image of the object is obtained from the positions A and B, respectively, and the focal length f of the lens is
It will be projected on the projection plane spaced apart by only. Then, the three-dimensional shape of the object is restored from the two images obtained at the positions A and B.

Stereo matching is often used as a method for restoring the three-dimensional shape of an object from two images of a stationary object. This stereo matching is to restore the shape of an object based on the triangulation principle from the positional shift (parallax) between corresponding points of two images.

[0004]

In the conventional method, the direction of the position B from the position A, that is, the moving direction of the television camera is set to a direction in which the television camera is easy to move, such as a horizontal direction, and It was decided regardless of the shape of the. As a result, even if the television camera is moved from the position A to the position B, the corresponding point of the object cannot always be obtained accurately.

The present invention has been made in view of such a situation, and it is possible to accurately find a corresponding point of an object and thereby accurately restore the three-dimensional shape of the object. Is.

[0006]

According to the three-dimensional shape restoration method of the present invention, the same object is photographed from different viewpoints to obtain a plurality of images, the corresponding points of the object are obtained from the plurality of images, and the three points of the object are obtained.
A three-dimensional shape restoration method for restoring a three-dimensional shape is characterized in that a direction in which a change in the feature amount of an image is large is obtained and the viewpoint is moved in that direction.

The different viewpoints may have, for example, a predetermined first viewpoint and a second viewpoint obtained by at least translating or rotating the first viewpoint.

Further, the characteristic amount can be luminance information or edge information of the image.

Further, it is possible to create a histogram for the direction in which the change in the characteristic amount is large and to determine the moving direction of the viewpoint in accordance with the peak value of the histogram.

It is also possible to calculate the direction in which the change in the characteristic amount is large, divide the direction obtained by the calculation into predetermined ranges, and move the viewpoint for each section.

[0011]

In the three-dimensional shape restoration method of the above construction, the direction in which the change in the feature amount such as the brightness information and the edge information of the image is large is obtained, and the viewpoint is moved in that direction. Therefore, from the images of the viewpoint before moving and the viewpoint after moving,
The corresponding points can be accurately obtained, and the three-dimensional shape of the object can be accurately restored.

[0012]

DESCRIPTION OF THE PREFERRED EMBODIMENTS First, the principle of three-dimensional shape restoration according to the present invention will be described with reference to FIGS.

FIG. 1 shows how an object is photographed in the present invention. As shown in the figure, in the present invention, an object is photographed by a television camera at position A, for example, and an image thereof projected on a projection surface is obtained. After that, the television camera moves on the moving surface, and when it reaches the position B, the optical axis of its photographing lens is rotated and directed toward the object, and the object is photographed there. That is, in the present invention, not only the television camera is moved, but also the optical axis is rotated at the moved position as needed.

As a result, for example, as shown in FIG. 2, by photographing the object at each position (viewpoint) such as A, B, C, and D so as to surround the outer circumference of the object, the entire range of the outer circumference of the object can be obtained. Can be obtained.

Next, the parallax of an object will be described with reference to FIGS. 3 and 4. Now, as shown in FIG.
When an image of an object is taken by the television camera 1 at the position B and the position B, images as shown in FIGS. 4A and 4B are obtained corresponding to each position. The coordinates of the reference point P of the object shown in FIG. 3 are (x _p , y _p ) on the screen shot at the position A, and (x _p ', on the screen shot at the position B.
y _p ') a which was a With, reference points P parallax d _v (hereinafter represented denoted by the subscript a vector _v) is, and FIG. 4 (b)
As shown in.

[0016] The above is more three-dimensionally expressed as shown in Figs. That is, FIG. 5 shows a state of photographing at the position A. Of the coordinate axes X, Y, Z, the Z axis is set to the coordinate in the object direction, and the optical axis of the lens of the television camera 1 is arranged on this Z axis.
The coordinates of the reference point P of the object can be represented by (X _p , Y _p , Z _p ). That is, when the origin of the X, Y, and Z axes is O,
The vector P _v obtained by connecting the origin O and the reference point P can be expressed by the following equation. P _v = (X _p , Y _p , Z _p ) ... (1)

The reference point P is projected as a point q on the projection plane separated from the origin O by the focal length f on the Z axis. This projection plane is a coordinate f on the Z axis parallel to the XY plane.
It is composed of a plane. The coordinates on the projection plane are X
It is represented by a coordinate axis x parallel to the axis and a coordinate axis y parallel to the Y axis. Since the reference point P is projected as a point q on the projection plane, if the coordinates of the point q are (x _p , y _p ), the coordinate x
When the origin of y is o, the vector q _v from the origin o to the point q is projected on the projection plane of the vector P _v . Therefore, the following equation is established. q _v = (x _p , y _p ) ... (2)

As a result, the following equation is established from the above equations (1) and (2). _{x p = f (X p /} Z p) ··· (3) y p = f (Y p / Z p) ··· (4)

On the other hand, FIG. 6 shows a state in which the object is photographed at the position B. That is, position B corresponds to position A (see FIG. 6).
From the origin O) by the vector t _v coordinate axis X, Y,
It is a position (origin O ′ in FIG. 6) obtained by translating Z. And further, in this embodiment, the coordinate axes are rotated in a predetermined direction. For example, the coordinate axis is rotated about the X axis (X ′ axis in FIG. 6) in the translated state by the angle θ. The coordinate axes at the new positions are X ', Y', and Z ', respectively. The origin is O '.

When rotated by θ around the X axis, the rotation matrix R can be expressed by the following equation.

[0021]

[Equation 1]

As in the case described above, when the coordinate axes of the projection plane at this position B are x'and y ', the projection point of the reference point P on the projection plane is q'. Then, the coordinates are (x _p ', y _p '). Further, the vector P _v ′ from the origin O ′ to the reference point P is q _v ′ on the projection plane. As a result, the following equation is established. _{P v '= (X p'} , Y p ', Z p') ··· (6) q v '= (x p', y p ') ··· (7) x p' = f (X p '/ Z _p ') ・ ・ ・ (8) y _p '= f (Y _p ' / Z _p ') ・ ・ ・ (9)

Inverse matrix R of translation component t _v and rotation matrix R
⁻¹ , P _v and P _v 'have the following relationship. In ^{_{P v '= R -1 (P}} v -t v) ··· (10) wherein, R ^-1 is expressed by the following equation.

[0024]

[Equation 2]

[0025] t _v is represented by the following equation. t _v = (t _x , t _y , t _z ) ... (15)

Expressions (1), (6), and (11) are added to Expression (10).
By substituting (15) to (15), the following equation is obtained.

[0027]

[Equation 3]

Here, S _vp is defined by the following equation. S _vp ≡ (x _p , y _p , f) (17)

By substituting the equations (12) to (14) into the equation (16), the following equation is obtained. _{X p '= r v1 · (} Z p S vp -ft v) / f ··· (18) Y p' = r v2 · (Z p S vp -ft v) / f ··· (19) Z p '= R _v3 · (Z _p S _vp −ft _v ) / f (20)

In addition to the above equations (8) and (9), the equation (18)
By substituting (20) to (20), the following equation is obtained. _{x p '= f (X p} ' / Z p ') = f {(r v1 · (Z p S vp -ft v)) / (r v3 · (Z p S vp -ft v))} ··· _{(21) y p '= f} (Y p' / Z p ') = f {(r v2 · (Z p S vp -ft v)) / (r v3 · (Z p S vp -ft v))} (22)

The parallax d _v can be expressed by the following equation. _{d v = q v -q v '} = (dx, dy) ··· (23)

Further, dx and dy can be expressed by the following equations. dx = xx '... (24) dy = yy' ... (25)

In the above equations (21) and (22), the subscript p
Is a general relational expression that represents the depth of the part on the object projected on each pixel of the image and the position of the corresponding point in the image when the viewpoint is changed. Parallax d of these equations
By substituting equations (24) and (25) into the relationship between _v and the depth Z, the following equation is obtained. Z = [f {(x- dx) r v3 -fr v1} t v] / [{(x-dx) r v3 -fr v1} S v] = [f {(y-dy) r v3 -fr v2 _{} t v] / [{(} y-dy) r v3 -fr v2} S v] ··· (26)

That is, by solving the parallax distributions dx and dy using an appropriate numerical solution, the distribution of the depth Z of the object, that is, the three-dimensional shape can be restored. If the movement of the viewpoint is limited to the X direction (t _y = t _z = 0) and no rotation is given (when R is a unit matrix), Z = f ×
(Tx / dx), which corresponds to normal stereo matching.

The parallax distribution d _v between the images is calculated by the same method as the ordinary stereo matching (see, for example, Japanese Patent Laid-Open No. 3-1
No. 67678). Further, d _v can be converted into the depth distribution of the object by the equation (26). Next, an example of the calculation method will be described.

Image information F ₀ (q _v ) at two positions,
F ₁ (q _v ) may be binary (black or white) information such as edge information, or luminance information that is not affected by the difference between the distance from the object to the two viewpoints and the perspective angle of the lens. For example, in the region where the corresponding points exist, the following equation holds. F ₀ (q _v ) = F ₁ (q _v + d _v (q _v )) (27)

As shown in the following expression, the parallax d _v is an offset amount d _v0 (a predetermined value close to d _v ) and Δd _v (q _v ) (= d _v
Assuming that it is represented by the sum of −d _v0 ), d _v = d _v0 + Δd _v (q _v ) ... (28) The above equation (27) can be approximately expressed as follows. F ₀ (q _v ) = F ₁ (q _v + d _v0 + Δd _v (q _v )) = F ₁ (q _v + d _v0 ) + Δd _v (q _v ) × ΔF ₁ (q _v + d _v0 ) ... (29 )

As a result, the following equation is obtained. _{Δdx (q v) (∂F 1} / ∂x) (q v + d v0) + Δdy (∂F 1 / ∂y) (q v + d v0) = F 0 (q v) -F 1 (q v + d v0) ... (30)

On the other hand, if the following conditions (31) and (32) are substituted into the above equation (26), the following equation (33) is obtained. _{dx = d 0x + Δdx ··· (} 31) dy = d 0y + Δdy ··· (32) Z = [f {(x- (d 0x + Δdx)) r v3 -fr v1} · t v] / [{( _{x- (d 0x + Δdx))} r v3 -fr v1} · S v] = [f {(y- (d 0y + Δdy)) r v3 -fr v2} · t v] / [{(y- (d 0y + Δdy)) r _v3 −fr _v2 } · S _v ] ... (33)

The above equations (30) and (33) form simultaneous equations, and by solving them, Δdx (q _v ), Δ
dy (q _v ) can be obtained. As a result, from the above equations (33), (3), and (4), the coordinates (X, Y, Z) of the point on the object corresponding to q _v on the image F ₀ (q _v ) in the three-dimensional space Can be asked. The parallax offset d _v0 is given by taking into account the position of the object in general, and the accuracy can be improved by calculating Δ d _v again with d _v0 + Δd _v0 as d _v1 .
When the same calculation is repeated, Δd _v gradually becomes a value close to 0, an appropriate value ε is set, and the operation is terminated when the absolute value of Δd _v becomes smaller than this ε.

The above calculation is executed for each pixel of the image F ₀ .

FIG. 7 shows a flowchart for executing the above calculation. First, in step S1, the variable n is initialized to 0. This variable n represents a pixel number and takes a value equal to or less than the total number N of pixels. Next, in step S2, m is initialized to 0, and Δd _vm is an initial value Δd _v0.
Is set to. Next, in step S3, (∂F ₁ / ∂
x) (q _vn + d _vm ) and (∂F ₁ / ∂y) (q _vn + d
_vm ) operation is executed. That is, the image information is differentiated here.

Further proceeding to step S4, the simultaneous equations of the above equations (30) and (33) are calculated to obtain Δd _vm . Next, in Step S5, d _vm + Δd _vm is d
Set to _{vm + 1} .

Next, at step S6, it is judged if the absolute value of Δd _vm is smaller than a predetermined constant ε. If YES is determined in step S6, the process proceeds to step S7, and the above equation (3),
From (4) and (33), X _n , Y _n , and Z _n are calculated.
That is, the coordinates of the three-dimensional space of the object are obtained.

If NO is determined in step S6, the process proceeds to step S9, and it is determined whether or not m is smaller than M. This M means the upper limit value of the number of iterations of the iterative calculation. When m is smaller than M, the number of calculations has not reached the upper limit value yet, so the process proceeds to step S10, m is incremented by 1, the process proceeds to step S3, and the subsequent processes are repeatedly executed.

When it is determined in step S9 that the variable m has become equal to or greater than M, the process proceeds to step S8 and the magnitudes of the variables n and N are compared, as in the case where the process of step S7 is completed. . If the variable n is smaller than N, that is, if the calculation has not been completed for all pixels, the process proceeds to step S11, the variable n is incremented by 1, and step S2 is performed.
Then, the subsequent processing is repeatedly executed. When it is determined in step S8 that the variable n is equal to or greater than N, the processing of all the pixels is completed, and thus the arithmetic processing is completed.

FIG. 8 shows a hardware configuration for executing the above calculation. As shown in the figure, when the television camera 1 photographs an object, it outputs a video signal thereof. This video signal is A / D converted by the A / D converter 2 and output to the CPU 3. The CPU 3 controls each unit according to the program stored in the ROM 6. That is, the data input from the A / D converter 2 is temporarily stored in the RAM.
Store in 7. Then, the television camera 1 is controlled via the camera control unit 5, the shooting position of the television camera 1 is moved, and the image data at a plurality of positions is stored in the RAM 7. Further, the data stored in the RAM 7 is read and processed, and the processing result is output to the CRT 4 and displayed. That is, by this, the restored three-dimensional shape is displayed on the CRT 4.

If the optical axis of the television camera 1 is not rotated (the rotation matrix R is a unit matrix) and the moving direction of the viewpoint is limited to one axis in the X direction (t _y = 0, t _z = 0). Then, from equations (21) and (22), y ′ = y (dy =
0), and parallax occurs only in the X direction. Image information F ₀
In, in the other image information F ₁ , the way of changing the characteristic amount (for example, the luminance information) of the predetermined height in the X direction is deformed corresponding to the depth distribution in the X direction of the same height,
As shown in FIG. 9, the luminance values at the corresponding points of the image information F ₀ and F ₁ are the same. In addition, in FIG.
The length of the arrow represents parallax, and the length of the arrow varies depending on the position.

On the other hand, if the moving direction of the viewpoint is limited to one axis in the Y direction, parallax occurs only in the Y direction, and the way of changing the luminance in the Y direction at a predetermined horizontal position of the image information F ₀ is the image information. The horizontal position in F ₁ is deformed corresponding to the depth distribution in the Y direction.

As described above, when the moving direction of the viewpoint is limited to one axis, the direction of parallax occurs only in that axial direction,
If the brightness change does not exist in the axial direction, or even if it exists, it is difficult to find the corresponding point when the change is small.

That is, for example, as shown in FIG. 10, when a cylindrical object is arranged with its axis along the x-axis direction, it moves when the television camera 1 moves along the x-axis in the -x direction. The image shown in FIG. 11A is obtained before, and the image shown in FIG. 11B is obtained after moving.
Moreover, when the TV camera 1 is moved along the y-axis direction,
Before moving, the image shown in FIG. 11 (a) is obtained,
After moving, the image shown in FIG. 11C is obtained. Then, a predetermined height (y =
12 shows the change in luminance in the x-axis direction in y _c ).
It becomes like (a). Similarly, FIG. 12B shows changes in luminance in the y-axis direction at x = x _c of the images in FIGS. 11A and 11C.

Due to the structure of an object called a cylinder, as shown in FIG. 12, there is a luminance change in the y-axis direction, but there is no luminance change in the x-axis direction. Therefore, when the moving direction of the viewpoint is the x-axis direction, it becomes difficult to find the corresponding point from the image before moving and the image after moving, and thus it is difficult to restore the three-dimensional shape. In other words, in the case of this embodiment, it is possible to obtain the corresponding points by moving the viewpoint in the y-axis direction, but it becomes difficult to obtain the corresponding points in the x-axis direction.

When the change in the luminance in the moving direction of the viewpoint is large, the distribution characteristic of the change in the luminance as shown in FIG. 13 can be obtained corresponding to the moving direction of the viewpoint. On the other hand, although there is a luminance change in the moving direction of the viewpoint, when it is extremely small, the change in the luminance distribution obtained in response to the movement of the viewpoint is as shown in FIG. That is, when the luminance change is small, the error between the true parallax and the measured parallax increases due to noise.

From the above, it is understood that, when the three-dimensional shape is restored by stereo matching, it is preferable that the moving direction of the viewpoint is the direction in which the luminance change is large.

By the way, in the case of a columnar object as shown in FIG. 10, the direction of the distribution of the brightness change is a constant direction. However, in general, the distribution of luminance changes depends on the shape of the object and is not regular. In addition, even if the same object image is used, the manner in which the luminance changes is changed by changing the illumination device and the direction of illumination. Therefore, it is preferable that the moving direction of the viewpoint is not limited to one but plural directions.

When the corresponding point of a predetermined point on one object image is obtained from the object image from another viewpoint, it is obtained from the object image whose viewpoint has moved in the direction in which the change in luminance is the largest. , It is possible to reduce the error of the corresponding points. In this way, when the corresponding points are obtained for all the pixels on one object image, the depth distribution for each pixel can be calculated accurately, and a highly reliable three-dimensional shape can be restored. it can.

Even when the rotation of the optical axis of the television camera 1 is allowed, if the rotation amount is small, the effect on parallax is large because the change component of the luminance caused by the parallel movement is large. You can think the same way. Due to the field of view of the television camera 1, it is often not necessary to set the rotation amount of the optical axis of the television camera 1 so large when the three-dimensional shape is restored by stereo matching.

For the reference image information F ₀ (q),
When the corresponding point of the point q is obtained for another image information F ₁ (q), the viewpoint movement for taking the image information F ₁ (q) is performed in the gradient direction of the luminance at the point q (the maximum luminance change is obtained). Direction), that is, the direction of grad (F ₀ (q)). However, if the actual moving direction of the viewpoint does not greatly deviate from the gradient direction, the error at the corresponding points does not increase significantly. Therefore, since it is not necessary to take the moving direction of the viewpoint very finely, the reference image information F
For each point of ₀ (q), the gradient direction is obtained in advance,
Depending on the distribution state, if you set the angle of the moving direction of the viewpoint and the number of movements, the viewpoint will move automatically,
It is possible to restore a three-dimensional shape according to an object or environment.

Next, an example of an algorithm for restoring such a three-dimensional shape will be described.

FIG. 15 shows an example of the first algorithm. In this embodiment, first, step S21.
At, the television camera 1 is set to a predetermined initial position. This initial position can be, for example, a position where the object image comes near the center of the screen. Next, in step S22, an image obtained by shooting an object with the television camera 1 is captured from the initially set position. Then, in step S23, the distribution of the gradient angle is calculated from the captured image data. That is, the direction (gradient angle) that maximizes the luminance change is calculated for all pixels. Next, the process proceeds to step S24, and a gradient angle histogram is created corresponding to the calculation result in step S23.

FIG. 16 (a) shows an example of the histogram created in this way. In the figure, the horizontal axis represents the absolute value of the gradient angle, and the vertical axis represents the number of pixels. The gradient angle θ is θ = tan ⁻¹ (F _y / F _x ). Here, F _x and F _y are partial derivatives of the image information in the x direction and the y direction, respectively.

In the case of this histogram, it can be seen that the number of pixels having a gradient angle of θ ₀ is the largest.

Next, in step S25, the moving direction of the television camera 1 is determined. For this moving direction, the peak value of the number of pixels of the histogram obtained in step S24 is obtained, and is determined as the gradient angle corresponding to this peak value. That is, in the embodiment of FIG. 16 (a),
θ ₀ . Then, in step S26, the television camera 1 is moved in the direction determined in step S25. This moving distance is set in advance to a predetermined value (for example, 10 cm) corresponding to the size of the object photographed by the television camera 1 and the distance between the television camera 1 and the object.

Next, in step S27, the object is photographed at the moved position and its image is captured. Then, in step S28, the corresponding points are calculated from the images captured in steps S22 and S27, and the depth distance of the corresponding points is calculated. This calculation is performed for pixels whose inclination angle is within a predetermined range (for example, within ± 30 degrees) around the angle θ ₀ of the moving direction. At this time, the calculation is not performed for pixels whose gradient is smaller than a predetermined threshold value set in advance. Then, the pixel for which the solution has been obtained (the corresponding point has been obtained) is excluded from the target of the subsequent calculation. Therefore, the process proceeds to step S29, and the histogram is updated to a histogram of only pixels for which corresponding points have not yet been obtained.

That is, as shown in FIG. 16B, the pixels for which corresponding points have been obtained within the range of ± 30 degrees around the gradient angle θ ₀ are excluded from the histogram and the corresponding points are still obtained. Histograms are created only for those pixels that are absent or if not so reliable.

Next, in step S30, it is determined whether or not the determination of corresponding points has been completed for substantially all pixels. If not, the process returns to step S25 and the subsequent processing is repeated. .

In this way, FIGS. 16 (b) to 16 (d)
As shown in, the gradient angle θ ₀ is followed by the gradient angles θ ₁ , θ ₂ , and θ ₃
In order (in order of gradient angle corresponding to larger number of pixels),
An image is captured, a corresponding point is calculated, a depth distance is calculated, and a histogram is updated. Then, as shown in FIG. 16E, at the time when the total number of remaining pixels does not change (that is, a flat portion where the gradient is smaller than a predetermined threshold value, or a pixel for which a corresponding pixel cannot be obtained due to hiding). Only when left), step S
It is determined in 30 that the process has ended, and the process ends.

FIG. 17 shows another embodiment of the algorithm. In this embodiment, first step S4
1, the initial position of the television camera 1 is set.
Then, in step S42, the image at the initial position is captured. Next, in step S43, the distribution of gradient angles is calculated. Then, in step S44, a histogram is created. Step S41 above
The processing from S44 to S44 is the same as the processing from S21 to S24 in FIG. As a result,
Histogram similar to that shown in FIG. 6 (a) (FIG. 18)
Is created.

Next, in step S45, pixel classification processing is performed according to the gradient angle. That is, in this embodiment, the range of the gradient angle of 180 degrees is divided into a predetermined number m in advance so that the range is, for example, equal intervals. In this embodiment, as shown in FIG. 18, the range of the inclination angle of 180 degrees is 6
It is divided into pieces (m = 6) and the range of the gradient angle of 30 degrees is regarded as one group (range). In other words, each pixel is classified into any of the groups from m = 1 to m = 6. The movement direction is set to the angles θ _{1 to} θ ₆ of the center of each group.

Next, in step S46, m is initialized to 1. Then, in step S47, it is moved by a predetermined distance (for example, 10 cm) in the direction of the gradient angle θ ₁ within the range set in step S46. Then, in step S48, an image of the object at the position after movement is photographed, captured, corresponding points of pixels belonging to the range of m = 1 are calculated, and a distance to the corresponding points is calculated.

Next, in step S49, m is incremented by 1, and it is determined in step S50 whether this m is equal to or smaller than the maximum value M (M = 7 in this embodiment). . When m is 6 or less, the process returns to step S47, and the subsequent processes are repeated. That is, by this, m = 1 to m = 6
Corresponding points in the range up to are required. And step S5
At 0, when it is determined that m is equal to 7,
The process ends.

In the above, the brightness information is used as the feature amount of the image, but it is also possible to use a value obtained by differentiating the brightness information. Furthermore, when edge information is used as the feature amount, the above-described gradient direction is the edge normal direction.

[0073]

As described above, according to the three-dimensional shape restoration method of the present invention, the direction in which the change in the feature amount of the image is large is obtained and the viewpoint is moved in that direction, so that the corresponding points of the object are surely obtained. And the three-dimensional shape of the object can be accurately restored.

[Brief description of drawings]

FIG. 1 is a diagram illustrating a method of capturing an object by a three-dimensional shape restoration method of the present invention.

FIG. 2 is a diagram for explaining another embodiment of the object photographing method by the three-dimensional shape restoration method of the present invention.

FIG. 3 is a diagram illustrating a shooting position where the screen shown in FIG. 4 is obtained.

FIG. 4 is a diagram illustrating an image obtained when the image is captured at the position shown in FIG.

FIG. 5 is a diagram illustrating coordinate axes when a reference point is photographed at a predetermined position.

FIG. 6 is a diagram illustrating a state when the coordinates obtained in FIG. 5 are moved.

FIG. 7 is a flowchart showing processing steps of an embodiment of the three-dimensional shape restoration method of the present invention.

FIG. 8 is a block diagram showing a hardware configuration for executing the three-dimensional shape restoration method of the present invention.

FIG. 9 is a diagram illustrating a change in luminance when a viewpoint is moved.

FIG. 10 is a diagram for explaining how a television camera shoots a cylindrical object.

FIG. 11 is a diagram illustrating an image obtained when the television camera is moved as shown in FIG.

FIG. 12 is a diagram illustrating a change in luminance in the x-axis direction or the y-axis direction in FIG. 11.

FIG. 13 is a diagram illustrating a change in brightness when a large brightness gradient is provided in the direction in which the viewpoint is moved.

FIG. 14 is a diagram illustrating a change in brightness when there is a slight brightness gradient in the direction in which the viewpoint is moved.

FIG. 15 is a flowchart showing processing steps of another embodiment of the three-dimensional shape restoration method of the present invention.

16 is a diagram illustrating the processing of steps S24 and S29 of FIG.

FIG. 17 is a flowchart showing the processing steps of still another embodiment of the three-dimensional shape restoration method of the present invention.

FIG. 18 is a diagram illustrating a process of step S45 of FIG.

[Explanation of Codes] 1 TV camera 2 A / D converter 3 CPU 4 CRT 5 Camera control unit 6 ROM 7 RAM

Claims (5)

[Claims] 1. A three-dimensional shape restoration method for reconstructing a three-dimensional shape of an object by obtaining the plurality of images of the same object from different viewpoints, obtaining corresponding points from the plurality of images. A three-dimensional shape restoration method, characterized in that a direction having a large change in the feature amount of the image is obtained and the viewpoint is moved in that direction. 2. The three-dimensional structure according to claim 1, wherein the different viewpoints include a predetermined first viewpoint and a second viewpoint obtained by at least translating or rotating the first viewpoint. Shape restoration method. 3. The feature amount is brightness information or edge information of the image.
The method for restoring a three-dimensional shape according to 1. 4. The histogram according to the direction in which the change in the characteristic amount is large, and the moving direction of the viewpoint is determined in accordance with the peak value of the histogram.
The method for restoring a three-dimensional shape according to 1. 5. A method in which a direction in which the change in the feature amount is large is calculated, the direction obtained by the calculation is divided into predetermined ranges, and the viewpoint is moved in each of the divided sections. The method for restoring a three-dimensional shape according to item 2 or 3.