JP2003262510A - Method of measuring three-dimensional shape and three- dimensional scanner - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a three-dimensional scanner that is simple in constitution, low in price, and high in resolution and accuracy. <P>SOLUTION: This three-dimensional scanner is provided with: a camera 1; a periscope 2 rotatably attached to the camera 1; and a three-dimensional shape analyzing section 5 which analyzes the movement of a desired point on a plurality of images of an object picked up by means of the camera 1 at every prescribed rotational angle synchronously to rotation between the images upon receiving the images, finds the parallax with respect to the desired point based on the analyzed results, and finds the distance to the point from the camera 1 based on the parallax. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional shape measuring method and a three-dimensional scanner for easily measuring the three-dimensional shape of an object.

[0002]

2. Description of the Related Art In the industry, dimensional accuracy of parts is defined in order to assemble parts accurately. For the measurement, a contact-type measuring instrument with good point position accuracy or a precision sensor for fixed point measurement is mainly used. This type of measuring instrument is also used in the fields of design and trial production. By the way, in these fields, it is necessary to faithfully measure models, prototypes, and product shapes mainly composed of free-form surfaces, and a large amount of point data is required for that purpose. With conventional measuring instruments, it is difficult to obtain such a large amount of point data in a short time, and due to the limitation of computer performance for data processing,
The CAD point interpolation process based on a small amount of point data had to correspond to a free-form surface. Therefore, the model shape often changed after CAD input. In order to avoid the problem caused by this, it was necessary to have a detailed meeting among the engineers at each stage, and the skill of the operator and a lot of working time were indispensable. Recently, especially in the automobile industry, non-contact measuring instruments that realize surface measurement in a short time have been demanded in order to shorten the development process including design. Further, even in the case of parts that have been roughly measured with a small number of measurement points in the past, many of them include free-form surfaces, and efforts for detailed front surface inspection are being actively promoted. Furthermore, it is becoming more common to measure objects while moving them.

[0003]

Currently, these requirements are met by a large-scale system specialized for an object. Since all of these are based on the positive methods such as the moire method and the CT method, the system must be expensive and large-scaled for improving the resolution.

On the other hand, in the industrial world, the demand for shape measurement is increasing more and more, and a high-speed, non-contact, highly accurate three-dimensional shape measuring instrument is required. In addition, there is an increasing demand for tools for displaying three-dimensional shapes in computer graphics (CG), internet communication, and WWW. In particular, in the Internet communication technology reform currently being promoted by the government, there are three areas for CG and WWW communication.
Dimensional data input device can be an innovative technology. However, for that purpose, convenience by simplification and miniaturization of measurement equipment and reliability by cost reduction and drastic improvement of spatiotemporal resolution are essential.

The present invention has been made to solve the above problems, and an object of the present invention is to provide a simple three-dimensional shape measuring method and a three-dimensional scanner having a simple structure and a low price.

[0006]

A three-dimensional shape measuring method according to the present invention comprises a first step of preparing a camera and a periscope rotatably attached to the camera, and a second step of rotating the periscope. The third step of capturing an image of an object with the camera in synchronization with rotation and obtaining a plurality of images at each predetermined rotation angle, and analyzing the movement between the plurality of images at a desired point on the image, It comprises a fourth step of obtaining a parallax for a point and a fifth step of obtaining a distance from the camera to the point based on the parallax.

The three-dimensional shape measuring method according to the present invention comprises a first step of defining an arc or a closed curve for moving the camera, a second step of moving the camera along the arc or the closed curve, and a movement of the camera. According to the third step of photographing an object with the camera to obtain a plurality of images at each predetermined rotation angle, and analyzing the movement between the plurality of images for a desired point on the image, And a fifth step of obtaining the distance from the camera to the point based on the parallax.

A three-dimensional shape measuring method according to the present invention comprises a camera, a first step of preparing a periscope rotatably attached to the camera, and a second step of projecting speckle light onto an object to be measured. A third step of rotating the periscope, a fourth step of capturing a speckle pattern in synchronization with the rotation to obtain a plurality of images for each predetermined rotation angle, and a step between the plurality of images at a desired point on the image. And a sixth step of analyzing the movement in the above to obtain a parallax about the point, and a sixth step of obtaining a distance from the camera to the point based on the parallax.

A three-dimensional shape measuring method according to the present invention comprises a first step of defining a circular arc or a closed curve for moving a camera, a second step of projecting speckle light onto an object to be measured, and a straight line along the circular arc or the closed curve. And a third step of moving the camera, a fourth step of photographing a speckle pattern according to the movement of the camera to obtain a plurality of images at predetermined rotation angles, and a plurality of desired points on the image. The fifth step of analyzing the movement between the images to obtain the parallax for the point and the sixth step of obtaining the distance from the camera to the point based on the parallax.

Preferably, a search window having a predetermined size is prepared, a search is performed along the moving direction of the periscope or the camera, cross-correlation between the plurality of images is obtained, and based on the result of the cross-correlation. The radius or diameter of the arc relating to the movement of the point is determined, and the distance from the camera to the point is determined based on the radius or diameter of the arc.

Preferably, the center of rotation is fixed with the point of any one of the images as a base point, and a fan shape having a degree of freedom of radius only is given, and the movement of the point is associated with the fan shape under this condition. To obtain the radius or diameter of the arc.

Preferably, the XY coordinates of the center of rotation and three degrees of freedom of the radius are given, and the radius or diameter of the arc is obtained by associating the movement of the point with the sector shape under this condition.

Preferably, the displacement amount between the plurality of images is added to obtain the circumference, and the radius or diameter of the arc is obtained from this circumference.

In the three-dimensional shape measuring method according to the present invention, a first step of obtaining a first image by photographing an object with a camera, a second step of moving the camera along a predetermined locus, and the camera On the object based on a vector included in the optical flow, a third step of capturing an object to obtain a second image in step 4, a fourth step of obtaining an optical flow between the first image and the second image, and The fifth step of obtaining the distance information to the point, the sixth step of affine transforming the distance information to integrate the object into the space viewed from a specific direction, and the camera along a predetermined trajectory. A step of repeating the first step to the sixth step while moving and averaging a plurality of integrated distance information to obtain a distance to a point on the object. It is intended and a step.

In the above method (the method according to claim 9), the position information of the camera at the time of photographing needs to be known (preferably strictly). Other techniques according to the present invention require the camera to move along a circular arc or a closed curve (including a circle and an ellipse). However, even if the accuracy of the information regarding the shooting conditions such as the camera position information is low, (Shape) can be obtained. "There is an advantage that there is only one camera so there is little information about the camera position information."
It can be said that it is suitable for relatively low precision applications when used for WWW communication and the like. If the shooting conditions such as the position information of the camera can be accurately grasped, in the above method (method according to claim 9), the restriction of the movement locus of the camera can be relaxed, and any movement of the camera ( (Trajectory) (for example, the present invention can be applied even when the camera makes a linear vertical or horizontal motion).

The three-dimensional scanner according to the present invention receives from the camera an image of a camera, a periscope rotatably attached to the camera, and an object photographed at each predetermined rotation angle of the periscope, A three-dimensional shape analysis unit that analyzes movement between the plurality of images for a desired point on the image, obtains parallax for the point based on the analysis result, and obtains a distance from the camera to the point based on the parallax. And with.

A three-dimensional scanner according to the present invention includes a camera, a moving means for moving the camera along a predetermined arc or a closed curve, and an image of an object taken at a predetermined rotation angle from the camera. Receiving and analyzing movement between the plurality of images for a desired point on the image,
And a three-dimensional shape analysis unit that obtains the parallax for the point based on the analysis result and the distance from the camera to the point based on the parallax.

Further, the light source, and a diffusing means for diffusing the light from the light source to generate a speckle pattern in which the intensity distribution of the light is distributed in a granular form, and the speckle pattern is projected on the object. Then, the three-dimensional shape analysis unit may process the speckle pattern on the object.

As a means for moving the camera, it is conceivable to provide a rotating shaft facing the object, an arm attached to the rotating shaft, and a motor for driving the rotating shaft. A sensor such as a rotary encoder may be provided on the rotation shaft to obtain a rotation angle signal. However, this is not mandatory. A camera is attached to the end of the arm. The optical axis of the camera lens is preferably parallel to the rotation axis, but is not limited thereto.

The locus of movement of the camera in the present invention (excluding the invention of claim 9 and the same in the following paragraphs) will be described. In the present invention, the movement locus of the camera is preferably a circle or an arc, but the present invention is not limited to this. For example, it can be applied when the camera makes an elliptical motion. More generally, the movement locus of the camera has a closed curve, that is, even if the locus has an arbitrary shape, the same effect can be obtained if the starting point and the end point are the same. In the case of circular motion, it is relatively easy to predict the effective amount of movement because there are vectors that are obvious to reduce to a certain point and a certain amount of movement (the magnitude of the vector in the optical flow) in many image pairs. This is because the prediction is possible even with a trajectory other than a circle.

[0021]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiment 1 of the Invention FIG. 1 shows a schematic diagram of a three-dimensional scanner according to a first embodiment of the present invention. The output of the CCD camera 1 is input to the three-dimensional shape / color analysis unit 5, where the processing described below is performed to obtain the three-dimensional shape of the object 10. A periscope (periscope) 2 is attached to the lens of the CCD camera 1. The periscope 2 is an optical instrument used to place a line of sight at a distant position. Mirrors and prisms are provided inside the device to separate the line of sight. Specifically, in the periscope 2, for example, two mirrors 21 and 22 that are parallel to each other are provided.
However, the mirrors 21 and 22 do not have to be parallel, and are preferably not parallel when a relatively small object is targeted. The mirror 22 on the front of the camera is preferably 45 degrees in terms of design and others. The light from the object 10 is reflected by the mirrors 21 and 22.
Is reflected by and is guided to the lens of the camera 1 along the optical path L in FIG. The optical path from the mirror 21 to the object 10 is preferably parallel to the optical axis of the lens of the camera 1 (the invention is not limited to this).

The periscope 2 is rotatable about the optical axis of the camera 1 as shown in FIG. 1, and is rotated at a predetermined speed by a drive mechanism (not shown). Alternatively, the stepping motor rotates at a predetermined time interval (for example, according to the processing time). A rotary encoder (not shown) may be provided on the rotary shaft of the periscope 2. This allows the position (rotation angle) of the periscope 2 to be known. The information on the rotation angle is sent to the three-dimensional shape / color analysis unit 5 as necessary.

In FIG. 1, the opening of the periscope 2 on the side of the mirror 21 has a radius r around the optical axis of the lens of the camera 1.
Draw a circle (arc) C of. That is, the camera 1 can take an image of the object 10 viewed from a position r away from the original lens position. With the rotation of the periscope 2, the camera 1 can capture two or more images with parallax captured from a position separated by a maximum of 2r. That is, a mirror 22 tilted by π / 4 degrees is installed in front of the center of a single CCD camera 1, and a mirror 21 tilted at an arbitrary angle (π / 4 for simplicity) is placed outside the mirror 22. By rotating the (periscope 2) around the line of sight of the camera, multiple continuous images are taken as if the camera were rotated while keeping the line of sight constant or the object was rotated vertically to the axis of the camera. it can.

FIG. 2 is a principle diagram of stereoscopic vision. When the focal length is f and the depth direction distance from the focus to the object is z, the parallax obtained when the amount of camera movement is r (corresponding to the diameter 2r in FIG. 1) is d. The smaller the distance z is, the parallax d
Is big. This parallax d is the diameter of the circular locus drawn by each point of the object while the periscope 2 makes one rotation, and is obtained by analyzing the image. Distance from parallax d to object
z can be calculated.

In short, when the camera is rotated in parallel,
Each point on the screen draws a circle, but the radius is larger for closer objects and smaller for farther objects. Therefore, first rotate the camera once to obtain multiple images (for example, 360
PIV (particle image velocimetry) is applied to one sheet) and the transition vector is obtained by performing the transition analysis. next,
By combining these, the locus (circle or arc) of each point on the screen can be obtained. From this radius, the distance to each point is obtained as follows. z = rf / d where z: depth (distance) from the camera to the target point,
r: radius of gyration (or diameter) of camera, d: radius of measured circle, f: focal length of camera

The concept of optical flow is used to obtain the trajectory of each point of the object on the image by PIV. FIG. 3 shows the principle diagram. The optical flow is information on changes before and after movement of points on the image obtained by matching, and indicates an apparent velocity vector. In FIG. 3, the circle C1 is the first image P1
The circle C2 is shown in the second image P2 taken a predetermined time after P2. Comparing the first image P1 and the second image P2, it can be seen that the circle C has moved from left to right. The optical flow showing this is P3, and the vector V shows the movement of the circle C. A mere dot indicates no movement.

The moving directions of the light and shade patterns by the objects in the image projected on the image pickup surface of the camera that is translating the stationary outside world at a constant speed perpendicular to the optical axis are all the same, but the farther the object is, the slower it moves. And the closer the object is, the faster it moves. That is, this optical flow has distance information, and the vector amount becomes large in the one near the camera and becomes small in the one far from the camera.
The apparent velocity of the object projected on the imaging surface provides depth information of the object in the three-dimensional space.

Next, the three-dimensional shape measuring method according to the first embodiment of the present invention will be described with reference to the flowchart of FIG.

S1: The periscope 2 is rotated at a predetermined angle (for example, 1 degree). It may be rotated once or more as a whole, or may be partially rotated (rotation angle <360 °).

S2: The object is photographed by the CCD camera 1. The rotation of the periscope 2 attached to the front part of the camera lens is synchronized with the frame of the CCD camera 1, and an image is recorded for each minute rotation angle. Each image is an image captured from a slightly different viewpoint, and the parallax is recorded as information. By tracking the pattern movement of this image using the cross-correlation method, the distance to the target object surface can be calculated (S3 to S7). In addition,
If the two mirrors are not both 45 degrees, preferably 3
By performing coordinate transformation in the dimensional space, it is transformed into an image that should be obtained when the two mirrors are parallel.

S3: Select a desired area on the image. An image (for example 512x512 pixels) can be displayed on a grid (5x5 o
r 10x10 etc.), and search for corresponding points in the area by the correlation method. Unlike general stereo vision methods,
In this method, convenient points are not given in advance or searched for and processed.

S4: Track the pattern movement between a plurality of images using the cross-correlation method. The cross-correlation method often used in PIV (Particle Image Velocity Measurement Method) is applied to the calculation of the movement vector between the obtained images. The cross-correlation method is one of the methods for performing pattern matching between two images. Specifically, when it is assumed that the image A and the image B are present, a part of the image A is cut out and a pattern similar to the image A is searched for in the image B. This is a method for performing pattern matching by calculating a cross-correlation coefficient between a part of the cut-out image A and an image of arbitrary same size of the image B, and acquiring the position where this correlation coefficient is maximum. . By applying the cross-correlation method PIV between the plurality of images, it is possible to perform pattern matching between the two images and calculate the movement vector.

The movement vector analysis by the correlation method will be described in more detail. Predetermined pixel (for example, 5 × 5 pixel)
The correlation is calculated only for the known moving direction (one-dimensional) using the search window of. Since the rotation direction of the periscope 2 is known, the search direction can be limited to one dimension.
Since only one-dimensional search is required, the amount of calculation can be saved. Saving the amount of calculation is effective for this system with many images.
Another advantage of this system is that it does not require the search for epipolar lines as seen in the stereo method.

(Reference: Epipolar geometry
When an object in three-dimensional space is projected by multiple cameras, a unique geometry appears between multiple images. Such a geometry is called an epipolar geometry. In FIG. 12, X is a point in a three-dimensional space, C and C ′ are viewpoints, π and π ′ are projection planes, Σ is an epipolar plane defined by C, C ′, and X, and an epipolar plane and an image plane π. The straight line L formed by the intersection is the epipolar line, and the line e, e ′ formed by the line connecting the viewpoints C and C ′ with the image plane π, π ′ is the epipole.

Subpixel analysis is performed in order to improve accuracy in the above search. Therefore, pixel interpolation by the linear or spline method is performed. At the same time, linear interpolation is performed on the movement vector calculated for each search window of each image pair. The sub-pixel analysis is a process of interpolating and calculating the brightness when the corresponding point is not in the center of the pixel.
In the search, if the correspondence is obtained in the unit of pixel center (or search window), the correspondence accuracy decreases. Therefore, the brightness is corrected between the pixel centers, and the correspondence is not limited to the pixel centers. This is called subpixel analysis. As the interpolation at this time, linear interpolation or spline interpolation can be applied. Spline interpolation is slightly more accurate.

Further, the displacement amount below the pixel may be obtained by calculating and calculating the time differential and the spatial differential of the brightness using the displacement amount obtained by measuring the displacement amount in pixel units.

If there is a hidden line, tracking by the correlation method becomes difficult, but in order to reduce this probability, movement vector calculation is performed from both forward and reverse directions of rotation. The calculation is aborted when a reliable corresponding point cannot be obtained. In addition,
The obtained correspondence vector does not always exist at the pixel center. Further, in tracking the locus in accordance with the rotation of the target (or the camera), unless the end point of the movement vector between the first and second sheets matches the start point of the vector obtained for the second and third sheets, for example, The vector needs to be interpolated.

It should be noted that the distance analysis method, or the method of calculating the correspondence while the search window is gradually reduced or the calculation is performed while the search windows are overlapped, as is commonly used in flow visualization, is also possible. The effect can be seen depending on the target object (especially when the shooting conditions are bad, the number of shots is small, etc.). A recursive correlation method can also be used.

In addition, whichever analysis method is used, the search window is made small at the same time when the movement amount is predicted.
And / or the grid may be fine. That is, the lattice spacing may be narrowed at the same time while predicting the movement amount of the lattice points from the distance information that is temporarily calculated, instead of the vector obtained by the correlation calculation and the variance thereof.

As described above, one rotation (one rotation or more or less than one rotation) may be performed on an arbitrary point of the object on the screen while performing linear interpolation or spline interpolation on the vector (optical flow) connecting the corresponding points. Track the trajectory of.

S5: The diameter (parallax) of the arc of the pattern movement at the designated point is obtained. As a method of calculating the diameter of the arc, for example, a 1-degree-of-freedom fan-shaped fitting method,
The 3-degree-of-freedom sector fitting method and the displacement direct calculation method can be applied. However, the present invention is not limited to these.

(1) One-degree-of-freedom fan-shaped fitting method This is a method in which the center of rotation is fixed with the start frame as a base point and only one radius of radius R is given. The locus is calculated by tracing the movement vector every 1/2 turn in both the forward and reverse directions with the calculation start frame as the base point. With respect to this locus, a fan shape in which only the radius (corresponding to parallax) r is unknown with the start frame as a base point is also considered, and the fan shape is fitted by the least square method. Since this method has few degrees of freedom, it is considered that hidden lines can be easily avoided, and the integration of the movement vector error due to the tracking of the trajectory can be relaxed to some extent. However, since the forward / backward calculation is performed with the start frame as the base point, a large error occurs when the start point = end point does not hold due to the shake of the rotation of the mirror.

(2) Three-degree-of-freedom fan-shaped fitting This is a method of giving three-degrees of freedom of the XY coordinates of the rotation center and radius R. Track the movement vector for one rotation in the positive direction. Even if some deviation occurs between the start point and the end point, the distance calculation error can be small.

(3) Direct displacement calculation The displacement amount in each frame is simply added to obtain the total circumference, and the radius of gyration is determined. Calculate only one rotation in the positive direction. The size of the association vector is added in one rotation. Since this sum corresponds to 2πr, r can be obtained by dividing by 2π.

S6: Obtain the distance from the camera to the designated point based on the diameter (parallax) (see FIG. 2 and its description). The diameter of the arc corresponds to the distance from the camera 1 (the diameter decreases in inverse proportion to the distance).

S7: By repeating the above processing at any point of the image, the three-dimensional shape of the target object surface can be captured. That is, the target object can be visualized.

According to the rotary periscope type three-dimensional shape measuring instrument / method according to the first embodiment of the present invention, simple shape measurement can be performed on a human face or the like.

The apparatus and method according to the first embodiment of the invention is a rotary periscope type three-dimensional shape measuring instrument and method using a single camera. With this device and method, only one camera is required, and the system can be downsized and simplified. As an application of the present apparatus / method, three-dimensional analysis that does not require very strict calibration, such as three-dimensional WWW communication, is conceivable.

When the object and its state are suitable for the analysis and the information of the system such as the camera and the mirror is exactly obtained, the 3D shape analysis can be performed from only these two images. However, problems such as shading due to lighting and thinness of texture must be solved. On the other hand, in the device / method according to the first embodiment of the invention, since a plurality of images shifted by a small angle are used and the moving direction and the moving amount of the camera (mirror) are almost known, the shadow and It is not affected much by textures and so on.

In the method according to the present invention, the accuracy of the association problem is significantly improved by feeding back the shape calculation value. Also, one-dimensional search works well.

All known methods are weak in occlusion, but this method is strong in occlusion. This is because the method according to the present invention uses an image for each minute angle captured in a known moving direction (“hidden” in an image with only a minute angle shift).
Because it hardly occurs).

(Reference: What is the occlusion problem? As shown in FIG. 11, when the same object is viewed from different viewpoints,
The problem is that the parts seen from each viewpoint differ due to the difference in viewpoints. In FIG. 11, A and B
Each part of is only displayed on the image of one camera. )

Further, in the method according to the present invention, it is not necessary to search for the epipolar constraint condition.

Embodiment 2 of the Invention Eliminating false vectors and narrowing the search range are very important in correlation calculation. In order to solve this problem, by feeding back the distance calculation result, an erroneous vector can be removed or the search range can be narrowed down. For example, the correlation calculation is performed again with reference to the movement amount predicted value obtained from the distance information obtained from the first correlation calculation result. This process is repeated several times. The movement amount prediction is a process in which the corresponding vector is back-calculated by the above algorithm based on the obtained distance. It should be noted that if the range of movement prediction is made too small, the result is simply rounding the shape (similar to the smoothing filter), and this range is set to a predetermined ratio of the magnitude of the prediction vector (for example, this range 5 times the size). This makes it possible to improve the accuracy of correlation calculation while avoiding smoothing.
It is possible to avoid repeating the correlation calculation many times.

Third Embodiment of the Invention The above-mentioned three algorithms, the one-degree-of-freedom sector fitting method, the three-degree-of-freedom sector fitting method, and the displacement direct calculation method, have the property of accumulating matching errors during trajectory tracking. Also, the ability to avoid occlusion cannot be fully exhibited.

In order to improve such a point, (1 round 3
An affine viewed from a specific direction is obtained by processing each image pair (rather than processing the entire 60 images) to obtain a corresponding vector, calculating the distance based on this, and immediately performing affine transformation. A method of integrating into a space (for example, directly in front of the camera) can be considered. In other words, it seeks to obtain a final distance by obtaining a large amount of distance information about a specific point of an object and integrating them. This processing flowchart is shown in FIG.

According to this method, it is not necessary to use the least square method because the distance is obtained based on one movement vector obtained from two images. Since the rotation angle and the end point of the fan are known, the radius can be obtained from them. However, at this stage, the accuracy of each vector is worse than that of the above algorithm. However, an accurate distance can be obtained by adding a large number of distance information based on a large number of vectors to one affine space (S18) and obtaining an average of the distances for each of the minute spaces (S19). In this case, it is desirable to average out the distances that are too large or too small. Since the space is treated as a polygon, it is not necessary to distinguish between points that look like voxels (volume pixels) and points that do not.

In the method according to the third embodiment of the present invention, since the trajectory tracking in the case of the first embodiment of the invention is not performed, it is said that the points obtained from one image pair result in a specific point. That is not obvious. Therefore, in the third embodiment of the present invention, movement prediction is performed by performing a process of temporarily projecting in a 3D space by affine transformation, then obtaining the average thereof, and performing affine transformation again.

Fourth Embodiment of the Invention In the apparatus of FIG. 1, the optical axes thereof were parallel while the periscope 2 was rotating. According to this configuration, the camera can be easily arranged, and the locus of rotation of each point (synthetic optical flow)
Is a circle with a radius according to the distance to the object. It can be used regardless of the size of the object.

On the other hand, the optical axis of the periscope 2 may be always directed toward the object. According to this fixed gaze point configuration, the camera parameters increase compared to the case where the optical axis is parallel,
The locus does not directly become a circle, but the object is relatively close,
There is an advantage that it can be observed in a wider range.

Fifth Embodiment of the Invention By applying the speckle method to the apparatus / method according to the first embodiment of the present invention, it is possible to measure three-dimensional information of an object with high resolution and high accuracy. The speckle method is known as a method for measuring minute fluctuations in an object.

FIG. 6 shows a schematic diagram of a three-dimensional scanner according to the fifth embodiment of the present invention. The CCD camera 1, the periscope 2, and the three-dimensional shape / color analysis unit 5 are the same as those shown in FIG.

A speckle pattern is projected on the object 10. Light emitted from a laser oscillator (laser) 3 is spread by a diffuser 4 to generate a speckle pattern. The laser oscillator 3 can continuously oscillate laser light. This laser light is passed through a diffuser 4 such as an optical fiber to be measured 10
To irradiate. The laser light passing through the diffuser 4 becomes a laser light containing random speckle noise due to multiple scattering and minute defects inside. Speckle
e) means that when irradiating a rough surface or an inhomogeneous medium with highly coherent light such as a laser and observing the scattered light, the intensity distribution of the light becomes random and the surface or medium becomes granular. It is a phenomenon of appearance. An optical fiber can be used as the diffuser 4. A laser speckle pattern (LS) is projected on the object 10, and this is photographed by the camera 1.

The speckle pattern (LS) will be briefly described. White spots randomly exist on the object 10 over the entire area. When this is photographed by the camera 1 while rotating the periscope 2, the particles of the speckle pattern are displaced according to the distance from the camera 1 to the speckle pattern of the object 10. Specifically, the speckle patterns each draw an arc according to the rotation of the periscope 2, but the diameter thereof corresponds to the distance from the camera 1 (the diameter decreases in inverse proportion to the distance). This state is shown in FIG. In FIG. 7, the diameter of the arc P1 drawn by the speckle pattern LS1 is smaller than the arc P2 of the speckle pattern LS2. Therefore, the distance to the speckle pattern LS1 (more precisely, the portion of the object 10 on which the LS1 is projected) is larger than the distance to the speckle pattern LS2. This invention utilizes this property. The absolute value of the distance to the speckle pattern can be obtained with high accuracy by using the high-precision sub-pixel solution method capable of obtaining a resolution 100 times the number of camera pixels.

Next, the three-dimensional shape measuring method according to the embodiment of the present invention will be described with reference to the flowchart of FIG. S21: The speckle light is projected onto the object 10 to generate a random speckle pattern LS on the object surface. S22: The periscope 2 is rotated. It may be rotated once or more, or may be partially rotated (rotation angle <360 °).

S23: In this state, the speckle pattern is photographed by the CCD camera 1. The rotation of the periscope 2 attached to the front part of the camera lens is synchronized with the frame of the CCD camera 1, and an image is recorded for each minute rotation angle. Each image is an image captured from a slightly different viewpoint, and the parallax is recorded as information. By tracking the pattern movement of this image using the cross-correlation method, the distance to the target object surface can be calculated (S24 to S28).

S24: Designate a desired point (speckle pattern) on the image. S25: Track the pattern movement between a plurality of images using the cross-correlation method. The cross-correlation method often used in PIV (Particle Image Velocity Measurement Method) is applied to the calculation of the movement amount of the speckle pattern. By applying the cross-correlation method PIV between the plurality of images, it is possible to perform pattern matching of speckle noise between both images and calculate the amount of speckle movement.

By interpolating the cross-correlation value,
A movement amount (vector) with sub-pixel accuracy can be calculated. For example, the displacement amount of pixels or less can be obtained by calculating and calculating the time derivative and the spatial derivative of the luminance using the displacement amount obtained by measuring the displacement amount in pixel units.

S26: Find the diameter (parallax) of the arc of pattern movement at the specified point (first embodiment of the invention)
As in the case of). S27: The distance from the camera to the designated point is calculated based on the diameter (parallax). The diameter of the arc corresponds to the distance from the camera 1 (the diameter decreases in inverse proportion to the distance).
(Similar to the case of the first embodiment of the invention) S28: By repeating the above processing at any point of the image, the three-dimensional shape of the target object surface can be captured.

As described above, the fifth embodiment of the present invention is
It realizes three-dimensional measurement of surface and interface fluctuations of an object based on the speckle method. The speckle method actively gives a light pattern to an object and observes the time change of this pattern with a single camera with a single field of view to obtain a three-dimensional shape with a relatively small curvature and length in the depth direction. This is the method that has been used for. On the other hand, the present invention is based on the speckle method and introduces a periscope-type rotating mirror to capture an object to be caught from many viewpoints, thereby making it possible to obtain more complicated curvature and depth with a single camera. The method can be well adapted to a long three-dimensional shape. In addition, the present invention does not actively give a specific light pattern as in the Moire method or the conventional speckle method, but the color information originally possessed by the object is obtained by allowing a single camera to have a plurality of visual fields. It is intended to provide a passive three-dimensional shape / color measurement system using the.

According to the fifth embodiment of the present invention, the "known minute angle-shifted image" photographed using the periscope type rotary mirror is used.
On the other hand, by applying this speckle method, it is possible to accurately obtain shape data for both a stationary object and a moving object.

Since the flow visualization technology by the speckle method is a non-contact method, it is possible to instantly obtain highly accurate data without giving disturbance to the measurement target and without selecting the measurement target or environment. Is. According to the embodiment of the present invention, the superiority of the three-dimensional flow visualization technique can be applied to the general computer information input device technical field. This will establish three-dimensional image communication in the field of Internet technology, as well as CG and CAD.
Innovative improvement / shortening of industrial production / design process such as data acquisition for realization can be expected, and not only reliability improvement and automation promotion but also stable development of economic society by high efficiency / cost reduction can be expected. .

Sixth Embodiment of the Invention First Embodiment of the Invention
In 5 and 5, the periscope 2 was rotated to obtain a plurality of images with parallax. Instead of this, a circle having a radius r may be determined in advance, and the camera 1 may be moved along this circumference. FIG. 9 schematically shows a sixth embodiment of the present invention. The speckle pattern LS is photographed one after another while the camera 1 is moved along the circumference C by a driving device (not shown). The optical axis of the lens of the moving camera 1 is preferably parallel to the rotation axis.

FIG. 10 shows a processing flowchart. S31: Project speckle light on the measurement object. In addition,
This step may be replaced with the next step S32. S32: Define an arc C for capturing the speckle pattern on the object. The radius r is determined according to the distance to the object 10 and the required measurement accuracy. A large radius r is desirable when measuring a distant object with high accuracy. S33: The speckle pattern is photographed while moving the camera on the arc C to obtain a plurality of continuously photographed images.

Since the subsequent steps S34 to S38 are the same as those in the first and fifth embodiments of the invention, the description thereof will be omitted.

The present invention is not limited to the above embodiments, and various modifications can be made within the scope of the invention described in the claims, and these are also included in the scope of the present invention. It goes without saying that this is what is done.

Further, in the present specification, the means does not necessarily mean a physical means, but also includes the case where the function of each means is realized by software.
Further, the function of one means may be realized by two or more physical means, or the functions of two or more means may be realized by one physical means.

[Brief description of drawings]

FIG. 1 is a schematic diagram of a three-dimensional scanner according to a first embodiment of the invention.

FIG. 2 is an explanatory diagram of an operation principle of the three-dimensional scanner according to the first embodiment of the invention.

FIG. 3 is an explanatory diagram of an optical flow.

FIG. 4 is a processing flowchart of a three-dimensional shape measuring method according to the first embodiment of the invention.

FIG. 5 is a processing flowchart of a three-dimensional shape measuring method according to a third embodiment of the invention.

FIG. 6 is a schematic diagram of a three-dimensional scanner according to a fifth embodiment of the invention.

FIG. 7 is an explanatory diagram of the operation principle of the three-dimensional scanner according to the fifth embodiment of the invention.

FIG. 8 is a processing flowchart of a three-dimensional shape measuring method according to a fifth embodiment of the invention.

FIG. 9 is a schematic diagram of a three-dimensional scanner according to a sixth embodiment of the invention.

FIG. 10 is a processing flowchart of a three-dimensional shape measuring method according to a sixth embodiment of the invention.

FIG. 11 is an explanatory diagram of occlusion.

FIG. 12 is an explanatory diagram of epipolar geometry.

[Explanation of symbols]

1 CCD camera 2 Periscope 21, 22 Mirror 3 laser light source 4 diffuser 5 3D shape / color analysis section 10 objects

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Satoshi Someya             1-1-1 East, Tsukuba City, Ibaraki Prefecture             Joint Research Center Tsukuba Center (72) Inventor Masakazu Inaba             6-33-7-107 Minamikarasuyama, Setagaya-ku, Tokyo (72) Inventor Koji Okamoto             91 Shirahata 91 Tokai-mura, Naka-gun, Ibaraki Prefecture Todai Yayoi Dormitory             Room 203 (72) Inventor Gentaro Tanaka             91 Shirahata 91 Tokai-mura, Naka-gun, Ibaraki Prefecture Todai Yayoi Dormitory             Room 501 F term (reference) 2F065 AA53 BB05 FF04 FF56 GG04                       HH01 JJ03 JJ26 LL00 LL02                       MM25 MM28 QQ24 QQ31 QQ38

Claims (12)

[Claims] 1. A first step of preparing a camera and a periscope rotatably attached to the camera, a second step of rotating the periscope, and an object being photographed by the camera in synchronization with the rotation, and a predetermined step. A third step of obtaining a plurality of images for each rotation angle of, a fourth step of analyzing movement between the plurality of images with respect to a desired point on the image, and obtaining a parallax of the point, based on the parallax And a fifth step of obtaining a distance from the camera to the point. 2. A first step of defining an arc or a closed curve for moving the camera, a second step of moving the camera along the arc or the closed curve, and an object being moved by the camera according to the movement of the camera. A third step of capturing and obtaining a plurality of images for each predetermined rotation angle; a fourth step of analyzing movement between the plurality of images at a desired point on the image to obtain a parallax for the point; A fifth step of obtaining a distance from the camera to the point based on the parallax. 3. A camera, a first step of preparing a periscope rotatably attached to the camera, a second step of projecting speckle light onto an object to be measured, and a third step of rotating the periscope. , A fourth step of capturing a speckle pattern in synchronization with rotation and obtaining a plurality of images at each predetermined rotation angle, and analyzing movement between the plurality of images at a desired point on the image, The three-dimensional shape measuring method using the speckle method, which comprises a fifth step of obtaining a parallax of the image and a sixth step of obtaining a distance from the camera to the point based on the parallax. 4. A first step of defining a circular arc or a closed curve for moving the camera, a second step of projecting speckle light onto an object to be measured, and a third step of moving the camera along the circular arc or the closed curve. And a fourth step of capturing a speckle pattern according to the movement of the camera to obtain a plurality of images at each predetermined rotation angle, and analyzing movement between the plurality of images at a desired point on the image. A three-dimensional shape measuring method using the speckle method, which comprises a fifth step of obtaining a parallax for the point, and a sixth step of obtaining a distance from the camera to the point based on the parallax. 5. A search window having a predetermined size is prepared, a search is performed along the moving direction of the periscope or the camera, cross-correlation between the plurality of images is obtained, and the cross-correlation is performed based on the result of the cross-correlation. The tertiary according to any one of claims 1 to 4, wherein a radius or a diameter of an arc relating to the movement of the point is obtained, and a distance from the camera to the point is obtained based on the radius or the diameter of the arc. Original shape measuring method. 6. A fan shape having a degree of freedom of radius only, in which the center of rotation is fixed with the point of any image as a base point,
The three-dimensional shape measuring method according to claim 5, wherein the radius or diameter of the arc is obtained by associating the movement of the point with the sector shape under this condition. 7. The radius or diameter of an arc is obtained by giving XY coordinates of a center of rotation and three degrees of freedom of radius, and by making the movement of the point correspond to the sector shape under this condition. Item 3. The three-dimensional shape measuring method according to Item 5. 8. The three-dimensional shape measuring method according to claim 5, wherein the displacement amount between the plurality of images is added to obtain a perimeter, and the radius or diameter of the arc is obtained from the perimeter. 9. A first step of photographing a target object with a camera to obtain a first image, a second step of moving the camera along a predetermined locus, and a second step of photographing the target object with the camera. A third step of obtaining an image, a fourth step of obtaining an optical flow between the first image and the second image, and a step of obtaining distance information to a point on the object based on a vector included in the optical flow. 5 steps, 6th step of subjecting the object to a space viewed from a specific direction by affine transformation of the distance information, and 1st step through moving the camera along a predetermined trajectory. Three-dimensional shape measurement, comprising: repeating the sixth step; and averaging a plurality of integrated distance information to obtain a distance to a point on the object. Method. 10. A camera, a periscope rotatably attached to the camera, and an image of an object taken at each predetermined rotation angle of the periscope, the image being received from the camera, and a desired point on the image. A three-dimensional shape analysis unit for analyzing the movement between the plurality of images, obtaining the parallax for the point based on the analysis result, and obtaining the distance from the camera to the point based on the parallax. 11. A camera, a moving means for moving the camera along a predetermined arc or a closed curve, and an image of an object photographed for each predetermined rotation angle from the camera, and the image on the image is received. A three-dimensional shape analysis unit that analyzes the movement between the plurality of images for a desired point, obtains the parallax for the point based on the analysis result, and obtains the distance from the camera to the point based on the parallax. Original scanner. 12. A light source, and a diffusing means for diffusing light from the light source to generate a speckle pattern in which the intensity distribution of the light is distributed in a granular form, and the speckle pattern is projected on the object. The three-dimensional scanner according to claim 10 or 11, wherein the three-dimensional shape analysis unit processes the speckle pattern on the object.