JP2010071782A - Three-dimensional measurement apparatus and method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately measure the three-dimensional shape of a mirror object, without being affected by the difference in the positions or the characteristics of cameras. <P>SOLUTION: A three-dimensional measuring device for measuring the three-dimensional shape of a mirror object to be measured comprises: a plurality of cameras; character-acquiring means for acquiring the direction of a normal, which is a physical features of the surface of the object to be measured, from each photographed image; corresponding pixel search means for searching the corresponding pixels among the images by using the physical features; and a measuring means for performing three-dimensional measurement, based on the parallax of the corresponding picture elements. The three-dimensional measuring device, preferably, comprises coordinate conversion means for converting the direction of the normals of the images to a common coordinate system. The parameters for this coordinate conversion can be calculated from parameters obtained at the calibration of the camera. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for measuring a three-dimensional shape of a measurement object, particularly a specular measurement object.

  As shown in FIG. 12, three-dimensional measurement (triangulation) is a technique for measuring a distance by examining the correspondence between pixels from images taken by a plurality of cameras having different imaging angles and calculating parallax. . Normally, the luminance value is used as the feature amount when examining the corresponding pixel.

  Here, when the measurement target is a specular object, the luminance value captured in the image does not represent the feature amount of the object surface itself, but is determined by the reflection of surrounding objects. Therefore, as shown in FIG. 13, when a specular object is photographed by the two cameras 101 and 102, the position of the object surface where the light emitted from the light source L1 is reflected is different. When these points are three-dimensionally measured as corresponding pixels, the point L2 in the figure is actually measured, and an error occurs. This error increases as the difference between the imaging angles of the cameras increases. In addition, errors due to differences in camera characteristics also occur.

In order to eliminate the influence of errors due to differences in camera position and characteristics, a normal map is obtained by illuminance difference stereo, and regions are divided from this normal map, and association is performed for each region using an average normal. Thus, a technique for performing three-dimensional surveying is known (Patent Document 1).
Japanese Patent Laid-Open No. 61-198815

  However, in the case of the prior art as described above, the following problems have occurred.

  When the conventional three-dimensional surveying method is applied to a specular object using the brightness value as a feature value, the brightness value of the captured image is affected by the difference in characteristics of the multiple cameras and the camera arrangement, which causes an error in pixel matching. End up. When the surface of the measurement target is a mirror surface, this influence becomes large.

  In the method described in Patent Document 1, attention is paid to information unique to a measurement object called a normal line, so that errors due to differences in camera arrangement and characteristics can be reduced. Arise. In particular, a measurement object having a smooth continuous surface such as a sphere can be measured only as an angular three-dimensional shape with a surface resolution rough by area division. In addition, when the correspondence is taken, it is assumed that the convergence angle of the camera is reduced and the same coordinate system is used between multiple cameras. Therefore, if the convergence angle is increased, the accuracy of the correspondence will be increased due to the difference in the normal coordinate system. Will fall.

  The present invention has been made in view of the above circumstances, and the object of the present invention is a technique that can accurately measure the three-dimensional shape of a specular object without being affected by differences in camera position and characteristics. Is to provide.

In order to achieve the above object, a three-dimensional measurement apparatus according to the present invention is a three-dimensional measurement apparatus that measures a three-dimensional shape of a measurement object that is a specular object, and is photographed by a plurality of cameras and the plurality of cameras. A feature acquisition unit that acquires a physical feature of the surface of the measurement object from each of the images obtained, and a corresponding pixel that searches for a corresponding pixel among images captured by the plurality of cameras using the physical feature Search means and surveying means for performing three-dimensional surveying based on the parallax between corresponding pixels are provided.

  An error occurs when pixel correspondence is achieved using luminance information reflected on the surface of a specular object because luminance information is not a feature of the surface of the specular object itself, but information that changes depending on conditions such as ambient lighting. is there. Therefore, in the present invention, by acquiring the physical characteristics of the surface of the specular object and taking the pixel correspondence using this characteristic, high-precision matching is possible without being affected by the position and orientation of the camera, Therefore, it is possible to accurately measure the three-dimensional shape of the measurement object.

  As the physical characteristics of the surface of the measurement object as described above, it is preferable to use the direction of the surface normal. In addition to the normal line, spectral characteristics and reflection characteristics of the surface of the measurement object may be used. Since these physical features are information unique to the measurement object, they are not affected by the position and orientation of the camera.

  In the present invention, it is preferable to further include coordinate conversion means for converting a coordinate system of images taken by a plurality of cameras into a common coordinate system using a conversion parameter. In this case, the corresponding pixel search means searches for a corresponding pixel between the images using the direction of the normal line converted into the common coordinate system by the coordinate conversion means.

  By performing the matching after performing coordinate transformation processing that unifies the coordinate system among a plurality of captured images, the matching accuracy does not decrease even if the convergence angle between the cameras is increased. Therefore, the arrangement of the cameras can be determined more flexibly.

  Note that the conversion parameters in the coordinate conversion means are preferably extracted from parameters obtained by camera calibration (calibration) performed in advance.

  In addition, it is preferable that the corresponding pixel search unit in the present invention searches for a corresponding pixel between images by comparing physical features in a predetermined area including the pixel of interest. By performing comparison including the surrounding physical features, more accurate matching can be performed.

  Note that the present invention can be understood as a three-dimensional measuring apparatus having at least a part of the above means. The present invention can also be understood as a three-dimensional measurement method including at least a part of the above processing, or a program for realizing such a method. Each of the above means and processes can be combined with each other as much as possible to constitute the present invention.

  According to the present invention, it is possible to accurately measure the three-dimensional shape of a specular object without being affected by the difference in camera position and characteristics.

  Exemplary embodiments of the present invention will be described in detail below with reference to the drawings.

(First embodiment)
<Overview>
FIG. 1 is a diagram showing an outline of a three-dimensional measuring apparatus according to this embodiment. FIG. 2 is a diagram illustrating functional blocks of the three-dimensional measurement apparatus according to the present embodiment. As shown in FIG. 1, the measurement object 4 arranged on the stage 5 is photographed by two cameras 1 and 2. The measurement object 4 is irradiated with white light from three illuminating devices 3a to 3c from different directions. The illuminating devices 3a to 3c are sequentially irradiated, and each of the cameras 1 and 2 displays three images. Take a picture. The captured image is taken into the computer 6 and subjected to image processing to perform three-dimensional measurement.

  The computer 6 functions as a surface shape calculation unit 7, a coordinate conversion unit 8, a corresponding point calculation unit 9, and a triangulation unit 10 as shown in FIG. Note that some or all of these functional units may be realized by dedicated hardware.

<Constitution>
[Camera layout]
FIG. 3 is a diagram for explaining the arrangement of the cameras. As shown in FIG. 3, the camera 1 captures the measurement object 4 from the vertical direction, and the camera 2 captures the measurement object 4 from a direction shifted by 40 degrees from the vertical direction.

[Lighting arrangement]
FIG. 4 is a diagram illustrating the arrangement of the lighting devices 3a to 3c. FIG. 4A is a diagram viewed from the vertical direction, and is a diagram illustrating the azimuth angle arrangement of the lighting devices 3a to 3c. FIG. 4B is a diagram seen from the lateral direction, and is a diagram illustrating the zenith angle arrangement of the illumination devices 3a to 3c. As shown in the figure, the three illumination devices 3a to 3c irradiate the measurement object with light from directions in which the azimuth angles are different from each other by 120 degrees and from a direction in which the zenith angle is 40 degrees.

  The arrangement of the cameras 1 and 2 and the illumination devices 3a to 3c described here is merely a specific example, and it is not always necessary to use such an arrangement. For example, it is not necessary to make the azimuth angles of the lighting device uniform. Moreover, although the zenith angles of the camera and the lighting device are the same, they may be different.

[Surface shape (normal) calculation]
The surface shape calculation unit 7 is a functional unit that calculates the direction of the normal line at each position of the measurement target from three images captured by the cameras 1 and 2. A more detailed functional block diagram of the surface shape calculation unit 7 is shown in FIG. As shown in the figure, the surface shape calculation unit 7 includes an image input unit 71, a normal / luminance table 72, and a normal calculation unit 73.

  The image input unit 71 is a functional unit that receives input of images taken by the cameras 1 and 2. The image input unit 71 converts analog data from the cameras 1 and 2 into digital data using a capture board or the like. The image input unit 71 may receive an image of digital data through a USB terminal, an IEEE1394 terminal, or the like. In addition, a configuration in which an image is read through a LAN cable or from a portable storage medium may be employed.

  The normal line-luminance table 72 is a storage unit that stores a correspondence relationship between the direction of the normal line and the luminance values in images captured by sequentially lighting the three illumination devices 3a to 3c. Note that this normal-luminance table 72 is prepared for each camera, and in this embodiment, two normal-luminance tables are used corresponding to the cameras 1 and 2.

  A method of creating the normal-luminance table 72 will be described with reference to FIG. First, the illumination devices 3a to 3c are sequentially turned on for an object whose surface shape is known, and three images 10a to 10c are taken. At this time, since the sphere has normals in all directions and the direction of the normal at each position can be easily calculated by calculation, it is preferable to target the sphere. Further, it is necessary that the reflection characteristics of the target used for creating the normal-luminance table and the measurement target for actually calculating the normal are the same and constant.

For each position of the table creation images 10a to 10c, the normal direction (the zenith angle θ
And the azimuth angle φ) and the luminance values (La, Lb, Lc) in each image are acquired and stored in association with each other. By associating the direction of the normal and the combination of the luminance value with all the points in the photographed image, the normal-luminance table 72 storing the combination of the direction of the normal and the luminance value can be created for all the directions of the normal. .

  As shown in FIG. 7, the normal line calculation unit 73 calculates the direction of the normal line at each position of the measurement object 4 from the three images 11 a to 11 c photographed by sequentially lighting the illumination devices 3 a to 3 c. To do. More specifically, the normal calculation unit 73 acquires a combination of luminance values at each position from the three input images 11a to 11c, and stores a normal-luminance table 72 corresponding to the camera that captured the image. Referring to the normal direction at that position.

[Coordinate transformation processing]
The coordinate conversion unit 8 represents the direction of the normal calculated from the images photographed by the cameras 1 and 2 by the coordinate conversion process in a unified coordinate system. Since the direction of the normal obtained from the images taken by the cameras 1 and 2 is expressed in the respective camera coordinate systems, an error occurs when compared as they are. In particular, this error increases when the difference in the imaging direction of the camera is large.

  In this embodiment, the coordinate system is unified by converting the direction of the normal line acquired from the image of the camera 2 taken from obliquely above into the coordinate system of the camera 1. However, the normal direction acquired from the image of the camera 1 may be converted to the coordinate system of the camera 2 to unify the coordinate system, or the normal direction acquired from the images of the cameras 1 and 2 may be different. You may unify the coordinate system by converting to the coordinate system.

When the camera model in the present embodiment that a positive projection, as shown in FIG. 8, to convert the world coordinate system (X, Y, Z) of the camera 1 of the coordinate system _{(x a, y a, z} a) to If the rotation matrix is R ₁ , and the rotation matrix for converting the world coordinate system (X, Y, Z) to the two coordinate systems (x _b , y _b , z _b ) of the camera is R ₂ , the coordinate system of the camera 2 is The rotation matrix R ₂₁ to be converted into the coordinate system of the camera 1 is R ₂₁ = R ₂ ⁻¹ · R ₁ .

  Further, the following calibration parameters are obtained in the camera calibration performed in advance.

なお、ｘ_１，ｙ_１はカメラ１の撮影画像内における座標、ｘ_２，ｙ_２はカメラ２の撮影画像内における座標を表す。

X ₁ and y ₁ represent coordinates in the captured image of the camera 1, and x ₂ and y ₂ represent coordinates in the captured image of the camera 2.

  In general, the rotation matrix R is expressed as follows.

In Equation _{1, p a11, p a12,} p a13, p a21, p a22, p a23 is the rotation matrix _{R 1, R 1_11, R 1_12} , R 1_13, R 1_21, R 1_22, as each equal to _{R 1_23} By solving the simultaneous equations, the rotation angles α, β, γ of the camera can be obtained, and therefore the rotation matrix R ₁ can be obtained. Similarly, the rotation matrix R ₂ can be obtained for the camera 2. Then, the rotation matrix R ₂₁ for converting the coordinate system of the camera 2 to the coordinate system of the camera 1 can be obtained by R ₂ ⁻¹ · R ₁ .

[Corresponding point search processing]
The corresponding point calculation unit 9 calculates corresponding pixels from two normal images whose coordinate systems are unified. This process is performed by obtaining a normal line in the same direction as the normal line in the pixel of interest in the normal image of the camera 1 from the normal image of the camera 2. The processing performed by the corresponding point calculation unit 9 will be described with reference to the flowchart of FIG.

  First, the corresponding point calculation unit 9 acquires two normal images A and B with a unified coordinate system (S1). Here, the normal image A acquired from the image of the camera 1 uses the image obtained from the surface shape calculation unit 7 as it is, and the normal image B obtained from the image of the camera 2 is used as a coordinate conversion unit. 8 converted into the coordinate system of the camera 1 is used.

  Next, an arbitrary pixel in one normal image (here, normal image A) is selected as a target point (target pixel) (S2). Then, a contrast point is selected from the epipolar line of the other normal image (here, normal image B) (S3).

  Then, the similarity between the attention point of the normal image A and the contrast point of the normal image B is calculated by the similarity evaluation function (S4). Here, since there is a possibility that erroneous determination may occur when the normal directions at one point are compared, the similarity is calculated using the normal directions at pixels around the attention point and the contrast point. FIG. 10A shows an example of a search window used for calculating the similarity. Here, an area of 5 pixels × 5 pixels centering on the point of interest is used as a search window.

  The similarity between the attention point and the contrast point is calculated based on the coincidence ratio of the normal directions in the entire search window. More specifically, the inner product of the normal vectors is calculated between the normal images A and B at each point in the search window by the following formula, and the similarity is calculated based on the sum (FIG. 10 (B )reference).

  Since the corresponding point is on the epipolar line, the calculation of the similarity is performed for pixels on the epipolar line. Therefore, after calculating the similarity for one point, it is determined whether the similarity calculation processing has been executed for all the points on the epipolar line. If there is a point for which the similarity is not yet calculated, the process returns to step S3. Similarity calculation is performed (S5).

  When the similarity is calculated for all points on the epipolar line, the point having the highest similarity is obtained from the points, and the point is determined as the corresponding point in the normal image B corresponding to the attention point of the normal image A ( S6).

  Since the above processing is performed for all the points of the normal image A for performing the triangulation, it is determined whether all the points have been processed, and if there are points that have not been processed yet, the process returns to step S2 A corresponding point corresponding to the point is searched (S7).

[Triangulation]
As described above, when the corresponding points in the two images are obtained, the triangulation unit 10 calculates the depth information (distance) for each position of the measurement object 4. Since this process is a known technique, a detailed description thereof will be omitted.

<Operation and effect of the embodiment>
In the three-dimensional measurement apparatus according to the present embodiment, the corresponding points between the two images are searched using the direction of the normal, which is a physical feature of the measurement object. 3D measurement is possible without being affected by the above. In the conventional corresponding point search processing based on the color (luminance value) of the physical surface, when the target surface is a mirror surface, an error becomes large and accurate three-dimensional measurement is difficult. If it is used, even if it is a mirror surface object, it will become possible to perform three-dimensional measurement with sufficient accuracy.

  In addition, since the coordinate system that is different among multiple cameras is converted into a common coordinate system using the conversion parameters extracted from the calibration parameters obtained by camera calibration, the corresponding points are searched for, so the camera congestion Even if the angle is increased, the matching accuracy does not decrease, and accurate three-dimensional measurement is possible.

(Modification 1)
In the embodiment described above, the normal direction is calculated from the captured image of the camera 2 by referring to the normal-luminance table, and then the process of matching the coordinate system of the normal image with the coordinate system of the camera 1 is performed by coordinate conversion. is doing. However, other methods may be used as long as the coordinate system is finally unified. For example, conversion processing for matching the coordinate system of the camera 2 to the coordinate system of the camera 1 may be performed on the normal data stored in the normal-luminance table corresponding to the camera 2. By doing so, the calculation result of the direction of the normal line by the surface shape calculation unit 7 with respect to the image of the camera 2 is expressed in the coordinate system of the camera 1.

(Modification 2)
In the above-described embodiment, the three illumination devices 3a to 3c that emit white light are used, these are sequentially turned on to capture images, and the normal direction is calculated from the three images. However, any method can be employed as long as it captures an image and acquires the direction of the normal. For example, the illuminating colors of the three lighting devices are set to three colors of R, G, and B, respectively, and these are simultaneously irradiated to acquire the intensity of each component light. Can be obtained.

(Second Embodiment)
In the first embodiment, the normal direction is used as the physical feature of the measurement target surface. However, in this embodiment, the corresponding points between the stereo images are searched using the spectral characteristics of the target.

  In order to measure the spectral characteristics of the measurement target surface, light sources having different spectral characteristics are sequentially irradiated onto the measurement target from the same position. For example, as shown in FIG. 11, a color filter having different spectral characteristics depending on the location (angle) is provided in front of the white light source, and this filter can be rotated. A simple spectral characteristic for each pixel can be calculated by observing an object with a color camera using such an illumination device and measuring a luminance value having the highest value.

  And it matches using the spectral characteristic map for every pixel obtained with the some camera. The subsequent processing is the same as in the first embodiment.

(Third embodiment)
In the present embodiment, the corresponding point between the stereo images is searched using the reflection characteristic as the physical characteristic of the measurement target surface.

  In order to measure the reflection characteristics of the surface to be measured, a plurality of light sources that irradiate light from different directions are arranged, and shooting is performed with a camera while sequentially turning on these light sources. Similarly to the first embodiment, a sample having a known shape such as a sphere and a known reflection characteristic is prepared. Here, a plurality of samples having different reflection characteristics are used, and the luminance value under each light source is stored as case data for each sample.

  Similarly, a plurality of light sources are sequentially turned on for the measurement target, and a combination of luminance values under each light source is acquired. The combination of the brightness values is compared as case data, and the reflection characteristic corresponding to each pixel is calculated.

  Using the reflection characteristic maps for each pixel obtained by a plurality of cameras, the pixels are associated between images captured by the plurality of cameras. Subsequent processing is the same as in the first embodiment.

FIG. 1 is a diagram showing an outline of a three-dimensional measuring apparatus. FIG. 2 is a diagram illustrating functional blocks of the three-dimensional measurement apparatus. FIG. 3 is a diagram for explaining the arrangement of the cameras. FIG. 4 is a diagram for explaining the arrangement of the illumination devices ((A) is an azimuth angle arrangement and (B) is a zenith angle arrangement). FIG. 5 is a functional block diagram of the surface shape calculation unit. FIG. 6 is a diagram for explaining a method of creating a normal-luminance table. FIG. 7 is a diagram for explaining a method for acquiring the direction of the normal line from a captured image. FIG. 8 is a diagram for explaining a conversion matrix for converting between the world coordinate system and the coordinate system of each camera. FIG. 9 is a flowchart showing the flow of corresponding point search processing by the corresponding point calculation unit. FIG. 10 is a diagram for explaining a search window and similarity calculation for corresponding point search. FIG. 11 is a diagram illustrating the lighting device according to the second embodiment. FIG. 12 is a diagram showing the principle of three-dimensional surveying. FIG. 13 is a diagram illustrating a case where a three-dimensional survey is performed on a specular object.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 2 Camera 3a, 3b, 3c Illuminating device 4 Measurement object 6 Computer 7 Surface shape calculation part 71 Image input part 72 Normal-luminance table 73 Normal calculation part 8 Coordinate conversion part 9 Corresponding point calculation part 10 Triangulation part

Claims (10)

A three-dimensional measurement device that measures a three-dimensional shape of a measurement object that is a specular object,
Multiple cameras,
Feature acquisition means for acquiring the physical characteristics of the surface of the measurement object from each of the images taken by the plurality of cameras;
Corresponding pixel search means for searching for corresponding pixels between images captured by the plurality of cameras using the physical features;
Surveying means for performing three-dimensional surveying based on the parallax between corresponding pixels;
A three-dimensional measuring apparatus comprising: The three-dimensional measurement apparatus according to claim 1, wherein the physical feature of the surface of the measurement object acquired by the feature acquisition unit is a direction of a normal of the surface. Coordinate conversion means for converting the coordinate system of images taken by a plurality of cameras into a common coordinate system using conversion parameters,
3. The three-dimensional image according to claim 2, wherein the corresponding pixel search unit searches for a corresponding pixel between images by using a direction of a normal line converted into a common coordinate system by the coordinate conversion unit. Measuring device. The three-dimensional measurement apparatus according to claim 3, wherein the conversion parameter in the coordinate conversion unit is extracted from a parameter obtained by camera calibration performed in advance. The said corresponding pixel search means compares the physical characteristics in the area | region of the predetermined area containing the pixel of interest, and searches for the corresponding pixel between images. 3D measuring device. A three-dimensional measurement method for measuring a three-dimensional shape of a measurement object that is a specular object,
A feature acquisition step of acquiring a physical feature of the surface of the measurement object from each of images taken by a plurality of cameras;
A corresponding pixel search step of searching for a corresponding pixel between images taken by the plurality of cameras using the physical feature;
A surveying step for performing a three-dimensional survey based on the parallax between corresponding pixels;
A three-dimensional measurement method comprising: The three-dimensional measurement method according to claim 6, wherein the physical feature of the surface of the measurement object acquired in the feature acquisition step is a direction of a normal of the surface. A coordinate conversion step of converting a coordinate system of images captured by a plurality of cameras into a common coordinate system using a conversion parameter;
8. The three-dimensional image according to claim 7, wherein, in the corresponding pixel search step, a corresponding pixel is searched between images using the direction of the normal line converted into a common coordinate system in the coordinate conversion step. Measurement method. The three-dimensional measurement method according to claim 8, wherein the conversion parameter used in the coordinate conversion step is extracted from a parameter obtained by camera calibration performed in advance. The said corresponding pixel search process compares the physical characteristics in the area | region of the predetermined area containing the pixel of interest, and searches for a corresponding pixel between images. 3D measurement method.

Images