JPH0886609A - Measurement with beam light projection-type three-dimensional sensor, calibration of the sensor and apparatus therefor - Google Patents

Abstract

PURPOSE: To provide a measuring method with a beam projection type three- dimensional sensor, a calibration method for the sensor, and an apparatus therefor enabling highly accurate three-dimensional measurements and correct calibration of an actual beam-camera system. CONSTITUTION: According to the method, modeling of a beam projection type three-dimensional sensor is conducted with the use of an affine inverse matrix so that a middle point of the common normal of a line of a beam light from a beam light source and a line of sight of a camera becomes an approximate solution of least squares (51). Image data of the camera as measuring data are subjected to coordinate transformation with the use of a line parameter of the beam light included in the model and a camera parameter, thereby the position of a measuring point on the three-dimensional coordinates in space is operated (52). Alternatively, the line parameter of the beam light included in the model and the camera parameter are calibrated. By the above constitution, a highly accurate three-dimensional measurements and the like are achieved.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a measuring method and a calibrating method using a beam light projection type three-dimensional sensor and their devices. More specifically, the present invention relates to a method of triangulation using a beam light source and a camera. The present invention relates to a measuring method and a calibrating method using a beam light projection type three-dimensional sensor for measuring a dimensional position, and devices thereof.

[0002]

2. Description of the Related Art A beam light projection type three-dimensional sensor using a beam light such as a laser beam uses a beam light source whose geometrical positional relationship is known and a single camera, and uses a triangulation principle. It measures the position of a certain target object in the dimensional space. Here, the setting / calibration accuracy of the geometrical positional relationship between the beam light source and one camera directly affects the measurement accuracy. Therefore, in order to obtain high measurement accuracy, it is necessary to accurately calibrate the geometrical positional relationship between the beam light source and one camera. Generally, the calibration of the beam light projection type three-dimensional sensor is performed by describing the geometrical positional relationship between the beam light source and one camera using a mathematical model and deriving the parameter values of the model. In such a conventional method of measuring a beam light projection type three-dimensional sensor, the measurement point in the three-dimensional space is on the straight line of the beam light, and the line of sight of the camera connecting the measurement point and the lens center of the camera and the image pickup of the camera. By using the assumption that the beam image exists at the intersection with the plane, one of the equations was solved to calculate the coordinates of the measurement point (3 for a large structure using an electronic theodolite). Dimensional measurement system, SICE'89,
Jul 25-27, Matsuyama).

[0003]

In the conventional measuring method using the beam light projection type three-dimensional sensor as described above, the measuring point in the three-dimensional space is on the straight line of the beam light and the measuring point and the lens of the camera. The assumption is that a beam image exists at the intersection of the camera line of sight that connects the centers and the imaging surface of the camera. However, in an actual beam light projection type three-dimensional sensor, the beam light straight line and the camera line of sight do not strictly cross each other due to a quantization error in measurement or the like. In that case, in the method of solving one equation using the above assumptions, the measurement accuracy is reduced because the influence of measurement noise is large. Therefore,
The actual beam-camera system cannot be calibrated accurately. The present invention has been made in view of the above circumstances, and a first object of the present invention is to perform highly accurate three-dimensional measurement by considering in advance an error existing in an actual beam-camera system such as measurement noise. It is an object of the present invention to provide a measurement method and a device therefor using a beam light projection type three-dimensional sensor that can perform A second object is to provide a method and apparatus for calibrating a beam light projection type three-dimensional sensor capable of accurately calibrating an actual beam-camera system.

[0004]

To achieve the above first object, a first invention is a beam light for measuring the position of a measuring point in a three-dimensional space by using a beam light source and a camera. In a measurement method using a projection type three-dimensional sensor, a beam light projection type three-dimensional method in which a midpoint of a common normal line between the straight line of the light beam from the light source and the line of sight of the camera is used as a least squares approximate solution using a pseudo inverse matrix The sensor is modeled, and the image data of the camera, which is the measurement data, is subjected to coordinate conversion using the straight line parameter of the light beam and the camera parameter included in the model, thereby measuring points in three-dimensional space coordinates. It is configured as a measuring method by a beam light projection type three-dimensional sensor characterized by calculating the position of. The second invention is a beam light source and
In a measuring device using a beam light projection type three-dimensional sensor that measures the position of a measurement point in a three-dimensional space using two cameras, a straight line of the beam light from the light source and the line of sight of the camera using a pseudo inverse matrix. Modeling means for modeling a beam light projection type three-dimensional sensor having a midpoint of a common normal line to and as a least square approximation solution, and camera data, which is the above measurement data, included in the above beam included in the above model. A beam light projection type three-dimensional sensor comprising: a calculation means for calculating the position of a measurement point in three-dimensional space coordinates by performing coordinate conversion using a linear parameter of light and a camera parameter. It is a measuring device. In order to achieve the above-mentioned second object, a third invention is a calibration method of a beam light projection type three-dimensional sensor for measuring the position of a measurement point in a three-dimensional space using a beam light source and one camera. , A pseudo-inverse matrix is used to calculate a beam light projection formula 3 in which the midpoint of the common normal line between the straight line of the light beam from the light source and the line of sight of the camera is used as the least squares approximate solution.
Modeling a three-dimensional sensor, partially differentiating the model with a linear parameter of the light beam and a camera parameter included in the model, measuring a position of a known measurement point in a three-dimensional space, and performing the known measurement. Calibration of a beam light projection type three-dimensional sensor characterized by deriving both of the above parameters that minimizes the norm between the data using the measured data of the position of the point and the partial differential data of the model. Is configured as an option method. Further, in a beam light projection type three-dimensional sensor calibration device that measures the position of a measurement point in a three-dimensional space using a beam light source and one camera, the beam from the light source is calculated using a pseudo inverse matrix. Modeling means for modeling a beam light projection type three-dimensional sensor having a midpoint of a common normal line of a straight line of light and a line of sight of the camera as a least squares approximate solution, and the beam light included in the model. Differentiating means for partially differentiating between the linear parameter and the camera parameter, measuring means for measuring the position of a known measuring point in the three-dimensional space, measurement data of the position of the known measuring point, and partial derivative data of the model. And a deriving means for deriving both of the above parameters so as to minimize the norm between the two data by using the carrier of the beam light projection type three-dimensional sensor. A configuration device.

[0005]

According to the first and second aspects of the invention, when the beam light projection type three-dimensional sensor for measuring the position of the measurement point in the three-dimensional space is used by using the beam light source and one camera, Using a pseudo-inverse matrix, modeling of a beam light projection type three-dimensional sensor in which the midpoint of the common normal line between the straight line of the light beam from the light source and the line of sight of the camera is used as the least squares approximate solution is performed. The image data of the camera, which is the above measurement data,
By performing coordinate conversion using the linear parameter of the light beam and the camera parameter included in the model,
The position of the measurement point in three-dimensional space coordinates is calculated. Since the above modeling considers errors existing in the actual beam-camera system such as measurement noise in advance, by performing the calculation by the least square method using a plurality of equations,
Highly accurate three-dimensional measurement is possible. Further, according to the third and fourth inventions, a beam light source and a single camera are used.
Beam light projection formula 3 for measuring the position of the measurement point in the dimensional space
When calibrating a three-dimensional sensor, a beam light projection type three-dimensional method that uses a pseudo-inverse matrix to set the midpoint of the common normal line between the line of the beam of light from the light source and the line of sight of the camera to the least squares approximate solution Modeling of the sensor is performed.
The model is partially differentiated by the linear parameter of the light beam and the camera parameter included in the model. The position of a known measurement point in the three-dimensional space is measured. Using the measurement data of the known measurement points and the partial differential data of the model, the above two parameters are derived so that the norm between the two data is minimized. As a result, the actual beam-camera system can be accurately calibrated.

[0006]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments embodying the present invention will be described below with reference to the accompanying drawings for the understanding of the present invention. The following embodiments are examples of embodying the present invention and are not intended to limit the technical scope of the present invention. Here, FIG. 1 is a diagram showing a schematic processing flow of a measuring method using a beam light projection type three-dimensional sensor according to an embodiment (first embodiment) of the first invention, and FIG. 2 is applicable to the measuring method. FIG. 3 is a block diagram showing a schematic configuration of a device A, FIG. 3 is an explanatory diagram of each coordinate system, and FIG. 4 is a calibration of a beam light projection type three-dimensional sensor according to an embodiment (second embodiment) of the third invention. FIG. 5 is a block diagram showing a schematic configuration of an apparatus B to which the above calibration method can be applied. As shown in FIG. 1, a measuring method using a beam light projection type three-dimensional sensor according to one embodiment (first embodiment) of the first invention uses a beam light source and one camera in a three-dimensional space. This is the same as the conventional example in that the measurement is performed by the beam projection type three-dimensional sensor that measures the position of the measurement point. However, in the first embodiment, a pseudo-inverse matrix is used for the beam light projection type three-dimensional method in which the midpoint of the common normal line between the straight line of the light beam from the light source and the line of sight of the camera is used as the least square approximation solution. The sensor is modeled (S1), and the image data of the camera, which is the measurement data, is subjected to coordinate conversion using the straight line parameter of the light beam and the camera parameter included in the model, thereby performing three-dimensional spatial coordinates. This is different from the conventional example in that it is configured to calculate the position of the measurement point in (S2). An apparatus to which the above measuring method can be applied is shown in FIG. That is, this measuring apparatus A is the second invention, and the modeling apparatus 3 (corresponding to modeling means) in the figure performs step S1 in the above method, and the measuring point position calculating apparatus 4 (corresponding to computing means) performs the above step S2. Respectively. The execution result is output by the output device 5.

The basic principle will be clarified below according to the procedure of the above method. Step S1 (Modeling of a beam light projection type three-dimensional sensor): In the first embodiment, first, the beam light source 1 and one camera 2 are used to create a world coordinate system Σ as shown in FIG.
it is assumed for measuring the position of the reference point ^w x described by _w. At this time, the relationship between the light beam from the light source 1, the camera 2, the measurement point ^w x, and the world coordinate system Σ _W is expressed as follows. That is, the light beam coordinate system sigma _R and from the world coordinate system sigma _W as defined in the figure, homogeneous transformation matrix of each coordinate transformation to the camera coordinate system _{^{Σ C 4 × 4 w A R}} , expressed in ^w A _C , expressed in homogeneous transformation matrix ^S a _F of the coordinate conversion 3 × 3 from the screen coordinate system sigma _S to the frame coordinate system sigma _F.
Further, the position and orientation of the image pickup surface (screen) of the camera 2 are described by the coordinate system Σ _S set on the scoon, and the position and orientation on the image memory are the frame coordinate system set on the image memory. Describe in Σ _F. here,

[Equation 1] The conversion from the screen (CCD surface) to the image memory (frame buffer) causes distortion due to the influence of sampling or the like. Let ( ^F u ₀ , ^F v ₀ ) be the translation of the origin, (s, t) be the contraction of the coordinate axes, and φ be the distortion of the coordinate system. The projection matrix for projecting the camera coordinate space onto the screen coordinate space is given by the following equation.

[Equation 2]

Here, f is the focal length. The beam light in the beam light coordinate system Σ _R is expressed as follows. ^R x = α · ^R m + ^R x ₀ (4) where α is a scalar parameter, ^R m is a direction vector,
^R x ₀ is a point on the light beam,

(Equation 3) Is. The point ^R x defined by the beam light coordinate system Σ _R is projected on the frame coordinate system Σ _F as follows.

[Equation 4]

However, ^F w is a scale factor, and 2
The point ( ^F X, ^F Y) on the three-dimensional frame is given by the following equation (the point in the three-dimensional space is projected onto the point in the two-dimensional space by the projective transformation).

(Equation 5) The sensor parameter c is a camera parameter expressing the mounting position of the camera 2, the focal length, the distortion of the frame buffer, and the like, and the straight line parameter of the light beam from the light source 1.
Is defined. In the first embodiment, the above (1),
Parameters in equations (3) and (6),

(Equation 6) Is the specific sensor parameter. Calibration is nothing but the accurate derivation of the values of these parameters. Step S2 (measurement method of beam light projection type three-dimensional sensor using pseudo inverse matrix):

When the straight line in the beam light coordinate system Σ _R is given by the above equation (4), the coordinate transformation matrix ^W A
_{Since R} , ^W A _C , ^F A _S , and ^S P _C are all known, the vector amount in the frame coordinate system Σ _F of the vector in the above equation (5)

(Equation 7) Has its value specified. At this time, the following equation is derived from the above equation (7).

[Equation 8] The equation (10) is redundant because there are two equations for one unknown variable α. Conventionally, it is assumed that the measurement point in the three-dimensional space is on the straight line of the light beam and that the beam image exists at the intersection of the camera line of sight connecting the measurement point and the center of the camera lens and the imaging surface of the camera. Was used to solve one equation. However, in the first embodiment, the least squares solution by the pseudo inverse matrix is obtained by using these redundant equations.

[Equation 9]

However, + represents a pseudo-inverse matrix, and is given by P ⁺ = P ^T (P ^T P) ^-1 (12) when P is an n × m matrix (n ≧ m), for example. By substituting the variable α obtained by the above equation (11) into the above equation (4), the position ^R x of the measurement point in the beam light coordinate system Σ _R is obtained, and the world coordinate system is obtained by the following coordinate conversion. The position of the measurement point at Σ _w can be obtained. _{^{W x = W A R · R}} x ... (13) As described above, in this first embodiment by using the least squares solution by pseudo-inverse, to reduce the deterioration of the measurement accuracy due to influence of measurement noise be able to. By the way, in the first embodiment described above, the measuring method and the apparatus using the beam light projection type three-dimensional sensor have been described. However, it is necessary to periodically calibrate the sensor parameter during modeling used for this measurement. The method is realized by the following third and fourth inventions. As shown in FIG. 4, a method of calibrating a beam light projection type three-dimensional sensor according to an embodiment (second embodiment) of the third invention uses a beam light source and a camera to form a three-dimensional space. When calibrating a beam light projection type three-dimensional sensor that measures the position of a measurement point inside, the midpoint of the common normal line between the straight line of the light beam from the light source and the line of sight of the camera is calculated by using the pseudo inverse Modeling of a beam light projection type three-dimensional sensor with S is the least squares approximate solution (S1
1), the model is partially differentiated by the linear parameter of the light beam and the camera parameter included in the model (S
12), the position of a known measurement point in the three-dimensional space is measured (S13), and the norm between the two is minimized by using the known measurement data of the known measurement point and the partial differential data of the model. Both of the above parameters are derived (S1
4) is configured.

An apparatus to which the above calibration method can be applied is shown in FIG. That is, the calibration device B is the fourth invention, and the modeling device 6 (corresponding to modeling means) in FIG. 5 performs step S11 in the above method, and the differentiating device 7 (corresponding to differentiating means) performs step S12. The beam light source 1 and the camera 2 (both correspond to measuring means) and the calibration data storage device 8 execute step S13, and the sensor parameter calculation device 9 (corresponding to deriving means) executes step S14. The execution result is stored in the sensor parameter storage device 10 and used for actual measurement by the beam light source 1 and the camera 2. Below, the basic principle is clarified according to the procedure of the above method. Step S11 (Modeling of a beam light projection type three-dimensional sensor): This step is the same as step S1 in the method of the first embodiment, so its explanation is omitted here. Step S12 (derivation of differential relation): Now, ^W A _R , ^W A
_{^{C, S P C, F A}} S parameter vector q _k parameters of the coordinate transformation matrix and the light beam having the N elements, such as
It is assumed that it is described in. q _k = [q _k1 q _k2 ... q _kN] ... (14) The parameter vector q _k is a generalization of the sensor parameter c in the first embodiment method.
In order to simplify the formulation, the straight line equation of the equation (4) is converted into the world coordinate system Σ _w and is represented by the function f (,).

[Equation 10]

Hereinafter, when the projection point ^F X of the frame coordinate system Σ _F is given, the error that occurs during the calculation for deriving the reference point ^w x of the world coordinate system Σ _w due to the error of the parameter vector q _k is derived. . The differential relation by the parameter vector q _k of the above equation (15) is as follows.

[Equation 11] Where J _l is the Jacobian matrix. In the following, this Jacobian matrix J _l is derived. In the above equation (16), the Jacobian matrix J _l can be transformed as follows.

[Equation 12]

The second and third terms on the right side of the equation (17) can be easily derived by the differential of the homogeneous conversion form. Rewrite the first term as follows.

[Equation 13] Then, the first term uses the Kronecker product,

[Equation 14] Is derived as follows. The above equations (17) and (19) are error equations of the light section measurement using the beam light. In addition, in the modeling method using the conventional assumption,

(Equation 15) As a result, the influence of the first term on the right side of the above equation (19) is neglected, and in that sense the rigor is lacking. According to the above equation (19), when the differential relation between the frame coordinate system Σ _F and the world coordinate system Σ _w , that is, the projection point ^F X of the frame coordinate system Σ _F is given, the camera parameter error (small change) and quantum Due to the error of the projection point itself such as the conversion, the error that occurs during the calculation for deriving the reference point ^w x of the world coordinate system Σ _w is calculated.

Step S13 (collection of calibration data): A measurement point ^w x _i whose position in the world coordinate system Σ _w is known is prepared.

[Equation 16] Data obtained by capturing them with the camera 2 is represented as follows.

[Equation 17] However, m is the number of measurement points ^w x _i . The positions and the number of measurement points in the world coordinate system Σ _w are selected so that the rank of the Jacobian matrix J of the equation (24) described later is N (the number of sensor parameters) or more. Step S14 (calibration calculation)

Data of the above equations (21) and (22),
The sensor parameters of equation (13) are derived using equations (15) and (19). Substep S14-1: Initial values of sensor parameters

(Equation 18) And the convergence criterion e (> 0). Sub-step S 14 - 2: by using the vector q is a common form known in which the coordinate values ^w x _D measurement point, projection point ^F X and the sensor parameters is a data captured by the camera 2 in the world coordinate system sigma _w The calculated coordinate value ^w x _M and the Jacobian matrix J are obtained.

[Formula 19]

[0017] The sub-step _{^{S14-3: Δx = w x D -}} w x M ... to calculate the (25), | Δx | terminated if ≦ e ... (26). Sub-step ^{S14-4: Δ w x = JΔq T} ... from (27), calculates the ^{= Δq T = J + Δ W} x [(J ^T J) ^-1 J ^T] Δ ^w x ... (28). Substep S14-5: q ← q + Δq (29), and the procedure returns to substep S14-2. The iterative calculation enables the parameter vector q to be derived with high accuracy. As described above, according to the second embodiment, the actual beam-camera system can be accurately calibrated. Although the least squares method is used in modeling in both the first and second embodiments, other types of optimization algorithms may be used in actual use. However, even in this case, according to the method of least squares, relatively reliable and highly accurate modeling can be obtained as compared with the case of using another type of optimization algorithm.

[0018]

Since the measuring method and apparatus using the beam light projection type three-dimensional sensor according to the first and second aspects of the invention are configured as described above, they can be applied to an actual beam-camera system such as measurement noise. The existing error can be taken into account in advance. Therefore, highly accurate three-dimensional measurement can be performed by performing the calculation by the least square method using a plurality of equations. Further, since the method and apparatus for calibrating the beam light projection type three-dimensional sensor according to the third and fourth aspects of the invention are configured as described above, the actual beam-camera system is calibrated accurately. be able to.

[Brief description of drawings]

FIG. 1 is a diagram showing a schematic processing flow of a measuring method using a beam light projection type three-dimensional sensor according to an embodiment (first embodiment) of the first invention.

FIG. 2 is a block diagram showing a schematic configuration of an apparatus A to which the above measuring method can be applied.

FIG. 3 is an explanatory diagram of each coordinate system.

FIG. 4 is a diagram showing a schematic processing flow of a calibration method for a beam light projection type three-dimensional sensor according to an embodiment (second embodiment) of the third invention.

FIG. 5 is a block diagram showing a schematic configuration of an apparatus B to which the above calibration method can be applied.

[Explanation of symbols]

A ... Measuring device using a beam light projection type three-dimensional sensor 1 ... Beam light source (corresponding to measuring means) 2 ... Camera (corresponding to measuring means) 3 ... Modeling device (corresponding to modeling means) 4 ... Measuring point position calculation device (calculation) B) ... calibration device for beam light projection type three-dimensional sensor 1 ... beam light source (corresponding to measuring device) 2 ... camera (corresponding to measuring device) 6 ... modeling device (corresponding to modeling device) 7 ... differential device (Corresponding to differentiating means) 9 ... Sensor parameter calculation device (corresponding to deriving means)

Claims (4)

[Claims] 1. A beam source and a camera
Beam light projection formula 3 for measuring the position of the measurement point in the dimensional space
In a measurement method using a three-dimensional sensor, modeling of a beam light projection type three-dimensional sensor in which a midpoint of a common normal line between the straight line of the light beam from the light source and the line of sight of the camera is used as a least squares approximate solution by using a pseudo inverse matrix The image data of the camera, which is the measurement data, is subjected to coordinate conversion using the linear parameters of the light beam and the camera parameters included in the model to determine the position of the measurement point in three-dimensional space coordinates. A measuring method using a beam light projection type three-dimensional sensor characterized by being calculated. 2. A beam source and a camera
Beam light projection formula 3 for measuring the position of the measurement point in the dimensional space
Modeling of a beam light projection type three-dimensional sensor using a pseudo-inverse matrix in which a midpoint of a common normal line between the straight line of the light beam from the light source and the line of sight of the camera is used as a least-squares approximate solution in a measuring device using a three-dimensional sensor By performing the coordinate conversion of the modeling means for performing the above and the image data of the camera, which is the measurement data, by using the linear parameter of the light beam and the camera parameter included in the model, 3
A measuring device using a beam light projection type three-dimensional sensor, comprising: a calculation means for calculating the position of a measurement point in dimensional space coordinates. 3. A beam source and a camera
Beam light projection formula 3 for measuring the position of the measurement point in the dimensional space
In a calibration method for a three-dimensional sensor, a beam light projection type three-dimensional sensor using a pseudo-inverse matrix in which a midpoint of a common normal line between the straight line of the light beam from the light source and the line of sight of the camera is used as a least squares approximate solution. Is modeled, the model is partially differentiated by the linear parameter of the light beam and the camera parameter included in the model, the position of the known measurement point in the three-dimensional space is measured, and the model of the known measurement point is calculated. A method of calibrating a beam light projection type three-dimensional sensor characterized by deriving both of the above parameters that minimizes the norm between the two data by using position measurement data and model partial differential data. . 4. A beam source and a camera
Beam light projection formula 3 for measuring the position of the measurement point in the dimensional space
In a calibration device for a three-dimensional sensor, a beam light projection type three-dimensional sensor that uses a pseudo-inverse matrix and uses a midpoint of a common normal line between the straight line of the light beam from the light source and the line of sight of the camera as a least squares approximate solution Means for modeling the above, a differentiating means for partially differentiating the model with the linear parameter of the light beam and the camera parameter included in the model, and a measurement for measuring the position of a known measurement point in the three-dimensional space. And means for deriving the two parameters so as to minimize the norm between the data by using the measurement data of the position of the known measurement point and the partial differential data of the model. A beam light projection type three-dimensional sensor calibration device characterized by: