JP3254475B2 - Calibration method for range sensor - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional position measuring device for measuring a three-dimensional position at the time of teaching a work to a work robot, and more particularly to a measuring device for measuring a three-dimensional position. The present invention relates to a calibration method capable of simultaneously estimating necessary parameters by efficiently estimating a required space.

[0002]

2. Description of the Related Art A range sensor device such as a range finder which irradiates an object with a laser beam, observes a three-dimensional position where the light is irradiated by a camera or the like, and measures the three-dimensional position based on distance information is widely used. Have been. (Reference books: 3
3D image measurement, Iguchi, Sato, Shokodo)

FIG. 7 shows a configuration of a general range sensor device. 7, the range sensor device, a light slit light source F _S emitted slit G
_A light irradiator (not shown) that scans the slit in a right angle direction while forming and irradiating linear light through _S, and an observation space 72 that receives the light and accommodates the object 71 to be observed.
When, TV having an imaging surface G _I which are synchronously scanned with the light irradiation device while receiving the image of the light projected onto the observation object 71
An image receiving device (not shown) such as a camera (hereinafter, referred to as a camera) is provided while maintaining a distance d between the devices.

[0004] Slit light source F_SIs a point light source instead of
Laser light source of the laser device described below (hereinafter referred to as laser)
F_LIf you use a spot-shaped light beam with strong directivity
The slit G_SCan be omitted and instead this
Laser light source F_LFor controlling the
User control surface G_LCan be set on the observation space side. Also this
Laser control surface G_LIs a laser seat that is on the XY coordinate plane
Mark X_LY _LZ_LCan be defined.

[0005] imaging plane G _I is in a position to rotate the principal point F _C having camera lens in point symmetry to the center can assume the camera image plane G _C is the image plane of the temporary, the camera image plane G _C
Camera coordinate system X _C Y _C Z _C but such that XY coordinate plane
Can be defined.

[0006] The observation space 72 can define an object coordinate system based on the shape and directionality of the object 71 to be observed. Therefore, the laser beam, the laser coordinate value at the intersection of the laser control surface G _L a (x _l, y _l), the line of sight of the camera, the camera image plane G _C
Has the camera coordinate values (x _c, y _c) to the intersection with the.

FIG. 8 shows a conventional measuring method using the laser shown in FIG.
It is explanatory drawing explaining a method. 7 and 8,
In this measurement method, first, a predetermined laser coordinate value (x_l, y _l)
To the laser to control it,_LOriginating from
The object is irradiated with laser light to scan the object two-dimensionally.

Next, a predetermined camera coordinate value (x _c, y _c ) is given to the camera to scan the camera image plane G _C , and a bright point is detected using the image projected by the laser beam as an object of image processing. , measured by converting the camera coordinate value when (x _c, y _c) known laser coordinate value based on the (x _l, y _l) a three-dimensional shape of the observed object from the distance information.

However, a laser coordinate system, a camera coordinate system,
And the observation space coordinate system are individually defined, and the units and directions of measurement are not uniform, and the measurement itself is meaningless as well as the measurement accuracy as it is. Incidentally, 81 and 82 indicate scanning lines on each image plane.

Therefore, prior to the measurement, a camera, a laser,
In addition, there is a step of obtaining the characteristics of the measurement environment based on the mutual relationship and calibrating and preparing the measurement values of the range sensor based on the characteristics. This preparation step involves the distances d,
Since the time the than measuring the focal length and ruler in such a principal point F _c of applied, practically by using the principle of triangulation, the position of the camera and the laser, the focal length position, and the lens Camera, which is a parameter determined by
A method for determining (calibrating) the parameter [A] and the laser parameter [B] has been implemented.

This calibration method is executed through the following two steps. i) First, a camera parameter [A], which is a parameter related to the camera, is obtained by a reference observation object serving as a reference. ii) Next, a three-dimensional position of the laser beam is measured using the camera parameter [A], and a laser parameter [B], which is a parameter related to the laser, is obtained.

[0012]

However, when the calibration method in the conventional range sensor is executed, there are the following problems. (1) Since the parameters are obtained in two steps for obtaining the camera parameters and the laser parameters, the work process is complicated and extra work time is required. (2) In addition, since both parameters are obtained separately and their measurement errors cannot be uniform, the measurement accuracy of the three-dimensional position when measuring the shape of the observed object is reduced. (3) Further, in a range sensor that obtains a welding line (Japanese Patent Application No. 5-89634), the laser beam does not scan the object to be observed regularly, but tends to be irradiated in a certain direction. Therefore, if the entire camera image plane is used as a target area for image processing, there is too much unnecessary processing, resulting in useless processing time. The present invention has been made in view of the above-mentioned problems, and has simplified the working process.
Another object of the present invention is to provide a calibration method for a range sensor that shortens the working time. In addition, by using this calibration method, it is possible to provide a robot path generation device that simplifies the preparation for measurement, can easily generate an instruction path in a short time, even a complicated work, and can efficiently teach. That is the task.

[0013]

[Means for Solving the Problems]

(1) A laser device for irradiating a laser beam to an object, and a camera device for observing the shape of the object. In a range sensor for measuring the shape, a laser parameter, which is a measurement characteristic based on the laser device, and A calibration method for determining and calibrating a camera parameter, which is a measurement characteristic based on a camera device, by a calibration jig, wherein the calibration jig has a shape formed in an n-polygonal pyramid shape. First, a position setting stage for setting a position for positioning the calibration jig and a control position for controlling the laser beam three-dimensionally, and then a calibration jig while controlling the laser beam. A trajectory measurement step of measuring the projection trajectory projected onto the tool three-dimensionally by a camera device, and subsequently, the measured projection trajectory and each of the set positions are determined according to the principle of triangulation. The measurement is performed by converting the positional relationship between the calibration jig, the laser device, and the camera device into a uniform three-dimensional position, and measuring the position. The measurement is repeated while the positioning position is moved three-dimensionally. And a parameter estimation step for simultaneously obtaining the laser parameters and the camera parameters based on the measured values to obtain the laser parameters and the camera parameters. A calibration method in a range sensor, comprising: (2) The step of measuring the trajectory includes a process of obtaining a change in the projection trajectory based on the displacement of the control position, and performing a prediction while limiting the range to be measured to a partial region. A calibration method for the range sensor as described.

[0014]

A jig having an n-polygonal concavity whose shape is known in advance is three-dimensionally moved while continuously irradiating a laser beam and observing the characteristics of the trajectory of the laser beam. The parameters and the laser parameters are determined simultaneously. Further, by using information for deflecting the laser light, the position of the laser light observed by the camera after the deflection is predicted, and the target area is limited to a partial area on the camera image plane.

[0015]

Embodiments of the present invention will be described below with reference to the drawings. Note that the same reference numerals in the drawings denote the same parts as those in the related art, and a description thereof will be omitted. FIG. 1 is an explanatory diagram schematically illustrating an embodiment of the present invention in a range sensor, for example.

In FIG. 1, a main part of this embodiment is a calibration jig 1 having an n-polygonal concave portion 10.
1, a laser beam (arrow) 12 is scanned and projected on this jig, and the projected image P is captured and measured with the line of sight (arrow) 13 of the camera, and is measured in a common measurement environment using a single jig.・
Parameter section 14 for simultaneously obtaining each parameter of the sensor
And a position prediction unit 15 for predicting the position where the projection image P exists. Incidentally, X _L Y _L Z _L is the laser coordinate system, X _P Y _P
Z _P is the measurement space coordinate system, X _C Y _C Z _C is the camera coordinate system.

In this calibration method, the laser coordinate value ( _xl, y) on the laser control surface _GL is moved while the calibration jig 11 is moved variously in the measurement environment described above.
Controls _l), the camera coordinate value in the camera image plane G _C
(x _c, y _c ) is measured, and this laser coordinate value (x _l, y _l ) is known camera coordinate value (x _c, y _c ) and known calibration coordinate value (x _p, y _p, z _p ) to estimate the laser parameter [A] and the camera parameter [B].

The n polygonal pyramid of the calibration jig 11 is, for example, a quadrangular pyramid, and a spot coordinate value is set at an intersection P between the concave 10 of the quadrangular pyramid and the optical axis of the laser beam (arrow) 12.
(x _s, y _s, z _s ), and a vertex P _S of a quadrangular pyramid having a known shape has a calibration coordinate value (x _p, y _p, z _p ). Are arranged toward the laser coordinate system and the camera coordinate system.

FIG. 2 is a diagram showing the locus of the laser beam in FIG.
Description that outlines a method for obtaining an image on a camera image plane
FIG. In FIG. 2, first, a laser beam is applied to a coordinate axis X._L
Scan in parallel with the spot plane S _SAnd generate this
At the position where the square plane and the square pyramid
Trajectory L_sIs formed by projection.

Next, determine the path image I _L by connecting the projected image of the spot trajectory L _s in the camera image plane G _C captured by the camera. The path image I _L is four each ridge line of the pyramid 21,2
A polygonal line having a feature point Q1, Q2 at the intersection between the 1 and the spot trajectory L _s.

[0021] Subsequently, the coordinate axis Y takes a predetermined interval along the _L sequentially forming a plurality of spots plane S _S, which to seek the same number of path image I _L sequentially corresponding feature points Q1
And a group of camera coordinate values (x _c, y _c ) of the feature point Q2. Finally, the image of the vertex P _S of the quadrangular pyramid is obtained from the camera coordinate values (x _c, y _c ) of the feature points Q1 and Q2.

FIG. 3 is an explanatory diagram for schematically explaining a method for obtaining the image of the apex of the quadrangular pyramid in FIG. 2 on the camera image plane. In FIG. 3, the above-mentioned feature point Q1 and feature point Q
The camera coordinate values (x _c, y _c ) are classified into two groups, and a straight line group formed by approximation is obtained for each group, and the camera coordinate value (x _c, y _c ) at the intersection is obtained. Ask. The obtained intersection is an image on which the vertices of the quadrangular pyramid are projected, and the ridge lines and vertices, which are the straight line groups, maintain a mutually fixed positional relationship in the concave of the quadrangular pyramid. .

This fixed positional relationship is based on the jig 1 described above.
Each calibration coordinate value at 1 (x _p, y _p, z _p )
Already given by the group. Therefore, via the known calibration coordinate values (x _p, y _p, z _p ), the laser coordinate values (x _l, y _l ) given as control values and the camera coordinate values (x _c, y _c ) obtained are obtained. The following equation can be established.

Next, the position of the calibration jig is moved or the posture is changed, and the laser coordinate value (x _p, y _p, z _p ) is changed from the new calibration coordinate value (x _p, y _p, z _p ) again by the above-described method. _l, y _l ) and the camera coordinate values (x _c, y _c ) are determined, and simultaneous equations are established using the respective numerical values. In other words, repeat the measurement more than the number of times corresponding to the number of necessary conditional expressions,
Each parameter of the laser and the camera can be obtained.

Here, the number of necessary conditional expressions is at least six in order to distribute each component of the three-dimensional coordinates to each parameter. By increasing this number, the accuracy of calibration is further improved. be able to.

[0026] By the above method, a predetermined number of calibration coordinates _{(x p, y p, z} p) with the same number of laser coordinate values (x _l, y _l) and the camera coordinate values (x _c, y _c) Is obtained. Next, a description will be given of a method for calculating the laser parameters [A] and the camera parameters [B] simultaneously by arithmetically processing these coordinate values.

First, each coordinate value is subjected to arithmetic processing using the conversion model described above, thereby generating a necessary number of linear simultaneous equations for obtaining each parameter. However, in general, the conversion formula based on the conversion model becomes non-linear, and it is extremely difficult to perform the arithmetic processing using the above conversion model as it is.

Then, each three-dimensional coordinate system is converted into a homogeneous coordinate system as shown in the following equation ₁ by using intermediate variables, and a laser matrix [L] is obtained from each laser coordinate value ( _{xl, yl} ).
The transposing each calibration coordinates _{(x p, y p, z} p) the calibration matrix (P) from each camera coordinate values (x _c, y _c) from the camera matrix [C], respectively Set as a matrix.

[0029]

[Number 1] [L] = _{[W l * x l, W l} * Y l, W l ] ^t [P] = _{[X p, Y p, Z p} , 1 ] ^t [C] = [W _c * x _c , W _c * Y _c , W _c ] ^t

Subsequently, the laser matrix [L] and the camera matrix [C] are associated with each other via the calibration matrix [P], and the relational expressions among the respective matrices [P], [L], [C] are obtained. Are set as shown in Equation 2, respectively.

[0031]

[L] = [A] [P] [C] = [B] [P]

FIG. 4 is a flowchart specifically showing the flow of processing in the parameter section of FIG. In FIG. 4, this parameter section is provided with an initial setting stage in which an initial value required for the following measurement is previously set in each register, and the calibration jig is positioned to control the laser beam. It is designed to direct.

In this initial setting step, a calibration coordinate value (x _p, y _p, z _p ) which is a predetermined value corresponding to the i-th positioning by the jig is subjected to processing 401 to execute the j-th laser beam irradiation. in Y-axis component y _l a process 402 of the coordinate values in the laser control surface G _L is a control value corresponding to the order of scanning lines, a control value corresponding to the k-th spot by the laser beam on the scanning line lasers each X-axis component x _l of coordinate values in the control plane G _L in process 403 are set by entering the order. Hereinafter, the i-th, j-th, and k-th subscripts are shown at the upper right shoulder of each coordinate value.

Next, a laser beam is irradiated and the scanning line is projected on a jig, and the projected image is measured by a camera. An image measurement stage and a feature extraction for extracting a characteristic shape in the projected image are performed. Steps are provided in order following the initial setting step, and positions Q1 and Q2 at which the projection image of one scanning line is bent by each ridge line are obtained.

The image measurement step is a process 404 for obtaining the camera coordinate value (x _c, y _c ) of the projection image of the k-th spot on the j-th scanning line in the i-th positioning by the image processing described above; th is provided in order the X-axis component x _l is determined whether or not it reaches the maximum value x _mx 405 of coordinate values in the spot until the laser beam spot finishes scanning one minute scan lines Each process 403, 405
Are sequentially repeated, and the projected image of each spot is sequentially obtained.

In the feature extraction step, the camera coordinate values (x
_c, a process 406 for collecting y _c), y _l-coordinate values are provided and whether the process of determining 407 reaches a maximum value y _mx sequentially, the laser beam scans the entire surface of the laser control surface Until the processing is completed, the processes 402 to 407 are repeated in order to obtain a projection image at each bending position.

Subsequently, the jig is positioned at a plurality of different positions, and for each vertex at each position, a numerical relationship between the projected image of these vertices and the position of the spot is obtained and arranged in a predetermined calibration table. The arrangement stage is provided following the feature extraction stage.

In the calibration table arrangement step, a projection image of each ridge line is formed from each camera coordinate value (x _c, y _c ) group at the bending positions Q1 and Q2, and an intersection point of the projection image group of each ridge line is formed. The camera coordinate value (x _c,
y _c ), and the camera coordinates (x _c, y _c ) of this intersection are
The laser coordinate value ( _{xl, yl} ) at the timing of obtaining this is represented by i
Calibration coordinates (x _p,
y _p, z _p ) and processing 4 for judging whether or not the number of times of positioning described above reaches a predetermined number.
09 are provided in order. Accordingly, each of the processes 401 to 409 until the number of times of positioning reaches a predetermined value.
Is repeated to obtain each coefficient of each equation.

Finally, a group of laser coordinate values ( _{xl, yl} ) arranged in the calibration table and a calibration coordinate value (x
_p, y _p, z _p ) group and the camera coordinate value (x _c, y _c ) group are used to simultaneously establish the above equation (2), and the laser parameter [A] and the camera parameter [B] are calculated. A parameter estimation step 410 for calculating and estimating at once is provided following the calibration table array step. Note that 400 is a start process, and 49 is a start process.
Reference numeral 9 denotes end processing.

The above is a method for obtaining each parameter. Next, a method for executing this method more efficiently will be described. When the laser matrix [L] is varied by δL, the variation δC of the corresponding camera matrix [C] also satisfies the relationship of the above equation (2), so that each parameter that minimizes the measurement error in the entire measured value is obtained. The equation is
If the inverse matrix of each parameter by the least squares method exists,
There is a relationship based on the following equation (3) using the least squares estimation amount as a coefficient.

[0041]

(3) ([A] ^t [A]) ^-1 [A] ^t [L] =
([B] ^t [B]) ^-1 [B] ^t [C]

FIG. 5 is a flowchart specifically showing the flow of processing in the position predicting unit of FIG. In FIG. 5, the position estimating unit firstly limits the area to which the projected image of the spot moves on the camera image plane, and initially provides a limited detection step of detecting the projected image. Can be reduced by predicting the destination.

At the stage of the limited detection, the laser coordinate values ( _xl,
y _l ), the camera coordinate value (x _c,
seeking a small variation δC of y _c) by calculating the number 3 above, the process 501 to be stored in a predetermined memory location, based on the small amount of change, an area on the to be detected camera image plane G _C A process 502 for limiting and a process 503 for determining whether or not a projected image of a spot exists in the limited area are provided in order.

The process 502 corresponds to the small change amount δC.
Camera coordinates (x_c, y_c) Value (δ x _c,δ y_c) To the coefficient k
^-, K⁺Multiplied by the camera image plane G_CDetermine the detection range of
Then, the coordinates 1 and 2 of the range are obtained by the following equation (4).
Camera coordinate values (x_c, y_c)
Set.

[0045]

Equation 4] coordinate range ^{_{1 :( x c - k - *}} δ x c, y c -
k ^- * δ y _c) coordinate range ^{_{2 :( x c + k + *}} δ x c, y c + k + * δ
y _c )

Next, when the limited area cannot be detected, the area expansion step of expanding the limited area is provided following the limited detection step, and the prediction of the destination is slightly enlarged to increase the detection time. Is suppressed.

In this area expansion step, the above-described coefficient
a process 504 for determining whether or not k ^- and k ⁺ reach a maximum allowable value; and a coefficient k ^- and k if not yet reached.
A process 505 for multiplying ⁺ by a predetermined number is provided. Through this process 505, the flow branches to process 502 and shifts to the process 502 until the coefficients k ⁻ and k ⁺ reach the maximum values.

Finally, a stage of whole area detection for detecting spots in all regions of the camera image plane without limiting the detection region is provided following the stage of region expansion. In this whole area detection step, a process 506 of setting the whole area as a target of image processing is performed.
And a process 503 for detecting a projected image of a spot similar to that described above.

FIG. 6 is a configuration diagram showing a configuration example of a range sensor incorporating the parameter unit and the position prediction unit in FIG. In FIG. 6, this range sensor is
A laser control device 61 for controlling the irradiation of laser light, a position control device 62 for obtaining a laser coordinate value ( _{xl, yl} ) on a laser control surface, and an image processing device 63 for processing a projection image to obtain distance information. , A jig position controller 64 for controlling the calibration jig, and a three-dimensional position calculator 65 for calculating the three-dimensional position of the jig.

The laser control unit 61 detects a control value consisting of a position and a posture for irradiating a laser beam and detects a laser coordinate value (x
_{l, yl} ) and sends it to the position controller 62. The image processing device 63 recognizes the position and the line of sight of the camera and matches the camera coordinate value (x _c, y) while collating it with the laser coordinate value ( _{xl, yl} ).
_c ) is sent to the three-dimensional position calculating device 65.

The jig position control device 64 has a calibration
Calibration coordinates from the position and orientation of the calibration jig
Value (x_p, y_p, z_p) To the three-dimensional position calculation device 65
You. The three-dimensional position calculator 65 calculates the camera coordinate value (x_c, 
y_c) And laser coordinates (x_l, y_l) And calibration seat
Standard value (x_p, y_p, z_p) And the laser parameters and camera
・ Estimate parameters and use each parameter
Camera coordinates (x_c, y _c) And laser coordinates (x_l, y_l) Or
Measure the shape of the object to be observed.

The present invention is not limited only to the above-described embodiment. For example, the calibration jig may be an n-polygonal pyramid in order to reduce the obstruction by a shield when the laser beam reaches. Although it is described as having a dent, a plurality of features such as a ridge line or a side can be extracted, and the corresponding laser coordinate values ( _{xl, yl} ) and the camera are measured or calculated based on these feature extraction points. Coordinate value (x _c, y _c )
Of course, various changes can be made within the scope of the present invention, as long as the jig can determine the irregularity of the jig.

[0053]

As described above, the present invention has the following effects. (1) Since the camera parameters and the laser parameters are obtained simultaneously by the same measurement, the work process is simplified, the work time is shortened, and the extra work man-hours can be reduced. (2) Since the measurement errors for both parameters are uniform, the measurement accuracy of the three-dimensional position with respect to the shape of the object to be observed can be improved. (3) Further, since the entire image plane of the camera is not used as a target area for image processing, the target area is limited to a partial area and image processing is performed, so that a laser beam is observed as in a range sensor for obtaining a welding line. Even if the body does not scan the entire surface and tends to be irradiated in a certain direction, unnecessary processing time can be reduced by predicting the range of image processing. As described in (1) to (3) above, by simultaneously obtaining the parameters for the camera, the operation process of the calibration process can be simplified and the operation time can be shortened. In addition, by using this calibration method, it is possible to provide a robot path generation device that simplifies the preparation for measurement, can easily generate an instruction path in a short time, even a complicated work, and can efficiently teach. You will be able to do it.

[Brief description of the drawings]

FIG. 1 is an explanatory view schematically illustrating an embodiment of the present invention.

FIG. 2 is an explanatory diagram schematically illustrating a method of obtaining a trajectory of a laser beam in FIG.

FIG. 3 is an explanatory diagram schematically illustrating a method for obtaining the vertices of the quadrangular pyramid in FIG. 2;

FIG. 4 is a flowchart specifically showing a flow of processing in a parameter section of FIG. 1;

FIG. 5 is a flowchart specifically showing a flow of processing in a position prediction unit in FIG. 1;

FIG. 6 is a configuration diagram schematically showing a configuration example of a range sensor in FIG. 1;

FIG. 7 is a configuration diagram showing a configuration of a conventional general range sensor device.

FIG. 8 is an explanatory diagram for explaining a conventional method of performing measurement using the device of FIG. 7;

[Explanation of symbols]

10 n-polygonal concave portion 11 calibration jig 12 laser light (arrow) 13 camera line of sight (arrow) 14 .... Parameter section 15 ...
... position prediction unit G _L · · · · · · · laser control surface G _C · · · ·
... camera image plane F _C ······· camera principal point F _L ····
・ ・ ・ Laser light source P ・ ・ ・ ・ ・ ・ Projected image P _S・ ・ ・
... pyramid vertex X _C Y _C Z _C ··· camera coordinate system X _L Y _L Z _L
... Laser coordinate system _XP Y _P Z _P ... Measurement space coordinate system (x _c, y _c )
・ ・ ・ Camera coordinate value (x _l, y _l ) ・ ・ ・ ・ ・ ・ Laser coordinate value (x _s, y _s,
z _s ) ・ ・ Spot coordinate value (x _p, y _p, z _p ) ・ ・ Calibration coordinate value

Claims (2)

(57) [Claims] 1. A range sensor for irradiating a laser beam onto an object and a camera device for observing the shape of the object, wherein a laser parameter is a measurement characteristic based on the laser device. And a camera parameter, which is a measurement characteristic based on the camera device, is calibrated by a calibration jig, and the calibration jig has a shape formed into an n-polygonal pyramid. First, a position setting step of three-dimensionally setting a position for positioning the calibration jig and a control position for controlling the laser beam, and then performing calibration while controlling the laser beam. A stage of trajectory measurement for measuring a projection trajectory projected onto the jig three-dimensionally by a camera device, and then triangulating the measured projection trajectory and each set position. Calculation is performed according to the principle, and the arrangement relationship between the calibration jig, the laser device, and the camera device is converted into a uniform three-dimensional position for measurement, and the measurement is repeated while moving the positioning position three-dimensionally. And a parameter estimation step of simultaneously estimating the laser parameter and the camera parameter based on the measured values to sequentially obtain the laser parameters and the camera parameters. A calibration method in a range sensor, comprising: 2. The method according to claim 1, wherein the step of measuring the trajectory includes a process of obtaining a change in the projection trajectory based on the displacement of the control position, and predicting the measurement range while limiting the range to be measured to a partial area. 2. A calibration method in the range sensor according to 1.