JPH08110807A - Method and device for automatic calibration - Google Patents

Abstract

PURPOSE: To automatically generate a geometrical model of slit light. CONSTITUTION: A work 23 for calibration which has flat surfaces in two stages differing in height and is provided with plural marks on the respective flat surfaces, is irradiated with the slit light from a projection device 22 to form intersection lines of the slit light on the respective flat surfaces, and an image of the intersection lines is picked up by an image pickup device 21. A CPU 12 detects the two-dimensional coordinates of the respective marks in the two-dimensional image stored in an image storage part 15, and arithmetic operation using the coordinates and the known three-dimensional coordinates of the respective marks is carried out to calculate a camera model. Then, when slight light irradiation and image pickup processing are performed again, the CPU 12 detects the coordinates of image points of slit images in the two-dimensional image and the arithmetic operation using the three-dimensional coordinates of the coordinates and the image points calculated from the camera model is performed to find the geometrical model of the slit light.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to, for example, a visual recognition technique in an industrial robot, and more particularly, the present invention projects slit light onto an object to be observed and intersects the slit light formed on the surface thereof. The present invention relates to an automatic calibration method and an apparatus for automatically calibrating a geometric model of slit light in a visual recognition device for observing.

[0002]

2. Description of the Related Art Conventionally, there are various methods for recognizing the shape of an object, and as one of them, a three-dimensional object recognition method using a plate-shaped slit light has been proposed. This method irradiates an object to be observed with a plate-shaped slit light by a light projecting device to generate a line of intersection of the slit light on its surface, and after capturing the line of intersection with an imaging device to obtain a slit image, The slit image is image-processed to recognize the object. In the case of this type of system, it is necessary to obtain the camera model and the slit light model in advance by executing the calibration of the imaging device and the light projecting device before the recognition processing.

Conventionally, a geometric model of slit light is calibrated by setting a predetermined calibration work at an observation position, irradiating the slit light, and imaging the intersecting line of the slit light generated on the surface thereof. After detecting the slit image on the obtained two-dimensional image, the detection result,
This is performed by giving a computer the slit light irradiation conditions, the setting conditions such as the installation position of the imaging device and the calibration work, and the correction coefficient, and executing the calibration calculation.

[0004]

However, in the case of such a calibration method, since it is necessary to perform arithmetic processing under each adjustment condition while adjusting the projection device and the image pickup device over a wide range, There is a problem that it takes a lot of time to perform. Further, since this adjustment is performed on a pixel-by-pixel basis, the adjustment work is difficult, and there is also the problem that it is necessary to prepare various correction coefficients.

The present invention has been made in view of the above-mentioned problems, and the geometry of the slit light is calculated by performing the calculation using the camera model of the image pickup device and the position of the intersection line of the slit light on the two-dimensional image. Automatically generates a geometric model,
An object of the present invention is to provide an automatic calibration method and an apparatus therefor which can perform the calibration work easily and in a short time.

[0006]

According to a first aspect of the present invention, there is provided a visual recognition device for irradiating an observation object with slit light and observing a line of intersection of the slit light generated on the surface of the observation object. A method for automatically calibrating the geometric model of the above, which has a calibration surface on the surface that cannot be regarded as a surface on the same plane, and a slit light is projected from the light projecting means onto the calibration work. A first step of irradiating the image with the image of the intersecting line of the slit light generated on the surface thereof by the image capturing means, and detecting the position of the intersecting line on the calibration surface on the obtained two-dimensional image; A predetermined calculation is performed using the image position of the intersecting line detected on the two-dimensional image and the camera model of the image pickup unit obtained in advance to determine the slit light It is characterized by carrying out the second step of generating a biological model in a series.

A second aspect of the present invention is an automatic calibration device for carrying out the above method, wherein the calibration work is irradiated with slit light and a crossing line of the slit light is generated on the surface thereof. Means, two-dimensional imaging means for imaging the intersecting line of the calibration work and the slit light to generate respective two-dimensional images, and the two-dimensional imaging means for imaging the two-dimensional images.
An image storage means for storing a three-dimensional image, a camera model storage means for storing a camera model of the two-dimensional image pickup means, and a line of intersection on the calibration surface on the two-dimensional image stored in the image storage means. Of the slit light by a predetermined calculation from the intersecting line position detecting means for detecting the position of the intersecting line, the camera model stored in the camera model storing means, and the image position of the intersecting line detected by the intersecting line position detecting means. And a calculation means for generating a dynamic model.

[0008]

The slitting light is radiated to the calibration work having the surface for calibration which cannot be regarded as coplanar surfaces, and the intersecting line of the slit light generated on the surface is imaged. The position of the intersecting line on the calibration surface is detected in the two-dimensional image. Then, a calculation is executed using the camera model of the image pickup means and the image position detected about the intersection line, which is obtained in advance, and the geometric model of the slit light is automatically generated.

[0009]

FIG. 1 shows the concept of an automatic calibration device 2 according to the present invention, which is electrically connected to an object recognition device 1 via an interface 3. The object recognizing device 1 is for recognizing the shape of an object, etc., and is composed of an image pickup device 4 and an image processing device 5 for recognizing an object by executing a predetermined image processing by capturing a video signal.

The automatic calibration device 2 is mainly controlled by the CPU 6 of the computer, and is provided with a ROM 7 for storing a program for executing the calibration of the image pickup device 4 and a calculation RAM 8 for storing various data. The CPU 6 decodes and executes the calibration program, takes in the image data of the calibration work 9 from the imaging device 4, and executes a predetermined calibration calculation. As a result, a camera model (details of which will be described later) is generated, and the coordinate conversion coefficient forming the camera model is output to the object recognition device 1.

2 and 3 show an object recognition device incorporating the above-mentioned automatic calibration device,
FIG. 2 is an example of a two-dimensional object recognition device, and FIG. 3 is an example of a three-dimensional object recognition device using plate-shaped slit light.

The device example of FIG. 2 is a CPU 1 of a computer.
2 is a control main body, and for this CPU 12, a ROM 13 in which an image processing program and a calibration program are stored, an image storage section 15 in which an image of a recognized object and an image of the calibration work 9 are stored, and a camera model. In addition to the coefficient storage unit 16 that stores the coordinate conversion coefficient that configures, the RAM 17 that is used for calculation, the display unit 18 that is a CRT and the operation unit 19 that is a keyboard are connected. The image storage unit 15 is composed of an image memory, and the imaging device 2 is connected via the interface 20.
Connected to 1. This image pickup device 21 is, for example, a two-dimensional CC.
It is composed of a D television camera, and images the object to be recognized and the calibration work 9 and outputs the video signal.

As shown in FIGS. 4 and 5, the calibration work 9 is a thin plate having a square planar shape, and a plurality of marks 11 are provided on a flat surface 10 having a constant height (thickness) h. There is. Each of these marks 11 is a circle having the same diameter, and when the calibration work 9 is imaged and binarized, the inside of the circle is black so that it can be clearly distinguished from the background such as the flat surface 10 and the floor surface. It is painted. Further, the total number of the marks 11 is 10, and as shown in FIG. 5, the labels “0” to “9” are assigned to the respective marks 11. The marks 11 are arranged in accordance with a predetermined rule. The marks 11 having labels 0, 5, 6, 7, 8, 9 are located substantially on a diagonal line orthogonal to each other, and the marks 11 of other labels have the lines. It is located outside. The marks 11 in the illustrated example have the same shape, but the shapes may be different for labeling later.

The device example shown in FIG. 3 is different from the device example shown in FIG.
There is a difference in that the form of the calibration work 23 is different and that the light projecting device 22 is provided. The other configurations are the same as those of the embodiment of FIG. 2, and the corresponding configurations are denoted by the same reference numerals and the description thereof will be omitted.

The calibration work 23 in this embodiment has a height (thickness) as shown in FIGS. 6 and 7.
The flat surfaces 24 and 25 having two different levels of h ₁ and h ₂ are provided, and five marks 26 (10 in total) similar to the above are provided on each of the flat surfaces 24 and 25. Labels "0" to "9" are assigned to each mark 11 as shown in FIG. 7, and each mark 11 is assigned based on the same rule as the above-mentioned embodiment.
Has been determined.

The light projecting device 22 is for irradiating the object to be recognized and the work 23 for calibration with a plate-shaped slit light obliquely from above so as to generate an intersection line 27 of the slit light on the surface thereof. The intersection line 27 is imaged by the imaging device 21 to obtain a slit image as an image to be processed.

FIGS. 8 and 9 show the structure of the camera model of the image pickup device 21. In the figure, O _C indicates the center point of the image pickup surface 31 of the image pickup device 21, and a camera coordinate system having the X _C axis, Y _C axis, and Z _C axis as orthogonal axes is set with this center point O _C as the origin. Has been done. R is the lens center of the image pickup device 21, and the imaging distance lemb is given by the distance between O _C R. Further, in FIG. 8, S ₀ is the origin of sampling during image processing, and PC _X and PC _Y are sampling pitches in the X _C axis direction and the Y _C axis direction. The IJ coordinate system in FIG. 8 is an image memory coordinate system fitted to the imaging surface 31, and the xyz coordinate system in FIG. 9 is a reference coordinate system that gives spatial coordinates to an object point.

Here, the camera model is a model expressing the three-dimensional position and orientation of the image pickup device, the image forming distance of the lens, and the digitized specifications of the video signal. The former two are external models and the latter are internal models. Say. The position and orientation of the image pickup device represent the camera coordinate system from the reference coordinate system, and the position and posture of this image pickup device and the image forming distance of the lens change due to movement of the image pickup device and focus adjustment.

When the position / orientation of the image pickup apparatus viewed from the reference coordinate system is [FRAME], this [FRAME] can be expressed as a matrix of 4 rows and 4 columns as in the following equation (1).

[0020]

[Equation 1]

Here, the vector having f ₁₁ , f ₂₁ , and f ₃₁ as components is a unit vector in the X _C direction of the camera coordinate system,
f ₁₁ , f ₂₁ , and f ₃₁ are xyz components in the reference coordinate system of the vector. A vector having f ₁₂ , f ₂₂ , and f ₃₂ as components and a vector having f ₁₃ , f ₂₃ , and f ₃₃ as components are unit vectors in the Y _C direction and Z _C direction of the camera coordinate system, and f ₁₂ , f ₂₂ , f ₃₂ and f ₁₃ , f ₂₃ , f ₃₃
Are xyz components in the reference coordinate system of the vector.

Further, the vector having the components f ₁₄ , f ₂₄ and f ₃₄ is the position coordinate of the origin O _C of the camera coordinate system,
f ₁₄ , f ₂₄ , and f ₃₄ are xyz components in the reference coordinate system. From the above, the following equations (2) to (6) are ideally established for each element of the matrix [FRAME].

[0023]

[Equation 2]

[0024]

(Equation 3)

[0025]

[Equation 4]

[0026]

(Equation 5)

[0027]

(Equation 6)

However, the expression (2) represents the condition of a unit vector, and the expressions (3) to (5) show that the axes of the camera coordinate system are orthogonal to each other.

Next, letting [LENZ] be a matrix representing image formation transformation, this [LENZ] is expressed by the following equation (7).

[0030]

(Equation 7)

Further, the internal model of the image pickup device is the image pickup surface 3
1 shows how the image is sampled, and if this is [SAMP], it can be expressed as a matrix with 3 rows and 3 columns as in the following equation (8).

[0032]

(Equation 8)

In the above equation, ORG _X and ORG _Y are X _C of the sampling origin S _{0 on} the image pickup surface 31 as shown in FIG.
The coordinates and Y _C coordinates are shown.

The three-dimensional coordinates of the object point B _i (shown in FIG. 9) expressed by the reference coordinate system and the camera coordinate system are now (x _i , y _i , z _i , 1) ^t ... e _i, f _i, and g _i, 1) ^t ··· camera coordinate system, also the coordinates of the image point P _i expressed in the camera coordinate system and the image memory coordinate system for the object point B _i, (p _i, q _i , 1) ^t ... Camera coordinate system (I _i , J _i , 1) ^t ... Image memory coordinate system. Note that each of these coordinates is represented by the homogeneous coordinate expression, and t indicates a transposed matrix.

In this case, the object point B _i expressed in the reference coordinate system
The process of converting the three-dimensional coordinates of to the three-dimensional coordinates expressed in the camera coordinate system is expressed by the following equation (9).

[0036]

[Equation 9]

Next, the process of converting the object point coordinates into the image point coordinates by the image formation conversion is expressed by the following equation (10).

[0038]

[Equation 10]

In the above equation, h _i is a variable representing the distortion of coordinates and the like. Further, the process of converting the coordinates of the image point P _i expressed in the camera coordinate system into the coordinates expressed in the image memory coordinate system is expressed by the following equation (11).

[0040]

[Equation 11]

Next, FIG. 10 shows an example of the apparatus of FIG.
The procedure for executing the calibration of the imaging device 21 is shown. This calibration is executed by using the calibration work 23 shown in FIGS. 6 and 7. The center point of each mark image is extracted on the image, and then the three-dimensional center point of each mark 26 is extracted. The camera model of the imaging device 21 is generated from the coordinates (known) and the coordinates on the image of the center point of each mark image.

First, in step 1 (shown as "ST1" in the figure) of the figure, after determining the threshold value TH for binarizing the mark image, the light projecting device 22 is turned off to perform calibration. The work 23 is imaged by the imaging device 21 (steps 2 and 3). The image pickup device 21 images the calibration work 23 and outputs the video signal to the interface 20.
The image of No. 3 is binarized by the threshold value TH, and the bin of the mark image is defined as a black pixel (pixel of data “1”) and the background portion is a white pixel (pixel of data “0”). The image is stored in the image storage unit 15 including an image memory.

Next, in step 4, the CPU 12
The center point of the mark image is extracted from the binary image of the calibration work. FIG. 11 shows an example of this center point extraction method.
2, a mark search window 34 and a mark center point extraction window 35 are set. The mark search window 34 has a vertically long rectangular shape, and the length of the long side is d.
j, the length of the short side is di. The mark center point extraction window 35 has a square shape, and the length of each side is w.

Assuming that the radius of each mark image 33 is r, the sizes of the windows 34 and 35 are the radius r.
It is calculated by the following equations (12) to (14) as a function of.

[0045]

(Equation 12)

[0046]

(Equation 13)

[0047]

[Equation 14]

However, the radius r is obtained from the total area S of black pixels and the number of marks (= 10) by the following equation (15), and even if the observation conditions such as illumination change, the mark image position The accuracy of extraction is less likely to deteriorate.

[0049]

(Equation 15)

First, in order to search the mark image 33, the mark searching window 34 is scanned in the direction of the arrow, and the position where the area of the black pixel in the window 34 occupies 80% of the window area is obtained. Then, the mark searching window 34 is stopped at that position, and the position of the center of gravity G of the black pixel in the window is obtained. The center of gravity is at the center point C of the mark image 33.
It is near _M.

Next, in order to obtain the position of the center point C _M , the mark center point extraction window 35 is set around the center of gravity G, and the center coordinates of the black pixels in this window are obtained as the center point C _M. . When the coordinates of the center point C _M are obtained, the processed mark image may be deleted in order to prevent the same mark image from being overlapped. Thereafter, the same process is executed while scanning the mark search window 34 over the entire surface of the image, and the center point C is calculated for all the mark images 33.
Extract _M.

Returning to FIG. 10, when the center point extraction in step 4 is completed, in the next step 5, the CPU 12 refers to the information on the arrangement of each mark based on the detected coordinate position of the center point C _M of each mark image 33. Then, each mark image 33 is labeled in correspondence with any of the marks.

That is, as shown in FIG. 12, the CPU 12 first obtains the center of gravity g of the center point C _M of the ten mark images 33, and the mark image of the center point C _M -9 closest to the center of gravity g. The label “9” is assigned to 33. Then assign the label "0" in the mark image 33 of the center point C _M -0 located farthest from the center point C _M -9 mark images 33 of the label "9". Then label "9" label from the center point C _M -9 mark image "0"
The unit vector directed toward the center point C _M -0 of the mark image is n, the unit vector perpendicular to the vector n and t. Then the center point C _M -9 labeled "9" as the origin, again representing the coordinates of the center point C _M of each mark image 33 by the coordinate system to the unit vector n, the direction t with the shaft, the i-th The n-coordinate and t-coordinate of the center point C _M of the detected mark image are defined as (n _i , t _i ).

[0054] Next, 1/4 of the distance between the center point C _M -0 of the center point C _M -9 and mark images 33 of the label "0" in the mark image 33 labeled "9" and a threshold ath, Using the center point C _M of each mark image 33 and the threshold value ath, another mark image 33 is generated based on the rules shown in Table 1 below.
Label. In the table, “positive” means n coordinate or t.
The coordinate value is greater than the threshold value ath, "0" means that the absolute value of the coordinate value is less than or equal to the threshold value ath, and "negative" means that the coordinate value is at the threshold value ath. Each is shown to be smaller.

[0055]

[Table 1]

Next, in step 6, the CPU 12 performs a predetermined calculation from the spatial coordinate position of the center point of each mark 26 and the detected coordinate position of the center point of each mark image 33 that has a corresponding relationship with the mark to capture an image. Generate an external model of the device 21.

Now, the matrix representing this external model is [EXT]
Then, this [EXT] is the following (1) from the above equations (9) and (10).
It is expressed as in Eqs. 6) and 17).

[0058]

[Equation 16]

[0059]

[Equation 17]

If each element of [FRAME] ^-1 is expressed as in equation (18), equations (7) and (18) are substituted into equation (16), and equation (19) is obtained. To get

[0061]

(Equation 18)

[0062]

[Formula 19]

Here, the image pickup apparatus 2 having the elements shown in the equation (19)
The external model No. 1 is calculated using the coordinates of the center point C _M of each mark image 33 in the image memory coordinate system and the coordinates of the center point of each mark 26 in the reference coordinate system.

Now, the center point C of the mark image 33 of the label i
_{For M} , the coordinates in the image memory coordinate system are (I _i , J
_i ), and the coordinates expressed in the camera coordinate system are (p _i , q _i ), the following (2
We get expression (0).

[0065]

(Equation 20)

Next, assuming three-dimensional coordinates (x _i , y _i , z _i ) in the reference coordinate system of the center point of the mark 26 of the label i, the above equation (17) is solved using the equation (20). [EX
T] can be obtained. In this case, Eq. (19) is changed to (1
Substituting into equation (7) and rearranging for p _i and q _i , the following (2
1) Equation (22) is obtained.

[0067]

[Equation 21]

[0068]

[Equation 22]

When the above equations (21) and (22) are modified and C ₃₄ = 1 is set, the following equations (23) and (24) are obtained.

[0070]

(Equation 23)

[0071]

[Equation 24]

Here, the matrix [C] is placed as shown in equation (25), and
Further, when the equations (23) and (24) are satisfied for N points (i = 1, ..., N), the matrices [A] and [R] are transformed into the following (2
6) Place as in equation (27).

[0073]

(Equation 25)

[0074]

(Equation 26)

[0075]

[Equation 27]

Thus, by rewriting the equations (23) and (24) for i = 1, ..., N using the above-mentioned respective matrices [C] [A] [R], the following (28 ) Equation matrix equation is obtained. Further, the matrix [C] is obtained by applying the least squares method as in the following expression (29).

[0077]

[Equation 28]

[0078]

[Equation 29]

Therefore, each parameter of the external model of the equation (19) can be obtained from the equations (25) and (29).

Returning to FIG. 10, when the external model is calculated in step 6, the image forming distance and the position / orientation of the image pickup device 21 are calculated from this external model (steps 7 and 8).

In this case, the external model is non-unique, and as is clear from the equations (21) and (22), it holds even if each element of [EXT] is multiplied by K (however, K is a proportional coefficient). . Therefore (2
[C] obtained by solving equation (9) may be K times the original value. Considering this point, the relational expression between each element of Eq. (19) is described as follows. t ₁₁ = C ₁₁ · K ··· (a) t ₁₂ = C ₁₂ · K ··· (b) t ₁₃ = C ₁₃ · K ··· (c) t ₁₄ = C ₁₄ · K · ... (d) t ₂₁ = C ₂₁ · K ··· (e) t ₂₂ = C ₂₂ · K ··· (f) t ₂₃ = C ₂₃ · K ··· (g) t ₂₄ = C ₂₄ · K ···· (h) t ₃₁ = -lenb · C ₃₁ · K ··· (i) t ₃₂ = -lenb · C ₃₂ · K ··· (j) t ₃₃ =- lenb ・ C ₃₃・ K ・ ・ ・ ・ (k) t ₃₄ = lenb-lenb ・ C ₃₄・ K ・ ・ ・ ・ (l)

Further, in each element t _ij of the matrix [FRAME] ⁻¹ , t ₁₁ ² + t ₁₂ ² + t ₁₃ ² = 1 ... (m) t ₂₁ ² + t ₂₂ ² + t ₂₃ ² = 1 ... since ^{_{· (n) t 31 2 +}} t 32 2 + t 33 2 = 1 the constraint · · · · (o) is satisfied, by solving the above (a) ~ (o) type, the following (30 ) Equation (31) is obtained.

[0083]

[Equation 30]

[0084]

[Equation 31]

[0085] Thus (30) (31) by substituting expression of (a) ~ (l) equation [FRAME] Each element t _ij ^-1 can be obtained, also [FRAME] ^-1 reverse [FRAME] is obtained by taking the matrix. Lenb thus obtained
Is the imaging distance, and [FRAME] is the position and orientation of the camera.

When the calibration of the image pickup device 21 is completed, the calibration of the light projecting device 22 is executed based on the procedure shown in FIG. In this calibration, after the camera model is known, the calibration work 23 is irradiated with a plate-shaped slit light to generate a slit image, and the slit image is analyzed, whereby the light projecting device 22 The slit light model of is generated.

Here, the slit beam model is a plane equation of the plane of the slit beam 40 as shown in FIG. The plane equation of the slit light 40 is given by the following equation (32), and therefore the slit light model can be expressed as a set of parameters a _S , b _S , c _S and d _S.

[0088]

[Equation 32]

First, in step 9 of FIG. 13, the threshold value TH 'for binarizing the slit image is determined, and then in step 10, the light projecting device 22 is turned on and the calibration work 23 is plate-shaped. The slit light 40 is emitted. As a result, the intersecting line 27 of the slit light as shown in FIG. 7 is generated on the flat surfaces 24 and 25 of the work 23 and the floor surface, and the intersecting line 27 is imaged by the imaging device 21 (step 1).
1). The image pickup device 21 obtains a slit image by imaging the intersecting line 27 and outputs the video signal to the interface 20. The CPU 12 binarizes the slit image by the threshold value TH ', for example, a portion of the slit image. Are stored as black pixels (pixels of data “1”) and the other portions are white pixels (pixels of data “0”), and the binary image is stored in the image storage unit 15 (step 12).

At the next step 13, the CPU 12 calculates the three-dimensional coordinates of the image point on the slit image. FIG. 15 shows a method of calculating coordinates of image points. In the figure, A _{1 to} A ₄ ,
B _{1 to} B ₄ are corner points of the upper and lower flat surfaces 24 and 25 of the calibration work 23, and the corner points on the work image 41 corresponding to the respective corner points are A ₁ ′ to A ₄ ′ and B ₁ ′. It is shown in ~B _4. An intersecting line 27 of the slit light 40 is generated on the upper and lower flat surfaces 24 and 25 and on the floor surface.
When 3 is imaged, a slit image 42 corresponding to the intersection line 27 appears in the work image 41. The intersecting line 27 is a set of object points B, and the positional relationship between each object point B and its image point P is determined by the line of sight 43 passing through the lens center R.

That is, when the image point P is given, the three-dimensional coordinates of the object point B corresponding to the image point P are the line of sight 43 of the image point P.
And the flat surface of the calibration work 23 (the upper flat surface 25 in the illustrated example). The line representing the line of sight 43 is given by the following equation (33) which is expressed as a parameter. However, k _R is a parameter.

[0092]

[Expression 33]

In the above equation, (x _R , y _R , z _R ) is the three-dimensional coordinate of the lens center R, and is obtained by the following equations (34) to (36) under the known camera model.

[0094]

(Equation 34)

[0095]

[Equation 35]

[0096]

[Equation 36]

Thus, the three-dimensional coordinates of the image point P are (x ₀ ,
y ₀ , z ₀ ), then n _x , n _y , n _z are
Given by the formula.

[0098]

(37)

The three-dimensional coordinates of the image point P (x ₀ , y ₀ , z
₀ ) is the following (38) when the camera model is known.
It is determined by the formula.

[0100]

(38)

Since the line-of-sight 43 of the image point P can be obtained from the above equations, the plane equation of the flat surface on which the object point B is located is next obtained. First, since the shape data of the camera model and the calibration work 23 are known,
Corner points A ₁ ′ -A ₄ ′ and B ₁ ′ -on the work image 41
The coordinates (I, J) of B ₄ ′ on the image are calculated by the following equations (39) and (40).

[0102]

[Formula 39]

[0103]

(Equation 40)

In the above equation, (x _B , y _B , z _B ) means the three-dimensional coordinates of the corner point on the work 23 which has a correspondence relationship with the corner point of the mark image 41 to be calculated.

All corner points A ₁ ′ to A ₄ ′, B ₁ ′ to
Once the three-dimensional coordinates of B ₄ ′ are obtained, the next image point P of interest
Is a square 44 determined by the corner points A ₁ ′ to A ₄ ′ and the corner point B ₁ ′
The following Table 2 determines the inclusion relationship with the quadrangle 45 determined by B ₄ ′ to B ₄ ′, thereby determining whether the object point B is located on the flat surface 24, 25 or the floor surface. To do.

[0106]

[Table 2]

When the plane on which the object point B is located is discriminated as described above, the three-dimensional coordinates of the object point B are calculated by simultaneously solving the equation of the plane and the equation of the line of sight 43. When the three-dimensional coordinates (x _i , y _i , z _i ) of each image point P on the slit image 42 are obtained in this way, the following equation (41) holds between the slit light 40 and the plane parameter. To do.

[0108]

[Formula 41]

However, in the expression (32), c _S = -1 is set. For where N points, by solving the following equation (42), a _S, b _S of the error term e _i in the above equation minimizing the sum of e _i, obtaining a d _S (step 14).

[0110]

(Equation 42)

Here, each matrix of the equation (42) is [R] [A]
Putting [Z], this equation (42) can be expressed by the following equation (43).

[0112]

[Equation 43]

In this equation, the matrix [A] representing the slit light model is given by the following equation (44).

[0114]

[Equation 44]

Although the above has described the calibration of the image pickup device 21 and the light projecting device 22 with respect to the example of the apparatus shown in FIG. 3, it is sufficient for the example of the apparatus shown in FIG. Since the method is similar to the above, the description thereof is omitted here.

[0116]

As described above, the present invention was obtained by irradiating a slitting light on a calibration work having a calibration surface which cannot be regarded as a coplanar surface on the surface, and imaged the slit work. Since the geometrical model of the slit light is automatically generated by using the position of the line of intersection of the slit light on the two-dimensional image and the camera model of the image pickup means which is obtained in advance, a three-dimensional object recognition device, etc. In this case, the geometrical model of the slit light required in the above can be automatically calibrated, and the remarkable effects of achieving the object of the invention are achieved.

[Brief description of drawings]

FIG. 1 is a block diagram showing the concept of an automatic calibration device according to the present invention.

FIG. 2 is a block diagram showing an embodiment of the present invention.

FIG. 3 is a block diagram showing another embodiment of the present invention.

FIG. 4 is a perspective view showing an example of a calibration work.

FIG. 5 is a plan view showing an example of a calibration work.

FIG. 6 is a perspective view showing another example of the calibration work.

FIG. 7 is a plan view showing another example of the calibration work.

FIG. 8 is an explanatory diagram illustrating a structure of a camera model of an imaging device.

FIG. 9 is an explanatory diagram illustrating a structure of a camera model of an imaging device.

FIG. 10 is a flowchart showing a procedure of performing calibration of the image pickup apparatus.

FIG. 11 is an explanatory diagram showing a method of extracting a center point of a mark image.

FIG. 12 is an explanatory diagram showing a method of labeling mark images.

FIG. 13 is a flowchart showing a procedure for executing calibration of the light projecting device.

FIG. 14 is an explanatory diagram illustrating a structure of a slit light model.

FIG. 15 is an explanatory diagram showing a method of calculating coordinates of an image point.

[Explanation of symbols]

 2 Automatic calibration device 12 CPU 21 Imaging device 22 Projection device 23 Work for calibration

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁶ Identification code Internal reference number FI Technical display location G06T 1/00 G06F 15/62 415 15/64 M

Claims (5)

[Claims] 1. A visual recognition device for irradiating an observation object with slit light and observing a line of intersection of the slit light generated on the surface of the observation object, wherein a geometric model of slit light is automatically calibrated. It is a method for performing the same, and it is generated on the surface by irradiating slit light from the light projecting means to the calibration work that has the surface for calibration that cannot be regarded as coplanar A first step of imaging the intersecting line of the slit light by an imaging means and detecting the position of the intersecting line on the calibration surface on the obtained two-dimensional image, and the intersecting detected on the two-dimensional image. A second calculation for generating a geometric model of slit light is performed by performing a predetermined calculation using the image position of the line and the camera model of the image pickup unit obtained in advance. Automatic calibration method which comprises carrying out the degree in a series. 2. The automatic calibration method according to claim 1, wherein the calibration work has a plurality of surfaces that are not located on the same plane. 3. An automatic calibration for automatically calibrating a geometric model of slit light using a calibration work having a surface for calibration which cannot be regarded as a surface on the same plane. A projection device for irradiating the calibration work with slit light to generate a line of intersection of the slit light on the surface thereof, and imaging the line of intersection of the calibration work and the slit light. Two-dimensional imaging means for generating each two-dimensional image, image storage means for storing the two-dimensional image captured by the two-dimensional imaging means, and camera model storage for storing a camera model of the two-dimensional imaging means Means and a calibration plane on the two-dimensional image stored in the image storage means. The intersecting line position detecting means for detecting the position of the intersecting line, the camera model stored in the camera model storing means, and the image position of the intersecting line detected by the intersecting line position detecting means are subjected to a predetermined calculation to determine the slit light An automatic calibration device comprising a calculation means for generating a geometric model. 4. The automatic calibration device according to claim 2, wherein the calibration work has a plurality of surfaces that are not located on the same plane. 5. The calibrating work is provided with each flat surface having two levels of height.
The automatic calibration device described in.