JP3207023B2 - Calibration method of image measurement device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION This invention relates to a plurality of cameras,
The present invention relates to an image measuring device that determines three-dimensional coordinates of a subject by arranging lenses such that their optical axes are parallel to each other and their focal lengths are arranged on a common plane perpendicular to the optical axis.

[0002]

2. Description of the Related Art The present inventors have disclosed, for example, JP-A-4-307.
No. 309 has been proposed. This is an apparatus in which a plurality of cameras are arranged so that their optical axes are parallel to each other, and their focal points are located on a common plane perpendicular to the optical axis. One point is observed with a plurality of cameras, and three-dimensional coordinates of the point are obtained from the position coordinates of the point on the screen. This is to obtain the coordinates of the same point on the screen of each camera using three or more cameras.

The principle will be described with reference to FIG. Although three or more cameras are used, for the sake of simplicity, here, a principle method for obtaining the distance to a point using two cameras will be described. There are right and left cameras. _OL is the position of the left camera focal point and the origin of the camera coordinates. O
_R is the focal point of the right camera and the origin of the coordinates of this camera. The plate before point O is the spread of the lens. Point P
Is the goal of position measurement in this case. D for camera spacing
And

In a camera optical system, a lens is centered,
There is an imaging surface at the position of the focal point. The imaging surface has a spread, and the lens is a point through which the optical axis passes. When an object is imaged by a camera, the object apertures converge at a single point with a lens, spread again on the rear imaging surface, and form a real inverted image. If this is faithfully described, the image will be inverted in the coordinate system before and after the lens. The image is on the opposite side of the object and all coordinates are inverted. The coordinates of the object and the image are in a direct proportional relationship with a negative proportional constant.

Of course, the image can be handled as it is, but in order to simplify the mathematical processing, the imaging surface is set to one point and the lens surface is made to be a wide surface. Then, the intersection between the line connecting the imaging center O and the object and the lens surface is an image of the object,
It becomes an erect image. Since the object and the image are on the same side as viewed from the point O, the size, coordinates, and the like of the image have a direct proportional relationship having a positive proportional constant. And the optical axis of the camera is parallel,
The focal length is also fixed. Left and right camera screen center at C
_L and C _R.

The image of the point P is displayed on the screen of the left and right cameras by M
_L and M _R points can be set. The deviation of the position on the screen of the M _L and M _R, called a parallax d. The x-coordinate of M _L X _L, when the X-coordinate of M _R and X _R, disparity d is d = | X _R -X _L | at given, and because the optical axis of the camera are parallel, to the image The distance L is

L = fD / d (1)

[0008] This is the case when the point P is on a plane including the centers O _L and O _{R of} the camera and the centers C _L and C _R of the lens. If you're far from it, you'll have to make some corrections. Anyway, the distance in the direction perpendicular to the lens surface can be obtained by such a method. On the other hand, coordinates in a direction orthogonal to the direction are obtained from the two-dimensional position of the image on the screen. Actually, since three or more cameras are used, the accuracy of the distance in the direction perpendicular to the lens surface is increased. Further, by arranging a plurality of cameras in a direction perpendicular to the base line O _L O _R of these group of cameras, even O _L O _R direction coordinate can be accurately obtained.

FIG. 5 shows a device closer to an actual device.
Three cameras are arranged with their optical axes aligned. An image of the same measurement point P is formed on a screen behind the lens. The deviation from the center of the image in plan view is set to _{d 1, d 2, d 3} . The displacement of the image in a direction perpendicular to this is represented by a common variable V. Assuming that the distance of the camera is D, the X of P
The coordinates in the direction, the coordinates Y perpendicular to the direction, and the coordinates Z in the direction perpendicular to the plane can also be obtained.

FIG. 2 shows an image processing system of each camera. TV
An image is captured by a camera to obtain an analog image, which is read out in raster order and A / D converted for the light intensity of each small area. Feature points are extracted as a black and white binary image. A feature point is a point on a black-and-white boundary line and can be obtained by differentiation or the like. It can be said that the continuation of the feature points is an outline. The coordinates of the feature points on the imaging screen are stored in the image memory. This is processed by the CPU.

[0011]

[Problems to be solved by the invention]

By using a calculation formula such as (1), the distance to the object can be easily obtained. At high speed, three-dimensional coordinates of an object can be obtained. However, this method has several assumptions. The distance between the cameras is constant and known, and the optical axes of the cameras are parallel.

Since the distance D is included in the equation (1), it is natural that the distance D needs to be constant. What is meant by the parallel optical axes is that if the optical axes are not parallel, the coordinates d ₁ , d ₂ ,
Are changed, and the calculation of the distance L is erroneous. If the optical axis of the camera deviates from parallel by an angle Θ, the distance is LΘ
Only errors will be included. The fact that the optical axes of the cameras are parallel is very important for this measurement method. In other words, in this measuring method, two conditions, that is, the camera distance is constant and the optical axis is parallel, must be satisfied.

Once the camera is fixed, it does not move.
The distance between cameras does not change. However, the condition parallel to the camera optical axis is not always satisfied. Since the optical axis of the camera tends to fluctuate, it is necessary to adjust the optical axis whenever the camera group is moved. However, this adjustment itself is manually performed, there is a variation between people, and there is also an error. Further, it is complicated and necessary to adjust the optical axis every time the camera group moves. In an actual system, the distance to the target is often 10 m or more, but the position measurement error is required to be 1% or less. SUMMARY OF THE INVENTION It is an object of the present invention to provide a method for automatically calibrating measurement data by knowing deviation of optical axes from parallelism without relying on manual adjustment for making optical axes of a plurality of cameras parallel. And

[0014]

According to a method of calibrating an image measuring apparatus according to the present invention, a plurality of cameras are arranged on a straight line, and all cameras are installed so that their optical axes are at right angles to the straight line, and the same point is measured. A method of calibrating an image monitoring apparatus in which the three-dimensional coordinates of the point are obtained from the coordinates of the image on the imaging plane of each camera, wherein the intersections with the optical axes of all the cameras and the vicinity of the optical axes of the respective cameras A target having at least two or more landmarks is set at a predetermined position in front of the apparatus, and the coordinates of the landmarks on the image obtained by each camera and the coordinates of the landmarks in the camera in the normal direction are given in advance. From the coordinates, the rotation angle from the normal position of the camera is calculated, and the camera position is corrected by rotating the camera by the amount of rotation in the opposite direction, or the calculation is performed without correction, thereby calculating the three-dimensional coordinates. Constituted by a calibration means for calibrating.

[0015]

The normal direction means that the direction of the camera is correct and points in a direction perpendicular to the base line (Z-axis direction). The target is set at a position distant from the camera group, but the present invention is established assuming that the coordinates of the mark on the imaging plane of the camera in the normal direction are known in advance. That is, it is assumed that the correct orientation of the camera is defined and that the target is correctly positioned on this extension. The following calculation is based on the condition that the two-dimensional coordinates of the mark in the camera in the normal direction are known. That is, the target is aligned and installed.

A base line on which a plurality of cameras are arranged is defined as an X axis. The imaging surface (taken on the lens surface) of the camera is parallel to the XY plane. That is, the Y axis is perpendicular to the camera baseline and parallel to the imaging plane. It is assumed that the Z axis is perpendicular to the imaging plane. Thus, the three-dimensional coordinates of the camera are defined. Consider a point represented by P (X, Y, Z) at these coordinates. Camera is X
It is assumed that the rotation is α around the axis, β around the Y axis, and γ around the Z axis. It is assumed that the rotation is performed in this order. The rotation order is not commutative.
The same point P is the camera coordinates after rotation (X ', Y', Z ')
Let's say that The relationship between the coordinates before and after the rotation of the angle α about the X axis is given by the following determinant.

[0017]

(Equation 2)

However, here, a parameter representing the ratio of dimensions is used.
Consider the coordinates of four variables including one parameter. Β around the Y axis
Is expressed as follows.

[0019]

(Equation 3)

The rotation of γ about the Z axis is as follows.

[0021]

(Equation 4)

By multiplying these, the coordinates (X ', Y', Z ') of the same point P in a camera rotated by γ around the Z axis, β around the Y axis, and α around the X axis are represented by a matrix. Given by W.

[0023]

(Equation 5)

[0024]

(Equation 6)

When the elements of the matrix W are expressed by {a _ij },

[0026]

(Equation 7)

Here, {a _ij } is defined as follows: a ₁₁ = cos βcos γ (8) a ₁₂ = −cos βsin γ (9) a ₁₃ = sin β (10)

A ₂₁ = sin αsin βcos γ + cos αsin γ (11) a ₂₂ = −sin αsin βsin γ + cos αcos γ (12) a ₂₃ = −sin αcos β (13)

A ₃₁ = −cos αsin βcos γ + sin αsin γ (14) a ₃₂ = cos αsin βsin γ + sin αcos γ (15) a ₃₃ = cos αcos β (16)

As described above, a coordinate conversion equation based on the rotation of the camera is obtained. Assuming that the image of the point P (X, Y, Z) is projected on the two-dimensional coordinates (x, y) on the camera imaging surface, the following relational expression holds.

[0031]

[Equation 17]

Since the imaging plane of the camera is located at a distance f from the origin O and the object is at a distance Z from the origin O, the coordinates X, Y of the object
And the two-dimensional coordinates x and y on the imaging surface are in direct proportion, and the proportionality constant is Z / f. X = (Z / f) x, Y =
(Z / f) y. The equation for Z in the third row of the determinant is an identity. The expression for 1 in the fourth row is included to express the enlargement ratio. Here, it is an identity. The same point P is set to (X, Y, Z) in this coordinate system due to the rotation of the camera. Since the length Z ′ in the Z-axis direction is used as a reference, when this is first obtained,

X ′ = a ₁₁ X + a ₁₂ Y + a ₁₃ Z = (a ₁₁ x + a ₁₂ y + a ₁₃ f) (Z / f) (18)

Y ′ = a ₂₁ X + a ₂₂ Y + a ₂₃ Z = (a ₂₁ x + a ₂₂ y + a ₂₃ f) (Z / f) (19)

Z ′ = a ₃₁ X + a ₃₂ Y + a ₃₃ Z = (a ₃₁ x + a ₃₂ y + a ₃₃ f) (Z / f) (20)

From the above relationship, the point P of the camera system shifted from the coordinate (x, y) of the image of the point P on the camera of the regular camera system
The relationship between the coordinates (x ', y') of the image is as follows.
Using X ′ = (Z ′ / f) x ′ and Y ′ = (Z ′ / f) y ′,

X ′ = (a ₁₁ x + a ₁₂ y + a ₁₃ f) f (a ₃₁ x + a ₃₂ y + a ₃₃ f) ⁻¹ (21) y ′ = (a ₂₁ x + a ₂₂ y + a ₂₃ f) f (a ₃₁ x + a ₃₂ y + a ₃₃ f) ^-1 (22)

## EQU4 ## This is a conversion formula of the coordinates on the imaging plane of the camera. Such an equation holds for each individual camera. The camera rotation angles α, β, and γ should also be determined for each camera. Here, one camera is described, but the same operation is performed for other cameras.

The reason for taking two landmarks for one camera is that two points are needed to determine three rotation angles. Here, for simplicity, it is assumed that one of the two marks P ₁ (X ₁ , Y ₁ , Z ₁ ) is set to be located on the extension of the optical axis. This is a strong condition, on the premise that the position of the mark is fixed. Because it is a strong condition,
Two of the rotation angles can be determined. Then X ₁ , Y
₁ is 0. The corresponding coordinates (x ₁ , y ₁ ) on the camera imaging surface are x ₁ = 0 and y ₁ = 0. Since the optical axis of the camera is rotated by α around the X axis, β around the Y axis, and γ around the Z axis, the image of the same object point P is (x ₁ ′,
y ₁ ′).

X ₁ ′ = a ₁₃ f / a ₃₃ = fsin β / cos αcos β = ftan β / cos α (23) y ₁ ′ = a ₂₃ f / a ₃₃ = −fsin αcos β / cos αcos β = − ftan α (24)

From this, conversely, the angles α and β of the deviation in the direction of the camera can be obtained.

Α = tan ⁻¹ (−y ′ / f) (25) β = tan ⁻¹ (x′cos α / f) (26)

What is desired is three rotation angles α, β, and γ. This equation gives α and β. You can see α and β.
Two rotation parameters can be determined with one mark. This, however, indicates that the landmark is on the camera's normal optical axis (x ₁ , y
₁ ) = (0,0).

If the position of the mark is not known a priori, such a simple equation cannot be obtained. The rotation angle is determined independently for each camera, but in order to be able to determine the rotation angle using this simple formula, the first landmark must be on the regular optical axis of each camera. Must be. For this purpose, for example, it is convenient to use a checkered sign as shown in FIG. The length of a predetermined number of eyes is set to be equal to the camera interval D. Since the optical axis of the legitimate camera cannot actually be known from the beginning, it is impossible to set the first landmark on the legitimate camera optical axis.

However, this is no problem. Assuming that any camera has an optical axis perpendicular to the mounting plane (xy plane) of the camera, this is set as a standard camera, and the position of the marker is determined so that the mark 1 of this camera comes on the optical axis of the standard camera. Since the optical axis is shifted from the mark 1 of the other camera, the angles α, β, and γ of the shift are obtained. The rotation angle is determined for each camera, but this operation is not performed for a standard camera.

Α and β can be obtained from equations (25) and (26). α is obtained first because the rotation equation described above is rotated around the Z axis, around the Y axis, and then around the X axis. Since the rotation around the X axis is the last rotation, α = tan ⁻¹
It becomes a simple expression like (−y ′ / f). This is intuitive. β is the rotation around the Y axis, which is affected by the rotation around the X axis, so that cos α enters. This is also intuitive.

Now, γ remains. As far as looking at a point on the optical axis, γ is not determined. This is because the optical axis is parallel to the Z axis. Further complicating the determination of γ is the order of the coordinate rotation described above. Since the rotation about the Z axis, the Y axis, and the X axis is performed in order, the rotation about the Z axis that has been rotated last can be easily understood. However, since the rotation about the Y axis is performed after that, the situation becomes slightly complicated. However, the rotation about the Z axis is rotated by the Y axis and the X axis after this, so that it is represented by a rather complicated equation. Z by the second landmark
Determine the rotation angle γ about the axis. α and β are already known. The known amounts contained in a _{31 to} a ₃₃ , a _{11 to} a ₁₃ , and a _{21 to} a ₂₃ are set as follows.

K = −cos αsin β (27) 1 = sin α (28) m = cos αcos β (29)

Q = cos β (30) r = sin β (31) t = sin αsin β (32)

U = cos α (33) v = −sin α cos β (34)

Using this definition, the coefficient a includes only γ as an unknown as follows.

A ₃₁ = k cos γ + I sin γ (35) a ₃₂ = −k sin γ + I cos γ (36) a ₃₃ = m (37)

A ₁₁ = q cos γ (38) a ₁₂ = −q sin γ (39) a ₁₃ = r (40)

A ₂₁ = t cos γ + usin γ (41) a ₂₂ = −tsin γ + ucos γ (42) a ₂₃ = v (43)

It is assumed that the coordinates of the second mark P _{2 in} the regular camera imaging system are (x ₂ , y ₂ ), and this is (x ′ ₂ , y ′ ₂ ) due to the rotation of the camera. I do. It is assumed that the imaging plane coordinates (x ₂ , y ₂ ) in regular coordinates are known. The coordinates of the imaging plane (x ′ ₂ ,
y ′ ₂ ) can be measured. From these, γ should be known. The relational expressions (21) and (22) hold between these coordinates.

X ′_Two = F (qcos γx_Two −qsin γy_Two + Rf) ｛(k cos γ + lsin γ) x _Two + (-Ksin γ + lcos γ) y_Two + Mf｝^-1 (44)

Y ′ ₂ = f ｛(t cos γ + usin γ) x ₂ + (− tsin γ + ucos γ) y ₂ + v f｝ ｛(k cos γ + lsin γ) x ₂ + (− ksin γ + lcos γ) y ₂ + mf｝ ⁻¹ ( 45)

Is as follows. When the coefficients are put together, the following quantities are put out, so that they are A and B.

A = x ₂ cosγ−y ₂ sinγ (46) B = x ₂ cosγ + y ₂ cosγ (47)

(A, B) is the coordinates of a point obtained by rotating (x ₂ , y ₂ ) around the Z axis by γ. As described above, since the rotation is performed in the order of Z, Y, and X, it is difficult to cut out the rotation around the Z axis independently. First, (x ₂ , y ₂ ) is converted to γ.
(A, B) is rotated by β around the Y axis and rotated around the X axis by (x ₂ ′, y).
₂ '). Conversely, from (x ₂ ′, y ₂ ′) to (A, B)
It is trying to ask for. Substituting (A, B) into the above equation,

X ′ ₂ = f (qA + rf) ｛(kA + 1B + mf｝ ⁻¹ (48) y ′ ₂ = f ｛(tA + uB + vf｝ ｛(kA + 1B + mf｝ ⁻¹ ) (49)

This is rewritten as an equation in which A and B are unknown numbers.

(Ky ₂ ′ -ft) A + (y ₂ ′ l-uf) B = vf ² −mfy ₂ (50) (kx ₂ ′ −qf) A + x ₂ ′ 1B = rf ² −mfx ₂ ′ (51 )

Since this is a quadratic equation for A and B having a known number as a coefficient, it can be easily solved.

[0065]

(Equation 52)

By solving this equation, A and B are obtained. Using this, from (46) and (47),

[0067] _{sin γ = (- y 2 A} + x 2 B) / (x 2 2 + y 2 2) (53)

Since the above holds, the value of γ is obtained by substituting it. Equations (46) and (47) are rotation equations, but since they are included angles of vectors of the same length, sin γ is obtained by dividing these cross products by the square of the vector length. Intuitively, the meaning of (53) is understood. If γ is known, all the rotation angles of α, β, and γ are known. The optical axis of the camera is -α around the X axis, -β around the Y axis,
Rotate −γ about the Z axis. Then, the optical axis of the camera is directed in the normal direction. This is the end of the camera direction correction of the present invention. Since the calculation is performed by a computer, such a calculation can be performed instantaneously.

However, in order to make the form of the solution easier to understand, a further explanation will be given. Substituting the expressions for α and β into the constant term,

A = (x′cos β−y ′ sin αsin β−fcos αsin β) / (x′sin β−y ′ sin αcos β + fcos αcos β) (54) B = (y′cos α + fsin α) / ( x′sin β−y′sin αcos β + fcos αcos β) (55)

Here, for the sake of simplicity, the x and y suffixes 2 are omitted.

Sin γ = {xy′cos α−x′ycos β + yy′sin αsin β + f (xsin α + ycos αsin β)} / (x′sin β−y′sin αcos β + fcos αcos β) (x ^Two + Y^Two ) (56)

This can directly give γ. α
And β have been found earlier. The second landmark takes the image plane coordinates (x ₂ , y ₂ ) when the camera should be, and the image plane coordinates (x ₂ ′, y ₂ ′) because the camera is rotating. Since both are known amounts, γ can be known.

Calculations can be made simpler by adopting special points. For example, x ₂ = 0. That is, when the point on the y-axis is adopted as the mark 2 when the camera is in the normal camera direction, the coordinates of the rotated camera on the imaging surface are (x ₂ ′, y ₂ ′), and the expression of sin γ is given. Is

[0075] _{sin γ = (- x 2 '} y 2 cosβ + y 2 y 2' sin αsin β + fy 2 cosαsin β) / (x 2 'sin β-y 2' sin αcos β + fcos αcos β) y 2 2 (57)

Is as follows. Some calculations are simplified. Or, if we choose a special point, y ₂ = 0, sin
The equation for γ is

Sin γ = {x ₂ y ₂ ′ cos α + fx ₂ sin α} / (x ₂ ′ sin β−y ₂ ′ sin αcos β + fcos αcos β) x ₂ ² (58)

This is further simplified. It can be calculated using any of the formulas. Mark 1 is P ₁ x ₁ =
The points at which 0 and y ₁ = 0 are selected, and α and β are determined so that γ is not included in the equation. The mark 2 selects the more general point P ₂ and obtains the remaining γ. Further, γ may be calculated using the mark 3, the mark 4,... If different values of γ are obtained, the optimal γ is obtained by the least squares method.
In any case, in the present invention, it is necessary that the positions of the images on the imaging surface of these marks with respect to the normal camera direction are known.

The position of the mark 1 is always x ₁ = 0, y ₁ =
It need not be zero. If a point where this condition is not satisfied is selected, the equation becomes the same as that of the mark 2, so that the angles α, β, and γ can be calculated by combining these. However, the calculation becomes slightly complicated because it does not become a linear equation.

There are two calibration methods. One is to reorient the camera. In this order, the camera is rotated by −α around the X axis, −β around the Y axis, and −γ around the Z axis. All cameras are strictly parallel, baseline (straight line with cameras)
At right angles to The other is to correct the coordinates (x, y) of the imaging plane of the camera without changing the direction of the camera. As shown in the lower part of FIG. 5, an image of the same point P is obtained below each. The coordinates (x, y) are out of order due to the tilt of the camera. Let the correct coordinates be (x ', y'). This relationship is derived from (21) and (22)

X = (a ₁₁ x '+ a ₁₂ y' + a ₁₃ f) f (a ₃₁ x '+ a ₃₂ y' + a ₃₃ f) ^-1 (59) y = (a ₂₁ x '+ a ₂₂ y' + a ₂₃ f) f (a ₃₁ x '+ a ₃₂ y' + a ₃₃ f) ^-1 (60)

Is satisfied. By solving this in reverse, the corrected coordinates (x ', y') are obtained. Here, since all the coefficients are known, this calculation can be performed. The inverse transformation may be performed, but here, the inverse transformation formula can be obtained more easily.
What is necessary is just to reverse the signs of α, β, and γ in the matrix coefficients a _ij . _Expressing this as matrix coefficients b _ij ,

X ′ = (b ₁₁ x + b ₁₂ y + b ₁₃ f) f (b ₃₁ x + b ₃₂ y + b ₃₃ f) ⁻¹ (61) y ′ = (b ₂₁ x + b ₂₂ y + b ₂₃ f) f (b ₃₁ x + b ₃₂ y + b ₃₃ f) ^-1 (62)

B ₁₁ = cos βcos γ (63) b ₁₂ = cos βsin γ (64) b ₁₃ = −sin β (65)

B ₂₁ = sin αsin β cos γ-cos αsin γ (66) b ₂₂ = sin αsin βsin γ + cos αcos γ (67) b ₂₃ = sin αcos β (68)

B ₃₁ = cos αsin βcos γ + sin αsin γ (69) b ₃₂ = cos αsin βsin γ-sin αcos γ (70) b ₃₃ = cos αcos β (71)

In this manner, instead of correcting the camera direction, the two-dimensional coordinates (x, y) on the imaging surface can be corrected by the rotation angle to obtain the correct coordinates (x ', y'). Since this can be done only by calculation, it is suitable for automatic correction.

[0088]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 illustrates a method for calibration according to the invention. A target for adjusting the optical axis of the lens is provided in front of the visual section composed of the camera group, having a mark 1 marked in accordance with the camera installation interval and a mark 2 shifted by a known distance from the mark 1. Marks are set on the target at two or more points per camera. Two points is the lowest number. Rotation angle is 3
You need at least two points. With three or more points, the accuracy can be improved. It is convenient to provide the landmark as a point, for example, an intersection of checkered targets. But there is no requirement that such patterns be present. It suffices if two or more marks can be defined. The target in which only the mark points are drawn may be used.

The camera posture is adjusted so that the optical axis position of each camera lens on the image substantially matches the position on the image of the mark 1 corresponding to each camera on the target (target). . This is a manual coarse adjustment.
The mark 1 is provided so as to form an image at the center of the imaging surface for a camera facing a normal direction. As shown in FIG. 2, an analog image obtained by a camera is digitized in an A / D converter. A feature point (a point corresponding to the contour of a camera subject) is extracted based on the level of digital data obtained by the feature point extraction unit, and written into an image memory.

The positions where the marks 1 and 2 appear are measured from the image memory. Referring to FIG. 3, a processing example in the case where a checkered intersection is used as the mark 1 will be described below. The measurement of the position where the mark 2 appears can be performed by substantially the same processing. a. A histogram is created in which the X-axis and the Y-axis are plotted on the abscissa and the number of feature points is plotted on the ordinate for feature points in a small area centered on the optical axis position of the camera lens. b. The peak value of the created histogram in each axis direction is obtained. Whether the peak value of the histogram is reliable or not is evaluated according to the following criteria. (A) Does the sum of the histogram frequencies exceed the threshold? (B) Is the distribution width d of the histogram around the peak value within the specified value? c. If the above criterion is satisfied, the coordinates of the point at which the peak value is taken are taken as the position of the mark. If not, adjust the camera posture again.

The amount of deviation on the image is calculated from the position where the mark appears and the position of the optical axis of the lens. The position (x _i ′, y _i ′) of the mark i on the screen is obtained by the peak of the histogram. Target position of the preset landmark _i (x i, y i) assumed that is known in advance. This is a two-dimensional coordinate on the camera imaging surface when the camera is facing normally. These two coordinates are obtained, and the deviation value in the camera direction is obtained from these two coordinates. Alternatively, their difference (Δx _i , Δy
_In some cases, _i ) is obtained and the calculation proceeds. That is, it is a shift amount. As shown in FIG. 3, the mark is 2 for one camera.
Take over. This is because the rotation angles α, β, and γ around the three main axes are determined. If there are two marks, the rotation angle can be obtained. 3
In the above case, the average value of the rotation angles can be obtained to improve the calculation accuracy. Here, Mark 1, Mark 2, Mark 3
Is taking. The above calculation is performed for all of these marks 1 to 3.

The coordinates of the imaging plane (x
_i , y _i ) and imaging plane coordinates (x _i ′,
y _i ′) or the amount of rotation (Δx _i , Δy _i ) of the camera posture, and the rotation amount α, β, γ around the axis with respect to the normal position is calculated. Expression (1) using rotation amounts α, β, and γ around the camera axis
D _L and d _R are corrected. The three-dimensional coordinates of the target point are more complicated equations, but since α, β, and γ are known, they can be corrected on the equations. Alternatively, since the amount of rotation is known, the camera is rotated to cancel the amount of rotation, and the direction of the camera is corrected to the normal direction.

[0093]

According to the present invention, in a monitoring apparatus in which a plurality of cameras are arranged on a straight line and the optical axes of all the cameras are set to be perpendicular to the straight lines, the rotation angles of the cameras from a normal direction are obtained. Can be. The camera position can be calibrated by actually rotating the camera by this angle. This is the simplest usage. It correctly calibrates all camera orientations.

Alternatively, without actually rotating and adjusting the camera, the rotation angle information α, β, γ can be obtained, and the observation result of the object can be corrected using the information, and the correct coordinates can be obtained by calculation. This is to calibrate by calculation while keeping the camera direction out of order.

[Brief description of the drawings]

FIG. 1 is a schematic perspective view showing an apparatus for calibrating a camera direction according to the present invention.

FIG. 2 is a block diagram of an image processing system of each camera.

FIG. 3 is a diagram illustrating a target in which a checkered pattern is observed with a group of cameras, a mark 1, a mark 2, and a mark 3 are observed, and the position of the mark is detected from a histogram of an outline.

FIG. 4 is a view for explaining that the distance measurement value is incorrect if the camera optical axis is shifted when observing the same point with two cameras having parallel optical axes.

FIG. 5 is an explanatory view of the principle of an apparatus for obtaining three-dimensional coordinates of an object using a group of parallel cameras.

──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Takehiko Kikuchi 1-3-1 Uchisaiwaicho, Chiyoda-ku, Tokyo Tokyo Electric Power Co., Inc. (72) Inventor Takeshi Ishibashi 1-1-3 East Uchisaiwaicho, Chiyoda-ku, Tokyo (56) References JP-A-4-181106 (JP, A) JP-A-5-99622 (JP, A) JP-A-5-38688 (JP, A) JP-A-62-56814 ( JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G01B 11/00-11/30 102 G01C 3/00-3/32 G01C 11/00-11/34 G06T 1/00

Claims (4)

(57) [Claims] 1. An image measuring apparatus for measuring three-dimensional coordinates of a subject by combining a plurality of cameras such that respective focal points are located on a common plane in which optical axes of lenses are parallel to each other and perpendicular to the optical axis. Intersects the optical axis of all cameras,
A target having at least two or more landmarks near the optical axis of each camera is installed in front of the apparatus, and the coordinates of the landmarks on the image obtained by each camera and the camera stored in advance are correct. An image characterized by calculating the amount of rotation of each camera from the coordinates of the landmarks in the direction, and calibrating the two-dimensional coordinates of the image of the subject captured by the camera from the calculated amount of camera rotation. Calibration method for measuring equipment. 2. An image measurement apparatus for measuring three-dimensional coordinates of an object by combining a plurality of cameras such that respective focal points are located on a common plane in which optical axes of lenses are parallel to each other and perpendicular to the optical axis. Intersects the optical axis of all cameras,
A target having at least two or more landmarks near the optical axis of each camera is installed in front of the apparatus, and the coordinates of the landmarks on the image obtained by each camera and the camera stored in advance are correct. An image characterized by calculating the amount of rotation of each camera from the coordinates of the landmarks in the direction, and calibrating the direction of the camera by rotating the camera in reverse by the amount of rotation from the calculated amount of camera rotation Calibration method for measuring equipment. 3. An image measuring apparatus for measuring three-dimensional coordinates of a subject by combining a plurality of cameras such that respective focal points are located on a common plane in which optical axes of lenses are parallel to each other and perpendicular to the optical axis. Intersects the optical axis of all cameras,
A target having a first landmark to form an image at the center of the imaging surface when each camera is oriented in the right direction, and a target having at least one or more landmarks 2 in the vicinity thereof is installed in front of the apparatus,
The rotation angle α around the X axis and the rotation angle β around the Y axis of the camera are obtained from the coordinates of the mark 1 on the image obtained by each camera, and the rotation around the Z axis is obtained from the other marks 2,. A method for calibrating an image measuring device, comprising: calculating an angle γ; and correcting two-dimensional coordinates of an image of the subject captured by the camera from the calculated camera rotation amounts α, β, and γ. 4. An image measuring apparatus for measuring three-dimensional coordinates of a subject by combining a plurality of cameras such that respective focal points are located on a common plane in which optical axes of lenses are parallel to each other and perpendicular to the optical axis. Intersects the optical axis of all cameras,
A target having a first landmark to form an image at the center of the imaging surface when each camera is oriented in the right direction, and a target having at least one or more landmarks 2 in the vicinity thereof is installed in front of the apparatus,
The rotation angle α around the X axis and the rotation angle β around the Y axis of the camera are obtained from the coordinates of the mark 1 on the image obtained by each camera, and the rotation around the Z axis is obtained from the other marks 2,. The angle γ is calculated, and the two-dimensional coordinates (x, x, y) of the image of the subject captured by the camera are calculated from the calculated camera rotation amounts α, β, γ.
y) is corrected to a correct value (x ′, y ′), and the relationship between the two is expressed by the following equation: x ′ = (b ₁₁ x + b ₁₂ y + b ₁₃ f) f (b ₃₁ x + b ₃₂ y
+ B ₃₃ f) ^-1 y '= (b ₂₁ x + b ₂₂ y + b ₂₃ f) f (b ₃₁ x + b ₃₂ y
^{_{+ B 33 f) -1 b 11}} = cos βcos γ b 12 = cos βsin γ b 13 = -sin β b 21 = sin αsin βcos γ-cos αsin γ b 22 = sin αsin βsin γ + cos αcos γ b 23 = sin αcos β _{b 31 = cos αsin βcos γ +} sin αsin γ b 32 = cos αsin βsin γ-sin αcos γ b 33 = calibrating method of an image measuring apparatus, characterized in that performed according to cos αcos β.