JP2007205929A - Instrument for measuring camera internal parameter - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an instrument for measuring a camera internal parameter capable of measuring precisely a camera internal parameter. <P>SOLUTION: A camera 208 has a lens 204 and a two-dimensional imaging element 206. This instrument for measuring the camera internal parameter has a parallel beam emission module 216 for emitting a parallel beam 212, a camera rotation module 211 for changing an incident angle of the parallel beam to the camera, and a calculation module for calculating a photoreception-face-to-photoreception-face distance in an image side main point of the lens and the two-dimensional imaging element 206, and an intersection position of a camera optical axis with the two-dimensional imaging element 206, which are the camera internal parameters, based on two out of three angle differences obtained from the three incident angles when making the parallel beam 212 get incident three times from the parallel beam emission module 216 into the camera 208 while changing the incident angle by the camera rotation module 211, and based on two out of positional differences in three points obtained from three convergence points converged on the two-dimensional imaging element 206 inside the camera 208 corresponding thereto. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  In computer vision, the distance between the image-side principal point of the lens and the light-receiving surface of the image sensor, which is an internal parameter of the camera, and the coordinates in the digital image coordinate system of the intersection of the optical axis center of the lens and the image sensor are calculated. Regarding seeking.

  A conventional camera internal parameter measurement device (for example, Non-Patent Document 1) will be described below. FIG. 1 shows an optical system for measurement of conventional camera internal parameter measurement.

  101 is a light source, 102 is a calibration pattern which is a pattern for obtaining camera parameters, 103 is a world coordinate system indicating the position of the calibration pattern, 104 is a camera lens, 105 is a camera coordinate system defined by the camera, 106 Is a two-dimensional image sensor of the camera, 107 is a digital image coordinate system defined by the two-dimensional image sensor, and 108 is a camera composed of 104 camera lenses and 106 two-dimensional image sensors.

Assume that the camera 108 can capture 102 calibration patterns at various positions with respect to the 103 world coordinates. C _i , i = 1, 2, 3,. And the position is related by i.

The method of determining each axis of the 105 camera coordinate system is as follows. The origin is the object-side principal point and the image-side principal point of the camera lens, the origin of coordinates related to the incidence of light on the lens is the object-side principal point, and the origin of coordinates related to the emission of light from the lens is the image side The main point. Z _Ci-axis coincides with the lens optical axis, the light incident on the camera to the opposite direction to the positive direction. The X _Ci axis is in a plane that includes the origin and is orthogonal to the Z _Ci axis, and is an axis parallel to the scanning direction of the two-dimensional imaging device. When the object is moved, the direction of the object moving in the same direction as the scanning direction is indicated. The positive side. The Y _Ci axis is an axis orthogonal to both the Z _Ci axis and the X _Ci axis, and determines the positive direction of the Y _Ci axis as a right-handed system.

In the digital image coordinate system 107, the origin of scanning of one frame is the origin, the axis in the scanning direction is the u _Ci axis, and the other axis is the v _Ci axis.
It is assumed that the calibration pattern 102 includes a plurality of visually characteristic points. The characteristic points are, for example, intersections of grid-like lines, vertices of checkered squares, etc. The procedure for obtaining camera internal parameters is as follows.

1) Measure the positions of a plurality of feature points of 102 calibration patterns on the 103 world coordinate system. Let these coordinates be [X _Wk , Y _Wk , Z _Wk ] ^T. Here, k is a number for distinguishing points. T represents transposition.

2) Fix 108 cameras in a certain position. The camera of this time is C _i. i is a number for distinguishing the position of the camera.

3) imaging the calibration pattern 102 in 108 camera _{C i} of.

4) The positions of a plurality of grid points of the calibration pattern on the digital image coordinate system 107 are detected. _Let these coordinates be [u _Cik , v _Cik ] ^T. Here, k is a number for distinguishing points, and a number is assigned to each of the feature points described above. Numbering by this association is called labeling.

5) _Find the projection matrix P _Ci for the camera C _i . How to find is as follows.
5-1) For the camera C _i , the following matrix B _Ci is obtained by using [X _Wk , Y _Wk , Z _Wk ] ^T and [u _Cik , v _Cik ] ^T.

5-2) determining the eigenvectors for the minimum solid value of _B ^Ci T _{B Ci} matrix using the _{B Ci.} If this is [p _Ci11 , p _Ci12 , p _Ci13 , p _Ci14 , p _Ci21 , p _Ci22 , p _Ci23 , p _Ci24 , p _Ci31 , p _Ci32 , p _Ci33 , p _Ci34 ] ^T , the following expression is Become.

6) The epipole e _Ci is obtained as follows using the projection matrix P _Ci obtained above.

ただし、ｓ_Ｃｉはスカラであり、Ｗ_Ｃｉは三次元の任意のベクトルである。ｓ_Ｃｉの大きさについては、〜付きｅ_Ｃｉの３つめのエレメントは１であるので、ｓ_Ｃｉは、そうなるように決める。Ｉは４×４の単位行列である。

However, s _Ci is a scalar, and W _Ci is a three-dimensional arbitrary vector. Regarding the size of s _Ci , since the third element of e _Ci with ~ is 1, s _Ci is determined to be so. I is a 4 × 4 unit matrix.

7) In addition to the camera C _{i at} a certain position, the above 2) to 6) are performed for the camera C _{j at} another position.

8) A Fundamental matrix is obtained as follows using the projection matrix P _Ci obtained above.

  9) When the camera internal matrix including the camera internal parameters is represented as A, it is assumed that a symmetric matrix K is created by A as follows.

ただし、ｆ_Ｃｉはカメラのレンズの像側主点と二次元撮像素子の受光面間の距離であり、［ｕ_ＣＯ，ｖ_ＣＯ］^Ｔはカメラの二次元撮像素子が規定しているディジタル画像座標系におけるカメラのレンズの光軸と二次元撮像素子との交点の座標であり、ｋ_Ｃｉはカメラの二次元撮像素子のピクセルピッチの逆数である。

Where f _Ci is the distance between the image side principal point of the camera lens and the light receiving surface of the two-dimensional image sensor, and [u _CO , v _CO ] ^T is the digital image coordinates defined by the camera two-dimensional image sensor. It is the coordinate of the intersection of the optical axis of the camera lens in the system and the two-dimensional image sensor, and k _Ci is the reciprocal of the pixel pitch of the two-dimensional image sensor of the camera.

It is known from the Self-Calibration theory that the following two expressions hold simultaneously when K _Ci in a camera C _{i at a} certain place and K _Cj in a camera C _j whose place is changed are used.

これらはτについての二次方程式になる。これらのτについての二次関数は等価であるので、それらの二次方程式の０次、１次、２次の係数をそれぞれｋ_Ｃｉ０、ｋ_Ｃｉ１、ｋ_Ｃｉ２、ｋ_Ｃｊ０、ｋ_Ｃｊ１、ｋ_Ｃｊ２で表すと、以下のようなKruppa方程式が得られる。

These are quadratic equations for τ. Since the secondary functions of these τ are equivalent, 0-order their quadratic equation, primary, secondary coefficient in each _{k Ci0, k Ci1, k Ci2} , k Cj0, k Cj1, k Cj2 When expressed, the following Kruppa equation is obtained.

ただし、Ｆ_ＣｉＣｊはFundamental行列であり、その要素を上記のようにＦ_{ＣｉＣｊｋｌ}という表記に置き換えている。また、カメラＣ_ｉとＣ_ｊは、カメラの設置位置だけが異なり、カメラ内部パラメタは同じであるためＫ_Ｃｉ＝Ｋ_Ｃｊとなるので、以降はＫ_ＣｊをＫ_Ｃｉに置き換えて述べる。

However, F _CiCj is a Fundamental matrix, and its elements are replaced with the notation F _CiCjkl as described above. The camera _{C i} and _{C j} differ by the installation position of the camera, because the camera internal parameter becomes _K Ci _{= K Cj} are the same, thereafter describes replacing _{K Cj} to _{K Ci.}

Note that, from this equation, three simultaneous quadratic equations for K _{Cikl stand} as follows, but any set may be used.

10) Repeat steps 2) to 9) while changing the position of the camera so that five or more of the above equations 1-8 can be achieved. For this purpose, for example, the above-mentioned 2) to 6) are performed for the camera C _{k at} a position different from the cameras C _i and C _j, and 8) to 9) are performed with the cameras C _j and C _k to obtain two equations. , And 8) to 9) are performed with the cameras C _k and C _i to derive two more equations, and a quadratic equation for a total of six K _{Cikl is created} .

11) from the secondary equation for _{K Cikl} made in the above _10), to derive the _{K Cikl.} For this purpose, for example, five _out of the quadratic equations for six K _Cikl are selected and the quadratic equation is solved. Alternatively, an optimization method may be used in which the left side of the quadratic equation for six K _{Cikl is} squared, and a function obtained by summing all six is used as an evaluation function, and K _Cikl is changed so that the evaluation function approaches zero. Use to get K _Cikl . As this optimization method, for example, the Levenberg-Marquardt method is used.

12) Camera internal parameters are calculated from the K _Ci matrix obtained in 11) above. For this purpose, for example, the inverse matrix of the K _Ci matrix is first subjected to Choleski decomposition as follows.

ここで、カメラ内部パラメタと上記の式とは、下記の関係がある。

Here, the camera internal parameters and the above formula have the following relationship.

ただし、ｆ_Ｃｉはカメラのレンズの像側主点と二次元撮像素子の受光面間の距離であり、［ｕ_ＣＯ，ｖ_ＣＯ］^Ｔはカメラの二次元撮像素子が規定しているディジタル画像座標系におけるカメラのレンズの光軸と二次元撮像素子との交点の座標であり、ｋ_Ｃｉはカメラの二次元撮像素子のピクセルピッチの逆数である。

However, f _Ci is the distance between the light receiving surface of the image side principal point and the two-dimensional imaging device of the camera _{lens, [u CO, v CO]} T digital image coordinate two-dimensional image sensor of the camera is defined It is the coordinate of the intersection of the optical axis of the camera lens in the system and the two-dimensional image sensor, and k _Ci is the reciprocal of the pixel pitch of the two-dimensional image sensor of the camera.

  Therefore, the camera internal parameters are derived, for example, by performing the following calculation.

あるいは、式１−１０の各エレメントについて、左辺と右辺の二乗和を取ったものを評価関数とし、その評価関数が０に近づくようにｕ_ＣｉＯ、ｖ_ＣｉＯ、ｆ_Ｃｉを変化させる最適化手法を使用してｕ_ＣｉＯ、ｖ_ＣｉＯ、ｆ_Ｃｉを得る。この最適化手法としては、たとえば、Levenberg-Marquardt法を使用する。

Alternatively, an optimization method for changing u _CiO , v _CiO , and f _Ci so that each element of Expression 1-10 is obtained by taking the sum of squares of the left and right sides as an evaluation function and the evaluation function approaches 0. Use to obtain u _CiO , v _CiO , f _Ci . As this optimization method, for example, the Levenberg-Marquardt method is used.

Q. T. Luong, O. D. Faugeras, “Self Calibration of a Moving Camera from Point Correspondences and Fundamental Matrices”, International Journal of Computer Vision, 22 (3), pp.261-289, 1997

However, in the above conventional method, even if the incident angle of the light from the feature point of the calibration pattern to the object side principal point of the camera lens is the same, the feature point of the calibration pattern and the object side principal point of the camera lens This causes a phenomenon that the position of the light reaching the two-dimensional image sensor is shifted depending on the distance, and the measurement accuracy of the internal parameters of the camera deteriorates due to the phenomenon. In addition, since the conventional camera parameter measurement method requires strict alignment for measurement, an accurate alignment mechanism is required, and the operation of the mechanism measurement for adjusting these is large.
The present invention has been made in view of the above, and an object of the present invention is to provide a camera internal parameter measurement device that accurately measures camera internal parameters.

  Of the inventions disclosed in this application, the outline of typical ones will be briefly described as follows.

  A first invention is a camera internal parameter measurement device for a camera having a lens and a two-dimensional imaging device, a parallel beam emitting module for emitting a parallel beam, and a relative position changing module for changing the incident angle of the parallel beam to the camera And two of three angle differences obtained from three incident angles when a parallel beam is incident on the camera from the parallel beam emitting module by changing the incident angle by the relative position changing module three times, and The image side principal of the lens, which is a camera internal parameter, with two of the differences in the positions of the three points obtained from the three focal points collected on the two-dimensional image sensor in the corresponding camera And a calculation module for calculating the distance between the point and the light receiving surface of the two-dimensional image sensor and the intersection position of the camera optical axis and the two-dimensional image sensor.

  According to a second aspect, in the first aspect, the relative position changing module is a camera rotation module that rotates the camera.

  According to a third invention, in the first or second invention, a third aspect of the invention includes a threshold value storage module that stores a threshold value of dispersion of camera internal parameter values. The calculation module includes three angular differences obtained from three incident angles. The camera internal parameters are calculated for the three methods for selecting two of the differences between the positions of the three points obtained from two of the three and the corresponding three focal points, and the calculated values of the camera internal parameters are calculated. Is larger than the threshold value stored in the threshold value storage module, the incident angle and the focal point when a parallel beam is incident on the camera from the parallel beam extraction module at a different incident angle than before. And using the incident angle and condensing point obtained so far, select three incident angles and condensing point position, and again as described above, to calculate the camera internal parameters, If the variance of the calculated camera internal parameter value is compared and the calculated variance of the camera internal parameter is smaller than the threshold stored in the threshold storage module, the calculated camera internal parameter value is used. The parameter is fixed.

  Hereinafter, in order to help understanding of the present invention, details of calculation performed in the calculation module of the camera internal parameter measurement device will be specifically described, but the present invention is not limited thereto. When calculating camera internal parameters, for example, the following calculation is performed.

The coordinates defined by the camera are called camera coordinates. In this camera coordinate, the origin is the object-side principal point and the image-side principal point of the camera lens, the origin of the coordinates relating to the incidence of light on the lens is the object-side principal point, and the coordinates relating to the emission of light from the lens Is the image side principal point. The Z _C axis, X _C axis, and Y _C axis of the camera coordinates are determined as follows. Z _C axis coincides with the lens optical axis, the light incident on the camera to the opposite direction to the positive direction. X _C-axis is in the plane orthogonal to the Z _C axis comprises the origin, a axis parallel to the scanning direction of the two-dimensional imaging device of the camera, of an object moving in the same direction as the scanning direction when moving the object The direction is the plus side. Y _C-axis is an axis orthogonal to both the Z _C axis and X _C axis, and to determine the positive direction of the Y _C axis as right-handed.

In the digital image coordinate system defined by the two-dimensional image sensor of the camera, the origin of scanning of one frame is the origin, the scanning direction axis is the u _C axis, and the other axis is the v _C axis.

The coordinates of the intersection of the optical axis of the camera lens and the two-dimensional image sensor in the digital image coordinate system is [u _CO , v _CO ] ^T, and the image side principal point of the camera lens and the light reception of the two-dimensional image sensor of the camera. The distance to the surface is f _C and the pixel pitch of the two-dimensional image sensor of the camera is 1 / k _C.

The coordinate in the digital image coordinate system of the point where the parallel beam emitted from the parallel beam extraction module is incident on the lens of the camera and is focused on the two-dimensional image sensor of the camera at that time is [u _C , v _C ]. ^T , and the coordinates in the camera coordinate system are [X _C , Y _C , Z _C ] ^T , the [X _C , Y _C , Z _C ] ^T is

の関係にある。

Are in a relationship.

  The directions when the direction of the camera is changed three times by the camera rotation module will be referred to as direction 0, direction 1, and direction 2, respectively.

The angle difference obtained by subtracting the direction 0 from the direction 1 is [Δθ _ZXCD10 , Δθ _ZYCD10 ] ^T , and the angle difference obtained by subtracting the direction 0 from the direction 2 is [Δθ _ZXCD20 , Δθ _ZYCD20 ] ^T , where [Y _C axis of the camera coordinate system angle to the rotational axis, and be referred to as angle to rotation axis X _C axis.

In addition, when the coordinates of the points focused on the two-dimensional image sensor of the camera in each direction are expressed in the digital image coordinate system, [u _CD0 , v _CD0 ] ^{T in} direction 0 and [u _CD0 in direction 1 _{respectively.} + Δu _CD10 , v _CD0 + Δv _CD10 ] ^T and [u _CD0 + Δu _CD20 , v _CD0 + Δv _CD20 ] ^T in direction 2.

If the camera lens is an fSinθ system, the distance f _C between the camera lens and the two-dimensional imaging device, the optical axis of the camera lens in the digital image coordinate system defined by the two-dimensional imaging device of the camera, and the two-dimensional imaging The coordinates [u _CO , v _CO ] ^T of the intersection with the element are

となる。

It becomes.

If the camera lens is an fTanθ system, the distance f _C between the camera lens and the two-dimensional image sensor, the optical axis of the camera lens in the digital image coordinate system defined by the camera two-dimensional image sensor, The coordinates [u _CO , v _CO ] ^T of the intersection with the two-dimensional image sensor are

となる。

It becomes.

  In the first and second aspects of the invention, since the measurement is performed using the light in which the parallel beam is incident on the camera, the object side main part of the camera lens of the light from the feature point of the calibration pattern as in the prior art. Even if the incident angle to the point is the same, there is no inconvenience that the position of the light reaching the two-dimensional imaging device is shifted depending on the distance between the feature point of the calibration pattern and the object side principal point of the camera lens. The position of the feature point can be extracted with high accuracy, and therefore, there is an advantage that the camera internal parameters can be measured with high accuracy.

  In the first and second aspects of the present invention, the angle used for the calculation is not an absolute angle of the camera with respect to the parallel beam, but is a difference value of angle data obtained from a plurality of points. There is an advantage that the angle relationship between the camera and the camera does not need to be strictly matched and can be easily measured.

  In the first and second inventions, since the parallel beam is incident on the camera, the distance relationship between the parallel beam emitting module and the camera does not need to be strictly aligned, and there is an advantage that it can be easily measured. .

  In the third aspect of the invention, an error that occurs when measuring the rotation angle of the camera, an error that occurs when measuring the position of the condensing point that is focused on the image sensor, or a camera internal parameter is calculated. If an error such as a calculation rounding error occurs in the camera, if the variance of the camera internal parameters obtained from the three combinations where the variance is originally 0 increases, there is a problem in the calculation accuracy of the camera internal parameters. Since it can be seen that there is an advantage that an error of the obtained camera internal parameter can be detected.

  Further, in the third aspect of the present invention, when an error is detected in the obtained camera internal parameter, the incident light is increased to increase the combination of the incident light angle difference and the corresponding condensing point position difference. Since the combination with the smallest variance of the calculated camera internal parameters can be selected, there is an advantage that the calculation accuracy of the camera internal parameters is improved.

  In the present invention, since the parallel beam is measured using the light incident on the camera, the incident angle of the light from the feature point of the calibration pattern to the object side principal point of the camera lens as in the prior art. Even if they are the same, there is no problem that the position of the light reaching the two-dimensional image sensor is shifted depending on the distance between the calibration pattern feature point and the object side principal point of the camera lens, and the feature point is accurate. Therefore, there is an advantage that the camera internal parameters can be measured with high accuracy.

  In addition, the angle used for the calculation is not the absolute angle of the camera with respect to the parallel beam, but the difference value of the angle data obtained from a plurality of points, so the angle relationship between the parallel beam extraction module and the camera must be strictly matched. There is an advantage that it can be measured easily.

  Further, since the parallel beam is incident on the camera, there is an advantage that the distance relationship between the parallel beam extraction module and the camera does not need to be strictly aligned and can be easily measured.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 2 is a configuration diagram of the camera internal parameter measurement device according to the first embodiment of the present invention. 204 is a camera lens; 205 is a camera coordinate system defined by the camera; 206 is a two-dimensional image sensor of the camera; 207 is a digital image coordinate system defined by the two-dimensional image sensor; It is a camera composed of a two-dimensional image sensor. Reference numeral 209 denotes a rotary stage that changes the direction of the camera, and a _θYC rotary stage that rotates about the Y axis of the camera coordinate system 208. A rotation stage 210 changes the orientation of the camera, and is a θ _XC rotation stage that rotates about the X axis of the camera coordinate system 208. _{Reference numeral} 211 denotes a camera rotation module composed of a 209 θ _YC rotation stage and a 210 θ _XC rotation stage. 212 is a parallel beam incident on the camera, 213 is a laser serving as a light source, 214 is a spatial filter for removing the spatial frequency noise of the laser 213, and approaches a plane wave, and 215 is for spreading the laser light emitted from 214. The beam expander 216 is a parallel beam extraction module composed of a laser 213, a spatial filter 214, and a beam expander 215. In addition, although not shown in FIG. 2, the camera internal parameter measurement device according to the first embodiment includes a calculation module including a computer and a program stored in a storage device. This calculation module includes three angle differences obtained from three incident angles when a parallel beam is incident from the 216 parallel beam emitting module to the 208 camera by changing the incident angle by the 211 camera rotating module three times. And two of the three point position differences obtained from the three condensing points collected on the 206 two-dimensional image sensors in the 208 cameras corresponding thereto, A distance between the image-side principal point of the lens and the light receiving surface of the two-dimensional image sensor, and an intersection position between the camera optical axis and the two-dimensional image sensor are calculated.

The 208 camera can be fixed at various angles with respect to the traveling direction of the 212 parallel beam by the 209 θ _YC rotation stage and the 210 θ _XC rotation stage. It shall be able to enter the lens at various angles. For each angle camera, C _Di , i = 0, 1, 2,. . . And the angle is related by i.

How to determine each axis of the 205 camera coordinate system is as follows. The origin is the object-side principal point and the image-side principal point of the camera lens, the origin of coordinates related to the incidence of light on the lens is the object-side principal point, and the origin of coordinates related to the emission of light from the lens is the image side The main point. The Z _CDi axis coincides with the lens optical axis, and the direction opposite to the incidence of light on the camera is the plus direction. The X _CDi axis is in a plane that includes the origin and is orthogonal to the Z _CDi axis, and is an axis parallel to the scanning direction of the two-dimensional imaging device. The direction of the object that moves in the same direction as the scanning direction when the object is moved The positive side. Y _CDi axis _is an axis orthogonal to both the _{Z CDi} axis and _{X CDi} axis, and to determine the positive direction of the _{Y CDi} axis as right-handed.

In the digital image coordinate system 207, the origin of scanning of one frame is the origin, the axis in the scanning direction is the u _CDi axis, and the other axis is the v _CDi axis.

The relationship between the two coordinate systems is as follows. If the coordinates of a certain point on the digital image coordinate system is [u _CDi , v _CDi ] ^T , the point is represented by coordinates [X _CDi , Y _CDi , Z _CDi ] ^T in the camera coordinate system as follows: Become.

ただし、［ｕ_ＣＯ，ｖ_ＣＯ］^Ｔはカメラのレンズの光軸と二次元撮像素子との交点のディジタル画像座標系での座標であり、ｆ_Ｃはカメラのレンズの像側主点とカメラの二次元撮像素子の受光面までの距離であり、１／ｋ_Ｃはカメラの二次元撮像素子のピクセルピッチである。

[U _CO , v _CO ] ^T is the coordinate in the digital image coordinate system of the intersection of the optical axis of the camera lens and the two-dimensional image sensor, and f _C is the image side principal point of the camera lens and the camera This is the distance to the light receiving surface of the two-dimensional image sensor, and 1 / k _C is the pixel pitch of the two-dimensional image sensor of the camera.

Parallel beam of 212, even if the position of the to 204 of lens rotation has changed by 208 Camera rotation module C _i is 211, as 204 of the lens does not protrude from the scope propagating parallel beams, Keep the beam width wide.

Hereinafter, the coordinates [u _CO , v _CO ] ^{T in} the digital image coordinate system of the intersection between the optical axis of the camera lens and the two-dimensional image sensor, which are internal parameters of the camera, the image side principal point of the camera lens, and the camera A procedure for obtaining the distance f _C to the light receiving surface of the two-dimensional image sensor will be described.

1) The 208 camera is set at a certain angle by the camera rotation module. In this case the item number i = 0 to be given to the camera, call this camera and C _D0.

2) As an angle of the camera at this time, [an angle with the _YCD0 axis as the rotation axis and an angle with the _XCD0 axis as the rotation axis] of the camera coordinate system is measured, and this is obtained as [ _θZXCD0 ′, _θZYCD0 ′]. ^T.

For example, when the reading of the scale of the 209 θ _YC rotary stage is θ _YCD0 ′ and the reading of the scale of the θ _XC rotary stage of 210 is θ _XCD0 ′, θ _ZXCD0 ′ and θ _ZYCD0 ′ are as follows: become that way.

ここで、角度にダッシュ「’」がついているのは、２０５のカメラ座標系と２０９のθ_ＹＣ回転ステージと２１０のθ_ＸＣ回転ステージが、２１２の平行ビームの進行方向に対して、その位置関係を何ら保証していないためである。

Here, it marked with a dash "'" The angle, theta _XC rotating stage of the camera coordinate system and 209 of the theta _YC rotating stage and 210 205, the traveling direction of the parallel beam 212, the positional relationship This is because there is no guarantee.

This will be described with reference to FIG. Reference numeral 204 denotes a camera lens, and reference numeral 212 denotes incident light incident on the lens 204, that is, a parallel beam shown in FIG. Reference numerals 301, 302, and 303 denote an X _CDi axis, a Y _CDi axis, and a Z _CDi axis of the camera coordinate system, respectively. 311 is a _X CDi _{-Z CDi} plane is a plane containing the _{Z CDi} axis _{X CDi} axis 303 of the 301, 312 is a plane including the _{Z CDi} axis _{Y CDi} axis 303 of the 302 _Y CDi _{-Z CDi} It is a plane. _{Reference numeral} 321 denotes a line obtained by projecting 212 incident light onto the X _CDi -Z _CDi plane of 311. _{Reference numeral} 322 denotes a line obtained by projecting 212 incident light onto the Y _CDi -Z _CDi plane of 312. _{Reference numeral} 331 denotes an angle θ _ZXCDi from the Z _CDi axis of 303 to the line 321, and 332 denotes an angle θ _ZYCDi from the Z _CDi axis of 303 to the line 322. When the camera item number i = 0, 331 is the angle θ _ZXCD0 and 332 is the angle θ _ZYCD0 . That is, this figure shows the incident light on the lens of the camera, the camera coordinate system [X _CDi , Y _CDi , Z _CDi ] ^T, and Z _CDi for the line in which the incident light is projected onto the X _CDi -Z _CDi plane of the camera coordinate system. the angle _{θ ZXCD0,} shows the relationship of the angle theta _ZYCD0 from _{Z CDi} incident light for the line was projected to _Y CDi _{-Z CDi} of the camera coordinate system. When the Z _CDi axis coincides with the direction opposite to the direction of the incident light, if the 205 camera and the 216 parallel beam extraction module are adjusted so that θ _ZXCD0 ′ and θ _ZYCD0 ′ become 0, θ _ZXCD0 ′ And θ _ZYCD0 ′ coincide with θ _ZXCD0 and θ _ZYCD0 , but since such adjustment is not performed here, θ _ZXCD0 ′ and θ _ZYCD0 ′ generally do not coincide with θ _ZXCD0 and θ _ZYCD0 . Therefore, in order to distinguish the current measurement from θ _ZXCD0 and θ _ZYCD0 , a dash “′” is _added to the angle.

In the above, the scale of the 209 θ _YC rotary stage and the scale of the 210 _{X XC} rotary stage were read and θ _ZXCD0 ′ and θ _ZYCD0 ′ were measured by calculation. flat portion of, or attached to a flat plate in a camera, it may be measured directly θ _{ZXCD0 'and} θ _ZYCD0' using goniometer.

  3) In this state, 212 parallel beams emitted from the 216 parallel beam emission module are made incident on the lens of the 204 camera.

4) Measure the coordinates on the digital image coordinate system of the point focused on the two-dimensional image sensor 206 at this time and set it as [u _CD0 , v _CD0 ] ^T.

5) The 208 camera is set to another angle by the 211 camera rotation module, and the above 2) to 4) are performed twice. The item number to be given to the camera, respectively the i = 1, 2, to obtain _{[θ ZXCDi ', θ ZYCDi'} ] T, i = 1,2 and ^{_{[u CDi, v CDi] T}} , the i = 1, 2.

  6) From the result of the above 5), the following angle difference and the difference in the position of the focal point are obtained.

7) The distance f _C between the image side principal point of the camera lens and the light receiving surface of the two-dimensional image sensor, the optical axis of the camera lens in the digital image coordinate system defined by the camera two-dimensional image sensor, and the two-dimensional image The coordinates [u _CO , v _CO ] ^T of the intersection with the image sensor are obtained according to the following equation.
When the camera lens is an fSinθ system, it is obtained as follows.

  When the camera lens is an fTanθ system, it is obtained as follows.

In the above procedure, the angle difference and the condensing point position difference in the combination i = 1, 2 with respect to the camera item number i = 0 are used, but the combination i = 2, 0 with respect to i = 1, i = In consideration of the angle difference and the condensing point position difference in the combination of 0 and 1 with respect to 2, the angle difference and the condensing point position difference in each combination are used to formulas 2-3, 2-4, and 2-5. a method of increasing the respective f _C and u _CO and v differencing accuracy by a reduced of _CO obtained from also described below.

As this method, f _C in each combination is set so that the linear sum of dispersion of u _CO and v _CO is minimized. This can be achieved by using an optimization method using a linear sum of variances of f _C , u _CO and v _CO as an evaluation function. As an optimization method, for example, there is a Levenberg-Marquardt method. This is the variable of the evaluation function _{[θ ZXCDi ', θ ZYCDi'} ] T, i = 0,1,2 and ^{_{[u CDi, v CDi] T}} , and i = 0,1,2, the evaluation function in the variable This is performed by using a differential coefficient obtained by partial differentiation.

This procedure is shown below. Coordinates [u _CO , v _CO ] ^{T in} the digital image coordinate system of the intersection of the optical axis of the camera lens and the two-dimensional image sensor, which are camera internal parameters, the image side principal point of the camera lens, and the camera The procedure for obtaining the distance f _C to the light receiving surface of the two-dimensional image sensor is the same as the above procedures 1) to 5). The subsequent procedure is described below. However, “′” is attached to the number indicating the procedure to distinguish it from the first embodiment.

  6 ') The following three combinations are obtained as the difference between the angle difference and the focal point position.

7 ′) For each, calculate the equation of step 7), and the coordinates of the intersection of the optical axis of the camera lens and the two-dimensional image element in the digital image coordinate system, the image side principal point of the camera lens, and the camera The distance to the light receiving surface of the two-dimensional image element is obtained. At this time, the values obtained by the expressions 2-6, 7, and 8 are [u _COj , v _COj ] ^T , f _Cj , j = 0, 1, and 2, respectively.

8 ′) Variances σ _uCO , σ _vCO , and σ _fC of u _COj , v _COj , and f _Cj are obtained, respectively, and the evaluation function J is obtained as a linear sum of variances as follows.

ここで、Ｃ_ｕ，Ｃ_ｖ，Ｃ_ｆは実数であり重みを表す。σ_ｕＣＯ、σ_ｖＣＯ、σ_ｆＣはｊによって変動するｕ_ＣＯｊ、ｖ_ＣＯｊ、ｆ_Ｃｊの分散であり、たとえば、σ_ｕＣＯについては、σ_ｕＣＯ＝（１／３）＊Σ（ｕ_ＣＯｊ）＾２−（（１／３）＊Σｕ_ＣＯｊ）＾２によって求められる。ここで、Σはｊ＝０〜２の和であり、「＾２」は２乗である。σ_ｖＣＯ、σ_ｆＣについても同様である。

Here, C _u , C _v , and C _f are real numbers and represent weights. σ _uCO , σ _vCO , and σ _fC are variances of u _COj , v _COj , and f _Cj that vary depending on j. For example, for σ _uCO , σ _uCO = (1/3) * Σ (u _COj ) ^ 2- ((1/3) * Σu _COj ) ^ 2. Here, Σ is the sum of j = 0 to 2, and “^ 2” is the square. The same _{applies to} σ _vCO and σ _fC .

  A vector whose elements are variables is expressed as follows.

このとき勾配∇Ｊとヘッセ行列Ｈを以下のように求める。

At this time, the gradient ∇J and the Hessian matrix H are obtained as follows.

ξの更新ルールは以下のようにする。

The update rule of ξ is as follows.

ただし、ξ^（Ｋ）はＫ回の反復後のξであり、Ｈ^（Ｋ）と∇Ｊ^（Ｋ）はξ^（Ｋ）を代入して得られたヘッセ行列と勾配である。ｃはξ^（Ｋ）を使って求めたＪ^{（Ｋ＋１）}の値が以前に求めたＪの最小値よりも小さいかどうかで変更する実数である。Ｄ［Ｈ^（Ｋ）］は、たとえば、Ｈ^（Ｋ）の対角要素のみ取り出した対角行列としてもよいし、単位行列に置き換えても良い。

Here, ξ ^(K) is ξ after K iterations, and H ^(K) and ∇J ^(K) are the Hessian and gradient obtained by substituting ξ ^(K) . c is a real number that changes depending on whether the value of J ^{(K + 1)} obtained using ξ ^(K) is smaller than the minimum value of J obtained previously. D [H ^(K)], for example, may be used as the diagonal matrix obtained by extracting only diagonal elements of H ^(K), it may be replaced with the identity matrix.

Using Equation 2-13, for example, the following procedure is used to converge ξ.
8′-1) Set c ← 0.0001. However, the value is not limited to 0.0001, and a small value may be determined as appropriate.
8′-2) An initial value of ξ is given. This may be, for example, the measured value itself.
8′-3) Formula 2-9 is calculated using the ξ, and J is obtained.
8′-4) The gradient ∇J and the Hessian matrix H are obtained by Equations 2-11 and 2-12.
8′-5) Solve the following simultaneous linear equations to obtain Δξ.

８’−６）次のようにξ’を計算する。

8′-6) ξ ′ is calculated as follows.

８’−７）そのξ’を用いて式２−９を計算した値を、Ｊ’とする。
８’−８）Ｊ’＞Ｊであれば、ｃ←１０ｃと更新して８’−５）へ戻る。
８’−９）そうでなければ次のように更新する。

8′-7) A value obtained by calculating Expression 2-9 using the ξ ′ is J ′.
8′-8) If J ′> J, update c ← 10c and return to 8′-5).
8′-9) Otherwise, update as follows.

８’−１０）ある小さな値δに対して、｜Δξ｜＜δであれば、ξが収束したと見なし、終了する。そうでなければ、８’−４）に戻る。

8′−10) If | Δξ | <δ with respect to a certain small value δ, it is considered that ξ has converged, and the process ends. Otherwise, return to 8'-4).

[U _CO , v _CO ] ^T , f _C is obtained by substituting ξ obtained above into Equation 2-3 and further calculating Equation 2-4.

  The above calculation is performed by a calculation module (not shown). The calculation module can be configured by a program stored in a computer and a storage device. Further, hardware may be used in place of part or all of the program.

  As an effect of the present embodiment, since a parallel beam is measured using light incident on the camera, the light from the calibration pattern feature point to the object side principal point of the camera lens as in the conventional technique. Even if the incident angles are the same, there is no problem that the position of the light reaching the two-dimensional image sensor is shifted depending on the distance between the feature point of the calibration pattern and the object side principal point of the camera lens. The position of the feature point can be extracted well, so that there is an advantage that the camera internal parameter can be measured with high accuracy.

  In addition, the angle used for the calculation is not the absolute angle of the camera with respect to the parallel beam, but the difference value of the angle data obtained from a plurality of points, so the angle relationship between the parallel beam extraction module and the camera must be strictly matched. There is an advantage that it can be measured easily.

  Further, since the parallel beam is incident on the camera, there is an advantage that the distance relationship between the parallel beam extraction module and the camera does not need to be strictly aligned and can be easily measured.

  In addition, since the parallel beam is obtained through a spatial filter using coherent light called laser light as a light source, the wave front of the parallel beam is close to a plane wave, and the condensing performance of the camera lens is very good compared to natural light, Since an image with little blur is obtained on the two-dimensional image sensor of the camera, there is an advantage that the coordinates of the condensing point can be obtained with high accuracy.

Example 2 will be described below.
The configuration diagram is the same as that of the first embodiment (FIG. 2). However, although not shown in FIG. 2, the camera internal parameter measurement device according to the second embodiment includes a calculation module including a computer and a program stored in a storage device, and a threshold storage module including a storage device. This calculation module uses two of the three angle differences obtained from three incident angles and two of the three point position differences obtained from the corresponding three condensing points. The camera internal parameter is calculated, and when the variance of the calculated camera internal parameter value is larger than the threshold value stored in the threshold value storage module, the parallel beam extraction module at the incident angle is different from the above. Using the incident angle and condensing point when the parallel beam is incident on the camera and the incident angle and condensing point obtained so far, three incident angles and condensing point positions are selected, again as described above, The camera internal parameter is calculated, the variance of the calculated camera internal parameter value is compared, and the calculated variance of the camera internal parameter is stored in the threshold storage module. If less than the threshold value, it determines the camera internal parameter using the value of the camera internal parameters are calculated.

Coordinates [u _CO , v _CO ] ^{T in} the digital image coordinate system of the intersection of the optical axis of the camera lens and the two-dimensional image sensor, which are camera internal parameters, the image side principal point of the camera lens, and the camera The procedure for obtaining the distance f _C to the light receiving surface of the two-dimensional imaging device is the same as the procedures 1) to 5) of the first embodiment. The subsequent procedure is described below. However, “” ”is added to the number indicating the procedure to distinguish it from the first embodiment.

  6 ″) The following three combinations are obtained as the difference between the angle difference and the focal point position.

7 ″) for each, calculate the expression of the procedure 7) of the first embodiment, coordinates of the intersection of the optical axis of the camera lens and the two-dimensional image sensor in the digital image coordinate system, and the image side main of the camera lens. The distance from the point to the light receiving surface of the two-dimensional image sensor of the camera is obtained by _calculating [u _COj , v _COj ] ^T , f _Cj , j = 0, 1 , 2

8 ″) The variances σ _uCO , σ _vCO , and σ _fC of u _COj , v _COj , and f _Cj are obtained. The method for obtaining the variance is as described in 8 ′). The threshold value T set for these variances _σuCO, T _σvCO, by comparing each of the variance and _T σfC, if the threshold is not exceeded for it any of the dispersion, _{[u ^COj,} v _COj] for j = 0, 1, 2 _T, which of _{f Cj} Alternatively, [u _COj , v _COj ] ^T , when the linear sum of dispersion shown in Equation 2-9 is minimized using the Levenberg-Marquardt method, which is the optimization method described in Example 1. f _Cj is adopted.

The variances σ _uCO , σ _vCO , and σ _fC are compared with the threshold _values T _σuCO , T _σvCO , and T _σfC , respectively, and if any of them exceeds [u _COj , v _COj ] All of ^T and f _Cj are not _adopted . Then, the incident angle and the position of the condensing point are different from any of the above [[theta] _ZXCDi ', [theta] _ZYCDi '] ^T , i = 0, 1, 2 and [ _uCDj , _vCDj ] ^T , i = 0, 1, 2. as a newly incident parallel light to the camera, it is a _{[θ ZXCDi ', θ ZYCDi'} ] T, i = 3 and ^{_{[u CDi, v CDi] T}} , i = 3.

  9 ″) For all the combinations in which i = 0, 1, 2, 3 are selected from three, the following angular difference and focal point difference are calculated, and for each of the above 7 ″), 8 ″) The same calculation is performed.

各３つを選ぶ組み合わせごとにｕ_ＣＯｊ、ｖ_ＣＯｊ、ｆ_Ｃｊのｊを変えた場合の変動を示す分散σ_ｕＣＯ、σ_ｖＣＯ、σ_ｆＣをそれぞれ求める。分散を求める方法は８’）で述べたとおりである。各組み合わせにおいて、分散σ_ｕＣＯ、σ_ｖＣＯ、σ_ｆＣに対してそれぞれ設定した閾値Ｔ_σｕＣＯ、Ｔ_σｖＣＯ、Ｔ_σｉＣとこれらの分散を比較し、いずれの分散もそれに対する閾値を超えない場合は、その組み合わせを［ｕ_ＣＯｊ，ｖ_ＣＯｊ］^Ｔ，ｆ_Ｃｊを採用するための候補とする。

Variances σ _uCO , σ _vCO , and σ _fC indicating fluctuations when j of u _COj , v _COj , and f _Cj is changed are _obtained for each combination of selecting three. The method for obtaining the variance is as described in 8 ′). In each combination, the threshold _values T _σuCO , T _σvCO , and T _σiC set for the variances σ _uCO , σ _vCO , and σ _fC are compared with these variances. If any variance does not exceed the threshold value, _Let the combination be a candidate for employing [u _COj , v _COj ] ^T , f _Cj .

  Select one of the candidate combinations. Alternatively, the combination having the smallest variance is selected from the combinations that are candidates.

One of the three sets [u _COj , v _COj ] ^T , f _Cj is adopted. Alternatively, [u _COj , v _COj ] ^T , f _Cj obtained by minimizing the linear sum of dispersion shown in Equation 2-9 using the Levenberg-Marquardt method, which is the optimization method described in the first embodiment, is employed. .

However, if a combination candidate for adopting [u _COj , v _COj ] ^T , f _Cj is not obtained, none of the combinations is a candidate, and the incident angle and the focal point position used so far are not used. Unlike the above, a new parallel light is incident on the camera, and the incident angle and the focal point position at that time are measured. 9 ″) _until the dispersions σ _uCO , σ _vCO , and σ _fC obtained for each of the combinations for selecting each of the three combinations do not exceed the threshold _values T _σuCO , T _σvCO , and T _σiC set for them. ,repeat.

  The above calculation is performed by a calculation module (not shown). The calculation module can be constituted by a program stored in a computer and a storage device. Further, hardware may be used in place of part or all of the program. Further, the threshold value of the dispersion of the camera internal parameter value is stored in a threshold value storage module (not shown). The threshold storage module can be configured by a storage device.

FIG. 4 shows a flowchart of the second embodiment. In step 401, the number N of incident angles is set to 3. In step 402, light is incident at N different angles, and camera internal parameters are calculated from the three angle differences obtained therefrom (calculated in all _N C _two ways of selecting angles). In step 403, the variance of the camera internal parameters according to the _N C _two ways of selection is evaluated. If the variance is small, in step 404, camera internal parameters are determined. If the dispersion is large, in step 405, light is further incident at a different angle, 1 is added to N, and the process returns to step 402. _N C ₂ is as described in (Note 1) of FIG.

As the effect of the present embodiment, the effect shown in the first embodiment can be obtained.
Also, errors that occur when measuring the rotation angle of the camera, errors that occur when measuring the position of the focal point that converges on the image sensor, and calculation rounding errors that occur when calculating the camera internal parameters When some kind of error occurs, it can be seen that if the variance of the camera internal parameters obtained from the three combinations whose variance is originally 0 becomes large, there is a problem in the calculation accuracy of the camera internal parameters. There is an advantage that an error of the obtained camera internal parameter can be detected.

  In addition, as an effect of the present embodiment, when an error is detected in the obtained camera internal parameter, the incident light is increased and the combination of the incident light angle difference and the corresponding condensing point position difference is increased. Since the combination with the smallest variance of the calculated camera internal parameters can be selected, there is an advantage that the calculation accuracy of the camera internal parameters is improved.

  In the first and second embodiments, as shown in FIG. 2, the incident angle of the parallel beam to the camera is changed by rotating the 208 camera with the 211 camera rotation stage. However, the present invention is not limited to this. The incident angle of the parallel beam to the camera may be changed by a relative position changing module that can change the relative position and angle of the parallel beam extraction modules 216 and 216.

  As mentioned above, the invention made by the present inventor has been specifically described based on the embodiment. However, the invention is not limited to the embodiment, and various modifications can be made without departing from the scope of the invention. Of course.

It is a block diagram of the conventional camera internal parameter measurement apparatus. It is a block diagram of the Example of this invention. It is a related figure of a camera coordinate system and an incident angle. It is a flowchart of Example 2 of this invention.

Explanation of symbols

101 ... light source, 102 ... calibration pattern, 103 ... world coordinate system, 104 ... lens, 105 ... camera coordinate system, 106 ... two-dimensional imaging element, 107 ... digital image coordinate system, 108 ... camera _C i, 204 ... lens, 205 ... camera coordinate system, 206 ... two-dimensional imaging element, 207 ... digital image coordinate system, 208 ... camera CDi, 209 ... theta _YC rotating stage, 210 ... theta _XC rotating stage, 211 ... camera rotation module, 212 ... parallel beam, 213 ... Laser, 214 ... Spatial filter, 215 ... Beam expander, 216 ... Parallel beam extraction module

Claims (3)

A camera internal parameter measurement device for a camera having a lens and a two-dimensional image sensor,
A parallel beam extraction module for emitting a parallel beam;
A relative position changing module that changes the incident angle of the parallel beam on the camera;
Two of three angular differences obtained from three incident angles obtained when a parallel beam is incident on the camera from the parallel beam emitting module three times by changing the incident angle by the relative position changing module, and corresponding thereto And two of the differences in the positions of the three points obtained from the three condensing points collected on the two-dimensional image sensor in the camera, and the image side principal point of the lens, which is a camera internal parameter, and A calculation module for calculating the distance between the light receiving surfaces of the two-dimensional image sensor and the intersection position of the camera optical axis and the two-dimensional image sensor;
A camera internal parameter measuring device.   The camera internal parameter measurement device according to claim 1, wherein the relative position change module is a camera rotation module that rotates the camera. A threshold storage module for storing a dispersion threshold of values of camera internal parameters;
The calculation module has two ways of selecting two of three angle differences obtained from three incident angles and two of three point positions obtained from three corresponding condensing points. The camera internal parameter is calculated, and when the variance of the calculated camera internal parameter value is larger than the threshold value stored in the threshold value storage module, the parallel beam extraction module at the incident angle is different from the above. Using the incident angle and condensing point when the parallel beam is incident on the camera and the incident angle and condensing point obtained so far, three incident angles and condensing point positions are selected, again as described above, The camera internal parameter is calculated, the variance of the calculated camera internal parameter value is compared, and the calculated variance of the camera internal parameter is stored in the threshold storage module. If less than the threshold value, determines the camera internal parameter using the value of the camera internal parameters are calculated,
The camera internal parameter measurement device according to claim 1 or 2, characterized in that

Images