JP2005322128A - Calibration method for stereo three-dimensional measurement and three-dimensional position calculating method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a stereo three-dimensional measurement calibrating method capable of easily performing calibration of a stereo camera required to measure a three-dimensional position of a target object by utilizing the stereo camera in stereo three-dimensional measurement. <P>SOLUTION: This calibrating method comprises the steps for: calculating a distance between calibration planes being a first calibration plane and a second calibration plane in parallel; calculating a first projective transformation matrix between stereo cameras for the first calibration plane and a second projective transformation matrix between the stereo cameras for the second calibration plane; calculating the three-dimensional position of the optical center of a reference camera in a world coordinate system fixed to a prescribed reference calibration plane; and calculating a third projective transformation matrix from a reference image plane to the reference calibration plane. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a three-dimensional measurement technique using a stereo camera (hereinafter also referred to as stereo three-dimensional measurement or stereo measurement), and in particular, a stereo camera calibration method for performing stereo three-dimensional measurement and a result of the calibration method. The present invention relates to a method for calculating a three-dimensional position of a target object used.

  In the stereo three-dimensional measurement field, in order to measure a target object using a stereo camera and obtain a three-dimensional position (three-dimensional coordinates) of the target object, it is generally necessary to calibrate the stereo camera in advance. .

  In other words, conventionally, in stereo three-dimensional measurement, the camera internal parameters representing the camera's intrinsic characteristics such as the camera's focal length and the camera external parameters representing the positional relationship between the cameras, or the perspective projection matrix of each camera are known. Otherwise, the three-dimensional position of the target object could not be obtained.

  However, in order to accurately obtain in advance information such as the internal parameters / external parameters of the camera or the perspective projection matrix of each camera, calibration of a stereo camera (hereinafter also referred to as strong calibration) is very laborious. This is usually done at the factory. Well-known strong calibration methods include, for example, the Tsai method disclosed in Non-Patent Document 1 for directly obtaining internal parameters and external parameters, and the perspective projection matrix disclosed in Non-Patent Document 2. There is a method of decomposing into an internal parameter and an external parameter after obtaining.

  Therefore, in a stereo 3D measurement site where a stereo camera is actually installed, even if it becomes necessary to change the focal length, focus, etc. of the camera according to the site situation, It is difficult to perform strong calibration of a stereo camera in the field.

  On the other hand, in the stereo three-dimensional measurement field, as another type of stereo camera calibration method, for example, a stereo that can measure only the relative positional relationship of a three-dimensional scene as disclosed in Non-Patent Document 3 is used. A camera calibration method (hereinafter also referred to as a weak calibration method) is known. As another example of the weak calibration method, for example, there is a “calibration method using a two-plane projective transformation matrix” as disclosed in Non-Patent Document 4 and Non-Patent Document 5.

According to such a weak calibration method, the calibration of the stereo camera can be easily performed, but the information obtained from the calibration result is only the context with respect to the calibration plane, and the three-dimensional of the target object. There is a problem that position information (three-dimensional coordinates) could not be obtained.
R. Wy. RYTsai, “A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the shelf TV” A high-accuracy 3D machine vision metrology using off-the shelf TV cameras and lenses) ”, I Triple E Journal of Robotics and Automation, RA-3, Number 4 (IEEE Journal of Robotics and Automation, RA-3, No.4), p.323-344, 1987 8 Moon Oh. O.Faugeras, "Three dimensional computer vision", The MIT Press, 1993 Zet. Z.Zhang, Earl. R. Deriche, Oh. O.Faugeras, cue. tea. Co-authored by QTLuong, “Robust Technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry”, INR・ A Technical Report, Number 2273 (INRIA technical report, No.2273), May 1994 Hiroshi Mosquito and Takeo Kanade, "Stereovision and Stereo Camera Calibration in Arbitrary Camera Arrangement", IEICE Transactions, 1996, Vol.11, No.11, p.1810-1818 Mosquito Hiroshi, Mino, N., Yasuda, M. and Osumi Masato, “Geometrical Properties of Weak Calibration of Stereo Cameras Using Two Planes”, IEICE Transactions, 1999, Vol. J82-D-II, No.3, p.561-565

  In short, in the conventional strong calibration method, the internal parameters and external parameters of the camera are obtained from the correspondence between a large number of reference points whose three-dimensional positions are known and their camera coordinates, and a precise calibration pattern and accurate positioning are obtained. There is a problem that a mechanism is necessary and it is very difficult to strongly calibrate a stereo camera using a strong calibration method.

  Further, since the distance information measured by the stereo camera weakly calibrated by the conventional weak calibration method is a relative distance from two planes (that is, calibration planes) used for calibration, There has been a problem that the dimension position information (three-dimensional coordinates) cannot be obtained.

  The present invention has been made under the circumstances as described above, and an object of the present invention is a stereo camera required for measuring the three-dimensional position of a target object using a stereo camera in stereo three-dimensional measurement. To provide a calibration method for stereo three-dimensional measurement capable of easily performing calibration, and a three-dimensional position calculation method capable of calculating the three-dimensional position of a target object using the result of the calibration method. It is in.

  The present invention relates to a calibration method for stereo three-dimensional measurement, and the above object of the present invention is a calibration method for performing stereo three-dimensional measurement, that is, three-dimensional measurement using a stereo camera. A distance between calibration planes between one calibration plane and a second calibration plane is obtained, and a first projective transformation matrix between the stereo cameras with respect to the first calibration plane and the second calibration plane. A second projective transformation matrix between stereo cameras with respect to is obtained, a three-dimensional position of the optical center of the reference camera in the world coordinate system fixed to a predetermined reference calibration plane is obtained, and a reference image plane is transferred to the reference calibration plane. Obtaining a third projective transformation matrix of Alternatively, the reference calibration plane is either the first calibration plane or the second calibration plane, or information necessary to obtain the first projective transformation matrix is: Corresponding points between four or more stereo images existing in the first calibration plane in the three-dimensional space observed from the stereo camera, and information necessary for obtaining the second projective transformation matrix are as follows: Necessary for obtaining the third projective transformation matrix by corresponding points between four or more stereo images existing in the second calibration plane in the three-dimensional space observed from the stereo camera. Information is effectively achieved by being world coordinate values of four or more sample points on the reference calibration plane.

The present invention also relates to a method for calculating a three-dimensional position, and the object of the present invention is to obtain the distance between the calibration planes obtained by using the calibration method for stereo three-dimensional measurement according to the present invention and the optics of the reference camera. Based on the three-dimensional position of the center, the first projective transformation matrix, the second projective transformation matrix, and the third projective transformation matrix, the world coordinates ( _Xw , _Yw , Z) of the target object by deriving a _w), alternatively, from the stereo three-dimensional measurement using two-plane projective transformation matrix, deriving the world coordinate component Z _w is perpendicular component calibration plane, the third The step of obtaining the world coordinate components X _w and Y _w based on the projective transformation matrix and the world coordinate component Z _w is effectively achieved.

  If the calibration method for stereo three-dimensional measurement according to the present invention is used, the stereo camera can be easily calibrated in stereo three-dimensional measurement.

  Further, since the four parameters determined by the stereo three-dimensional measurement calibration method according to the present invention are parameters not directly related to the internal parameters and the external parameters of the stereo camera, the stereo 3 in which the stereo camera is actually installed is used. In a three-dimensional measurement site, the internal parameters and external parameters such as the focal length and focus of the stereo camera are adjusted in accordance with the measurement site conditions, and an excellent effect is achieved such that three-dimensional measurement can be performed with high accuracy.

  In other words, if the stereo camera calibrated by the calibration method of the present invention is used, even if the internal parameters / external parameters of the stereo camera are adjusted in accordance with the measurement site conditions at various 3D measurement sites, 3 Calibration required for the three-dimensional position measurement can be easily performed on site, and a remarkable effect is achieved that a very practical three-dimensional measurement can be realized.

  Furthermore, the three-dimensional position calculation method according to the present invention has an excellent effect that the three-dimensional position of the target object can be easily calculated using the parameters determined by the calibration method of the present invention.

The best mode for carrying out the present invention will be described below with reference to the drawings. An embodiment in which the present invention is applied to traffic flow measurement (vehicle detection, tracking) which is one of the actual three-dimensional measurement fields will be described as follows, but the present invention is not limited thereto. Needless to say, the present invention can be applied in various three-dimensional measurement fields.

<1> Stereo 3D Measurement Calibration Method According to the Present Invention Stereo 3D measurement calibration method according to the present invention (hereinafter also simply referred to as the calibration method of the present invention) is a stereo camera in stereo 3D measurement. In order to measure the three-dimensional position of the target object using the method, it is a calibration method that can easily perform the necessary calibration of the stereo camera. As will be described later, the three-dimensional position calculation method according to the present invention is a method capable of calculating the three-dimensional position of the target object using the result of the calibration method of the present invention.

  In other words, the present invention configured by the stereo three-dimensional measurement calibration method and the three-dimensional position calculation method can easily measure the three-dimensional coordinates of the target object and does not require strong calibration (that is, Stereo camera does not require internal parameters and external parameters) This is a three-dimensional measurement method.

What is obtained by the calibration method of the present invention is the following four parameters shown in FIG. In FIG. 1, the reference camera and the reference camera constitute a stereo camera for stereo three-dimensional measurement, and the parallel planes ₀ and ₁ existing in the three-dimensional space observed from the stereo camera are calibrated. A plane.
(1) Distance D between calibration plane _{０ 0} and calibration plane Π ₁
(2) and the projective transformation matrix H ₀ between the stereo camera for the calibration plane [pi _0, projective transformation matrix H ₁ between the stereo camera for the calibration plane [pi ₁
(3) Three-dimensional position (X _c , Y _c , Z _c ) of the optical center of the reference camera in the world coordinate system (X, Y, Z) fixed on the reference calibration plane
In FIG. 1, although the calibration plane [pi ₁ being based calibration plane, without being limited thereto in the present invention, it is of course possible to the calibration plane [pi ₀ as a reference calibration plane. Further, the reference calibration plane in the present invention means a calibration plane in which the world coordinate system (X, Y, Z) is fixed.
(4) Projection transformation matrix H _IC from the reference image plane to the reference calibration plane (in FIG. 1, calibration plane Π ₁ )
The information necessary for the calibration method of the present invention is only the above four parameters, and it is necessary to know all the internal parameters and external parameters of the stereo camera used for the stereo three-dimensional measurement necessary for the conventional strong calibration method. No.

Hereinafter, a method for obtaining the above four parameters necessary for the calibration method of the present invention will be described in detail.

<1-1> Distance between two parallel calibration planes First, two calibration planes that exist in the three-dimensional space observed from the stereo camera and are parallel are set, and the set calibration planes Measure the distance.

Specifically, first, it is assumed that two planes (also called calibration planes) used for calibration are parallel. As shown in FIG. 2 (A), and set the calibration plane [pi ₀ and calibration plane [pi ₁ parallel to each other, the calibration plane [pi ₀ is _set, actually measuring the distance D between [pi _1.

As shown in FIG. 2A, setting the distance between the calibration planes _{０ 0} and Π ₁ to D means that the stereo camera pair is moved to the plane as shown in FIG. This is equivalent to moving the distance D without changing the direction of the optical axis. That is, in FIG. 2B, when the plane Π observed from the stereo camera pair at a position close to the plane と す る is set as the calibration plane _{０ 0} , the stereo camera pair can be moved by a distance D in the vertical direction. For example, the plane plane observed from the stereo camera pair at that time can be used as the calibration plane plane ₁ .

<1-2> Projection Transformation Matrix Between Stereo Cameras for Calibration Plane Information necessary for obtaining a projection transformation matrix between stereo cameras for the calibration plane is a calibration plane in a three-dimensional space observed from the stereo camera. 4 or more points, and the correspondence relationship between the stereo images of the determined points (that is, between the reference image taken by the reference camera and the reference camera taken by the reference camera).

Specifically, for example, when the calibration planes _{０ 0} and Π ₁ as shown in FIG. 2A exist, between stereo images via the calibration planes _{０ 0} and Π ₁ (that is, A projection transformation matrix H ₀ , H ₁ between stereo cameras is obtained.

  FIG. 3 is an example of corresponding points between stereo images and applied to an actual scene. In FIG. 3, the road surface is assumed to be a calibration plane, and corresponding points 1, 2, 3, and 4 are selected. 3A is a reference image taken by the reference camera, and FIG. 3B is a reference image taken by the reference camera.

は基準画像の画像座標の斉次座標表示で、

は参照画像の画像座標の斉次座標表示である。また、数１を展開すると、数２になる。 From the relationship between corresponding points 1, 2, 3, and 4 shown in FIGS. 3 (A) and 3 (B), a projective transformation matrix H in Equation 1 below is obtained. here,

Is a homogeneous coordinate display of the image coordinates of the reference image,

Is a homogeneous coordinate display of the image coordinates of the reference image. Further, when formula 1 is expanded, formula 2 is obtained.

ここで、射影変換行列Ｈは３×３の行列である。この射影変換行列Ｈは、定数倍の不定性を含むため、実際には未知数は８個存在する。

Here, the projective transformation matrix H is a 3 × 3 matrix. Since this projective transformation matrix H includes an indefiniteness of a constant multiple, there are actually eight unknowns.

  When the above equation 2 is transformed, the following equation 3 is obtained, and two equations can be established from one set of corresponding points.

そこで、未知数が８個なので、図３に示されるような対応点の組を４点以上求めると、下記数４のようになり、

Therefore, since there are eight unknowns, when four or more pairs of corresponding points as shown in FIG. 3 are obtained, the following equation 4 is obtained.

上記数４を下記数５の形に置き換えることができ、Ａの最小固有値に対応する固有ベクトルをｈとすることで、射影変換行列Ｈを求めることができる（つまり、最小２乗法を適用している事と等価）。

The above equation 4 can be replaced by the following equation 5, and the projective transformation matrix H can be obtained by setting the eigenvector corresponding to the minimum eigenvalue of A to h (that is, the least square method is applied). Equivalent to that).

上述した射影変換行列Ｈの求め方を用いて、図２（Ａ）に示されるような平行する２つのキャリブレーション平面Π_０,Π_１に対するステレオカメラ間の射影変換行列Ｈ_０,Ｈ_１をそれぞれ導出する。

＜１−３＞基準キャリブレーション平面に固定されたワールド座標系における基準カメラの光学中心の３次元位置
図４はステレオカメラを真横から見た図である。図４において、下側のカメラを基準カメラ、上側のカメラを参照カメラとしているが、実際にはどちらを基準カメラとしても良い。また、図４において、下側に位置するキャリブレーション平面Π_１を基準キャリブレーション平面と呼ぶこととするが、それに限定されることがなく、例えば、上側に位置するキャリブレーション平面Π_０を基準キャリブレーション平面と呼ぶこともできる。

Using the above-described method for obtaining the projection transformation matrix H, the projection transformation matrices H ₀ and H ₁ between the stereo cameras for two parallel calibration planes Π ₀ and Π ₁ as shown in FIG. To derive.

<1-3> Three-dimensional position of the optical center of the reference camera in the world coordinate system fixed on the reference calibration plane FIG. 4 is a diagram of the stereo camera viewed from the side. In FIG. 4, the lower camera is the reference camera and the upper camera is the reference camera, but in actuality, either may be the reference camera. Further, in FIG. 4, but will be referred calibration plane [pi ₁ located on the lower side with reference calibration plane, without being limited thereto, for example, based on the calibration plane [pi ₀ positioned above calibration It can also be called a motion plane.

  Here, in order to simplify the explanation, in FIG. 4, the world coordinate origin is assumed to be placed with a perpendicular foot drawn from the optical center of the reference camera to the reference calibration plane, but the position of the world coordinate origin is there. Without being limited, it is possible to place the world coordinate origin at an arbitrary position on the reference calibration plane, and the Z axis of the world coordinate system is directed from the world coordinate origin to the optical center of the reference camera. Furthermore, the X axis and Y axis of the world coordinate system are defined as orthogonal coordinates on the reference calibration plane.

  Under the preconditions as shown in FIG. 4, the three-dimensional position (three-dimensional coordinates) of the optical center of the reference camera in the world coordinate system fixed to the reference calibration plane is obtained as follows.

Here, from the way of placing the world coordinate system as shown in FIG. 4, the optical center of the reference camera can be placed as (0, 0, D ₁ ). Accordingly, as shown in FIG. 4, if actually measured only distance D ₁ of the from the optical center of the base camera and the reference calibration plane, mean three-dimensional coordinates of the optical center of the base camera in the world coordinate system is obtained is there. That is, the three-dimensional coordinates (X _c , Y _c , Z _c ) of the optical center of the reference camera in the world coordinate system (X, Y, Z) are (0, 0, D ₁ ).

Also, as shown in FIG. 4, the distance D ₀ between the optical center of the reference camera and the calibration plane _{０ 0} is the calibration plane _{０ 0} and the reference calibration plane (here, the calibration plane _{１ 1} ). Since the distance D between and is known, it can be expressed as the following formula 6.

＜１−４＞基準画像平面から基準キャリブレーション平面への射影変換行列Ｈ_ＩＣ
基準画像平面から基準キャリブレーション平面への射影変換行列Ｈ_ＩＣを導出するのに必要な情報とは、基準キャリブレーション平面上の４点以上のサンプル点のワールド座標値である。

<1-4> Projection transformation matrix H _IC from the reference image plane to the reference calibration plane
The information required to derive a projective transformation matrix H _IC from the reference image plane to the reference calibration plane, a world coordinate values of four points or more sample points on the reference calibration plane.

In order to derive the projective transformation matrix H _IC from the reference image plane to the reference calibration plane, the following preconditions are assumed.

First, as shown in FIG. 4, the calibration plane Π ₁ located on the lower side of the two calibration planes is used as a reference calibration plane, and is lowered from the optical center of the reference camera to the reference calibration plane. The vertical foot is the origin of the world coordinates, and the direction from the world coordinate origin to the optical center of the reference camera (that is, the direction perpendicular to the reference calibration plane) is the Z axis of the world coordinate system, and the optical axis of the reference camera The direction orthogonally projected to the reference calibration plane is defined as the Y axis of the world coordinate system. Further, as shown in FIG. 5A, it is defined that the X axis of the world coordinate system is taken to be perpendicular to the Y axis and the Z axis of the world coordinate system.

In FIG. 5, a point M _cp (X _cp , Y _cp , 0) is a sample point on the reference calibration plane, and a point m (u, v) is a point M _cp ( X _cp , Y _cp , 0).

  In the present invention, the method of taking the Z axis of the world coordinate system is as described above, but the remaining world coordinate system axes, that is, the X axis and the Y axis of the world coordinate system are particularly There is no restriction on how to take them, and the X-axis and Y-axis of the world coordinate system only need to be orthogonal coordinates on the reference calibration plane.

Under the above-mentioned preconditions, a projective transformation matrix H _IC from the reference image plane to the reference calibration plane is derived as follows.

  First, as shown in FIG. 6, four or more world coordinates of sample points on the reference calibration plane (on the road surface in the figure) are obtained. In FIG. 6A, the world coordinates of the selected sample points (that is, point 1, point 2, point 3, point 4, point 5, point 6, point 7, and point 8 in the figure) are shown in FIG. It is as the table shown in B).

In the table shown in FIG. 6B, the optical center is the optical center of the reference camera, and the parameters X _cp and Y _cp are the sample points M on the reference calibration plane as shown in FIG. _{represents the cp} world coordinates (X _cp , Y _cp , 0), X _cp , Y _cp , and parameters u, v are the two-dimensional coordinates (u, v) of the point m on the reference image plane as shown in FIG. v) represents u and v.

Next, the projection transformation matrix H _IC from the reference image plane to the reference calibration plane as shown in FIG. 5 from the eight sample points on the reference calibration plane shown in FIG. To derive.

ここで、行列Ｈ_ＩＣ（ここの_ＩＣは基準画像平面から基準キャリブレーション平面へを意味し、つまり、ImagePlane to CalibrationPlaneを意味する）は、基準画像平面から基準キャリブレーション平面への射影変換行列である。

＜２＞本発明に係る３次元位置算出方法
本発明に係る３次元位置算出方法とは、上述したような本発明に係るステレオ３次元計測用キャリブレーション方法によって得られた情報（つまり、＜１＞で述べられた４つのパラメータ）に基づいて、ステレオ３次元計測対象である対象物体を表す点の３次元位置（つまり、対象物体を表す点のワールド座標（Ｘ_ｗ,Ｙ_ｗ,Ｚ_ｗ））を導出する方法であって、その手順は次のようになっている。なお、本発明では、説明を簡単にするために、対象物体を表す点を単に対象物体とも称し、また、対象物体を表す点の３次元位置を単に対象物体の３次元位置とも称するようにしている。また、本発明でいう３次元座標は、ワールド座標を意味する。
ステップ１：
まず、２平面射影変換行列を利用したステレオ３次元計測から、対象物体の３次元座標のキャリブレーション平面の法線方向の成分Ｚ_ｗを導出する。
ステップ２：
そして、基準画像平面から基準キャリブレーション平面への射影変換行列Ｈ_ＩＣと、ステップ１で求められた３次元座標のキャリブレーション平面の法線方向の成分Ｚ_ｗとに基づいて、対象物体の３次元座標の残りの２つの成分Ｘ_ｗ,Ｙ_ｗを求める。

Here, the matrix H _IC (the _IC here means from the reference image plane to the reference calibration plane, that is, ImagePlane to CalibrationPlane) is a projective transformation matrix from the reference image plane to the reference calibration plane. .

<2> Three-dimensional position calculation method according to the present invention The three-dimensional position calculation method according to the present invention is information obtained by the above-described stereo three-dimensional measurement calibration method according to the present invention (that is, <1 > The three-dimensional position of the point representing the target object that is a stereo three-dimensional measurement target (that is, the world coordinates (X _w , Y _w , Z _w ) of the point representing the target object) ), And the procedure is as follows. In the present invention, for the sake of simplicity, the point representing the target object is also simply referred to as the target object, and the three-dimensional position of the point representing the target object is also simply referred to as the three-dimensional position of the target object. Yes. Further, the three-dimensional coordinates referred to in the present invention mean world coordinates.
Step 1:
First, from the stereo three-dimensional measurement using two-plane projective transformation matrix to derive the normal direction of the component Z _w calibration plane of the three-dimensional coordinates of the target object.
Step 2:
Then, a projective transformation matrix H _IC from the reference image plane to the reference calibration plane based on the normal direction of the component Z _w calibration plane of the three-dimensional coordinates calculated in step 1, 3-dimensional object Find the remaining two components X _w , Y _w of the coordinates.

The procedure of the three-dimensional position calculation method of the present invention will be described in more detail as follows.

<2-1> Step 1: By performing stereo three-dimensional measurement using the projective transformation matrix derivation 2 plane components Z _w of the three-dimensional coordinates of the object, perpendicular to the calibration plane in the world coordinate of the target object it can be derived directional component Z _w.

On the other hand, the necessary information in performing stereo three-dimensional measurement using the projection transformation matrix of two planes, parallel to the two calibration planes [pi _0, a projective transformation matrix H _0, H ₁ relates [pi _1, reference camera The distance D ₀ from the optical center to the calibration plane _{０ 0} and the distance D ₁ from the optical center of the reference camera to the calibration plane _{１ 1} . These pieces of information are obtained by the calibration method of the present invention as described above. Incidentally, the distance _{D 0} and the distance _{D 1,} referring to FIG.

Here, a stereo three-dimensional measurement scene using a projective transformation matrix relating to two calibration planes as shown in FIG. 7 is assumed. In FIG. 7, when projection transformation matrices H ₀ and H ₁ relating to two parallel calibration planes _{０ 0} and _{１ 1} are known, a plane 内 that is interpolated (or extrapolated) into the calibration planes _{０ 0} and Π _1. the projective transformation matrix H _alpha related _alpha, can be expressed as the following equation 8.

ステレオ計測を行うことにより、対象物体の奥行きと同一の位置にある平面Π_αを推定することができ、よって、上記数８から対象物体の奥行き係数αを導出することができる。

By performing stereo measurement, it is possible to estimate the plane [pi _alpha in the same location and depth of the target object, thus, it is possible to derive the depth factor of the object alpha from the number 8.

Next, as shown in FIG. 8, the two calibration planes ₀ and Π ₁ are parallel, and the perpendicular distances D ₀ and D ₁ dropped from the optical center of the reference camera to each calibration plane are known. If so, the depth D _α of the target object corresponding to the depth coefficient α of the target object obtained by the above equation 8 can be obtained based on the following equation 9.

よって、下記数１０に基づいて、対象物体のワールド座標の射影平面（つまり、キャリブレーション平面）に垂直な方向成分Ｚ_ｗを求めることが可能になる。

Therefore, on the basis of the following Expression 10, projective plane in the world coordinate of the target object (i.e., the calibration plane) it is possible to determine a direction perpendicular component Z _w to.

＜２−２＞ステップ２：対象物体の３次元座標の残りの２つの成分Ｘ_ｗ,Ｙ_ｗの導出
対象物体の３次元座標（つまり、ワールド座標）の残りの２つの成分Ｘ_ｗ,Ｙ_ｗの導出には、基準画像平面から基準キャリブレーション平面への射影変換行列Ｈ_ＩＣ、ステップ１で得られた対象物体のワールド座標の成分Ｚ_ｗ、基準カメラの光学中心から基準キャリブレーション平面までの距離Ｄ_１（図９参照）といった情報を用いる。

<2-2> Step 2: The remaining two components _{X w} of the remaining two components _{X w} of the 3-dimensional coordinates of the object, three-dimensional coordinates of the derived target object _{Y w} (i.e., world coordinates), _{Y w} Is derived from the reference image plane to the reference calibration plane by the projective transformation matrix H _IC , the world coordinate component Z _{w of} the target object obtained in step 1, and the distance from the optical center of the reference camera to the reference calibration plane. Information such as D ₁ (see FIG. 9) is used.

Here, in FIG. 9, a point on the reference image plane (more specifically, a point representing a target object on the reference image captured by the reference camera) is displayed on the reference calibration plane using the projective transformation matrix H _IC . _{Project to the} point M _cp (X _cp , Y _cp , 0). In the present invention, the world coordinate components X _w and Y _w of the point M _w representing the target object from the point M _cp (X _cp , Y _cp , 0) mapped to the reference calibration plane (here, the road surface) are changed. It is a reason for seeking.

As shown in FIG. 9, using the similar relationship between the triangle OCM _cp and the triangle OCM _w , the world coordinate component Y _w , of the point M _w representing the unknown target object based on the following equations 12 and 14, _Xw can be calculated.

  That is, since the following formula 11 is established, the following formula 12 is obtained. Moreover, since the following formula 13 is established, the following formula 14 is obtained.

上述したように、本発明に係る３次元位置算出方法によれば、対象物体（つまり、対象物体を表す点Ｍ_ｗ）のワールド座標の全ての成分、つまり、（Ｘ_ｗ,Ｙ_ｗ,Ｚ_ｗ）を求めることができる。

As described above, according to the three-dimensional position calculation method according to the present invention, all the components of the world coordinates of the target object (that is, the point M _w representing the target object), that is, (X _w , Y _w , Z _w ).

It is a schematic diagram for demonstrating the parameter calculated | required with the calibration method of this invention. In this invention, it is a schematic diagram for demonstrating a stereo camera and a calibration plane. It is the figure applied to the actual scene in the example of the corresponding point between stereo images. It is the figure which looked at the stereo camera from the side. It is a schematic diagram for demonstrating conversion from the coordinate system of a reference | standard image plane to a world coordinate system. It is a figure which shows an example of how to take the sample point on a reference | standard calibration plane, and a world coordinate value. It is a schematic diagram for demonstrating the three-dimensional measurement using the projection transformation matrix regarding two calibration planes. In the case of the three-dimensional measurement shown in FIG. 7, it is the figure which looked at the stereo camera from right side. FIG. 6 is a schematic diagram for explaining derivation of the remaining two components X _w and Y _w of the three-dimensional coordinates of the target object.

Claims (6)

A calibration method for performing stereo 3D measurement, that is, 3D measurement using a stereo camera,
A distance between calibration planes between the parallel first calibration plane and the second calibration plane is obtained, and the first projective transformation matrix between the stereo cameras with respect to the first calibration plane and the second calibration plane are calculated. A second projection transformation matrix between stereo cameras with respect to the calibration plane is obtained, a three-dimensional position of the optical center of the reference camera in the world coordinate system fixed to a predetermined reference calibration plane is obtained, and the reference calibration is obtained from the reference image plane. A calibration method for stereo three-dimensional measurement, comprising a step of obtaining a third projective transformation matrix onto a projection plane.   The stereo three-dimensional measurement calibration method according to claim 1, wherein the reference calibration plane is one of the first calibration plane and the second calibration plane.   The information necessary for obtaining the first projective transformation matrix is corresponding points between four or more stereo images existing on the first calibration plane in the three-dimensional space observed from a stereo camera, and The information necessary for obtaining the second projective transformation matrix is corresponding points between four or more stereo images existing on the second calibration plane in the three-dimensional space observed from a stereo camera. Item 3. A calibration method for stereo three-dimensional measurement according to Item 2.   The calibration method for stereo three-dimensional measurement according to claim 2, wherein the information necessary for obtaining the third projective transformation matrix is world coordinate values of four or more sample points on the reference calibration plane. 5. The distance between the calibration planes, the three-dimensional position of the optical center of the reference camera, obtained by using the stereo three-dimensional measurement calibration method according to claim 1; Deriving world coordinates (X _w , Y _w , Z _w ) of the target object based on one projection transformation matrix, the second projection transformation matrix, and the third projection transformation matrix; A three-dimensional position calculation method. Stereo three-dimensional measurement using two-plane projective transformation matrix, deriving the world coordinate component Z _w is perpendicular component calibration plane,
Obtaining the world coordinate components X _w and Y _w based on the third projective transformation matrix and the world coordinate component Z _w ;
The three-dimensional position calculation method according to claim 5, wherein:

Images