JPH0814828A - Calibration method for stereoscopic image sensor and device therefor - Google Patents

Abstract

PURPOSE:To provide a calibration method for a stereoscopic image sensor and a calibration device capable of calibrating the real camera system where the right and left lines of sight do not intersect each other in a strict sense. CONSTITUTION:A stereoscopic image sensor is modelled, by use of a pseudo- inverse matrix at a step S1, with the midpoint of normals drawn to the right and left sight lines of a stereoscopic camera taken as the approximate solution of least squares, and this model data is partially differentiated at a step S2, using a camera parameter as a conversion factor between the coordinate system of three-dimensional space and the coordinate system of stereoscopic camera screen. Then, the known position of a measurement object in the three- dimensional space is measured with the stereoscopic camera at a step S4. In this case, constitution is so made that the camera parameter is derived at a step S5 for minimizing a norm between both data, using the above-mentioned measurement data and the differentiated value of the model data.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a stereo image sensor calibration method and apparatus, and more particularly to a stereo image sensor calibration method for performing three-dimensional position measurement by triangulation using two cameras. It relates to the device.

[0002]

2. Description of the Related Art A stereo image sensor measures the position of a target object in a three-dimensional space by the principle of triangulation using two cameras whose geometrical positional relationships are known in advance.
Here, the setting / calibration accuracy of the geometrical positional relationship between the two cameras directly affects the measurement accuracy. Therefore, in order to obtain high measurement accuracy, it is necessary to accurately calibrate the geometrical positional relationship between the two cameras. Generally, the calibration of a stereo image sensor is performed by describing the geometrical positional relationship between two cameras using a mathematical model and deriving the parameter values of the model. In the conventional modeling of a stereo image sensor, for example, as shown in FIG. 9, a measurement reference point P in a three-dimensional space, camera centers O _L and O _{R of} left and right cameras, and imaging points P _L and P _R are all It was performed using the assumption called the so-called epipolar condition that they are on the same plane.

[0003]

The conventional stereo image sensor calibration method as described above uses the epipolar condition. However, in a real camera, the left and right lines of sight are strict due to a quantization error in measurement. Will no longer cross. For this reason, it is difficult to accurately calibrate the actual camera system including errors such as measurement noise with the modeling method using the epipolar condition. The present invention
The present invention has been made in view of the above circumstances, and its purpose is to provide a calibration method for a stereo image sensor capable of accurately performing calibration of an actual camera system so that the left and right eyes do not actually intersect. And to provide the device.

[0004]

In order to achieve the above object, the first invention is a stereo system which uses a pseudo-inverse matrix to set the midpoint of the common normal line of the left and right lines of sight of a stereo camera as a least squares approximate solution. The image sensor is modeled, and the model data is partially differentiated by a camera parameter that is a conversion coefficient between the coordinate system of the three-dimensional space and the coordinate system of the screen of the stereo camera,
A camera parameter that measures the position of a known measurement target in a three-dimensional space with a stereo camera and uses the measurement data and the partial differential value of the model data to minimize the norm of the measurement data and the model data. It is configured as a calibration method of the stereo image sensor obtained by deriving A second aspect of the present invention uses a pseudo-inverse matrix to model a stereo image sensor in which the midpoint of the common normal line of the left and right lines of sight of the stereo camera is used as the least squares approximate solution, and the common normal line included in the model data is modeled. The evaluation function consisting of the length is partially differentiated by the camera parameter which is a conversion coefficient between the coordinate system of the three-dimensional space and the coordinate system of the screen of the stereo camera, and 3
The position of an arbitrary measurement object in the dimensional space is measured by a stereo camera, and camera parameters that minimize the norm of the evaluation function are derived using the measurement data and the partial differential value of the evaluation function. This is a calibration method for a stereo image sensor. Furthermore, it is a calibration method for a stereo image sensor that uses the least squares method to derive the camera parameters.

A third aspect of the present invention uses a pseudo inverse matrix to model a stereo image sensor in which the midpoint of the common normal line of the left and right lines of sight of the stereo camera is used as a least squares approximate solution, and the model data is converted into a three-dimensional space. And a first computing means for performing partial differentiation with a camera parameter that is a conversion coefficient between the coordinate system of the stereo camera and the coordinate system of the screen of the stereo camera, and the first measuring the position of the known measurement target in the three-dimensional space with the stereo camera. And a first calibration means for deriving a camera parameter that minimizes the norm of the measurement data and the model data by using the measurement data and the partial differential value of the model data. This is a calibration device for a stereo image sensor. A fourth invention is a second modeling means for modeling a stereo image sensor using a pseudo-inverse matrix, wherein a midpoint of a common normal line of left and right lines of sight of a stereo camera is used as a least squares approximate solution, and the model data. Second calculation means for partially differentiating an evaluation function consisting of the length of the included common normal with a camera parameter which is a conversion coefficient between the coordinate system of the three-dimensional space and the coordinate system of the screen of the stereo camera, and in the three-dimensional space Second measurement means for measuring the position of an arbitrary measurement object with a stereo camera, and camera parameters that minimize the norm of the evaluation function are derived using the measurement data and the partial differential value of the evaluation function. A calibration device for a stereo image sensor, comprising a second calibration means. Furthermore, it is a calibration device for a stereo image sensor that uses the least squares method to derive the camera parameters.

[0006]

According to the first and third aspects of the present invention, the stereo image sensor is first modeled using the pseudo-inverse matrix with the midpoint of the common normal line of the left and right lines of sight of the stereo camera as the least squares approximate solution. Next, the model data is partially differentiated by a camera parameter which is a conversion coefficient between the coordinate system of the three-dimensional space and the coordinate system of the screen of the stereo camera. Then, the position of the known measurement target in the three-dimensional space is measured by a stereo camera, and the norm of the measurement data and the model data is minimized by using the measurement data and the partial differential value of the model data. Parameters are derived. Since the above modeling takes into consideration errors existing in a real camera system such as measurement noise in advance, it is necessary to accurately calibrate the real camera system so that the left and right eyes do not exactly intersect, that is, the epipolar condition is not established. It becomes possible to do it. According to the second and fourth aspects of the invention, the modeling of the stereo image sensor is first performed using the pseudo-inverse matrix with the midpoint of the common normal line of the left and right lines of sight of the stereo camera as the least squares approximate solution. Next, the evaluation function consisting of the length of the common normal included in the model data is partially differentiated by the camera parameter which is a conversion coefficient between the coordinate system of the three-dimensional space and the coordinate system of the screen of the stereo camera. Then, the position of an arbitrary measurement object in the three-dimensional space is measured by a stereo camera, and camera parameters that minimize the norm of the evaluation function are derived using the measurement data and the partial value of the evaluation function. It In this case as well, since errors existing in the actual camera system such as measurement noise are taken into consideration in advance, it is necessary to accurately calibrate the actual camera system so that the left and right eyes do not exactly intersect, that is, the epipolar condition is not established. It becomes possible to do. Furthermore, since an evaluation function that is a function of only camera parameters is used here, self-calibration that does not require position data of the measurement target in the three-dimensional space can be performed. Furthermore, if the least squares method is used for deriving the camera parameters, it is possible to obtain a highly reliable and highly accurate calibration result as compared with the case of using another type of optimization algorithm.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments embodying the present invention will be described below with reference to the accompanying drawings for the understanding of the present invention. The following embodiments are examples of embodying the present invention and are not intended to limit the technical scope of the present invention. 1 is a block diagram showing a schematic processing flow of a calibration method for a stereo image sensor according to a first embodiment of the present invention, and FIG. 2 is a schematic diagram of an apparatus to which the calibration method for a stereo image sensor can be applied. Block diagram showing the configuration, FIG. 3 is an explanatory diagram of the coordinate system of the stereo vision sensor,
4 is an explanatory diagram of projective transformation, FIG. 5 is a conceptual diagram of mapping from the world coordinate system to the screen coordinate system, FIG. 6 is an explanatory diagram showing the geometrical relationship between the left and right lines of sight, and FIG. 7 is the creation of calibration data. FIG. 8 is a block diagram showing a schematic processing flow of a calibration method for a stereo image sensor according to a second embodiment of the present invention. As shown in FIG. 1, a stereo image sensor calibration method according to an embodiment (first embodiment) of the first aspect of the present invention uses a pseudo inverse matrix to determine the left and right line of sight of a stereo camera including two cameras. A stereo image sensor is modeled with the midpoint of the common normal line of the above as a least squares approximate solution (S1), and the model data is converted into a camera that is a conversion coefficient between the coordinate system of the three-dimensional space and the coordinate system of the screen of the stereo camera Partial differentiation with a parameter (S2), and the position of a known measurement target in the three-dimensional space is measured by a stereo camera (S2).
3), using the measurement data and the partial differential value of the model data, a camera parameter that minimizes the norm of the measurement data and the model data is derived (S4). Furthermore, the least squares method may be used to derive the camera parameters.

An apparatus to which the above method can be applied is shown in FIG.
That is, the calibration device 1 is the third invention, and the step S1 in the above method is performed by the stereo image modeling device 3 (corresponding to the first modeling means) in FIG. 2 and the differentiating device 4 (first computing means). Corresponding to step S2), the stereo camera 2 (corresponding to the first measuring means) and the calibration data storage device 5 through step S3, and the stereo camera parameter calculating device 6 (corresponding to the first calibrating means). Due to
The above step S4 is executed. The execution result is stored in the camera parameter storage means 7, and the stereo camera 2
It is used for the actual measurement by. Below, the basic principle is clarified according to the procedure of the above method. (Step S1) Modeling of Stereo Image Measuring Device Using Pseudo Inverse Matrix The position of the reference point ^w x described in the world coordinate system Σ _{w as} shown in FIG. 3 is measured using two cameras. _Let the imaging planes (screens) of the left and right cameras be π _L and π _R , respectively. The position and orientation of the left and right screens are described by the coordinate systems Σ _LS and Σ _RS set on each screen. As shown in FIG. 3, the screen coordinate system has an origin at the intersection of the screen normal line passing through the center of the lens and the screen, the normal line direction is the k axis, and the scanning line direction orthogonal to the k axis is the i axis. And the direction orthogonal to these two axes is defined as the j axis (right-handed system).

Next, the positions and orientations of the coordinate systems of the left and right screens viewed from the world coordinate system are converted into ^w A _LS and ^w A _RS , respectively, using simultaneous conversion.

[Equation 1] Next reference points ^S x written in the screen coordinate system sigma _S as shown in FIG. 4 is projected to the point ^P X on the screen coordinate system sigma _S by projective transformation described below.

[Equation 2] However, f (> 0) is the distance (focal length) between the screen coordinate origin and the camera center. Here, since the z component has no meaning, it is deleted.

(Equation 3) Next, in the homogeneous conversion format, each point is described as follows.

[Equation 4] If the above equation (4) is described for the left and right screens,
It looks like this:

(Equation 5)

Here, the projective transformation of the above equation (6) is defined as follows.

(Equation 6) Then, the above expression (6) is rewritten by the expression of the above expression (5).

(Equation 7) If the above equation (10) is put together into one, the following equation is obtained.

(Equation 8) After all, the above equation (11) and

[Equation 9] As a result, the reference point ^Wx in the world coordinate system is mapped to the projection point ^{Px in the} screen coordinate system. In reality, the conversion from the screen (CCD surface) to the frame buffer is distorted due to the influence of sampling or the like. This distortion is due to the CCD element used, the data input device,
Since it depends on hardware characteristics such as the mounting accuracy of the CCD, it is difficult to express a general distortion conversion formula. However, in any case, since the distortion transformation of the structure according to the characteristics of the hardware is applied to the above equation (10), it is only necessary to redefine it as ^P X. Therefore, the discussion on the distortion of the frame buffer is performed here. I will not.

A parameter expressing the mounting position of each camera, the focal length, the distortion of the frame buffer, etc. is defined as a camera parameter c. Here, the above (1), (6)
Each parameter in the expression,

[Equation 10] However, it becomes a concrete camera parameter, and calibration is nothing but derivation of the values of these parameters. The point ^P X in the screen space is mapped by the function g having the reference point ^W x in the world coordinate system and the camera parameter c as variables. FIG. 5 is a schematic representation of this relationship and its inverse relationship. Next, conversely to the above mapping, the projection point of the screen coordinate system
Mapping from ^P X to the reference point ^W x in the world coordinate system. For this purpose, first, the transformation matrix of the above equation (11) is defined as follows.

[Equation 11] Here, M _L and M _R are 3 × 4 matrices, m _Li and m _Ri (i =
1, 2, 3) is a 1 × 4 matrix. Regarding the above equations (10) and (14), considering the screen on the left side is as follows.

(Equation 12)

Further, only the first and second lines of the above equation (15) will be described as follows.

(Equation 13) The following equation is similarly derived on the right screen.

[Equation 14] The above equations (17) and (19) are combined into one.

(Equation 15) The right side of the above equation (20) is transposed to the left side to obtain the following equation.

[Equation 16] The matrix of the above equation (21) is set as follows.

[Equation 17]

Here, L is a 4 × 4 matrix which is a function of the camera parameter c and the projection point ^P X on the screen coordinates. Let the function L be a 4 × 3 matrix L ₁ and a 4 × 1 matrix l as
Separate into ₂ .

(Equation 18) At this time, the above equation (21) becomes as follows.

[Formula 19] By solving the above equation (24), the point in the world coordinate space ^{(W x, W y, W} z) is determined. The above equation (24) is a ternary simultaneous equation with unknown variables ^W x, ^W y, and ^W z, but there are four equations in total. The necessary and sufficient conditions for the solution to exist in the equation (24) are as follows.

[Equation 20]Considering this meaning geometrically, the above equation (25) holds.
That is, as shown in the equation (a) of FIG.
The viewpoint of the camera system on the right intersects (= [^Wx^Wy ^Wz])
Having, ie assuming an epipolar condition, etc.
Yes. On the other hand, in reality, the lines of sight of the left and right camera systems have an intersection.
In addition, the position of the twist as shown in FIG.
The lines of sight as shown in (c) may match.

Here, when the lines of sight of the left and right camera systems do not have an intersection (FIG. 6B), the midpoint of the common normal to the lines of sight is considered to be the solution. At this time, the solution of the (24) becomes the least square approximation solution, i.e., pseudo-inverse of the matrix L ₁ L ₁ ⁺
Using

[Equation 21] Ideally, the left and right lines of sight have an intersection, but in reality, it is considered that the state is as shown in FIG. 6B due to a physical error. The projection point ^P X in the screen coordinate system is mapped to the reference point ^w x in the world coordinate system by the equation (26). This solution gives the midpoint of the common normal line of the straight lines (line of sight) connecting the left and right lens centers and the imaging point. (Step S2) Derivation of differential relationship From the above equations (11), (12), and (26), the projection point ^P X of the screen coordinate system and the reference point of the world coordinate system
Mapping with ^W x is performed by the functions f and g as shown in FIG. 5 via the camera parameter c.

[Equation 22]

Here, the matrix is differentiated as follows. The matrix A of m × n order is b
_kl (k = 1,2, ..., p; l = 1,2, ..., q) is each element b _kl of a p × q-order matrix B having (k, l) elements, and the matrix A is When differentiable, the matrix A is differentiated with respect to B in the following mp × nq order matrix.

(Equation 23) Particularly, when the matrix B is the row vector b of the 1 × q order, the differentiation of the matrix A with respect to the row vector b is a matrix of the m × nq order as follows.

[Equation 24] Hereinafter, when the projection point ^P X of the screen coordinate system is given, the error that occurs in the calculation for deriving the reference point ^W x of the world coordinate system is derived due to the error of the camera parameter c.
At this time, the differential relationship is as follows.

(Equation 25) Here, N is the number of camera parameters c (14 in the first embodiment). To facilitate the calculation, the above (2
The calculation of the above equation (30) is executed based on the equation (6).

[0016]

(Equation 26) Hereinafter, the partial differential of the matrix product of the above equation (31) is calculated. During the calculation, the camera parameter c is a p × q matrix (actually, a 1 × N matrix), and I _k is a k × k unit matrix. Next, the Kronecker product in the matrix A of m × n order and the matrix B of p × q order is defined as follows.

[Equation 27] That is, the Kronecker product is a block matrix in which a _ij B is an (i, j) block. As a preparation for calculation, the matrix H = ABC in which the product is defined is differentiated with the camera parameter c.

[Equation 28] In the above equation (33), the matrix A = (L ₁ ^t L ₁ ) ⁻¹ ,
B = L ₁ ^t, by substituting C = L _2, executes the partial differential of the equation (31).

[0017]

[Equation 29] Here, with respect to the above equation (37),

[Equation 30] Substituting for, we obtain

[Equation 31] The above equation (39) is organized and substituted into the above equation (30).

[Equation 32] Assuming the epipolar condition of the conventional method, that is, (2)
In the modeling method that assumes equation (5) as a constraint,

[Expression 33] As a result, the influence of the second term on the right side of the above equation (40) was ignored, and lack of rigor. In the first embodiment, when the differential relationship between the screen coordinate system and the world coordinate system, that is, the projection point ^P X of the screen coordinate system is given by the above equation (40), the error (small change in the camera parameter c) ) And the error of the projection point itself such as quantization, the error that occurs during the derivation of the reference point ^w x in the world coordinate system is calculated.

(Step S3) Collection of Calibration Data As shown in FIG. 7, measurement points whose positions in the world coordinate system are known.

(Equation 34) Are prepared, and the data obtained by capturing them with the left and right cameras are obtained as follows.

[Equation 35] However, m is the number of measurement points. The positions and the number of measurement points in the world coordinate system are selected so that the rank of the matrix J of the equation (45) described later is N (the number of camera parameters c) or more. (Step S4) Calibration Calculation The data of the above equations (42) and (43) and the above (26),
The camera parameter c of the equation (13) is derived using the equation (40). (Step S4a) Initial values of camera parameters

[Equation 36] And the convergence criterion e (> 0).

(Step S4b) Above (42), (4
The following amount is calculated using the measurement data of the equation 3).

(37) (Step _{^{S4c) Δx = w x D -}} w x N ... calculate the (46), | Δx | ≦ e ... (47) if completed. From (Step ^{S4d) Δ w x = JΔc T} ... (48), calculates the Δc ^T = J ⁺ Δ ^w x [(J ^T J) ^-1 J ^T] Δ ^w x ... (49). (Step S4e) c ← c + Δc (50) and the process returns to step S4b. With the above iterative calculation, the camera parameter c can be derived with high accuracy.

As described above, since the modeling in the first embodiment takes into consideration the error existing in the actual camera system such as the measurement noise in advance, the left and right lines of sight do not exactly intersect, that is, the epipolar condition. It is possible to accurately perform the calibration of the actual camera system that does not satisfy the above condition. In the first embodiment, the calibration can be performed accurately by deriving the camera parameter that minimizes the norm between the actual data of the measurement object and the model data by the least square method. In use, other types of optimization algorithms may be used. However, according to the least-squares method, which is excellent in reliability, it is possible to obtain a highly reliable and highly accurate calibration result as compared with the case of using another type of optimization algorithm. By the way, in the first embodiment, the known position data of the measuring object in the three-dimensional space is required. However, the self-calibration that does not require the data is realized in the following second and fourth embodiments. It is an invention. . Second
In a calibration method for a stereo image sensor according to an embodiment (second embodiment) of the invention, the pseudo-inverse matrix is used to set the midpoint of the common normal line of the left and right lines of sight of the stereo camera to the least squares approximate solution. A stereo image sensor is modeled (S11), and an evaluation function consisting of the length of the common normal included in the model data is set as a camera parameter which is a conversion coefficient between the coordinate system of the three-dimensional space and the coordinate system of the screen of the stereo camera. Partial differentiation (S12), and the position of an arbitrary measurement object in the three-dimensional space is measured by a stereo camera (S1
3), it is configured to derive a camera parameter that minimizes the norm of the evaluation function using the measurement data and the partial differential value of the evaluation function (S14).

Further, the least squares method may be used to derive the camera parameters. Also for this method, the apparatus used in the first embodiment can be used. That is, the calibration device 1 in that case is the fourth invention, and the stereo image modeling device 3 in FIG.
(Corresponding to the second modeling means) is the above step S1.
1, the differentiating device 4 (corresponding to the second calculating means) performs the above step S12, the stereo camera 2 (corresponding to the second measuring means), and the calibration data storage device 5 performs the above step S13. Parameter device 6
(Corresponding to the second calibration means) executes the above step S14. The execution result is stored in the camera parameter storage device 7 and used for actual measurement by the stereo camera 2. The basic principle will be clarified below according to the procedure of this method. The step S11 is the same as the step S1 of the first embodiment.
However, the following evaluation function is used after step S12 of the second embodiment. (Step S12) Setting of evaluation function derived above its differential relationship (26), x = ^[w x ^w y ^w z] ^T ... is rewritten as _{(27) ', L 1 x} + l 2 = 0 ... (28 ) ′. Here, the least squares solution is used as follows.

(38)

That is, in the above equation (29) ', the solution using the pseudo-inverse matrix is the least squares solution, and its least squares value is shown. Geometrically, this least square value is the distance between the natural normals on the left and right, and is 0 when the epipolar condition is satisfied. However, in reality, since the epipolar condition is not satisfied, it has a value larger than 0. Therefore, the value of the equation (29) ′ is considered to be the satisfaction degree (evaluation function) for the epipolar condition,

[Formula 39] The camera parameters are calculated so that the epipolar condition is satisfied as shown in. That is, here, the evaluation function is made as small as possible. Here, it should be noted that the evaluation function M _e (c) is a function of only the camera parameter c and does not require the position information of the reference point x. As a preparation for differentiating this evaluation function by the camera parameters, the formula is expanded and simplified.

(Equation 40) Then partially differentiated by the camera parameters c an evaluation function M _e of (33) 'type. In the second embodiment, the matrix vector differentiation is performed as follows.

The matrix A of m × n order is b _kl (k = 1,
, ..., p; l = 1,2, ..., q) is a (k, l) element, and when the matrix A is differentiable by each element b _kl of the matrix B at p × q, The differential with respect to B becomes the following mp × nq-th matrix.

[Formula 41] In particular, when the matrix B is a row vector b of 1 × q degree, the differentiation of the matrix A with respect to b is m ×
It becomes an nqth order matrix.

(Equation 42) In the middle of the calculation, the camera parameter c is a p × q matrix, actually 1 × N matrix: N is the number of camera parameters c (14 in the second embodiment), and I _k is a k × k unit matrix. Is. Next, the Kronecker product in the matrix A of m × n order and the matrix B of p × q order is defined as follows.

[Equation 43] That is, the Kronecker product is a block matrix in which a _ij B is an (i, j) block. As a preparation for calculation, the matrix H = ABC in which the product is defined is differentiated with the camera parameter c.

[Equation 44]

The differential relationship of the evaluation function M _e , which is an epipolar error measure, with the camera parameter c is given by the following equation.

[Equation 45] (Step S13) measuring points in the world coordinate system as shown in Collection Figure 7 data for calibration ^W x _i = _{^{[W x i W y i W z}} i ] ^{T (i = 1, ...,} m) ... (45 ) ′ Is prepared, and the data obtained by capturing them with the left and right cameras is obtained as follows.

[Equation 46] Here, m is the number of measurement points. The position and number of measurement points in the world coordinate system are the matrix J _{s of the} equation (48) ′ described later.
Is selected so that the rank is N or more (the number of camera parameters c). (Step S14) Calibration Calculation The data of the equation (46) ′ and the equation (35) ′, (4
3) ′ equation is used to derive the parameters of the equation (13). (Step S14a) Initial values of camera parameters

[Equation 47] And a convergence determination condition e (> 0) are given.

(Step S14b) The following quantity is calculated using the measurement data of the above equation (46) '.

[Equation 48] (Step S14c) From ΔM = 0-M = -M (49) ', if | ΔM | ≤e (50)', the process ends. (Step S14d) From ΔM = J _s Δc ^T (51) ′, Δc ^T = J _s ⁺ ΔM = [(J _s ^T J _s ) ⁻¹ J _s ^T ] ΔM (52) ′ is calculated. (Step S14e) c ← c + Δc (53) ′, and the process returns to step S14b.

By the above iterative calculation, the camera parameter c can be derived with high accuracy. in this way,
Also in the second embodiment, as in the first embodiment,
Since the errors existing in the actual camera system such as measurement noise are taken into consideration in advance, it is possible to accurately calibrate the actual camera system so that the left and right eyes do not actually intersect, that is, the epipolar condition does not hold. It will be possible. In addition to this, in the second embodiment, the evaluation function is a function of only camera parameters and does not need the position information of the reference point, so that the position data of the measurement object in the three-dimensional space is not needed. Calibration can be performed. Also in the second embodiment, as in the first embodiment, it is possible to perform accurate calibration by using the least squares method for deriving the camera parameters. However, in actual use, other types of optimization are possible. A conversion algorithm may be used. However, also in this case, according to the method of least squares, it is possible to obtain a highly reliable and highly accurate calibration result as compared with the case of using another type of optimization algorithm.

[0027]

The method and apparatus for calibrating a stereo image sensor according to the first and third aspects of the invention have the following effects because they are configured as described above. That is, since the above modeling considers errors existing in an actual camera system such as measurement noise in advance, calibration of an actual camera system in which the left and right eyes do not exactly intersect, that is, the epipolar condition is not established, is performed. It can be done accurately. Further, according to the second and fourth inventions, the following effects can be obtained. That is, in this case as well, since errors existing in the actual camera system such as measurement noise are taken into consideration in advance, it is necessary to accurately calibrate the actual camera system so that the left and right eyes do not exactly intersect, that is, the epipolar condition is not established. It becomes possible to do it. Furthermore, since an evaluation function that is a function of only camera parameters is used here, self-calibration that does not require position data of the measurement target in the three-dimensional space can be performed. Furthermore, if the least squares method is used for deriving the camera parameters, it is possible to obtain a highly reliable and highly accurate calibration result as compared with the case of using another type of optimization algorithm.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic processing flow of a calibration method for a stereo image sensor according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a schematic configuration of an apparatus to which the calibration method for the stereo image sensor can be applied.

FIG. 3 is an explanatory diagram of a coordinate system of a stereo vision sensor.

FIG. 4 is an explanatory diagram of projective transformation.

FIG. 5 is a conceptual diagram of mapping from the world coordinate system to the screen coordinate system.

FIG. 6 is an explanatory diagram showing a geometrical relationship between left and right lines of sight.

FIG. 7 is an explanatory diagram showing how calibration data is created.

FIG. 8 is a block diagram showing a schematic processing flow of a calibration method for a stereo image sensor according to a second embodiment of the present invention.

FIG. 9 is a configuration diagram of a reference stereo camera.

[Explanation of symbols]

1 ... Calibration device 2 ... Stereo camera (corresponding to first and second measuring means) 3 ... Stereo camera image modeling device (corresponding to first and second modeling means) 4 ... Differentiating device (first and second measuring device) 5 ... Calibration data storage device 6 ... Stereo camera parameter calculation device (corresponding to first and second calibration means) 7 ... Camera parameter storage device

Claims (6)

[Claims] 1. A stereo image sensor is modeled using a pseudo-inverse matrix, where the midpoint of the common normal line of the left and right lines of sight of a stereo camera is used as a least squares approximate solution, and the model data is used as a coordinate system in a three-dimensional space. Partial differentiation is performed with the camera parameter that is a conversion coefficient with respect to the coordinate system of the screen of the stereo camera, the position of the known measurement target in the three-dimensional space is measured by the stereo camera, and the partial differential value of the measurement data and the model data is calculated. A method for calibrating a stereo image sensor, wherein a camera parameter that minimizes the norm between the measurement data and the model data is derived using. 2. A stereo image sensor is modeled using a pseudo-inverse matrix, where the midpoint of the common normal line of the left and right lines of sight of the stereo camera is used as a least squares approximate solution, and the length of the common normal line included in the model data is modeled. The partial evaluation of the evaluation function consisting of the three-dimensional space by the camera parameter that is the conversion coefficient between the coordinate system of the three-dimensional space and the coordinate system of the screen of the stereo camera, and the position of any measurement target in the three-dimensional space is measured by the stereo camera. A calibration method for a stereo image sensor, which uses the measurement data and the partial differential value of the evaluation function to derive a camera parameter that minimizes the norm of the evaluation function. 3. The method for calibrating a stereo image sensor according to claim 1, wherein a least squares method is used for deriving the camera parameters. 4. A stereo image sensor is modeled using a pseudo-inverse matrix, where the midpoint of the common normal line of the left and right lines of sight of the stereo camera is used as a least squares approximate solution, and the model data is used as a coordinate system in a three-dimensional space. Partial differentiation with camera parameters that are conversion coefficients with the coordinate system of the screen of the stereo camera 1
And the first measurement means for measuring the position of the known measurement target in the three-dimensional space by the stereo camera, and the measurement data and the model data using the measurement data and the partial differential value of the model data. And a first calibration means for deriving a camera parameter that minimizes the norm of 5. A second modeling means for modeling a stereo image sensor in which a midpoint of a common normal line of left and right lines of sight of a stereo camera is used as a least squares approximate solution by using a pseudo inverse matrix, and included in the model data. Second calculation means for partially differentiating an evaluation function consisting of the length of the common normal with a camera parameter which is a conversion coefficient between the coordinate system of the three-dimensional space and the coordinate system of the screen of the stereo camera, and an arbitrary function within the three-dimensional space Second measuring means for measuring the position of the object to be measured by a stereo camera, and deriving a camera parameter for minimizing the norm of the evaluation function by using the measurement data and the partial differential value of the evaluation function. A calibration device for a stereo image sensor, comprising the calibration means of 2. 6. The calibration device for a stereo image sensor according to claim 4, wherein the least squares method is used for deriving the camera parameter.