CN105809706A - Global calibration method of distributed multi-camera system - Google Patents

Abstract

The invention discloses a global calibration method of a distributed multi-camera system. The global calibration method includes the steps of using two-dimensional calibration targets formed by two groups of mutually orthogonal parallel straight lines to obtain relative position and attitude initial values through vanishing line equations and known target geometric dimensions based on vanishing point and vanishing line estimation cameras and relative positions and attitudes of the targets, and seeking minimum values of re-projection error functions so as to obtain optimum values of the cameras and the calibration targets; using auxiliary cameras to shoot adjacent target images to obtain a coordinate transformation matrix between the adjacent targets, seeking relative position and attitude initial values of the calibration targets relative to a reference target by an increment method and obtaining optimum values of transformation matrixes of the calibration targets relative to the reference target by seeking minimum values of re-projection error functions based on closed image sequences; through corresponding coordinate transformation, seeking optimum values of the transformation matrixes of the cameras relative to a reference camera to conduct global calibration of the distributed multi-camera system.

Description

Global calibration method of distributed multi-camera system

Technical Field

The invention belongs to the technical field of photogrammetry of an optical system, and particularly relates to a global calibration method for a distributed multi-camera system.

Background

The optical measuring system has the advantages of flexibility, diversity and higher precision. The distributed multi-camera system is a typical optical measurement system, has a wide field coverage, can fuse images acquired by multiple cameras, and is widely used in the fields of vision measurement, object detection and the like. The global calibration of the multi-camera system is to acquire the relative position and attitude relationship between cameras by a certain method, so that the multi-camera coordinate system is unified to the global coordinate system, which is one of the preconditions for optical measurement.

At present, the commonly used global calibration method for multiple cameras includes: the calibration method based on the precise instrument, the self-calibration method based on the feature matching, the calibration method based on the mirror reflection structure overlapping view field and the photogrammetry method based on the target.

The calibration method based on the precision instrument is generally realized by adopting a precision calibration target, a plurality of total stations or a three-dimensional laser measuring device, the measurement precision is high, but the cost of the precision measuring instrument is high. The self-calibration method does not need a special calibration target and is realized by carrying out characteristic detection and matching on the same scene under different visual angles; however, the self-calibration method has low accuracy, requires a sufficiently large overlap of the fields of view of adjacent cameras, and is not suitable for low-brightness and low-ambient texture situations. In order to avoid field shading and obtain better field distribution, the multi-camera system is generally distributed, the field overlap between adjacent cameras is small, and the calibration is difficult to be completed by adopting the method. The calibration method of constructing the overlapped view field based on the mirror reflection is difficult to ensure that each camera can clearly image the calibration target.

Disclosure of Invention

The invention aims to solve the problem of accumulated errors caused by multiple times of coordinate transformation in the global calibration of a distributed multi-camera system, and provides a global calibration method of the distributed multi-camera system.

The invention designs a two-dimensional calibration target consisting of two groups of mutually orthogonal parallel straight lines, and provides a relative position posture (pose) estimation method of a camera and the target based on vanishing point and vanishing line.

The invention relates to a global calibration method of distributed targets, which comprises the steps of shooting images of adjacent targets by using an auxiliary camera to obtain a coordinate transformation matrix between the adjacent targets, and obtaining initial values of relative poses of the calibration targets relative to a reference target by an incremental coordinate transformation method; and obtaining the optimal value of the transformation matrix of each target relative to the reference target by calculating the minimum value of the re-projection error function based on the closed image sequence. After the relative pose estimation of the cameras and the calibration target and the global calibration of the distributed targets are completed, the optimal value of the transformation matrix of each camera relative to the reference camera is obtained through corresponding coordinate transformation, and the global calibration of the distributed multi-camera system is completed.

The invention has the advantages that:

(1) the plane calibration target designed by the invention consists of two groups of mutually orthogonal parallel lines, and compared with point characteristics, the straight line characteristics can better overcome the influence of image noise;

(2) the designed target is used for camera calibration, attitude angle information is deduced according to a vanishing line equation, and a translation vector is determined by combining the known length of a parallel line;

(3) the method uses the auxiliary camera to obtain the closed image sequence of the adjacent target, and is suitable for calibrating a distributed multi-camera system with overlapped view fields and a distributed multi-camera system without the overlapped view fields;

(4) in order to correct accumulated errors caused by multiple times of coordinate transformation, the invention minimizes a re-projection error function according to the closed image sequence constraint, thereby obtaining the optimal value of a transformation matrix of each target relative to a reference target.

Drawings

FIG. 1 is a flow chart of a global calibration method for a distributed camera system;

FIG. 2 is a top view of a calibration target;

FIG. 3 is a schematic diagram of global calibration of a distributed camera system;

FIG. 4 is a schematic representation of adjacent targets (i, j) taken using an auxiliary camera;

FIG. 5 is a schematic representation of the vanishing point and vanishing line of the planar target;

FIG. 6 is a diagram of global calibration result error results.

Detailed Description

The present invention will be described in further detail with reference to the accompanying drawings and examples.

The distributed multi-camera system is composed of M cameras, C_kCF (1. ltoreq. k. ltoreq.M) and A_iCF (i is more than or equal to 1 and less than or equal to M) respectively represents a camera coordinate system of the kth camera and a camera coordinate system of an auxiliary camera for shooting an adjacent target (i, j); i is_kCF (1. ltoreq. k. ltoreq.M) denotes the image pixel coordinate system of the kth camera, I_kThe origin of coordinates of the CF is located at the center of the image plane.

As shown in FIG. 2, the target is composed of two sets of parallel lines orthogonal to each other and having a known length L₁At a pitch of L₂；Respectively representing the mth angular point and the mth characteristic straight line of the target k. T is_kCF (1. ltoreq. k. ltoreq.M) denotes the coordinate system of the target k, o_tIs the origin of a coordinate system, x_t,z_tRepresenting the x-axis and z-axis, respectively. The ECF represents a ground coordinate system, and the origin of the ECF is fixedly connected with the ground and is a coordinate system of the northeast.

Measurement model

Let p be [ u, v ]]^TAnd P ═ X, Y, Z]^TRepresenting points in a two-dimensional image plane and in a three-dimensional space, respectively, u and v are pixel coordinates in an image coordinate system, X, Y, Z represent three-dimensional coordinates,respectively, represent the corresponding homogeneous coordinates,the projection of a point within the target coordinate system TCF onto the camera image plane can be expressed as:

$s\tilde{p}=\[K|0_{3\×1}\]T\tilde{P},K=\left[\begin{array}{ccc}{f}_x& 0& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right],T=\left[\begin{array}{cc}{R}_{3\×3}& t_{3\×1}\\ 0_{1\×3}& 1\end{array}\right],R=\left[\begin{array}{ccc}{R}_{11}& R_{12}& R_{13}\\ R_{21}& R_{22}& R_{23}\\ R_{31}& R_{32}& R_{33}\end{array}\right]---\left(1\right)$

wherein: s represents a scale factor, K is an internal parameter matrix, f_xAnd f_yIs the equivalent focal length, (u)₀,v₀) T denotes a coordinate transformation matrix from a target coordinate system to a camera coordinate system, R is a 3 × 3-dimensional rotation matrix,t is a translation vector of 3 × 1 dimensions, the rotation matrix R is the available Euler angle (Y-X-Z) and the yaw anglePitch angle θ and roll angle φ:

in the present invention, the coordinate transformation matrix is defined as follows:

TABLE 1 definition of coordinate transformation matrix

The global calibration method provided by the invention is shown in fig. 3, in the invention, a camera 1 is selected as a reference camera, and a target 1 is selected as a reference target, and the specific implementation steps of the global calibration method of the distributed multi-camera system provided by the invention are as follows:

estimating the relative pose of a camera and a calibration target;

using a two-dimensional calibration target consisting of two groups of mutually orthogonal parallel straight lines, estimating the relative position posture (pose) of the camera and the target based on vanishing points and vanishing lines, obtaining a relative pose initial value through a vanishing line equation and the known geometric dimension of the target, and obtaining the optimal value of the relative pose of the camera and the calibration target through solving the minimum value of a reprojection error function;

step 1.1, calibrating internal parameters of each camera respectively, wherein the internal parameters of the cameras are regarded as fixed constants, and the posture of the cameras is kept unchanged in the calibration process;

step 1.2, placing the target in the field of view of each camera, wherein the symmetry axis of the target points to the corresponding camera, I_kRepresenting an image of the target k taken by the camera k, to obtainTo image sequence I ═ { I ═ I_k|1≤k≤M}；

Step 1.3, calculating a transformation matrix according to the image sequence IThe method specifically comprises the following steps:

step 1.3.1, obtaining the equation of the line extinguishing

As shown in FIG. 5, two sets of parallel lines converge to vanishing point v, respectively, on the image plane₁And v₂Passing point v₁,v₂The straight line of (1) is the extinction line. In the target coordinate system, two linear equations which are not parallel to each other are as follows:

a_ix+c_iz+d_i＝0,y＝0,(i＝1,2)(3)

wherein: a is₁c₂-a₂c₁≠0，a_i,c_i,d_iIs the correlation coefficient of the straight line equation.

V₁And V₂Respectively representing points at infinity on two straight lines, there are:

$s_i{\tilde{v}}_i=\[K|0_{3\×1}\]T_k^{tc}{\tilde{V}}_i,(i=1,2)---\left(4\right)$

wherein:s_iis a scale factor, and is a function of,and K represents an internal parameter matrix of the camera K, and is the homogeneous coordinate of the ith vanishing point in an image coordinate system.

From formulae (1) and (4), there are:

${\tilde{v}}_i={\[f_x\frac{-c_i{R}_{11}+a_i{R}_{13}}{-c_i{R}_{31}+a_i{R}_{33}}+u_0,f_y\frac{-c_i{R}_{21}+a_i{R}_{23}}{-c_i{R}_{31}+a_i{R}_{33}}+v_0,1\]}^T---\left(5\right)$

wherein R is_ijRepresenting the ith row and jth column element of the rotation matrix R.

The line can be extinguishedCalculated, according to equation (5), there are:

$l=\left[\begin{array}{c}{f}_y(R_{23}{R}_{31}-R_{21}{R}_{33})\\ f_x(R_{11}{R}_{33}-R_{13}{R}_{31})\\ f_x{f}_y(R_{21}{R}_{13}-R_{11}{R}_{23})+f_x{v}_0(R_{13}{R}_{31}-R_{11}{R}_{33})+f_y{u}_0(R_{21}{R}_{33}-R_{23}{R}_{31})\end{array}\right]---\left(6\right)$

according to equations (2) and (6), the equation for the line out can be expressed as:

$\frac{sin\mathrm{\φ}}{f_x}(u-u_0)+\frac{cos\mathrm{\φ}}{f_y}(v-v_0)-tan\mathrm{\θ}=0---\left(7\right)$

step 1.3.2, obtaining a rotation matrix

And obtaining a vanishing line equation by using a least square method according to the characteristic points on the characteristic straight line:

$\tilde{a}u+\tilde{b}v+\tilde{c}=0---\left(8\right)$

wherein,are the coefficients of the resulting linear equation.

From equations (7) and (8), the roll angle Φ and the pitch angle θ can be obtained:

$\left\{\begin{array}{c}\mathrm{\φ}={\tan }^{-1}\[f_x\tilde{a}/\left(f_y\tilde{b}\right)\]\\ \mathrm{\θ}={\tan }^{-1}(-\frac{\tilde{c}}{\tilde{a}}\frac{sin\mathrm{\φ}}{f_x}-\frac{sin\mathrm{\φ}}{f_x}{u}_0-\frac{cos\mathrm{\φ}}{f_y}{v}_0)\end{array}\right.---\left(9\right)$

the vanishing point coordinate and the direction vector of the parallel lines in the camera coordinate system have the following relations:

${\tilde{v}}_i={\mathrm{Kd}}_i,(i=1,2)---\left(10\right)$

wherein d is_iIs a straight line at C_kDirection vector of 3 × 1 dimensions in CF.

Andrespectively with T_kThe z-axis and x-axis of the CF coincide with:

$\{\begin{array}{c}{d}_1/||d_1||=R\·{\left[\begin{array}{ccc}0& 0& 1\end{array}\right]}^T\\ d_2/||d_2||=R\·{\left[\begin{array}{ccc}1& 0& 0\end{array}\right]}^T\end{array}---\left(11\right)$

from the formulas (2) and (11), it is possible to obtainAndto thereby obtain a value ofThe rotation matrix R of (a).

Step 1.3.3, obtaining translation vector

Setting characteristic pointsIs a virtual point on the target plane, becauseAndin the vector d₂The projection on is equal to that:

from formulas (1) and (12), there are:

$\frac{z_1}{z_2}=\frac{d_2^T{K}^{-1}{\tilde{p}}_2^k}{d_2^T{K}^{-1}{\tilde{p}}_1^k}---\left(13\right)$

wherein: z is a radical of₁And z₂Are respectively pointsAndat C_kZ-axis coordinate of CF.

Due to the fact thatThe length of (a) is known as:

$||z_1{K}^{-1}{\tilde{p}}_1^k-z_2{K}^{-1}{\tilde{p}}_2^k||=L_1---\left(14\right)$

from the formulae (13) and (14), z is obtained₁And z₂Thereby obtainingAt C_kPosition coordinates in CF; whileIs T_kOrigin of coordinates of CF, thereforeThe translation vector t of (a) may be expressed as:

$t=z_1{K}^{-1}{\tilde{p}}_1^k---\left(15\right)$

step 1.3.4, nonlinear optimization

The mth corner point representing the target k is at T_kHomogeneous coordinates in CF, which is in image I_kCorresponding secondary coordinate of (A) isM is more than or equal to 1 and less than or equal to 6, and the formula comprises the following components:

$s_m^k{\tilde{p}}_m^k=\[K|0_{3\×1}\]T_k^{tc}{\tilde{p}}_m^k---\left(16\right)$

wherein:is a scale factor.

Let the image points be disturbed by an independent and equally distributed gaussian noiseObtaining maximum likelihood estimation by minimizing the square sum of the distances between the characteristic straight line and the reprojection point, and obtaining the minimum value of the function by using a Levenberg-Marquardt method so as to obtain a transformation matrixOptimum value of (2):

$f\left(\mathrm{\Ω}\right)=\underset{m=1}{\overset{6}{\Σ}}\[d^2({\tilde{p}}_m^k,l_m^k)+d^2({\tilde{p}}_m^k,l_n^k)\]---\left(17\right)$

$n=\left\{\begin{array}{ccc}m-1,& if& m\≥2\\ 6,& if& m=1\end{array}\right.---\left(18\right)$

wherein: space of function arguments Respectively representing images I_kThe m and n lines of the target k, d (-) represents the point-to-line distance.

Step two, global calibration of the distributed targets;

shooting images of adjacent targets by using an auxiliary camera to obtain a coordinate transformation matrix between the adjacent targets, obtaining a relative pose initial value of each calibration target relative to a reference target by an incremental coordinate transformation method, and obtaining an optimal value of the transformation matrix of each target relative to the reference target by obtaining a minimum value of a re-projection error function based on a closed image sequence;

step 2.1, obtaining a sequence of closed images by an auxiliary camera, as shown in fig. 4Wherein the target (i, j) isIt can be seen that:

step 2.2 is illustrated by figure 4,representing the transformation matrices from target i and target j to the auxiliary camera coordinate system, respectively.Can be obtained by the same method as in step 1.3, thereby obtaining

$T_{ij}^{tt}={\left(T_{ji}^{ta}\right)}^{-1}{T}_{ii}^{ta}---\left(20\right)$

Step 2.3, calculating a transformation matrix by adopting an incremental coordinate transformation methodAnd then optimized.

The initial values of (a) are obtained by a plurality of coordinate transformations:

according to the imaging model, there are:

$\left\{\begin{array}{c}{s}_m^{ii}{\tilde{p}}_m^{ii}=\[K|0_{3\×1}\]T_{i2}^{ta}{\left(T_{j1}^{tt}\right)}^{-1}{T}_{i1}^{tt}{\tilde{P}}_m^i\\ s_m^{ji}{\tilde{p}}_m^{ji}=\[K|0_{3\×1}\]T_{i1}^{ta}{\left(T_{i1}^{tt}\right)}^{-1}{T}_{j1}^{tt}{\tilde{P}}_m^j\end{array}\right.---\left(22\right)$

wherein:andm angles representing targets i and j, respectivelyIs spotted onThe coordinates of the image on the upper side,is a scale factor.

Calculating the minimum value of the reprojection error function by using a Levenberg-Marquardt methodOptimum value of (2):

$f\left(\mathrm{\Ω}\right)=\underset{i=1}{\overset{M}{\Σ}}\underset{m=1}{\overset{6}{\Σ}}\[d^2({\tilde{p}}_m^{ii},l_m^{ii})+d^2({\tilde{p}}_m^{ii},l_n^{ii})+d^2({\tilde{p}}_m^{ji},l_m^{ji})+d^2({\tilde{p}}_m^{ji},l_n^{ji})\]---\left(23\right)$

wherein: space of independent variablesFor a four-dimensional identity matrix, the iteration initial values are provided by equations (20) and (21).

Step three, calculating a transformation matrixAnd finishing the global calibration.

Using the target as a medium to obtainThe optimized value of (c):

$T_{k1}^{cc}=T_1^{tc}{T}_{k1}^{tt}{\left(T_k^{tc}\right)}^{-1},(2\≤k\≤M)---\left(24\right)$

and obtaining a transformation matrix of each camera relative to the reference camera, thereby completing the calibration of the distributed multi-camera system.

Example (b):

in this embodiment, the distributed multi-camera system consists of 8 cameras, and the position coordinates of the cameras in the ECF and the euler angles with the ECF are shown in table 1. Target size L₁＝500mm,L₂Calculated as 200mmThe accuracy of the global calibration method provided by the invention is evaluated according to the error.

TABLE 1 Camera position and attitude

100 independent experiments are carried out after the mean value of 0 and the standard deviation of 0.2 pixel Gaussian noise are added into the characteristic point coordinates.

To illustrate the effect of the proposed method, two methods were used to calculate the calibration error: an incremental calibration method and a global calibration method. The only difference between the two methods is whether to perform the target global non-linear optimization represented by formula (23). Of incremental calibration methodsThe global calibration method is the method provided by the invention, and the global calibration method is directly obtained by using the formula (21) without global optimization.

As can be seen in fig. 6, the rotation matrix errors and the translation vector errors accumulate gradually as the number of coordinate transformations increases and reach a maximum at the camera 5 because the camera 5 is farthest from the reference camera (camera 1). The global calibration method provided by the invention can effectively reduce the accumulated error caused by the incremental coordinate transformation.

Claims (4)

1. A global calibration method for a distributed multi-camera system, wherein:

the distributed multi-camera system is composed of M cameras, C_kCF and A_iCF respectively represents a camera coordinate system of a kth camera and a camera coordinate system of an auxiliary camera for shooting an adjacent target (i, j), k is more than or equal to 1 and less than or equal to M, and i is more than or equal to 1 and less than or equal to M; i is_kCF denotes the image pixel coordinate system of the kth camera, I_kThe origin of coordinates of the CF is located at the center of the image plane;

the target consists of two groups of parallel lines which are mutually orthogonal and have known lengths, and the lengths of the parallel lines areIs L₁At a pitch of L₂；Respectively representing the mth angular point and the mth characteristic straight line of the target k, wherein m is more than or equal to 1 and less than or equal to 6; t is_kCF denotes the coordinate system of target k, o_tIs the origin of a coordinate system, x_t,z_tRespectively representing the x-axis and the z-axis; ECF represents a ground coordinate system, and the origin of the ECF is fixedly connected to the ground and is a coordinate system of the northeast;

let p be [ u, v ]]^TAnd P ═ X, Y, Z]^TRepresenting points in a two-dimensional image plane and in a three-dimensional space, respectively, u and v are pixel coordinates in an image coordinate system, X, Y, Z represent three-dimensional coordinates,respectively, represent the corresponding homogeneous coordinates,the projection of a point within the target coordinate system TCF onto the camera image plane is represented as:

$s\tilde{p}=\[K|0_{3\×1}\]T\tilde{P},K=\left[\begin{array}{ccc}{f}_x& 0& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right],T=\left[\begin{array}{cc}{R}_{3\×3}& t_{3\×1}\\ 0_{1\×3}& 1\end{array}\right],R=\left[\begin{array}{ccc}{R}_{11}& R_{12}& R_{13}\\ R_{21}& R_{22}& R_{23}\\ R_{31}& R_{32}& R_{33}\end{array}\right]---\left(1\right)$

wherein: s represents a scale factor, K is an internal parameter matrix, f_xAnd f_yIs the equivalent focal length, (u)₀,v₀) Is the coordinate of the principal point, T represents the coordinate transformation matrix from the target coordinate system to the camera coordinate system, R is the rotation matrix of 3 × 3D, T is the translation vector of 3 × 1D, and the rotation matrix R can be the Euler angle (Y-X-Z): yaw anglePitch angle θ and roll angle φ:

is provided withRepresents T_kCF to C_kThe coordinate conversion matrix of the CF is,represents T_iCF to A_iThe coordinate conversion matrix of the CF is,represents T_jCF to A_iThe coordinate conversion matrix of the CF is,represents T_iCF to T_jThe coordinate conversion matrix of the CF is,is represented by C_iCF to C_jA coordinate transformation matrix of the CF;

the global calibration method of the distributed multi-camera system is characterized by comprising the following steps:

estimating the relative pose of a camera and a calibration target;

using a two-dimensional calibration target consisting of two groups of mutually orthogonal parallel straight lines, estimating the relative position posture of the camera and the target based on vanishing points and vanishing lines, obtaining a relative posture initial value through a vanishing line equation and the known geometric dimension of the target, and obtaining the optimal value of the relative posture of the camera and the calibration target through solving the minimum value of a reprojection error function;

step two, global calibration of the distributed targets;

shooting images of adjacent targets by using an auxiliary camera to obtain a coordinate transformation matrix between the adjacent targets, obtaining a relative pose initial value of each calibration target relative to a reference target by an incremental coordinate transformation method, and obtaining an optimal value of the transformation matrix of each target relative to the reference target by obtaining a minimum value of a re-projection error function based on a closed image sequence;

and step three, calculating a transformation matrix to finish global calibration.

2. A global calibration method for a distributed multi-camera system according to claim 1, said step one comprising:

step 1.1, calibrating internal parameters of each camera respectively, wherein the internal parameters of the cameras are regarded as fixed constants, and the posture of the cameras is kept unchanged in the calibration process;

step 1.2, placing the target in the field of view of each camera, wherein the symmetry axis of the target points to the corresponding camera, I_kAn image sequence I is obtained by representing an image of a target k photographed by a camera k, I_k|1≤k≤M}；

Step 1.3, calculating a transformation matrix according to the image sequence IThe method specifically comprises the following steps:

step 1.3.1, obtaining the equation of the line extinguishing

Two groups of parallel lines converge to vanishing point v on image plane₁And v₂Passing point v₁,v₂The straight line of (1) is the line of extinction; in the target coordinate system, two linear equations which are not parallel to each other are as follows:

a_ix+c_iz+d_i＝0,y＝0(3)

wherein: a is₁c₂-a₂c₁≠0，a_i,c_i,d_iIs the correlation coefficient of the linear equation, i is 1, 2;

V₁and V₂Respectively representing points at infinity on two straight lines, there are:

$s_i{\tilde{v}}_i=\[K|0_{3\×1}\]T_k^{tc}{\tilde{V}}_i---\left(4\right)$

wherein:s_iis a scale factor, and is a function of,the homogeneous coordinate of the ith vanishing point in an image coordinate system is shown, and K represents an internal parameter matrix of a camera K;

from formulae (1) and (4), there are:

${\tilde{v}}_i={\[f_x\frac{-c_i{R}_{11}+a_i{R}_{13}}{-c_i{R}_{31}+a_i{R}_{33}}+u_0,f_y\frac{-c_i{R}_{21}+a_i{R}_{23}}{-c_i{R}_{31}+a_i{R}_{33}}+v_0,1\]}^T---\left(5\right)$

wherein R is_ijThe ith row and the jth column of the rotation matrix R are represented;

the line is extinguishedCalculated, according to equation (5), there are:

$l=\left[\begin{array}{c}{f}_y(R_{23}{R}_{31}-R_{21}{R}_{33})\\ f_x(R_{11}{R}_{33}-R_{13}{R}_{31})\\ f_x{f}_y(R_{21}{R}_{13}-R_{11}{R}_{23})+f_x{v}_0(R_{13}{R}_{31}-R_{11}{R}_{33})+f_y{u}_0(R_{21}{R}_{33}-R_{23}{R}_{31})\end{array}\right]---\left(6\right)$

according to equations (2) and (6), the equation for the line out is expressed as:

$\frac{\sin \mathrm{\φ}}{f_x}(u-u_0)+\frac{cos\mathrm{\φ}}{f_y}(v-v_0)-tan\mathrm{\θ}=0---\left(7\right)$

step 1.3.2, obtaining a rotation matrix

And obtaining a vanishing line equation by using a least square method according to the characteristic points on the characteristic straight line:

$\tilde{a}u+\tilde{b}v+\tilde{c}=0---\left(8\right)$

wherein,the coefficients of the obtained linear equation;

from equations (7) and (8), the roll angle Φ and the pitch angle θ are obtained:

$\left\{\begin{array}{c}\mathrm{\φ}={\tan }^{-1}\[f_x\tilde{a}/\left(f_y\tilde{b}\right)\]\\ \mathrm{\θ}={\tan }^{-1}(-\frac{\tilde{c}}{\tilde{a}}\frac{sin\mathrm{\φ}}{f_x}-\frac{sin\mathrm{\φ}}{f_x}{u}_0-\frac{cos\mathrm{\φ}}{f_y}{v}_0)\end{array}\right.---\left(9\right)$

the vanishing point coordinate and the direction vector of the parallel lines in the camera coordinate system have the following relations:

${\tilde{v}}_i={\mathrm{Kd}}_i---\left(10\right)$

wherein d is_iIs a straight line at C_kA 3 × 1 dimensional direction vector in CF;

andrespectively with T_kThe z-axis and x-axis of the CF coincide with:

$\left\{\begin{array}{c}{d}_1/\left|\right|d_1\left|\right|=R\·{\left[\begin{array}{ccc}0& 0& 1\end{array}\right]}^T\\ d_2/\left|\right|d_2\left|\right|=R\·{\left[\begin{array}{ccc}1& 0& 0\end{array}\right]}^T\end{array}\right.---\left(11\right)$

from the formulas (2) and (11), theAndto thereby obtain a value ofThe rotation matrix R of (2);

step 1.3.3, obtaining translation vector

Setting characteristic pointsIs a virtual point on the target plane, becauseAndin the vector d₂The projection on is equal to that:

from formulas (1) and (12), there are:

$\frac{z_1}{z_2}=\frac{d_2^T{K}^{-1}{\tilde{p}}_2^k}{d_2^T{K}^{-1}{\tilde{p}}_1^k}---\left(13\right)$

wherein: z is a radical of₁And z₂Are respectively a point P₁ ^kAndat C_kZ-axis coordinates of CF;

due to P₁ ¹ The length of (a) is known as:

$\left|\right|z_1{K}^{-1}{\tilde{p}}_1^k-z_2{K}^{-1}{\tilde{p}}_2^k\left|\right|=L_1---\left(14\right)$

from the formulae (13) and (14), z is obtained₁And z₂To obtain P₁ ^kAt C_kPosition coordinates in CF; p₁ ^kIs T_kOrigin of coordinates of CF, thenIs represented as:

$t=z_1{K}^{-1}{\tilde{p}}_1^k---\left(15\right)$

step 1.3.4, nonlinear optimization

The mth corner point representing the target k is at T_kHomogeneous coordinates in CF, which is in image I_kCorresponding secondary coordinate of (A) isM is more than or equal to 1 and less than or equal to 6, and the formula comprises the following components:

$s_m^k{\tilde{p}}_m^k=\[K|0_{3\×1}\]T_k^{tc}{\tilde{P}}_m^k--\left(16\right)$

wherein:is a scale factor;

the image points are interfered by independent Gaussian noise with the same distribution, maximum likelihood estimation is obtained by minimizing the sum of squares of the distances between the characteristic straight lines and the reprojected points, the minimum value of the function is obtained by using a Levenberg-Marquardt method, and a transformation matrix is obtainedOptimum value of (2):

$f\left(\mathrm{\Ω}\right)=\underset{m=1}{\overset{6}{\mathrm{\Σ}}}\[d^2({\tilde{p}}_m^k,l_m^k)+d^2({\tilde{p}}_m^k,l_n^k)\]---\left(17\right)$

wherein: space of function argumentsRespectively representing images I_kThe m and n lines of the target k, d (-) represents the point-to-line distance.

3. The global calibration method of the distributed multi-camera system according to claim 1, wherein the second step specifically comprises:

step 2.1, obtaining a sequence of closed images by an auxiliary cameraWherein the target (i, j) isIt can be seen that:

step 2.2,Respectively representing transformation matrices from target i and target j to the auxiliary camera coordinate system, thenComprises the following steps:

$T_{ij}^{tt}={\left(T_{ji}^{ta}\right)}^{-1}{T}_{ii}^{ta}---\left(20\right)$

step 2.3, calculating a transformation matrix by adopting an incremental coordinate transformation methodAnd then optimized;

respectively through multiple coordinate transformationsObtaining:

wherein k1 is more than or equal to 2 and less than or equal to M, and according to the imaging model, the method comprises the following steps:

wherein:andthe mth angular points respectively representing the target i and the target j are arranged atThe coordinates of the image on the upper side,is a scale factor;

calculating the minimum value of a reprojection error function by using a Levenberg-Marquardt method to obtainOptimum value of (2):

$f\left(\mathrm{\Ω}\right)=\underset{i=1}{\overset{M}{\mathrm{\Σ}}}\underset{m=1}{\overset{6}{\mathrm{\Σ}}}\[d^2\left({\tilde{p}}_m^{ii},l_m^{ii}\right)+d^2\left({\tilde{p}}_m^{ii},l_n^{ii}\right)+d^2\left({\tilde{p}}_m^{ji},l_m^{ji}\right)+d^2\left({\tilde{p}}_m^{ji},l_n^{ji}\right)\]---\left(23\right)$

wherein: space of independent variablesIs a four-dimensional identity matrix.

4. The global calibration method of the distributed multi-camera system according to claim 1, wherein the third step is specifically:

using the target as a medium to obtainThe optimized value of (c):

$T_{k1}^{cc}=T_1^{tc}{T}_{k1}^{tt}{\left(T_k^{tc}\right)}^{-1}---\left(24\right)$

and after the transformation matrix of each camera relative to the reference camera is obtained, the calibration of the distributed multi-camera system is finished.