CN105809706A

CN105809706A - Global calibration method of distributed multi-camera system

Info

Publication number: CN105809706A
Application number: CN201610354114.5A
Authority: CN
Inventors: 吴晓龙; 吴森堂
Original assignee: Beihang University
Current assignee: Beihang University
Priority date: 2016-05-25
Filing date: 2016-05-25
Publication date: 2016-07-27
Anticipated expiration: 2036-05-25
Also published as: CN105809706B

Abstract

The invention discloses a global calibration method for a distributed multi-camera system, comprising the following steps: using a two-dimensional calibration target composed of two sets of mutually orthogonal parallel straight lines, estimating the relative relationship between the camera and the target based on the vanishing point and the vanishing line Position and attitude (pose), the initial value of the relative pose is obtained through the vanishing line equation and the known target geometry, and the optimal value of the relative pose between the camera and the calibration target is obtained by finding the minimum value of the reprojection error function; using The auxiliary camera captures the images of adjacent targets to obtain the coordinate transformation matrix between adjacent targets, and obtains the initial value of the relative pose of each calibration target relative to the reference target through the incremental method, and obtains the reprojection error function based on the closed image sequence The minimum value of the transformation matrix of each target relative to the reference target is obtained; through the corresponding coordinate transformation, the optimal value of the transformation matrix of each camera relative to the reference camera is obtained, and the distributed multi-camera system global calibration.

Description

A Global Calibration Method for Distributed Multi-camera System

技术领域technical field

本发明属于光学系统的摄影测量学技术领域，具体来说是一种用于分布式多像机系统的全局标定方法。The invention belongs to the technical field of photogrammetry of optical systems, in particular to a global calibration method for a distributed multi-camera system.

背景技术Background technique

光学测量系统有着灵活多样、精度较高的优点。分布式多像机系统作为一种典型的光学测量系统，其视场覆盖范围大，可对多像机所获取的图像进行融合，因此广泛用于视觉测量、物体探测等领域。多像机系统的全局标定是通过一定的方法获取像机之间的相对位置与姿态关系，从而将多像机坐标系统一至全局坐标系下，是进行光学测量的前提条件之一。The optical measurement system has the advantages of flexibility, variety and high precision. As a typical optical measurement system, the distributed multi-camera system has a large field of view and can fuse the images acquired by the multi-camera, so it is widely used in visual measurement, object detection and other fields. The global calibration of the multi-camera system is to obtain the relative position and attitude relationship between the cameras through a certain method, so that the coordinate system of the multi-camera is integrated into the global coordinate system, which is one of the prerequisites for optical measurement.

目前，常用的多像机全局标定方法有：基于精密仪器的标定方法、基于特征匹配的自标定方法、基于镜面反射构造重叠视场的标定方法和基于靶标的摄影测量学方法。At present, the commonly used multi-camera global calibration methods are: calibration method based on precision instruments, self-calibration method based on feature matching, calibration method based on mirror reflection to construct overlapping fields of view, and target-based photogrammetry method.

基于精密仪器的标定方法一般采用精密标定靶标、多台全站仪或三维激光测量装置实现，测量精度高，但精密测量仪器成本高。自标定方法不需要特殊的标定靶标，它通过对不同视角下的同一景物进行特征检测与匹配实现；然而自标定方法精度较低，需要相邻像机具有足够大的视场重叠，且不适用于低亮度、低环境纹理场合。而为了得到避免视场遮蔽、获取更好的视场分布，多像机系统一般呈分布式布置，相邻像机间的视场重叠较小，难以采用上述方法完成标定。基于镜面反射构造重叠视场的标定方法，难以保证每一个像机对标定靶标都能清晰成像。The calibration method based on precision instruments is generally realized by precise calibration targets, multiple total stations or three-dimensional laser measurement devices, which has high measurement accuracy, but the cost of precision measurement instruments is high. The self-calibration method does not require a special calibration target, and it is realized by feature detection and matching of the same scene under different viewing angles; however, the self-calibration method has low accuracy, requires adjacent cameras to have a large enough field of view overlap, and is not applicable In low brightness, low environment texture occasions. In order to avoid field of view shadowing and obtain better field of view distribution, multi-camera systems are generally arranged in a distributed manner, and the overlapping of fields of view between adjacent cameras is small, making it difficult to complete the calibration with the above method. The calibration method based on mirror reflection to construct overlapping fields of view is difficult to ensure that each camera can clearly image the calibration target.

发明内容Contents of the invention

本发明的目的是为了解决分布式多像机系统的全局标定中由于多次坐标变换导致的积累误差的问题，提出一种分布式多像机系统的全局标定方法。The object of the present invention is to propose a global calibration method for a distributed multi-camera system in order to solve the problem of accumulated errors caused by multiple coordinate transformations in the global calibration of a distributed multi-camera system.

本发明设计了由两组相互正交平行直线组成的二维标定靶标，在此基础上提出了一种基于灭点和灭线的像机与靶标的相对位置姿态(位姿)估计方法，通过灭线方程和已知的靶标几何尺寸得到相对位姿初始值，通过求取重投影误差函数最小值得到像机与标定靶标的相对位姿的最优值。The present invention designs a two-dimensional calibration target composed of two groups of mutually orthogonal parallel straight lines, and on this basis, proposes a method for estimating the relative position and posture (pose) of the camera and the target based on the vanishing point and the vanishing line, through The initial value of the relative pose is obtained by the vanishing line equation and the known geometric size of the target, and the optimal value of the relative pose between the camera and the calibration target is obtained by calculating the minimum value of the reprojection error function.

本发明的一种分布式靶标的全局标定方法，通过使用辅助像机拍摄相邻靶标图像得到相邻靶标间的坐标变换矩阵，通过增量坐标变换方法求得各标定靶标相对于参考靶标的相对位姿初始值；通过求取基于闭合图像序列的重投影误差函数的最小值，得到各靶标相对于参考靶标的变换矩阵的最优值。在完成像机与标定靶标的相对位姿估计和分布式靶标的全局标定后，通过相应的坐标变换，求得各像机相对于参考像机的变换矩阵的最优值，完成分布式多像机系统的全局标定。A global calibration method for distributed targets of the present invention obtains the coordinate transformation matrix between adjacent targets by using an auxiliary camera to capture images of adjacent targets, and obtains the relative position of each calibration target relative to the reference target through an incremental coordinate transformation method. The initial value of the pose; by calculating the minimum value of the reprojection error function based on the closed image sequence, the optimal value of the transformation matrix of each target relative to the reference target is obtained. After completing the relative pose estimation of the camera and the calibration target and the global calibration of the distributed target, the optimal value of the transformation matrix of each camera relative to the reference camera is obtained through the corresponding coordinate transformation, and the distributed multi-image Global calibration of the computer system.

本发明的优点在于：The advantages of the present invention are:

(1)本发明设计的平面标定靶标由两组相互正交的平行线组成，相对于点特征，直线特征能更好克服图像噪声的影响；(1) The planar calibration target designed by the present invention is composed of two groups of mutually orthogonal parallel lines. Compared with point features, straight line features can better overcome the influence of image noise;

(2)本发明将所设计的靶标用于像机标定中，本发明根据灭线方程推导姿态角信息，结合平行线的已知长度确定平移向量；(2) The present invention uses the designed target in the camera calibration, the present invention deduces the attitude angle information according to the disappearing line equation, and determines the translation vector in conjunction with the known length of the parallel line;

(3)本发明使用辅助像机获取相邻靶标的闭合图像序列，既适用于存在重叠视场的分布式多像机系统的标定，也适用于无重叠视场的分布式多像机系统的标定；(3) The present invention uses an auxiliary camera to acquire closed image sequences of adjacent targets, which is applicable to the calibration of a distributed multi-camera system with overlapping fields of view, and is also applicable to the calibration of a distributed multi-camera system without overlapping fields of view calibration;

(4)本发明为了校正由于多次坐标变换带来的积累误差，根据闭合图像序列约束使得重投影误差函数最小，从而求得各靶标相对于参考靶标的变换矩阵的最优值。(4) In order to correct the accumulated errors caused by multiple coordinate transformations, the present invention minimizes the reprojection error function according to the constraints of the closed image sequence, thereby obtaining the optimal value of the transformation matrix of each target relative to the reference target.

附图说明Description of drawings

图1.分布式像机系统的全局标定方法流程图；Figure 1. The flow chart of the global calibration method of the distributed camera system;

图2.标定靶标俯视图；Figure 2. Top view of the calibration target;

图3.分布式像机系统的全局标定示意图；Figure 3. Schematic diagram of the global calibration of the distributed camera system;

图4.使用辅助像机拍摄相邻靶标(i,j)示意图；Figure 4. Schematic diagram of shooting adjacent targets (i, j) with auxiliary cameras;

图5.平面靶标的灭点和灭线示意图；Figure 5. Schematic diagram of the vanishing point and vanishing line of the planar target;

图6.全局标定结果误差结果图。Figure 6. Global calibration result error graph.

具体实施方式detailed description

下面将结合附图和实施例对本发明作进一步的详细说明。The present invention will be further described in detail with reference to the accompanying drawings and embodiments.

设分布式多像机系统由M个像机构成，C_kCF(1≤k≤M)和A_iCF(1≤i≤M)分别表示第k个像机的像机坐标系和拍摄相邻靶标(i,j)的辅助像机的像机坐标系；I_kCF(1≤k≤M)表示第k个像机的图像像素坐标系，I_kCF的坐标原点位于像平面的中心。Assuming that the distributed multi-camera system is composed of M cameras, C _k CF (1≤k≤M) and A _i CF (1≤i≤M) represent the camera coordinate system and the camera coordinate system of the kth camera respectively. The camera coordinate system of the auxiliary camera adjacent to the target (i, j); I _k CF (1≤k≤M) represents the image pixel coordinate system of the kth camera, and the coordinate origin of I _k CF is located at the center of the image plane .

如图2所示，靶标由两组相互正交且长度已知的平行线组成，平行线长度为L₁，间距为L₂；分别表示靶标k的第m个角点和第m条特征直线。T_kCF(1≤k≤M)表示靶标k的坐标系，o_t为坐标系原点，x_t,z_t分别表示x轴和z轴。ECF表示地面坐标系，ECF的原点固联于地面，为北东地坐标系。As shown in Figure 2, the target consists of two groups of parallel lines that are orthogonal to each other and whose length is known. The length of the parallel lines is L ₁ and the distance is L ₂ ; respectively represent the mth corner point and the mth feature line of the target k. T _k CF (1≤k≤M) represents the coordinate system of target k, o _t is the origin of the coordinate system, x _t , z _t represent the x-axis and z-axis respectively. ECF represents the ground coordinate system, and the origin of ECF is fixed on the ground, which is the northeast ground coordinate system.

测量模型measurement model

设p＝[u,v]^T和P＝[X,Y,Z]^T分别表示二维像平面和三维空间内的点，u和v为图像坐标系内的像素坐标，X,Y,Z表示三维坐标，分别表示对应的齐次坐标，靶标坐标系TCF内的点向像机像平面的投影可表示为：Let p=[u,v] ^T and P=[X,Y,Z] ^T represent points in the two-dimensional image plane and three-dimensional space respectively, u and v are the pixel coordinates in the image coordinate system, X, Y, Z represent three-dimensional coordinates, respectively represent the corresponding homogeneous coordinates, The projection of a point in the target coordinate system TCF to the camera image plane can be expressed as:

$s the s \overset{~ ~}{p p} = = [[K K | | 00_{33 \times \times 11}]] T T \overset{~ ~}{P P},, K K = = [\begin{matrix} {f f}_{x x} & 00 & {u u}_{00} \\ 00 & {f f}_{y the y} & {v v}_{00} \\ 00 & 00 & 11 \end{matrix}],, T T = = [\begin{matrix} {R R}_{33 \times \times 33} & {t t}_{33 \times \times 11} \\ 00_{11 \times \times 33} & 11 \end{matrix}],, R R = = [\begin{matrix} {R R}_{1111} & {R R}_{1212} & {R R}_{1313} \\ {R R}_{21 twenty one} & {R R}_{22 twenty two} & {R R}_{23 twenty three} \\ {R R}_{3131} & {R R}_{3232} & {R R}_{3333} \end{matrix}] - - - - - - ((11))$

其中：s表示尺度因子，K为内参数矩阵，f_x和f_y为等效焦距，(u₀,v₀)为主点坐标。T表示从标靶坐标系到像机坐标系的坐标变换矩阵，R是3×3维的旋转矩阵，t是3×1维的平移向量。旋转矩阵R可用欧拉角(Y-X-Z)：偏航角俯仰角θ和滚转角φ来表示：Where: s represents the scale factor, K is the internal parameter matrix, f _x and f _y are the equivalent focal lengths, and (u ₀ , v ₀ ) are the principal point coordinates. T represents the coordinate transformation matrix from the target coordinate system to the camera coordinate system, R is a 3×3 dimensional rotation matrix, and t is a 3×1 dimensional translation vector. Rotation matrix R available Euler angles (YXZ): yaw angle Pitch angle θ and roll angle φ to represent:

在本发明中，坐标转换矩阵的定义如下：In the present invention, the coordinate transformation matrix is defined as follows:

表1.坐标转换矩阵的定义Table 1. Definition of coordinate transformation matrix

本发明提出的全局标定方法如图3所示，在本发明中，选取像机1作为参考像机，选取靶标1作为参考靶标，本发明提出的一种分布式多像机系统的全局标定方法的具体实施步骤如下：The global calibration method proposed by the present invention is shown in Figure 3. In the present invention, camera 1 is selected as the reference camera, and target 1 is selected as the reference target. A global calibration method for a distributed multi-camera system proposed by the present invention The specific implementation steps are as follows:

步骤一、像机与标定靶标的相对位姿估计；Step 1. Relative pose estimation between the camera and the calibration target;

使用由两组相互正交平行直线组成的二维标定靶标，基于灭点和灭线估计像机与靶标的相对位置姿态(位姿)，通过灭线方程和已知的靶标几何尺寸得到相对位姿初始值，通过求取重投影误差函数最小值得到像机与标定靶标的相对位姿的最优值；Using a two-dimensional calibration target composed of two sets of mutually orthogonal parallel lines, the relative position and attitude (pose) of the camera and the target is estimated based on the vanishing point and the vanishing line, and the relative position is obtained by the vanishing line equation and the known target geometry. The initial value of the pose, the optimal value of the relative pose between the camera and the calibration target is obtained by finding the minimum value of the reprojection error function;

步骤1.1、分别完成各像机的内参数标定，将像机内参数视为固定常量，且像机的姿态在标定过程中保持不变；Step 1.1. Complete the internal parameter calibration of each camera respectively, regard the internal parameters of the camera as fixed constants, and the attitude of the camera remains unchanged during the calibration process;

步骤1.2、将靶标放置在各像机的视场内，靶标的对称轴指向其所对应的像机，I_k表示被像机k所拍摄的靶标k的图像，从而得到图像序列I＝{I_k|1≤k≤M}；Step 1.2, place the target in the field of view of each camera, the symmetry axis of the target points to its corresponding camera, I _k represents the image of the target k captured by the camera k, thereby obtaining the image sequence I={I _k |1≤k≤M};

步骤1.3、根据图像序列I计算变换矩阵具体为：Step 1.3, calculate transformation matrix according to image sequence I Specifically:

步骤1.3.1、获取灭线方程Step 1.3.1, Obtain the line-extinguishing equation

如图5所示，两组平行线在像平面上分别会聚于灭点v₁和v₂，过灭点v₁,v₂的直线即是灭线。在靶标坐标系中，两条相互不平行的直线方程为：As shown in Figure 5, two sets of parallel lines converge at the vanishing points v ₁ and v ₂ respectively on the image plane, and the straight line passing through the vanishing points v ₁ and v ₂ is the vanishing line. In the target coordinate system, the equations of two non-parallel straight lines are:

a_ix+c_iz+d_i＝0,y＝0,(i＝1,2)(3)a _i x+c _i z+d _i =0,y=0,(i=1,2)(3)

其中：a₁c₂-a₂c₁≠0，a_i,c_i,d_i为直线方程的相关系数。Among them: a ₁ c ₂ -a ₂ c ₁ ≠0, a _i , c _i , d _i are the correlation coefficients of the straight line equation.

V₁和V₂分别表示两条直线上的无穷远点，有：V ₁ and V ₂ respectively denote the points at infinity on the two straight lines, there are:

${s the s}_{i i} {\overset{~ ~}{v v}}_{i i} = = [[K K | | 00_{33 \times \times 11}]] {T T}_{k k}^{t t c c} {\overset{~ ~}{V V}}_{i i},, ((i i = = 11,, 22)) - - - - - - ((44))$

其中：s_i为尺度因子，为第i个灭点在图像坐标系下的齐次坐标，K表示像机k的内参数矩阵。in: s _i is the scale factor, is the homogeneous coordinate of the i-th vanishing point in the image coordinate system, and K represents the internal parameter matrix of camera k.

由式(1)和式(4)，有：From formula (1) and formula (4), we have:

${\overset{~ ~}{v v}}_{i i} = = {[[{f f}_{x x} \frac{- - {c c}_{i i} {R R}_{1111} + + {a a}_{i i} {R R}_{1313}}{- - {c c}_{i i} {R R}_{3131} + + {a a}_{i i} {R R}_{3333}} + + {u u}_{00},, {f f}_{y the y} \frac{- - {c c}_{i i} {R R}_{21 twenty one} + + {a a}_{i i} {R R}_{23 twenty three}}{- - {c c}_{i i} {R R}_{3131} + + {a a}_{i i} {R R}_{3333}} + + {v v}_{00},, 11]]}^{T T} - - - - - - ((55))$

其中，R_ij表示旋转矩阵R的第i行第j列元素。Among them, R _ij represents the i-th row and j-th column element of the rotation matrix R.

灭线l可由计算得到，根据式(5)，有：Off line l can be determined by Calculated, according to formula (5), we have:

$l l = = [\begin{matrix} {f f}_{y the y} (({R R}_{23 twenty three} {R R}_{3131} - - {R R}_{21 twenty one} {R R}_{3333})) \\ {f f}_{x x} (({R R}_{1111} {R R}_{3333} - - {R R}_{1313} {R R}_{3131})) \\ {f f}_{x x} {f f}_{y the y} (({R R}_{21 twenty one} {R R}_{1313} - - {R R}_{1111} {R R}_{23 twenty three})) + + {f f}_{x x} {v v}_{00} (({R R}_{1313} {R R}_{3131} - - {R R}_{1111} {R R}_{3333})) + + {f f}_{y the y} {u u}_{00} (({R R}_{21 twenty one} {R R}_{3333} - - {R R}_{23 twenty three} {R R}_{3131})) \end{matrix}] - - - - - - ((66))$

根据式(2)和式(6)，灭线方程可表示为：According to formula (2) and formula (6), the line-extinguishing equation can be expressed as:

$\frac{s the s i i n no φ φ}{{f f}_{x x}} ((u u - - {u u}_{00})) + + \frac{c c o o s the s φ φ}{{f f}_{y the y}} ((v v - - {v v}_{00})) - - t t a a n no θ θ = = 00 - - - - - - ((77))$

步骤1.3.2、获取旋转矩阵Step 1.3.2, get the rotation matrix

根据特征直线上的特征点，使用最小二乘法得到灭线方程：According to the feature points on the feature line, use the least square method to get the line disappearing equation:

$\overset{~ ~}{a a} u u + + \overset{~ ~}{b b} v v + + \overset{~ ~}{c c} = = 00 - - - - - - ((88))$

其中，为所得的直线方程的系数。in, are the coefficients of the resulting straight line equation.

根据式(7)和式(8)，可得到滚转角φ和俯仰角θ：According to formula (7) and formula (8), the roll angle φ and pitch angle θ can be obtained:

$\{\begin{matrix} φ φ = = {tan the tan}^{- - 11} [[{f f}_{x x} \overset{~ ~}{a a} / / (({f f}_{y the y} \overset{~ ~}{b b}))]] \\ θ θ = = {tan the tan}^{- - 11} ((- - \frac{\overset{~ ~}{c c}}{\overset{~ ~}{a a}} \frac{s the s i i n no φ φ}{{f f}_{x x}} - - \frac{s the s i i n no φ φ}{{f f}_{x x}} {u u}_{00} - - \frac{c c o o s the s φ φ}{{f f}_{y the y}} {v v}_{00})) \end{matrix} - - - - - - ((99))$

灭点坐标与平行线在像机坐标系中的方向向量有如下关系：The coordinates of the vanishing point and the direction vector of the parallel line in the camera coordinate system have the following relationship:

${\overset{~ ~}{v v}}_{i i} = = {Kd k}_{i i},, ((i i = = 11,, 22)) - - - - - - ((1010))$

其中，d_i是直线在C_kCF中的3×1维方向向量。Among them, d _i is the 3×1-dimensional direction vector of the line in C _k CF.

和分别与T_kCF的z轴和x轴重合，有： and Coinciding with the z-axis and x-axis of T _k CF respectively, there are:

${{\begin{matrix} {d d}_{11} / / || || {d d}_{11} || || = = R R \cdot &Center Dot; {[\begin{matrix} 00 & 00 & 11 \end{matrix}]}^{T T} \\ {d d}_{22} / / || || {d d}_{22} || || = = R R \cdot &Center Dot; {[\begin{matrix} 11 & 00 & 00 \end{matrix}]}^{T T} \end{matrix} - - - - - - ((1111))$

由式(2)和式(11)，可求得和的值，从而求得的旋转矩阵R。From formula (2) and formula (11), we can get and value, so that The rotation matrix R of .

步骤1.3.3、获取平移向量Step 1.3.3, get the translation vector

设特征点是靶标平面上的一个虚拟点，由于和在向量d₂上的投影相等，有：feature point is a virtual point on the target plane, since and The projections onto the vector d ₂ are equal, with:

由式(1)和式(12)，有：From formula (1) and formula (12), we have:

$\frac{{z z}_{11}}{{z z}_{22}} = = \frac{{d d}_{22}^{T T} {K K}^{- - 11} {\overset{~ ~}{p p}}_{22}^{k k}}{{d d}_{22}^{T T} {K K}^{- - 11} {\overset{~ ~}{p p}}_{11}^{k k}} - - - - - - ((1313))$

其中：z₁和z₂分别是点和在C_kCF的z轴坐标。Where: z ₁ and z ₂ are points respectively and The z-axis coordinate of C _k CF.

由于的长度已知，有：because The length of is known, there are:

$|| || {z z}_{11} {K K}^{- - 11} {\overset{~ ~}{p p}}_{11}^{k k} - - {z z}_{22} {K K}^{- - 11} {\overset{~ ~}{p p}}_{22}^{k k} || || = = {L L}_{11} - - - - - - ((1414))$

由式(13)和式(14)，求得z₁和z₂，从而得到在C_kCF中的位置坐标；而是T_kCF的坐标原点，因此的平移向量t可表示为：According to formula (13) and formula (14), z ₁ and z ₂ are obtained, so that position coordinates in C _k CF; and is the coordinate origin of T _k CF, so The translation vector t of can be expressed as:

$t t = = {z z}_{11} {K K}^{- - 11} {\overset{~ ~}{p p}}_{11}^{k k} - - - - - - ((1515))$

步骤1.3.4、非线性优化Step 1.3.4, nonlinear optimization

表示靶标k的第m个角点在T_kCF中的齐次坐标，其在图像I_k中的对应其次坐标是1≤m≤6，有： Indicates the homogeneous coordinates of the mth corner point of the target k in T _k CF, and its corresponding secondary coordinates in the image I _k are 1≤m≤6, there are:

${s the s}_{m m}^{k k} {\overset{~ ~}{p p}}_{m m}^{k k} = = [[K K | | 00_{33 \times \times 11}]] {T T}_{k k}^{t t c c} {\overset{~ ~}{p p}}_{m m}^{k k} - - - - - - ((1616))$

其中：为尺度因子。in: is the scale factor.

设图像点受到独立且相同分布的高斯噪声的干扰，通过使得特征直线与重投影点的距离的平方和最小得到最大似然估计，使用Levenberg-Marquardt方法求得函数的最小值，从而得到变换矩阵的最优值：Assuming that the image points are interfered by independent and identically distributed Gaussian noise, the maximum likelihood estimation is obtained by minimizing the sum of the squares of the distances between the characteristic line and the reprojection point, and the minimum value of the function is obtained by using the Levenberg-Marquardt method, thereby obtaining the transformation matrix The optimal value of :

$f f ((Ω Ω)) = = {Σ Σ}_{m m = = 11}^{66} [[{d d}^{22} (({\overset{~ ~}{p p}}_{m m}^{k k},, {l l}_{m m}^{k k})) + + {d d}^{22} (({\overset{~ ~}{p p}}_{m m}^{k k},, {l l}_{n no}^{k k}))]] - - - - - - ((1717))$

$n no = = \{\begin{matrix} m m - - 11,, & i i f f & m m &GreaterEqual; &Greater Equal; 22 \\ 66,, & i i f f & m m = = 11 \end{matrix} - - - - - - ((1818))$

其中：函数自变量空间分别表示图像I_k中靶标k的第m条和第n条直线，d(·)表示点到直线的距离。where: function argument space represent the mth and nth straight lines of the target k in the image I _k respectively, and d(·) represents the distance from the point to the straight line.

步骤二、分布式靶标的全局标定；Step 2. Global calibration of distributed targets;

使用辅助像机拍摄相邻靶标图像得到相邻靶标间的坐标变换矩阵，通过增量坐标变换方法求得各标定靶标相对于参考靶标的相对位姿初始值，通过求取基于闭合图像序列的重投影误差函数的最小值，得到各靶标相对于参考靶标的变换矩阵的最优值；The coordinate transformation matrix between adjacent targets is obtained by using the auxiliary camera to capture the images of adjacent targets, and the initial value of the relative pose of each calibration target relative to the reference target is obtained by the incremental coordinate transformation method. The minimum value of the projection error function is used to obtain the optimal value of the transformation matrix of each target relative to the reference target;

步骤2.1、如图4所示，由辅助像机得到闭合图像序列其中靶标(i,j)在中可见，满足：Step 2.1, as shown in Figure 4, the closed image sequence is obtained by the auxiliary camera where target (i,j) is in It can be seen that, satisfying:

步骤2.2由图4所示，分别表示从靶标i和靶标j到辅助像机坐标系的变换矩阵。的初始值可由与步骤1.3中相同的方法得到，从而得到 Step 2.2 is shown in Figure 4, Denote the transformation matrices from target i and target j to the auxiliary camera coordinate system, respectively. The initial value of can be obtained by the same method as in step 1.3, so that

${T T}_{i i j j}^{t t t t} = = {(({T T}_{j j i i}^{t t a a}))}^{- - 11} {T T}_{i i i i}^{t t a a} - - - - - - ((2020))$

步骤2.3、采用增量坐标变换方法计算变换矩阵的初始值，随后对其进行优化。Step 2.3, using the incremental coordinate transformation method to calculate the transformation matrix The initial value of , which is then optimized.

的初始值分别通过多次坐标变换得到： The initial value of is obtained by multiple coordinate transformations respectively:

根据成像模型，有：According to the imaging model, there are:

$\{\begin{matrix} {s the s}_{m m}^{i i i i} {\overset{~ ~}{p p}}_{m m}^{i i i i} = = [[K K | | 00_{33 \times \times 11}]] {T T}_{i i 22}^{t t a a} {(({T T}_{j j 11}^{t t t t}))}^{- - 11} {T T}_{i i 11}^{t t t t} {\overset{~ ~}{P P}}_{m m}^{i i} \\ {s the s}_{m m}^{j j i i} {\overset{~ ~}{p p}}_{m m}^{j j i i} = = [[K K | | 00_{33 \times \times 11}]] {T T}_{i i 11}^{t t a a} {(({T T}_{i i 11}^{t t t t}))}^{- - 11} {T T}_{j j 11}^{t t t t} {\overset{~ ~}{P P}}_{m m}^{j j} \end{matrix} - - - - - - ((22 twenty two))$

其中：和分别表示靶标i和靶标j的第m个角点在上的图像坐标，为尺度因子。in: and Respectively represent the mth corner of target i and target j in image coordinates on the is the scale factor.

使用Levenberg-Marquardt方法求取重投影误差函数的最小值，从而求得的最优值：Use the Levenberg-Marquardt method to find the minimum value of the reprojection error function, thus obtaining The optimal value of :

$f f ((Ω Ω)) = = {Σ Σ}_{i i = = 11}^{M m} {Σ Σ}_{m m = = 11}^{66} [[{d d}^{22} (({\overset{~ ~}{p p}}_{m m}^{i i i i},, {l l}_{m m}^{i i i i})) + + {d d}^{22} (({\overset{~ ~}{p p}}_{m m}^{i i i i},, {l l}_{n no}^{i i i i})) + + {d d}^{22} (({\overset{~ ~}{p p}}_{m m}^{j j i i},, {l l}_{m m}^{j j i i})) + + {d d}^{22} (({\overset{~ ~}{p p}}_{m m}^{j j i i},, {l l}_{n no}^{j j i i}))]] - - - - - - ((23 twenty three))$

其中：自变量空间为四维单位矩阵，迭代初始值由式(20)和式(21)提供。where: the independent variable space is a four-dimensional unit matrix, and the initial value of iteration is provided by formula (20) and formula (21).

步骤三、计算变换矩阵完成全局标定。Step 3. Calculate the transformation matrix Complete global calibration.

将靶标作为媒介，得到的优化值：Using the target as the medium, we get The optimal value for :

${T T}_{k k 11}^{c c c c} = = {T T}_{11}^{t t c c} {T T}_{k k 11}^{t t t t} {(({T T}_{k k}^{t t c c}))}^{- - 11},, ((22 \leq \leq k k \leq \leq M m)) - - - - - - ((24 twenty four))$

得到各像机相对于参考像机的变换矩阵后，即完成了分布式多像机系统的标定。After obtaining the transformation matrix of each camera relative to the reference camera, the calibration of the distributed multi-camera system is completed.

实施例：Example:

在本实施例子中，分布式多像机系统由8台像机组成，像机在ECF中的位置坐标和与ECF的欧拉角如表1所示。靶标尺寸L₁＝500mm,L₂＝200mm，通过计算得到的误差来评估本发明提出的全局标定方法的精度。In this implementation example, the distributed multi-camera system consists of 8 cameras, and the position coordinates of the cameras in the ECF and the Euler angles with the ECF are shown in Table 1. Target size L ₁ =500mm, L ₂ =200mm, obtained by calculation The error is used to evaluate the accuracy of the global calibration method proposed in the present invention.

表1像机位置与姿态Table 1 Camera position and attitude

在对特征点坐标加入均值为0，标准差为0.2像素高斯噪声，进行100次独立实验。Add Gaussian noise with a mean value of 0 and a standard deviation of 0.2 pixels to the coordinates of feature points, and conduct 100 independent experiments.

为了说明所提方法的效果，分别采用两种方法计算标定误差：增量标定方法和全局标定方法。两种方法的唯一区别是是否进行式(23)所表示的靶标全局非线性优化。增量标定方法的使用式(21)直接求得而不进行全局优化，全局标定方法即是本发明所提出的方法。In order to illustrate the effect of the proposed method, two methods are used to calculate the calibration error: the incremental calibration method and the global calibration method. The only difference between the two methods is whether to perform the global nonlinear optimization of the target represented by formula (23). Incremental Calibration Method Use formula (21) to obtain directly without global optimization, and the global calibration method is the method proposed by the present invention.

从图6中可以看出，旋转矩阵误差和平移向量误差随着坐标变换次数的增多而逐渐积累，并在像机5处达到最大值，这是因为像机5与参考像机(像机1)的最远。而本发明所提出的全局标定方法能有效减小由于增量坐标变换而带来的积累误差。It can be seen from Figure 6 that the error of the rotation matrix and the error of the translation vector gradually accumulate with the increase of the number of coordinate transformations, and reach the maximum at camera 5, because the difference between camera 5 and the reference camera (camera 1 ) is the farthest. However, the global calibration method proposed by the present invention can effectively reduce the accumulated error caused by the incremental coordinate transformation.

Claims

1. A global calibration method for a distributed multi-camera system, wherein:

Assuming that the distributed multi-camera system is composed of M cameras, C _k CF and A _i CF represent the camera coordinate system of the k-th camera and the camera of the auxiliary camera that shoots the adjacent target (i, j) respectively Coordinate system, 1≤k≤M, 1≤i≤M; I _k CF represents the image pixel coordinate system of the kth camera, and the coordinate origin of I _k CF is located at the center of the image plane;

Assume that the target is composed of two sets of mutually orthogonal parallel lines with known lengths, the length of the parallel lines is L ₁ , and the distance is L ₂ ; respectively represent the mth corner point and the mth characteristic line of target k, 1≤m≤6; T _k CF represents the coordinate system of target k, o _t is the origin of the coordinate system, x _t , z _t represent the x-axis and z-axis; ECF represents the ground coordinate system, and the origin of ECF is fixed on the ground, which is the north-east coordinate system;

Let p=[u,v] ^T and P=[X,Y,Z] ^T represent points in the two-dimensional image plane and three-dimensional space respectively, u and v are the pixel coordinates in the image coordinate system, X, Y, Z represent three-dimensional coordinates, respectively represent the corresponding homogeneous coordinates, The projection of a point in the target coordinate system TCF to the camera image plane is expressed as:

s the s \overset{~ ~}{p p} = = [[K K | | 00_{33 \times \times 11}]] T T \overset{~ ~}{P P},, K K = = [\begin{matrix} {f f}_{x x} & 00 & {u u}_{00} \\ 00 & {f f}_{y the y} & {v v}_{00} \\ 00 & 00 & 11 \end{matrix}],, T T = = [\begin{matrix} {R R}_{33 \times \times 33} & {t t}_{33 \times \times 11} \\ 00_{11 \times \times 33} & 11 \end{matrix}],, R R = = [\begin{matrix} {R R}_{1111} & {R R}_{1212} & {R R}_{1313} \\ {R R}_{21 twenty one} & {R R}_{22 twenty two} & {R R}_{23 twenty three} \\ {R R}_{3131} & {R R}_{3232} & {R R}_{3333} \end{matrix}] - - - - - - ((11))

Among them: s represents the scale factor, K is the internal parameter matrix, f _x and f _y are the equivalent focal length, (u ₀ , v ₀ ) is the principal point coordinates; T represents the coordinate transformation from the target coordinate system to the camera coordinate system Matrix, R is a 3×3-dimensional rotation matrix, t is a 3×1-dimensional translation vector; the rotation matrix R can use Euler angles (YXZ): yaw angle Pitch angle θ and roll angle φ to represent:

Assume Represents the coordinate transformation matrix from T _k CF to C _k CF, Represents the coordinate transformation matrix from T _i CF to A _i CF, Represents the coordinate transformation matrix from T _j CF to A _i CF, Represents the coordinate transformation matrix from T _i CF to T _j CF, Represents the coordinate transformation matrix from C _i CF to C _j CF;

Selecting camera 1 as a reference camera, selecting target 1 as a reference target, characterized in that, a global calibration method for a distributed multi-camera system includes the following steps:

Step 1. Relative pose estimation between the camera and the calibration target;

Using a two-dimensional calibration target composed of two sets of mutually orthogonal parallel lines, the relative position and attitude of the camera and the target are estimated based on the vanishing point and the vanishing line, and the initial value of the relative pose is obtained through the vanishing line equation and the known target geometry. The optimal value of the relative pose between the camera and the calibration target is obtained by calculating the minimum value of the reprojection error function;

Step 2. Global calibration of distributed targets;

The coordinate transformation matrix between adjacent targets is obtained by using the auxiliary camera to capture the images of adjacent targets, and the initial value of the relative pose of each calibration target relative to the reference target is obtained by the incremental coordinate transformation method. The minimum value of the projection error function is used to obtain the optimal value of the transformation matrix of each target relative to the reference target;

Step 3: Calculate the transformation matrix and complete the global calibration.

2. the global calibration method of a kind of distributed multi-camera system according to claim 1, described step one specifically comprises:

Step 1.1. Complete the internal parameter calibration of each camera respectively, regard the internal parameters of the camera as fixed constants, and the attitude of the camera remains unchanged during the calibration process;

Step 1.2, place the target in the field of view of each camera, the symmetry axis of the target points to the corresponding camera, I _k represents the image of the target k captured by the camera k, and the image sequence I={I _k |1≤k≤M};

Step 1.3, calculate transformation matrix according to image sequence I Specifically:

Step 1.3.1, Obtain the line-extinguishing equation

Two sets of parallel lines converge at the vanishing points v ₁ and v ₂ respectively on the image plane, and the straight line passing through the vanishing points v ₁ and v ₂ is the vanishing line; in the target coordinate system, the equations of two non-parallel straight lines are:

a _i x+c _i z+d _i =0,y=0(3)

Among them: a ₁ c ₂ -a ₂ c ₁ ≠0, a _i , c _i , d _i are the correlation coefficients of the straight line equation, i=1,2;

V ₁ and V ₂ respectively denote the points at infinity on the two straight lines, there are:

{s the s}_{i i} {\overset{~ ~}{v v}}_{i i} = = [[K K | | 00_{33 \times \times 11}]] {T T}_{k k}^{t t c c} {\overset{~ ~}{V V}}_{i i} - - - - - - ((44))

in: s _i is the scale factor, is the homogeneous coordinate of the i-th vanishing point in the image coordinate system, and K represents the internal parameter matrix of camera k;

From formula (1) and formula (4), we have:

{\overset{~ ~}{v v}}_{i i} = = {[[{f f}_{x x} \frac{- - {c c}_{i i} {R R}_{1111} + + {a a}_{i i} {R R}_{1313}}{- - {c c}_{i i} {R R}_{3131} + + {a a}_{i i} {R R}_{3333}} + + {u u}_{00},, {f f}_{y the y} \frac{- - {c c}_{i i} {R R}_{21 twenty one} + + {a a}_{i i} {R R}_{23 twenty three}}{- - {c c}_{i i} {R R}_{3131} + + {a a}_{i i} {R R}_{3333}} + + {v v}_{00},, 11]]}^{T T} - - - - - - ((55))

Among them, R _ij represents the i-th row and j-th column element of the rotation matrix R;

off line l by Calculated, according to formula (5), we have:

l l = = [\begin{matrix} {f f}_{y the y} (({R R}_{23 twenty three} {R R}_{3131} - - {R R}_{21 twenty one} {R R}_{3333})) \\ {f f}_{x x} (({R R}_{1111} {R R}_{3333} - - {R R}_{1313} {R R}_{3131})) \\ {f f}_{x x} {f f}_{y the y} (({R R}_{21 twenty one} {R R}_{1313} - - {R R}_{1111} {R R}_{23 twenty three})) + + {f f}_{x x} {v v}_{00} (({R R}_{1313} {R R}_{3131} - - {R R}_{1111} {R R}_{3333})) + + {f f}_{y the y} {u u}_{00} (({R R}_{21 twenty one} {R R}_{3333} - - {R R}_{23 twenty three} {R R}_{3131})) \end{matrix}] - - - - - - ((66))

According to formula (2) and formula (6), the line-extinguishing equation is expressed as:

\frac{sin sin φ φ}{{f f}_{x x}} ((u u - - {u u}_{00})) + + \frac{c c o o s the s φ φ}{{f f}_{y the y}} ((v v - - {v v}_{00})) - - t t a a n no θ θ = = 00 - - - - - - ((77))

Step 1.3.2, get the rotation matrix

According to the feature points on the feature line, use the least square method to get the line disappearing equation:

\overset{~ ~}{a a} u u + + \overset{~ ~}{b b} v v + + \overset{~ ~}{c c} = = 00 - - - - - - ((88))

in, are the coefficients of the obtained straight line equation;

According to formula (7) and formula (8), the roll angle φ and pitch angle θ are obtained:

\{\begin{matrix} φ φ = = {tan the tan}^{- - 11} [[{f f}_{x x} \overset{~ ~}{a a} / / (({f f}_{y the y} \overset{~ ~}{b b}))]] \\ θ θ = = {tan the tan}^{- - 11} ((- - \frac{\overset{~ ~}{c c}}{\overset{~ ~}{a a}} \frac{s the s i i n no φ φ}{{f f}_{x x}} - - \frac{s the s i i n no φ φ}{{f f}_{x x}} {u u}_{00} - - \frac{c c o o s the s φ φ}{{f f}_{y the y}} {v v}_{00})) \end{matrix} - - - - - - ((99))

The coordinates of the vanishing point and the direction vector of the parallel line in the camera coordinate system have the following relationship:

{\overset{~ ~}{v v}}_{i i} = = {Kd k}_{i i} - - - - - - ((1010))

Among them, d _i is the 3×1-dimensional direction vector of the line in C _k CF;

and Coinciding with the z-axis and x-axis of T _k CF respectively, there are:

\{\begin{matrix} {d d}_{11} / / | | | | {d d}_{11} | | | | = = R R \cdot &Center Dot; {[\begin{matrix} 00 & 00 & 11 \end{matrix}]}^{T T} \\ {d d}_{22} / / | | | | {d d}_{22} | | | | = = R R \cdot &Center Dot; {[\begin{matrix} 11 & 00 & 00 \end{matrix}]}^{T T} \end{matrix} - - - - - - ((1111))

From formula (2) and formula (11), get and value, so that The rotation matrix R;

Step 1.3.3, get the translation vector

feature point is a virtual point on the target plane, since and The projections onto the vector d ₂ are equal, with:

From formula (1) and formula (12), we have:

\frac{{z z}_{11}}{{z z}_{22}} = = \frac{{d d}_{22}^{T T} {K K}^{- - 11} {\overset{~ ~}{p p}}_{22}^{k k}}{{d d}_{22}^{T T} {K K}^{- - 11} {\overset{~ ~}{p p}}_{11}^{k k}} - - - - - - ((1313))

Where: z ₁ and z ₂ are points P ₁ ^k and The z-axis coordinate at C _k CF;

Thanks P ₁ ¹ The length of is known, there are:

| | | | {z z}_{11} {K K}^{- - 11} {\overset{~ ~}{p p}}_{11}^{k k} - - {z z}_{22} {K K}^{- - 11} {\overset{~ ~}{p p}}_{22}^{k k} | | | | = = {L L}_{11} - - - - - - ((1414))

Calculate z ₁ and z ₂ from formula (13) and formula (14), and obtain the position coordinates of P ₁ ^k in C _k CF; P ₁ ^k is the coordinate origin of T _k CF, then The translation vector t is expressed as:

t t = = {z z}_{11} {K K}^{- - 11} {\overset{~ ~}{p p}}_{11}^{k k} - - - - - - ((1515))

Step 1.3.4, nonlinear optimization

Indicates the homogeneous coordinates of the mth corner point of the target k in T _k CF, and its corresponding secondary coordinates in the image I _k are 1≤m≤6, there are:

{s the s}_{m m}^{k k} {\overset{~ ~}{p p}}_{m m}^{k k} = = [[K K | | 00_{33 \times \times 11}]] {T T}_{k k}^{t t c c} {\overset{~ ~}{P P}}_{m m}^{k k} - - - - ((1616))

in: is the scale factor;

Assuming that the image points are interfered by independent and identically distributed Gaussian noise, the maximum likelihood estimation is obtained by minimizing the sum of the squares of the distances between the characteristic line and the reprojection point, and the minimum value of the function is obtained by using the Levenberg-Marquardt method to obtain the transformation matrix The optimal value of :

f f ((Ω Ω)) = = {Σ Σ}_{m m = = 11}^{66} [[{d d}^{22} (({\overset{~ ~}{p p}}_{m m}^{k k},, {l l}_{m m}^{k k})) + + {d d}^{22} (({\overset{~ ~}{p p}}_{m m}^{k k},, {l l}_{n no}^{k k}))]] - - - - - - ((1717))

where: function argument space represent the mth and nth straight lines of the target k in the image I _k respectively, and d(·) represents the distance from the point to the straight line.

3. the global calibration method of a kind of distributed multi-camera system according to claim 1, described step 2 specifically comprises:

Step 2.1, Obtain a closed image sequence from the auxiliary camera where target (i,j) is in It can be seen that, satisfying:

Step 2.2, respectively represent the transformation matrix from target i and target j to the auxiliary camera coordinate system, then for:

{T T}_{i i j j}^{t t t t} = = {(({T T}_{j j i i}^{t t a a}))}^{- - 11} {T T}_{i i i i}^{t t a a} - - - - - - ((2020))

Step 2.3, using the incremental coordinate transformation method to calculate the transformation matrix The initial value of , and then optimize it;

The initial value of is obtained by multiple coordinate transformations respectively:

Among them, 2≤k1≤M, according to the imaging model, there are:

in: and Respectively represent the mth corner of target i and target j in image coordinates on the is the scale factor;

Using the Levenberg-Marquardt method to find the minimum value of the reprojection error function, obtain The optimal value of :

f f ((Ω Ω)) = = {Σ Σ}_{i i = = 11}^{M m} {Σ Σ}_{m m = = 11}^{66} [[{d d}^{22} (({\overset{~ ~}{p p}}_{m m}^{i i i i},, {l l}_{m m}^{i i i i})) + + {d d}^{22} (({\overset{~ ~}{p p}}_{m m}^{i i i i},, {l l}_{n no}^{i i i i})) + + {d d}^{22} (({\overset{~ ~}{p p}}_{m m}^{j j i i},, {l l}_{m m}^{j j i i})) + + {d d}^{22} (({\overset{~ ~}{p p}}_{m m}^{j j i i},, {l l}_{n no}^{j j i i}))]] - - - - - - ((23 twenty three))

where: the independent variable space is a four-dimensional identity matrix.

4. the global calibration method of a kind of distributed multi-camera system according to claim 1, described step 3 is specifically:

Using the target as the medium, we get The optimal value for :

{T T}_{k k 11}^{c c c c} = = {T T}_{11}^{t t c c} {T T}_{k k 11}^{t t t t} {(({T T}_{k k}^{t t c c}))}^{- - 11} - - - - - - ((24 twenty four))

After obtaining the transformation matrix of each camera relative to the reference camera, the calibration of the distributed multi-camera system is completed.