CN102930544B - Parameter calibration system of vehicle-mounted camera - Google Patents

Abstract

Disclosed is a parameter calibration method of a vehicle-mounted camera. The method includes steps of self-making a three-dimensional calibration board, and establishing a world coordinate system on the basis of the self-made three-dimensional calibration board; establishing a camera calibration model on the basis of the three-dimensional calibration board, acquiring world coordinates of chessboard corners of the three-dimensional calibration board, and extracting pixel coordinates of the chessboard corners on the basis of a corner detection algorithm; and using a least square method to calibrate inner and outer parameters of a camera in combination of the corner world coordinates and the pixel coordinates.

Description

A kind of parameter calibration system of vehicle-mounted vidicon

Technical field

The present invention relates to intelligent transportation field, relate more specifically to the demarcation of vehicle-mounted vidicon inside and outside parameter in vehicle DAS (Driver Assistant System).

Background technology

In recent years, intelligent transportation becomes the focus of research, and DAS (Driver Assistant System), as the important component part of intelligent transportation, receives much concern.Wherein, lane detection, vehicle detection and tracking and range finding etc. are the gordian techniquies of DAS (Driver Assistant System).These technology are all to rely on vehicle-mounted photography/videography machine collection traffic environment around, therefore vehicle-mounted vidicon are demarcated and are had very important significance accurately.

Camera marking method is divided into traditional camera marking method and self-calibrating method.Common calibrating method has: Tsai(tradition) and Zhang Zhengyou (between tradition and self calibration) etc.Traditional camera calibration method is applied widely, and stated accuracy is high, but calibration process complexity; Self-calibration method process is simple but precision is on the low side.

Chessboard calibration plate based on two-dimentional is demarcated video camera, and the method for Zhang Zhengyou is very representative.Video camera need to catch 2 above images and solve the internal reference of video camera, and the outer ginseng of corresponding every width image also needs to solve according to internal reference and the picture of having tried to achieve.The demarcation of vehicle-mounted vidicon, in order to facilitate subsequent operation, outside wishing, ginseng is fixed.Gu the subject matter that existing vehicle-mounted vidicon scaling method exists is: demarcate with two-dimentional scaling board the trouble relatively that seems, be unfavorable for determining of world coordinate system, and be unfavorable for detection and range finding etc. below.If demarcate with two-dimentional scaling board, just need two scaling boards, and need to know two world coordinates relations between scaling board; Or take a scaling board to clap two width pictures, and need to know the motion track of scaling board.Homemade stereo calibration plate is actual to be vertically made up of two two-dimentional scaling boards, and angle point world coordinates is in two planes, and all known, easily extraction and the calibrated and calculated of coordinate.

Two dimension scaling board comparison is simple, and also comparative maturity of the algorithm of Zhang Zhengyou, therefore a lot of people has adopted two-dimentional scaling board to carry out the demarcation of vehicle-mounted vidicon.In prior art, common scheme for example: " a kind of simple calibrating method of vehicle-mounted vidicon " (Li Qing etc., " robot " z1 phase in 2003), wherein position to two-dimentional scaling board with aids such as vertical, meter ruler and large set squares, video camera is carried out to the demarcation of inside and outside parameter.The method calibrates after camera parameters, can obtain fixing perspective transformation matrix." the single image camera calibrations based on two orthogonal one-dimensional objects " (Xue Junpeng, Su Xianyu, " Acta Optica ", the 32nd the 1st phase of volume, 0115001--1 page--0115001--7 page, in January, 2012) in, propose to demarcate target based on the T-shape of two orthogonal one-dimensional object compositions, and set up the scaling method of space coordinates, only need to take a width picture and just can calibrate the inside and outside parameter of distortion parameter and the video camera of camera lens.

But the vehicles such as lane detection of the prior art, vehicle detection and tracking and range finding are assisted driving technology, need to clap the picture of getting to vehicle-mounted vidicon carries out perspective transform, therefore needs fixing perspective transformation matrix.With two-dimentional scaling board, vehicle-mounted vidicon is demarcated, be can not get fixing world coordinate system, perspective transformation matrix is unfixing yet, inconvenient follow-up perspective transform.Much utilize aid to determine that the scaling method of world coordinate system is also more loaded down with trivial details, easy not, operation is trouble relatively.

Terminological interpretation:

Video camera internal reference: comprise focal length and the image pixel centre coordinate of video camera, specifically comprise: camera sensor equivalent focal length f in the x and y direction _xand f _y; Image pixel centre coordinate (u ₀, v ₀);

Video camera is joined outward: comprise the installation site of video camera setting angle and the relative world coordinate system of video camera, specifically comprise: the setting angle of video camera: crab angle γ, angle of pitch φ and roll angle ψ; Camera coordinate system ties up to the translational movement in x, y and z direction: t with respect to world coordinates _x, t _yand t _z.

Summary of the invention

For above-mentioned technical matters, the present invention adopts and a kind ofly based on homemade stereo calibration plate, the inside and outside parameter of vehicle-mounted vidicon is demarcated, vehicle-mounted vidicon is demarcated with the stereo calibration plate of homemade two two-dimentional scaling boards composition, set up fixing world coordinate system and perspective transformation matrix, angle point world coordinate system coordinate and image pixel coordinate conveniently obtain, only need bat to get a width picture and just can carry out to vehicle-mounted vidicon the demarcation of inside and outside parameter, facilitate the operations such as follow-up lane detection and vehicle detection, tracking and range finding.

As shown in Figure 1, the invention provides a kind of vehicle-mounted vidicon scaling method based on stereo calibration plate, comprise step: 1, self-control stereo calibration plate, set up camera coordinate system model, set up world coordinate system based on stereo calibration plate, obtain the world coordinates of stereo calibration plate chessboard angle point; 2, set up camera calibration model, extract the pixel coordinate of above-mentioned chessboard angle point based on Corner Detection Algorithm; 3,, in conjunction with angle point world coordinates and pixel coordinate, utilize least square method to demarcate camera interior and exterior parameter.

Brief description of the drawings

Fig. 1 is particular flow sheet of the present invention;

Fig. 2 is for self-control three-dimensional calibration plate and set up world coordinate system;

Fig. 3 is the foundation of camera coordinate system.

Embodiment

Below, by reference to the accompanying drawings, the specific implementation process of the vehicle-mounted vidicon scaling method based on stereo calibration plate provided by the invention is elaborated.

1, self-control is three-dimensional calibrates plate and sets up world coordinate system model

Self-control solid calibration plate as shown in Figure 2, this stereo calibration plate becomes 90 degree angles to form by the black and white chessboard scaling board of two 4*4 sizes, and gridiron pattern size is 150mm*150mm.If the XOY coordinate system that ground scaling board is world coordinate system, the XOZ coordinate system that endways scaling board is world coordinate system, and on two plate boundary lines, the tessellated right corner point of first left black and white is world coordinates initial point O.

2, set up camera calibration model

If P _w(X _w, Y _w, Z _w) be a bit under world coordinate system, the coordinate under its corresponding camera coordinate system is P _c(x _c, y _c, z _c), the coordinate under corresponding image physical coordinates system is (x _u, y _u), the coordinate under corresponding image pixel coordinate system is (u, v), camera coordinate system is as shown in Figure 3.

Under linear pinhole camera modeling, the corresponding relation of four kinds of coordinate systems is as follows:

(1) world coordinates is tied to camera coordinate system:

$\left[\begin{array}{c}{x}_c\\ y_c\\ z_c\\ 1\end{array}\right]=\left[\begin{array}{cc}R& T\\ 0& 1\end{array}\right]\left[\begin{array}{c}{X}_W\\ Y_W\\ Z_W\\ 1\end{array}\right]---\left(1\right)$

In formula (1): $R=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]$ For rotation matrix, and be orthogonal matrix, by crab angle γ, angle of pitch φ, roll angle ψ, expression formula is as follows

$R=\left[\begin{array}{ccc}\cos \mathrm{\ψ}\cos \mathrm{\γ}& \sin \mathrm{\ψ}\cos \mathrm{\γ}& -\sin \mathrm{\γ}\\ -\sin \mathrm{\ψ}\cos \mathrm{\φ}+\cos \mathrm{\ψ}\sin \mathrm{\γ}\sin \mathrm{\φ}& \cos \mathrm{\ψ}\cos \mathrm{\φ}+\sin \mathrm{\ψ}\sin \mathrm{\γ}\sin \mathrm{\φ}& \cos \mathrm{\γ}\sin \mathrm{\φ}\\ \sin \mathrm{\ψ}\sin \mathrm{\φ}+\cos \mathrm{\ψ}\sin \mathrm{\γ}\cos \mathrm{\φ}& -\cos \mathrm{\ψ}\sin \mathrm{\φ}+\sin \mathrm{\ψ}\sin \mathrm{\γ}\cos \mathrm{\φ}& \cos \mathrm{\γ}\cos \mathrm{\φ}\end{array}\right];$ $T=\left[\begin{array}{c}{t}_x\\ t_y\\ t_z\end{array}\right]$ For camera coordinate system is with respect to the translation vector of world coordinate system.

(2) camera coordinates is tied to desirable non-fault image physical coordinates system:

$x_u=f\frac{x_c}{y_c}$

$y_u=f\frac{y_c}{z_c}$

The focal length that wherein f is camera lens.

(3) image physical coordinates is tied to image pixel coordinate system:

$u=\frac{x_u}{\mathrm{dx}}+u_0$

$v=\frac{y_u}{\mathrm{dy}}+v_0$

Wherein (u ₀, v ₀) be the coordinate at image pixel center, dx, dy is respectively the pixel cell distance in camera sensor x and y direction.

To sum up, object point P in camera coordinate system _ctransformational relation to image pixel coordinate points (u, v) is:

$\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_x& 0& u_0\\ 0& f_y& v_0\\ 0& 0& 0\end{array}\right]\left[\begin{array}{c}{x}_c/z_c\\ y_c/z_c\\ 1\end{array}\right]---\left(2\right)$

In formula (2): f _x=f/dx, f _y=f/dy, is respectively camera sensor equivalent focal length in the x and y direction.Order $A=\left[\begin{array}{ccc}{f}_x& o& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right]$ For the internal reference matrix of video camera.

World coordinate point P _wtransformational relation to image pixel coordinate points (u, v) is:

$z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=A\left[\begin{array}{cc}R& T\\ 0& 1\end{array}\right]\left[\begin{array}{c}{X}_W\\ Y_W\\ Z_W\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_x& o& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_x\\ r_{21}& r_{22}& r_{23}& t_y\\ r_{31}& r_{32}& r_{33}& t_z\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_W\\ Y_W\\ Z_W\\ 1\end{array}\right]---\left(3\right)$

3, solve in conjunction with least square method

Order $M=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\\ m_{31}& m_{32}& m_{33}& m_{34}\end{array}\right]=A\left[\begin{array}{cc}R& T\\ 0& 1\end{array}\right],$

$z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=M\left[\begin{array}{c}{X}_W\\ Y_W\\ Z_W\\ 1\end{array}\right]---\left(4\right)$

The projection matrix that in formula (4), M is 3*4, solves to it inside and outside parameter that obtains video camera.

In above formula, comprise following two about m _ijlinear equation:

X _Wm ₁₁+Y _Wm ₁₂+Z _Wm ₁₃+m ₁₄-uX _Wm ₃₁-uY _Wm ₃₂-uZ _Wm ₃₃=um ₃₄

X _Wm ₂₁+Y _Wm ₂₂+Z _Wm ₂₃+m ₂₄-vX _Wm ₃₁-vY _Wm ₃₂-vZ _Wm ₃₃=vm ₃₄

If there be n world coordinates point coordinate P _wkand corresponding image pixel coordinate (u _k, v _k), wherein, k=1 ... n, can obtain a following 2n linear equation:

$\left[\begin{array}{ccccccccccc}{X}_{W1}& Y_{W1}& Z_{W1}& 1& 0& 0& 0& 0& -X_{W1}{u}_1& -Y_{W1}{u}_1& -Y_{W1}{u}_1\\ 0& 0& 0& 0& X_{W1}& Y_{W1}& Z_{W1}& 1& -X_{W1}{v}_1& {-Y}_{W1}{v}_1& {-Z}_{W1}{v}_1\\ \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·\\ \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·\\ X_{\mathrm{Wn}}& Y_{\mathrm{Wn}}& Z_{\mathrm{Wn}}& 1& 0& 0& 0& 0& {-X}_{\mathrm{Wn}}{u}_n& {-Y}_{\mathrm{Wn}}{u}_n& {-Z}_{\mathrm{Wn}}{u}_n\\ 0& 0& 0& 0& X_{\mathrm{Wn}}& Y_{\mathrm{Wn}}& Z_{\mathrm{Wn}}& 1& {-X}_{\mathrm{Wn}}{v}_n& {-Y}_{\mathrm{Wn}}{v}_n& -Z_{\mathrm{Wn}}{v}_n\end{array}\right]\×\left[\begin{array}{c}{m}_{11}\\ m_{12}\\ m_{13}\\ m_{14}\\ m_{21}\\ m_{22}\\ m_{23}\\ m_{24}\\ m_{31}\\ m_{32}\\ m_{33}\end{array}\right]=\left[\begin{array}{c}{u}_1{m}_{34}\\ v_1{m}_{34}\\ \·\·\·\\ \·\·\·\\ u_n{m}_{34}\\ v_n{m}_{34}\end{array}\right]---\left(5\right)$

Without loss of generality, make m ₃₄=1,2n linear equation of other 11 elements of Metzler matrix, can be abbreviated as:

Km=U （6）

In formula (6), m is 11 dimensional vectors of 11 element compositions, and K is the 11*2n dimension matrix of world coordinates point coordinate and image pixel point coordinate composition, and U is the 2n dimensional vector of image pixel point coordinate composition.K and U are known vector, and in the time of 2n>11, available least square method solves m:

m=K ^-1U （7）

In formula (7): K ^-1for the inverse matrix of K.

Ask for the inside and outside parameter of video camera below by m.

If R=[r ₁r ₂r ₃] ^t, wherein r ₁=[r ₁₁r ₁₂r ₁₃], r ₂=[r ₂₁r ₂₂r ₂₃], r ₃=[r ₃₁r ₃₂r ₃₃].

:

$m_{34}\left[\begin{array}{cc}{m}_1^{\′}& m_{14}^{\′}\\ m_2^{\′}& m_{24}^{\′}\\ m_3^{\′}& 1\end{array}\right]=\left[\begin{array}{cccc}{f}_x& 0& u_0& 0\\ 0& f_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}{r}_1& t_x\\ r_2& t_y\\ r_3& t_z\\ 0& 1\end{array}\right]=\left[\begin{array}{cc}{f}_x{r}_1+u_0{r}_3& f_x{t}_x+u_0{t}_z\\ f_y{t}_y+v_0{r}_3& f_y{t}_y+v_0{t}_z\\ r_3& t_z\end{array}\right]---\left(8\right)$

In formula (8): m' ₁=[m ₁₁m ₁₂m ₁₃]/m ₃₄, m' ₂=[m ₂₁m ₂₂m ₂₃]/m ₃₄, m' ₃=[m ₃₁m ₃₂m ₃₃]/m ₃₄, m' ₁₄=m ₁₄/ m ₃₄, m' ₂₄=m ₂₄/ m ₃₄.

From above formula, m ₃₄m' ₃=r ₃, be that orthogonal matrix can obtain by R again | r ₃|=1, so:

$m_{34}=\frac{1}{\left|m_3^{\′}\right|}$

Obtain the calibration result of camera parameters according to above-mentioned calculation procedure:

(a) video camera internal reference

Image pixel center horizontal ordinate:

Image pixel center ordinate:

The equivalent focal length of camera sensor in x direction:

The equivalent focal length of camera sensor in y direction:

(b) video camera is joined outward

$r_1=\frac{m_{34}}{f_x}(m_1^{\′}-u_0{m}_3^{\′}),$

$r_2=\frac{m_{34}}{f_x}(m_2^{\′}-v_0{m}_3^{\′}),$

r ₃=m ₃₄m’ ₃，

Crab angle: γ=| arcsin (r ₁₃) |,

The angle of pitch: φ=arcsin (r ₁₂/ cos γ),

Roll angle: ψ=0(is defaulted as 0),

Camera coordinate system ties up to the translational movement in z direction: t with respect to world coordinates _z=m ₃₄,

Camera coordinate system ties up to the translational movement in x direction with respect to world coordinates:

Camera coordinate system ties up to the translational movement in y direction with respect to world coordinates:

The present invention first utilize homemade stereo calibration plate convenient and clear and definite world coordinate system, fixing outer ginseng and perspective transformation matrix is provided, demarcate fixing outer ginseng and perspective transformation matrix is provided to vehicle-mounted vidicon, thereby facilitate the operations such as follow-up lane detection, vehicle detection and tracking and range finding.The stereo calibration plate that wherein adopted is demarcated target compared with the T-shape of two orthogonal one-dimensional objects compositions, the determining of more convenient world coordinate system.

Claims (2)

1. the vehicle-mounted vidicon scaling method based on stereo calibration plate, comprises step: a, self-control stereo calibration plate, set up world coordinate system based on described self-control stereo calibration plate; B, set up camera calibration model: set up camera coordinate system model based on stereo calibration plate, obtain the world coordinates of stereo calibration plate chessboard angle point, and extract the pixel coordinate of above-mentioned chessboard angle point based on Corner Detection Algorithm; C, in conjunction with angle point world coordinates and pixel coordinate, utilize least square method to demarcate camera interior and exterior parameter; Wherein, described self-control stereo calibration plate becomes 90 degree angles to form by the black and white chessboard scaling board of two 4*4 sizes, and gridiron pattern size is 150mm*150mm; Ground scaling board is the XOY coordinate system of world coordinate system, the XOZ coordinate system that endways scaling board is world coordinate system, and on two plate boundary lines, the tessellated right corner point of first left black and white is world coordinates initial point O;

If P _w(X _w, Y _w, Z _w) be a bit under world coordinate system, the coordinate under its corresponding camera coordinate system is P _c(x _c, y _c, z _c), the coordinate under corresponding image physical coordinates system is (x _u, y _u), the coordinate under corresponding image pixel coordinate system is (u, v),

Under linear pinhole camera modeling, be tied to according to world coordinates that camera coordinate system, camera coordinates are tied to image physical coordinates system and image physical coordinates is tied to the transformational relation between image pixel coordinate system, obtain world coordinate point P _w(X _w, Y _w, Z _w) to the following transformational relation of image pixel coordinate points (u, v):

$z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=A\left[\begin{array}{cc}R& T\\ 0& 1\end{array}\right]\left[\begin{array}{c}{X}_W\\ Y_W\\ Z_W\\ 1\end{array}\right]=\left[\begin{array}{ccc}{f}_x& o& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right]\left[\begin{array}{cccc}{r}_{11}& r_{12}& r_{13}& t_x\\ r_{21}& r_{22}& r_{23}& t_y\\ r_{31}& r_{32}& r_{33}& t_z\\ 0& 0& 0& 1\end{array}\right]\left[\begin{array}{c}{X}_W\\ Y_W\\ Z_W\\ 1\end{array}\right]$

Wherein, $A=\left[\begin{array}{ccc}{f}_x& o& u_0\\ 0& f_y& v_0\\ 0& 0& 1\end{array}\right]$ For the internal reference matrix of video camera; $R=\left[\begin{array}{ccc}{r}_{11}& r_{12}& r_{13}\\ r_{21}& r_{22}& r_{23}\\ r_{31}& r_{32}& r_{33}\end{array}\right]$ For rotation matrix, and be orthogonal matrix, wherein, r ₁₁=cos ψ cos γ, r ₁₂=sin ψ cos γ, r ₁₃=-sin γ, r ₂₁=-sin ψ cos φ+cos ψ sin γ sin φ, r ₂₂=cos ψ cos φ+sin ψ sin γ sin φ, r ₂₃=cos γ sin φ, r ₃₁=sin ψ sin φ+cos ψ sin γ cos φ, r ₃₂=-cos ψ sin φ+sin ψ sin γ cos φ, r ₃₃=cos γ cos φ, γ is that crab angle, φ are that the angle of pitch, ψ are roll angle; $T=\left[\begin{array}{c}{t}_x\\ t_y\\ t_z\end{array}\right]$ For camera coordinate system is with respect to the translation vector of world coordinate system, f _x, f _ybe respectively the equivalent focal length in camera sensor x and y direction, (u ₀, v ₀) be the coordinate at image pixel center.

2. according to the process of claim 1 wherein, utilize the step that least square method is demarcated camera interior and exterior parameter to comprise:

Order $M=\left[\begin{array}{cccc}{m}_{11}& m_{12}& m_{13}& m_{14}\\ m_{21}& m_{22}& m_{23}& m_{24}\\ m_{31}& m_{32}& m_{33}& m_{34}\end{array}\right]=A\left[\begin{array}{cc}R& T\\ 0& 1\end{array}\right],$

$z_c\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=M\left[\begin{array}{c}{X}_W\\ Y_W\\ Z_W\\ 1\end{array}\right]---\left(4\right)$

The projection matrix that wherein M is 3*4, solves to it inside and outside parameter that obtains video camera, in formula (4), comprises two about m _ijlinear equation, wherein i=1,2,3; J=1,2,3,4:

X _Wm ₁₁+Y _Wm ₁₂+Z _Wm ₁₃+m ₁₄-uX _Wm ₃₁-uY _Wm ₃₂-uZ _Wm ₃₃=um ₃₄

X _Wm ₂₁+Y _Wm ₂₂+Z _Wm ₂₃+m ₂₄-vX _Wm ₃₁-vY _Wm ₃₂-vZ _Wm ₃₃=vm ₃₄

, for n world coordinates point coordinate P _wk, k=1 ... n, and corresponding image pixel coordinate (u _k, v _k), obtain a following 2n linear equation:

$\left[\begin{array}{ccccccccccc}{X}_{W1}& Y_{W1}& Z_{W1}& 1& 0& 0& 0& 0& -X_{W1}{u}_1& -Y_{W1}{u}_1& -Y_{W1}{u}_1\\ 0& 0& 0& 0& X_{W1}& Y_{W1}& Z_{W1}& 1& -X_{W1}{v}_1& {-Y}_{W1}{v}_1& {-Z}_{W1}{v}_1\\ \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·\\ \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·& \·\·\·\\ X_{\mathrm{Wn}}& Y_{\mathrm{Wn}}& Z_{\mathrm{Wn}}& 1& 0& 0& 0& 0& {-X}_{\mathrm{Wn}}{u}_n& {-Y}_{\mathrm{Wn}}{u}_n& {-Z}_{\mathrm{Wn}}{u}_n\\ 0& 0& 0& 0& X_{\mathrm{Wn}}& Y_{\mathrm{Wn}}& Z_{\mathrm{Wn}}& 1& {-X}_{\mathrm{Wn}}{v}_n& {-Y}_{\mathrm{Wn}}{v}_n& -Z_{\mathrm{Wn}}{v}_n\end{array}\right]\×\left[\begin{array}{c}{m}_{11}\\ m_{12}\\ m_{13}\\ m_{14}\\ m_{21}\\ m_{22}\\ m_{23}\\ m_{24}\\ m_{31}\\ m_{32}\\ m_{33}\end{array}\right]=\left[\begin{array}{c}{u}_1{m}_{34}\\ v_1{m}_{34}\\ \·\·\·\\ \·\·\·\\ u_n{m}_{34}\\ v_n{m}_{34}\end{array}\right]---\left(5\right)$

Make m ₃₄=1,2n linear equation of other 11 elements of Metzler matrix, is abbreviated as:

Km=U （6）

In formula (6), m is 11 dimensional vectors of 11 element compositions, K is the 11*2n dimension matrix of world coordinates point coordinate and image pixel coordinate composition, U is the 2n dimensional vector of image pixel coordinate composition, K and U are known vector, in the time of 2n>11, can enough least square methods solve m:

m=K ^-1U （7）

In formula (7): K ^-1for the inverse matrix of K;

Ask for the inside and outside parameter of video camera below by m;

If R=[r ₁r ₂r ₃] ^t, wherein r ₁=[r ₁₁r ₁₂r ₁₃], r ₂=[r ₂₁r ₂₂r ₂₃], r ₃=[r ₃₁r ₃₂r ₃₃];

:

$m_{34}\left[\begin{array}{cc}{m}_1^{\′}& m_{14}^{\′}\\ m_2^{\′}& m_{24}^{\′}\\ m_3^{\′}& 1\end{array}\right]=\left[\begin{array}{cccc}{f}_x& 0& u_0& 0\\ 0& f_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}{r}_1& t_x\\ r_2& t_y\\ r_3& t_z\\ 0& 1\end{array}\right]=\left[\begin{array}{cc}{f}_x{r}_1+u_0{r}_3& f_x{t}_x+u_0{t}_z\\ f_y{t}_y+v_0{r}_3& f_y{t}_y+v_0{t}_z\\ r_3& t_z\end{array}\right]---\left(8\right)$

In formula (8): m' ₁=[m ₁₁m ₁₂m ₁₃]/m ₃₄, m' ₂=[m ₂₁m ₂₂m ₂₃]/m ₃₄, m' ₃=[m ₃₁m ₃₂m ₃₃]/m ₃₄, m' ₁₄=m ₁₄/ m ₃₄, m' ₂₄=m ₂₄/ m ₃₄,

Known m by above formula ₃₄m' ₃=r ₃, be that orthogonal matrix obtains by R again | r ₃|=1, so:

$m_{34}=\frac{1}{\left|m_3^{\′}\right|}$

Finally obtain the following calibration result of camera parameters:

Video camera internal reference:

Image pixel center horizontal ordinate:

Image pixel center ordinate:

The equivalent focal length of camera sensor in x direction:

The equivalent focal length of camera sensor in y direction:

Video camera is joined outward:

$r_1=\frac{m_{34}}{f_x}(m_1^{\′}-u_0{m}_3^{\′}),$

$r_2=\frac{m_{34}}{f_x}(m_2^{\′}-v_0{m}_3^{\′}),$

r ₃=m ₃₄m’ ₃，

Crab angle: γ=| arcsin (r ₁₃) |,

The angle of pitch: φ=arcsin (r ₁₂/ cos γ),

Roll angle: ψ is defaulted as 0,

Camera coordinate system ties up to the translational movement in z direction: t with respect to world coordinates _z=m ₃₄,

Camera coordinate system ties up to the translational movement in x direction with respect to world coordinates:

Camera coordinate system ties up to the translational movement in y direction with respect to world coordinates:

Images