CN105701837A - Geometric calibration processing method and apparatus for camera - Google Patents

Abstract

The embodiment of the invention provides a geometric calibration processing method and apparatus for a camera. The method comprises: a coordinate of a spatial point and a coordinate of an image point are obtained, a panorama camera shoots the spatial point to obtain an image point corresponding to the spatial point, wherein the spatial point is a point on a spatial coordinate system and the image point is a point on an image coordinate system; a panoramic camera imaging model is obtained, wherein the panoramic camera imaging model is used for expressing a conversion relationship between the point on the spatial coordinate system and a point on an image coordinate system; and on the basis of the coordinate of the spatial point, the coordinate of the image point, and the panoramic camera imaging model, an external parameter of the panorama camera is determined. According to the invention, a defect that spatial points are not on a common plane according to a two-step calibration method and a defect that an initial value error is large because various distortion factors are not considered according to the calibration method put forwarded by Zhang Youzheng can be overcome.

Description

A kind of camera geometric calibration processing method and device

Technical field

The present invention relates to image processing field, particularly to a kind of camera geometric calibration processing method and device。

Background technology

Pan-shot, it is common that refer to carry out horizontal 360-degree and vertical 180 degree of shootings centered by certain point, captured plurality of pictures is spliced into shooting and the picture joining method of a Zhang Quanjing picture。In general, pan-shot at least can include panoramic picture and two kinds of forms of panoramic video。

Generally, when utilizing multiple captured original images to be spliced into a Zhang Quanjing picture, can relate to mapping and splicing two parts。Wherein, mapping can be understood as and projects on the position that panoramic pictures is corresponding by the pixel on original image, and splicing can be understood as the overlapping region to adjacent two original images and carries out merging transition。

In order to determine the three-dimensional geometry position of space object surface point and its mutual relation between corresponding point in original image, it is possible to by the mode of camera geometric calibration, it is thus achieved that camera parameter, in order to follow-up described camera parameter can be utilized to carry out pixel projection。Generally, camera parameter can include the outer ginseng of camera and the internal reference of camera。

At present, the conventional outer ginseng estimation technique mainly has two-stage calibration method and Zhang Zhengyou standardizition。

For two-stage calibration method, require that when solving outer ginseng spatial point is non-coplanar, if coplanar, cannot obtain outer ginseng, therefore for coplanar flat board calibrating block, it is necessary to use additive method to carry out outer ginseng and estimate。It addition, two step standardizitions being assumed, camera lens only has radial distortion, the estimation difference of panorama picture of fisheye lens is very big。

For Zhang Youzheng standardizition, first it is left out various distortion, but solves substituting into a little, but, the distortion of the pixel being typically remote from picture centre is all very big, if these pixels also being regarded the pixel not distorted substitute into the words solved as, it is clear that can strengthen the error solving initial value。Similarly, Zhang Youzheng standardizition also only considered radial distortion, is not particularly suited for fish eye lens。

Summary of the invention

The embodiment of the present invention provides a kind of camera geometric calibration processing method and device, it is possible to the defect being prevented effectively from existing scaling scheme to exist。

A kind of camera geometric calibration processing method, described method includes:

Obtaining the coordinate of spatial point and the coordinate of picture point, panorama camera shoots described spatial point, it is thus achieved that the described picture point corresponding with described spatial point, described spatial point is the point that space coordinates is fastened, and described picture point is the point that image coordinate is fastened；

Obtaining panorama camera imaging model, described panorama camera imaging model is for representing the transformational relation between point that described space coordinates fastens and the point that described image coordinate is fastened；

Utilize the coordinate of described spatial point, the coordinate of described picture point and described panorama camera imaging model, it is determined that the outer ginseng of panorama camera。

Preferably, described acquisition panorama camera imaging model, including:

What described space coordinates was fastened clicks on line linearity conversion, it is thus achieved that the point in the camera lens coordinate system of panorama camera；

The line nonlinearity that clicks in described camera lens coordinate system is converted, it is thus achieved that the point that the sensor coordinates of camera lens is fastened；

The point that described sensor coordinates is fastened carries out affine transformation, it is thus achieved that the point that described image coordinate is fastened；

The point that the point fastened based on described space coordinates and described image coordinate are fastened, sets up described panorama camera imaging model。

Preferably, by below equation, the point described space coordinates fastened is converted to the point in described camera lens coordinate system:

${\mathrm{\λ}}_{ij}\·p_{ij}={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{u}_{ij}^{\′\′}\\ v_{ij}^{\′\′}\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right]=P^i\·X=\left[\begin{array}{cccc}{r}_1^i& r_2^i& r_3^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ z^{ij}\\ 1\end{array}\right]$

Wherein, (x_ij, y_ij, z_ij) point in representation space coordinate system, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system,λ_ijRepresent normalized parameter, PⁱRepresent that the first spin matrix r and D translation vector t, i represent the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened。

Preferably, by below equation, the point in described camera lens coordinate system is converted to the point that described sensor coordinates is fastened:

g(u″_ij, v "_ij)=(u "_ij, v "_ij, f (u "_ij, v "_ij))^T

Wherein, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system, (u "_ij, v "_ij) representing the point that sensor coordinates is fastened, T represents that transposition, i represent the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened,

Preferably, by below equation, the point fastened by described sensor coordinates is converted to the point that described image coordinate is fastened:

u″_ij=Au '_ij+t₁, v "_ij=Av '_ij+t₁

Wherein, (u "_ij, v "_ij) represent the point that sensor coordinates is fastened, (u '_ij, v '_ij) representing the point that image coordinate is fastened, i represents the i-th photographic head of panorama camera, and the jth point that j denotation coordination is fastened, A represents the second spin matrix, t₁Represent translation matrix。

Preferably, the point that the point fastened based on described space coordinates and described image coordinate are fastened, set up described panorama camera imaging model, including:

Obtain the corresponding relation formula between point that described space coordinates fastens and the point that described image coordinate is fastened:

$P^i\·X=\left[\begin{array}{cccc}{r}_1^i& r_2^i& r_3^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ z^{ij}\\ 1\end{array}\right]={\mathrm{\λ}}_{ij}\·p_{ij}={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{u}_{ij}^{\′\′}\\ v_{ij}^{\′\′}\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right]={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{\mathrm{Au}}_{ij}^{\′}+t_1\\ {\mathrm{Av}}_{ij}^{\′}+t_1\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right];$

Wherein, (x_ij, y_ij, z_ij) point in representation space coordinate system, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system,(u′_ij, v '_ij) represent the point that image coordinate is fastened, λ_ijRepresent normalized parameter, PⁱRepresenting that the first spin matrix r and D translation vector t, i represent the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened, A represents the second spin matrix, t₁Represent translation matrix；

If z_ijBe 0, then the corresponding relation formula between described space coordinates is fastened point and the point that described image coordinate is fastened is:

$P^i\·X=\left[\begin{array}{cccc}{r}_1^i& r_2^i& r_3^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ 0\\ 1\end{array}\right]=\left[\begin{array}{ccc}{r}_1^i& r_2^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ 1\end{array}\right]={\mathrm{\λ}}_{ij}\·p_{ij}={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{\mathrm{Au}}_{ij}^{\′}+t_1\\ {\mathrm{Av}}_{ij}^{\′}+t_1\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right];$

If equation two ends are multiplication cross p simultaneously_ij, then the corresponding relation formula between described space coordinates is fastened point and the point that described image coordinate is fastened is:

Decompose the corresponding relation formula that multiplication cross obtains, it is thus achieved that the imaging model of the i-th photographic head of panorama camera:

v′_j·(r₃₁x_j+r₃₂y_j+t₃)-f(ρ_j)·(r₂₁x_j+r₂₂y_j+t₂)=0

f(ρ_j)·(r₁₁x_j+r₁₂y_j+t₁)-u′_j·(r₃₁x_j+r₃₂y_j+t₃)=0。

u′_j·(r₂₁x_j+r₂₂y_j+t₂)-v′_j·(r₁₁x_j+r₁₂y_j+t₁)=0

A kind of camera geometric calibration processes device, and described device includes:

Coordinate acquiring unit, the coordinate of coordinate and picture point for obtaining spatial point, panorama camera shoots described spatial point, it is thus achieved that the described picture point corresponding with described spatial point, described spatial point is the point that space coordinates is fastened, and described picture point is the point that image coordinate is fastened；

Imaging model acquiring unit, is used for obtaining panorama camera imaging model, and described panorama camera imaging model is for representing the transformational relation between point that described space coordinates fastens and the point that described image coordinate is fastened；

Outer ginseng determines unit, for utilizing the coordinate of described spatial point, the coordinate of described picture point and described panorama camera imaging model, it is determined that the outer ginseng of panorama camera。

Preferably, described imaging model acquiring unit includes:

Linear transform unit, clicks on line linearity conversion for what described space coordinates was fastened, it is thus achieved that the point in the camera lens coordinate system of panorama camera；

Non-linear conversion unit, for converting the line nonlinearity that clicks in described camera lens coordinate system, it is thus achieved that the point that the sensor coordinates of camera lens is fastened；

Affine transformation unit, carries out affine transformation for the point that described sensor coordinates is fastened, it is thus achieved that the point that described image coordinate is fastened；

Imaging model sets up unit, for the point that the point fastened based on described space coordinates and described image coordinate are fastened, sets up described panorama camera imaging model。

Preferably, described linear transform unit, for by below equation, the point described space coordinates fastened is converted to the point in described camera lens coordinate system:

${\mathrm{\λ}}_{ij}\·p_{ij}={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{u}_{ij}^{\′\′}\\ v_{ij}^{\′\′}\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right]=P^i\·X=\left[\begin{array}{cccc}{r}_1^i& r_2^i& r_3^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ z^{ij}\\ 1\end{array}\right];$

Described non-linear conversion unit, for by below equation, the point in described camera lens coordinate system being converted to the point that described sensor coordinates is fastened:

g(u″_ij, v "_ij)=(u "_ij, v "_ij, f (u "_ij, v "_ij))^T；

Described affine transformation unit, for by below equation, the point fastened by described sensor coordinates is converted to the point that described image coordinate is fastened:

u″_ij=Au '_ij+t₁, v "_ij=Av '_ij+t₁

Wherein, (x_ij, y_ij, z_ij) point in representation space coordinate system, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system,λ_ijRepresent normalized parameter, PⁱRepresent that the first spin matrix r and D translation vector t, i represent the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened, (u "_ij, v "_ij) representing the point that sensor coordinates is fastened, T represents transposition, (u '_ij, v '_ij) representing the point that image coordinate is fastened, A represents the second spin matrix, t₁Represent translation matrix。

Preferably, imaging model sets up unit, for obtaining the imaging model of the i-th photographic head of panorama camera:

Obtain the corresponding relation formula between point that described space coordinates fastens and the point that described image coordinate is fastened:

$P^i\·X=\left[\begin{array}{cccc}{r}_1^i& r_2^i& r_3^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ z^{ij}\\ 1\end{array}\right]={\mathrm{\λ}}_{ij}\·p_{ij}={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{u}_{ij}^{\′}\\ v_{ij}^{\′\′}\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right]={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{\mathrm{Au}}_{ij}^{\′}+t_1\\ {\mathrm{Av}}_{ij}^{\′}+t_1\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right];$

Wherein, (x_ij, y_ij, z_ij) point in representation space coordinate system, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system,(u′_ij, v '_ij) represent the point that image coordinate is fastened, λ_ijRepresent normalized parameter, PⁱRepresenting that the first spin matrix r and D translation vector t, i represent the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened, A represents the second spin matrix, t₁Represent translation matrix；

If z_ijBe 0, then the corresponding relation formula between described space coordinates is fastened point and the point that described image coordinate is fastened is:

$P^i\·X=\left[\begin{array}{cccc}{r}_1^i& r_2^i& r_3^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ 0\\ 1\end{array}\right]=\left[\begin{array}{ccc}{r}_1^i& r_2^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ 1\end{array}\right]={\mathrm{\λ}}_{ij}\·p_{ij}={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{\mathrm{Au}}_{ij}^{\′}+t_1\\ {\mathrm{Av}}_{ij}^{\′}+t_1\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right];$

If equation two ends are multiplication cross p simultaneously_ij, then the corresponding relation formula between described space coordinates is fastened point and the point that described image coordinate is fastened is:

Decompose the corresponding relation formula that multiplication cross obtains, it is thus achieved that the imaging model of the i-th photographic head of panorama camera:

v′_j·(r₃₁x_j+r₃₂y_j+t₃)-f(ρ_j)·(r₂₁x_j+r₂₂y_j+t₂)=0

f(ρ_j)·(r₁₁x_j+r₁₂y_j+t₁)-u′_j·(r₃₁x_j+r₃₂y_j+t₃)=0。

u′_j·(r₂₁x_j+r₂₂y_j+t₂)-v′_j·(r₁₁x_j+r₁₂y_j+t₁)=0

Compared with prior art, the present invention program converts the process of picture point to by studying spatial point, obtain the panorama camera imaging model of transformational relation between representation space point and picture point, simultaneously, also can obtain the picture point coordinate of spatial point coordinate and correspondence in real time, and then determine the outer ginseng of panorama camera。Such scheme, had both been absent from the defect that two-stage calibration method requirement spatial point is non-coplanar, is also absent from Zhang Youzheng standardizition and is left out the defect that various distortion causes that initial value error is big。

Additionally, the present invention program is when setting up imaging model, take into account that space coordinates is tied to camera lens coordinate system linear transformation, camera lens coordinate is tied to the nonlinear transformation of sensor coordinate system and sensor coordinates is tied to the affine transformation of image coordinate system, not only allow for radial distortion, also contemplate barrel-type distortion, be favorably improved outside the present invention to join the precision of estimation。

It addition, when solving the outer ginseng occurrence of panorama camera, it is possible to solve least square problem by the method for machine learning, so, it is possible to amount of calculation is greatly decreased, make the present invention program be applicable to the outer in real time of embedded system and join estimation。

Accompanying drawing explanation

In order to be illustrated more clearly that the technical scheme in the embodiment of the present invention, below the accompanying drawing used required during embodiment is described is briefly described, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the premise not paying creative work, it is also possible to obtain other accompanying drawing according to these accompanying drawings。

Fig. 1 is the flow chart of camera geometric calibration processing method of the present invention；

Fig. 2 is the flow chart of the method setting up panorama camera imaging model in the present invention；

Fig. 3 is the structural representation that camera geometric calibration of the present invention processes device。

Detailed description of the invention

Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is only a part of embodiment of the present invention, rather than whole embodiments。Based on the embodiment in the present invention, the every other embodiment that those of ordinary skill in the art obtain under not making creative work premise, broadly fall into the scope of protection of the invention。

With reference to Fig. 1, it is shown that the flow chart of embodiment of the present invention camera geometric calibration processing method, it is possible to comprise the following steps:

S101, obtains the coordinate of spatial point and the coordinate of picture point, and panorama camera shoots described spatial point, it is thus achieved that the described picture point corresponding with described spatial point, described spatial point is the point that space coordinates is fastened, and described picture point is the point that image coordinate is fastened。

In order to realize horizontal 360-degree and vertical 180 degree of shootings, panorama camera generally includes multiple photographic head, is used for carrying out multi-angled shooting。Each photographic head all can shoot the spatial point that space coordinates is fastened, so as to form the corresponding diagram picture point that image coordinate is fastened。Such as, panorama camera includes m photographic head, and this step can for each photographic head, it is thus achieved that L group spatial point coordinate and picture point coordinate, i.e. can obtain m*L group spatial point coordinate and picture point coordinate for this panorama camera。Generally, the value of L can be tried one's best greatly, namely can try one's best for each photographic head and many choose spatial point and picture point。As a kind of example, L can be not less than 10。

S102, obtains panorama camera imaging model, and described panorama camera imaging model is for representing the transformational relation between point that described space coordinates fastens and the point that described image coordinate is fastened。

Panorama camera imaging model in the present invention is it is to be understood that each self-corresponding imaging model of M photographic head that includes of panorama camera, say, that the imaging model of M photographic head is collectively forming panorama camera imaging model。Set up the process of camera imaging model to may refer to FIG. 2 below place and introduce, wouldn't describe in detail herein。

As a kind of example, acquisition panorama camera imaging model in the present invention, can be pre-build and preserve imaging model, and carry out directly reading when outer ginseng is estimated at needs, such as, imaging model can be saved in panorama camera this locality or other third party devices that can communicate with panorama camera, and this can be not specifically limited by the present invention。It addition, the acquisition panorama camera imaging model in the present invention, it is also possible to it is when needs carry out outer ginseng estimation, sets up imaging model in real time。

It should be noted that the present invention program as it is shown in figure 1, first obtain the coordinate of spatial point and picture point, then can obtain panorama camera imaging model again；Or, it is also possible to first obtain panorama camera imaging model, then obtain the coordinate of spatial point and picture point；Furthermore, it is also possible to performing two steps, this can be not specifically limited by the present invention simultaneously。

S103, utilizes the coordinate of described spatial point, the coordinate of described picture point and described panorama camera imaging model, it is determined that the outer ginseng of panorama camera。

It should be noted that, panorama camera shooting spatial point also generates the process of corresponding diagram picture point, Camera extrinsic plays an important role, therefore the panorama camera imaging model of the present invention includes Camera extrinsic, namely can embody outer ginseng by imaging model and be converted to role in the process of picture point in spatial point。

When utilizing the present invention program to carry out geometric calibration, imaging model known in spatial point and correspondence image point coordinates is known, just can determine that the outer ginseng of panorama camera。Detailed process can referring to hereafter, wouldn't describe in detail herein。

With reference to Fig. 2, it is shown that the present invention sets up the flow chart of the method for panorama camera imaging model。That is, the method setting up the imaging model of each photographic head that panorama camera includes, it is possible to comprise the following steps:

S201, what described space coordinates was fastened clicks on line linearity conversion, it is thus achieved that the point in the camera lens coordinate system of panorama camera。

Spatial point is converted to picture point at least can include three transformation processs: space coordinates are converted to camera lens coordinate system, camera lens ordinate transform is sensor coordinate system, sensor coordinate system is converted to image coordinate system, explain separately below。

As a kind of example, what space coordinates was tied to camera lens coordinate system is converted to linear transformation, specifically can be obtained by the first spin matrix r and D translation vector t。Wherein, what r and t described is the parameter of the outer scene of camera, i.e. Camera extrinsic in the present invention。

The transformational relation of space coordinates and camera lens coordinate system can be presented as equation below:

${\mathrm{\λ}}_{ij}\·p_{ij}={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{u}_{ij}^{\′\′}\\ v_{ij}^{\′\′}\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right]=P^i\·X=\left[\begin{array}{cccc}{r}_1^i& r_2^i& r_3^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ z^{ij}\\ 1\end{array}\right]---\left(1\right)$

Wherein, the point in X representation space coordinate system, (x can be embodied as_ij, y_ij, z_ij)；P_ijRepresent the point in camera lens coordinate system, can be embodied as (u "_ij, v "_ij, f (u "_ij, v "_ij))；PⁱRepresent the first spin matrix r and D translation vector t；λ_ijRepresenting normalized parameter, i represents the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened。

S202, converts the line nonlinearity that clicks in described camera lens coordinate system, it is thus achieved that the point that the sensor coordinates of camera lens is fastened。

The light at the spatial point X place optical center by camera lens, through too much organizing the refraction of eyeglass, light path bends, and the some position being imaged onto on sensor (Sensor) there will be skew, and the conversion of this process is nonlinear。It is to say, camera lens coordinate be tied to Sensor coordinate system be converted to nonlinear transformation。

As a kind of example, it is possible to represent the vague generalization model of camera lens projection pattern with a Taylor polynomial。Specifically, camera lens coordinate system can be presented as equation below with the transformational relation of Sensor coordinate system:

p_ij=g (u "_ij, v "_ij)=(u "_ij, v "_ij, f (u "_ij, v "_ij))^T(2)

Wherein, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system, (u "_ij, v "_ij) representing the point that sensor coordinates is fastened, T represents transposition,The value of N can be not specifically limited by the present invention, and N can be canceled out in follow-up processing procedure。

It should be noted that f (u "_ij, v "_ij) be a nonlinear function about ρ, therefore be also denoted as f (ρ), wherein, ρ represent point (u "_ij, v "_ij) to the distance of sensor coordinates initial point。So so that the imaging model of the present invention had both considered radial distortion, it is also contemplated that barrel-type distortion, it is favorably improved outside the present invention to join the precision of estimation。

S203, the point that described sensor coordinates is fastened carries out affine transformation, it is thus achieved that the point that described image coordinate is fastened。

As a kind of example, the coordinate system set up on virtual-sensor imaging plane is in units of physical unit mm, and the image coordinate system being ultimately imaged is in units of pixel, and both zero positions are different。Therefore by affine transformation, sensor coordinate system can be transformed into image coordinate system。

The transformational relation of sensor coordinate system and image coordinate system can be presented as equation below:

u″_ij=Au '_ij+t₁, v "_ij=Av '_ij+t₁(3)

Wherein, (u "_ij, v "_ij) represent the point that sensor coordinates is fastened, (u '_ij, v '_ij) representing the point that image coordinate is fastened, A represents the second spin matrix, t₁Represent translation matrix, A and t₁Depend primarily on the photographic head of selection。

S204, the point that the point fastened based on described space coordinates and described image coordinate are fastened, set up described panorama camera imaging model。

Specifically, based on above-mentioned three kinds of conversion formulas, the corresponding relation formula between point that space coordinates fastens and the point that image coordinate is fastened just can be obtained:

$P^i\·X=\left[\begin{array}{cccc}{r}_1^i& r_2^i& r_3^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ z^{ij}\\ 1\end{array}\right]={\mathrm{\λ}}_{ij}\·p_{ij}={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{u}_{ij}^{\′\′}\\ v_{ij}^{\′\′}\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right]={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{\mathrm{Au}}_{ij}^{\′}+t_1\\ {\mathrm{Av}}_{ij}^{\′}+t_1\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right]---\left(4\right)$

As a kind of example, when without loss of generality, it is possible to choose some special spatial point, in order to obtain the imaging model of the present invention。Such as, z is chosen_ijBe the spatial point of 0, then the corresponding relation formula between space coordinates is fastened point and the point that image coordinate is fastened can be:

$P^i\·X=\left[\begin{array}{cccc}{r}_1^i& r_2^i& r_3^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ 0\\ 1\end{array}\right]=\left[\begin{array}{ccc}{r}_1^i& r_2^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ 1\end{array}\right]={\mathrm{\λ}}_{ij}\·p_{ij}={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{\mathrm{Au}}_{ij}^{\′}+t_1\\ {\mathrm{Av}}_{ij}^{\′}+t_1\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right]---\left(5\right)$

As a kind of example, for the ease of calculating imaging model, it is possible to multiplication cross p the equation two ends of formula (5) while_ij, making equation is 0:

That is, the corresponding relation formula between the point fastened of the point fastened of space coordinates and image coordinate can be:

Finally, decomposition formula (7), just can obtain the imaging model of each photographic head that panorama camera includes, for instance the imaging model of i-th photographic head can be:

v′_j·(r₃₁x_j+r₃₂y_j+t₃)-f(ρ_j)·(r₂₁x_j+r₂₂y_j+t₂)=0

f(ρ_j)·(r₁₁x_j+r₁₂y_j+t₁)-u′_j·(r₃₁x_j+r₃₂y_j+t₃)=0。

u′_j·(r₂₁x_j+r₂₂y_j+t₂)-v′_j·(r₁₁x_j+r₁₂y_j+t₁)=0

To sum up, just can obtain the imaging model of each photographic head, and then obtain the panorama camera imaging model in the present invention。

As S103 place above introduce, when spatial point coordinate corresponding to each photographic head, picture point coordinate and imaging model are all known, just can determine the Camera extrinsic of the present invention according to these Given informations。Below the solution procedure of ginseng outside each photographic head is explained。

First, the imaging model of photographic head is rewritten into vector form: M H=0, wherein,

H=[r₁₁, r₁₂, r₂₁, r₂₂, t₁, t₂]^T；

After carrying out model rewriting, just can obtain a super positive definite equation, it is possible to adopt method of least square to solve this equation group, | | the M H | | that namely solves min²。

Method of least square is adopted to solve r₁₁, r₁₂, r₂₁, r₂₂, t₁, t₂Value after, it is contemplated that the first spin matrix r has the property that | r₁r₂r₃|=1, therefore the r solved can be utilized₁₁, r₁₂, r₂₁, r₂₂Continue to solve r₃₁, r₃₂Value。So, just can obtain and outside photographic head, join r₁₁, r₁₂, r₂₁, r₂₂, r₃₁, r₃₂, t₁, t₂Concrete value。

As a kind of example, when adopting method of least square solving equation group, it is possible to adopt SVD (English: SingularValueDecomposition, Chinese: singular value decomposition) algorithm to realize；Or, it is also possible to being realized by following machine learning mode, this can be not specifically limited by the present invention。

Relative to svd algorithm, solve least square problem based on machine learning mode, it is possible to amount of calculation is greatly decreased, make the present invention program be applicable to the outer in real time of embedded system and join estimation。It is explained below explanation。

It should be noted that, the manner is mainly by the method for machine learning, it is determined that goes out a set including some iterative parameters, and then just can join on the basis of initial value outside photographic head, utilize iterative parameter to carry out iteration optimization progressively, finally determine preferably Camera extrinsic。

As a kind of example, it is possible to arrange outer ginseng initial value in conjunction with practical operation experience；Or, it is contemplated that the priori value of camera parameter has been preferably be worth generally, therefore also can arrange outer ginseng initial value according to priori value, for instance priori value can be defined as outer ginseng initial value；Or, it is possible on the basis of priori value, increase random disturbance, it is thus achieved that join outward initial value。For example, it is possible to according to practical operation experience, random disturbance value is set, or, random disturbance value also can be set to ± (priori value/100), this can be not specifically limited by the present invention。

Outer ginseng x in the present invention program, before iteration_k-1With the outer ginseng x after iteration_kBetween relation, it is possible to be presented as following iterative formula: x_k=x_k-1+P_k-1M·H(x_k-1)+Q_k-1, therefore, solve least square problem and translated into and solve iterative parameter P_k-1And Q_k-1。

Specifically, the process solving iterative parameter can be divided into study stage and Qualify Phase。

1. the study stage

Choose training sample, and by the thought of the supervised learning in machine learning, use training sample study to obtain following iterative parameter set: { P₀, P₁..., P_k-1, P_kAnd { Q₀, Q₁..., Q_k-1, Q_k}。

It should be noted that the parameter of training sample can embody as follows in the present invention: the identity coding of the sample photographic head that sample panorama camera includes；Spatial point coordinate that each sample photographic head is corresponding and picture point coordinate；The outer ginseng initial value of each sample photographic head；The outer ginseng estimated value of each sample photographic head, i.e. preferably Camera extrinsic。

2. Qualify Phase

Generally, during k=5, namely iterative parameter set is { P₀, P₁, P₂, P₃, P₄, P₅And { Q₀, Q₁, Q₂, Q₃, Q₄, Q₅Time, join outside photographic head and the basis of initial value is iterated, it is possible to obtain this photographic head and preferably join estimated value outward。Iterations can be not specifically limited by the present invention, it is possible to determines in conjunction with practical situations。

With method as described above accordingly, the embodiment of the present invention also provides for a kind of camera geometric calibration and processes device, and referring to Fig. 3, described device comprises the steps that

Coordinate acquiring unit 301, the coordinate of coordinate and picture point for obtaining spatial point, panorama camera shoots described spatial point, it is thus achieved that the described picture point corresponding with described spatial point, described spatial point is the point that space coordinates is fastened, and described picture point is the point that image coordinate is fastened；

Imaging model acquiring unit 302, is used for obtaining panorama camera imaging model, and described panorama camera imaging model is for representing the transformational relation between point that described space coordinates fastens and the point that described image coordinate is fastened；

Outer ginseng determines unit 303, for utilizing the coordinate of described spatial point, the coordinate of described picture point and described panorama camera imaging model, it is determined that the outer ginseng of panorama camera。

Alternatively, described imaging model acquiring unit includes:

Linear transform unit, clicks on line linearity conversion for what described space coordinates was fastened, it is thus achieved that the point in the camera lens coordinate system of panorama camera；

Non-linear conversion unit, for converting the line nonlinearity that clicks in described camera lens coordinate system, it is thus achieved that the point that the sensor coordinates of camera lens is fastened；

Affine transformation unit, carries out affine transformation for the point that described sensor coordinates is fastened, it is thus achieved that the point that described image coordinate is fastened；

Imaging model sets up unit, for the point that the point fastened based on described space coordinates and described image coordinate are fastened, sets up described panorama camera imaging model。

Alternatively,

Described linear transform unit, for by below equation, the point described space coordinates fastened is converted to the point in described camera lens coordinate system:

${\mathrm{\λ}}_{ij}\·p_{ij}={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{u}_{ij}^{\′\′}\\ v_{ij}^{\′\′}\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right]=P^i\·X=\left[\begin{array}{cccc}{r}_1^i& r_2^i& r_3^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ z^{ij}\\ 1\end{array}\right];$

Described non-linear conversion unit, for by below equation, the point in described camera lens coordinate system being converted to the point that described sensor coordinates is fastened:

g(u″_ij, v "_ij)=(u "_ij, v "_ij, f (u "_ij, v "_ij))^T；

Described affine transformation unit, for by below equation, the point fastened by described sensor coordinates is converted to the point that described image coordinate is fastened:

u″_ij=Au '_ij+t₁, v "_ij=Av '_ij+t₁

Wherein, (x_ij, y_ij, z_ij) point in representation space coordinate system, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system,λ_ijRepresent normalized parameter, PⁱRepresent that the first spin matrix r and D translation vector t, i represent the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened, (u "_ij, v "_ij) representing the point that sensor coordinates is fastened, T represents transposition, (u '_ij, v '_ij) representing the point that image coordinate is fastened, A represents the second spin matrix, t₁Represent translation matrix。

Alternatively, imaging model sets up unit, for obtaining the imaging model of the i-th photographic head of panorama camera:

Obtain the corresponding relation formula between point that described space coordinates fastens and the point that described image coordinate is fastened:

$P^i\·X=\left[\begin{array}{cccc}{r}_1^i& r_2^i& r_3^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ z^{ij}\\ 1\end{array}\right]={\mathrm{\λ}}_{ij}\·p_{ij}={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{u}_{ij}^{\′}\\ v_{ij}^{\′\′}\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right]={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{\mathrm{Au}}_{ij}^{\′}+t_1\\ {\mathrm{Av}}_{ij}^{\′}+t_1\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right];$

Wherein, (x_ij, y_ij, z_ij) point in representation space coordinate system, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system,(u′_ij, v '_ij) represent the point that image coordinate is fastened, λ_ijRepresent normalized parameter, PⁱRepresenting that the first spin matrix r and D translation vector t, i represent the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened, A represents the second spin matrix, t₁Represent translation matrix；

If z_ijBe 0, then the corresponding relation formula between described space coordinates is fastened point and the point that described image coordinate is fastened is:

$P^i\·X=\left[\begin{array}{cccc}{r}_1^i& r_2^i& r_3^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ 0\\ 1\end{array}\right]=\left[\begin{array}{ccc}{r}_1^i& r_2^i& t^i\end{array}\right]\·\left[\begin{array}{c}{x}_{ij}\\ y_{ij}\\ 1\end{array}\right]={\mathrm{\λ}}_{ij}\·p_{ij}={\mathrm{\λ}}_{ij}\·\left[\begin{array}{c}{\mathrm{Au}}_{ij}^{\′}+t_1\\ {\mathrm{Av}}_{ij}^{\′}+t_1\\ f(u_{ij}^{\′\′},v_{ij}^{\′\′})\end{array}\right];$

If equation two ends are multiplication cross p simultaneously_ij, then the corresponding relation formula between described space coordinates is fastened point and the point that described image coordinate is fastened is:

Decompose the corresponding relation formula that multiplication cross obtains, it is thus achieved that the imaging model of the i-th photographic head of panorama camera:

v′_j·(r₃₁x_j+r₃₂y_j+t₃)-f(ρ_j)·(r₂₁x_j+r₂₂y_j+t₂)=0

f(ρ_j)·(r₁₁x_j+r₁₂y_j+t₁)-u′_j·(r₃₁x_j+r₃₂y_j+t₃)=0。

u′_j·(r₂₁x_j+r₂₂y_j+t₂)-v′_j·(r₁₁x_j+r₁₂y_j+t₁)=0

It should be noted that each embodiment in this specification all adopts the mode gone forward one by one to describe, what each embodiment stressed is the difference with other embodiments, between each embodiment identical similar part mutually referring to。For device class embodiment, due to itself and embodiment of the method basic simlarity, so what describe is fairly simple, relevant part illustrates referring to the part of embodiment of the method。

Finally, it can further be stated that, in this article, term " includes ", " comprising " or its any other variant are intended to comprising of nonexcludability, so that include the process of a series of key element, method, article or equipment not only include those key elements, but also include other key elements being not expressly set out, or also include the key element intrinsic for this process, method, article or equipment。When there is no more restriction, statement " including ... " key element limited, it is not excluded that there is also other identical element in including the process of described key element, method, article or equipment。

Above scheme provided by the present invention being described in detail, principles of the invention and embodiment are set forth by specific case used herein, and the explanation of above example is only intended to help to understand method and the core concept thereof of the present invention；Simultaneously for one of ordinary skill in the art, according to the thought of the present invention, all will change in specific embodiments and applications, in sum, this specification content should not be construed as limitation of the present invention。

Claims (10)

1. a camera geometric calibration processing method, it is characterised in that described method includes:

Obtaining the coordinate of spatial point and the coordinate of picture point, panorama camera shoots described spatial point, it is thus achieved that the described picture point corresponding with described spatial point, described spatial point is the point that space coordinates is fastened, and described picture point is the point that image coordinate is fastened；

Obtaining panorama camera imaging model, described panorama camera imaging model is for representing the transformational relation between point that described space coordinates fastens and the point that described image coordinate is fastened；

Utilize the coordinate of described spatial point, the coordinate of described picture point and described panorama camera imaging model, it is determined that the outer ginseng of panorama camera。

2. method according to claim 1, it is characterised in that described acquisition panorama camera imaging model, including:

What described space coordinates was fastened clicks on line linearity conversion, it is thus achieved that the point in the camera lens coordinate system of panorama camera；

The line nonlinearity that clicks in described camera lens coordinate system is converted, it is thus achieved that the point that the sensor coordinates of camera lens is fastened；

The point that described sensor coordinates is fastened carries out affine transformation, it is thus achieved that the point that described image coordinate is fastened；

The point that the point fastened based on described space coordinates and described image coordinate are fastened, sets up described panorama camera imaging model。

3. method according to claim 2, it is characterised in that by below equation, the point described space coordinates fastened is converted to the point in described camera lens coordinate system:

Wherein, (x_ij, y_ij, z_ij) point in representation space coordinate system, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system,λ_ijRepresent normalized parameter, PⁱRepresent that the first spin matrix r and D translation vector t, i represent the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened。

4. method according to claim 2, it is characterised in that by below equation, the point in described camera lens coordinate system is converted to the point that described sensor coordinates is fastened:

g(u″_ij, v "_ij)=(u "_ij, v "_ij, f (u "_ij, v "_ij))^T

Wherein, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system, (u "_ij, v "_ij) representing the point that sensor coordinates is fastened, T represents that transposition, i represent the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened,

5. method according to claim 2, it is characterised in that by below equation, the point fastened by described sensor coordinates is converted to the point that described image coordinate is fastened:

u″_ij=Au '_ij+t₁, v "_ij=Av '_ij+t₁

Wherein, (u "_ij, v "_ij) represent the point that sensor coordinates is fastened, (u '_ij, v '_ij) representing the point that image coordinate is fastened, i represents the i-th photographic head of panorama camera, and the jth point that j denotation coordination is fastened, A represents the second spin matrix, t₁Represent translation matrix。

6. method according to claim 2, it is characterised in that the point that the point fastened based on described space coordinates and described image coordinate are fastened, sets up described panorama camera imaging model, including:

Obtain the corresponding relation formula between point that described space coordinates fastens and the point that described image coordinate is fastened:

Wherein, (x_ij, y_ij, z_ij) point in representation space coordinate system, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system,(u′_ij, v '_ij) represent the point that image coordinate is fastened, λ_ijRepresent normalized parameter, PⁱRepresenting that the first spin matrix r and D translation vector t, i represent the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened, A represents the second spin matrix, t₁Represent translation matrix；

If z_ijBe 0, then the corresponding relation formula between described space coordinates is fastened point and the point that described image coordinate is fastened is:

If equation two ends are multiplication cross p simultaneously_ij, then the corresponding relation formula between described space coordinates is fastened point and the point that described image coordinate is fastened is:

Decompose the corresponding relation formula that multiplication cross obtains, it is thus achieved that the imaging model of the i-th photographic head of panorama camera:

v′_j·(r₃₁x_j+r₃₂y_j+t₃)-f(ρ_j)·(r₂₁x_j+r₂₂y_j+t₂)=0

f(ρ_j)·(r₁₁x_j+r₁₂y_j+t₁)-u′_j·(r₃₁x_j+r₃₂y_j+t₃)=0

u′_j·(r₂₁x_j+r₂₂y_j+t₂)-v′_j·(r₁₁x_j+r₁₂y_j+t₁)=0。

7. a camera geometric calibration processes device, it is characterised in that described device includes:

Coordinate acquiring unit, the coordinate of coordinate and picture point for obtaining spatial point, panorama camera shoots described spatial point, it is thus achieved that the described picture point corresponding with described spatial point, described spatial point is the point that space coordinates is fastened, and described picture point is the point that image coordinate is fastened；

Imaging model acquiring unit, is used for obtaining panorama camera imaging model, and described panorama camera imaging model is for representing the transformational relation between point that described space coordinates fastens and the point that described image coordinate is fastened；

Outer ginseng determines unit, for utilizing the coordinate of described spatial point, the coordinate of described picture point and described panorama camera imaging model, it is determined that the outer ginseng of panorama camera。

8. device according to claim 7, it is characterised in that described imaging model acquiring unit includes:

Linear transform unit, clicks on line linearity conversion for what described space coordinates was fastened, it is thus achieved that the point in the camera lens coordinate system of panorama camera；

Non-linear conversion unit, for converting the line nonlinearity that clicks in described camera lens coordinate system, it is thus achieved that the point that the sensor coordinates of camera lens is fastened；

Affine transformation unit, carries out affine transformation for the point that described sensor coordinates is fastened, it is thus achieved that the point that described image coordinate is fastened；

Imaging model sets up unit, for the point that the point fastened based on described space coordinates and described image coordinate are fastened, sets up described panorama camera imaging model。

9. device according to claim 8, it is characterised in that

Described linear transform unit, for by below equation, the point described space coordinates fastened is converted to the point in described camera lens coordinate system:

Described non-linear conversion unit, for by below equation, the point in described camera lens coordinate system being converted to the point that described sensor coordinates is fastened:

g(u″_ij, v "_ij)=(u "_ij, v "_ij, f (u "_ij, v "_ij))^T；

Described affine transformation unit, for by below equation, the point fastened by described sensor coordinates is converted to the point that described image coordinate is fastened:

u″_ij=Au '_ij+t₁, v "_ij=Av '_ij+t₁

Wherein, (x_ij, y_ij, z_ij) point in representation space coordinate system, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system,λ_ijRepresent normalized parameter, PⁱRepresent that the first spin matrix r and D translation vector t, i represent the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened, (u "_ij, v "_ij) representing the point that sensor coordinates is fastened, T represents transposition, (u '_ij, v '_ij) representing the point that image coordinate is fastened, A represents the second spin matrix, t₁Represent translation matrix。

10. device according to claim 8, it is characterised in that imaging model sets up unit, for obtaining the imaging model of the i-th photographic head of panorama camera:

Obtain the corresponding relation formula between point that described space coordinates fastens and the point that described image coordinate is fastened:

Wherein, (x_ij, y_ij, z_ij) point in representation space coordinate system, (u "_ij, v "_ij, f (u "_ij, v "_ij)) represent the point in camera lens coordinate system,(u′_ij, v '_ij) represent the point that image coordinate is fastened, λ_ijRepresent normalized parameter, PⁱRepresenting that the first spin matrix r and D translation vector t, i represent the i-th photographic head of panorama camera, the jth point that j denotation coordination is fastened, A represents the second spin matrix, t₁Represent translation matrix；

If z_ijBe 0, then the corresponding relation formula between described space coordinates is fastened point and the point that described image coordinate is fastened is:

If equation two ends are multiplication cross p simultaneously_ij, then the corresponding relation formula between described space coordinates is fastened point and the point that described image coordinate is fastened is:

Decompose the corresponding relation formula that multiplication cross obtains, it is thus achieved that the imaging model of the i-th photographic head of panorama camera:

v′_j·(r₃₁x_j+r₃₂y_j+t₃)-f(ρ_j)·(r₂₁x_j+r₂₂y_j+t₂)=0

f(ρ_j)·(r₁₁x_j+r₁₂y_j+t₁)-u′_j·(r₃₁x_j+r₃₂y_j+t₃)=0

u′_j·(r₂₁x_j+r₂₂y_j+t₂)-v′_j·(r₁₁x_j+r₁₂y_j+t₁)=0。