CN103268612B - Based on the method for the single image fisheye camera calibration of low-rank characteristic recovery - Google Patents

Abstract

The present invention discloses a kind of method of the single image fisheye camera calibration based on low-rank characteristic recovery.The image with low-rank texture is regarded as a matrix, based on the optimization method minimizing rank of matrix, recover low-rank image, and the conversion between original image and low-rank image, the parameter of fisheye camera is just included among this conversion.Instant invention overcomes the shortcoming of a large amount of manpower of classic method hand labeled angle point consumption, only need single image simultaneously, and do not need to arrange Calibration Field, under the prerequisite ensureing certain robustness, there is advantage easily and efficiently.

Description

Based on the method for the single image fisheye camera calibration of low-rank characteristic recovery

Technical field

The present invention relates to camera calibration method, be specifically related to a kind of method of the single image fisheye camera calibration based on low-rank characteristic recovery.

Background technology

Camera calibration is the problem on basis the most in computer vision and mapping science, only carries out camera calibration, and the pixel on image correctly could be mapped on three dimensions, or be correctly mapped on image by the point on three dimensions.Therefore, in computer vision field, as long as relate to the application of 3-d recovery, all need the calibrating parameters of camera, and the precision of camera calibration, also determine the precision of these application to a great extent.

Recent decades, camera calibration problem is a hot issue all the time, and scholars propose the method for many camera calibrations.

Current, the fisheye camera scaling method of single image nearly all depends on the correspondence of seeking unique point or line, and by certain geometrical constraint, simultaneous equations solve parameter.If adopt automatic extract minutiae, then the method that Feature point correspondence gets up to carry out retraining is depended critically upon the precision of extract minutiae and Feature Points Matching, and the unique point automatically extracted can not extract all unique points of demarcating and needing usually, this just constrains the precision of camera calibration.Also have a kind of method to be manually select the corresponding point in several figure, but this need operator very careful and careful, so it makes camera calibration become a very consuming time and warm work.

Summary of the invention

The object of the present invention is to provide a kind of method of the single image fisheye camera calibration based on low-rank characteristic recovery, the image with low-rank texture is regarded as a matrix, based on the optimization method minimizing rank of matrix, recover low-rank image, and the conversion between original image and low-rank image, the parameter of fisheye camera is just included among this conversion.

The technical solution used in the present invention is:

(1) manually select the rectangular area in former figure with low-rank texture, only the rectangular area of low-rank texture is operated;

(2) parameter initialization: adopt unified Sphere Measurement Model and double ends degree projection model, and use the initialization of TILT method to convert parameter, comprising focal distance f _x, f _y, principal point o _x, o _y, minute surface parameter ξ, rotation matrix R;

(3) optimization problem minimizing order is solved:

Wherein I is the image of input, and τ is conversion parameter, and ο is the operational symbol of the geometric transformation adopting unified Sphere Measurement Model and double precision projection model, I ⁰for the low-rank image obtained, λ is a constant, is taken as width is the width of output image, and E is a sparse matrix, the pollution be used in token image, the order that rank () is compute matrix, || || ₀for the zeroth order norm of compute matrix;

(4) from τ, take out the internal reference of camera.

The concrete grammar of above-mentioned steps (2) parameter initialization is:

Unified Sphere Measurement Model is defined as:

$\left\{\begin{array}{c}x=\frac{X}{\mathrm{\ξ}+Z}\\ y=\frac{Y}{\mathrm{\ξ}+Z}\end{array}\right.---\left(2\right)$

Wherein (x, y) is the point in picture plane, and (X, Y, Z) is the three-dimensional coordinate in unit sphere in camera coordinates system;

Double ends degree projection model is defined as:

$\left\{\begin{array}{c}\mathrm{\θ}=arctan\left(\frac{Z}{Y}\right)\\ \mathrm{\α}=arctan\left(\frac{Z}{X}\right)\end{array}\right.---\left(3\right)$

The longitude that wherein θ is is south poles with X-axis two end points, the longitude that α is is south poles with Y-axis two end points, tiles θ and the α value of each three-dimensional point in u and v both direction, then obtain the image projected;

Use the approximate value of TILT method estimation parameter, the camera model of use is perspective camera model, and minute surface parameter ξ is initialized to 1.

The concrete grammar that above-mentioned steps (3) solves the optimization problem minimizing order is:

1) carry out convex relaxing to formula (1), use nuclear norm to replace order, and use L1 norm to replace L0 norm, the problem that separate changes to:

Wherein || || _*for the nuclear norm of compute matrix, || || ₁for the single order norm of compute matrix.

2) carry out linearization to formula (4), the problem that separate changes to and solves following problem in outer loop, until the change of optimized variable is less than threshold value:

Wherein J be in previous cycle I ο τ to the Jacobian matrix of τ, Δ τ be previous cycle solve variable quantity, use τ+Δ τ to upgrade current τ after each circulation;

Formula (3) uses LADMAP algorithm to solve, its method is, when outside circulant solution formula (5), three optimized variables are become two, inner loop individually solve this Two Variables until convergence, each variable solve closed solution;

3) when after τ convergence, outer loop is jumped out.

The concrete grammar that above-mentioned steps (4) takes out the internal reference of camera from τ is:

F is taken out from τ _x, f _y, o _x, o _y, ξ camera internal parameter, be the result of camera calibration.

Compared with background technology, the beneficial effect that the present invention has is:

Instant invention overcomes the shortcoming of a large amount of manpower of classic method hand labeled angle point consumption, only need single image simultaneously, and do not need to arrange Calibration Field, under the prerequisite ensureing certain robustness, there is advantage easily and efficiently

Accompanying drawing explanation

Fig. 1 is the step schematic diagram of method of the present invention.

Embodiment

Below in conjunction with accompanying drawing and embodiment, the present invention is further described.

Fig. 1 gives the process flow diagram of the method for the single image fisheye camera calibration based on low-rank characteristic recovery.

(1) rectangular area in former figure with low-rank feature is manually selected, practical operation only need select upper left and the Liang Ge summit, bottom right of this rectangle, document 1 (Wright J, Ganesh A, Rao S, et al.Robust principal component analysis:Exact recovery of corrupted low-rank matrices via convex optimization [J] .Advances in neuralinformation processing systems, 2009, 22:2080-2088.) prove selected by matrix larger, the probability that formula (1) obtains solving is larger, therefore the region chosen should be the region large as far as possible comprising low-rank feature.

(2) parameter initialization: adopt unified Sphere Measurement Model and double ends degree projection model, and use branch and bound method initialization to convert parameter, comprising focal distance f _x, f _y, principal point o _x, o _y, minute surface parameter ξ, rotation matrix R, unified Sphere Measurement Model is defined as:

$\left\{\begin{array}{c}x=\frac{X}{\mathrm{\ξ}+Z}\\ y=\frac{Y}{\mathrm{\ξ}+Z}\end{array}\right.---\left(1\right)$

Wherein (x, y) is the point in picture plane, and (X, Y, Z) is the three-dimensional coordinate in unit sphere in camera coordinates system;

Double ends degree projection model is defined as:

$\left\{\begin{array}{c}\mathrm{\θ}=arctan\left(\frac{Z}{Y}\right)\\ \mathrm{\α}=arctan\left(\frac{Z}{X}\right)\end{array}\right.---\left(2\right)$

The longitude that wherein θ is is south poles with X-axis two end points, the longitude that α is is south poles with Y-axis two end points, tiles θ and the α value of each three-dimensional point in u and v both direction, then can obtain the image projected;

Use document 2 (Zhang Z, Liang X, Ganesh A, et al.TILT:transform invariant low-ranktextures [M] .Computer Vision – ACCV 2010.Springer Berlin Heidelberg, 2011:314-328.) approximate value of the TILT method that proposes estimation parameter, TILT input region identical with the region that this method inputs, minute surface parameter ξ is not calculated in TILT, initialization its be 1.

(3) optimization problem minimizing order is solved:

Carry out formula (3) convex lax, use nuclear norm to replace order, and use L1 norm to replace L0 norm, the problem that separate changes to:

Wherein || || _*for the nuclear norm of compute matrix, || || ₁for the single order norm of compute matrix.

2) carry out linearization to formula (4), the problem that separate changes to and solves following problem in outer loop, until the change of optimized variable is less than threshold value:

Wherein J be in previous cycle I ο τ to the Jacobian matrix of τ, Δ τ be previous cycle solve variable quantity, use τ+Δ τ to upgrade current τ after each circulation;

Formula (3) can use LADMAP algorithm to solve, its thought is, when outside circulant solution formula (3), three optimized variables are become two, this Two Variables is solved respectively until convergence in inner loop, each variable solve closed solution, its detailed step is as follows:

1. J is made ^⊥=I ₀-J (J ^tj) ^-1j ^t, formula (3) can be converted to:

2. circulation solves I ⁰and E, its closed solution is:

$I_{k+1}^0=S_{\frac{1}{{\mathrm{\μ}}_k}}\left(M_k\right)---\left(7\right)$

$E_{k+1}=T_{\frac{\mathrm{\λ}}{{\mathrm{\μ}}_k}}\left(N_k\right)---\left(8\right)$

Wherein:

μ _k+1＝min(μ _max,ρμ _k) (12)

Wherein, k represents the number of times of previous cycle, ρ ₀and ε ₂two default constants; S () is singular value contraction operator, is defined as:

S _ε(W)＝UT _ε(Σ)V ^T(14)

T _εx () is scalar contraction operator, be defined as:

T _ε(x)＝sgn(x)max(|x|-ε,0) (15)

3. step loop stop conditions is 2.:

Wherein ε ₁a default constant, ε ₂identical with mentioned above.

4) from τ, take out the internal reference of camera.

Experimental example:

Experiment uses FUJINON FE185C086HA-1 fish eye lens, has carried out two groups of experiments, the scaling board that first group of shooting is indoor, the well-regulated ceiling that second group of shooting is outdoor.Test the image collecting 10 512 × 512 respectively for two groups.Because camera calibration does not have true value, so give use this method to single width figure demarcates resultful average and standard deviation, simultaneously in order to the validity of this method is described, select calibration tool case (the document 3:C.Mei and P.Rives based on scaling board, inProceedings of IEEE Interna-tional Conference on Robotics and Automation (2007), pp.3945{3950) result and the calibration result of this method compare, calibration tool case needs the angle point of the figure simultaneously inputting several scaling boards, and need the size of known calibration plate medium square.

Table 1 gives the calibration result of this law to scaling board and ceiling, and compares with the result of the calibration tool case using scaling board.

The calibration result of table 1 this method calibration result and calibration tool case

Claims (2)

1. based on a method for the single image fisheye camera calibration of low-rank characteristic recovery, it is characterized in that: by minimizing the optimization method of order, obtaining the conversion parameter of image and the low-rank image inputted, thus obtaining camera internal reference; The method concrete steps are as follows:

(1) manually select the rectangular area in the image of input with low-rank texture, only the rectangular area of low-rank texture is operated;

(2) parameter initialization: adopt unified Sphere Measurement Model and double ends degree projection model, and use the initialization of TILT method to convert parameter, comprising focal distance f _x, f _y, principal point o _x, o _y, minute surface parameter ξ, rotation matrix R;

(3) optimization problem minimizing order is solved:

Wherein I is the image of input, and τ is conversion parameter, and o is the operational symbol of the geometric transformation adopting unified Sphere Measurement Model and double ends degree projection model, I ⁰for the low-rank image obtained, λ is a constant, is taken as width is the width of low-rank image, and E is a sparse matrix, the pollution be used in the image of sign input, the order that rank () is compute matrix, || || ₀for the zeroth order norm of compute matrix;

(4) from τ, take out the internal reference of camera;

The concrete grammar of described step (2) parameter initialization is:

Unified Sphere Measurement Model is defined as:

$\left\{\begin{array}{c}x=\frac{X}{\mathrm{\ξ}+Z}\\ y=\frac{Y}{\mathrm{\ξ}+Z}\end{array}\right.---\left(2\right)$

Wherein (x, y) is the point in picture plane, and (X, Y, Z) is the three-dimensional coordinate in unit sphere in camera coordinates system;

Double ends degree projection model is defined as:

$\left\{\begin{array}{c}\mathrm{\θ}=arctan\left(\frac{Z}{Y}\right)\\ \mathrm{\α}=arctan\left(\frac{Z}{X}\right)\end{array}\right.---\left(3\right)$

The longitude that wherein θ is is south poles with X-axis two end points, the longitude that α is is south poles with Y-axis two end points, tiles θ and the α value of each three-dimensional point in u and v both direction, then obtain the image projected;

Use the approximate value of TILT method estimation parameter, the camera model of use is perspective camera model, and minute surface parameter ξ is initialized to 1;

The concrete grammar that described step (3) solves the optimization problem minimizing order is:

3.1) carry out convex relaxing to formula (1), use nuclear norm to replace order, and use L1 norm to replace L0 norm, the problem that separate changes to:

Wherein || || _*for the nuclear norm of compute matrix, || || ₁for the single order norm of compute matrix;

3.2) carry out linearization to formula (4), the problem that separate changes to and solves following problem in outer loop, until the change of optimized variable is less than threshold value:

Wherein J be in previous cycle I o τ to the Jacobian matrix of τ, Δ τ be previous cycle solve variable quantity, use τ+Δ τ to upgrade current τ after each circulation;

Formula (5) uses LADMAP algorithm to solve, its method is, when outside circulant solution formula (5), three optimized variables are become two, inner loop individually solve this Two Variables until convergence, each variable solve closed solution;

3.3) when after τ convergence, jump out outer loop.

2. the method for a kind of single image fisheye camera calibration based on low-rank characteristic recovery according to claim 1, is characterized in that: the concrete grammar that described step (4) takes out the internal reference of camera from τ is:

F is taken out from τ _x, f _y, o _x, o _y, ξ camera internal parameter, be the result of camera calibration.