CN105430333A - Method and device for calculating gun-type camera distortion coefficient in real time - Google Patents

Abstract

The invention discloses a method and a device for calculating a gun-type camera distortion coefficient in real time. The method comprises the following steps: acquiring an image from a gun-type camera and acquiring two or more images from a dome camera, wherein the contents of every two adjacent images from the dome camera are overlapped, and the images from the dome camera are partially overlapped in content with the image from the gun-type camera; separately extracting feature points of the images from the dome camera for matching; establishing a homography matrix between two adjacent images from the dome camera according to the feature points matched between the two adjacent dome camera images; calculating a dome camera distortion coefficient according to the homography matrix between the two adjacent dome camera images; separately extracting feature points of one image from the dome camera and the image from the gun-type camera for matching; establishing a homography matrix between the images of the dome camera and the gun-type camera according to the feature points matched between the images of the dome camera and the gun-type camera; and calculating the gun-type camera distortion coefficient according to the dome camera distortion coefficient and the homography matrix between the images of the dome camera and the gun-type camera. Through adoption of the method and the device, the accurate gun-type camera distortion coefficient can be calculated in real time with the changing focal length; and meanwhile, the method and the device are simple in algorithm and high in efficiency.

Description

A kind of method of real-time time calculation gunlock distortion factor and device

Technical field

The present invention relates to technical field of video monitoring, be specifically related to a kind of method and device of real-time time calculation gunlock distortion factor.

Background technology

Safety monitoring is an important ring during current safe city is built, if traditional video monitoring also exists under large scene captured target detailed information cannot pay close attention to target detail information, other moving targets and cannot realize the shortcomings such as real-time intellectual analysis, target following, track record, feature image candid photograph can ignore around, therefore modern safety monitoring needs intelligentized security protection means to assist the protecting, monitoring of being correlated with.And (the so-called rifle ball interlock of the video monitoring scheme of rifle ball linked system, refer to " the many balls of a rifle ", wherein " rifle " is high-definition network gun shaped video camera, " many balls " is high-definition network ball-shaped camera), adopt the technology such as advanced video analysis algorithm and image procossing, accomplish to improve greatly in the practicality of safety-protection system and efficiency, compared with traditional video surveillance, not only can " that sees be complete in scene, that sees is clear ", and seizure warning can be carried out to suspicious event and suspicious crowd/individual goal more efficiently, thus avoid and fail to report phenomenon in artificial monitoring, truly accomplish the intellectuality of modern security protection.

Detect target in gunlock image, and determine the orientation of target, ball machine image by following gunlock image registration, and constructs projection matrix to obtain the orientation of target under ball machine coordinate system, thus obtains the initial position for controlling marble forming machine tracking target.But traditional gunlock is because need to obtain guarded region as far as possible on a large scale, often be the camera lens of wide-angle, there is larger distortion, thus the virtual borderlines between rifle ball has not been linear mapping relations, had a strong impact on the accuracy of calibration, therefore the calibration of camera distortion coefficient has been technical problem urgently to be resolved hurrily.Along with the progress of technology, the resolution of CCTV camera is more and more higher, the requirement that also can meet engineering gradually of picture distortion calibration precision.But traditional calibration method, be utilize controlling filed to carry out strict calibration, cost is too expensive, and workload is also very large, and efficiency is not high.The Zhang Zhengyou professor of Microsoft Research, invention utilize gridiron pattern to carry out the method for calibration, i.e. Zhang Shi standardization, required experimental site and equipment require very low, greatly reduce calibration cost, obtain a wide range of applications.But Zhang Zhengyou calibration method needs to carry out calibration separately in a pre-installation to every complete equipment, otherwise because individual difference, all devices all adopts same set of distortion factor to be forbidden; And equipment is in installation process, need to carry out Real-time Focusing according to monitoring scene, and distortion factor also can change along with the change of focal length, and obviously Zhang Zhengyou calibration method cannot adapt to the requirement of real-time calibration.And other camera self calibration method, utilize bundle adjustment model, such as SFM technology (StructurefromMotion, from movable information, recover three-dimensional scene structure), or photogrammetric in sky three technology, but these methods be not suitable for the situation of gunlock and ball machine limitation of movement, such as, gunlock is motionless, and when ball machine is similar to Concentric rotation, this algorithm is with regard to complete failure.

Summary of the invention

Therefore; the technical problem to be solved in the present invention is to overcome in prior art to be needed to carry out prior calibration distortion factor by controlling filed or scaling board; the situation of the individual difference of equipment cannot be applicable to; and equipment is after installation; the defect that the focal length variations of gunlock can cause distortion factor to change, thus a kind of method and device of real-time time calculation gunlock distortion factor are provided.

Another technical problem that the present invention will solve is to overcome the self-calibrating method based on the bundle adjustment model of stereo vision three-dimensional rebuilding in prior art, gunlock cannot be applicable to motionless, ball machine is similar to the defect of the limited situation of the equipment moving of Concentric rotation, thus provides a kind of method and the device that return calculation gunlock distortion factor in real time that do not need back the comparison of calculating scenery three-dimensional structure to simplify.

For this reason, the invention provides following technical scheme:

A method for real-time time calculation gunlock distortion factor, comprises the steps:

Utilize gunlock to obtain a gunlock picture, under utilizing ball machine to remain on same focal length, rotary taking obtains two with Apparatus for feeding balls as disintegrating members image, and at least often the content of shooting of adjacent two ball machine images has overlap, and ball machine image has overlapping with the content of shooting of gunlock image;

Extract the characteristic point of ball machine image respectively, and mate the characteristic point of often adjacent two ball machine images;

Set up the homography matrix between these adjacent two ball machine images according to the characteristic point of adjacent two ball machine images match respectively, in this homography matrix, comprise ball machine distortion factor to be calculated;

The homography matrix between adjacent two ball machine images is utilized to calculate ball machine distortion factor;

Extract the characteristic point of a ball machine image and gunlock image respectively, and mate the characteristic point of this ball machine image and this gunlock image;

Utilize the characteristic point of mating between ball machine image and gunlock image to set up homography matrix between ball machine image and gunlock image, this homography matrix comprises gunlock distortion factor to be calculated;

Ball machine distortion factor, homography matrix between ball machine image and gunlock image is utilized to calculate gunlock distortion factor.

Preferably, the step of the homography matrix set up between these adjacent two ball machine images according to the characteristic point of adjacent two ball machine images match respectively comprises:

Obtain the undistorted coordinate of the Feature point correspondence mated between adjacent two ball machine images respectively, the function of this is undistorted coordinate is ball machine distortion factor to be calculated;

The undistorted coordinate corresponding respectively according to the characteristic point of mating between these adjacent two ball machine images sets up the homography matrix between these adjacent two ball machine images.

Preferably, the step obtaining the undistorted coordinate of the Feature point correspondence mated between adjacent two ball machine images respectively comprises:

The characteristic point calculating ball machine image is respectively relative to the coordinate of ball owner point and radius size;

Corresponding undistorted coordinate is obtained relative to the coordinate of the coordinate of ball owner point and radius size, ball owner point and ball machine distortion factor to be calculated according to the characteristic point of ball machine image.

Preferably, the step utilizing the homography matrix between adjacent two ball machine images to calculate ball machine distortion factor comprises:

Preset the initial value of ball machine distortion factor, and the initial value of the homography matrix of corresponding acquisition often between adjacent two ball machine images;

According to the homography matrix often between adjacent two ball machine images, single order Newton iteration method is utilized to obtain the increment of ball machine distortion factor;

Ball machine distortion factor is obtained according to the initial value of ball machine distortion factor and the increment of ball machine distortion factor.

Preferably, the increment of ball machine distortion factor is obtained by following formulae discovery:

${\mathrm{\Δ}}_1=-{\left(J_1^T{J}_1\right)}^{-1}{J}_1^{T}F\left(P_0\right),$

Wherein,

F (P ₀) for presetting the initial value of the linear fractional function between m corresponding to the initial value of ball machine distortion factor adjacent two ball machine images, the homography matrix one_to_one corresponding of this m linear fractional function respectively and between individual adjacent two the ball machine images of m;

Wherein, $\left\{\begin{array}{c}P=\left(\begin{array}{ccccccccc}{k}_1& k_2& k_3& p_1& p_2& H_1& H_2& ...& H_m\end{array}\right)\\ H_i=\left(\begin{array}{ccccc}{h}_{i1}& h_{i2}& h_{i3}& ...& h_{i9}\end{array}\right)\end{array}\right\},$ Namely P is ball machine distortion factor (k ₁, k ₂, k ₃, p ₁, p ₂) and m homography matrix (H ₁, H ₂h _m); F (P) is linear fractional function corresponding to m homography matrix comprising ball machine distortion factor; N is the number of same place, F ₁₁f _nmit is a n same place corresponding m linear fractional function respectively.

Preferably, step ball machine distortion factor being converted to forward distortion factor is also comprised:

What obtain multiple ball machine image has distorted image coordinate sampled point, and calculates corresponding orthoscopic image coordinate according to ball machine distortion factor;

Set up orthoscopic image coordinate, forward distortion factor respectively and have the corresponding equation of the coordinate sampled point that distorts;

Solve forward distortion factor.

Preferably, the step utilizing ball machine distortion factor, homography matrix between ball machine image and gunlock image to calculate gunlock distortion factor comprises:

Preset the initial value of gunlock distortion factor, and the initial value of homography matrix between corresponding acquisition ball machine image and gunlock image;

According to the homography matrix between ball machine image and gunlock image, single order Newton iteration method is utilized to obtain the increment of gunlock distortion factor;

Gunlock distortion factor is obtained according to the distortion factor of ball machine and the increment of gunlock distortion factor.

Preferably, the increment of gunlock distortion factor is obtained by following formulae discovery:

${\mathrm{\Δ}}_2=-{\left(J_2^T{J}_2\right)}^{-1}{J}_2^T{F}^{\′}\left(P_0^{\′}\right)$

Wherein, $J_2=\frac{\∂F^{\′}\left(P^{\′}\right)}{\∂P^{\′}}=\left[\begin{array}{cccccc}\frac{\∂F_1^{\′}}{\∂k_1^{\′}}& \frac{\∂F_1^{\′}}{\∂k_2^{\′}}& \frac{\∂F_1^{\′}}{\∂k_3^{\′}}& \frac{\∂F_1^{\′}}{\∂p_1^{\′}}& \frac{\∂F_1^{\′}}{\∂p_2^{\′}}& \frac{\∂F_1^{\′}}{\∂H}\\ \frac{\∂F_2^{\′}}{\∂k_1^{\′}}& \frac{\∂F_2^{\′}}{\∂k_2^{\′}}& \frac{\∂F_2^{\′}}{\∂k_3^{\′}}& \frac{\∂F_2^{\′}}{\∂p_1^{\′}}& \frac{\∂F_2^{\′}}{\∂p_2^{\′}}& \frac{\∂F_2^{\′}}{\∂H}\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ \frac{\∂F_r^{\′}}{\∂k_1^{\′}}& \frac{\∂F_r^{\′}}{\∂k_2^{\′}}& \frac{\∂F_r^{\′}}{\∂k_3^{\′}}& \frac{\∂F_r^{\′}}{\∂p_1^{\′}}& \frac{\∂F_r^{\′}}{\∂p_2^{\′}}& \frac{\∂F_r^{\′}}{\∂H}\end{array}\right],$

F ' (P ' ₀) be the initial value of the linear fractional function between ball machine image corresponding to the initial value of gunlock distortion factor preset and gunlock image, this ball machine image is corresponding with the linear fractional function between gunlock image and its homography matrix;

Wherein, $P^{\′}=\left(\begin{array}{cccccc}{k}_1^{\′}& k_2^{\′}& k_3^{\′}& p_1^{\′}& p_2^{\′}& H^{\′}\end{array}\right),$ Namely P is gunlock distortion factor $\left(\begin{array}{ccccc}{k}_1^{\′}& k_2^{\′}& k_3^{\′}& p_1^{\′}& p_2^{\′}\end{array}\right)$ And the homography matrix (H ') between ball machine image and gunlock image, F (P ') is linear fractional function corresponding to the homography matrix between ball machine image and gunlock image; F ' ₁f ' _rit is the linear fractional function between ball machine image and gunlock image that r same place is corresponding respectively.

A device for real-time time calculation gunlock distortion factor, comprising:

Image acquisition unit, a gunlock picture is obtained for utilizing gunlock, under utilizing ball machine to remain on same focal length, rotary taking obtains two with Apparatus for feeding balls as disintegrating members image, and at least often the content of shooting of adjacent two ball machine images has overlap, and ball machine image has overlapping with the content of shooting of gunlock image;

Fisrt feature point extracts and matching unit, for extracting the characteristic point of ball machine image respectively, and mates the characteristic point of often adjacent two ball machine images;

First homography matrix sets up unit, for setting up the homography matrix between these adjacent two ball machine images according to the characteristic point of adjacent two ball machine images match respectively, comprises ball machine distortion factor to be calculated in this homography matrix;

Ball machine distortion factor computing unit, calculates ball machine distortion factor for utilizing the homography matrix between adjacent two ball machine images;

Second feature point extracts and matching unit, for extracting the characteristic point of a ball machine image and gunlock image respectively, and mates the characteristic point of this ball machine image and this gunlock image;

Second homography matrix sets up unit, and for utilizing the characteristic point of mating between ball machine image and gunlock image to set up homography matrix between ball machine image and gunlock image, this homography matrix comprises gunlock distortion factor to be calculated;

Gunlock distortion factor computing unit, calculates gunlock distortion factor for utilizing ball machine distortion factor, homography matrix between ball machine image and gunlock image.

Preferably, the first homography matrix is set up unit and is comprised:

First undistorted coordinate acquiring unit, for obtaining the undistorted coordinate of the Feature point correspondence mated between adjacent two ball machine images respectively, the function of this is undistorted coordinate is ball machine distortion factor to be calculated;

First homography matrix acquiring unit, sets up the homography matrix between these adjacent two ball machine images for the undistorted coordinate corresponding respectively according to the characteristic point of mating between these adjacent two ball machine images.

Preferably, the first undistorted coordinate acquiring unit comprises:

First computing unit, for the characteristic point that calculates ball machine image respectively relative to the coordinate of ball owner point and radius size;

First acquiring unit, obtains corresponding undistorted coordinate according to the characteristic point of ball machine image relative to the coordinate of the coordinate of ball owner point and radius size, ball owner point and ball machine distortion factor to be calculated.

Preferably, ball machine distortion factor computing unit comprises:

First initial value computing unit, for the initial value of default ball machine distortion factor, and the initial value of the homography matrix of corresponding acquisition often between adjacent two ball machine images;

First incremental computations unit, for according to the homography matrix often between adjacent two ball machine images, utilizes single order Newton iteration method to obtain the increment of ball machine distortion factor;

Ball machine distortion factor acquiring unit, for obtaining ball machine distortion factor according to the initial value of ball machine distortion factor and the increment of ball machine distortion factor.

Preferably, also comprise converting unit, comprising:

Computing unit, has distorted image coordinate sampled point for what obtain multiple ball machine image, and calculates corresponding orthoscopic image coordinate according to ball machine distortion factor;

Establishing equation unit, for setting up orthoscopic image coordinate, forward distortion factor respectively and having the corresponding equation of the coordinate sampled point that distorts;

Forward distortion factor acquiring unit, for solving forward distortion factor.

Preferably, gunlock distortion factor computing unit comprises:

Second initial value computing unit, for the initial value of default gunlock distortion factor, and the initial value of homography matrix between corresponding acquisition ball machine image and gunlock image;

Second incremental computations unit, for according to the homography matrix between ball machine image and gunlock image, utilizes single order Newton iteration method to obtain the increment of gunlock distortion factor;

Gunlock distortion factor acquiring unit, for obtaining gunlock distortion factor according to the distortion factor of ball machine and the increment of gunlock distortion factor.

Technical solution of the present invention, tool has the following advantages:

The method of real-time time calculation gunlock distortion factor provided by the invention and device, solve the problem that Conventional distortion factor check method cannot again be demarcated after rifle ball linked system installs, the distortion factor of change can be calculated along with the change of focal length in real time, meet the demand needing to focus at any time according to monitoring site; And the method does not need the three-dimensional structure of reconstruction point cloud and camera, the coefficient of gunlock picture distortion accurately just can be calculated only by Auto-matching and single should relation, simplify flow process and Mathematical Modeling, efficiency of algorithm is also higher, therefore also can not increase the weight of the computation burden of video camera arithmetic element.

Accompanying drawing explanation

In order to be illustrated more clearly in the specific embodiment of the invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.

Fig. 1 is the method flow diagram returning calculation gunlock distortion factor in the embodiment of the present invention 1 in real time;

Fig. 2 is the flow chart of the homography matrix set up in the embodiment of the present invention 1 between adjacent two ball machine images;

Fig. 3 is the flow chart setting up undistorted equation in coordinates corresponding to ball machine image characteristic point in the embodiment of the present invention 1;

Fig. 4 is the flow chart calculating ball machine distortion factor in the embodiment of the present invention 1;

Fig. 5 is the flow chart in the embodiment of the present invention 1, ball machine distortion factor being converted to forward distortion factor;

Fig. 6 is the flow chart calculating gunlock distortion factor in the embodiment of the present invention 1;

Fig. 7 is the theory diagram of the device returning calculation gunlock distortion factor in the embodiment of the present invention 2 in real time.

Embodiment

Be clearly and completely described technical scheme of the present invention below in conjunction with accompanying drawing, obviously, described embodiment is the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.

Embodiment 1

Present embodiments provide a kind of method of real-time time calculation gunlock distortion factor, as shown in Figure 1, comprise the steps:

S1: utilize gunlock to obtain a gunlock picture, under utilizing ball machine to remain on same focal length, rotary taking obtains two with Apparatus for feeding balls as disintegrating members image, and at least often the content of shooting of adjacent two ball machine images has overlap, and ball machine image has overlapping with the content of shooting of gunlock image.In general rifle ball linked system, the field range of gunlock is comparatively large, and ball machine carries out rotary taking within sweep of the eye at gunlock.In this step, in the ball machine image of more than two, often the degree of overlapping of the content of shooting of adjacent two ball machine images is preferably about 80%, and the angular speed that can be rotated by control marble forming machine and shooting interval are obtained.In ball machine image, also the overlap of content of shooting can be had between other two ball machine images except adjacent two, if the content degree of overlapping between these two ball machine images meets the demands, the feature of namely mating abundant words of counting also can calculate homography matrix between these two ball machine images.In addition, ball machine distortion factor is relevant to the focal length of ball machine, and therefore, ball machine needs rotary taking under same focal length to obtain ball machine image.

S2: the characteristic point extracting ball machine image respectively, and the characteristic point of mating often adjacent two ball machine images.Sift algorithm specifically can be adopted to extract the characteristic point of ball machine image respectively, and the Euclidean distance adopting knn to search for calculated characteristics point descriptor mate.

S3: set up the homography matrix between corresponding two ball machine images according to the characteristic point of adjacent two ball machine images match respectively, specifically can adopt conventional DLT direct linear transformation, and coordinate ransac algorithm to carry out excluding gross error point.Ball machine distortion factor to be calculated is comprised in this homography matrix.

S4: utilize the homography matrix between adjacent two ball machine images to calculate ball machine distortion factor.Specifically utilize initial homography matrix as restricted model, utilize nonlinear least square method to calculate the distortion factor of ball machine.

S5: the characteristic point extracting a ball machine image and gunlock image respectively, and the characteristic point of mating this ball machine image and this gunlock image.

S6: utilize the characteristic point of mating between ball machine image and gunlock image to set up homography matrix between ball machine image and gunlock image, this homography matrix comprises gunlock distortion factor to be calculated.

S7: utilize ball machine distortion factor, homography matrix between ball machine image and gunlock image to calculate gunlock distortion factor.

The method of returning calculation gunlock distortion factor in real time that the embodiment of the present invention provides, solve the problem that conventional method cannot again be demarcated after rifle ball linked system installs, the distortion factor of change can be calculated along with the change of focal length in real time, meet the demand needing to focus at any time according to monitoring site; And the method does not need the three-dimensional structure of reconstruction point cloud and camera, the coefficient of gunlock picture distortion accurately just can be calculated only by Auto-matching and single should relation, simplify flow process and Mathematical Modeling, efficiency of algorithm is also higher, therefore also can not increase the weight of the computation burden of video camera arithmetic element.

The quantity of the ball machine image obtained in the present embodiment be preferably three and more than, can constraint be increased, to weaken the impact that indivedual big error brings, improve stability and precision that ball machine distortion factor calculates, and then improve the precision of subsequent calculations gunlock distortion factor.Such as, image p1, p2, p3 have overlap, find characteristic point t1 respectively, t2, t3, if t1 and t2 can match, t2 and t3 can match, if but t3 and t1 may to unmatch, the same place of this candidate will be disallowable.Same place refers to an object point in the Feature point correspondence real world mated between different images, and the characteristic point of these couplings is just called same place.

Particularly, as shown in Figure 2, above-mentioned steps S3, namely sets up the step of the homography matrix between corresponding two ball machine images respectively, comprising according to the characteristic point of adjacent two ball machine images match:

S31: the undistorted coordinate obtaining the Feature point correspondence mated between adjacent two ball machine images respectively, the function of this is undistorted coordinate is ball machine distortion factor to be calculated;

S32: the undistorted coordinate corresponding respectively according to the characteristic point of mating between adjacent two ball machine images sets up the homography matrix between these adjacent two ball machine images, and the element of this homography matrix is the function of ball machine distortion factor to be calculated.

Particularly, as shown in Figure 3, above-mentioned steps S31, the step namely obtaining the undistorted coordinate of the Feature point correspondence mated between adjacent two ball machine images respectively comprises:

S311: the characteristic point calculating ball machine image is respectively relative to the coordinate of ball owner point and radius size;

S312: obtain corresponding undistorted coordinate relative to the coordinate of the coordinate of ball owner point and radius size, ball owner point and ball machine distortion factor to be calculated according to the characteristic point of ball machine image, the function of this is undistorted coordinate is ball machine distortion factor to be calculated.

Particularly, in above-mentioned steps S311, a pair characteristic point coordinate wherein mated between adjacent two ball machine images is respectively (x _d, y _d), (x _d', y _d'), ball owner point coordinates is (x ₀, y ₀), wherein a ball machine image characteristic point relative to ball owner point coordinate and radius obtained by following formulae discovery:

$\begin{array}{c}{\tilde{x}}_d=\frac{x_d-x_0}{f}\\ {\tilde{y}}_d=\frac{y_d-y_0}{f}\\ r^2={\tilde{x}}_d^2+{\tilde{y}}_d^2\end{array},$ Wherein, f is the focal length of ball machine when taking this ball machine image, for the characteristic point of a wherein ball machine image is relative to the coordinate of ball owner point.

The characteristic point of another ball machine image is obtained by following formulae discovery relative to the coordinate of ball owner point and radius:

$\begin{array}{c}{\tilde{x}}_d^{\′}=\frac{x_d^{\′}-x_0}{f}\\ {\tilde{y}}_d^{\′}=\frac{y_d^{\′}-y_0}{f}\\ r^{\′2}={\tilde{x}}_d^{\′2}+{\tilde{y}}_d^{\′2}\end{array},$ Wherein, f is the focal length of ball machine when taking this ball machine image, for the coordinate that the individual features point of another ball machine image is put relative to ball owner.

In above-mentioned steps S312, the undistorted coordinate of the Feature point correspondence wherein mated between adjacent two ball machine images is respectively: $\begin{array}{c}{x}_{ud}=f_u\left(x_d\right)=x_0+f\[{\tilde{x}}_d(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)\\ +2p_1{\tilde{x}}_d{\tilde{y}}_d+p_2(r^2+2{\tilde{x}}_d^2)\]\\ y_{ud}=f_u\left(y_d\right)=y_0+f\[{\tilde{y}}_d(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)\\ +2p_2{\tilde{x}}_d{\tilde{y}}_d+p_1(r^2+2{\tilde{y}}_d^2)\]\end{array},$ $\begin{array}{c}{x}_{ud}^{\′}=f_u\left(x_d^{\′}\right)=x_0+f\[{\tilde{x}}_d^{\′}(1+k_1{r}^{\′2}+k_2{r}^{\′4}+k_3{r}^{\′6})\\ +2p_1{\tilde{x}}_d^{\′}{\tilde{y}}_d^{\′}+p_2(r^{\′2}+2{\tilde{x}}_d^{\′2})\]\\ y_{ud}^{\′}=f_u\left(y_d^{\′}\right)=y_0+f\[{\tilde{y}}_d^{\′}(1+k_1{r}^{\′2}+k_2{r}^{\′4}+k_3{r}^{\′6})\\ +2p_2{\tilde{x}}_d^{\′}{\tilde{y}}_d^{\′}+p_1(r^{\′2}+2{\tilde{y}}_d^{\′2})\]\end{array},$ Wherein, k ₁, k ₂, p ₁, p ₂, p ₃for ball machine distortion factor to be calculated.

Equation of singly answering between these adjacent two ball machine images is x ' _ud=Hx _ud, be also f _u(x ' _d)=Hf _u(x _d), wherein H is the homography matrix between these adjacent two ball machine images.Because homography matrix has 9 variablees, wherein have a variable to be set to 1, and the characteristic point of every a pair coupling can list two equations, therefore at least need 4 to the characteristic point of mating.And in order to reduce error, preferably 5 to above equally distributed matching characteristic point.

Singly answered equation can obtain linear fractional function to be by above-mentioned:

$F\left(P\right)=0=f_u\left(x_d^{\′}\right)-{\mathrm{Hf}}_u\left(x_d\right)=(x_{ud}^{\′}-\frac{h_1{x}_{ud}+h_2{y}_{ud}+h_3}{h_7{x}_{ud}+h_8{y}_{ud}+h_9},y_{ud}^{\′}-\frac{h_4{x}_{ud}+h_5{y}_{ud}+h_6}{h_7{x}_{ud}+h_8{y}_{ud}+h_9})$

Wherein, homography matrix $H=\left[\begin{array}{ccc}h1& h2& h3\\ h4& h5& h6\\ h7& h8& h9\end{array}\right].$

According to single order Newton iteration: F (P)=F (P ₀+ Δ)=F (P ₀)+J Δ, as F (P)=F (P ₀+ Δ)=0 time, J Δ=-F (P ₀), this equation is be the iterative equation that nonlinearity in parameters function linearization to be estimated draws by target function.When always having the homography matrix between individual adjacent two the ball machine images of m, parameter vector to be estimated is $\begin{array}{c}P=\left(\begin{array}{ccccccccc}{k}_1& k_2& k_3& p_1& p_2& H_1& H_2& ...& H_m\end{array}\right)\\ H_i=\left(\begin{array}{ccccc}{h}_{i1}& h_{i2}& h_{i3}& ...& h_{i9}\end{array}\right)\end{array},$ Wherein, k ₁, k ₂, k ₃, p ₁, p ₂for ball machine distortion factor, H ₁, H ₂h _mfor m homography matrix, h _i1, h _i2, h _i3h _i9for 9 elements of one of them homography matrix.

Particularly, as shown in Figure 4, step S4, the step namely utilizing the homography matrix between adjacent two ball machine images to calculate ball machine distortion factor comprises:

S41: the initial value presetting ball machine distortion factor, and the initial value of the homography matrix of corresponding acquisition often between adjacent two ball machine images, in the present embodiment, the initial value of ball machine distortion factor is preset as 0, the element of the homography matrix often between adjacent two ball machine images is all the function about ball machine distortion factor, and the initial value of the homography matrix therefore between each adjacent two ball machine images can be determined according to the initial value of ball machine distortion factor.The initial value of ball machine distortion factor also can based on experience value or factory-said value preset.

S42: according to the homography matrix often between adjacent two ball machine images, utilizes single order Newton iteration method to obtain the increment of ball machine distortion factor.

S43: obtain ball machine distortion factor according to the initial value of ball machine distortion factor and the increment of ball machine distortion factor.

In above-mentioned steps S41, the initial value of ball machine distortion factor is preset as 0, priori value or factory-said value time, according to the initial value of the homography matrix between adjacent two ball machine images that the initial value of this ball machine distortion factor calculates be inaccurate (but with the homography matrix of reality be also closer like), namely utilize this homography matrix to be the characteristic point of a wherein ball machine image can not be mapped on the exact position of the same place of another ball machine image.But can, using this homography matrix as initial value, progressively to reduce error for target, utilize nonlinear least square method to calculate homography matrix new more accurately and ball machine distortion factor in follow-up step.

Particularly, the increment of ball machine distortion factor obtains by following formulae discovery:

${\mathrm{\Δ}}_1=-{\left(J_1^T{J}_1\right)}^{-1}{J}_1^{T}F\left(P_0\right),$

${\mathrm{\Δ}}_1=\[\begin{array}{ccccccccc}{\mathrm{dk}}_1& {\mathrm{dk}}_2& {\mathrm{dk}}_3& {\mathrm{dp}}_1& {\mathrm{dp}}_2& {\mathrm{dH}}_1& {\mathrm{dH}}_2& ...& {\mathrm{dH}}_m\end{array}\]$

Wherein,

F (P ₀) for presetting the initial value of the linear fractional function between m corresponding to the initial value of ball machine distortion factor adjacent two ball machine images, the homography matrix one_to_one corresponding of this m linear fractional function respectively and between individual adjacent two the ball machine images of m;

Wherein, $\left\{\begin{array}{c}P=\left(\begin{array}{ccccccccc}{k}_1& k_2& k_3& p_1& p_2& H_1& H_2& ...& H_m\end{array}\right)\\ H_i=\left(\begin{array}{ccccc}{h}_{i1}& h_{i2}& h_{i3}& ...& h_{i9}\end{array}\right)\end{array}\right\},$ Namely P is ball machine distortion factor (k ₁, k ₂, k ₃, p ₁, p ₂) and m homography matrix (H ₁, H ₂h _m); F (P) is linear fractional function corresponding to m homography matrix comprising ball machine distortion factor; N is the number of same place, F ₁₁f _nmit is a n same place corresponding m linear fractional function respectively.

In the present embodiment, each in a said n same place preferably has overlapping ball machine image to choose according at least 3 contents of shooting.After employing sift algorithm or other characteristic point algorithms extract the characteristic point of each ball machine image, carry out the Feature Points Matching between adjacent two ball machine images again, again according to 3 with the same same place that Apparatus for feeding balls as disintegrating members image is corresponding whether can mutually mate on screen the same place of the larger Feature point correspondence of fractional error, to improve the precision calculating ball machine distortion factor and gunlock distortion factor.That is, this n same place needs to choose according to calculating in the candidate's same place be left after the same place of the larger Feature point correspondence of error is fallen in screening.

In basis ${\mathrm{\Δ}}_1=-{\left(J_1^T{J}_1\right)}^{-1}{J}_1^{T}F\left(P_0\right)$ After trying to achieve the increment of the homography matrix between the increment of ball machine distortion factor and m adjacent two ball machine images, ball machine distortion factor can be obtained through successive ignition.

In addition, above-mentioned obtained ball machine distortion factor is reverse distortion factor, conveniently subsequent module goes distortion process to there being the source images resampling of distortion, the pixel of distortionless result images to calculate get back to distortion source images on go for corresponding pixel, therefore also comprise step ball machine distortion factor being converted to forward distortion factor in the present embodiment, comprising:

First, what obtain multiple ball machine image has distorted image coordinate sampled point, and calculates corresponding orthoscopic image coordinate according to ball machine distortion factor, and being specially s has distorted image coordinate sampled point $\left(\begin{array}{cccc}{(x_d,y_d)}_0& {(x_d,y_d)}_1& ...& {(x_d,y_d)}_s\end{array}\right),$ This s s the orthoscopic image coordinate having distorted image coordinate sampled point corresponding is $\left(\begin{array}{cccc}{(x_{ud},y_{ud})}_0& {(x_{ud},y_{ud})}_1& ...& {(x_{ud},y_{ud})}_s\end{array}\right);$

Then, set up orthoscopic image coordinate, forward distortion factor respectively and have the corresponding equation of the coordinate sampled point that distorts, for:

$\left[\begin{array}{cccccc}1& 1& 1& 1& 1& 1\\ {\tilde{x}}_{ud}& {\tilde{x}}_{ud}{r_u}^2& {\tilde{x}}_{ud}{r_u}^4& {\tilde{x}}_{ud}{r_u}^6& 2{\tilde{x}}_{ud}{\tilde{y}}_{ud}& ({r_u}^2+2{\tilde{x}}_{ud}^2)\\ 1& 1& 1& 1& 1& 1\\ {\tilde{y}}_{ud}& {\tilde{y}}_{ud}{r_u}^2& {\tilde{y}}_{ud}{r_u}^4& {\tilde{y}}_{ud}{r_u}^6& ({r_u}^2+2{\tilde{y}}_{ud}^2)& 2{\tilde{x}}_{ud}{\tilde{y}}_{ud}\\ 2& 2& 2& 2& 2& 2\\ {\tilde{x}}_{ud}& {\tilde{x}}_{ud}{r_u}^2& {\tilde{x}}_{ud}{r_u}^4& {\tilde{x}}_{ud}{r_u}^6& 2{\tilde{x}}_{ud}{\tilde{y}}_{ud}& ({r_u}^2+2{\tilde{x}}_{ud}^2)\\ 2& 2& 2& 2& 2& 2\\ {\tilde{y}}_{ud}& {\tilde{y}}_{ud}{r_u}^2& {\tilde{y}}_{ud}{r_u}^4& {\tilde{y}}_{ud}{r_u}^6& ({r_u}^2+2{\tilde{y}}_{ud}^2)& 2{\tilde{x}}_{ud}{\tilde{y}}_{ud}\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ n& n& n& n& n& n\\ {\tilde{x}}_{ud}& {\tilde{x}}_{ud}{r_u}^2& {\tilde{x}}_{ud}{r_u}^4& {\tilde{x}}_{ud}{r_u}^6& 2{\tilde{x}}_{ud}{\tilde{y}}_{ud}& ({r_u}^2+2{\tilde{x}}_{ud}^2)\\ n& n& n& n& n& n\\ {\tilde{y}}_{ud}& {\tilde{y}}_{ud}{r_u}^2& {\tilde{y}}_{ud}{r_u}^4& {\tilde{y}}_{ud}{r_u}^6& ({r_u}^2+2{\tilde{y}}_{ud}^2)& 2{\tilde{x}}_{ud}{\tilde{y}}_{ud}\end{array}\right]\left[\begin{array}{c}1\\ K_1\\ K_2\\ K_3\\ P_1\\ P_2\end{array}\right]=\left[\begin{array}{c}\frac{{\left(x_d\right)}_1-x_0}{f}\\ \frac{{\left(y_d\right)}_1-y_0}{f}\\ \frac{{\left(x_d\right)}_2-x_0}{f}\\ \frac{{\left(y_d\right)}_2-y_0}{f}\\ .\\ .\\ .\\ \frac{{\left(x_d\right)}_n-x_0}{f}\\ \frac{{\left(y_d\right)}_n-y_0}{f}\end{array}\right]$

Wherein, $\begin{array}{c}{\tilde{x}}_{ud}=\frac{x_{ud}-x_0}{f}\\ {\tilde{y}}_{ud}=\frac{y_{ud}-y_0}{f}\\ r_u^2={\tilde{x}}_{ud}^2+{\tilde{y}}_{ud}^2\end{array};$

Finally, according to above-mentioned equation group, solve forward distortion factor (K ₁, K ₂, K ₃, P ₁, P ₂).

Particularly, in step S5, namely the characteristic point of a ball machine image and gunlock image is extracted respectively and to mate in the concrete grammar of the characteristic point of this ball machine image and this gunlock image and above-mentioned steps S2 similar, utilize sift algorithm to extract the characteristic point of this ball machine image and gunlock image respectively, and adopt knn search to mate.The technical scheme that the present embodiment provides in the rifle ball linked system that is suitable for, gunlock is motionless, therefore an image can only be obtained, and ball machine is all generally taking within sweep of the eye at gunlock, so the content of shooting of ball machine image is by all overlapping with the content of shooting of gunlock image, in other application system, the content of shooting of ball machine image also can have very most overlap with the content of shooting of gunlock image.Therefore, can the extraction of carrying out characteristic point of an optional ball machine image and gunlock image with mate.

Particularly, step S6 is similar with step S3, adopt conventional Method of Direct Liner Transformation, calculate the homography matrix between them by ball machine image and the characteristic point of mating between gunlock image, and carry out excluding gross error point with the use of RANSAC (RandomSampleConsensus) algorithm.As shown in Figure 5, detailed process is as follows:

S61: calculate the characteristic point of mating between ball machine image and gunlock image respectively relative to the coordinate of corresponding principal point and radius size;

S62: the undistorted coordinate obtaining this Feature point correspondence according to the characteristic point of ball machine image relative to the principal point coordinate of the coordinate of ball owner point and radius size, ball machine and the above-mentioned ball machine distortion factor calculated; The undistorted coordinate of this Feature point correspondence is obtained relative to the principal point coordinate of the coordinate of gunlock principal point and radius size, gunlock and gunlock distortion factor to be calculated, the function of this is undistorted coordinate is gunlock distortion factor to be calculated according to the characteristic point of gunlock image;

S63: utilize the homography matrix between the undistorted coordinate acquisition gunlock image of the Feature point correspondence of the undistorted coordinate of the Feature point correspondence of above-mentioned gunlock image and ball machine image and ball machine image, the element of this homography matrix is the function of gunlock distortion factor.

Particularly, as shown in Figure 6, in above-mentioned steps S7, the step namely utilizing ball machine distortion factor, homography matrix between ball machine image and gunlock image to calculate gunlock distortion factor comprises:

S71: preset the initial value of gunlock distortion factor, and the initial value of homography matrix between corresponding acquisition ball machine image and gunlock image;

S72: according to the homography matrix between ball machine image and gunlock image, utilizes single order Newton iteration method to obtain the increment of gunlock distortion factor;

S73: obtain gunlock distortion factor according to the distortion factor of ball machine and the increment of gunlock distortion factor.

This step S7 and step S4 is similar, utilizes homography matrix between ball machine image and gunlock image as restricted model, and ball machine distortion factor, as fixing initial value, utilizes nonlinear least square method to calculate the distortion factor of gunlock.

Particularly, the increment of gunlock distortion factor is obtained by following formulae discovery:

${\mathrm{\Δ}}_2=-{\left(J_2^T{J}_2\right)}^{-1}{J}_2^T{F}^{\′}\left(P_0^{\′}\right)$

Wherein, $J_2=\frac{\∂F^{\′}\left(P^{\′}\right)}{\∂P^{\′}}=\left[\begin{array}{cccccc}\frac{\∂F_1^{\′}}{\∂k_1^{\′}}& \frac{\∂F_1^{\′}}{\∂k_2^{\′}}& \frac{\∂F_1^{\′}}{\∂k_3^{\′}}& \frac{\∂F_1^{\′}}{\∂p_1^{\′}}& \frac{\∂F_1^{\′}}{\∂p_2^{\′}}& \frac{\∂F_1^{\′}}{\∂H}\\ \frac{\∂F_2^{\′}}{\∂k_1^{\′}}& \frac{\∂F_2^{\′}}{\∂k_2^{\′}}& \frac{\∂F_2^{\′}}{\∂k_3^{\′}}& \frac{\∂F_2^{\′}}{\∂p_1^{\′}}& \frac{\∂F_2^{\′}}{\∂p_2^{\′}}& \frac{\∂F_2^{\′}}{\∂H}\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ \frac{\∂F_r^{\′}}{\∂k_1^{\′}}& \frac{\∂F_r^{\′}}{\∂k_2^{\′}}& \frac{\∂F_r^{\′}}{\∂k_3^{\′}}& \frac{\∂F_r^{\′}}{\∂p_1^{\′}}& \frac{\∂F_r^{\′}}{\∂p_2^{\′}}& \frac{\∂F_r^{\′}}{\∂H}\end{array}\right],$

F ' (P ' ₀) be the initial value of the linear fractional function between ball machine image corresponding to the initial value of gunlock distortion factor preset and gunlock image, this ball machine image is corresponding with the linear fractional function between gunlock image and its homography matrix;

Wherein, $P^{\′}=\left(\begin{array}{cccccc}{k}_1^{\′}& k_2^{\′}& k_3^{\′}& p_1^{\′}& p_2^{\′}& H^{\′}\end{array}\right),$ Namely P is gunlock distortion factor $\left(\begin{array}{ccccc}{k}_1^{\′}& k_2^{\′}& k_3^{\′}& p_1^{\′}& p_2^{\′}\end{array}\right)$ And the homography matrix (H ') between ball machine image and gunlock image, F ' (P ') is linear fractional function corresponding to the homography matrix between ball machine image and gunlock image; F ' ₁f ' _rit is the linear fractional function between ball machine image and gunlock image that r same place is corresponding respectively.This r same place is also need to select according to calculating after the same place of the larger Feature point correspondence of error is fallen in screening.

Embodiment 2

Originally execute the device that example provides a kind of real-time time calculation gunlock distortion factor, as shown in Figure 7, comprising:

Image acquisition unit U1, a gunlock picture is obtained for utilizing gunlock, under utilizing ball machine to remain on same focal length, rotary taking obtains two with Apparatus for feeding balls as disintegrating members image, and at least often the content of shooting of adjacent two ball machine images has overlap, and ball machine image has overlapping with the content of shooting of gunlock image;

Fisrt feature point extracts and matching unit U2, for extracting the characteristic point of ball machine image respectively, and mates the characteristic point of often adjacent two ball machine images;

First homography matrix sets up unit U3, for setting up the homography matrix between these adjacent two ball machine images according to the characteristic point of adjacent two ball machine images match respectively, comprises ball machine distortion factor to be calculated in this homography matrix;

Ball machine distortion factor computing unit U4, calculates ball machine distortion factor for utilizing the homography matrix between adjacent two ball machine images;

Second feature point extracts and matching unit U5, for extracting the characteristic point of a ball machine image and gunlock image respectively, and mates the characteristic point of this ball machine image and this gunlock image;

Second homography matrix sets up unit U6, and for utilizing the characteristic point of mating between ball machine image and gunlock image to set up homography matrix between ball machine image and gunlock image, this homography matrix comprises gunlock distortion factor to be calculated;

Gunlock distortion factor computing unit U7, calculates gunlock distortion factor for utilizing ball machine distortion factor, homography matrix between ball machine image and gunlock image.

The device returning calculation gunlock distortion factor in real time that the embodiment of the present invention provides, can calculate the distortion factor of change in real time, meet the demand needing to focus at any time according to monitoring site along with the change of focal length.

Particularly, the first homography matrix is set up unit U3 and is comprised:

First undistorted coordinate acquiring unit, for obtaining the undistorted coordinate of the Feature point correspondence mated between adjacent two ball machine images respectively, the function of this is undistorted coordinate is ball machine distortion factor to be calculated;

First homography matrix acquiring unit, sets up the homography matrix between these adjacent two ball machine images for the undistorted coordinate corresponding respectively according to the characteristic point of mating between these adjacent two ball machine images.

Particularly, the first undistorted coordinate acquiring unit comprises:

First computing unit, for the characteristic point that calculates ball machine image respectively relative to the coordinate of ball owner point and radius size;

First acquiring unit, obtains corresponding undistorted coordinate according to the characteristic point of ball machine image relative to the coordinate of the coordinate of ball owner point and radius size, ball owner point and ball machine distortion factor to be calculated.

Particularly, in above-mentioned first computing unit, a pair characteristic point coordinate wherein mated between adjacent two ball machine images is respectively (x _d, y _d), (x _d', y _d'), ball owner point coordinates is (x ₀, y ₀), wherein a ball machine image characteristic point relative to ball owner point coordinate and radius obtained by following formulae discovery:

$\begin{array}{c}{\tilde{x}}_d=\frac{x_d-x_0}{f}\\ {\tilde{y}}_d=\frac{y_d-y_0}{f}\\ r^2={\tilde{x}}_d^2+{\tilde{y}}_d^2\end{array},$ Wherein, f is the focal length of ball machine when taking this ball machine image, for the characteristic point of a wherein ball machine image is relative to the coordinate of ball owner point.

The characteristic point of another ball machine image is obtained by following formulae discovery relative to the coordinate of ball owner point and radius:

$\begin{array}{c}{\tilde{x}}_d^{\′}=\frac{x_d^{\′}-x_0}{f}\\ {\tilde{y}}_d^{\′}=\frac{y_d^{\′}-y_0}{f}\\ r^{\′2}={\tilde{x}}_d^{\′2}+{\tilde{y}}_d^{\′2}\end{array},$ Wherein, f is the focal length of ball machine when taking this ball machine image, for the coordinate that the individual features point of another ball machine image is put relative to ball owner.

In above-mentioned first acquiring unit, the undistorted coordinate of the Feature point correspondence wherein mated between adjacent two ball machine images is respectively: $\begin{array}{c}{x}_{ud}=f_u\left(x_d\right)=x_0+f\[{\tilde{x}}_d(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)\\ +2p_1{\tilde{x}}_d{\tilde{y}}_d+p_2(r^2+2{\tilde{x}}_d^2)\]\\ y_{ud}=f_u\left(y_d\right)=y_0+f\[{\tilde{y}}_d(1+k_1{r}^2+k_2{r}^4+k_3{r}^6)\\ +2p_2{\tilde{x}}_d{\tilde{y}}_d+p_1(r^2+2{\tilde{y}}_d^2)\]\end{array},$ $\begin{array}{c}{x}_{ud}^{\′}=f_u\left(x_d^{\′}\right)=x_0+f\[{\tilde{x}}_d^{\′}(1+k_1{r}^{\′2}+k_2{r}^{\′4}+k_3{r}^{\′6})\\ +2p_1{\tilde{x}}_d^{\′}{\tilde{y}}_d^{\′}+p_2(r^{\′2}+2{\tilde{x}}_d^{\′2})\]\\ y_{ud}^{\′}=f_u\left(y_d^{\′}\right)=y_0+f\[{\tilde{y}}_d^{\′}(1+k_1{r}^{\′2}+k_2{r}^{\′4}+k_3{r}^{\′6})\\ +2p_2{\tilde{x}}_d^{\′}{\tilde{y}}_d^{\′}+p_1(r^{\′2}+2{\tilde{y}}_d^{\′2})\]\end{array},$ Wherein, k ₁, k ₂, k ₃, p ₁, p ₂for ball machine distortion factor.

Particularly, ball machine distortion factor computing unit U4 comprises:

First initial value computing unit, for the initial value of default ball machine distortion factor, and the initial value of the homography matrix of corresponding acquisition often between adjacent two ball machine images;

First incremental computations unit, for according to the homography matrix often between adjacent two ball machine images, utilizes single order Newton iteration method to obtain the increment of ball machine distortion factor;

Ball machine distortion factor acquiring unit, for obtaining ball machine distortion factor according to the initial value of ball machine distortion factor and the increment of ball machine distortion factor.

Particularly, the increment of ball machine distortion factor obtains by following formulae discovery:

${\mathrm{\Δ}}_1=-{\left(J_1^T{J}_1\right)}^{-1}{J}_1^{T}F\left(P_0\right),$

${\mathrm{\Δ}}_1=\[\begin{array}{ccccccccc}{\mathrm{dk}}_1& {\mathrm{dk}}_2& {\mathrm{dk}}_3& {\mathrm{dp}}_1& {\mathrm{dp}}_2& {\mathrm{dH}}_1& {\mathrm{dH}}_2& ...& {\mathrm{dH}}_m\end{array}\]$

Wherein,

F (P ₀) for presetting the initial value of the linear fractional function between m corresponding to the initial value of ball machine distortion factor adjacent two ball machine images, the homography matrix one_to_one corresponding of this m linear fractional function respectively and between individual adjacent two the ball machine images of m;

Wherein, $\left\{\begin{array}{c}P=\left(\begin{array}{ccccccccc}{k}_1& k_2& k_3& p_1& p_2& H_1& H_2& ...& H_m\end{array}\right)\\ H_i=\left(\begin{array}{ccccc}{h}_{i1}& h_{i2}& h_{i3}& ...& h_{i9}\end{array}\right)\end{array}\right\},$ Namely P is ball machine distortion factor (k ₁, k ₂, k ₃, p ₁, p ₂) and m homography matrix (H ₁, H ₂h _m); F (P) is linear fractional function corresponding to m homography matrix comprising ball machine distortion factor; N is the number of same place, F ₁₁f _nmit is a n same place corresponding m linear fractional function respectively.

As other embodiment, also comprise converting unit, comprising:

Computing unit, has distorted image coordinate sampled point for what obtain multiple ball machine image, and calculates corresponding orthoscopic image coordinate according to ball machine distortion factor, and being specially s has distorted image coordinate sampled point $\left(\begin{array}{cccc}{(x_d,y_d)}_0& {(x_d,y_d)}_1& ...& {(x_d,y_d)}_s\end{array}\right)$ , this s s the orthoscopic image coordinate having distorted image coordinate sampled point corresponding is $\left(\begin{array}{cccc}{(x_{ud},y_{ud})}_0& {(x_{ud},y_{ud})}_1& ...& {(x_{ud},y_{ud})}_s\end{array}\right)$ ；

Establishing equation unit, for setting up orthoscopic image coordinate, forward distortion factor respectively and having the corresponding equation of the coordinate sampled point that distorts, for: $\left[\begin{array}{cccccc}1& 1& 1& 1& 1& 1\\ {\tilde{x}}_{ud}& {\tilde{x}}_{ud}{r}^{\′2}& {\tilde{x}}_{ud}{r}^{\′4}& {\tilde{x}}_{ud}{r}^{\′6}& 2{\tilde{x}}_{ud}{\tilde{y}}_{ud}& (r^{\′2}+2{\tilde{x}}_{ud}^2)\\ 1& 1& 1& 1& 1& 1\\ {\tilde{y}}_{ud}& {\tilde{y}}_{ud}{r}^{\′2}& {\tilde{y}}_{ud}{r}^{\′4}& {\tilde{y}}_{ud}{r}^{\′6}& (r^{\′2}+2{\tilde{y}}_{ud}^2)& 2{\tilde{x}}_{ud}{\tilde{y}}_{ud}\\ 2& 2& 2& 2& 2& 2\\ {\tilde{x}}_{ud}& {\tilde{x}}_{ud}{r}^{\′2}& {\tilde{x}}_{ud}{r}^{\′4}& {\tilde{x}}_{ud}{r}^{\′6}& 2{\tilde{x}}_{ud}{\tilde{y}}_{ud}& (r^{\′2}+2{\tilde{x}}_{ud}^2)\\ 2& 2& 2& 2& 2& 2\\ {\tilde{y}}_{ud}& {\tilde{y}}_{ud}{r}^{\′2}& {\tilde{y}}_{ud}{r}^{\′4}& {\tilde{y}}_{ud}{r}^{\′6}& (r^{\′2}+2{\tilde{y}}_{ud}^2)& 2{\tilde{x}}_{ud}{\tilde{y}}_{ud}\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ s& s& s& s& s& s\\ {\tilde{x}}_{ud}& {\tilde{x}}_{ud}{r}^{\′2}& {\tilde{x}}_{ud}{r}^{\′4}& {\tilde{x}}_{ud}{r}^{\′6}& 2{\tilde{x}}_{ud}{\tilde{y}}_{ud}& (r^{\′2}+2{\tilde{x}}_{ud}^2)\\ s& s& s& s& s& s\\ {\tilde{y}}_{ud}& {\tilde{y}}_{ud}{r}^{\′2}& {\tilde{y}}_{ud}{r}^{\′4}& {\tilde{y}}_{ud}{r}^{\′6}& (r^{\′2}+2{\tilde{y}}_{ud}^2)& 2{\tilde{x}}_{ud}{\tilde{y}}_{ud}\end{array}\right]\left[\begin{array}{c}1\\ K_1\\ K_2\\ K_3\\ P_1\\ P_2\end{array}\right]=\left[\begin{array}{c}\frac{{\left(x_d\right)}_1-x_0}{f}\\ \frac{{\left(y_d\right)}_1-y_0}{f}\\ \frac{{\left(x_d\right)}_2-x_0}{f}\\ \frac{{\left(y_d\right)}_2-y_0}{f}\\ .\\ .\\ .\\ \frac{{\left(x_d\right)}_s-x_0}{f}\\ \frac{{\left(y_d\right)}_s-y_0}{f}\end{array}\right];$

Forward distortion factor acquiring unit, for solving forward distortion factor (K ₁, K ₂, K ₃, P ₁, P ₂).

Particularly, gunlock distortion factor computing unit U7 comprises:

Second initial value computing unit, for the initial value of default gunlock distortion factor, and the initial value of homography matrix between corresponding acquisition ball machine image and gunlock image;

Second incremental computations unit, for according to the homography matrix between ball machine image and gunlock image, utilizes single order Newton iteration method to obtain the increment of gunlock distortion factor;

Gunlock distortion factor acquiring unit, for obtaining gunlock distortion factor according to the distortion factor of ball machine and the increment of gunlock distortion factor.

Particularly, the increment of gunlock distortion factor is obtained by following formulae discovery:

${\mathrm{\Δ}}_2=-{\left(J_2^T{J}_2\right)}^{-1}{J}_2^T{F}^{\′}\left(P_0^{\′}\right)$

Wherein, $J_2=\frac{\∂F^{\′}\left(P^{\′}\right)}{\∂P^{\′}}=\left[\begin{array}{cccccc}\frac{\∂F_1^{\′}}{\∂k_1^{\′}}& \frac{\∂F_1^{\′}}{\∂k_2^{\′}}& \frac{\∂F_1^{\′}}{\∂k_3^{\′}}& \frac{\∂F_1^{\′}}{\∂p_1^{\′}}& \frac{\∂F_1^{\′}}{\∂p_2^{\′}}& \frac{\∂F_1^{\′}}{\∂H}\\ \frac{\∂F_2^{\′}}{\∂k_1^{\′}}& \frac{\∂F_2^{\′}}{\∂k_2^{\′}}& \frac{\∂F_2^{\′}}{\∂k_3^{\′}}& \frac{\∂F_2^{\′}}{\∂p_1^{\′}}& \frac{\∂F_2^{\′}}{\∂p_2^{\′}}& \frac{\∂F_2^{\′}}{\∂H}\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ \frac{\∂F_r^{\′}}{\∂k_1^{\′}}& \frac{\∂F_r^{\′}}{\∂k_2^{\′}}& \frac{\∂F_r^{\′}}{\∂k_3^{\′}}& \frac{\∂F_r^{\′}}{\∂p_1^{\′}}& \frac{\∂F_r^{\′}}{\∂p_2^{\′}}& \frac{\∂F_r^{\′}}{\∂H}\end{array}\right],$

F ' (P ' ₀) be the initial value of the linear fractional function between ball machine image corresponding to the initial value of gunlock distortion factor preset and gunlock image, this ball machine image is corresponding with the linear fractional function between gunlock image and its homography matrix;

Wherein, $P^{\′}=\left(\begin{array}{cccccc}{k}_1^{\′}& k_2^{\′}& k_3^{\′}& p_1^{\′}& p_2^{\′}& H^{\′}\end{array}\right)$ , namely P is gunlock distortion factor $\left(\begin{array}{ccccc}{k}_1^{\′}& k_2^{\′}& k_3^{\′}& p_1^{\′}& p_2^{\′}\end{array}\right)$ And the homography matrix (H ') between ball machine image and gunlock image, F ' (P ') is linear fractional function corresponding to the homography matrix between ball machine image and gunlock image; F ' ₁f ' _rit is the linear fractional function between ball machine image and gunlock image that r same place is corresponding respectively.

Obviously, above-described embodiment is only for clearly example being described, and the restriction not to execution mode.For those of ordinary skill in the field, can also make other changes in different forms on the basis of the above description.Here exhaustive without the need to also giving all execution modes.And thus the apparent change of extending out or variation be still among the protection range of the invention.

Claims (14)

1. return a method for calculation gunlock distortion factor in real time, it is characterized in that, comprise the steps:

Gunlock is utilized to obtain a gunlock picture, under utilizing ball machine to remain on same focal length, rotary taking obtains two with Apparatus for feeding balls as disintegrating members image, at least often the content of shooting of adjacent two described ball machine images has overlap, and described ball machine image has overlapping with the content of shooting of described gunlock image;

Extract the characteristic point of described ball machine image respectively, and mate the characteristic point of often adjacent two described ball machine images;

Set up the homography matrix between these adjacent two ball machine images according to the characteristic point of adjacent two described ball machine images match respectively, in this homography matrix, comprise ball machine distortion factor to be calculated;

The homography matrix between described adjacent two ball machine images is utilized to calculate described ball machine distortion factor;

Extract the characteristic point of a described ball machine image and described gunlock image respectively, and mate the characteristic point of this ball machine image and this gunlock image;

Utilize the characteristic point of mating between described ball machine image and described gunlock image to set up homography matrix between described ball machine image and described gunlock image, this homography matrix comprises described gunlock distortion factor to be calculated;

Utilize described ball machine distortion factor, homography matrix between described ball machine image and described gunlock image calculates described gunlock distortion factor.

2. method according to claim 1, is characterized in that, the step of the described homography matrix set up between these adjacent two ball machine images according to the characteristic point of adjacent two described ball machine images match respectively comprises:

Obtain the undistorted coordinate of the Feature point correspondence mated between adjacent two ball machine images respectively, the function of this is undistorted coordinate is described ball machine distortion factor to be calculated;

The undistorted coordinate corresponding respectively according to the characteristic point of mating between these adjacent two described ball machine images sets up the homography matrix between these adjacent two ball machine images.

3. method as claimed in claim 2, it is characterized in that, the described step obtaining the undistorted coordinate of the Feature point correspondence mated between adjacent two ball machine images respectively comprises:

The characteristic point calculating described ball machine image is respectively relative to the coordinate of ball owner point and radius size;

Corresponding undistorted coordinate is obtained relative to the coordinate of the coordinate of described ball owner point and radius size, described ball owner point and described ball machine distortion factor to be calculated according to the characteristic point of described ball machine image.

4. the method according to any one of claim 1-3, is characterized in that, the described step utilizing the homography matrix between described adjacent two ball machine images to calculate described ball machine distortion factor comprises:

Preset the initial value of described ball machine distortion factor, and the initial value of the homography matrix of corresponding acquisition often between adjacent two ball machine images;

According to the homography matrix often between adjacent two ball machine images, single order Newton iteration method is utilized to obtain the increment of described ball machine distortion factor;

Described ball machine distortion factor is obtained according to the initial value of described ball machine distortion factor and the increment of described ball machine distortion factor.

5. method according to claim 4, is characterized in that, the increment of described ball machine distortion factor is obtained by following formulae discovery:

${\mathrm{\Δ}}_1=-{\left(J_1^T{J}_1\right)}^{-1}{J}_1^{T}F\left(P_0\right),$

Wherein,

F (P ₀) for presetting the initial value of the linear fractional function between m corresponding to the initial value of ball machine distortion factor adjacent two ball machine images, the homography matrix one_to_one corresponding of this m linear fractional function respectively and between individual adjacent two the ball machine images of m;

Wherein, $\left\{\begin{array}{c}P=\left(\begin{array}{ccccccccc}{k}_1& k_2& k_3& p_1& p_2& H_1& H_2& ...& H_m\end{array}\right)\\ H_i=\left(\begin{array}{ccccc}{h}_{i1}& h_{i2}& h_{i3}& ...& h_{i9}\end{array}\right)\end{array}\right\},$ Namely P is described ball machine distortion factor (k ₁, k ₂, k ₃, p ₁, p ₂) and m homography matrix (H ₁, H ₂h _m); F (P) is linear fractional function corresponding to m homography matrix comprising described ball machine distortion factor; N is the number of same place, F ₁₁f _nmit is a n same place corresponding m linear fractional function respectively.

6. the method according to any one of claim 1-5, is characterized in that, also comprises the step described ball machine distortion factor being converted to forward distortion factor:

What obtain multiple ball machine image has distorted image coordinate sampled point, and calculates corresponding orthoscopic image coordinate according to described ball machine distortion factor;

Set up described orthoscopic image coordinate, described forward distortion factor and the described corresponding equation having the coordinate sampled point that distorts respectively;

Solve described forward distortion factor.

7. the method according to any one of claim 1-6, is characterized in that, describedly utilizes described ball machine distortion factor, step that homography matrix between described ball machine image and described gunlock image calculates described gunlock distortion factor comprises:

Preset the initial value of described gunlock distortion factor, and the initial value of homography matrix between corresponding acquisition described ball machine image and gunlock image;

According to the homography matrix between described ball machine image and gunlock image, single order Newton iteration method is utilized to obtain the increment of described gunlock distortion factor;

Described gunlock distortion factor is obtained according to the distortion factor of described ball machine and the increment of described gunlock distortion factor.

8. the method according to any one of claim 1-7, is characterized in that, the increment of described gunlock distortion factor is obtained by following formulae discovery:

${\mathrm{\Δ}}_2=-{\left(J_2^T{J}_2\right)}^{-1}{J}_2^T{F}^{\′}\left(P_0^{\′}\right)$

Wherein, $J_2=\frac{\∂F^{\′}\left(P^{\′}\right)}{\∂P^{\′}}=\left[\begin{array}{cccccc}\frac{\∂F_1^{\′}}{\∂k_1^{\′}}& \frac{\∂F_1^{\′}}{\∂k_2^{\′}}& \frac{\∂F_1^{\′}}{\∂k_3^{\′}}& \frac{\∂F_1^{\′}}{\∂p_1^{\′}}& \frac{\∂F_1^{\′}}{\∂p_2^{\′}}& \frac{\∂F_1^{\′}}{\∂H}\\ \frac{\∂F_2^{\′}}{\∂k_1^{\′}}& \frac{\∂F_2^{\′}}{\∂k_2^{\′}}& \frac{\∂F_2^{\′}}{\∂k_3^{\′}}& \frac{\∂F_2^{\′}}{\∂p_1^{\′}}& \frac{\∂F_2^{\′}}{\∂p_2^{\′}}& \frac{\∂F_2^{\′}}{\∂H}\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ .& .& .& .& .& .\\ \frac{\∂F_r^{\′}}{\∂k_1^{\′}}& \frac{\∂F_r^{\′}}{\∂k_2^{\′}}& \frac{\∂F_r^{\′}}{\∂k_3^{\′}}& \frac{\∂F_r^{\′}}{\∂p_1^{\′}}& \frac{\∂F_r^{\′}}{\∂p_2^{\′}}& \frac{\∂F_r^{\′}}{\∂H}\end{array}\right],$

the initial value of the linear fractional function between the ball machine image corresponding for the initial value of the gunlock distortion factor preset and gunlock image, this ball machine image is corresponding with the linear fractional function between gunlock image and its homography matrix;

Wherein, P '=(k ' ₁k ' ₂k ' ₃p ' ₁p ' ₂h), namely P be described gunlock distortion factor (k ' ₁k ' ₂k ' ₃p ' ₁p ' ₂) and homography matrix (H ') between described ball machine image and gunlock image, F (P ') is linear fractional function corresponding to the homography matrix between described ball machine image and gunlock image; F ' ₁f ' _rit is the linear fractional function between described ball machine image and gunlock image that r same place is corresponding respectively.

9. return a device for calculation gunlock distortion factor in real time, it is characterized in that, comprising:

Image acquisition unit, a gunlock picture is obtained for utilizing gunlock, under utilizing ball machine to remain on same focal length, rotary taking obtains two with Apparatus for feeding balls as disintegrating members image, at least often the content of shooting of adjacent two described ball machine images has overlap, and described ball machine image has overlapping with the content of shooting of described gunlock image;

Fisrt feature point extracts and matching unit, for extracting the characteristic point of described ball machine image respectively, and mates the characteristic point of often adjacent two described ball machine images;

First homography matrix sets up unit, for setting up the homography matrix between these adjacent two ball machine images according to the characteristic point of adjacent two described ball machine images match respectively, comprises ball machine distortion factor to be calculated in this homography matrix;

Ball machine distortion factor computing unit, calculates described ball machine distortion factor for utilizing the homography matrix between described adjacent two ball machine images;

Second feature point extracts and matching unit, for extracting the characteristic point of a described ball machine image and described gunlock image respectively, and mates the characteristic point of this ball machine image and this gunlock image;

Second homography matrix sets up unit, and for utilizing the characteristic point of mating between described ball machine image and described gunlock image to set up homography matrix between described ball machine image and described gunlock image, this homography matrix comprises described gunlock distortion factor to be calculated;

Gunlock distortion factor computing unit, for utilizing described ball machine distortion factor, homography matrix between described ball machine image and described gunlock image calculates described gunlock distortion factor.

10. device according to claim 9, is characterized in that, described first homography matrix is set up unit and comprised:

First undistorted coordinate acquiring unit, for obtaining the undistorted coordinate of the Feature point correspondence mated between adjacent two ball machine images respectively, the function of this is undistorted coordinate is described ball machine distortion factor to be calculated;

First homography matrix acquiring unit, sets up the homography matrix between these adjacent two ball machine images for the undistorted coordinate corresponding respectively according to the characteristic point of mating between these adjacent two described ball machine images.

11. methods as described in claim 9 or 10, it is characterized in that, described first undistorted coordinate acquiring unit comprises:

First computing unit, for the characteristic point that calculates described ball machine image respectively relative to the coordinate of ball owner point and radius size;

First acquiring unit, obtains corresponding undistorted coordinate according to the characteristic point of described ball machine image relative to the coordinate of the coordinate of described ball owner point and radius size, described ball owner point and described ball machine distortion factor to be calculated.

12. devices according to any one of claim 9-11, it is characterized in that, described ball machine distortion factor computing unit comprises:

First initial value computing unit, for presetting the initial value of described ball machine distortion factor, and the initial value of the homography matrix of corresponding acquisition often between adjacent two ball machine images;

First incremental computations unit, for according to the homography matrix often between adjacent two ball machine images, utilizes single order Newton iteration method to obtain the increment of described ball machine distortion factor;

Ball machine distortion factor acquiring unit, for obtaining described ball machine distortion factor according to the initial value of described ball machine distortion factor and the increment of described ball machine distortion factor.

13. devices according to any one of claim 9-12, is characterized in that, also comprise converting unit, comprising:

Computing unit, has distorted image coordinate sampled point for what obtain multiple ball machine image, and calculates corresponding orthoscopic image coordinate according to described ball machine distortion factor;

Establishing equation unit, for setting up described orthoscopic image coordinate, described forward distortion factor and the described corresponding equation having the coordinate sampled point that distorts respectively;

Forward distortion factor acquiring unit, for solving described forward distortion factor.

14. devices according to any one of claim 9-13, it is characterized in that, described gunlock distortion factor computing unit comprises:

Second initial value computing unit, for presetting the initial value of described gunlock distortion factor, and the initial value of homography matrix between corresponding acquisition described ball machine image and gunlock image;

Second incremental computations unit, for according to the homography matrix between described ball machine image and gunlock image, utilizes single order Newton iteration method to obtain the increment of described gunlock distortion factor;

Gunlock distortion factor acquiring unit, for obtaining described gunlock distortion factor according to the distortion factor of described ball machine and the increment of described gunlock distortion factor.