CN105488766B - Fisheye image bearing calibration and device - Google Patents
Fisheye image bearing calibration and device Download PDFInfo
- Publication number
- CN105488766B CN105488766B CN201510819265.9A CN201510819265A CN105488766B CN 105488766 B CN105488766 B CN 105488766B CN 201510819265 A CN201510819265 A CN 201510819265A CN 105488766 B CN105488766 B CN 105488766B
- Authority
- CN
- China
- Prior art keywords
- fish eye
- coordinate system
- image
- coordinate
- fisheye
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 241000251468 Actinopterygii Species 0.000 claims abstract description 137
- 238000012937 correction Methods 0.000 claims description 39
- 238000000034 method Methods 0.000 claims description 10
- 238000003384 imaging method Methods 0.000 claims description 9
- 230000000007 visual effect Effects 0.000 claims description 7
- 238000012897 Levenberg–Marquardt algorithm Methods 0.000 claims description 6
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 230000014509 gene expression Effects 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 4
- 238000013139 quantization Methods 0.000 claims description 4
- 238000013519 translation Methods 0.000 claims description 4
- 230000000694 effects Effects 0.000 abstract description 4
- 238000002594 fluoroscopy Methods 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 12
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 210000003128 head Anatomy 0.000 description 5
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 239000011521 glass Substances 0.000 description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 230000008878 coupling Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 238000003702 image correction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/80—Geometric correction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20172—Image enhancement details
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The present invention provides a kind of fisheye image bearing calibration and device.Fisheye image bearing calibration of the present invention, including：According to the collected uncalibrated image of fish eye lens, the model parameter of panorama picture of fisheye lens model is obtained；Fish-eye effective angular field of view is determined according to the model parameter and default constraints；The fisheye image acquired in effective angular field of view to fish eye lens is corrected, the fisheye image after being corrected.The present invention can be reduced to the fisheye image of distortion the fluoroscopy images for meeting human eye vision effect.
Description
Technical field
The present invention relates to digital image processing techniques field more particularly to a kind of fisheye image bearing calibration and dresses
It puts.
Background technology
Fish eye lens is a kind of bugeye lens, and visual angle generally reaches or more than 180 degree, diameter of lens is very short and is in throw
Object shape protrudes therefore named fish eye lens similar with the eyes of fish to camera lens forepart.Fish eye lens is in video monitoring, virtual reality, three
Dimension modeling, vision guided navigation field are widely used.But hemispherical object plane is imaged as plane by fish eye lens, so flake mirror
Head captured by image Severe distortion, visually experience very unnatural, it is therefore desirable to by these distorted image corrections be people
The perspective projection image being accustomed to.
Restore pattern distortion in the prior art and fish eye lens calibration algorithm may be used, it is specific by establishing fish eye lens
Then imaging model obtains fish-eye correction parameter with caliberating device, finally using the correction parameter of acquisition come correction chart
Picture.Being generally designated device is needed corresponding calibration masterplate and the calibration masterplate is shot using fish eye lens, according to
The image of shooting obtains fish-eye correction parameter, demarcates masterplate such as plane black and white chessboard.But usual fish eye lens pair
In the image of the black and white chessboard shooting, black and white chessboard cannot be covered with the image entirely shot, the school that caliberating device is caused to obtain
Positive parameter is not necessarily global optimum, therefore may serious distortion based on the calibrated correction image of fish eye lens.
Invention content
The present invention provides a kind of fisheye image bearing calibration and device, and fish eye lens is based in the prior art to overcome
It is calibrated correction image may serious distortion the problem of.
In a first aspect, the present invention provides a kind of fisheye image bearing calibration, including：
According to the collected uncalibrated image of fish eye lens, the model parameter of panorama picture of fisheye lens model is obtained；
Fish-eye effective angular field of view is determined according to the model parameter and default constraints；
The fisheye image acquired in effective angular field of view to fish eye lens is corrected, after obtaining correction
Fisheye image.
Second aspect, the present invention provide a kind of fisheye image means for correcting, including：
Acquisition module, for according to the collected uncalibrated image of fish eye lens, obtaining the model of panorama picture of fisheye lens model
Parameter；
Determining module, for determining fish-eye effective visual angle model according to the model parameter and default constraints
It encloses；
Correction module, the fisheye image for being acquired in effective angular field of view to fish eye lens carry out school
Just, the fisheye image after being corrected.
Fisheye image bearing calibration of the present invention and device, by according to the collected uncalibrated image of fish eye lens, obtaining
The model parameter of panorama picture of fisheye lens model is taken, and is determined according to the model parameter and default constraints fish-eye
Effective angular field of view；The fisheye image acquired in effective angular field of view to fish eye lens is corrected, and is obtained
Fisheye image after correction since the black and white chessboard in uncalibrated image cannot be covered with entire uncalibrated image, is got
Model parameter be not necessarily global optimum, compared to existing technologies, the present invention in after model parameter is determined, further
Determine fish-eye effective angular field of view, the fisheye image acquired in effective angular field of view to fish eye lens
It is corrected, therefore, it is possible to the fisheye image of distortion is reduced to the fluoroscopy images for meeting human eye vision effect, solves
In the prior art based on fish eye lens it is calibrated correction image may serious distortion the problem of.
Description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, to embodiment or will show below
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is this hair
Some bright embodiments, for those of ordinary skill in the art, without having to pay creative labor, can be with
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is the flow diagram of one embodiment of fisheye image bearing calibration of the present invention；
Fig. 2 is the relation schematic diagram of fish eye lens coordinate system and fish eye images plane coordinate system；
Fig. 3 is the structure diagram of one embodiment of fisheye image means for correcting of the present invention.
Specific embodiment
Purpose, technical scheme and advantage to make the embodiment of the present invention are clearer, below in conjunction with the embodiment of the present invention
In attached drawing, the technical solution in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is
Part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
All other embodiments obtained without creative efforts shall fall within the protection scope of the present invention.
Fig. 1 is the flow diagram of one embodiment of fisheye image bearing calibration of the present invention.Fig. 2 is fish eye lens coordinate
System and the relation schematic diagram of fish eye images plane coordinate system.As shown in Figure 1, the method for the present embodiment includes：
Step 101, according to the collected uncalibrated image of fish eye lens, obtain the model parameter of panorama picture of fisheye lens model；
Step 102 determines fish-eye effective angular field of view according to the model parameter and default constraints；
Step 103, the fisheye image acquired in effective angular field of view to fish eye lens are corrected, and are obtained
Fisheye image after to correction.
Specifically, plane chessboard calibration plate is made by oneself, with fish-eye video camera to the acquisition n width calibration of chessboard calibration plate
Image, n>>1；Tessellated black and white is chosen on every uncalibrated image and intersects vertex, and obtain the world coordinates on those vertex
The coordinate under coordinate and fish eye lens coordinate under system, and obtained according to the coordinate and panorama picture of fisheye lens model on those vertex
The model parameter of the panorama picture of fisheye lens model.
As shown in Fig. 2, OcXcYcZc is fish eye lens coordinate system, OwXwYwZw is world coordinate system, it is assumed that world coordinates
It is next point Pw (Xw, Yw, Zw), Xw is the coordinate value of Pw Xw axis under world coordinate system, and Yw is Pw under world coordinate system
The coordinate value of Yw axis, Zw are the coordinate value of Pw Zw axis under world coordinate system；Coordinate of the Pw points under fish eye lens coordinate system be
Pc (Xc, Yc, Zc), Xc are the coordinate value of Pc Xc axis under fish eye lens coordinate system, and Yc is Pc Yc under fish eye lens coordinate system
The coordinate value of axis, Zc are the coordinate value of Pc Zc axis under fish eye lens coordinate system；Line segmentWith the Zc of fish eye lens coordinate system
Axis positive direction is (i.e.) angle be θ；OiXiYiZi be fish eye images plane coordinate system, f be fish-eye focal length, Pi
For fish eye lens acquisition image on point corresponding with the Pc, it is assumed that Pr (a, b) be correct after image on described Pc pairs
The point answered, enables θdDistance for pixel Pi and fish eye images plane coordinate system origin Oi；A and b is respectively the Pr points in flake
Level (Xi axis) and vertical (Yi axis) coordinate (space actual coordinate value) under plane of delineation coordinate system；R is the distance of Pr to Oi.
Panorama picture of fisheye lens model has following several：
[1]For stereo projection (stereographic projection), using seldom；
[2]For waiting three-dimensional angular projection (equisolid angle projection), civilian fish
Glasses head；
[3]θd=kf sin (θ)；For rectangular projection (orthogonal projection), a small number of camera lenses use；
[4]θd=kf θ；It is the most frequently used for equidistant projection (equidistanee projection).
Wherein, k is the coefficient of panorama picture of fisheye lens model.
General pearl eye lens imaging model will can be obtained after above-mentioned model Taylor expansion：θd=k1θ+k3θ3+k5θ5+k7θ7+
k9θ9+...；
Wherein, k1、k2、k3、k4Coefficient for the panorama picture of fisheye lens model.
It is illustrated by taking general pearl eye lens imaging model as an example in the embodiment of the present invention.
Obtain the model parameter of the panorama picture of fisheye lens model, first initialization model parameter；Then Burger-Ma Kua is used
Special Levenberg-Marquardt algorithm optimizations cost function obtains model parameter.The algorithm is widely used in non-linear minimum
Change problem, it is the deformation of one Newton's algorithm of Gauss.It between Newton method and gradient descent method, for parametrization the problem of
It is insensitive, nuisance parameter problem can be effectively handled, greatly reduces the chance that cost function is absorbed in local minimum.
During model parameter is obtained, due to not to the coefficient k of panorama picture of fisheye lens model1、k2、k3、k4It carries out about
Beam, therefore obtained model parameter is not necessarily global optimum.From panorama picture of fisheye lens model：θ is in the range of 0 and pi/2, θd
More than 0 and less than θ, and it is the monotonically increasing function of the θ；According to known models parameter k obtained by calibrating1、k2、k3、k4, with
And variations of the θ in the range of 0 and pi/2 draws θdFigure, take out more than 0 and understood less than the part of θ and monotonic increase
The effective angular field of view in road, so as to know maximum effective visual angle.
The fisheye image acquired in effective angular field of view to fish eye lens is corrected, after obtaining correction
Fisheye image.Bilinear interpolation specifically may be used to be corrected.
Fisheye image bearing calibration provided in this embodiment, by according to the collected uncalibrated image of fish eye lens,
The model parameter of panorama picture of fisheye lens model is obtained, and fish eye lens is determined according to the model parameter and default constraints
Effective angular field of view；The fisheye image acquired in effective angular field of view to fish eye lens is corrected, and is obtained
Fisheye image after to correction since the black and white chessboard in uncalibrated image cannot be covered with entire uncalibrated image, obtains
To model parameter be not necessarily global optimum, compared to existing technologies, the present invention in after model parameter is determined, into one
Step determines fish-eye effective angular field of view, the fish eye lens figure acquired in effective angular field of view to fish eye lens
As being corrected, therefore, it is possible to the fisheye image of distortion is reduced to the fluoroscopy images for meeting human eye vision effect, solve
In the prior art based on the calibrated correction image of fish eye lens may serious distortion the problem of.
On the basis of above-described embodiment, optionally, before step 101, it can proceed as follows：
The panorama picture of fisheye lens model is established, and establishes fish eye lens projection on the panorama picture of fisheye lens model and closes
System；
The fish eye lens projection relation such as formula (1)：
Wherein, θd=θ (1+k1θ2+k2θ4+k3θ6+k4θ8) it is the panorama picture of fisheye lens model；k1、k2、k3、k4It is described
The coefficient of panorama picture of fisheye lens model, θdFor the pixel Pi of the fisheye image and fish eye images plane coordinate system origin
Distance；θ is line segmentWith the angle of the Zc axis positive directions of fish eye lens coordinate system；Oc is the original of fish eye lens coordinate system
Point, Pc are coordinate points Pc (Xc, Yc, Zc) corresponding with the Pi under fish eye lens coordinate system；Wherein, Xc is Pc in fish eye lens
The coordinate value of Xc axis under coordinate system, Yc are the coordinate value of Pc Yc axis under fish eye lens coordinate system, and Zc is sat for Pc in fish eye lens
The coordinate value of the lower Zc axis of mark system；Pw (Xw, Yw, Zw) is Pc (Xc, Yc, Zc) corresponding points under world coordinate system；Xw exists for Pw
The coordinate value of Xw axis under world coordinate system, Yw are the coordinate value of Pw Yw axis under world coordinate system, and Zw is Pw in world coordinate system
The coordinate value of lower Zw axis；R is spin matrix, and T is translation vector；Fx and fy is respectively fish eye images plane coordinate system to flake mirror
The quantization parameter horizontally and vertically of head coordinate system, u and v are respectively water of the Pc under fish eye images plane coordinate system
It is respectively that fisheye image center is horizontal and vertical under fish eye images plane coordinate system that gentle vertical coordinate, cx and cy, which are,
Coordinate；X' and y' is intermediate variable；A, b, r are as described in above-described embodiment.
Optionally, following manner realization specifically may be used in step 101：
N width uncalibrated images are acquired using fish eye lens, obtain the uncalibrated image in world coordinate system and in fish eye images
N × m group corresponding points under plane coordinate system；The coordinate of n × m groups corresponding points is (Xwij, Ywij, 0) and (uij, vij)；
Wherein, uncalibrated image described in every width includes the tessellated vertex of m black and white；The value range of i is [1, n], and the value range of j is
[1, m]；Xwij is the coordinate value of the i-th j-th point of Xw axis under world coordinate system of width image, and Ywij is j-th point of the i-th width image
The coordinate value of Yw axis under world coordinate system；Uij and vij is respectively j-th point of the i-th width image in fish eye images plane coordinate system
Under horizontal and vertical coordinate；N and m is the integer more than 1；
According to n × m groups corresponding points the fish is determined using Burger-Marquart Levenberg-Marquardt algorithms
The model parameter of glasses head imaging model；The model parameter includes：R,T,k1,k2,k3,k4,fx,fy,cx,cy；
Wherein, the function that min () expressions are minimized；
mijRepresent the i-th j-th point of coordinate under fish eye images plane coordinate system of width image,Represent j-th point of the i-th width image under world coordinate system coordinate (Xwij,
Ywij, 0) coordinate under fish eye images plane coordinate system being calculated by formula (1).
Specifically, black and white tessellated vertex of n × m groups corresponding points for n black and white chessboard, the seat of n × m group corresponding points
It is designated as (Xwij, Ywij, 0) and (uij, vij)；Xwij is the coordinate of the i-th j-th point of Xw axis under world coordinate system of width image
Value, Ywij are the coordinate value of the i-th j-th point of Yw axis under world coordinate system of width image；0 for j-th point of the i-th width image in the world
The coordinate value of Zw axis under coordinate system；Uij and vij is respectively the i-th j-th point of water under fish eye images plane coordinate system of width image
Gentle vertical coordinate.
Initialization model parameter R first is unit battle array, and T is three-dimensional null vector, k1、k2、k3、k4Be zero, cx it is fish eye lens
The horizontal centre coordinate of image, cy are the vertical centre coordinate of fisheye image, and fx and fy are 2000, are then used
Levenberg-Marquardt algorithm optimizations cost function obtains model parameter.
Wherein, model parameter R and T belongs to external parameter, k1, k2, k3, k4, fx, fy, and cx, cy belong to inner parameter,
Only inner parameter need to be used during correction.
For by (Xwij, Ywij, 0) substitute into formula (1) be calculated
Coordinate under fish eye images plane coordinate systemmijFor coordinate (uij, vij), calculate 2 points of distance and cause minimum, into
After row successive ignition, optimal model parameter can be obtained.
Optionally, step 102 can be realized specifically in the following way：
According to the model parameter, the default initial range of θ and θdConstraints determine effective angular field of view；It is described
The effective range that effective angular field of view is the θ；
Wherein, the θdConstraints be θdMore than 0, and less than θ, and it is the monotonically increasing function of the θ；
The default initial range of θ is [0, pi/2].
Specifically, during model parameter is obtained, due to not to the coefficient k of panorama picture of fisheye lens model1、k2、
k3、k4It is constrained, therefore obtained model parameter is not necessarily global optimum.From panorama picture of fisheye lens model：θ 0 and π/
In the range of 2, θdMore than 0 and less than θ, and it is the monotonically increasing function of the θ；According to known models parameter k obtained by calibrating1、
k2、k3、k4And variations of the θ in the range of 0 and pi/2 draws θd=θ (1+k1θ2+k2θ4+k3θ6+k4θ8) figure, taking-up is more than
0 and the part less than θ and monotonic increase may know that effective angular field of view, so as to know maximum effective visual angle.
Optionally, step 103 can be realized specifically in the following way：
In effective angular field of view, the pixel that is obtained according to the resolution ratio of the fisheye image after correction (u',
V'), the value of a and b are calculated using formula (2)；
The pixel (u, v) of the image of the acquisition is calculated using formula (1) according to the value of a and b；
The pixel value of pixel (u', v') that the fisheye image after correction is obtained according to the pixel (u, v) is：
F (u', v')=(1-s) (1-t) f (i, j)+(1-s) tf (i, j+1)+s (1-t) f (i+1, j)+stf (i+1, j+
1)；
Wherein, i represents the integer part of u, and s represents the fractional part of u, and j represents the integer part of v, and t represents the decimal of v
Part：
F (i, j), f (i, j+1), f (i+1, j), f (i+1, j+1) are respectively the image of the acquisition in fish eye images plane
Coordinate under coordinate system is (i, j), (i+1, j), (i, j+1), 4 pixels corresponding to (i+1, j+1) pixel value.
Specifically, it is assumed that the value range that correction image resolution ratio is 800 × 800, u' and v' is [0,800]；A and b
It is equal to 5 centimetres for space actual range size, such as a, b is equal to 5 centimetres；And the size that u' and v' is pixel space, as u' is equal to 5
A pixel, v' are equal to 5 pixels.
Assuming that obtained maximum effectively visual angle is 76 degree, fx=100, y=100；
Assuming that correction image resolution ratio is 1000 × 1000, picture centre in (500,500), i.e. cx=500, cy=50,
Work as u'=0, during v'=0, then a=-5, b=-5, r=sqrt (a × a+b × b)=5, θ=atan (r)=78.69 degree are more than 76
Degree, therefore the image meeting distortion distortion after the correction of resolution ratio 1000 × 1000, therefore cannot be in the case where resolution ratio is 1000 × 1000
Correction；Sqrt () is the function of extraction of square root；Tan () is the function for seeking tangent value.
Assuming that correction image resolution ratio is 800 × 800, picture centre in (400,400), i.e. cx=400, cy=400, when
When u'=0, v'=0, then a=-4, b=-4, r=sqrt (a × a+b × b)=4, θ=atan (r)=75.96 degree are less than 76
Degree, therefore can be 800 × 800 times corrections in resolution ratio, the value of u and v are obtained using formula (1), and utilize above-mentioned calculating picture
The formula of the pixel value of vegetarian refreshments (u', v') calculates the pixel of each pixel in the fisheye image after correction.
Fig. 3 is the structure diagram of one embodiment of fisheye image means for correcting of the present invention.As shown in figure 3, this implementation
The fisheye image means for correcting of example, including：Acquisition module 301, determining module 302 and correction module 303；
Wherein, acquisition module 301, for according to the collected uncalibrated image of fish eye lens, obtaining panorama picture of fisheye lens mould
The model parameter of type；
Determining module 302, for determining fish-eye effectively to regard according to the model parameter and default constraints
Angular region；
Correction module 303, in effective angular field of view to the fisheye image that fish eye lens is acquired into
Row correction, the fisheye image after being corrected.
Optionally, the acquisition module 301, is additionally operable to：
The panorama picture of fisheye lens model is established, and establishes fish eye lens projection on the panorama picture of fisheye lens model and closes
System；
The fish eye lens projection relation such as formula (1)：
Wherein, θd=θ (1+k1θ2+k2θ4+k3θ6+k4θ8) it is the panorama picture of fisheye lens model；k1、k2、k3、k4It is described
The coefficient of panorama picture of fisheye lens model, θdFor the pixel Pi of the fisheye image and fish eye images plane coordinate system origin
Distance；θ is line segmentWith the angle of the Zc axis positive directions of fish eye lens coordinate system；Oc is the original of fish eye lens coordinate system
Point, Pc are coordinate points Pc (Xc, Yc, Zc) corresponding with the Pi under fish eye lens coordinate system；Wherein, Xc is Pc in fish eye lens
The coordinate value of Xc axis under coordinate system, Yc are the coordinate value of Pc Yc axis under fish eye lens coordinate system, and Zc is sat for Pc in fish eye lens
The coordinate value of the lower Zc axis of mark system；Pw (Xw, Yw, Zw) is Pc (Xc, Yc, Zc) corresponding points under world coordinate system；Xw exists for Pw
The coordinate value of Xw axis under world coordinate system, Yw are the coordinate value of Pw Yw axis under world coordinate system, and Zw is Pw in world coordinate system
The coordinate value of lower Zw axis；R is spin matrix, and T is translation vector；Fx and fy is respectively fish eye images plane coordinate system to flake mirror
The quantization parameter horizontally and vertically of head coordinate system, u and v are respectively water of the Pc under fish eye images plane coordinate system
It is respectively that fisheye image center is horizontal and vertical under fish eye images plane coordinate system that gentle vertical coordinate, cx and cy, which are,
Coordinate；Point Pr in fisheye images of a and b respectively with Pc (Xc, Yc, Zc) after corresponding correction is in fish eye images plane
Horizontal and vertical space actual coordinate under coordinate system, r are the distance between Pr and the origin of fish eye images plane coordinate system；
X' and y' is intermediate variable.
Optionally, the acquisition module 301, is specifically used for：
N width uncalibrated images are acquired using fish eye lens, obtain the uncalibrated image in world coordinate system and in fish eye images
N × m group corresponding points under plane coordinate system；The coordinate of n × m groups corresponding points is (Xwij, Ywij, 0) and (uij, vij)；
Wherein, uncalibrated image described in every width includes the tessellated vertex of m black and white；The value range of i is [1, n], and the value range of j is
[1, m]；Xwij is the coordinate value of the i-th j-th point of Xw axis under world coordinate system of width image, and Ywij is j-th point of the i-th width image
The coordinate value of Yw axis under world coordinate system；Uij and vij is respectively j-th point of the i-th width image in fish eye images plane coordinate system
Under horizontal and vertical coordinate；N and m is the integer more than 1；
According to n × m groups corresponding points the fish is determined using Burger-Marquart Levenberg-Marquardt algorithms
The model parameter of glasses head imaging model；The model parameter includes：R,T,k1,k2,k3,k4,fx,fy,cx,cy；
Wherein, the function that min () expressions are minimized；
mijRepresent the i-th j-th point of coordinate under fish eye images plane coordinate system of width image,Represent j-th point of the i-th width image under world coordinate system coordinate (Xwij, Ywij,
0) coordinate under fish eye images plane coordinate system being calculated by formula (1).
Optionally, the determining module 302, is specifically used for：
According to the model parameter, the default initial range of θ and θdConstraints determine effective angular field of view；It is described
The effective range that effective angular field of view is the θ；
Wherein, the θdConstraints be θdMore than 0, and less than θ, and it is the monotonically increasing function of the θ；
The default initial range of θ is [0, pi/2].
Optionally, the correction module 303, is specifically used for：
In effective angular field of view, the pixel that is obtained according to the resolution ratio of the fisheye image after correction (u',
V'), the value of a and b are calculated using formula (2)；
The pixel (u, v) of the image of the acquisition is calculated using formula (1) according to the value of a and b；
The pixel value of pixel (u', v') that the fisheye image after correction is obtained according to the pixel (u, v) is：
F (u', v')=(1-s) (1-t) f (i, j)+(1-s) tf (i, j+1)+s (1-t) f (i+1, j)+stf (i+1, j+
1)；
Wherein, i represents the integer part of u, and s represents the fractional part of u, and j represents the integer part of v, and t represents the decimal of v
Part：
F (i, j), f (i, j+1), f (i+1, j), f (i+1, j+1) are respectively the image of the acquisition in fish eye images plane
Coordinate under coordinate system is (i, j), (i+1, j), (i, j+1), 4 pixels corresponding to (i+1, j+1) pixel value.
The device of above-described embodiment, can be used for perform Fig. 1 shown in embodiment of the method technical solution, realization principle and
Technique effect is similar, and details are not described herein again.
In several embodiments provided herein, it should be understood that disclosed device and method can pass through it
Its mode is realized.For example, apparatus embodiments described above are only schematical, for example, the unit or module
It divides, only a kind of division of logic function can have other dividing mode, such as multiple units or module in actual implementation
It may be combined or can be integrated into another system or some features can be ignored or does not perform.Another point, it is shown or
The mutual coupling, direct-coupling or communication connection discussed can be the indirect coupling by some interfaces, equipment or module
It closes or communicates to connect, can be electrical, machinery or other forms.
The module illustrated as separating component may or may not be physically separate, be shown as module
The component shown may or may not be physical module, you can be located at a place or can also be distributed to multiple
In network element.Some or all of module therein can be selected according to the actual needs to realize the mesh of this embodiment scheme
's.
One of ordinary skill in the art will appreciate that：Realizing all or part of step of above method embodiment can pass through
The relevant hardware of program instruction is completed, and aforementioned program can be stored in a computer read/write memory medium, the program
When being executed, step including the steps of the foregoing method embodiments is performed；And aforementioned storage medium includes：ROM, RAM, magnetic disc or light
The various media that can store program code such as disk.
Finally it should be noted that：The above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Pipe is described in detail the present invention with reference to foregoing embodiments, it will be understood by those of ordinary skill in the art that：Its according to
Can so modify to the technical solution recorded in foregoing embodiments either to which part or all technical features into
Row equivalent replacement；And these modifications or replacement, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme.
Claims (6)
1. a kind of fisheye image bearing calibration, which is characterized in that including：
Panorama picture of fisheye lens model is established, and fish eye lens projection relation is established on the panorama picture of fisheye lens model；
The fish eye lens projection relation such as formula (1)：
Wherein, θd=θ (1+k1θ2+k2θ4+k3θ6+k4θ8) it is the panorama picture of fisheye lens model；k1、k2、k3、k4For the flake
The coefficient of lens imaging model, θdPixel Pi and fish eye images plane coordinate system origin for the fisheye image away from
From；θ is line segmentWith the angle of the Zc axis positive directions of fish eye lens coordinate system；Origins of the Oc for fish eye lens coordinate system, Pc
For coordinate points Pc (Xc, Yc, Zc) corresponding with the Pi under fish eye lens coordinate system；Wherein, Xc is Pc in fish eye lens coordinate
The coordinate value of the lower Xc axis of system, Yc is the coordinate value of Pc Yc axis under fish eye lens coordinate system, and Zc is Pc in fish eye lens coordinate system
The coordinate value of lower Zc axis；Pw (Xw, Yw, Zw) is Pc (Xc, Yc, Zc) corresponding points under world coordinate system；Xw is Pw in the world
The coordinate value of Xw axis under coordinate system, Yw are the coordinate value of Pw Yw axis under world coordinate system, and Zw is Pw Zw under world coordinate system
The coordinate value of axis；R is spin matrix, and T is translation vector；Fx and fy is respectively that fish eye images plane coordinate system is sat to fish eye lens
Mark the quantization parameter horizontally and vertically of system, u and v be respectively levels of the Pc under fish eye images plane coordinate system and
It is respectively horizontal and vertical seat of the fisheye image center under fish eye images plane coordinate system that vertical coordinate, cx and cy, which are,
Mark；A and b is respectively that the point Pr in the fisheye image with Pc (Xc, Yc, Zc) after corresponding correction is sat in fish eye images plane
Horizontal and vertical space actual coordinate under mark system, r are the distance between Pr and the origin of fish eye images plane coordinate system；x'
It is intermediate variable with y'；
According to the collected uncalibrated image of fish eye lens, the model parameter of the panorama picture of fisheye lens model is obtained；
According to the model parameter, the default initial range of θ and θdConstraints determine fish-eye effective visual angle model
It encloses；Effective angular field of view is the effective range of the θ；
Wherein, the θdConstraints be θdMore than 0, and less than r, and it is the monotonically increasing function of the θ；
The default initial range of θ is [0, pi/2]；
The fisheye image acquired in effective angular field of view to fish eye lens is corrected, the fish after being corrected
Eye lens image.
2. according to the method described in claim 1, it is characterized in that, described according to the collected uncalibrated image of fish eye lens, obtain
The model parameter of panorama picture of fisheye lens model is taken, including：
N width uncalibrated images are acquired using fish eye lens, obtain the uncalibrated image in world coordinate system and in fish eye images plane
N × m group corresponding points under coordinate system；The coordinate of n × m groups corresponding points is (Xwij, Ywij, 0) and (uij, vij)；Wherein,
Uncalibrated image described in every width includes the tessellated vertex of m black and white；The value range of i is [1, n], the value range of j for [1,
m]；Xwij is the coordinate value of the i-th j-th point of Xw axis under world coordinate system of width image, and Ywij exists for j-th point of the i-th width image
The coordinate value of Yw axis under world coordinate system；Uij and vij is respectively j-th point of the i-th width image under fish eye images plane coordinate system
Horizontal and vertical coordinate；N and m is the integer more than 1；
According to n × m groups corresponding points the flake mirror is determined using Burger-Marquart Levenberg-Marquardt algorithms
The model parameter of head imaging model；The model parameter includes：R,T,k1,k2,k3,k4,fx,fy,cx,cy；
Wherein, the cost function of the algorithm is
Wherein, the function that min () expressions are minimized；
mijRepresent the i-th j-th point of coordinate under fish eye images plane coordinate system of width image,
Represent coordinate (Xwij, Ywij, 0) of j-th point of the i-th width image under world coordinate system by formula (1) be calculated
Coordinate under fish eye images plane coordinate system.
3. according to the method described in claim 1, it is characterized in that, it is described in effective angular field of view to fish eye lens institute
The image of acquisition is corrected, the fisheye image after being corrected, including：
In effective angular field of view, the pixel (u', v') that is obtained according to the resolution ratio of the fisheye image after correction,
The value of a and b are calculated using formula (2)；
The pixel (u, v) of the image of the acquisition is calculated using formula (1) according to the value of a and b；
The pixel value of pixel (u', v') that the fisheye image after correction is obtained according to the pixel (u, v) is：
F (u', v')=(1-s) (1-t) f (i, j)+(1-s) tf (i, j+1)+s (1-t) f (i+1, j)+stf (i+1, j+1)；
Wherein, i represents the integer part of u, and s represents the fractional part of u, and j represents the integer part of v, and t represents the fractional part of v：
F (i, j), f (i, j+1), f (i+1, j), f (i+1, j+1) are respectively the image of the acquisition in fish eye images plane coordinates
The lower coordinate of system is (i, j), (i+1, j), (i, j+1), 4 pixels corresponding to (i+1, j+1) pixel value.
4. a kind of fisheye image means for correcting, which is characterized in that including：
Acquisition module, for according to the collected uncalibrated image of fish eye lens, obtaining the model parameter of panorama picture of fisheye lens model；
The acquisition module, is additionally operable to：
The panorama picture of fisheye lens model is established, and fish eye lens projection relation is established on the panorama picture of fisheye lens model；
The fish eye lens projection relation such as formula (1)：
Wherein, θd=θ (1+k1θ2+k2θ4+k3θ6+k4θ8) it is the panorama picture of fisheye lens model；k1、k2、k3、k4For the flake
The coefficient of lens imaging model, θdPixel Pi and fish eye images plane coordinate system origin for the fisheye image away from
From；θ is line segmentWith the angle of the Zc axis positive directions of fish eye lens coordinate system；Origins of the Oc for fish eye lens coordinate system, Pc
For coordinate points Pc (Xc, Yc, Zc) corresponding with the Pi under fish eye lens coordinate system；Wherein, Xc is Pc in fish eye lens coordinate
The coordinate value of the lower Xc axis of system, Yc is the coordinate value of Pc Yc axis under fish eye lens coordinate system, and Zc is Pc in fish eye lens coordinate system
The coordinate value of lower Zc axis；Pw (Xw, Yw, Zw) is Pc (Xc, Yc, Zc) corresponding points under world coordinate system；Xw is Pw in the world
The coordinate value of Xw axis under coordinate system, Yw are the coordinate value of Pw Yw axis under world coordinate system, and Zw is Pw Zw under world coordinate system
The coordinate value of axis；R is spin matrix, and T is translation vector；Fx and fy is respectively that fish eye images plane coordinate system is sat to fish eye lens
Mark the quantization parameter horizontally and vertically of system, u and v be respectively levels of the Pc under fish eye images plane coordinate system and
It is respectively horizontal and vertical seat of the fisheye image center under fish eye images plane coordinate system that vertical coordinate, cx and cy, which are,
Mark；A and b is respectively that the point Pr in the fisheye image with Pc (Xc, Yc, Zc) after corresponding correction is sat in fish eye images plane
Horizontal and vertical space actual coordinate under mark system, r are the distance between Pr and the origin of fish eye images plane coordinate system；x'
It is intermediate variable with y'；
Determining module, for the default initial range and θ according to the model parameter, θdConstraints determine effective visual angle
Range；Effective angular field of view is the effective range of the θ；
Wherein, the θdConstraints be θdMore than 0, and less than r, and it is the monotonically increasing function of the θ；
The default initial range of θ is [0, pi/2]；
Correction module, the fisheye image for being acquired in effective angular field of view to fish eye lens are corrected,
Fisheye image after being corrected.
5. device according to claim 4, which is characterized in that the acquisition module is specifically used for：
N width uncalibrated images are acquired using fish eye lens, obtain the uncalibrated image in world coordinate system and in fish eye images plane
N × m group corresponding points under coordinate system；The coordinate of n × m groups corresponding points is (Xwij, Ywij, 0) and (uij, vij)；Wherein,
Uncalibrated image described in every width includes the tessellated vertex of m black and white；The value range of i is [1, n], the value range of j for [1,
m]；Xwij is the coordinate value of the i-th j-th point of Xw axis under world coordinate system of width image, and Ywij exists for j-th point of the i-th width image
The coordinate value of Yw axis under world coordinate system；Uij and vij is respectively j-th point of the i-th width image under fish eye images plane coordinate system
Horizontal and vertical coordinate；N and m is the integer more than 1；
According to n × m groups corresponding points the flake mirror is determined using Burger-Marquart Levenberg-Marquardt algorithms
The model parameter of head imaging model；The model parameter includes：R,T,k1,k2,k3,k4,fx,fy,cx,cy；
Wherein, the cost function of the algorithm is
Wherein, the function that min () expressions are minimized；
mijRepresent the i-th j-th point of coordinate under fish eye images plane coordinate system of width image,
Represent coordinate (Xwij, Ywij, 0) of j-th point of the i-th width image under world coordinate system by formula (1) be calculated
Coordinate under fish eye images plane coordinate system.
6. device according to claim 5, which is characterized in that the correction module is specifically used for：
In effective angular field of view, the pixel (u', v') that is obtained according to the resolution ratio of the fisheye image after correction,
The value of a and b are calculated using formula (2)；
The pixel (u, v) of the image of the acquisition is calculated using formula (1) according to the value of a and b；
The pixel value of pixel (u', v') that the fisheye image after correction is obtained according to the pixel (u, v) is：
F (u', v')=(1-s) (1-t) f (i, j)+(1-s) tf (i, j+1)+s (1-t) f (i+1, j)+stf (i+1, j+1)；
Wherein, i represents the integer part of u, and s represents the fractional part of u, and j represents the integer part of v, and t represents the fractional part of v：
F (i, j), f (i, j+1), f (i+1, j), f (i+1, j+1) are respectively the image of the acquisition in fish eye images plane coordinates
The lower coordinate of system is (i, j), (i+1, j), (i, j+1), 4 pixels corresponding to (i+1, j+1) pixel value.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510819265.9A CN105488766B (en) | 2015-11-23 | 2015-11-23 | Fisheye image bearing calibration and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510819265.9A CN105488766B (en) | 2015-11-23 | 2015-11-23 | Fisheye image bearing calibration and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105488766A CN105488766A (en) | 2016-04-13 |
| CN105488766B true CN105488766B (en) | 2018-06-19 |
Family
ID=55675731
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510819265.9A Active CN105488766B (en) | 2015-11-23 | 2015-11-23 | Fisheye image bearing calibration and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105488766B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106127701A (en) * | 2016-06-16 | 2016-11-16 | 深圳市凌云视迅科技有限责任公司 | Fisheye image distortion correction method and device |
| CN106780389B (en) * | 2016-12-23 | 2021-06-15 | 浙江宇视科技有限公司 | Fisheye image correction method and device based on coordinate transformation |
| CN107665483B (en) * | 2017-09-27 | 2020-05-05 | 天津智慧视通科技有限公司 | Calibration-free convenient monocular head fisheye image distortion correction method |
| CN107808367B (en) * | 2017-11-29 | 2020-10-16 | 天津聚飞创新科技有限公司 | Fisheye image correction method and device, unmanned aerial vehicle and storage medium |
| CN108230241B (en) * | 2018-01-09 | 2021-03-16 | 济南中维世纪科技有限公司 | Fisheye image correction method for equipment with angle side mounting |
| CN108692921B (en) * | 2018-05-30 | 2020-10-02 | 信利光电股份有限公司 | Camera module analysis force and lens analysis force approval method |
| CN112150554B (en) * | 2019-06-28 | 2023-08-04 | 杭州海康威视数字技术股份有限公司 | Picture display method, device, terminal and storage medium |
| CN113421300B (en) * | 2021-06-28 | 2023-05-12 | 上海迈外迪网络科技有限公司 | Method and device for determining actual position of object in fisheye camera image |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101577002A (en) * | 2009-06-16 | 2009-11-11 | 天津理工大学 | Calibration method of fish-eye lens imaging system applied to target detection |
| CN102096923A (en) * | 2011-01-20 | 2011-06-15 | 上海杰图软件技术有限公司 | Fisheye calibration method and device |
| CN103426168A (en) * | 2013-07-18 | 2013-12-04 | 北京航空航天大学 | General calibration method for common video cameras, panorama video cameras and fisheye three-dimensional video cameras and based on one-dimensional calibration rod |
| CN104240236A (en) * | 2014-08-26 | 2014-12-24 | 中山大学 | Fisheye image correction method after calibration conducted by fisheye lens |
| CN104574289A (en) * | 2013-10-29 | 2015-04-29 | 深圳市中航比特通讯技术有限公司 | Fish-eye image distortion correction algorithm based on ellipsoidal model |
-
2015
- 2015-11-23 CN CN201510819265.9A patent/CN105488766B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101577002A (en) * | 2009-06-16 | 2009-11-11 | 天津理工大学 | Calibration method of fish-eye lens imaging system applied to target detection |
| CN102096923A (en) * | 2011-01-20 | 2011-06-15 | 上海杰图软件技术有限公司 | Fisheye calibration method and device |
| CN103426168A (en) * | 2013-07-18 | 2013-12-04 | 北京航空航天大学 | General calibration method for common video cameras, panorama video cameras and fisheye three-dimensional video cameras and based on one-dimensional calibration rod |
| CN104574289A (en) * | 2013-10-29 | 2015-04-29 | 深圳市中航比特通讯技术有限公司 | Fish-eye image distortion correction algorithm based on ellipsoidal model |
| CN104240236A (en) * | 2014-08-26 | 2014-12-24 | 中山大学 | Fisheye image correction method after calibration conducted by fisheye lens |
Non-Patent Citations (4)
| Title |
|---|
| A Generic Camera Model and Calibration Method for Conventional, Wide-Angle, and Fish-Eye Lenses;Juho Kannala 等;《IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE》;20060831;第28卷(第8期);第1335-1340页 * |
| OpenCV 3.0.0-dev documentation- OpenCV API reference-calib3d.camera calibration and 3d reconstruction;opencv dev team;《https://docs.opencv.org/3.0-beta/modules/calib3d/doc/camera-calibration_and_3d_reconstruction.html》;20141110;第1-3页 * |
| 鱼眼图像变形校正算法研究;赵丹;《中国优秀硕士学位论文全文数据库信息科技辑》;20121115;第2012年卷(第11期);第4.2节，第4.2.2节 * |
| 鱼眼镜头的标定和畸变校正研究;黄岩岩 等;《计算机工程与设计》;20140930;第35卷(第9期);第3132-3135,3212页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105488766A (en) | 2016-04-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105488766B (en) | Fisheye image bearing calibration and device | |
| CN109146794B (en) | A kind of light field image rotation error bearing calibration | |
| CN108564617B (en) | Three-dimensional reconstruction method and device for multi-view camera, VR camera and panoramic camera | |
| CN109767474B (en) | Multi-view camera calibration method and device and storage medium | |
| CN109003311B (en) | Calibration method of fisheye lens | |
| CN107705333B (en) | Space positioning method and device based on binocular camera | |
| CN108257183B (en) | Camera lens optical axis calibration method and device | |
| CN110197466B (en) | Wide-angle fisheye image correction method | |
| TWI555379B (en) | An image calibrating, composing and depth rebuilding method of a panoramic fish-eye camera and a system thereof | |
| CN109272570A (en) | A kind of spatial point three-dimensional coordinate method for solving based on stereoscopic vision mathematical model | |
| CN105243637B (en) | One kind carrying out full-view image joining method based on three-dimensional laser point cloud | |
| CN108629829B (en) | Three-dimensional modeling method and system of the one bulb curtain camera in conjunction with depth camera | |
| CN109615663A (en) | Panoramic video bearing calibration and terminal | |
| CN107424118A (en) | Based on the spherical panorama mosaic method for improving Lens Distortion Correction | |
| CN110288527B (en) | Panoramic aerial view generation method of vehicle-mounted panoramic camera | |
| CN108053373A (en) | One kind is based on deep learning model fisheye image correcting method | |
| CN109559349A (en) | A kind of method and apparatus for calibration | |
| CN112215880B (en) | Image depth estimation method and device, electronic equipment and storage medium | |
| CN108269234A (en) | A kind of lens of panoramic camera Attitude estimation method and panorama camera | |
| CN109769110B (en) | Method and device for generating 3D asteroid dynamic graph and portable terminal | |
| KR20200013800A (en) | A multicamera imaging system | |
| CN110458952A (en) | A kind of three-dimensional rebuilding method and device based on trinocular vision | |
| CN107845056A (en) | Fish eye images panorama generation method based on cylinder model | |
| CN108648141A (en) | A kind of image split-joint method and device | |
| CN103546680B (en) | A kind of deformation-free omni-directional fisheye photographic device and a method for implementing the same |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |