CN104182961A - Fisheye image distortion correction method based on reverse polynomial model - Google Patents
Fisheye image distortion correction method based on reverse polynomial model Download PDFInfo
- Publication number
- CN104182961A CN104182961A CN201310203257.2A CN201310203257A CN104182961A CN 104182961 A CN104182961 A CN 104182961A CN 201310203257 A CN201310203257 A CN 201310203257A CN 104182961 A CN104182961 A CN 104182961A
- Authority
- CN
- China
- Prior art keywords
- model
- reverse
- correction
- image
- distortion correction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Processing (AREA)
Abstract
The invention discloses a fisheye image distortion correction method based on a reverse polynomial model. The fisheye image distortion correction method adopts a mathematic means that reverse Taylor series fitting is carried out on a polynomial function expression built by a forward model, and a corrected image coordinate which takes a pixel as a unit is converted into an image coordinate which is not corrected and takes a pixel as the unit. Since each corrected image coordinate inevitably corresponds to a certain pixel value in the image which is not corrected through a reverse correction model, a phenomenon that individual pixels of the image are lost after correction can be successfully avoided by reverse correction so as to avoid additional operation, such as interpolation and the like, and meanwhile, a good correction effect and better real-time performance can be obtained. Under a condition that correction precision is almost unchanged, the phenomenon that the pixels of the corrected image are lost is avoided, the real-time performance of the whole correction process is improved, and for an embedded system and a site application occasion, the reverse polynomial model is superior to a traditional forward polynomial correction model.
Description
Technical field
The present invention relates to image and process and field of machine vision, be specifically related to a kind of fish eye images distortion correction method based on reverse multinomial model.
Background technology
Owing to having the advantages such as large visual field, short focal length, fish eye lens can be used for expanding the field range of common pinhole camera.At many computer vision fields such as robot navigation, video conference, supervision and virtual realities, all need to use wide-angle or the flake video camera with larger visual field.Most computer vision fields generally believe that camera follows linear pin-hole model.But FISH EYE LENS OPTICS system is introduced strong optical distortion, cause the hypothesis of pin-hole model in fact no longer to be set up.The image perspective projection information that has serious distortion in order to make full use of these need to be desirable linear perspective projected image by the image rectification after distortion.Radial distortion is the most significant distortion type that fish eye lens is introduced, and the zone line that embodies fault image in FISH EYE LENS OPTICS presents high resolving power, and presents the non-linear decline of image resolution ratio in the marginal portion of image.How the key issue of distortion correction is that radial distortion is proofreaied and correct to fish eye images.
The mainly camera calibration theory based on traditional of research of the current distortion correction problem for flake camera review.Fish eye lens calibration algorithm is to set up on the basis of projective invariant model, considers main distortion type, the especially radial distortion of panorama picture of fisheye lens, sets up accordingly accurate flake camera model and calculates the each component in corresponding internal reference matrix.Camera marking method mainly comprises that camera vision calibration, self-calibration and Zhang Zhengyou demarcate, and all need special demarcating module and template.Conventional is, and precision is high, strong robustness and more flexibly Zhang Zhengyou demarcate.Zhang Zhengyou calibration request video camera is at plane reference plate of at least two orientation photographs, and video camera and two dimensional surface scaling board can move freely, and inner parameter remains constant.Because two dimensional surface scaling board is in world coordinate system
z=0, therefore can solve the optimization solution of intrinsic parameters of the camera and external parameter by linear model analysis, then carry out non-linear refinement by maximal possibility estimation and Levenberg-Marquardt.Finally adopt the succinct and real-time forward polynomial expression calibration model of algorithm to carry out matching radial distortion function, realize the distortion correction of flake camera review.
In sum, flake Camera calibration model great majority have adopted forward to go distortion correction thinking, and the image pixel coordinate before proofreading and correct is converted to the image coordinate after correction by calibration model.Because this mapping relations are not one to one, so the image after proofreading and correct will inevitably cause some pixel disappearance of image, namely cavitation.Traditional flake distortion correction must be accompanied by successive image interpolation processing, comprising nearest-neighbor interpolation, bilinear interpolation, bicubic interpolation method etc.In general,, owing to having introduced interpolation operation, can cause the real-time of algorithm to reduce, and then affect final calibration result.
Summary of the invention
For the deficiency of forward polynomial expression correcting algorithm, the invention provides the fish eye images distortion correction method based on reverse multinomial model, forward calibration model is obtained to reverse polynomial expression calibration model by the matching of linear Taylor progression, by reverse calibration model, the image coordinate after proofreading and correct has been converted to the image coordinate before correction.Experiment shows the phenomenon that this algorithm has effectively avoided the rear image pixel of correction to lack, thereby has avoided follow-up interpolation processing, has effectively improved the real-time of whole trimming process when having reduced operand.Contrary polynomial expression calibration model only relates to addition and multiplication in mathematical operation, is extremely conducive to hardware and realizes.For embedded system and application scenario, place, reverse polynomial expression correcting algorithm has obvious advantage.
The present invention relates to the fish eye images distortion correction method based on reverse multinomial model.Particularly: the forward polynomial expression calibration model based on definite and concrete forward distortion correction parameter, calculate expression formula and the corresponding distortion correction coefficient of reverse calibration model, finally carry out non-linear refinement by maximal possibility estimation.If the situation that correction accuracy declines, re-starts maximum likelihood refinement, until correction accuracy meets the demands.
Technical scheme of the present invention is achieved in that the inner parameter and the external parameter that first adopt Zhang Zhengyou calibration algorithm to calculate fish-eye camera, set up the distortion model of fish-eye camera, in the situation that only considering radial distortion, fish-eye camera is set up to forward polynomial expression distortion correction model.Because forward polynomial expression distortion correction model tormulation formula cannot directly be inverted, therefore adopt the method for Taylor progression matching to set up reverse polynomial expression calibration model.
Fish eye images distortion correction method based on reverse multinomial model, comprises the steps:
Step 1: flake video camera is carried out to Zhang Zhengyou demarcation, the each component in internal reference matrix and the outer ginseng matrix of acquisition video camera;
Step 2: first set up forward polynomial expression distortion correction
r u with
r d model, only considering to calculate the correction coefficient in forward multinomial model under radial distortion condition.Due to
r d with
r u the calibration model of setting up cannot directly be realized image rectification, it need to be converted to before and after proofreading and correct to the relation between image coordinate or the relational model of physical coordinates.In order to realize coordinate conversion, can do following hypothesis: (1) optical axis is strictly perpendicular to imaging plane, and principal point does not have slip chart inconocenter.We can ignore centrifugal distortion like this; (2) ignore tangential distortion, only consider radial distortion.According to geometric relationship, the coordinate relation under the coordinate system that easily obtains representing with physical unit before and after proofreading and correct, then arranges and obtains proofreading and correct the image coordinate relation under the image coordinate system of front and back taking pixel as unit.
Step 3: adopt Levenberg-Marquardt algorithm to carry out non-linear refinement to demarcating the correction coefficient of the internal reference, outer ginseng and the forward multinomial model that obtain, obtain relatively accurate calibrating parameters value;
Step 4: demarcate distortion correction parameter after the optimization drawing according to forward multinomial model, calculate the distortion correction coefficient of reverse polynomial expression calibration model; By reverse Taylor progression pair
r d carry out linear polynomial matching, easily derive the coordinate relation in the physical coordinates system of correction front and back in reverse model, and obtain the parameter expression in reverse calibration model; By maximal possibility estimation, the parameter values such as internal reference and distortion correction coefficient are optimized, can obtain the optimal value of undetermined parameter.Finally, can obtain the parameter containing in reverse multinomial model.
Step 5: the correction accuracy of checking reverse calibration model: be, the image after output calibration; No, continue the process of parameter optimization, return to step 3;
Step 6: export image after final correction, use as successive image identification, analysis, processing.
beneficial effect
The inventive method has taken into full account that the pixel that the correction of forward polynomial expression produces lacks the impact of problem on trimming process real-time, in the situation that ensureing correction accuracy, increase substantially the real-time of algorithm, avoid the phenomenon of image section pixel disappearance, thereby avoid interpolation processing, effectively improved the operation efficiency of trimming process.For embedded system and application scenario, place, reverse correcting algorithm is better than traditional correcting algorithm.
Brief description of the drawings
The method flow diagram of Fig. 1 specific embodiment of the invention;
The flake fault image of Fig. 2 specific embodiment of the invention;
The forward polynomial expression calibration result (not interpolation) of Fig. 3 specific embodiment of the invention;
The forward polynomial expression calibration result (interpolation) of Fig. 4 specific embodiment of the invention;
The reverse polynomial expression calibration result (iteration once) of Fig. 5 specific embodiment of the invention;
The reverse polynomial expression calibration result (same view angle) of Fig. 6 specific embodiment of the invention;
The reverse polynomial expression calibration result (other visual angle) of Fig. 7 specific embodiment of the invention.
Embodiment
Below in conjunction with accompanying drawing, specific embodiment of the invention is elaborated.
The method of present embodiment, software environment is WINDOWS 7 systems, simulated environment is MATLAB2008a, flow process as shown in Figure 1:
Step 1: fish-eye camera is demarcated.Wherein the model of fish-eye camera is PC1030, and resolution is 576*720, and the calibrating parameters obtaining is as follows:
From internal reference matrix analysis above, the internal reference of fish-eye camera outbalance is mainly as follows: (1) principal point coordinate figure:
u 0 =267.3437,
v 0 =362.0111.Unit length pixel count:
dx=242.9073,
dy=245.7855.The resolution that flake fault image is chosen in experiment is 576*720 pixel, and the scaling board of choosing is square gridiron pattern, comprises altogether 19*15 unique point.
Step 2: set up forward polynomial expression distortion correction model, only considering to calculate the correction coefficient in forward multinomial model under radial distortion condition
k i .Ignore the distortion form that other fish-eye camera is introduced, such as centrifugal distortion and tangential distortion, only retain radial distortion, forward multinomial model is set up
r u with
r d relational expression is as follows:
（1）
Wherein
k 1 ,
k 2 ,
k 3 ......
k n for the distortion correction parameter of optimization undetermined.
According to geometric relationship, the coordinate relation under the coordinate system that easily obtains representing with physical unit before and after proofreading and correct:
（2）
Wherein (
x,
y) be the physical coordinates of image before proofreading and correct, (
x',
y') for proofreading and correct the physical coordinates of rear image.Physical coordinates is converted to image coordinate, arranges and obtain proofreading and correct the image coordinate relation under the image coordinate system of front and back taking pixel as unit:
（3）
By forward multinomial model, easily obtain:
（4）
With
d,
d,kcarry out simplification matrix as follows:
（5）
Utilize least square method easily to obtain
kinitializer:
（6）
So far the parameter in model is all relevant with the internal reference of camera, all can obtain by camera calibration, and therefore the model of above-mentioned formula can be realized forward distortion correction.Demarcate the distortion correction parameter obtaining in conjunction with Zhang Zhengyou
k i value as follows:
k 1 =0.36；
k 2 =0.38；
k 3 =0.0016；
k 4 =0.00000078；
k 5 = -1.3297*10
-10
The flake fault image of the binaryzation that this enforcement adopts as shown in Figure 2.
Step 3: carry out non-linear refinement to demarcating the inner parameter, external parameter and the forward polynomial expression distortion correction coefficients by using Levenberg Marquardt that obtain, obtain relatively accurate calibrating parameters value.The parameter value more than providing is all through optimizing.
Step 4: according to the forward polynomial expression distortion correction model of setting up, obtain reverse multinomial model by the method for Taylor progression matching, demarcate distortion correction parameter after the optimization drawing according to forward multinomial model
k i , calculate the distortion correction coefficient of reverse polynomial expression calibration model
a i .
By reverse Taylor progression pair below
r d carry out linear polynomial matching:
（7）
Easily derive the coordinate relation in the physical coordinates system of correction front and back in reverse model:
（8）
And the coordinate relation in image coordinate system before and after proofreading and correct:
（9）
Calculate the distortion correction coefficient of reverse polynomial expression calibration model
a i , specific as follows:
Distortion correction parameter in reverse model
a 1 ,
a 2 ......
a n with the distortion correction coefficient in forward calibration model
k 1 ,
k 2 ,
k 3 ......
k n between there is specific relation.In order to seek this relation, order
r d value be sequence of natural numbers 1,2 ....
nso, can obtain corresponding
r u value; Utilize linear algebra theory, will
r d with
r u respective value is brought formula (1) into, and the simplification expression formula that can obtain linear equation is as follows:
（10）
Wherein
a,
x,
bexpression formula as follows:
（11）
Thus, can obtain the parameter expression in reverse calibration model:
（12）
By maximal possibility estimation, the parameter values such as internal reference and distortion correction coefficient are optimized, can obtain undetermined parameter
k 1 ,
k 2 ,
k 3 ......
k n optimal value.Finally, formula (12) can be obtained the parameter containing in reverse multinomial model
a 1 ,
a 2 ......
a n .
Fig. 3 has shown the effect after forward calibration model is proofreaied and correct, and owing to not doing interpolation arithmetic, can be clear that in the image after correction and have obvious pixel disappearance, also referred to as cavitation.
Fig. 4 has shown the calibration result figure after interpolation, has promoted much although compare calibration result with Fig. 3, can cause the real-time of algorithm to decline owing to additionally having introduced interpolation arithmetic.
According to the method for Taylor progression matching, can calculate
a i initial value, computation process is as follows: (1) calculates one group
r d with
r u corresponding relation; (2) set up
aa=
bsystem of linear equations, wherein
a,
a,bexpression formula is as follows:
(3) provide
a 1 =0.6378;
a 2 =-0.0369;
a 3 =0.0004;
a 4 ,
a 5 be approximately 0.
Fig. 5 has provided the calibration result after iteration for the first time, and correction accuracy also needs further raising, and to this, we adopt maximal possibility estimation to be optimized.
Fig. 6 has provided the calibration result of reverse calibration model under same visual angle, and wherein Optimal Parameters is as follows:
a 1 =0.8371;
a 2 =-0.08395;
a 3 =0.0008;
a 4 with
a 5 value is approximately 0.Intuitively correction accuracy declines to some extent, but this method has been avoided interpolation arithmetic.
Step 5: the correction accuracy of checking reverse calibration model: be that the image after output calibration, processes and use as successive image; No, continue the process of parameter optimization, return to step 3.
Fig. 7 has provided the reverse calibration result under other visual angles.Can find out through optimization process correction accuracy repeatedly and promote to some extent, but image is larger to external diffusion, lost partial pixel information.
In sum, realized efficient flake distortion correction process based on reverse polynomial distortion correction method.The method, in ensureing correction accuracy, has been avoided successive image interpolation processing, thereby has been increased substantially real-time and the dirigibility of algorithm.
Claims (2)
1. the fish eye images distortion correction method based on reverse multinomial model, the mathematical measure adopting is that the polynomial function expression formula that forward model is set up is carried out the matching of reverse Taylor progression, and the image coordinate taking pixel as unit after proofreading and correct is converted to the image coordinate taking pixel as unit before correction; Its technical characterictic and advantage are: (1) demarcates distortion correction parameter after the optimization drawing according to forward multinomial model, calculate the distortion correction coefficient of reverse polynomial expression calibration model, and adopt alternative manner to check the correction accuracy of reverse calibration model; (2) image coordinate after each correction will inevitably be corresponded to and be proofreaied and correct a certain pixel value in front image by reverse calibration model, effectively avoid the image pixel deficient phenomena (cavitation) after proofreading and correct, thereby avoid the extra operations such as interpolation, obtained good calibration result and more excellent real-time.
2. the fish eye images distortion correction method method based on reverse multinomial model described in, is characterized in that: comprise the steps:
Step 1: adopt Zhang Zhengyou calibration algorithm to obtain inner parameter and the external parameter of fish-eye camera;
Step 2: set up forward polynomial expression distortion correction model, only considering to calculate the correction coefficient in forward multinomial model under radial distortion condition;
Step 3: carry out non-linear refinement to demarcating the internal reference, outer ginseng and the forward polynomial expression distortion correction coefficients by using Levenberg-Marquardt that obtain, obtain relatively accurate calibrating parameters value;
Step 4: demarcate distortion correction parameter after the optimization drawing according to forward multinomial model, calculate the distortion correction coefficient of reverse polynomial expression calibration model;
Step 5: the correction accuracy of the reverse calibration model of iteration check, meet the demands and carry out next step, do not meet the demands and return to step 3;
Step 6: the image after output calibration.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310203257.2A CN104182961A (en) | 2013-05-28 | 2013-05-28 | Fisheye image distortion correction method based on reverse polynomial model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201310203257.2A CN104182961A (en) | 2013-05-28 | 2013-05-28 | Fisheye image distortion correction method based on reverse polynomial model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104182961A true CN104182961A (en) | 2014-12-03 |
Family
ID=51963979
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201310203257.2A Pending CN104182961A (en) | 2013-05-28 | 2013-05-28 | Fisheye image distortion correction method based on reverse polynomial model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104182961A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109961410A (en) * | 2019-02-28 | 2019-07-02 | 中航华东光电有限公司 | A kind of image pre-distortion method of optimization |
| CN110809114A (en) * | 2018-08-03 | 2020-02-18 | 半导体元件工业有限责任公司 | Transform processor for stepwise switching between image transforms |
| CN110852976A (en) * | 2019-11-22 | 2020-02-28 | 昆明物理研究所 | Infrared image brightness unevenness correction method and computer program product |
| CN115272110A (en) * | 2022-07-21 | 2022-11-01 | 四川大学 | Projector distortion correction method and device in structured light three-dimensional reconstruction |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6611282B1 (en) * | 1999-01-04 | 2003-08-26 | Remote Reality | Super wide-angle panoramic imaging apparatus |
| CN101739707A (en) * | 2009-12-16 | 2010-06-16 | 合肥工业大学 | Elliptic fisheye image-based distortion correction method |
| CN101783011A (en) * | 2010-01-08 | 2010-07-21 | 宁波大学 | Distortion correction method of fish eye lens |
-
2013
- 2013-05-28 CN CN201310203257.2A patent/CN104182961A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6611282B1 (en) * | 1999-01-04 | 2003-08-26 | Remote Reality | Super wide-angle panoramic imaging apparatus |
| CN101739707A (en) * | 2009-12-16 | 2010-06-16 | 合肥工业大学 | Elliptic fisheye image-based distortion correction method |
| CN101783011A (en) * | 2010-01-08 | 2010-07-21 | 宁波大学 | Distortion correction method of fish eye lens |
Non-Patent Citations (2)
| Title |
|---|
| WONPIL YU: "An Embedded Camera Lens Distortion Correction Method for Mobile Computing Applications", 《IEEE TRANSACTIONS ON CONSUMER ELECTRONICS》 * |
| 杨雪荣 等: "视觉测量中的相机标定方法进展研究", 《机械设计与制造》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110809114A (en) * | 2018-08-03 | 2020-02-18 | 半导体元件工业有限责任公司 | Transform processor for stepwise switching between image transforms |
| US10757324B2 (en) | 2018-08-03 | 2020-08-25 | Semiconductor Components Industries, Llc | Transform processors for gradually switching between image transforms |
| CN109961410A (en) * | 2019-02-28 | 2019-07-02 | 中航华东光电有限公司 | A kind of image pre-distortion method of optimization |
| CN109961410B (en) * | 2019-02-28 | 2022-12-20 | 中航华东光电有限公司 | Optimized image pre-distortion method |
| CN110852976A (en) * | 2019-11-22 | 2020-02-28 | 昆明物理研究所 | Infrared image brightness unevenness correction method and computer program product |
| CN110852976B (en) * | 2019-11-22 | 2023-04-18 | 昆明物理研究所 | Infrared image brightness unevenness correction method and computer program product |
| CN115272110A (en) * | 2022-07-21 | 2022-11-01 | 四川大学 | Projector distortion correction method and device in structured light three-dimensional reconstruction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104182933A (en) | Wide-angle lens image distortion correcting method based on reverse division model | |
| CN102810205B (en) | The scaling method of a kind of shooting or photographic means | |
| CN102096923B (en) | Fisheye calibration method and device | |
| CN105096329B (en) | Method for accurately correcting image distortion of ultra-wide-angle camera | |
| US9189889B2 (en) | Method for building a three-dimensional model and apparatus thereof | |
| CN110070598B (en) | Mobile terminal for 3D scanning reconstruction and 3D scanning reconstruction method thereof | |
| CN107358633A (en) | Join scaling method inside and outside a kind of polyphaser based on 3 points of demarcation things | |
| US11216979B2 (en) | Dual model for fisheye lens distortion and an algorithm for calibrating model parameters | |
| EP3376762B1 (en) | Depth image composition method and apparatus | |
| CN104200454B (en) | Fisheye image distortion correction method and device | |
| CN102609941A (en) | Three-dimensional registering method based on ToF (Time-of-Flight) depth camera | |
| CN111461963B (en) | Fisheye image stitching method and device | |
| CN104809739A (en) | Real-time ultra wide-angle lens camera video correcting method | |
| CN114399554A (en) | Calibration method and system of multi-camera system | |
| CN104778656A (en) | Fisheye image correction method on basis of spherical perspective projection | |
| CN103077524A (en) | Calibrating method of hybrid vision system | |
| CN106570907B (en) | Camera calibration method and device | |
| CN107665483A (en) | Exempt from calibration easily monocular camera lens fish eye images distortion correction method | |
| CN107492080B (en) | Calibration-free convenient monocular head image radial distortion correction method | |
| Micušık et al. | Para-catadioptric camera auto-calibration from epipolar geometry | |
| JP2022085910A (en) | Camera calibration method, apparatus, electronic device, storage medium, program, and roadside apparatus | |
| CN104182961A (en) | Fisheye image distortion correction method based on reverse polynomial model | |
| CN110288527A (en) | The vehicle-mounted camera panorama of looking around of one kind gets a bird's eye view drawing generating method | |
| CN111383264B (en) | Positioning method, positioning device, terminal and computer storage medium | |
| CN105488766A (en) | Fish-eye lens image correcting method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20141203 |
|
| WD01 | Invention patent application deemed withdrawn after publication |