CN113538283A - Distortion correction method for images shot by redundant fisheye cameras - Google Patents

Distortion correction method for images shot by redundant fisheye cameras Download PDF

Info

Publication number
CN113538283A
CN113538283A CN202110829146.7A CN202110829146A CN113538283A CN 113538283 A CN113538283 A CN 113538283A CN 202110829146 A CN202110829146 A CN 202110829146A CN 113538283 A CN113538283 A CN 113538283A
Authority
CN
China
Prior art keywords
distortion
image
resp
correction
coordinate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110829146.7A
Other languages
Chinese (zh)
Other versions
CN113538283B (en
Inventor
肖文平
石川
程旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Heqian Electronic Technology Co ltd
Original Assignee
Zhejiang Heqian Electronic Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Heqian Electronic Technology Co ltd filed Critical Zhejiang Heqian Electronic Technology Co ltd
Priority to CN202110829146.7A priority Critical patent/CN113538283B/en
Priority claimed from CN202110829146.7A external-priority patent/CN113538283B/en
Publication of CN113538283A publication Critical patent/CN113538283A/en
Application granted granted Critical
Publication of CN113538283B publication Critical patent/CN113538283B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G06T5/80

Abstract

The invention provides a distortion correction method for images shot by a redundant fisheye camera, which comprises the steps of carrying out secondary distortion correction on the shot images, obtaining a first distortion correction image after carrying out first distortion correction on the distortion images by utilizing an optical distortion parameter mapping table, and solving a homography matrix H by utilizing perspective change in the second correction through obtaining a first coordinate point corresponding to a characteristic point of a preset target image in the first distortion correction image and a first coordinate point corresponding to a difference correction planedapos, using a homography matrix Hdapos inverse transforms the first distorted image for a second distortion correction. The secondary distortion correction is carried out on the images shot by the redundant fisheye cameras, so that a good imaging effect can be obtained, and the calculated amount is greatly reduced.

Description

Distortion correction method for images shot by redundant fisheye cameras
Technical Field
The invention relates to the field of cameras, in particular to a distortion correction method for images shot by a redundant fisheye camera.
Background
In the existing automobile, only one camera is arranged at the position of the same area of the automobile, when the camera is damaged or the lens is polluted, the image at the corresponding position is lacked, and certain functions of the intelligent automobile, such as around-looking splicing, reversing images and the like, cannot be completed. In addition, in order to expand the imaging range of the camera, in the existing smart car, a fisheye camera is used for shooting to obtain an imaging picture in a large range, but the fisheye camera is distorted during shooting, so that the finally shot image is distorted, and therefore the fisheye camera needs to be subjected to distortion correction. The distortion correction technology is already in the prior artAs is well established, the current orthodox distortion correction schemes have been opencv cured using polynomials with r (θ) k0θ+k1θ3+k2θ5+k3θ7+k4θ9Correction of fisheye cameras, k0、k1、k2、k3、k4Is a distortion correction coefficient, but the distortion correction is performed on the basis of the assumption that the camera conforms to an ideal fish-eye camera model. However, in practice, the fisheye camera is limited by the processing conditions, and thus, a large error still exists when the camera module is aligned and assembled. In order to solve the above drawbacks, the inventor of the present application discloses (CN107248178B) a fisheye camera calibration method based on distortion parameters, which considers the object field angle and the actual image height in the production process of a fisheye camera, but the calculation process of the fisheye camera calibration method increases the calculation amount and consumes more resources along with a plurality of complex matrix operations, and the operation efficiency is low when the vehicle-mounted host adopts embedded hardware. In addition, the applicant finds that space difference exists in the installation position of the redundant cameras through a large amount of accidental discoveries in the research process of the redundant cameras, so that difference exists in the images of the shooting fields of view and precision difference exists in the production process, and the images corrected according to the existing distortion correction method still have distortion.
Disclosure of Invention
Based on the defects in the prior art, the invention provides a method for correcting the distortion of images shot by a redundant fisheye camera, which is characterized by at least comprising the following steps:
the method comprises the steps of performing first distortion correction, namely acquiring a distorted image formed by a preset target image vertical to the ground through a redundant camera, and performing first distortion correction on the distorted image through an optical distortion parameter mapping table to acquire a first distortion corrected image;
second distortion correction, establishing a difference correction plane coordinate system, respectively obtaining a first coordinate point corresponding to the feature point of the preset target image in the first distortion correction image and a first coordinate point corresponding to the difference correction plane, and solving the homography matrix by utilizing perspective transformationHdapos, using a homography matrix Hdapos inverse transforms the first distorted image for a second distortion correction.
A distortion correction method for images shot by a redundant fisheye camera further comprises the step of acquiring a homography matrix H in the second distortion correctiondapos, utilizing inverse perspective transformation to enable all first coordinate points corresponding to the established difference correction plane to correspond to second coordinate points in the first distortion correction image, obtaining pixel values of the second coordinate points in the first distortion correction image, and assigning the pixel values to the first coordinate points.
A method for correcting the distortion of the image shot by redundant fisheye camera features that the characteristic points (T) of target image are preset in the coordinate system of difference correction planexn,Tyn) First coordinate (R) corresponding to the difference correction planexn,Ryn) The mapping function of (d) is:
Rxn=Txn×SC+OFSTx
Ryn=Tyn×SC+OFSTy
where SC is a planar coordinate scale transformation factor, OFSTxAnd OFSTyHorizontal and vertical pixel offsets, respectively.
A distortion correction method for images shot by a redundant fisheye camera further comprises the following steps of:
step S101, acquiring a preset target map vertical to the ground through a redundant camera, and acquiring actual physical coordinates (T) of a plurality of characteristic points in the preset target map on a target map planexn,Tyn);
Step S102, according to the actual physical coordinate (T)xn,Tyn) Solving the actual physical coordinate (T) in the actual shooting surface U after distortion correctionxn,Tyn) Pixel coordinates (RESP) of corresponding feature pointsxn,RESPyn);
Step S103, establishing a difference correction plane coordinate system, and solving the feature point coordinate (R) which is mapped to the corresponding feature point coordinate under the difference correction plane coordinate system by the coordinates of a plurality of feature points in the target mapxn,Ryn);
Step S104, according to the perspective transformation, solving (R)xn,Ryn) To (RESP)xn,RESPyn) Homography matrix H ofdapos:
Perspective transformation:
Figure BDA0003174851440000031
Hdapos:
Figure BDA0003174851440000032
wherein h1, h2, h3, h4, h5, h6, h7 and h8 represent element values in a matrix;
step S105, again using perspective transformation change, correcting all pixel coordinates (R) of image with differencexi,Ryi) Replacement (R)xn,Ryn) Using H obtained in step S104dapos, calculating corresponding coordinates (RESP) of the corresponding corrected imagexi,RESPyi)。
A distortion correction method for images shot by a redundant fisheye camera is further provided, wherein the first distortion correction comprises the following steps:
step S1, acquiring optical distortion parameters of the vehicle-mounted Ethernet camera, wherein the optical distortion parameters comprise a discrete view field angle theta and a mapping table of corresponding image height gamma of the discrete view field angle theta, which is used for acquiring actual experiment environment images by the experiment module under the condition that the camera lens designates the CMOS chip;
step S2, according to the pinhole imaging principle, establishing the space coordinate relation of each pixel point of the actual shooting plane U corresponding to each point of the fisheye image on the imaging plane I, and establishing a first mapping function of the discrete view field angle theta and the object height lambda;
λ=tan(θ)
step S3, constructing a second mapping function θ of the image height γ and the discrete field angle θ as G (γ) according to the optical distortion parameter mapping table;
step S4, according to the imaging geometrical relation, solving the correction scaling factor S between the actual shooting plane U and the corresponding correction imaging plane and distortion imaging planeUAnd distortion scaling factor SI
Step S5, using the first mapping function, the second mapping function and the rectification scaling factor SUAnd distortion scaling factor SIEstablishing pixel-level coordinates RESP of the corrected undistorted imagex,RESPyAnd pixel level coordinates RESp 'of distorted image'x,RESp′yAnd RESp 'to a corresponding third mapping function'x,RESp′yAssignment of pixel values of coordinate locations to RESPx,RESPyThe coordinate position is expressed, thereby acquiring the image after the distortion correction.
A distortion correction method for images shot by a redundant fisheye camera further comprises the step of satisfying theta (n) more than or equal to theta (n +1) for any angle theta in step S2; in the distortion parameter table, searching for theta (n) and gamma (n +1) corresponding to theta (n +1), and then satisfying that gamma (n) is less than or equal to gamma (n +1) and gamma (n +1) is less than or equal to gamma;
wherein n is the serial number of the pixel point, theta (n) and theta (n +1) represent adjacent discrete angles, gamma (n) and gamma (n +1) represent adjacent image heights, and theta (n), theta (n +1), gamma (n) and gamma (n +1) are obtained by searching from the optical distortion parameter table according to the known theta or gamma.
In a method for correcting distortion of images captured by a redundant fisheye camera, in step S3, a formula for calculating a first mapping function between a field angle θ and an image height γ includes:
when the value of y is known, then,
Figure BDA0003174851440000041
when the value of theta is known, the value of theta,
Figure BDA0003174851440000051
distortion correction of image shot by redundant fisheye cameraMethod, further, correcting the scaling factor SUThe calculation formula of (2) is as follows:
Figure BDA0003174851440000052
distortion scaling factor SIThe calculation formula is as follows:
Figure BDA0003174851440000053
wherein, RESHIRepresenting the maximum horizontal resolution, RESH, of the actual fish-eye distorted imageURepresenting maximum horizontal resolution, gamma, of the distortion corrected imagemaxThe CMOS imaging surface of the camera has half of the maximum width in the horizontal direction.
A method for correcting distortion of images shot by a redundant fisheye camera, and further RESPx,RESPyAnd RESp'x,RESp′yThe corresponding third mapping function of (a) is:
Figure BDA0003174851440000054
Figure BDA0003174851440000055
wherein gamma is the corresponding image height of P point, gammamaxHalf of the maximum width of the CMOS imaging surface of the camera in the horizontal direction;
Figure BDA0003174851440000056
is an included angle between the connecting line of the point P and the central point O of the shooting plane U and the horizontal direction, RESHI、RESVIRepresenting the maximum horizontal and vertical resolution, RESH, of the actual fish-eye distorted imageU、RESVURepresenting the maximum horizontal resolution and the maximum vertical resolution of the distortion corrected image.
Redundant fisheye camera shootingThe method for correcting distortion of an image further includes (RESp 'in step S5'x,RESp′y) Is interpolated from its neighboring surrounding pixel values.
A method for correcting distortion of images shot by a redundant fisheye camera, further, a pixel value VALpThe interpolation obtaining specifically includes: obtaining (RESp'x,RESp′y) 4 pixel point coordinates with nearest peripheral distance
Figure BDA0003174851440000061
Figure BDA0003174851440000062
Obtaining p0、p1、p2、p3The corresponding pixel values are respectively VALp0、VALp1、VALp2、VALp3
Calculating p0、p1、p2、p3Corresponding weight coefficient:
Figure BDA0003174851440000063
Figure BDA0003174851440000064
Figure BDA0003174851440000065
Figure BDA0003174851440000066
calculating a pixel value VAL according to the weight coefficientp
VALp=VALp0×w0+VALp1×w1+VALp2×w2+VALp4×w3
The method for correcting the distortion of the images shot by the redundant fisheye cameras further comprises the steps that the redundant fisheye cameras comprise a plurality of cameras and are installed in the same area, and all the cameras in the same installation area are subjected to primary distortion correction and secondary distortion correction to obtain a final image.
Has the advantages that:
1. in the embodiment of the invention, better imaging effect can be obtained by carrying out secondary distortion correction on the images shot by the redundant fisheye cameras.
2. In the embodiment of the invention, in the distortion correction process of the fisheye camera for the first time, the distortion correction is not carried out on the imaging of the fisheye camera by adopting the traditional polynomial, but on the basis of distortion parameters based on the discrete view field angle and the corresponding image height, the actual discrete view field angle and the image height myopia are further assumed to meet the preset relational expression, and the problem that the complicated calculation amount is increased by solving a high-order function in the traditional method is avoided. The correction result shows that a better distortion correction effect can be obtained by the secondary method.
3. In the embodiment of the invention, in the second distortion process, the difference correction plane is established, and according to the relation between the characteristic point of the coordinate of the difference correction plane and the characteristic point in the first distortion correction process, certain optical center deviation and focal plane angle error between an imaging surface and a CMOS plane, which are inevitably caused by the production process precision of different cameras in the production process, can be eliminated.
Drawings
The following drawings are only schematic illustrations and explanations of the present invention, and do not limit the scope of the present invention.
Fig. 1 is a mapping table of discrete view field angles and corresponding image heights obtained by acquiring actual experimental environment images by an experimental module under the condition of LENS configuration and designated CMOS in an embodiment of the present invention.
Fig. 2 is a schematic diagram of an imaging model principle of a fisheye camera in an embodiment of the invention.
Fig. 3 is a checkerboard picture without distortion correction taken by a fisheye camera in an embodiment of the invention.
FIG. 4 is a checkerboard picture with a first distortion correction according to an embodiment of the present invention.
FIG. 5 is a second distortion corrected picture according to an embodiment of the present invention.
Detailed Description
For a more clear understanding of the technical features, objects, and effects herein, embodiments of the present invention will now be described with reference to the accompanying drawings, in which like reference numerals refer to like parts throughout. For the sake of simplicity, the drawings are schematic representations of relevant parts of the invention and are not intended to represent actual structures as products. In addition, for simplicity and clarity of understanding, only one of the components having the same structure or function is schematically illustrated or labeled in some of the drawings.
As for the control system, the functional module, application program (APP), is well known to those skilled in the art, and may take any suitable form, either hardware or software, and may be a plurality of functional modules arranged discretely, or a plurality of functional units integrated into one piece of hardware. In its simplest form, the control system may be a controller, such as a combinational logic controller, a micro-programmed controller, or the like, so long as the operations described herein are enabled. Of course, the control system may also be integrated as a different module into one physical device without departing from the basic principle and scope of the invention.
The term "connected" in the present invention may include direct connection, indirect connection, communication connection, and electrical connection, unless otherwise specified.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, values, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, values, steps, operations, elements, components, and/or groups thereof. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items
It should be understood that the term "vehicle" or "vehicular" or other similar terms as used herein generally includes motor vehicles such as passenger automobiles including Sport Utility Vehicles (SUVs), buses, trucks, various commercial vehicles, watercraft including a variety of boats, ships, aircraft, and the like, and includes hybrid vehicles, electric vehicles, plug-in hybrid electric vehicles, hydrogen-powered vehicles, and other alternative fuel vehicles (e.g., fuels derived from non-petroleum sources). As referred to herein, a hybrid vehicle is a vehicle having two or more power sources, such as both gasoline-powered and electric-powered vehicles.
Further, the controller of the present disclosure may be embodied as a non-transitory computer readable medium on a computer readable medium containing executable program instructions executed by a processor, controller, or the like. Examples of computer readable media include, but are not limited to, ROM, RAM, Compact Disc (CD) -ROM, magnetic tape, floppy disk, flash drive, smart card, and optical data storage device. The computer readable recording medium CAN also be distributed over network coupled computer systems so that the computer readable medium is stored and executed in a distributed fashion, such as by a telematics server or Controller Area Network (CAN).
This embodiment provides redundant fisheye cameras, and redundant cameras are installed in the car, and redundant cameras are installed in different regions on the automobile body, such as position about, preceding, back, left and right.
At the position of the left side of the vehicle body, the redundant cameras are arranged in the same area of the left side vehicle body, when one camera is polluted or damaged, the other camera is automatically switched to be used, and the reliability is improved.
The redundant cameras can comprise two or more;
in order to enlarge the shooting angle, the camera adopts a fisheye camera.
The image data transmission adopts a vehicle-mounted Ethernet protocol and a TSN protocol so as to obtain high transmission distance and large flow transmission;
the embodiment provides a method for correcting distortion of images shot by a redundant fisheye camera, which specifically comprises the following steps:
the method comprises the steps of performing first distortion correction, namely acquiring a distorted image formed by a preset target image vertical to the ground through a redundant camera, and performing first distortion correction on the distorted image by utilizing an optical distortion parameter mapping table to acquire a first distortion corrected image;
second distortion correction, establishing a difference correction plane coordinate system, respectively obtaining a first coordinate point corresponding to the feature point of the preset target image in the first distortion correction image and a first coordinate point corresponding to the difference correction plane, and solving a homography matrix H by utilizing perspective transformationdapos, using a homography matrix Hdapos inverse transforms the first distorted image for a second distortion correction.
The first time of the aberration correction specifically includes:
the vehicle-mounted Ethernet camera selects a fisheye LENS basically, a LENS LENS provider provides an optical distortion parameter table for the LENS, and the optical distortion parameter table is a mapping table of discrete view field angles and corresponding image heights obtained by acquiring actual experimental environment images of an experimental module under the condition that the LENS is matched with a designated CMOS (complementary metal oxide semiconductor) as shown in figure 1.
Acquiring optical distortion parameters of a vehicle-mounted Ethernet camera, wherein the optical distortion parameters comprise a discrete view field angle theta and a mapping table of corresponding image height gamma of the discrete view field angle theta, which is acquired by an experiment module under the condition that a camera lens designates a CMOS chip, for actual experiment environment image acquisition;
according to the pinhole imaging principle, establishing a spatial coordinate relation of each pixel point of an actual shooting plane U corresponding to each point of a fisheye image on an imaging plane I, and establishing a first mapping function of a discrete view field angle theta and an object height lambda;
in particular, with reference to figure 2,
the object height is λ, the angle θ unit of the optical distortion parameter table in fig. 1 is degree, and the image height γ unit is mm.
If the distance from the LENS optical center to the actual shooting plane is 1, the relationship is satisfied, then:
λ ═ tan (θ), (γ - > θ) formula (1)
θ ═ arctan (λ), (θ - > γ) formula (2)
The first mapping function: λ tan (θ)
Constructing a second mapping function theta of the image height gamma and the discrete view field angle theta, G (gamma), according to the optical distortion parameter mapping table,
specifically, in the present embodiment, it is assumed that:
for any angle theta, theta (n) is more than or equal to theta (n + 1); in the distortion parameter table, searching for theta (n) and gamma (n +1) corresponding to theta (n +1), and then satisfying that gamma (n) is less than or equal to gamma (n +1) and gamma (n +1) is less than or equal to gamma;
n is the serial number of the pixel point, theta (n) and theta (n +1) represent adjacent discrete angles, gamma (n) and gamma (n +1) are adjacent image heights, and theta (n), theta (n +1), gamma (n) and gamma (n +1) are obtained by searching an optical distortion parameter table according to the known theta or gamma;
when the value of y is known, then,
Figure BDA0003174851440000111
when the value of theta is known, the value of theta,
Figure BDA0003174851440000112
according to the imaging geometric relation, solving a correction scaling factor S between an actual shooting plane U and a corresponding correction imaging plane and a corresponding distortion imaging planeUAnd distortion scaling factor SI
The method specifically comprises the following steps:
at this time, if the included angle between the OP connecting line and the horizontal direction is psi and the numeric area of psi is 0-180 degrees, the horizontal coordinate P of the point P on the actual shooting plane U is determinedxAnd a vertical coordinate PyAnd point p' is on the imaging plane IHorizontal coordinate p'xAnd vertical coordinate p'ySatisfies the relationship:
Px=λ×cos(ψ),Pyλ x sin (ψ) formula (5)
p′x=γ×cos(ψ),p′yFormula (6) ═ γ × sin (ψ)
Selecting the maximum width 2 gamma of the CMOS imaging surface of the practical Ethernet camera in the horizontal directionmaxAnd the maximum horizontal resolution RESH of the actual fish-eye distortion imageIMaximum vertical resolution RESVIAnd maximum horizontal resolution RESH of distortion corrected imageUMaximum vertical resolution RESVUThe method can obtain the lambda of the undistorted actual shot image by substituting the formula 1 and the formula 2maxAnd distortion scaling factor SIAnd correct the scaling factor SUAnd satisfies the relation:
Figure BDA0003174851440000113
Figure BDA0003174851440000114
using a first mapping function, a second mapping function and a corrective scaling factor SUAnd distortion scaling factor SIEstablishing pixel-level coordinates RESP of the corrected undistorted imagex,RESPyAnd pixel level coordinates RESp 'of distorted image'x,RESp′yAnd RESp 'to a corresponding third mapping function'x,RESp′yAssignment of pixel values of coordinate locations to RESPx,RESPyTo obtain an image with corrected distortion.
The method specifically comprises the following steps:
pixel-level coordinates RESp 'of imaging-plane fish-eye distortion image'x,RESp′yAnd pixel-level coordinates RESP of an undistorted image of an actual shooting planex,RESPyThe relationship is satisfied,
Figure BDA0003174851440000121
Figure BDA0003174851440000122
Figure BDA0003174851440000123
Figure BDA0003174851440000124
dividing both sides of the equation of the formula (9) and the equation of the formula (11) respectively, substituting the formula (1), the formula (5), the formula (6), the formula (7) and the formula (8) to obtain the RESPxAnd RESp'xCorresponding third mapping function of:
Figure BDA0003174851440000125
the same principle is that: dividing both sides of the equation of the formula (10) and the equation of the formula (12) respectively, substituting the formula (1), the formula (5), the formula (6), the formula (7) and the formula (8) to obtain the RESPyAnd RESp'yCorresponding third mapping function of:
Figure BDA0003174851440000126
wherein gamma is the corresponding image height of P point, gammamaxHalf of the maximum width of a CMOS imaging surface of the camera in the horizontal direction;
Figure BDA0003174851440000127
is an included angle between the connecting line of the point P and the central point O of the shooting plane U and the horizontal direction, RESHI、RESVIRepresenting the maximum horizontal and vertical resolution, RESH, of the actual fish-eye distorted imageU、RESVUTo representMaximum horizontal resolution of the distortion corrected image maximum horizontal resolution and maximum vertical resolution.
Taking a certain CMOS horizontal length of 4mm as an example, when the imaging surface just covers the horizontal direction of the CMOS, the horizontal resolution RESH of the imaged fish-eye distortion pictureI1280 pixels, when op corresponds to γ in horizontal directionmaxIs 2;
setting the minimum precision of the optical distortion parameter table to meet the requirement that gamma (n) is more than or equal to gamma (n) and less than or equal to gamma (n +1) and is 1.95;
γ (n +1) is 2.03, θ (n) is 55.5, θ (n +1) is 56,
then, the formula 1 is substituted, and the formula 2 is substituted to obtain the lambda of the corresponding point P on the top surface OP of the actual shooting plane U in the horizontal directionmaxIn the order of 1.4721, is,
setting horizontal resolution RESH of distortion-free image in actual shootingU1280 pixels, S can be obtained according to equation 4IIs 320, SUIs 434.75.
Distortion corrected image is actual shooting surface U image, and any pixel point coordinate (RESP) on the imagex,RESPy) With the formula (14) and the formula (15), the pixel coordinates (RESp ') corresponding to the image plane I image, which is the fish-eye distortion image, can be obtained'x,RESp′y) Because the pixel coordinates of the image are integers, the calculated corresponding pixel coordinates are floating point numbers (RESpx, RESpy), and therefore the pixel value of the actual P point can be obtained by estimating the values of the surrounding pixels of the P' point;
the specific estimation method is as follows:
(RESp′x,RESp′y) When the floating point is floating, 4 pixels p with the nearest distance around the floating point0、p1、p2、p3The coordinates are respectively
Figure BDA0003174851440000131
Figure BDA0003174851440000132
Obtaining p0、p1、p2、p3The corresponding pixel values are respectively VALp0、VALp1、VALp2、VALp3
Calculating p0、p1、p2、p3Corresponding weight coefficient:
Figure BDA0003174851440000133
Figure BDA0003174851440000134
Figure BDA0003174851440000135
Figure BDA0003174851440000136
calculating a pixel value VAL according to the weight coefficientp
VALp=VALp0×w0+VALp1×w1+VALp2×w2+VALp4×w3
Traversing P-point pixel coordinates (RESP) according to image resolution of distortion corrected imagesx,RESPy) And calculating pixel coordinates (RESp ') of the corresponding fish-eye distortion image p ' point 'x,RESp′y) Assigning the pixel value of P point as VAL calculated by P' point interpolationpThen, the first distortion correction of the fish-eye basic image is completed. The fish eye distortion image is shown in fig. 3, and the distortion correction image is shown in fig. 4.
The installation positions of the redundant cameras are different in space, so that the images of the shooting view fields of the redundant cameras are different, and the different Ethernet cameras are different in production process due to the fact that certain optical center deviation and focal plane angle errors exist between an imaging plane and a CMOS plane, and the fish eye distortion images are restored to the distortion correction images according to the established distortion correction mathematical model.
And the redundant camera difference correction eliminates the difference of the ROI (region of interest) of the distortion correction images of different redundant Ethernet cameras to the maximum extent.
Therefore, a second distortion correction needs to be performed on the basis of the first distortion corrected image;
second distortion correction, establishing a difference correction plane coordinate system, respectively obtaining a first coordinate point corresponding to the feature point of the preset target image in the first distortion correction image and a first coordinate point corresponding to the difference correction plane, and solving a homography matrix H by utilizing perspective transformationdapos, using a homography matrix Hdapos inverse transforms the first distorted image for a second distortion correction.
In the second distortion correction, a homography matrix H is useddaAnd pos utilizes inverse perspective transformation to enable all first coordinate points corresponding to the established difference correction plane to correspond to second coordinate points in the first distortion correction image, pixel values of the second coordinate points in the first distortion correction image are obtained, and the pixel values are assigned to the first coordinate points.
Presetting the characteristic points (T) of the target map in the coordinate system of the difference correction planexn,Tyn) At a first coordinate (R) corresponding to the plane of correction of the differencexn,Ryn) The mapping function of (d) is:
Rxn=Txn×SC+OFSTx
Ryn=Tyn×SC+OFSTy
where SC is a planar coordinate scale transformation factor, OFSTxAnd OFSTyHorizontal and vertical pixel offsets, respectively;
step S101, acquiring a preset target map vertical to the ground through a redundant camera, and acquiring actual physical coordinates (T) of a plurality of characteristic points in the preset target map on a target map planexn,Tyn);
Step S102, according to the actual physical coordinate (T)xn,Tyn) Solving the actual physical coordinate (T) in the actual shooting surface U after distortion correctionxn,Tyn) Pixel coordinates (RESP) of corresponding feature pointsxn,RESPyn);
Step S103, establishing a difference correction plane coordinate system, and solving the feature point coordinate (R) which is mapped to the corresponding feature point coordinate under the difference correction plane coordinate system by the coordinates of a plurality of feature points in the target mapxn,Ryn);
Step S104, according to the perspective transformation, solving (R)xn,Ryn) To (RESP)xn,RESPyn) Homography matrix H ofdapos:
Perspective transformation:
Figure BDA0003174851440000151
Hdapos:
Figure BDA0003174851440000152
step S105, again using perspective transformation change, correcting all pixel coordinates (R) of image with differencexi,Ryi) Carry-over 7 substitution (R)xn,Ryn) Using H obtained in step S104dapos, calculating corresponding coordinates (RESP) of the corresponding corrected imagexi,RESPyi)。
What has been described above is only a preferred embodiment of the present invention, and the present invention is not limited to the above examples. It is clear to those skilled in the art that the form in this embodiment is not limited thereto, and the adjustable manner is not limited thereto. It is to be understood that other modifications and variations, which may be directly derived or suggested to one skilled in the art without departing from the basic concept of the invention, are to be considered as included within the scope of the invention.

Claims (12)

1. A method for correcting distortion of images shot by a redundant fisheye camera is characterized by at least comprising the following steps:
the method comprises the steps of performing first distortion correction, namely acquiring a distorted image formed by a preset target image vertical to the ground through a redundant camera, and performing first distortion correction on the distorted image through an optical distortion parameter mapping table to acquire a first distortion corrected image;
second distortion correction, establishing a difference correction plane coordinate system, respectively obtaining a first coordinate point corresponding to the feature point of the preset target image in the first distortion correction image and a first coordinate point corresponding to the difference correction plane, and solving a homography matrix H by utilizing perspective transformationdapos, using a homography matrix Hdapos inverse transforms the first distorted image for a second distortion correction.
2. The method of claim 1, wherein a homography H is obtained in the second distortion correctiondapos, utilizing inverse perspective transformation to enable all first coordinate points corresponding to the established difference correction plane to correspond to second coordinate points in the first distortion correction image, obtaining pixel values of the second coordinate points in the first distortion correction image, and assigning the pixel values to the first coordinate points.
3. The method for correcting distortion of images captured by a redundant fisheye camera of claim 1, wherein the characteristic points (T) of the target map are preset in the plane coordinate system for difference correctionxn,Tyn) First coordinate (R) corresponding to the difference correction planexn,Ryn) The mapping function of (d) is:
Rxn=Txn×SC+OFSTx
Ryn=Tyn×SC+OFSTy
where SC is a planar coordinate scale transformation factor, OFSTxAnd OFSTyHorizontal and vertical pixel offsets, respectively.
4. The method of claim 1, wherein the second distortion correction further comprises:
step S101, acquiring a preset target map vertical to the ground through a redundant cameraAcquiring the actual physical coordinates (T) of a plurality of characteristic points in the preset target map on the target map planexn,Tyn);
Step S102, according to the actual physical coordinate (T)xn,Tyn) Solving the actual physical coordinate (T) in the actual shooting surface U after distortion correctionxn,Tyn)
Pixel coordinates (RESP) of corresponding feature pointsxn,RESPyn);
Step S103, establishing a difference correction plane coordinate system, and solving the feature point coordinate (R) which is mapped to the corresponding feature point coordinate under the difference correction plane coordinate system by the coordinates of a plurality of feature points in the target mapxn,Ryn);
Step S104, according to the perspective transformation, solving (R)xn,Ryn) To (RESP)xn,RESPyn) Homography matrix H ofdapos:
Perspective transformation:
Figure FDA0003174851430000021
Hdapos:
Figure FDA0003174851430000022
wherein h1, h2, h3, h4, h5, h6, h7 and h8 represent element values in a matrix;
step S105, again using perspective transformation, all pixel coordinates (R) of the difference-corrected imagexi,Ryi) Replacement (R)xn,Ryn) Using H obtained in step S104dapos, calculating corresponding coordinates (RESP) of the corresponding corrected imagexi,RESPyi)。
5. The method of claim 1, wherein the first distortion correction comprises:
step S1, acquiring optical distortion parameters of the vehicle-mounted Ethernet camera, wherein the optical distortion parameters comprise a discrete view field angle theta and a mapping table of corresponding image height gamma of the discrete view field angle theta, which is used for acquiring actual experiment environment images by the experiment module under the condition that the camera lens designates the CMOS chip;
step S2, according to the pinhole imaging principle, establishing the space coordinate relation of each pixel point of the actual shooting plane U corresponding to each point of the fisheye image on the imaging plane I, and establishing a first mapping function of the discrete view field angle theta and the object height lambda;
λ=tan(θ)
step S3, constructing a second mapping function θ of the image height γ and the discrete field angle θ as G (γ) according to the optical distortion parameter mapping table;
step S4, according to the imaging geometrical relation, solving the correction scaling factor S between the actual shooting plane U and the corresponding correction imaging plane and distortion imaging planeUAnd distortion scaling factor SI
Step S5: using a first mapping function, a second mapping function and a corrective scaling factor SUAnd distortion scaling factor SIEstablishing pixel-level coordinates RESP of the corrected undistorted imagex,RESPyAnd pixel level coordinates RESp 'of distorted image'x,RESp′yAnd RESp 'to a corresponding third mapping function'x,RESp′yAssignment of pixel values of coordinate locations to RESPx,RESPyThe coordinate position is expressed, thereby acquiring the image after the distortion correction.
6. The method for correcting distortion of an image captured by a redundant fisheye camera as claimed in claim 5, wherein in step S2, θ (n) ≦ θ (n +1) is satisfied for any angle θ; in the distortion parameter table, searching for gamma (n) and gamma (n +1) corresponding to theta (n) and theta (n +1), and then the gamma corresponding to theta should satisfy that gamma (n) is less than or equal to gamma (n + 1);
wherein n is the serial number of the pixel point, theta (n) and theta (n +1) represent adjacent discrete angles, gamma (n) and gamma (n +1) represent adjacent image heights, and theta (n), theta (n +1), gamma (n) and gamma (n +1) are obtained by searching from the optical distortion parameter table according to the known theta or gamma.
7. The method of claim 5, wherein in step S3, the formula for calculating the first mapping function between the angle of field θ and the image height γ comprises:
when the value of y is known, then,
Figure FDA0003174851430000031
when the value of theta is known, the value of theta,
Figure FDA0003174851430000032
8. the method of claim 5, wherein the distortion correction of the redundant fisheye camera images is performed by correcting the scaling factor SUThe calculation formula of (2) is as follows:
Figure FDA0003174851430000041
distortion scaling factor SIThe calculation formula is as follows:
Figure FDA0003174851430000042
wherein, RESHIRepresenting the maximum horizontal resolution, RESH, of the actual fish-eye distorted imageURepresenting maximum horizontal resolution, gamma, of the distortion corrected imagemaxThe CMOS imaging surface of the camera has half of the maximum width in the horizontal direction.
9. The method of claim 5, wherein the RESP is a representation of a video image captured by a redundant fisheye camerax,RESPyAnd RESp′x,RESp′yThe corresponding third mapping function of (a) is:
Figure FDA0003174851430000043
Figure FDA0003174851430000044
wherein gamma is the corresponding image height of the P point,
γmaxhalf of the maximum width of the CMOS imaging surface of the camera in the horizontal direction;
Figure FDA0003174851430000045
is an included angle between the connecting line of the point P and the central point O of the shooting plane U and the horizontal direction, RESHI、RESVIRepresenting the maximum horizontal and vertical resolution, RESH, of the actual fish-eye distorted imageU、RESVURepresenting the maximum horizontal resolution and the maximum vertical resolution of the distortion corrected image.
10. The method for correcting distortion of a picture captured by a redundant fish-eye camera according to claim 5, wherein in step S5, (RESp'x,RESp′y) Is interpolated from its neighboring surrounding pixel values.
11. The method of claim 10, wherein the pixel value VAL is a value of a pixel valuepThe interpolation obtaining specifically includes: obtaining (RESp'x,RESp′y) 4 pixel point coordinates with nearest peripheral distance
Figure FDA0003174851430000051
Figure FDA0003174851430000052
Obtaining p0、p1、p2、p3The corresponding pixel values are respectively VALp0、VALp1、VALp2、VALp3
Calculating p0、p1、p2、p3Corresponding weight coefficient:
Figure FDA0003174851430000053
Figure FDA0003174851430000054
Figure FDA0003174851430000055
Figure FDA0003174851430000056
calculating a pixel value VAL according to the weight coefficientp
VALp=VALp0×w0+VALp1×w1+VALp2×w2+VALp4×w3。
12. The method as claimed in claim 1, wherein the redundant fisheye camera comprises a plurality of cameras and is installed in the same area, and the first distortion correction and the second distortion correction are performed on all the cameras in the same installation area to obtain the final image.
CN202110829146.7A 2021-07-22 Distortion correction method for images shot by redundant fisheye cameras Active CN113538283B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110829146.7A CN113538283B (en) 2021-07-22 Distortion correction method for images shot by redundant fisheye cameras

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110829146.7A CN113538283B (en) 2021-07-22 Distortion correction method for images shot by redundant fisheye cameras

Publications (2)

Publication Number Publication Date
CN113538283A true CN113538283A (en) 2021-10-22
CN113538283B CN113538283B (en) 2024-04-30

Family

ID=

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115082355A (en) * 2022-07-20 2022-09-20 北京智汇云舟科技有限公司 Fisheye camera full-pixel image correction method and device, fisheye camera and medium
CN115314689A (en) * 2022-08-05 2022-11-08 深圳海翼智新科技有限公司 Projection correction method, projection correction device, projector and computer program product

Citations (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060232569A1 (en) * 2005-04-15 2006-10-19 Microsoft Corporation Direct homography computation by local linearization
US20090141148A1 (en) * 2007-12-04 2009-06-04 Kosuke Imamura Distortion-corrected image generation unit and distortion-corrected image generation method
TW201019715A (en) * 2008-11-12 2010-05-16 Avisonic Technology Corp Fisheye correction with perspective distortion reduction method and related image processor
CN103996172A (en) * 2014-05-08 2014-08-20 东北大学 Fish-eye image correction method based on multistep correction
WO2015043507A1 (en) * 2013-09-27 2015-04-02 比亚迪股份有限公司 Image processing method and apparatus for cars, method for generating car surround view image, and car surround view system
CN106408511A (en) * 2016-09-22 2017-02-15 苏州大学 Overlook conversion method, overlook image acquisition method and mapping table construction method of fisheye image
WO2017092631A1 (en) * 2015-11-30 2017-06-08 宁波舜宇光电信息有限公司 Image distortion correction method for fisheye image, and calibration method for fisheye camera
CN107160046A (en) * 2017-04-18 2017-09-15 上海柏楚电子科技有限公司 A kind of vision aids in the camera calibration method of large format lathe plate cutting
RU2631765C1 (en) * 2016-04-26 2017-09-26 Общество с ограниченной ответственностью "Аби Девелопмент" Method and system of correcting perspective distortions in images occupying double-page spread
CN107248178A (en) * 2017-06-08 2017-10-13 上海赫千电子科技有限公司 A kind of fisheye camera scaling method based on distortion parameter
CN107274352A (en) * 2017-04-26 2017-10-20 睿舆自动化(上海)有限公司 A kind of image processing method and the real-time sampling system applied to lens distortion and photography distortion correction
WO2018119565A1 (en) * 2016-12-26 2018-07-05 深圳大学 Method and device for correcting and reconstructing a barrel distorted image
KR101912396B1 (en) * 2017-06-13 2018-10-26 주식회사 아이닉스 Apparatus and Method for Generating Image at any point-view based on virtual camera
CN108830810A (en) * 2018-06-07 2018-11-16 辽宁工业大学 A kind of fisheye image distortion correction method based on rectangular projection
KR20180127671A (en) * 2017-05-22 2018-11-30 주식회사 와이즈오토모티브 Method of automatic calibration of AVM system
CN108986172A (en) * 2018-07-25 2018-12-11 西北工业大学 A kind of single-view linear camera scaling method towards small depth of field system
JP2019012361A (en) * 2017-06-29 2019-01-24 キヤノン株式会社 Information processor, program, and method for information processing
WO2019019172A1 (en) * 2017-07-28 2019-01-31 Qualcomm Incorporated Adaptive Image Processing in a Robotic Vehicle
CN109544484A (en) * 2019-02-20 2019-03-29 上海赫千电子科技有限公司 A kind of method for correcting image and device
CN109961484A (en) * 2017-12-22 2019-07-02 比亚迪股份有限公司 Camera calibration method, device and vehicle
CN110020656A (en) * 2019-01-30 2019-07-16 阿里巴巴集团控股有限公司 Bearing calibration, device and the equipment of image
CN110084842A (en) * 2019-05-05 2019-08-02 广东电网有限责任公司 A kind of secondary alignment methods of machine user tripod head servo and device
KR102076635B1 (en) * 2018-10-16 2020-02-12 (주)캠시스 Apparatus and method for generating panorama image for scattered fixed cameras
CN111199528A (en) * 2020-01-15 2020-05-26 江苏北斗星通汽车电子有限公司 Fisheye image distortion correction method
US20200273205A1 (en) * 2017-09-08 2020-08-27 Sony Interactive Entertainment Inc. Calibration apparatus, calibration system, and calibration method
CN111626227A (en) * 2020-05-28 2020-09-04 深圳市博维远景科技有限公司 Method for realizing vehicle bottom perspective panoramic system based on binocular vision

Patent Citations (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060232569A1 (en) * 2005-04-15 2006-10-19 Microsoft Corporation Direct homography computation by local linearization
US20090141148A1 (en) * 2007-12-04 2009-06-04 Kosuke Imamura Distortion-corrected image generation unit and distortion-corrected image generation method
TW201019715A (en) * 2008-11-12 2010-05-16 Avisonic Technology Corp Fisheye correction with perspective distortion reduction method and related image processor
WO2015043507A1 (en) * 2013-09-27 2015-04-02 比亚迪股份有限公司 Image processing method and apparatus for cars, method for generating car surround view image, and car surround view system
CN103996172A (en) * 2014-05-08 2014-08-20 东北大学 Fish-eye image correction method based on multistep correction
WO2017092631A1 (en) * 2015-11-30 2017-06-08 宁波舜宇光电信息有限公司 Image distortion correction method for fisheye image, and calibration method for fisheye camera
RU2631765C1 (en) * 2016-04-26 2017-09-26 Общество с ограниченной ответственностью "Аби Девелопмент" Method and system of correcting perspective distortions in images occupying double-page spread
CN106408511A (en) * 2016-09-22 2017-02-15 苏州大学 Overlook conversion method, overlook image acquisition method and mapping table construction method of fisheye image
WO2018119565A1 (en) * 2016-12-26 2018-07-05 深圳大学 Method and device for correcting and reconstructing a barrel distorted image
CN107160046A (en) * 2017-04-18 2017-09-15 上海柏楚电子科技有限公司 A kind of vision aids in the camera calibration method of large format lathe plate cutting
CN107274352A (en) * 2017-04-26 2017-10-20 睿舆自动化(上海)有限公司 A kind of image processing method and the real-time sampling system applied to lens distortion and photography distortion correction
KR20180127671A (en) * 2017-05-22 2018-11-30 주식회사 와이즈오토모티브 Method of automatic calibration of AVM system
CN107248178A (en) * 2017-06-08 2017-10-13 上海赫千电子科技有限公司 A kind of fisheye camera scaling method based on distortion parameter
KR101912396B1 (en) * 2017-06-13 2018-10-26 주식회사 아이닉스 Apparatus and Method for Generating Image at any point-view based on virtual camera
JP2019012361A (en) * 2017-06-29 2019-01-24 キヤノン株式会社 Information processor, program, and method for information processing
WO2019019172A1 (en) * 2017-07-28 2019-01-31 Qualcomm Incorporated Adaptive Image Processing in a Robotic Vehicle
US20200273205A1 (en) * 2017-09-08 2020-08-27 Sony Interactive Entertainment Inc. Calibration apparatus, calibration system, and calibration method
CN109961484A (en) * 2017-12-22 2019-07-02 比亚迪股份有限公司 Camera calibration method, device and vehicle
CN108830810A (en) * 2018-06-07 2018-11-16 辽宁工业大学 A kind of fisheye image distortion correction method based on rectangular projection
CN108986172A (en) * 2018-07-25 2018-12-11 西北工业大学 A kind of single-view linear camera scaling method towards small depth of field system
KR102076635B1 (en) * 2018-10-16 2020-02-12 (주)캠시스 Apparatus and method for generating panorama image for scattered fixed cameras
CN110020656A (en) * 2019-01-30 2019-07-16 阿里巴巴集团控股有限公司 Bearing calibration, device and the equipment of image
CN109544484A (en) * 2019-02-20 2019-03-29 上海赫千电子科技有限公司 A kind of method for correcting image and device
CN110084842A (en) * 2019-05-05 2019-08-02 广东电网有限责任公司 A kind of secondary alignment methods of machine user tripod head servo and device
CN111199528A (en) * 2020-01-15 2020-05-26 江苏北斗星通汽车电子有限公司 Fisheye image distortion correction method
CN111626227A (en) * 2020-05-28 2020-09-04 深圳市博维远景科技有限公司 Method for realizing vehicle bottom perspective panoramic system based on binocular vision

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
徐芳;刘晶红;: "航空大视场角镜头畸变校正技术", 计算机应用, no. 09, pages 2623 - 2626 *
李伟;: "图像几何校正算法和反混叠综述", 现代计算机, no. 03, pages 83 - 87 *
王赛男;刘涛;左震宇;: "鱼眼镜头畸变校正算法的设计", 科技创新导报, no. 17, pages 9 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115082355A (en) * 2022-07-20 2022-09-20 北京智汇云舟科技有限公司 Fisheye camera full-pixel image correction method and device, fisheye camera and medium
CN115314689A (en) * 2022-08-05 2022-11-08 深圳海翼智新科技有限公司 Projection correction method, projection correction device, projector and computer program product

Similar Documents

Publication Publication Date Title
TWI491524B (en) Surrounding bird view monitoring image generation method and training method, automobile-side device, and training device thereof
US8837858B2 (en) Apparatus and method for correcting image distortion of rear camera
CN102915525B (en) Method for correcting barrel distortion of images for full-frame fish-eye lenses
KR20170077223A (en) Online calibration of a motor vehicle camera system
CN104917955A (en) Image Transformation And Multi-View Output Systems And Methods
CN106780374B (en) Fisheye image distortion correction method based on fisheye imaging model
JP2008187566A (en) Camera calibration apparatus and method and vehicle
US20110285856A1 (en) Image correction method for camera system
CN111199528A (en) Fisheye image distortion correction method
CN111292278B (en) Image fusion method and device, storage medium and terminal
CN108495066B (en) Distortion correction method, device and system for -angle lens
CN106886976B (en) Image generation method for correcting fisheye camera based on internal parameters
Chang et al. Auto-calibration around-view monitoring system
CN111652937B (en) Vehicle-mounted camera calibration method and device
CN113034616A (en) Camera external reference calibration method and system for vehicle all-round looking system and all-round looking system
CN111798375A (en) All-round-view image splicing method, system, device, storage medium and mixer truck
KR101465607B1 (en) Distorted Image Processing Method For Fish-Eye Lens
CN105335927B (en) Fish eye images processing method, device and vehicle
CN113538283A (en) Distortion correction method for images shot by redundant fisheye cameras
CN113538283B (en) Distortion correction method for images shot by redundant fisheye cameras
CN105323447B (en) More fish eye images processing methods, device and vehicle
CN110910311A (en) Automatic splicing method for multi-channel panoramic camera based on two-dimensional code
CN116740192A (en) Calibration method and calibration system of vehicle-mounted looking-around system and intelligent automobile
CN106097357B (en) The bearing calibration of auto-panorama camera
KR101230909B1 (en) Apparatus and method for processing wide angle image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant