CN109035170B - Self-adaptive wide-angle image correction method and device based on single grid image segmentation mapping - Google Patents
Self-adaptive wide-angle image correction method and device based on single grid image segmentation mapping Download PDFInfo
- Publication number
- CN109035170B CN109035170B CN201810834335.1A CN201810834335A CN109035170B CN 109035170 B CN109035170 B CN 109035170B CN 201810834335 A CN201810834335 A CN 201810834335A CN 109035170 B CN109035170 B CN 109035170B
- Authority
- CN
- China
- Prior art keywords
- grid
- distortion
- correction
- distorted
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 73
- 238000013507 mapping Methods 0.000 title claims abstract description 41
- 238000003702 image correction Methods 0.000 title claims abstract description 15
- 238000003709 image segmentation Methods 0.000 title abstract description 4
- 238000012937 correction Methods 0.000 claims abstract description 79
- 230000011218 segmentation Effects 0.000 claims abstract description 18
- 238000012545 processing Methods 0.000 claims abstract description 12
- 238000005457 optimization Methods 0.000 claims abstract description 10
- 238000007781 pre-processing Methods 0.000 claims abstract description 5
- 230000003044 adaptive effect Effects 0.000 claims description 11
- 238000009826 distribution Methods 0.000 claims description 10
- 230000001174 ascending effect Effects 0.000 claims description 6
- 239000000126 substance Substances 0.000 claims description 4
- 238000001914 filtration Methods 0.000 claims description 2
- 238000009499 grossing Methods 0.000 claims description 2
- 238000012163 sequencing technique Methods 0.000 claims description 2
- 230000002194 synthesizing effect Effects 0.000 claims 1
- 238000000605 extraction Methods 0.000 abstract description 8
- 230000008569 process Effects 0.000 description 11
- 230000000694 effects Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 7
- 230000003287 optical effect Effects 0.000 description 6
- 238000003860 storage Methods 0.000 description 6
- 238000011084 recovery Methods 0.000 description 5
- 238000001514 detection method Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 238000010845 search algorithm Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000005452 bending Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005192 partition Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
- 239000011800 void material Substances 0.000 description 1
Images
Classifications
-
- G06T5/80—
Abstract
The invention discloses a self-adaptive wide-angle image correction method and a self-adaptive wide-angle image correction device based on single grid image segmentation mapping, which have the characteristics of simple operation and strong practicability, not only realize the automatic extraction of control point pairs, but also can self-adaptively carry out the optimal segmentation correction according to the distortion degree of a lens, realize the automatic correction processing and improve the local and global correction precision. The method comprises the following steps: preprocessing the distorted grid template picture to obtain a single-pixel binary distorted grid template picture; extracting control point pairs in the distorted grid template graph by using the neighborhood characteristics of the grid intersection points; establishing a minimum piecewise optimization model of radial distortion; establishing a reverse correction mapping table to obtain a coordinate corresponding relation between a correction result image and an original distorted image; carrying out coordinate reverse mapping processing to obtain a first correction result image; and performing pixel interpolation on the first correction result image by using a bilinear interpolation algorithm, and recovering the gray value of the first correction result image to obtain a complete second correction result image.
Description
Technical Field
The invention relates to the technical field of wide-angle image correction, in particular to a self-adaptive wide-angle image correction method and device based on single grid image segmentation mapping.
Background
Compared with a common lens, the wide-angle lens has the characteristics of short focal length, large field of view, long depth of field and the like, can acquire images containing more target information in a wider field of view, and greatly reduces visual blind areas. Therefore, the method is widely applied to various visual fields such as industrial monitoring, video monitoring, auxiliary driving, robot navigation, medical endoscope, virtual reality and the like. However, due to the manufacturing defects of the wide-angle lens and the non-linear imaging mechanism, the photographed image has serious barrel distortion. The wide-angle barrel distortion mainly includes thin prism distortion, tangential distortion, and radial distortion. The nonlinear distortion not only distorts and deforms the original geometric structure of the image to affect the normal visual effect, but also reduces the geometric position accuracy of the scene target in the image, so that the wide-angle distortion needs to be corrected to actually put the wide-angle lens into practical application. Radial distortion is a leading factor of wide-angle distortion, and causes pixel points to shrink in the radial direction, so that the scaling of annular regions with different distances from an optical axis in an image is inconsistent. The farther a scene target is from the optical axis of the lens or the larger the included angle formed by the connecting line of the light from the target point to the center of the lens and the optical axis is, the more serious the shrinkage distortion degree is, therefore, most correction researches are mainly directed to wide-angle correction based on radial distortion.
The basic operation steps of distortion correction are: (1) firstly, determining a distortion center, namely a point without distortion in a distorted image; (2) then modeling and analyzing the distortion according to the distortion principle, and solving out a distortion coefficient; (3) then mapping pixel points of the correction graph to corresponding positions in the distortion graph through reverse mapping; (4) and finally, restoring the gray value of the pixel point by adopting an interpolation algorithm.
At present, distortion correction of wide-angle lenses is mainly classified into two categories: template method and parameter-known method. The parameter-known method is used for establishing an equivalent spherical model by using camera parameters (such as focal length and field angle of a lens) without a template, so as to simulate wide-angle distortion. Although the principle is simple, the correction precision is not high, and the related parameters of the lens need to be known in advance. 201611008743.9 patent application, a method and system for correcting distortion of super wide angle camera, uses a parameter known method to obtain a correction mapping relation from a rectangular coordinate system to a spherical seating system on a distorted image projection value panoramic spherical surface. The template method obtains the coordinate mapping relation between the distorted image and the ideal image point pair by manufacturing a calibration template, so that the correction precision is high, but the correction effect depends on the acquisition quality of a template image and the optical axis of a lens is required to be perpendicular to the template. High quality template maps meeting the constraint requirements can be obtained by dedicated acquisition means. The template has various structural shapes, and commonly used templates comprise a grid template, a concentric circle template, a checkerboard template, an equihexagonal lattice template and the like. In the conventional template correction method, a high-order polynomial model or a division model is mainly used for modeling the wide-angle distortion, but the method is only suitable for wide-angle lenses with small distortion degrees. For a large-field-angle lens with serious distortion, the two models can only effectively correct the area around the distortion center, but the correction precision is low and the effect is poor for the four corner edges of the image. For example, in patent application No. 201410424349.8, a method for correcting an image after calibration of a fisheye lens is similar to a zhang's calibration method, parameters of a high-order polynomial model are obtained by using a checkerboard grid calibration method, and due to the integrity requirement of grid blocks of a checkerboard template graph, control points in the template graph cannot be paved in the whole view angle area, so that the corner area of the image cannot be calibrated, and therefore, all areas of a distorted image cannot be effectively recovered. 201510514238.0 patent application, a method for accurately correcting image distortion of super wide-angle camera, improves the method on the basis, and improves the correction accuracy and the correction visual angle range to a certain extent, but the whole correction effect is poor.
Disclosure of Invention
At least one of the objectives of the present invention is to overcome the above problems in the prior art, and provide a method and an apparatus for adaptive wide-angle image correction based on single-grid-map segmentation mapping, which have the characteristics of simple operation and strong practicability, and not only implement automatic extraction of control point pairs, but also perform adaptive optimal segmentation correction according to the distortion degree (field angle) of the lens, implement automatic correction processing, and improve the local and global correction accuracy.
In order to achieve the above object, the present invention adopts the following aspects.
An adaptive wide-angle image correction method based on single-grid-map segmentation mapping, comprising the following steps:
preprocessing the distorted grid template picture to obtain a single-pixel binary distorted grid template picture; based on the single-pixel binary distortion grid template graph, extracting control point pairs in the distortion grid template graph by using neighborhood characteristics of grid intersections; establishing a minimum piecewise optimization model of radial distortion based on control point pairs in the distorted grid template graph;
establishing a reverse correction mapping table to obtain a coordinate corresponding relation between a correction result image and an original distorted image; based on the correction mapping table, carrying out coordinate reverse mapping processing to obtain a first correction result image; and performing pixel interpolation on the first correction result image by using a bilinear interpolation algorithm, and recovering the gray value of the first correction result image to obtain a complete second correction result image.
An adaptive wide-angle image correction apparatus based on single-grid-map segmentation mapping, comprising at least one processor, and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform any of the methods described herein.
In summary, due to the adoption of the technical scheme, the invention at least has the following beneficial effects:
the extraction of the control point pairs based on the directional search of the grid lines in the subareas realizes the automatic acquisition of the control point pairs of the single-grid template graph, and reduces the operation complexity of manual marking and positioning of the control point pairs in the traditional method; in addition, compared with an angular point detection algorithm, the provided algorithm improves the positioning accuracy of the distorted grid points to a certain extent;
the high-precision correction of the large-distortion wide-angle lens is realized through the self-adaptive distortion correction model based on polynomial piecewise optimization, and compared with a single curve fitting method, the method improves the local correction precision; compared with the traditional template method, the method has better integral correction effect, can well correct the image corners with the most serious distortion, and obtains satisfactory recovery effect; furthermore, this correction advantage in overall performance becomes more pronounced as the field angle of the wide-angle lens increases.
Drawings
Fig. 1 is a flowchart of an adaptive wide-angle image correction method based on single-grid-map segmentation mapping according to an exemplary embodiment of the present invention.
Fig. 2 is a diagram of a distorted mesh template according to an exemplary embodiment of the present invention.
Fig. 3 is a refined single-pixel binary distortion mesh template map in accordance with an exemplary embodiment of the present invention.
Fig. 4 is a schematic diagram of a relative positional relationship index coordinate system according to an exemplary embodiment of the present invention.
Fig. 5 is a schematic diagram of ideal distance versus distortion distance in accordance with an exemplary embodiment of the present invention.
Fig. 6 is a diagram illustrating CoD searched in a refined binary template graph according to an exemplary embodiment of the present invention.
Fig. 7 is a diagram of four corner distance point pair information according to an exemplary embodiment of the present invention.
Fig. 8 is a single distorted grid template diagram according to an exemplary embodiment of the present invention.
Fig. 9 is a schematic structural diagram of an adaptive wide-angle image correction apparatus based on single-grid-map segmentation mapping according to an exemplary embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and embodiments, so that the objects, technical solutions and advantages of the present invention will be more clearly understood. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiments of the invention disclose a self-adaptive correction algorithm based on single-grid template image polynomial sectional mapping for a large-distortion high-resolution wide-angle lens. The method belongs to an improved method based on a template radial distortion correction algorithm, and mainly comprises a distortion calibration process and a distortion correction process. For Distortion calibration, firstly, a partition orientation search algorithm is adopted in a refined single-grid template graph to obtain an actual Coordinate (DCP) of a Distortion grid Point and a Coordinate (Ideal Coordinate Point, ICP) corresponding to an Ideal regular grid graph, and automatic extraction of a Control Point Pair (CPP) is realized. Then, the optimal Distortion Center (CoD) is estimated using a point-to-point search algorithm, thereby obtaining the Distortion distance and ideal distance of the CPP to the CoD. And further, a distortion model of low-order polynomial piecewise fitting is established, and local radial distortion is fitted in a self-adaptive mode. And solving the optimal segmentation number and the fitting parameters by a least square algorithm. Finally, a coordinate transformation mapping table from the correction image to the distortion image is generated by adopting an inverse mapping method. In the distortion correction process, a final correction result is rapidly recovered through a reverse mapping table and a bilinear interpolation algorithm.
The invention selects the grid template to carry out distortion calibration, because the grid template can be paved over the view field range of the whole lens, and more control point pairs reflecting edge distortion information can be obtained. According to the self-adaptive correction algorithm for polynomial piecewise optimization, on one hand, the extraction method of control point pairs is improved, and automatic processing is realized; on the other hand, the self-adaptive piecewise polynomial model is adopted to replace the traditional single polynomial distortion model so as to improve the local and global correction precision. The wide-angle lens self-adaptive subsection correction method can self-adaptively determine the optimal subsection number of the radial distortion model according to the distortion degree of the lens, has semi-automatic characteristic in the operation process, can achieve higher correction precision, and can ensure that the image corners with serious distortion also obtain satisfactory recovery effect. In addition, as the angle of view of the lens is increased, the advantage is more prominent.
Fig. 1 illustrates an adaptive wide-angle image correction method based on single-grid-map segmentation mapping, which includes two processes of distortion calibration and distortion correction, according to an exemplary embodiment of the present invention. The method of this embodiment essentially comprises the steps of:
step 101: preprocessing the distorted grid template picture to obtain a single-pixel binary distorted grid template picture
Due to the uneven ambient lighting and the special configuration of the lens, the wide-angle image may exhibit a "dark angle" phenomenon (as shown in fig. 2) in which the hue becomes gradually deeper from the center to the edge, which may affect the extraction of the grid points. The conventional global thresholding algorithm cannot achieve an ideal segmentation effect, and can cause line breakage and edge noise. For the areas outside the focus in the template map, the common corner detection algorithm may cause missing detection or false detection, and the positioning accuracy is also poor. In addition, most template-based methods utilize distance criteria to obtain the four neighborhoods of distorted grid intersections, thereby obtaining the point-to-point positional relationship relative to the normalized grid template and obtaining ideal undistorted coordinates. However, this method is not suitable for a wide-angle lens with a large field of view and severe distortion, because the line bending degree is severe and the distance is no longer reliable as a criterion of relative position distribution.
Therefore, in the process of this step, the distorted mesh template map is grayed, and is divided into a plurality of blocks with different sizes (the size of the central area is large, and the size of the peripheral edge is smaller) to be respectively subjected to adaptive threshold segmentation, and then a complete binary image of the distorted mesh template map is synthesized. Further, smoothing is carried out by using median filtering, and finally, a refined single-pixel binary distortion grid template map (shown in figure 3) is obtained through reverse color processing.
Step 102: based on the single-pixel binary distortion grid template graph, the neighborhood characteristics of the grid intersection points are utilized to extract control point pairs in the distortion grid template graph.
Specifically, the neighborhood characteristics (branch structure information) of the grid intersection may be used to accurately locate the control point. And then, by combining a method of directional search along grid lines in subareas, obtaining the relative position distribution among the distorted grid points, thereby solving the ideal coordinates corresponding to the distorted points.
The control point pair automatic extraction processing procedure is as follows:
1. a relative position Relationship index Coordinate System (DRCS) is established as shown in fig. 4, and is used to describe the relative position Relationship (the orientation and the number of grid blocks spaced) of the grid points in the ideal normalized template corresponding to the distorted grid template. The grid point positions are described using Distribution Index Coordinates (DIC). Wherein the origin P of the coordinate system(0,0)Is the closest grid intersection to the distorted template map center point. The horizontal direction to the right is the x-axis indexing direction, and the vertical direction to the bottom is the y-axis indexing direction. P is(0,-1)、P(0,1)、P(-1,0)And P(1,0)Are respectively P(0,0)And nearest neighbor index positions in four directions of up, down, left and right.
2. And (3) extracting distortion grid intersection points and relative distribution index coordinates in four areas in the refined single-pixel binary distortion grid template graph by referring to a relative position relation index coordinate system. The search area is seen at blocks A, B, C and D in FIG. 4.
The grid point locations can be located by refining the 8 neighborhood characteristics of the grid intersection points. The distortion can cause the bending deformation of lines, grid lines outside a focusing region in the refined grid template picture are jagged to a large extent, some grid points can be split into two adjacent intersection points, and the middle points of the two grid points are taken as the coordinates of the refined grid points.
The reason why the directional search of the distorted grid points is performed separately in the regions is that the deformation directions of the grid lines in the regions A, B, C and D are different from each other, but the deformation tendencies of the grid lines in the same region (in both the horizontal and vertical directions) are the same. All mutually communicated grid intersections can be effectively searched along the grid lines by utilizing the line deformation direction, and the corresponding distribution index coordinates can be conveniently acquired. Ordered set SsearchThe searched grid points are recorded one by one.
The following describes in detail the directional search step of the control point pair, taking the upper right area B in fig. 4 as an example:
(1) initializing a search set Ssearch＝{P(0,-1)}；
(2) If S issearchIf the result is null, the search is terminated;
(3) for SsearchIf the coordinate is the first element of the set, searching the next connected neighbor grid point upwards and rightwards along the grid line by taking the coordinate as a starting point, and otherwise, only searching rightwards. The moving step size can be increased by sliding the window, so that the searching process is accelerated;
(4) reference SsearchThe relative distribution index coordinates of the grid points detected in the step (3) are easily determined by combining the detected characteristics that the neighboring grid points are only separated from the search starting point by one grid distance;
(5) updating SsearchAll the neighbor grid points searched in the step (3) are searched, and then the step (2) is carried out.
The entire visualization of the search is illustrated in fig. 4, with the arrow direction indicating the search direction and the coordinate point subscripts indicating the corresponding distribution index coordinates. The neighboring grid points obtained each time step (2) is performed are marked with different colors. In addition, for isolated grid points (e.g., P) of image corner regions(7,-6)) The distribution index coordinates thereof can be acquired with reference to the detected grid point that is closest thereto.
3. For distorted grid point PgCor_rAnd the ideal coordinate corresponding thereto is PgCor_iThen, it can be calculated by the following formula:
PgCor_i＝Pb+(DICgCor_r-DICb)·Dgrid_i (1)
there is an important assumption here: two grid points P with the largest distanceaAnd PbIs distortion free. This maximum distance is taken as the ideal grid distance Dgrid_i。
4. Obtaining all control point pairs < PgCor_i,PgCor_r>. generating set S(gCor_i,gCor_r)＝{＜PgCor_i,PgCor_r＞}。
Step 103: determining the position of the optimal distortion center by utilizing the nonlinear increasing relation between the distortion distance and the ideal distance
Some existing correction methods roughly consider the image center to be the location of the center of distortion (CoD). However, in practice there is some offset of the distortion center from the geometric center of the image. Further, only by accurately locating the position of the CoD, the distortion distance and the ideal distance from each distortion grid point to the CoD can be acquired more accurately. Aiming at the characteristics of a template method, the invention adopts a point distance search-based CoD search algorithm to locate the position of the optimal CoD. The method searches in the maximum grid by utilizing the nonlinear incremental relation between the distortion distance and the ideal distance, and judges whether the candidate point is the optimal distortion center. And the threshold requirement of the root mean square error of the four times of curve fitting can be further increased so as to improve the accuracy and the execution efficiency of the search. The method comprises the following concrete steps:
(1) traversing coordinate points P within a search areakCalculating ideal distances and distortion distances of all distortion grid points by the formula (2) to generate a distance pair set
(2) For is to
Performing curve fitting for four times, and calculating mean square error epsilon of fittingkIf epsilonkIf the value is smaller than the threshold value sigma (generally, sigma is 2.5), continuing to execute the step (3), otherwise, selecting the next candidate point, and then executing the step (1);
(3) set in ascending order by the ideal distance r
Carrying out sequencing operation; calculating a distortion distance
The distance number m in accordance with the ascending sort rulekAnd update the maximum distance number m0。
m0Corresponding search point P0Is the optimal CoD. At this time, the relationship between the corresponding ideal distance and distortion distance is shown in fig. 5, where the abscissa is the ideal distance and the ordinate is the distortion distance, and the unit is pixels. The CoD searched in the refined binary template map is shown as the point pointed by the arrow in fig. 6.
Step 104: establishing a minimum piecewise optimization model of radial distortion based on control point pairs in the distorted mesh template graph
And establishing a radial distortion model by utilizing a polynomial method of a concentric circle template based on the relation between the distortion distance and the ideal distance. If a single polynomial curve is used for fitting, it is only suitable for low resolution or small distortion wide-angle lenses. For high resolution wide-angle images with large distortions, this method not only requires higher polynomial orders, but also increases the computational load of the correction process. Therefore, the invention provides a self-adaptive low-order piecewise polynomial method, which specifically comprises the following steps:
firstly, sorting a set M of distance point pairs according to the ascending rule of an ideal distance r, and then dividing the set into K subsets
Wherein the content of the first and second substances,
and the number of pairs of distance points is N. Respectively adopting a least square algorithm shown in the formula (3) to perform curve fitting:
wherein, the first and the second end of the pipe are connected with each other,
is a curved line segment fi(r) and fi+1A segmentation point of (r). Suppose that
Since the CoD is not distorted, the first curve f1(r) past the origin, i.e.
Goodness of fit available for fitting accuracy of curve
The measurements were made as follows:
wherein the content of the first and second substances,
the more toward 1 the goodness of fit indicates the higher the quality of fit of the corresponding curve segment to the data point pair.
The end curve is used to estimate the distortion of the image edge region, and the corresponding maximum point (r) is corrected to ensure that the corner point farthest from the CoD can also be correctedsymAxis,max(fK) ) need to satisfy:
wherein, the first and the second end of the pipe are connected with each other,
rMaxthe distortion distance and the ideal distance corresponding to the corner point farthest from the CoD are shown. Four corners ptl、ptr、pblAnd pbrThe distance point pair information of (2) is shown in fig. 7.
Is easy to know，rMax＝max{rtl,rtr,rbl,rbr}、
In other words, there are:
further, the width W 'and height H' of the calibration chart are obtained by the similar trigonometric transformation in fig. 7, and the size region thereof is indicated by the matrix frame on the outer side.
Combining the above analysis, a minimum piecewise optimization model of radial distortion can be established by an inequality constrained optimization problem with two cost functions, as shown in equations (7) and (8):
through repeated experiments, a group of preferred parameter range selections of the embodiment of the invention are obtained: delta epsilon (0,1) can eliminate distortion center shrinkage
And distorted center expansion
Xi is set to be 0.9, so that good fitting quality can be ensured, and overfitting can be prevented; theta is less than or equal to 5 pi/180 (rad) and lambda is less than or equal to 2, so that a satisfactory smooth transition effect of the curve section can be obtained, and the optimal number of sections of the sectional curve is 4.
Step 105: establishing a reverse correction mapping table to obtain the coordinate corresponding relation between the correction result image and the original distorted image
If the forward mapping is adopted, namely the coordinates of distortion points are used for directly obtaining the corresponding ideal point coordinates, the size of the image after distortion correction is larger than that of the original image, so that distortion points corresponding to certain pixel points in the corrected recovery image cannot be found, and a 'void' phenomenon is formed. In order to eliminate the phenomenon of 'holes' caused by the non-full shot relation between the distortion image and the ideal recovery image, the invention adopts a reverse mapping method to obtain the coordinate corresponding relation between the correction result image and the original distortion image.
In order to obtain the inverse coordinate mapping from the Calibration map to the distortion map and accelerate the distortion correction process, a Calibration Mapping Table (CMT) with the same size as the Calibration map is established. Each element T (x, y) in the CMT holds that the ideal point p (x, y) corresponds to a distorted coordinate position p' (x, y) in the distorted image. The corresponding relationship is as follows:
wherein, CoDidealCorresponds to the centre of distortion in the correction map. r (p (x, y)) is p (x, y) to CoDidealThe distance of (a) to (b),
is the distance from p' (x, y) to CoD, and can be calculated by equation (3).
Step 106: based on the correction mapping table, coordinate reverse mapping processing is carried out to obtain a first correction result image
In the distortion correction process, aiming at the input distorted wide-angle image, the distortion coordinate position of each correction pixel point in the distorted wide-angle image is obtained through CMT table look-up.
Step 107: and performing pixel interpolation on the first correction result image by using a bilinear interpolation algorithm, and recovering the gray value of the first correction result image to obtain a complete second correction result image.
The exemplary implementation of the present invention is divided into two stages, a distortion calibration generation inverse correction mapping table and a distortion correction process. The distortion calibration is to perform a series of operations such as control point pair extraction, distortion center estimation, piecewise optimization distortion model solution and the like according to the shot single distortion grid template graph, and further generate a reverse coordinate mapping relation table from the ideal correction graph to the distortion graph. The distortion correction is to perform correction processing (coordinate mapping transformation and interpolation processing) on an image captured by the wide-angle lens by using a reverse correction mapping table to obtain a corrected recovery result map. In the distorted grid template image, a high-quality grid template image can be captured by using a professional auxiliary shooting device, such as an AEE SD-21 wide-angle lens with a large field angle (170 degrees), in an environment with sufficient illumination, as shown in fig. 8, the number of control points (grid points) is sufficient, and the entire image is fully covered and uniformly distributed in all areas of the image.
Fig. 9 illustrates an adaptive wide-angle image correction apparatus based on single-grid-map segmentation mapping according to an exemplary embodiment of the present invention, namely, an electronic device 310 (e.g., a computer server with program execution functions) including at least one processor 311, a power supply 314, and a memory 312 and an input-output interface 313 communicatively connected to the at least one processor 311; the memory 312 stores instructions executable by the at least one processor 311, the instructions being executable by the at least one processor 311 to enable the at least one processor 311 to perform a method disclosed in any one of the embodiments; the input/output interface 313 may include a display, a keyboard, a mouse, and a USB interface for inputting/outputting data; the power supply 314 is used to provide power to the electronic device 310.
Those skilled in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as a removable Memory device, a Read Only Memory (ROM), a magnetic disk, or an optical disk.
When the integrated unit of the present invention is implemented in the form of a software functional unit and sold or used as a separate product, it may also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present invention or portions thereof contributing to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a removable storage device, a ROM, a magnetic or optical disk, or other various media that can store program code.
The foregoing is merely a detailed description of specific embodiments of the invention and is not intended to limit the invention. Various alterations, modifications and improvements will occur to those skilled in the relevant art without departing from the spirit and scope of the invention.
Claims (9)
1. An adaptive wide-angle image correction method based on single-grid-map segmentation mapping, the method comprising:
preprocessing the distorted grid template picture to obtain a single-pixel binary distorted grid template picture; extracting control point pairs in the distorted grid template graph by using neighborhood characteristics of grid intersections based on the single-pixel binary distorted grid template graph; establishing a minimum piecewise optimization model of radial distortion based on control point pairs in the distorted grid template graph;
establishing a reverse correction mapping table to obtain a coordinate corresponding relation between a correction result image and an original distorted image; based on the correction mapping table, carrying out coordinate reverse mapping processing to obtain a first correction result image; performing pixel interpolation on the first correction result image by using a bilinear interpolation algorithm, and recovering the gray value of the first correction result image to obtain a complete second correction result image;
the extracting the control point pairs in the distorted grid template graph by using the neighborhood characteristics of the grid intersections comprises the following steps:
the method comprises the steps of accurately positioning control points by using neighborhood characteristics of grid intersections, further combining a method of directionally searching by regions along grid lines, obtaining relative position distribution among distorted grid points, and solving ideal coordinates corresponding to the distorted points.
2. The method of claim 1, wherein pre-processing the distorted mesh template map comprises:
firstly graying a distorted grid template picture, dividing the distorted grid template picture into a plurality of blocks with different sizes, respectively carrying out self-adaptive threshold segmentation, and then synthesizing a complete binary image of the distorted grid template picture; and further smoothing by using median filtering, and finally obtaining a refined single-pixel binary distortion grid template picture through color inversion processing.
3. The method of claim 1, further comprising:
establishing a relative position relation index coordinate system DRCS for describing the relative position relation of grid points in an ideal normalized template corresponding to the distorted grid template; wherein, the origin P (0,0) of the coordinate system is the nearest grid intersection point from the central point of the distorted template picture; the horizontal rightward direction is the x-axis indexing direction, and the vertical downward direction is the y-axis indexing direction; p (0, -1), P (0,1), P (-1,0) and P (1,0) are the nearest neighbor index positions of P (0,0) in the four directions of up, down, left and right, respectively;
extracting distortion grid intersection points and relative distribution index coordinates in four areas in the refined single-pixel binary distortion grid template graph by referring to a relative position relation index coordinate system;
for distorted grid point PgCor_rCorresponding ideal coordinate PgCor_iCalculated by the following formula:
PgCor_i＝Pb+(DICgCor_r-DICb)·Dgrid_i (1)
obtaining all control point pairs < PgCor_i,PgCor_r> (S) generating a set SgCor_i,gCor_r)＝{＜PgCor_i,PgCor_r> -, wherein Dgrid_iIs the maximum pitch.
4. The method of claim 3, wherein determining the location of the optimal distortion center comprises the steps of:
traversing coordinate points P within a search areakCalculating ideal distances and distortion distances of all the distortion grid points by equation (2),
generating a set of distance pairs
To pair
Performing curve fitting for four times, and calculating mean square error epsilon of fittingkIf epsilonkIf the value is less than the threshold value sigma, continuing to execute the next step, otherwise, selecting the next candidate point and executing the previous step;
set in ascending order by the ideal distance r
Carrying out sequencing operation; calculating a distortion distance
The distance number m in accordance with the ascending sort rulekAnd update the maximum distance number m0；
m0Corresponding search point P0The optimum distortion center CoD.
5. The method of claim 4, further comprising: grouping pairs of distance points
Sorting according to the rule of ascending the ideal distance r, and then dividing the set into K subsets
Wherein the content of the first and second substances,
and the number of the distance point pairs is N; respectively adopting a least square algorithm shown in the formula (3) to perform curve fitting:
wherein the content of the first and second substances,
is a curved line segment fi(r) and fi+1(r) a segmentation point, and
6. the method of claim 5, wherein the minimum piecewise optimization model of the radial distortion is as shown in equations (7) and (8):
7. the method of claim 6, wherein δ e (0,1), ξ is set as 0.9, θ ≦ 5 π/180(rad), λ ≦ 2, and the optimal number of segments for the segmentation curve is 4.
8. The method according to claim 7, characterized in that each element T (x, y) in the correction mapping table CMT holds that the ideal point p (x, y) corresponds to a distorted coordinate position p' (x, y) in the distortion map; the corresponding relationship is as follows:
wherein CoDial corresponds to the center of distortion in the correction map; r (p (x, y)) is the distance of p (x, y) to the CoDideal,
is the distance of p' (x, y) to the CoD.
9. An adaptive wide-angle image correction apparatus based on single-grid-map segmentation mapping, comprising at least one processor, and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810834335.1A CN109035170B (en) | 2018-07-26 | 2018-07-26 | Self-adaptive wide-angle image correction method and device based on single grid image segmentation mapping |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810834335.1A CN109035170B (en) | 2018-07-26 | 2018-07-26 | Self-adaptive wide-angle image correction method and device based on single grid image segmentation mapping |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109035170A CN109035170A (en) | 2018-12-18 |
| CN109035170B true CN109035170B (en) | 2022-07-01 |
Family
ID=64646543
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810834335.1A Active CN109035170B (en) | 2018-07-26 | 2018-07-26 | Self-adaptive wide-angle image correction method and device based on single grid image segmentation mapping |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109035170B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110084763B (en) * | 2019-04-29 | 2020-12-25 | 北京达佳互联信息技术有限公司 | Image restoration method and device, computer equipment and storage medium |
| CN110276734B (en) | 2019-06-24 | 2021-03-23 | Oppo广东移动通信有限公司 | Image distortion correction method and device |
| CN111612720B (en) * | 2020-05-21 | 2023-11-07 | 烟台艾睿光电科技有限公司 | Wide-angle infrared image optimization method, system and related components |
| CN112053369A (en) * | 2020-08-27 | 2020-12-08 | 西安迪威码半导体有限公司 | Low-delay image distortion removal and barrel mapping fusion algorithm |
| US20230292020A1 (en) * | 2020-10-27 | 2023-09-14 | Qualcomm Incorporated | Lens distortion correction for image processing |
| CN117455818A (en) * | 2023-12-11 | 2024-01-26 | 深圳精智达技术股份有限公司 | Correction method and device for intelligent glasses screen, electronic equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020944A (en) * | 2013-01-08 | 2013-04-03 | 华东师范大学 | Method for two-dimensional geometric correction of magnetic resonance image |
| CN103426149A (en) * | 2013-07-24 | 2013-12-04 | 玉振明 | Large-viewing-angle image distortion correction and processing method |
| CN103634527A (en) * | 2013-12-12 | 2014-03-12 | 南京华图信息技术有限公司 | Multi-camera real-time scene splicing system capable of resisting camera disturbance |
| CN103729841A (en) * | 2013-12-18 | 2014-04-16 | 同济大学 | Camera distortion correcting method based on square target model and perspective projection |
| CN104537616A (en) * | 2014-12-20 | 2015-04-22 | 中国科学院西安光学精密机械研究所 | Correction method for fisheye image distortion |
| CN105872413A (en) * | 2016-05-31 | 2016-08-17 | 成都德善能科技有限公司 | Intelligent aerial photographing system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102375978A (en) * | 2010-08-17 | 2012-03-14 | 富士通株式会社 | Method and equipment for processing images |
-
2018
- 2018-07-26 CN CN201810834335.1A patent/CN109035170B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103020944A (en) * | 2013-01-08 | 2013-04-03 | 华东师范大学 | Method for two-dimensional geometric correction of magnetic resonance image |
| CN103426149A (en) * | 2013-07-24 | 2013-12-04 | 玉振明 | Large-viewing-angle image distortion correction and processing method |
| CN103634527A (en) * | 2013-12-12 | 2014-03-12 | 南京华图信息技术有限公司 | Multi-camera real-time scene splicing system capable of resisting camera disturbance |
| CN103729841A (en) * | 2013-12-18 | 2014-04-16 | 同济大学 | Camera distortion correcting method based on square target model and perspective projection |
| CN104537616A (en) * | 2014-12-20 | 2015-04-22 | 中国科学院西安光学精密机械研究所 | Correction method for fisheye image distortion |
| CN105872413A (en) * | 2016-05-31 | 2016-08-17 | 成都德善能科技有限公司 | Intelligent aerial photographing system |
Non-Patent Citations (6)
| Title |
|---|
| A Piecewise Correction Method for High-resolution Image of Wide-angle Lens;Ruiqing Wu等;《2013 6th International Congress on Image and Signal Processing (CISP)》;20131231;331-335 * |
| 一种基于控制点自动提取的图像畸变校正算法;郭永刚等;《计算工程与应用》;20070111;第43卷(第2期);第214-216页 * |
| 一种广角镜头光学畸变的数字校正算法;丁莹等;《长春理工大学学报（自然科学版）》;20090615;第32卷(第2期);正文第1-3节、图1、图3 * |
| 低空航拍图像校正与增强方法的研究;王莉莉;《中国优秀硕士学位论文全文数据库_信息科技辑》;20160315(第03期);正文第2章 * |
| 基于网格图像的双线性插值畸变校正的方法研究;王鹏等;《科技创新与节能减排——吉林省第五届科学技术学术年会论文集（上册）》;20081001;第389-391页 * |
| 航天发射安全控制决策的空间信息分析与处理研;郭茂耘;《中国博士学位论文全文数据库_工程科技Ⅱ辑》;20111215;C031-13 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109035170A (en) | 2018-12-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109035170B (en) | Self-adaptive wide-angle image correction method and device based on single grid image segmentation mapping | |
| CN109544456B (en) | Panoramic environment sensing method based on two-dimensional image and three-dimensional point cloud data fusion | |
| CN107665483B (en) | Calibration-free convenient monocular head fisheye image distortion correction method | |
| CN111091567B (en) | Medical image registration method, medical device and storage medium | |
| CN110738273B (en) | Image feature point matching method, device, equipment and storage medium | |
| CN113920205B (en) | Calibration method of non-coaxial camera | |
| CN109816706B (en) | Smooth constraint and triangulation equal-proportion image pair dense matching method | |
| CN110415304B (en) | Vision calibration method and system | |
| CN111105452A (en) | High-low resolution fusion stereo matching method based on binocular vision | |
| CN103700082B (en) | Image split-joint method based on dual quaterion relative orientation | |
| CN114693760A (en) | Image correction method, device and system and electronic equipment | |
| CN110223356A (en) | A kind of monocular camera full automatic calibration method based on energy growth | |
| CN111126418A (en) | Oblique image matching method based on planar perspective projection | |
| CN107492080B (en) | Calibration-free convenient monocular head image radial distortion correction method | |
| US20120038785A1 (en) | Method for producing high resolution image | |
| CN110969650B (en) | Intensity image and texture sequence registration method based on central projection | |
| CN116993612A (en) | Nonlinear distortion correction method for fisheye lens | |
| CN114943823B (en) | Unmanned aerial vehicle image splicing method and system based on deep learning semantic perception | |
| CN100371944C (en) | Greyscale image partition method based on light distribution character of reflection or penetrance | |
| CN112669360B (en) | Multi-source image registration method based on non-closed multi-dimensional contour feature sequence | |
| CN115239559A (en) | Depth map super-resolution method and system for fusion view synthesis | |
| CN114964206A (en) | Monocular vision odometer target pose detection method | |
| JP7003291B2 (en) | Correction method and device for correcting image data | |
| CN114399540A (en) | Heterogeneous image registration method and system based on two-dimensional iteration closest point | |
| JP2006145419A (en) | Image processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |