CN1937698A

CN1937698A - Image processing method for image distortion automatic correction

Info

Publication number: CN1937698A
Application number: CN 200610117277
Authority: CN
Inventors: 赵群飞; 张森; 孙明; 森泽太平; 赖尾修三
Original assignee: Shanghai Jiao Tong University
Current assignee: Shanghai Jiao Tong University
Priority date: 2006-10-19
Filing date: 2006-10-19
Publication date: 2007-03-28

Abstract

An image processing method for automatic correction of image distortion. The invention uses radon transformation to quickly extract the four-sided contours of business cards and texts, uses the contour vertex information to correct the distorted image, and establishes a mathematical model of the internal imaging of the shooting device, including For the equations of business card and text aspect ratio parameters, the aspect ratio of the rectangular object is obtained by solving the equations and applied to the correction process, so that the final correction result will not be deformed. Finally, through character recognition, it is recognized whether the corrected image is inverted, and it is judged whether the corrected image needs to be rotated, so as to obtain a corrected result that meets the requirements. The invention can quickly realize distortion correction, does not need manual participation in the whole process, and truly realizes automatic correction. The invention can be used in most imaging devices such as medical imaging, monitoring equipment, mobile phones, digital cameras, digital video cameras, etc., and is easy to realize and easy for users to operate.

Description

Picture distortion is from the image processing method of dynamic(al) correction

Technical field

What the present invention relates to is a kind of method of technical field of image processing, specifically is the image processing method of a kind of picture distortion from dynamic(al) correction.

Background technology

When taking data such as business card, text with capture apparatus such as digital camera or mobile phones, the image that was photographed tends to tilt, and the object of original rectangles such as business card, text can take place to distort and become arbitrary quadrilateral.Its reason is when taking business card or text, photographer not over against and perpendicular to subject, but certain horizontal range and deviation angle are arranged apart from subject.And photographer is when taking these data, seldom can go over against and vertical, therefore, this distortion phenomenon is very common, to people read these above data information or cause some difficulties when carrying out word processing.

Find by prior art documents, Chinese patent publication number: CN1607824A, open day is on April 20th, 2005, denomination of invention: image processing system and image processing method and electronic camera and image processing apparatus.A cover image processing system and a processing method thereof described in this invention, and its image processing system is made up of Electrofax and image processing apparatus.Wherein, Electrofax comprises image pickup part, the image recording portion, machining information obtaining section and processed record information portion, image pickup part is taken the body that is taken, the image that image recording portion record is photographed by this image pickup part, and the machining information obtaining section is according to the image that is recorded in this image recording portion, obtain the machining information that uses in the regulation processing to this image, processed record information portion will be by this obtaining section machining information of obtaining and the image corresponding record that obtains the source.Image processing apparatus, it possesses taken the image of record by Electrofax, by the machining information of described Electrofax record, carries out the image processing part that regulation is processed according to answering with this image degree.In this invention, the image that Electrofax photographs is presented on the monitor with regeneration mode, if the user needs the correction image distortion, the user need do it yourself to operate the tetragonal outline line that becomes benchmark when identification is revised so, after finding the outline line of wanting, determine operation, the coordinate information on 4 summits of the outline line selected is write the title portion of the image file of display image.Afterwards, when revising, image processing apparatus therefrom reads coordinate information and carries out the correction operation.This technological invention needs the user to do it yourself to operate, and can not realize that from dynamic(al) correction, process complexity, accuracy are not high, except digital camera, also needs personal computer (PC) and projecting apparatus, the equipment complexity, and processing procedure is slower.

In addition, Photoshop software (software towards Digital Image Processing of U.S. Adobe company exploitation) also can be realized the function of image distortion correction, but must on personal computer, implement, need the user to start to regulate and proofread and correct angle, therefore in the operating process error can appear, accuracy is not high, can not realize zero offset capability.And this software can only proofread and correct the trapezoidal distortion of buildings such as building, and the shape that can not proofread and correct business card, text tilts and the distortion of platform shape, and range of application is little.

Summary of the invention

The present invention is directed to the deficiencies in the prior art, the image processing method of a kind of picture distortion from dynamic(al) correction is provided.The shape that the present invention can proofread and correct business card, text tilts and the distortion of platform shape, can realize distortion correction fast, and whole process does not need artificial participation, really realizes from dynamic(al) correction, is easy to realize, and is simple to operate.

The present invention is achieved by the following technical solutions, comprises five steps:

(1) extracts profile: utilize otsu algorithm (big Tianjin algorithm) to carry out automatic threshold and cut apart, adopt morphological method to carry out edge contour and extract;

(2) resolve profile: utilize radon conversion (a kind of projective transformation that is used for straight-line detection that Austrian mathematician Radon proposes) will go up edge contour that the step extracts and separate out, and try to achieve the coordinate of four end points with the formal solution of linear equation;

(3) calculate aspect ratio: be created as the Mathematical Modeling of picture device interior optical imagery, utilize optical imagery knowledge, set up equation group, try to achieve aspect ratio;

(4) distortion correction: utilize two groups of coordinates to try to achieve the distortion correction matrix image is proofreaied and correct, the aspect ratio of image remains unchanged in the trimming process;

(5) rectification building-out: in the time of digital above the method for utilizing digit recognition is judged the image after proofreading and correct inverted phenomenon takes place, select whether need image rotating, the image of finishing after the correction can not proofreaied and correct.

Among the present invention, at first extract tetragonal edge contours such as business card, text, four edges is separated.Wherein, profile extracts used big Tianjin algorithm, is the fast method that a kind of automatic threshold is cut apart.This method can calculate the segmentation threshold of cromogram or gray-scale map very soon, and converts former figure to binary map.Among the present invention, utilize business card, text etc. and the difference of background on brightness and colorfulness, big Tianjin algorithm is cut apart when converting binary map in threshold value background is made as black automatically, and business card etc. are made as white, therefore, in the binary map that obtains, comprise two parts: the business card or the text of the background of black and white.When morphology extracts the border, utilize the Boundary Extraction operator that binary map is carried out computing, edge contour can be extracted, like this, obtain a tetragonal binary map of white.

When resolving profile, the radon conversion of using is a kind of algorithm of straight-line detection.The white tetragonal binary map that step (1) obtains is used the radon conversion, obtain the radon conversion figure of four corresponding bright spots.The corresponding tetragonal limit of each bright spot by reading the XY coordinate figure of the correspondence of bright spot on radon conversion figure, can be obtained the straight line analytic expression on every limit come.Tetragonal four end points coordinates can be obtained in per two limits of simultaneous.

Imaging device internal imaging process is followed lens image formation rule.Suppose that rectangle thing length and width such as business card or text are respectively l, m, with respect to photographer's rectangle anglec of rotation is α, take the lens primary optical axis and the camera plane angle theta of equipment, the coordinate of rectangular centre in the rectangular coordinate system at shooting area place is (x0, y0), the shooting area center is D to the optical center of lens distance, optical coefficient is k, one has 8 unknown parameters, can four end points coordinate representations of rectangle be come out with these 8 parameters, utilize two special light then: the light of crossing the light of primary optical axis and being parallel to primary optical axis is obtained the coordinate of four end points on imaging screen come.Like this, utilize tetragonal four end points coordinates of trying to achieve in the step (2) again, can form an equation group by simultaneous, comprise 8 equations and 8 unknown numbers,, can solve the length-width ratio of rectangle, just aspect ratio by the group of solving an equation.

Carry out timing, need to proofread and correct the coordinate of preceding four end points, i.e. the coordinate of being tried to achieve in the step (2), and the coordinate of proofreading and correct back four end points.Can be with the coordinate before proofreading and correct, and after proofreading and correct quadrangle become the condition of rectangle and proofread and correct after keep the condition of the aspect ratio of trying to achieve, obtain the coordinate of proofreading and correct back four end points.Utilizing these two groups of coordinates to try to achieve the distortion correction matrix proofreaies and correct.

Trimming process becomes rectangle with the quadrangle of distortion on the one hand, also will keep aspect ratio constant on the other hand.But after the correction, business card or the Word message above the text be it seems for the reader, may be inverted.Therefore, need utilize the technology of literal identification, whether numeral or the English alphabet differentiated above the business card are inverted.If inverted, image rotating is judged again so, to guarantee that literal is not put upside down among the last result.

The present invention can be used as the central processing module that the distortion correction submodule embeds imaging device inside, makes imaging device have the automatic distortion correction function.The distortion correction submodule is shared identical digital signal processing chip with central processing module on hardware, use digital image processing method of the present invention on the software, realizes automatic distortion correction on function.

Compared with prior art, the present invention can realize distortion correction fast, and whole process does not need artificial participation, really realizes from dynamic(al) correction.And, introduce the optical imagery Mathematical Modeling, try to achieve the former aspect ratio of business card, text, make and proofread and correct the result and keep original aspect ratio constant, guarantee that last correction result is undistorted in shape.At last, use literal identification, determine the not inversion of business card, the Word message above the text, use simple and convenient.The present invention can be used in most of imaging devices such as medical imaging, watch-dog, mobile phone, digital camera, Digital Video, and need not do big change on hardware, realizes simple.

Description of drawings

Fig. 1 is the inventive method flow chart

Fig. 2 is an imaging device internal optics imaging Mathematical Modeling.

Fig. 3 is the plane graph of shooting area.

Embodiment

The present invention utilizes the radon conversion to extract four cincture exterior features of business card, text fast, utilize the profile vertex information, fault image is proofreaied and correct, and, set up capture apparatus internal imaging Mathematical Modeling, foundation comprises the equation group of aspect ratio parameters such as business card, text, and the method for organizing by solving an equation is obtained the aspect ratio of rectangle object and is applied in the trimming process, makes last correction result can not produce distortion.At last, by literal identification, whether the image after identification is proofreaied and correct occurs being inverted, and judges whether and need the image after proofreading and correct be rotated, thereby obtain satisfactory correction result.

As shown in Figure 1, image processing method of the present invention comprises five steps, is respectively and extracts profile, resolves profile, calculates aspect ratio, distortion correction, rectification building-out.

Extract the profile step and comprise two parts, automatic threshold is cut apart with the morphology profile and is extracted.What automatic threshold was cut apart employing is big Tianjin algorithm.Consider the difference of background in colorfulness and brightness on business card or text and next door, adopt big Tianjin algorithm to separate business card or text with background simply fast.Use big Tianjin algorithm, cromogram or gray-scale map are become binary map BW, background parts is a black on binary map BW, and business card etc. become the quadrangle of a white.Then, to binary map BW applied morphology boundary extraction method, its processing procedure is: with one 3 * 3 value is the binary map BW that the corrosion of 1 matrix obtains above entirely, obtain a new binary map BW1, deduct among the binary map BW2 that boundary profile binary map BW2. that BW1 just obtains needing in the end obtains with BW then, background and tetragonal inside become black entirely, and have only tetragonal four edges to keep white.Like this, white tetragonal boundary profile just is extracted out.

What resolve the use of profile step is the algorithm of radon change detection straight line.The radon conversion can be understood as the projection of image in ρ-θ space, the every bit correspondence image space straight line in ρ-θ space, and the radon conversion is the integration of image pixel point on every straight line, also can be regarded as the image projection on trunnion axis after the θ angle that turns clockwise.Therefore every straight line can form a bright spot in ρ-θ space in the image, and the detection of straight line is converted in the detection of ρ-θ transform domain to bright spot.Therefore, the boundary profile binary map BW2 that the profile extraction step is obtained uses the radon conversion, can find has four tangible bright spots on the radon conversion figure that obtains, they are corresponding tetragonal four edges respectively, utilize the ρ of four bright spots, the θ value, the linear equation analytic expression of four edges writes out accurately.Therefore, boundary profile is resolved successfully.The equation on two limits of simultaneous can find the solution out with the coordinate of four end points respectively.These four end points are called as input point in distortion correction.

Calculate in the aspect ratio step, at first will set up the Mathematical Modeling of imaging device internal optics imaging.As shown in Figure 2, the lens centre O of imaging device and imaging screen and irradiated area (being the imaging zone) center O ' three point on a straight line, set up coordinate system as shown in Figure 2.Wherein, lens centre and imaging region centre distance are D, and the angle on the plane at lens primary optical axis and imaging region place is θ.During shooting, the set of lenses focal length of imaging device is f, and device interior image optics coefficient is k.Fig. 3 is the plane graph of imaging region.The length of supposing rectangular article such as business card, text is l, and wide is m, and the coordinate of the center of rectangle in the imaging region coordinate system is that (x0, y0), with respect to reference axis, rectangle has rotated the α angle.For convenience of calculation, order

R = \sqrt{{(\frac{l}{2})}^{2} + {(\frac{m}{2})}^{2}}

{&PartialD;}_{1} = \arctan (\frac{m}{l}) - &PartialD;

{&PartialD;}_{2} = \arctan (\frac{l}{m}) - &PartialD;

So, in coordinate system shown in Figure 3, the coordinate of four end points of rectangle is respectively

(R×cos( ₁)+x ₀，R×sin( ₁)×sin(θ)+y ₀，-D+R×sin( ₁)×cos(θ)+y ₀×sin(θ))

(-R×sin( ₂)+x ₀，R×cos( ₂)×sin(θ)+y ₀，-D+R×cos( ₂)×cos(θ)+y ₀×sin(θ))

(R×sin( ₂)+x ₀，-R×cos( ₂)×sin(θ)+y ₀，-D-R×cos( ₂)×cos(θ)+y ₀×sin(θ))

(-R×cos( ₁)+x ₀，-R×sin( ₁)×sin(θ)+y ₀，-D-R×sin( ₁)×cos(θ)+y ₀×sin(θ))

Utilize these four end points to cross two special light that are refracted to after the lens on the imaging screen, can obtain the coordinate of the picture of these four end points on imaging screen.These two straight lines, one was the straight line of optical center of lens, and direction is constant, and other one is the straight line that is parallel to primary optical axis, crosses lens refraction afterwards through falling on the imaging screen after the overfocus.Just can obtain the coordinate of picture point according to the equation of these two straight lines.The XY coordinate of the picture point of four end points correspondences of trying to achieve behind the work in coordinate system shown in Figure 2 is respectively:

(\frac{- f \times [R \times COS ({&PartialD;}_{1}) + x_{0}]}{D - f - R \times \sin ({&PartialD;}_{1}) \times \cos (θ) - y_{0} \times \sin (θ)},

\frac{- f \times [R \times \sin ({&PartialD;}_{1}) \times \sin (θ) + y_{0}]}{D - f - R \times \sin ({&PartialD;}_{1}) \times \cos (θ) - y_{0} \times \sin (θ)})

(\frac{f \times [R \times \sin ({&PartialD;}_{2}) - x_{0}]}{D - f - R \times \cos ({&PartialD;}_{2}) \times \cos (θ) - y_{0} \times \sin (θ)},

\frac{- f \times [R \times \cos ({&PartialD;}_{2}) \times \sin (θ) + y_{0}]}{D - f - R \times \cos ({&PartialD;}_{2}) \times \cos (θ) - y_{0} \times \sin (θ)})

(\frac{- f \times [R \times \sin ({&PartialD;}_{2}) + x_{0}]}{D - f + R \times \cos ({&PartialD;}_{2}) \times \cos (θ) - y_{0} \times \sin (θ)},

\frac{f \times [R \times \cos ({&PartialD;}_{2}) \times \sin (θ) - y_{0}]}{D - f + R \times \cos ({&PartialD;}_{2}) \times \cos (θ) - y_{0} \times \sin (θ)})

(\frac{f \times [R \times \cos ({&PartialD;}_{1}) - x_{0}]}{D - f + R \times \sin ({&PartialD;}_{1}) \times \cos (θ) - y_{0} \times \sin (θ)},

\frac{f \times [R \times \sin ({&PartialD;}_{1}) \times \sin (θ) - y_{0}]}{D - f + R \times \sin ({&PartialD;}_{1}) \times \cos (θ) - y_{0} \times \sin (θ)})

Utilize again in the step (2), the coordinate of tetragonal four end points in the image of trying to achieve, simultaneous:

\frac{- f \times [R \times \cos ({&PartialD;}_{1}) + x_{0}]}{D - f - R \times \sin ({&PartialD;}_{1}) \times \cos (θ) - y_{0} \times \sin (θ)} \times k = x_{1} - \frac{s}{2}

\frac{- f \times [R \times \sin ({&PartialD;}_{1}) \times \sin (θ) + y_{0}]}{D - f - R \times \sin ({&PartialD;}_{1}) \times \cos (θ) - y_{0} \times \sin (θ)} \times k = y_{1} - \frac{t}{2}

\frac{f \times [R \times \sin ({&PartialD;}_{2}) - x_{0}]}{D - f - R \times \cos ({&PartialD;}_{2}) \times \cos (θ) - y_{0} \times \sin (θ)} \times k = x_{2} - \frac{s}{2}

\frac{- f \times [R \times \cos ({&PartialD;}_{2}) \times \sin (θ) + y_{0}]}{D - f - R \times \cos ({&PartialD;}_{2}) \times \cos (θ) - y_{0} \times \sin (θ)} \times k = y_{2} - \frac{t}{2}

\frac{- f \times [R \times \sin ({&PartialD;}_{2}) + x_{0}]}{D - f + R \times \cos ({&PartialD;}_{2}) \times \cos (θ) - y_{0} \times \sin (θ)} \times k = x_{3} - \frac{s}{2}

\frac{f \times [R \times \cos ({&PartialD;}_{2}) \times \sin (θ) - y_{0}]}{D - f + R \times \cos ({&PartialD;}_{2}) \times \cos (θ) - y_{0} \times \sin (θ)} \times k = y_{3} - \frac{t}{2}

\frac{f \times [R \times \cos ({&PartialD;}_{1}) - x_{0}]}{D - f + R \times \sin ({&PartialD;}_{1}) \times \cos (θ) - y_{0} \times \sin (θ)} \times k = x_{4} - \frac{s}{2}

\frac{f \times [R \times \sin ({&PartialD;}_{1}) \times \sin (θ) - y_{0}]}{D - f + R \times \sin ({&PartialD;}_{1}) \times \cos (θ) - y_{0} \times \sin (θ)} \times k = y_{4} - \frac{t}{2}

Wherein, k is the scaling coefficient between imaging screen and the image, and s and t are the dimensions of image array, x1, x2, x3, x4, y1, y2, y3, y4 be respectively clapping of calculating in the step (2) image in the coordinate of four end points of white quadrangle.Like this, above equation group totally 8 unknown numbers (focal distance f is known), 8 equations, unknown number can solve respectively, like this, the ratio of l and m is that aspect ratio can be readily solved.

During distortion correction, utilize two groups of coordinate figures.One group is the coordinate figure of white tetragonal four end points of trying to achieve in step (2), another group be proofread and correct after the coordinate figure of four end points of rectangle of these four end points correspondences, this group end points is become datum mark.The upper left end points of fixed white quadrangle at first, the upper left end points of rectangle also is this point after proofreading and correct, and is motionless on the position, coordinate figure is also constant.Selecting then with the fixed endpoint is a limit of end points, calculates its length of side, and with the length of side of a this edge length of side as the rectangle after proofreading and correct, the aspect ratio that utilization is calculated above can calculate the length of side on an other limit of the rectangle after proofreading and correct.After the length and width of rectangle are all determined, utilize the coordinate of fixing point again, can respectively the coordinate Calculation of its excess-three point be come out.Therefore, the coordinate of four end points of rectangle after proofreading and correct is decided, and promptly the coordinate of datum mark is decided.Utilize the coordinate of input point and datum mark correction matrix can be obtained.Solution procedure is as follows: before supposing to proofread and correct four point coordinates for (x1, y1), (x2, y2), (x3, y3), (x4, y4), four point coordinates are (x1 ', y1 ') after proofreading and correct, (x2 ', y2 '), (x3 ', y3 '), (x4 ', y4 '), the distortion correction matrix H,

H = \begin{matrix} h_{11} & h_{12} & h_{13} \\ h_{21} & h_{22} & h_{23} \\ h_{31} & h_{32} & h_{33} \end{matrix}

h＝(h ₁₁，h ₁₂，h ₁₃，h ₂₁，h ₂₂，h ₂₃，h ₃₁，h ₃₂，h ₃₃) ^T

There is relational expression between them:

[\begin{matrix} x_{1} & y_{1} & 1 & 0 & 0 & 0 & - {x_{1}}^{'} x_{1} & - {x_{1}}^{'} y_{1} & - x_{1} \\ 0 & 0 & 0 & x_{1} & y_{1} & 1 & - {y_{1}}^{'} x_{1} & - {y_{1}}^{'} y_{1} & {- y}_{1} \\ x_{2} & y_{2} & 1 & 0 & 0 & 0 & - {x_{2}}^{'} x_{2} & - {x_{2}}^{'} y_{2} & - x_{2} \\ 0 & 0 & 0 & x_{2} & y_{2} & 1 & - {y_{2}}^{'} x_{2} & - {y_{2}}^{'} y_{2} & - y_{2} \\ x_{3} & y_{3} & 1 & 0 & 0 & 0 & - {x_{3}}^{'} x_{3} & - {x_{3}}^{'} y_{3} & - x_{3} \\ 0 & 0 & 0 & x_{3} & y_{3} & 1 & - {y_{3}}^{'} x_{3} & - {y_{3}}^{'} y_{3} & - y_{3} \\ x_{4} & y_{4} & 1 & 0 & 0 & 0 & - {x_{4}}^{'} x_{4} & - {x_{4}}^{'} y_{4} & - x_{4} \\ 0 & 0 & 0 & x_{4} & y_{4} & 1 & - {y_{4}}^{'} x_{4} & - {y_{4}}^{'} y_{4} & - y_{4} \end{matrix}] \times h = 0

Utilize this relational expression can solve h, corresponding distortion correction matrix H can solve.Utilize the coordinate and the distortion correction matrix of the picture element of original image to proofread and correct original image, in the image that obtains after proofreading and correct, business card and text become rectangle, and have kept original aspect ratio.

Using literal identification compensates the image after proofreading and correct.The development of present character recognition technology rapidly, and because business card or the numeral above the text all are block letter, than handwritten text identification simple a lot.The rectification building-out step comprises four parts: image segmentation, feature extraction, literal identification, judgement compensation.After image after proofreading and correct carried out preliminary treatment, image binaryzation, make the image after the binaryzation, literal etc. are black, and other backgrounds become white.The word of asking that utilizes morphology connected region method will need to discern splits, specific practice is as follows: detect first black picture element point on binary map, suppose that it is a black picture element point on the connected domain, its beginning as recursive procedure, recurrence formula is: X _k=(X _K-1 B) ∩ A, X _kThe connected region that representative extracts, the binary map above the A representative, B is one 3 * 38 connected region templates.After satisfactory connected region is found out with all, carry out normalization again, all connected regions are all carried out interpolation become 20 * 10 matrix.Because the overwhelming majority comprises Arabic numerals in business card or the text, so the present invention discerns Arabic numerals emphatically.If do not have numeral in business card or the text, can discern English alphabet and Chinese character successively so.For example, when the present invention discerns numeral, by to 2,3,4,5,7 (because 0,1,6,8, even 9 be inverted and also have corresponding digital corresponding with it, for example, 0 and 0,6 and 9, or the like.) analysis of feature, adopt four kinds of numerical characteristics: the horizontal line feature, vertical line feature, horizontal direction are crossed the line number, and vertical direction is crossed the line number, and these digital numerical characteristics are as shown in table 1 below.Therefore, set up the respective classified device, as shown in the following Table 2, can be respectively with these digit recognition.For example, identify numeral 2 now, explanation so, business cards etc. are not inverted, if, utilize grader not identify numeral, explanation so, business cards etc. are inverted.Therefore, need the image after proofreading and correct be rotated again, the image that obtains then is only last needs.

Table 1 is the numerical characteristic form.

Numerical characteristic		2	3	4	5	7
Numerical characteristic		2	3	4	5	7	The horizontal line feature	Last horizontal line	0	0	0	1	1
Following horizontal line	1	0	0	0	0			Last horizontal line	0	0	0	1	1
Following horizontal line	1	0	0	0	0	The vertical line feature		Left side vertical line	0	0	0	0	0
Right vertical line	0	0	0	0	0			Left side vertical line	0	0	0	0	0
Right vertical line	0	0	0	0	0		Horizontal direction is crossed the line feature	Went up the line number	2	2	2	1	2
Played the line number	2	2	1，2	2	1			Went up the line number	2	2	2	1	2
Played the line number	2	2	1，2	2	1	Vertical direction is crossed the line feature		The line number is crossed on a left side	4，3	2，3，4	2，3	4，3	1，2
The right line number of crossing	3	4	2	3	2			The line number is crossed on a left side	4，3	2，3，4	2，3	4，3	1，2

Table 2 is the digital sort device.

Numerical characteristic	2	3	4	5	7
Numerical characteristic	2	3	4	5	7	Last horizontal line	0	0	0	1	1
Following horizontal line	1	0	0	0	0	Last horizontal line	0	0	0	1	1
Following horizontal line	1	0	0	0	0	Left side vertical line	0	0	0	0	0
Right vertical line	0	0	0	0	0	Left side vertical line	0	0	0	0	0
Right vertical line	0	0	0	0	0	Went up the line number	2	2	2	1	2
The right line number of crossing	3	4	2	3	2	Went up the line number	2	2	2	1	2
The right line number of crossing	3	4	2	3	2	Last 1/3 highly located the line number	1	1	2	1	1

The present invention can be used as the central processing module that the distortion correction submodule embeds imaging device inside, makes imaging device have the automatic distortion correction function.Its implementation is, the distortion correction submodule is embedded in the central processing module of imaging device as function sub-modules, on hardware, share identical digital signal processing chip with central processing module, use digital image processing method of the present invention on the software, on function, realize automatic distortion correction.

Claims

1. An image processing method for automatic correction of image distortion, comprising five steps:

(1) Contour extraction: use the Otsu algorithm for automatic threshold segmentation, and use morphological methods for edge contour extraction;

(2) Analyzing the contour: use the radon transformation to analyze the edge contour extracted in the previous step in the form of a straight line equation, and obtain the coordinates of the four endpoints;

(3) Calculate the aspect ratio: establish a mathematical model of the optical imaging inside the imaging device, use the knowledge of optical imaging to establish a system of equations, and obtain the aspect ratio;

(4) Distortion correction: use two sets of coordinates to obtain the distortion correction matrix to correct the image, and the aspect ratio of the image remains unchanged during the correction process;

(5) Correction and compensation: Use the method of digital recognition to judge the inversion of the above numbers on the corrected image, choose whether to rotate the image, and the corrected image cannot be corrected.

2. The image processing method for automatic correction of image distortion according to claim 1, characterized in that, the step of extracting contours includes two parts: automatic threshold segmentation and morphological contour extraction. Threshold segmentation to obtain a binary image. On the binary image, the business card and text become white, and the other background becomes black; use the morphological method to process the white area, leaving only the edges of the white quadrilateral Contour, in this way, the contour extraction is completed.

3. The image processing method for automatic correction of image distortion according to claim 1, characterized in that, in the step of analyzing the contour, the ρ-θ spatial information of the four white edge straight lines on the binary image is analyzed by using radon transformation, Find the straight line equations of the four white edges, and then solve the coordinates of the four endpoints by using the method of solving the equation system.

4. The image processing method for automatic correction of image distortion according to claim 1, characterized in that, in the step of calculating the aspect ratio, by establishing a mathematical model of optical imaging inside the imaging device, the following equations are established:

\frac{- - f f \times \times [[R R \times \times cos cos (({&PartialD; &PartialD;}_{11})) + + {x x}_{00}]]}{D D. - - f f - - R R \times \times sin sin (({&PartialD; &PartialD;}_{11})) \times \times cos cos ((θ θ)) - - {y the y}_{00} \times \times sin sin ((θ θ))} \times \times k k = = {x x}_{11} - - \frac{s the s}{22}

\frac{- - f f \times \times [[R R \times \times sin sin (({&PartialD; &PartialD;}_{11})) \times \times sin sin ((θ θ)) + + {y the y}_{00}]]}{D D. - - f f - - R R \times \times sin sin (({&PartialD; &PartialD;}_{11})) \times \times cos cos ((θ θ)) - - {y the y}_{00} \times \times sin sin ((θ θ))} \times \times k k = = {y the y}_{11} - - \frac{t t}{22}

\frac{f f \times \times [[R R \times \times sin sin (({&PartialD; &PartialD;}_{22})) - - {x x}_{00}]]}{D D. - - f f - - R R \times \times cos cos (({&PartialD; &PartialD;}_{22})) \times \times cos cos ((θ θ)) - - {y the y}_{00} \times \times sin sin ((θ θ))} \times \times k k = = {x x}_{22} - - \frac{s the s}{22}

\frac{- - f f \times \times [[R R \times \times cos cos (({&PartialD; &PartialD;}_{22})) \times \times sin sin ((θ θ)) + + {y the y}_{00}]]}{D D. - - f f - - R R \times \times cos cos (({&PartialD; &PartialD;}_{22})) \times \times cos cos ((θ θ)) - - {y the y}_{00} \times \times sin sin ((θ θ))} \times \times k k = = {y the y}_{22} - - \frac{t t}{22}

\frac{- - f f \times \times [[R R \times \times sin sin (({&PartialD; &PartialD;}_{22})) + + {x x}_{00}]]}{D D. - - f f + + R R \times \times cos cos (({&PartialD; &PartialD;}_{22})) \times \times cos cos ((θ θ)) - - {y the y}_{00} \times \times sin sin ((θ θ))} \times \times k k = = {x x}_{33} - - \frac{s the s}{22}

\frac{f f \times \times [[R R \times \times cos cos (({&PartialD; &PartialD;}_{22})) \times \times sin sin ((θ θ)) - - {y the y}_{00}]]}{D D. - - f f + + R R \times \times cos cos (({&PartialD; &PartialD;}_{22})) \times \times cos cos ((θ θ)) - - {y the y}_{00} \times \times sin sin ((θ θ))} \times \times k k = = {y the y}_{33} - - \frac{t t}{22}

\frac{f f \times \times [[R R \times \times cos cos (({&PartialD; &PartialD;}_{11})) - - {x x}_{00}]]}{D D. - - f f + + R R \times \times sin sin (({&PartialD; &PartialD;}_{11})) \times \times cos cos ((θ θ)) - - {y the y}_{00} \times \times sin sin ((θ θ))} \times \times k k = = {x x}_{44} - - \frac{s the s}{22}

\frac{f f \times \times [[R R \times \times sin sin (({&PartialD; &PartialD;}_{11})) \times \times sin sin ((θ θ)) - - {y the y}_{00}]]}{D D. - - f f + + R R \times \times sin sin (({&PartialD; &PartialD;}_{11})) \times \times cos cos ((θ θ)) - - {y the y}_{00} \times \times sin sin ((θ θ))} \times \times k k = = {y the y}_{44} - - \frac{t t}{22}

Among them, the length of the rectangle is l, the width is m, the distance between the center of the lens and the center of the imaging area is D, the angle between the main optical axis of the lens and the plane where the imaging area is located is θ, the focal length f is known, and the center of the rectangle is in the coordinate system of the imaging area The coordinates are (x0, y0), relative to the coordinate axis, the rectangle is rotated by α angle, k is the scaling factor between the imaging screen and the image, s and t are the dimensions of the image matrix, x1, x2, x3, x4, y1, y2, y3, y4 are the coordinates of the four endpoints of the white quadrilateral in the captured image calculated in step (2), assuming

R R = = \sqrt{{((\frac{l l}{22}))}^{22} + + {((\frac{m m}{22}))}^{22}}

{&PartialD; &PartialD;}_{11} = = arctan arctan ((\frac{m m}{l l})) - - &PartialD; &PartialD;

{&PartialD; &PartialD;}_{22} = = arctan arctan ((\frac{l l}{m m})) - - &PartialD; &PartialD;

In this way, the above equations can be solved, and these parameters can be solved to obtain the aspect ratio of the rectangle.

5. The image processing method for automatic correction of image distortion according to claim 1, characterized in that, in the distortion correction step, first, the coordinate values of the four endpoints calculated in step (2) are used as input points, and Input points and aspect ratio to calculate the reference point, then use two sets of coordinate values to obtain the distortion correction matrix H, and then use the distortion correction matrix to correct the original image, and the rectangle in the corrected image keeps the aspect ratio unchanged.

6. The image processing method for automatic correction of image distortion according to claim 1, characterized in that, in the correction and compensation step, a text recognition algorithm is used for compensation, and if the number or English letter is correctly recognized in the recognition result, Then there is no need to rotate the corrected image. If the recognition result does not recognize numbers or English letters, then the image needs to be rotated and corrected.