CN110309828B - Inclined license plate correction method - Google Patents
Inclined license plate correction method Download PDFInfo
- Publication number
- CN110309828B CN110309828B CN201910570623.5A CN201910570623A CN110309828B CN 110309828 B CN110309828 B CN 110309828B CN 201910570623 A CN201910570623 A CN 201910570623A CN 110309828 B CN110309828 B CN 110309828B
- Authority
- CN
- China
- Prior art keywords
- license plate
- character
- characters
- correction
- formula
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/146—Aligning or centring of the image pick-up or image-field
- G06V30/1475—Inclination or skew detection or correction of characters or of image to be recognised
- G06V30/1478—Inclination or skew detection or correction of characters or of image to be recognised of characters or characters lines
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Character Input (AREA)
- Character Discrimination (AREA)
Abstract
The invention discloses a method for correcting an inclined license plate, and belongs to the technical field of intelligent traffic. The method comprises the steps of detecting license plate characters through deep learning, connecting the central points of rectangular frames of all candidate characters pairwise to construct a map, then counting the distribution condition of slope intervals of all line segments passing through the central points of the characters, finally obtaining the inclination angle of a license plate through weighted calculation, and finishing inclination correction. The inclined license plate correction method obtained by the technology converts the inclined license plate correction problem into the mature target detection problem at present, can perform inclined correction on the common single-row license plate and double-row license plate, combines the traditional inclined license plate correction method and the deep learning method, realizes advantage complementation and higher reliability, allows the omission and false detection of license plate characters, and has high robustness on the inclined license plate needing to be corrected.
Description
Technical Field
The invention relates to the technical field of intelligent transportation, in particular to a method for correcting a tilted license plate.
Background
In the past two decades, the intelligent traffic related technology has been developed rapidly, wherein the automatic license plate recognition technology has made a major breakthrough in recognition accuracy and efficiency. However, tilting license plates remains a troublesome process for traditional three-stage license plate recognition and the latest deep learning-based license plate recognition techniques. The inclined license plate is quite common in an actual scene, and the inclined license plate correction is beneficial to widening the application range of the license plate recognition technology and enhancing the robustness of the license plate recognition technology.
The method for efficiently and accurately correcting the inclined license plate is an urgent problem to be solved in the field. The existing license plate correcting methods for inclined license plates can be divided into two types: methods based on conventional techniques and methods based on deep learning.
The method based on the traditional technology utilizes the characteristics of the inclined license plate to calculate the inclination angle for subsequent correction. The traditional method for correcting the inclined license plate mainly comprises the following steps: a Hough transform based method, a character detection analysis based method and a Radon transform based method. And calculating tilt correction parameters by detecting the upper, lower, left and right frames of the license plate based on the Hough transformation method. However, the method requires a high-contrast frame for the license plate, and is harsh in actual conditions and not suitable for most scenes. The most common method based on Radon transformation is to perform Radon transformation on a license plate image in a fixed interval range, to obtain the accumulated sum of absolute values of first-order derivatives of the transformed result, and to use the angle of Radon transformation corresponding to the maximum value of the accumulated sum as an inclination angle. The literature (Jiaxiandan; Leiwei; Wanghai Jiaojiao. a new license plate inclination correction method [ J ] based on Radon transformation, computer engineering and application, 2008,44(3):245 and 248.) carries out Radon transformation on a license plate image in the range of [ -20 degrees and 20 degrees ], and the inclined license plate is effectively corrected. The invention patent (patent number: 201510695122.1, name: a method and device for recognizing inclined license plate) performs affine transformation on character images in a fixed interval range, and calculates the inclination angle of each license plate character by combining gray value analysis, thereby determining an integral inclination angle to finish inclination correction. The invention discloses a method for correcting a horizontal inclined license plate in a surveillance video (the publication number is CN107563330A, the name is a method for correcting the horizontal inclined license plate in the surveillance video), which is characterized in that three straight lines are obtained by respectively fitting the obtained middle points of the upper edge, the middle point and the middle point of the lower edge of a plurality of character areas, the horizontal inclined angles of the three straight lines are respectively calculated, and finally the final inclined angle is obtained by weighted calculation to finish the inclination correction. However, in addition to single-line license plates, double-line license plates are also common in practice, and this method is obviously only applicable to single-line license plates.
The traditional method has the problem that a plurality of character areas can not be accurately segmented from the inclined license plate, which increases the difficulty of the method based on Radon transformation and character area analysis. The current deep learning-based method obtains a correction parameter or a correction result through a large amount of data sample training. The literature (j.wang, h.huang, x.qian, j.cao, and y.dai, "Sequence registration of Chinese license plates," neuro-projection, vol.317, pp.149-158, nov.2018.) uses a Spatial Transformer Network (STN) for oblique license plate correction based on deep learning, and obtains the correction result through a large number of data sample training. The literature (L.Xie, T.Ahmad, L.jin, Y.Liu, and S.Zhang, "A New CNN-Based Method for Multi-Directional Car License Plate Detection," IEEE Trans. inner. Trans. vol.19, No.2, pp.507-517, Feb.2018.) obtains a New network model which can regress the quadrilateral envelope of the inclined License Plate Based on the YOLO training. However, deep learning requires a large number of training samples to be prepared, and depends heavily on the quality of the training samples.
In summary, the current license plate recognition result correction method has the following defects: 1) the partial correction algorithm is applicable to single-row license plates, but not to double-row license plates; 2) a part of correction algorithms require that the license plate has a clear boundary; 3) part of the correction algorithm needs to accurately detect the character area; 4) the robustness of the tilt correction method based on deep learning is determined by the training samples.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides a method for correcting a tilted license plate. Firstly, license plate characters are detected by deep learning, then the center points of all characters are connected pairwise, then the distribution condition of the slope intervals of all line segments passing through the center points of the characters is counted, finally, the inclination angle of the license plate is obtained through weighted calculation, and inclination correction is completed.
The technical scheme of the invention is as follows:
a method for correcting an inclined license plate is characterized by comprising the following steps:
step 1: firstly, preparing a license plate character detection data set, and marking a position rectangular frame R and a category label A of each license plate character on each license plate, wherein A belongs to B, and B is a character index table; then training a deep convolution neural network model M for detecting license plate characters based on the prepared data set;
step 2: inputting the inclined license plate image I into a deep convolution neural network model M for license plate character detection, and outputting a candidate license plate character set H ═ Hi|i＝1,2,3…,nHIn which n isHIndicates the number of elements of the set H, HiThe ith candidate license plate character of the set H, HiIs a triple formed by (B, t, r), B represents the category label of the candidate character, B belongs to B, t represents the confidence coefficient of the candidate character, t belongs to [0,1 ]]R is a quadruple formed by (x, y, w, h), r represents a circumscribed rectangular frame of the candidate character, and x, y, w and h respectively represent the abscissa of the upper left corner, the ordinate of the upper left corner, the width and the height of the rectangular frame;
and step 3: to step 2 obtainThe license plate character set H connects the central points of all the characters in pairs, and records KjlThe slope of the line segment connecting the jth character and the ith character is calculated by formula (1):
wherein (ex)j,eyj) Denotes the abscissa and ordinate of the jth element in the set H, (ex)l,eyl) Represents the abscissa and ordinate of the first element in the set H, j is more than or equal to 1 and less than or equal to nH，1≤l≤nHAnd j is not equal to l;
and 4, step 4: k obtained in the step 3jlConverting into an inclination angle, and calculating the belonged angle interval according to a formula (2):
wherein arctan (×) is an arctangent function; bins represents the angular interval fraction;
returning a maximum approximate integer value of the floating point number; rjlIndication KjlThe index of the belonged angle interval;
and 5: r obtained according to step 4jlCounting the interval voting condition of each character of the license plate according to the formula (3):
wherein, χ () is an indicative function, and returns 1 when the input condition is true, otherwise returns 0; vjrIndicating the number of votes for the jth element of the set H within the equal partition r;
step 6: determining the final inclination rate of the license plate, which comprises the following specific steps:
and 7: finding the final license plate gradient k obtained in the step 6 by utilizing a beam-search algorithmHaving characters in the main row direction, and sorting the characters from small to large according to the abscissa, and recording the first character as CfThe last character is ClThen from CfTop left corner vertex coordinates, CfC and ClCalculating an affine matrix X by using the midpoint coordinates, and using the affine matrix X for the inclination correction of the license plate;
and 8: the algorithm ends.
The method for correcting the inclined license plate is characterized in that the step 6 specifically comprises the following steps:
step 6.1: v obtained according to step 5jrCalculating the maximum ticket number interval value tau according to a formula (4);
step 6.2: according to the tau determined in the step 6.1, further calculating the contribution weight of each character to the determination of the final license plate tilt rate, as shown in a formula (5):
wherein, WjThe contribution weight of the jth element in the set H to the final license plate gradient is calculated;
step 6.3: based on τ determined in step 6.1 and W determined in step 6.2jAnd calculating according to a formula (6) to obtain a final license plate gradient k:
by using the method, the inclination correction can be carried out on the common single-row license plate and the double-row license plate, and the advantage complementation is realized by combining the traditional inclined license plate correction method and the deep learning method. Compared with the existing deep learning related method, the method provided by the invention has the advantages that the problem of correcting the inclined license plate is converted into the problem of detecting the target which is mature at present, and the reliability is higher. In addition, the method of the invention allows the omission and the false detection of the license plate characters, and has no strict requirement on the inclined license plate needing to be corrected.
Drawings
FIG. 1 is an input oblique license plate image of the present invention;
FIG. 2 is a visual image of the detection result of the input license plate by the deep convolutional neural network of the present invention;
FIG. 3 is a visualization of the result of the processing of step 3 of the present invention;
FIG. 4 is a schematic diagram of step 6 of the present invention;
FIG. 5 is a diagram of an output rectified license plate image of the present invention.
Detailed Description
The following describes a specific embodiment of the method for correcting a tilted license plate according to the present invention in detail with reference to the following embodiments.
Step 1: firstly, preparing a license plate character detection data set, and marking a position rectangular frame R and a category label A of each license plate character on each license plate, wherein A belongs to B, and B is a character index table; then training a deep convolution neural network model M for detecting license plate characters based on the prepared data set; in the example, an official Yolov3 neural network structure is selected to be trained to obtain a model M;
step 2: inputting the inclined license plate image I into a license plate character detection network M as shown in FIG. 1, and outputting a candidate license plate character set H ═ Hi|i＝1,2,3…,nHIn which n isHIndicates the number of elements of the set H, HiThe ith candidate license plate character of the set H, HiIs a triple formed by (B, t, r), B represents the category label of the candidate character, B belongs to B, t represents the confidence coefficient of the candidate character, t belongs to [0,1 ]]R is a quadruple formed by (x, y, w, h), r represents a circumscribed rectangular frame of the candidate character, and x, y, w and h respectively represent the abscissa of the upper left corner, the ordinate of the upper left corner, the width and the height of the rectangular frame; in this example, a schematic view of the detection result of the license plate characters is shown in fig. 2.
And step 3: connecting the central points of all the characters pairwise in the license plate character set H obtained in the step 2, and recording K according to the graph shown in FIG. 3jlThe slope of the line segment connecting the jth character and the ith character is calculated by formula (1):
wherein (ex)j,eyj) Denotes the abscissa and ordinate of the jth element in the set H, (ex)l,eyl) Represents the abscissa and ordinate of the first element in the set H, j is more than or equal to 1 and less than or equal to nH，1≤l≤nHAnd j is not equal to l;
and 4, step 4: k obtained in the step 3jlConverting into an inclination angle, and calculating the belonged angle interval according to a formula (2):
wherein arctan (×) is an arctangent function; bins represents the angular interval fraction;
returning a maximum approximate integer value of the floating point number; rjlIndication KjlThe index of the belonged angle interval; in this example, take bins 5;
and 5: r obtained according to step 4jlCounting the interval voting condition of each character of the license plate according to the formula (3):
wherein, χ () is an indicative function, and returns 1 when the input condition is true, otherwise returns 0; vjrIndicating the number of votes for the jth element of the set H within the equal partition r;
step 6: a schematic diagram of the step of determining the final license plate inclination rate is shown in fig. 4, which specifically includes the following steps:
step 6.1: v obtained according to step 5jrCalculating the maximum ticket number interval tau according to the formula (4)；
Step 6.2: according to the tau determined in the step 6.1, further calculating the contribution weight of each character to the determination of the final license plate tilt rate, as shown in a formula (5):
wherein, WjThe contribution weight of the jth element in the set H to the final license plate gradient is calculated;
step 6.3: based on τ determined in step 6.1 and W determined in step 6.2jAnd calculating according to a formula (6) to obtain a final license plate gradient k:
and 7: according to the final license plate gradient k obtained in the step 6, finding out characters in all the main line directions by using a beam-search algorithm, sorting the characters from small to large according to the abscissa, and recording the first character as CfThe last character is ClThen from CfTop left corner vertex coordinates, CfC and ClCalculating an affine matrix X by using the midpoint coordinates, and using the affine matrix X for the inclination correction of the license plate; in this example, the corrected license plate image is shown in FIG. 5.
And 8: the algorithm ends.
Claims (1)
1. A method for correcting an inclined license plate is characterized by comprising the following steps:
step 1: firstly, preparing a license plate character detection data set, and marking a position rectangular frame R and a category label A of each license plate character on each license plate, wherein A belongs to B, and B is a character index table; then training a deep convolution neural network model M for detecting license plate characters based on the prepared data set;
step 2: inputting the inclined license plate image I into a deep convolution neural network model M for license plate character detection, and outputting a candidate license plate character set H ═ Hi|i＝1,2,3…,nHIn which n isHIndicates the number of elements of the set H, HiThe ith candidate license plate character of the set H, HiIs a triple formed by (B, t, r), B represents the category label of the candidate character, B belongs to B, t represents the confidence coefficient of the candidate character, t belongs to [0,1 ]]R is a quadruple formed by (x, y, w, h), r represents a circumscribed rectangular frame of the candidate character, and x, y, w and h respectively represent the abscissa of the upper left corner, the ordinate of the upper left corner, the width and the height of the rectangular frame;
and step 3: connecting the central points of all the characters pairwise in the license plate character set H obtained in the step 2, and recording KjlThe slope of the line segment connecting the jth character and the ith character is calculated by formula (1):
wherein (ex)j,eyj) Denotes the abscissa and ordinate of the jth element in the set H, (ex)l,eyl) Represents the abscissa and ordinate of the first element in the set H, j is more than or equal to 1 and less than or equal to nH，1≤l≤nHAnd j is not equal to l;
and 4, step 4: k obtained in the step 3jlConverting into an inclination angle, and calculating the belonged angle interval according to a formula (2):
wherein arctan (×) is an arctangent function; bins represents the angular interval fraction;
returning a maximum approximate integer value of the floating point number; rjlIndication KjlPertaining angleAn interval index;
and 5: r obtained according to step 4jlCounting the interval voting condition of each character of the license plate according to the formula (3):
wherein, χ () is an indicative function, and returns 1 when the input condition is true, otherwise returns 0; vjrIndicating the number of votes for the jth element of the set H within the equal partition r;
step 6: determining the final license plate gradient k:
the step 6 specifically comprises the following steps:
step 6.1: v obtained according to step 5jrCalculating the maximum ticket number interval value tau according to a formula (4);
step 6.2: according to the tau determined in the step 6.1, further calculating the contribution weight of each character to the determination of the final license plate tilt rate, as shown in a formula (5):
wherein, WjThe contribution weight of the jth element in the set H to the final license plate gradient is calculated;
step 6.3: based on τ determined in step 6.1 and W determined in step 6.2jAnd calculating according to a formula (6) to obtain a final license plate gradient k:
and 7: according to the final license plate gradient k obtained in the step 6, finding all the main line directions by using a beam-search algorithmThe characters are sorted from small to large according to the abscissa, and the first character is marked as CfThe last character is ClThen from CfTop left corner vertex coordinates, CfC and ClCalculating an affine matrix X by using the midpoint coordinates, and using the affine matrix X for the inclination correction of the license plate;
and 8: the algorithm ends.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910570623.5A CN110309828B (en) | 2019-06-27 | 2019-06-27 | Inclined license plate correction method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910570623.5A CN110309828B (en) | 2019-06-27 | 2019-06-27 | Inclined license plate correction method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110309828A CN110309828A (en) | 2019-10-08 |
| CN110309828B true CN110309828B (en) | 2021-08-17 |
Family
ID=68077797
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910570623.5A Active CN110309828B (en) | 2019-06-27 | 2019-06-27 | Inclined license plate correction method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110309828B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110807459B (en) * | 2019-10-31 | 2022-06-17 | 深圳市捷顺科技实业股份有限公司 | License plate correction method and device and readable storage medium |
| CN111144390B (en) * | 2019-12-20 | 2023-10-27 | 武汉瑞纳捷半导体有限公司 | Missing character searching method and system in license plate recognition and electronic equipment |
| CN113435446B (en) * | 2021-07-07 | 2023-10-31 | 南京云创大数据科技股份有限公司 | Deep learning-based inclined license plate correction method |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH08320990A (en) * | 1995-05-26 | 1996-12-03 | Japan Radio Co Ltd | Vehicle number reader |
| JP5442408B2 (en) * | 2009-11-26 | 2014-03-12 | 株式会社立山システム研究所 | Image correction method |
| US9344695B2 (en) * | 2011-09-15 | 2016-05-17 | Nec Corporation | Automatic projection image correction system, automatic projection image correction method, and non-transitory storage medium |
| CN103488978B (en) * | 2013-09-26 | 2017-08-01 | 浙江工业大学 | A kind of license plate locating method based on Gray Level Jump and character projection interval pattern |
| MY172797A (en) * | 2014-12-15 | 2019-12-12 | Mimos Berhad | A system and method to isolate characters for vehicle license plate recognition |
| CN106780351B (en) * | 2016-12-09 | 2019-11-29 | 浙江浩腾电子科技股份有限公司 | A kind of license plate sloped antidote |
| CN107563330B (en) * | 2017-09-04 | 2020-10-16 | 南京邮电大学 | Horizontal inclined license plate correction method in surveillance video |
| CN107704858A (en) * | 2017-10-31 | 2018-02-16 | 上海集光安防科技股份有限公司 | A kind of detection algorithm of license plate sloped angle |
-
2019
- 2019-06-27 CN CN201910570623.5A patent/CN110309828B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN110309828A (en) | 2019-10-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110309828B (en) | Inclined license plate correction method | |
| CN112818988B (en) | Automatic identification reading method and system for pointer instrument | |
| CN110929710B (en) | Method and system for automatically identifying meter pointer reading based on vision | |
| CN110866430B (en) | License plate recognition method and device | |
| CN110889398B (en) | Multi-modal image visibility detection method based on similarity network | |
| CN112801094B (en) | Pointer instrument image inclination correction method | |
| CN112257676A (en) | Pointer instrument reading method and system and inspection robot | |
| CN110910378A (en) | Bimodal image visibility detection method based on depth fusion network | |
| CN111553201A (en) | Traffic light detection method based on YOLOv3 optimization algorithm | |
| CN106127205A (en) | A kind of recognition methods of the digital instrument image being applicable to indoor track machine people | |
| CN111724355A (en) | Image measuring method for abalone body type parameters | |
| CN103065163B (en) | A kind of fast target based on static images detects recognition system and method | |
| CN110059713A (en) | Precipitation type identification method based on precipitation particle multi-feature parameters | |
| CN111724566A (en) | Pedestrian falling detection method and device based on intelligent lamp pole video monitoring system | |
| CN112381806A (en) | Double centromere aberration chromosome analysis and prediction method based on multi-scale fusion method | |
| CN116844147A (en) | Pointer instrument identification and abnormal alarm method based on deep learning | |
| CN113947714B (en) | Multi-mode collaborative optimization method and system for video monitoring and remote sensing | |
| KR101874968B1 (en) | Visibility measuring system base on image information and method for using the same | |
| CN111582270A (en) | Identification tracking method based on high-precision bridge region visual target feature points | |
| CN111259833A (en) | Vehicle counting method based on traffic images | |
| CN110516524A (en) | Vehicle number of axle recognition methods based on Mask R-CNN in a kind of traffic scene | |
| CN112924037A (en) | Infrared body temperature detection system and detection method based on image registration | |
| CN109325487B (en) | Full-category license plate recognition method based on target detection | |
| CN112347926A (en) | High-resolution image urban village detection method based on building form distribution | |
| CN116486212A (en) | Water gauge identification method, system and storage medium based on computer vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |