CN112528994B - Free angle license plate detection method, license plate recognition method and recognition system - Google Patents
Free angle license plate detection method, license plate recognition method and recognition system Download PDFInfo
- Publication number
- CN112528994B CN112528994B CN202011501675.6A CN202011501675A CN112528994B CN 112528994 B CN112528994 B CN 112528994B CN 202011501675 A CN202011501675 A CN 202011501675A CN 112528994 B CN112528994 B CN 112528994B
- Authority
- CN
- China
- Prior art keywords
- license plate
- image
- detection
- distorted
- plate detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 122
- 238000000034 method Methods 0.000 title claims abstract description 28
- 230000009466 transformation Effects 0.000 claims abstract description 40
- 238000012937 correction Methods 0.000 claims abstract description 30
- 238000012549 training Methods 0.000 claims abstract description 25
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims abstract description 21
- 238000000605 extraction Methods 0.000 claims abstract description 18
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims description 34
- 238000011176 pooling Methods 0.000 claims description 15
- 230000004913 activation Effects 0.000 claims description 12
- 230000003416 augmentation Effects 0.000 claims description 5
- 238000012544 monitoring process Methods 0.000 claims description 5
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 claims description 3
- 238000013527 convolutional neural network Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 239000006185 dispersion Substances 0.000 description 2
- 238000004880 explosion Methods 0.000 description 2
- 230000003321 amplification Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000010365 information processing Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 230000001537 neural effect Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/63—Scene text, e.g. street names
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/14—Transformations for image registration, e.g. adjusting or mapping for alignment of images
- G06T3/147—Transformations for image registration, e.g. adjusting or mapping for alignment of images using affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a free angle license plate detection method, which comprises the following steps: 1. constructing a distorted license plate detection network, wherein the network is used for positioning the license plate position and correcting the distorted license plate and comprises a feature extraction subnet and a license plate detection and correction subnet which are cascaded; the feature extraction subnetwork is used for extracting image features; the license plate detection and correction sub-network comprises a parallel license plate detection branch and a space transformation branch, and is used for carrying out affine correction on a license plate region according to affine transformation parameters when a license plate is detected; 2. constructing a training sample set, wherein the training sample is an image containing a license plate and the position coordinates of 4 corner points of the license plate in the image; 3. training the distorted license plate detection network by adopting a training sample set; 4. and inputting the image to be detected into a trained distorted license plate detection network, and outputting the image to be detected into a positioned and corrected orthographic license plate image. The method is suitable for license plate positioning detection of free angle shooting, has a correction function on the distorted image, and can obtain an orthographic license plate image.
Description
Technical Field
The invention relates to a free angle license plate recognition method, and belongs to the technical field of intelligent traffic.
Background
As a unique identification of a vehicle, reliable and accurate license plate recognition is the basis of an intelligent traffic system. The existing license plate recognition system is widely applied to positions such as a road gate, an entrance gate and the like, and images containing license plates are acquired through a camera, corresponding license plate information is extracted, and reliable information is provided for identity recognition and space positioning of vehicles. License plate recognition is divided into two processes of license plate positioning detection and character recognition, wherein the license plate positioning detection is a core link, and the image quality of a license plate region directly influences the precision of license plate character recognition, so that the final precision of license plate recognition is influenced. The current license plate positioning detection method is mainly divided into two methods based on traditional characteristics and depth characteristics.
License plate positioning methods based on traditional features have evolved earlier and mostly rely on related prior knowledge of license plate images, which can be divided into four classes based on edge features, color features, texture features and character region features (Halina kwassnica, bartosz wawrzyniak.license plate localization and recognition in camera pictures [ J ]. Artificial Intelligence methods.2002, 11:13-15.). Such methods require a large number of manual design features, rely heavily on designer experience, and have poor robustness to the effects of external conditions such as background interference, illumination variation, shooting angle, etc. In recent years, this conventional feature-based approach has gradually been replaced by a depth feature-based approach.
With the intensive research and rapid development of deep learning in recent years, a deep learning method based on convolutional neural networks (Convolutional Neural Networks, CNN) has achieved remarkable results in target detection tasks. The method also promotes a plurality of scholars at home and abroad to apply a plurality of target detection methods (Li H, shen C.reading Car License Plates Using Deep Convolutional Neural Networks and LSTMs [ J ].2016;Rafique M A,Pedrycz W,Jeon M.Vehicle license plate detection using region-based convolutional neural networks [ J ]. Soft Computing,2017,22 (3): 1-12, etc.) based on convolutional neural networks (Convolutional Neural Networks, CNN) to license plate positioning detection tasks. Compared with the traditional license plate positioning method, the license plate positioning method based on the convolutional neural network requires less manual intervention and has better robustness to shooting environment interference. However, under the conditions of large inclination angle, distortion and the like of the generated image, the traditional neural network method is adopted to extract license plate region images, so that the problems of distortion, inclination and the like are often caused, the phenomena of adhesion, distortion and the like of characters are easy to generate, a large challenge is caused to license plate character recognition, and recognition accuracy is difficult to guarantee. Such methods therefore still have less than ideal recognition results in the face of relatively free shooting angles.
Disclosure of Invention
The invention aims to: aiming at the problems in the prior art, the invention provides a free angle license plate detection method which is suitable for license plate positioning detection of free angle shooting, has a correction function on a distorted image and can obtain an orthographic license plate image.
The technical scheme is as follows: the invention adopts the following technical scheme:
the invention discloses a free angle license plate detection method, which comprises a training stage and a detection stage, wherein the training stage comprises the following steps:
s1, constructing a distorted license plate detection network, wherein the distorted license plate detection network is used for positioning the license plate position, correcting the distorted license plate, inputting the distorted license plate into an image containing the license plate and outputting the image of the license plate into an orthographic license plate; the distorted license plate detection network comprises a feature extraction subnet and a license plate detection and correction subnet which are cascaded;
the feature extraction sub-network is used for extracting image features and comprises three convolution units, each convolution unit is formed by cascading one convolution layer and one activation unit, a maximum pooling layer is arranged between the second convolution unit and a third convolution unit, three residual block units are connected behind the third convolution unit, and a maximum pooling layer is connected behind each residual block unit; each residual block unit includes at least one residual block;
the license plate detection and correction subnet comprises a parallel license plate detection branch and a space transformation branch; the license plate detection branch is used for detecting whether a license plate exists in an input image or not, and the space transformation branch is used for generating affine transformation parameters; the vehicle license plate detection system further comprises a connection unit, wherein when a license plate detection branch detection result is that a license plate exists, the connection unit carries out affine correction on a license plate region according to affine transformation parameters generated by the space transformation branch;
s2, constructing a training sample set, wherein samples in the training sample set are images containing license plates and position coordinates of 4 corner points of the license plates in the images;
s3, training the distorted license plate detection network by adopting a training sample set to obtain a distorted license plate detection network with distortion correction capability;
the detection phase comprises:
s4, inputting the image to be detected into a trained distorted license plate detection network, and outputting the image to be detected into a positioned and corrected orthographic license plate image.
Preferably, in step S3, an Adam optimizer is used to train the distorted license plate detection network.
In order to improve generalization capability of a detection network, when a training sample set is constructed in the step S2, firstly, N images containing license plates are collected, and 4 corner points of the license plates in the images are marked manually; then generating M images from each image by adopting an augmentation method to obtain N (1+M) samples;
the augmentation method comprises the following steps: rotation, cropping, mirroring, color transformation, scaling, centering, and changing aspect ratio.
On the other hand, the invention discloses a free angle license plate recognition method, which comprises the following steps:
step 1, acquiring a vehicle image, wherein the vehicle image adopts the free angle license plate detection method to position and correct a license plate, and acquires an orthographic license plate image in the vehicle image;
and 2, performing character recognition on the acquired license plate image to obtain a license plate recognition result.
Preferably, in the step 1, a single frame image in the monitoring video is acquired, and the vehicle information of the single frame image is detected by using a YOLOv3 target detection method to acquire a vehicle image.
Preferably, in the step 2, a YOLOv3 character recognition model is used to perform character recognition on the license plate image.
The invention also discloses a free angle license plate recognition system, which comprises:
the vehicle detection positioning module is used for detecting and positioning the vehicle image range in the input image;
the distorted license plate detection network is used for positioning license plate positions in a vehicle image range in an input image, correcting distorted license plates, inputting the distorted license plates into an image containing the license plates and outputting an orthographic license plate image; the distorted license plate detection network comprises a feature extraction subnet and a license plate detection and correction subnet which are cascaded;
the feature extraction sub-network is used for extracting image features and comprises three convolution units, each convolution unit is formed by cascading one convolution layer and one activation unit, a maximum pooling layer is arranged between the second convolution unit and a third convolution unit, three residual block units are connected behind the third convolution unit, and a maximum pooling layer is connected behind each residual block unit; each residual block unit includes at least one residual block;
the license plate detection and correction subnet comprises a parallel license plate detection branch and a space transformation branch; the license plate detection branch is used for detecting whether a license plate exists in an input image or not, and the space transformation branch is used for generating affine transformation parameters; the vehicle license plate detection system further comprises a connection unit, wherein when a license plate detection branch detection result is that a license plate exists, the connection unit carries out affine correction on a license plate region according to affine transformation parameters generated by the space transformation branch;
and the license plate image character recognition module is used for carrying out character recognition on the orthographic license plate image output by the distorted license plate detection network to obtain a license plate recognition result.
The beneficial effects are that: according to the free angle license plate detection method disclosed by the invention, the image feature extraction and the space transformation are combined, so that the accurate positioning and correction of the free angle license plate image are realized, the identification accuracy of license plate information under the condition of a large inclination angle is improved, the layout limit of license plate identification cameras is reduced, and the utilization rate of road monitoring cameras is improved. Has important significance for reducing resource waste and improving traffic supervision efficiency.
Drawings
FIG. 1 is a schematic diagram of a distorted license plate detection network according to the present disclosure;
FIG. 2 is a schematic diagram of a convolution unit configuration;
fig. 3 is a schematic diagram of a residual block unit structure;
FIG. 4 is a schematic diagram of a residual block structure;
FIG. 5 is a schematic diagram of a license plate detection and correction subnet;
fig. 6 is a schematic diagram of vehicle image detection in embodiment 2;
fig. 7 is a schematic diagram of the license plate image detection result in embodiment 2;
FIG. 8 is a diagram showing license plate recognition results in embodiment 2;
fig. 9 is a schematic diagram of the composition of the license plate recognition system with free angle according to the present invention.
Detailed Description
The invention is further elucidated below in connection with the drawings and the detailed description.
Example 1:
the embodiment discloses a free angle license plate detection method, which comprises a training stage and a detection stage, wherein the training stage comprises the following steps:
s1, constructing a distorted license plate detection network, wherein the distorted license plate detection network is used for positioning the license plate position, correcting the distorted license plate, inputting the distorted license plate into an image containing the license plate and outputting the image of the license plate into an orthographic license plate; the distorted license plate detection network comprises a feature extraction subnet 100 and a license plate detection and correction subnet 200 which are cascaded, as shown in fig. 1;
the feature extraction subnet 100 is used for extracting image features, and comprises three convolution units, as shown by Conv Unit 1-3 in FIG. 1; each convolution unit consists of a convolution layer 111 with a convolution kernel of 3X3 and an activation unit 112, which as shown in fig. 2 adopts a ReLu activation function;
a Max Pooling maximum Pooling layer 120 is arranged between the second convolution Unit Conv Unit 2 and the third convolution Unit Conv Unit 3, and three residual block units are connected behind the third convolution Unit, as shown in ResBock units 1-3 in FIG. 1; each residual block unit is connected with a Max Pooling maximum Pooling layer; each residual block unit comprises at least one residual block ResBock.
In order to extract the image information of a deeper layer, the network needs to be deepened, and the conditions such as gradient dispersion and gradient explosion are easy to generate in the deeper network layer, and the phenomena such as gradient dispersion and gradient explosion can be reduced by inserting a residual block ResBlock into the backbone of the feature extraction sub-network. The ResBlock may be continuously set, in this embodiment, the first residual block unit 130 includes one residual block 131, the second residual block unit 140 includes two residual blocks, and the third residual block unit 150 includes three residual blocks, as shown in fig. 3; the structure of each residual block ResBock is shown in fig. 4, comprising: the main branch consists of a first convolution layer, a first activation unit, a second convolution layer, a summation unit SUM and a second activation unit which are connected in sequence, and the side branch which is input to the summation unit from ResBock; the convolution kernels of the first convolution layer and the second convolution layer are 3X3, and the first activation unit and the second activation unit adopt a ReLu activation function.
As shown in fig. 5, the license plate detection and correction subnet 200 includes a parallel license plate detection branch 210 and a space transformation branch 220; the license plate detection branch is used for detecting whether a license plate exists in an input image or not, and the space transformation branch is used for generating affine transformation parameters; the vehicle license plate detection system further comprises a connection unit 230, wherein when the detection result of the vehicle license plate detection branch is that the vehicle license plate exists, the connection unit performs affine correction on a vehicle license plate region according to affine transformation parameters generated by the space transformation branch;
the license plate detection branch 210 first determines a virtual rectangle for each pixel in the feature map extracted by the feature extraction subnet 100, where the virtual rectangle is centered on the pixel in the feature map and has a fixed size of m×n.
The height of the input image of the distorted license plate detection network is H, the width of the input image is W, and the characteristic extraction subnet 100 has 4 maximum pooling layers MaxPooling, and the size m×n of the characteristic image after convolution is: m=h/16, n=w/16. And determining a virtual rectangle for each pixel in the feature map, wherein M is equal to N in total. The convolution layer (the convolution kernel size is 3X 3) and the SoftMax layer cascaded in the license plate detection branch 210 are used for predicting the probability that each virtual rectangle is a license plate, and if the probability is greater than a preset license plate threshold, the region covered by the virtual rectangle is considered to be a license plate region; the spatial transformation branch 220 generates affine transformation parameters for the license plate region. The spatial transformation branch 220 adopts a spatial transformation network (Spatial Transformer Network, STN), and the specific content is as follows: jaderberg M, simonyan K, zisselman A.spatial transformer networks [ C ]// Advances in neural information processing systems.2015:2017-2025.
If the license plate detection branch 210 detects a plurality of license plate regions in one feature map, that is, the probability that a plurality of virtual rectangles are license plates is greater than a preset license plate threshold, at this time, calculating the intersection ratio between every two of the plurality of virtual rectangles, and if the intersection ratio is greater than the preset intersection ratio threshold, considering that the two corresponding rectangular regions are the same license plate, thereby merging the same license plate regions, and preventing the same license plate from being detected with a plurality of license plate images.
S2, constructing a training sample set, wherein samples in the training sample set are images containing license plates and position coordinates of 4 corner points of the license plates in the images;
in order to improve generalization capability of a distorted license plate detection network, when a training sample set is constructed, firstly, 200 images containing license plates are collected, and 4 corner points of the license plates in the images are marked manually; then generating 9 images for each image by adopting an augmentation method to obtain 2000 samples in total; the amplification method adopted in the embodiment comprises the following steps: rotation, cropping, mirroring, color transformation, scaling, centering, changing aspect ratios, etc.
S3, training the distorted license plate detection network by adopting a training sample set to obtain a distorted license plate detection network with distortion correction capability; in this embodiment, an Adam optimizer is used to train the distorted license plate detection network.
The detection phase comprises:
s4, inputting the image to be detected into a trained distorted license plate detection network, and outputting the image to be detected into a positioned and corrected orthographic license plate image.
Example 2:
the embodiment discloses a free angle license plate recognition method, which comprises the following steps:
step 1, acquiring a vehicle image;
in this embodiment, license plate recognition is performed on a vehicle image in a monitoring video acquired by an outdoor parking lot camera. Firstly, a single frame image in a monitoring video is acquired, and vehicle information detection is carried out on the single frame image by adopting a YOLOv3 target detection method to acquire a vehicle image. As shown in fig. 6, images of 3 vehicles are detected. Wherein the vehicle twist angle on the left side is larger. Carrying out license plate positioning and correction by adopting the free angle license plate detection method in the embodiment 1, and obtaining an orthographic license plate image in the vehicle image; the result is shown in fig. 7, wherein (a) in fig. 7 is the corrected license plate image of the left-hand vehicle in fig. 6, and (b) in fig. 7 is the corrected license plate image of the right-hand lower vehicle in fig. 6; although the upper right vehicle is detected in fig. 6, the license plate image is not detected because of the undersize thereof, and thus the subsequent recognition step cannot be performed.
And 2, performing character recognition on the acquired license plate image to obtain a license plate recognition result.
In this embodiment, the YOLOv3 character recognition model is used to perform character recognition on the license plate image in fig. 7, the recognition result is shown in fig. 8, and the confidence of the recognition result is given. In this embodiment, the confidence of the recognition result is all above 0.97.
The embodiment also discloses a recognition system for realizing the free angle license plate recognition method, as shown in fig. 9, which comprises the following steps:
the vehicle detection positioning module is used for detecting and positioning the vehicle image range in the input image;
the distorted license plate detection network is used for positioning license plate positions in a vehicle image range in an input image, correcting distorted license plates, inputting the distorted license plates into an image containing the license plates and outputting an orthographic license plate image; the distorted license plate detection network comprises a feature extraction subnet and a license plate detection and correction subnet which are cascaded;
the feature extraction sub-network is used for extracting image features and comprises three convolution units, each convolution unit is formed by cascading one convolution layer and one activation unit, a maximum pooling layer is arranged between the second convolution unit and a third convolution unit, three residual block units are connected behind the third convolution unit, and a maximum pooling layer is connected behind each residual block unit; each residual block unit includes at least one residual block;
the license plate detection and correction subnet comprises a parallel license plate detection branch and a space transformation branch; the license plate detection branch is used for detecting whether a license plate exists in an input image or not, and the space transformation branch is used for generating affine transformation parameters; the vehicle license plate detection system further comprises a connection unit, wherein when a license plate detection branch detection result is that a license plate exists, the connection unit carries out affine correction on a license plate region according to affine transformation parameters generated by the space transformation branch;
and the license plate image character recognition module is used for carrying out character recognition on the orthographic license plate image output by the distorted license plate detection network to obtain a license plate recognition result.
Claims (7)
1. The free angle license plate detection method is characterized by comprising a training stage and a detection stage, wherein the training stage comprises the following steps:
s1, constructing a distorted license plate detection network, wherein the distorted license plate detection network is used for positioning the license plate position, correcting the distorted license plate, inputting the distorted license plate into an image containing the license plate and outputting the image of the license plate into an orthographic license plate; the distorted license plate detection network comprises a feature extraction subnet and a license plate detection and correction subnet which are cascaded;
the feature extraction sub-network is used for extracting image features and comprises three convolution units, each convolution unit is formed by cascading one convolution layer and one activation unit, a maximum pooling layer is arranged between the second convolution unit and a third convolution unit, three residual block units are connected behind the third convolution unit, and a maximum pooling layer is connected behind each residual block unit; each residual block unit includes at least one residual block;
the license plate detection and correction subnet comprises a parallel license plate detection branch and a space transformation branch; the license plate detection branch is used for detecting whether a license plate exists in an input image or not, and the space transformation branch is used for generating affine transformation parameters; the vehicle license plate detection system further comprises a connection unit, wherein when a license plate detection branch detection result is that a license plate exists, the connection unit carries out affine correction on a license plate region according to affine transformation parameters generated by the space transformation branch;
s2, constructing a training sample set, wherein samples in the training sample set are images containing license plates and position coordinates of 4 corner points of the license plates in the images;
s3, training the distorted license plate detection network by adopting a training sample set to obtain a distorted license plate detection network with distortion correction capability;
the detection phase comprises:
s4, inputting the image to be detected into a trained distorted license plate detection network, and outputting the image to be detected into a positioned and corrected orthographic license plate image.
2. The free angle license plate detection method of claim 1, wherein the distorted license plate detection network is trained using an Adam optimizer in step S3.
3. The method for detecting the license plate at the free angle according to claim 1, wherein when the training sample set is constructed in the step S2, firstly, N images containing the license plate are collected, and 4 corner points of the license plate in the images are marked manually; then generating M images from each image by adopting an augmentation method to obtain N (1+M) samples;
the augmentation method comprises the following steps: rotation, cropping, mirroring, color transformation, scaling, centering, and changing aspect ratio.
4. The method for identifying the license plate at the free angle is characterized by comprising the following steps of:
step 1, acquiring a vehicle image, wherein the vehicle image adopts the free angle license plate detection method of any one of claims 1-3 to perform license plate positioning and correction, and acquires an orthographic license plate image in the vehicle image;
and 2, performing character recognition on the acquired license plate image to obtain a license plate recognition result.
5. The method according to claim 4, wherein in step 1, the vehicle image is obtained by obtaining a single frame image in the monitoring video, and detecting the vehicle information of the single frame image by using YOLOv3 target detection method.
6. The method for recognizing a license plate at a free angle according to claim 4, wherein in the step 2, a YOLOv3 character recognition model is used to recognize characters in the license plate image.
7. A free angle license plate recognition system, comprising:
the vehicle detection positioning module is used for detecting and positioning the vehicle image range in the input image;
the distorted license plate detection network is used for positioning license plate positions in a vehicle image range in an input image, correcting distorted license plates, inputting the distorted license plates into an image containing the license plates and outputting an orthographic license plate image; the distorted license plate detection network comprises a feature extraction subnet and a license plate detection and correction subnet which are cascaded;
the feature extraction sub-network is used for extracting image features and comprises three convolution units, each convolution unit is formed by cascading one convolution layer and one activation unit, a maximum pooling layer is arranged between the second convolution unit and a third convolution unit, three residual block units are connected behind the third convolution unit, and a maximum pooling layer is connected behind each residual block unit; each residual block unit includes at least one residual block;
the license plate detection and correction subnet comprises a parallel license plate detection branch and a space transformation branch; the license plate detection branch is used for detecting whether a license plate exists in an input image or not, and the space transformation branch is used for generating affine transformation parameters; the vehicle license plate detection system further comprises a connection unit, wherein when a license plate detection branch detection result is that a license plate exists, the connection unit carries out affine correction on a license plate region according to affine transformation parameters generated by the space transformation branch;
and the license plate image character recognition module is used for carrying out character recognition on the orthographic license plate image output by the distorted license plate detection network to obtain a license plate recognition result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011501675.6A CN112528994B (en) | 2020-12-18 | 2020-12-18 | Free angle license plate detection method, license plate recognition method and recognition system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011501675.6A CN112528994B (en) | 2020-12-18 | 2020-12-18 | Free angle license plate detection method, license plate recognition method and recognition system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112528994A CN112528994A (en) | 2021-03-19 |
| CN112528994B true CN112528994B (en) | 2024-03-29 |
Family
ID=75001347
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011501675.6A Active CN112528994B (en) | 2020-12-18 | 2020-12-18 | Free angle license plate detection method, license plate recognition method and recognition system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112528994B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113128507B (en) * | 2021-05-12 | 2023-03-21 | 锐捷网络股份有限公司 | License plate recognition method and device, electronic equipment and storage medium |
| CN113435446B (en) * | 2021-07-07 | 2023-10-31 | 南京云创大数据科技股份有限公司 | Deep learning-based inclined license plate correction method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106548155A (en) * | 2016-10-28 | 2017-03-29 | 安徽四创电子股份有限公司 | A kind of detection method of license plate based on depth belief network |
| CN110427937A (en) * | 2019-07-18 | 2019-11-08 | 浙江大学 | A kind of correction of inclination license plate and random length licence plate recognition method based on deep learning |
| CN111325203A (en) * | 2020-01-21 | 2020-06-23 | 福州大学 | American license plate recognition method and system based on image correction |
| CN111461113A (en) * | 2020-03-03 | 2020-07-28 | 华南理工大学 | Large-angle license plate detection method based on deformed plane object detection network |
| CN111695561A (en) * | 2020-05-25 | 2020-09-22 | 南京博雅集智智能技术有限公司 | License plate detection and correction recognition method and recognition system based on SSD |
-
2020
- 2020-12-18 CN CN202011501675.6A patent/CN112528994B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106548155A (en) * | 2016-10-28 | 2017-03-29 | 安徽四创电子股份有限公司 | A kind of detection method of license plate based on depth belief network |
| CN110427937A (en) * | 2019-07-18 | 2019-11-08 | 浙江大学 | A kind of correction of inclination license plate and random length licence plate recognition method based on deep learning |
| CN111325203A (en) * | 2020-01-21 | 2020-06-23 | 福州大学 | American license plate recognition method and system based on image correction |
| CN111461113A (en) * | 2020-03-03 | 2020-07-28 | 华南理工大学 | Large-angle license plate detection method based on deformed plane object detection network |
| CN111695561A (en) * | 2020-05-25 | 2020-09-22 | 南京博雅集智智能技术有限公司 | License plate detection and correction recognition method and recognition system based on SSD |
Non-Patent Citations (2)
| Title |
|---|
| 基于深度学习的倾斜车牌矫正识别;李晓燃;;软件;20181015(第10期);全文 * |
| 轻量级车牌识别卷积网络;贺智龙;肖中俊;严志国;;齐鲁工业大学学报;20200815(第04期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112528994A (en) | 2021-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109886896B (en) | Blue license plate segmentation and correction method | |
| CN108121991B (en) | Deep learning ship target detection method based on edge candidate region extraction | |
| CN105046196B (en) | Front truck information of vehicles structuring output method based on concatenated convolutional neutral net | |
| CN104700072B (en) | Recognition methods based on lane line historical frames | |
| CN106971185B (en) | License plate positioning method and device based on full convolution network | |
| CN112528994B (en) | Free angle license plate detection method, license plate recognition method and recognition system | |
| CN113052170B (en) | Small target license plate recognition method under unconstrained scene | |
| CN112329776B (en) | License plate detection method and device based on improved CenterNet network | |
| Yang et al. | A vehicle license plate recognition system based on fixed color collocation | |
| CN111444778A (en) | Lane line detection method | |
| CN114973207B (en) | Road sign identification method based on target detection | |
| CN114782770A (en) | License plate detection and recognition method and system based on deep learning | |
| CN110969164A (en) | Low-illumination imaging license plate recognition method and device based on deep learning end-to-end | |
| CN112784834A (en) | Automatic license plate identification method in natural scene | |
| CN114898353B (en) | License plate recognition method based on video sequence image characteristics and information | |
| CN111062381B (en) | License plate position detection method based on deep learning | |
| Wang et al. | Vehicle license plate recognition based on wavelet transform and vertical edge matching | |
| CN115294558A (en) | Large-angle license plate recognition system and method thereof | |
| CN111444916A (en) | License plate positioning and identifying method and system under unconstrained condition | |
| CN116994236A (en) | Low-quality image license plate detection method based on deep neural network | |
| CN111401364A (en) | License plate positioning algorithm based on combination of color features and template matching | |
| CN112053407B (en) | Automatic lane line detection method based on AI technology in traffic law enforcement image | |
| CN105022992A (en) | Automatic identification method of vehicle license plate | |
| CN104408430B (en) | License plate positioning method and device | |
| CN114758002B (en) | Photovoltaic string position determining method and system based on aerial image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |