CN110781882A - License plate positioning and identifying method based on YOLO model - Google Patents

Abstract

The invention discloses a license plate positioning and identifying method based on deep learning. An improved YOLO (you Only look one) algorithm and an image super-resolution technology are adopted for optimization, and an improved YOLO convolutional neural network and a convolution enhanced SRCNN (super resolution) convolutional neural network are respectively trained. The method comprises the steps of firstly, positioning a license plate region by adopting an improved deep learning YOLO algorithm, then, correcting a detection frame by using a correction detector, solving the problem that the existing license plate positioning method cannot be correctly positioned in certain specific scenes, then, carrying out super-resolution technology processing on an image of the license plate region by using an enhanced convolutional neural network SRCNN model to obtain a picture with higher resolution and resolution ratio, and then, carrying out optical character recognition by using a neural network. In the training of the YOLO convolutional neural network, the maxout activation function is adopted to replace the activation function of the original model, the fitting capability is enhanced, the non-maximum value inhibition is improved by adjusting the threshold value, and the screening speed of the bounding box can be effectively accelerated. When the SRCNN convolutional neural network is trained, the size of a convolutional kernel and the number of convolutional layers are increased, and the image processing effect can be effectively improved, so that the method meets the requirements of real-time performance and accuracy.

Description

License plate positioning and identifying method based on YOLO model

Technical Field

The invention belongs to the technical field of artificial intelligence and computer vision recognition, and particularly relates to an automatic detection and recognition method related to a deep neural network.

Technical Field

With the rapid development of economy and the large-scale expansion of cities, traffic becomes an indispensable link of the modern society. The ever-increasing travel demands of people have prompted changes in the mode of traffic management. The early proposed intelligent traffic system can effectively relieve the imbalance contradiction, and the license plate recognition technology is to automatically extract the vehicle license plate information from the graphic data of the license plate and carry out information recognition. The license plate recognition technology is one of important components of an intelligent traffic system, and plays an important role in the aspects of stolen vehicle tracking, traffic monitoring, speed limit enforcement, automatic parking and the like.

Generally, a license plate recognition system mainly comprises license plate region detection and license plate character recognition, and complexity of external environment factors brings a crucial problem to detection, namely for license plate positioning, a method based on feature extraction cannot accurately position pictures affected by illumination, low in resolution and unclear, and has the problem of wrong license plate positioning, and a method based on gray feature needs a large amount of calculation time. Meanwhile, for character recognition, because the license plate is influenced by factors such as illumination, angle change of the license plate, dust covering on the license plate and the like, the accurate detection of each character in the license plate is also a technical problem. At present, there are many methods for recognizing license plate characters, such as a template matching-based method, a support vector machine-based method, and the like, but the existing methods have respective advantages and limitations.

The neural network method has good learning ability, fault tolerance ability and strong classification ability, and meanwhile, the great improvement of the current computing ability also allows us to compute more data, and a proper network model is selected for training according to the characteristics (color, area, central point and the like) of the license plate.

Disclosure of Invention

The invention provides a license plate positioning and identifying method based on a YOLO (you Only look one) model, aiming at the problems that the existing license plate character segmentation-based license plate identifying method cannot position a license plate in specific natural scenes such as darkness, inclination and the like, or the fuzzy license plate cannot be correctly segmented and the license plate character identifying effect is influenced. The method specifically comprises the following steps:

s1: positioning a license plate region by adopting an improved deep learning YOLO algorithm;

s2: correcting the detection frame by using a correction detector;

s3: processing the image of the license plate area by using a super-resolution technology by using an enhanced convolutional neural network (SRCNN) model;

s4: carrying out optical character recognition by utilizing a neural network;

further, the step S1 specifically includes the following steps:

A. and constructing a YOLO convolutional neural network and entering corresponding network parameter setting, wherein the model consists of 24 convolutional layers and 2 full-connection layers. For the detection task, the detection task is arranged to be realized in the first 20 convolutional layers in the preprocessing, and then 4 convolutional layers are added, and then 2 full-connection layers are added;

B. replacing a leak activation function with a maxout activation function, and performing normalization operation on all predicted values;

C. initializing the weight, wherein on one hand, the weight of the shared convolution layer is initialized by using an Imagenet classification training model, and the other layers are initialized by using zero-mean Gaussian distribution;

D. adjusting the threshold value to improve non-maximum value inhibition, thereby accelerating the screening speed of the bounding box;

E. training a convolutional neural network, setting different weights of loss during training, extracting features through a convolutional layer, inputting the features into a full-link layer for final prediction, and selecting one with the highest confidence level in a boundary frame predicted by each unit grid by using YOLO (YOLO) to perform regression on the boundary frame;

further, the step S2 specifically includes the following steps:

A. according to the comparison of the detection threshold values, an affine matrix for transforming the imaginary rectangle into the inclined license plate region is constructed;

B. carrying out affine transformation on the image;

further, the step S3 specifically includes the following steps:

A. constructing an SRCNN convolutional neural network and entering corresponding network parameter setting, wherein the network is composed of 6 convolutional layers;

B. initializing the SRCNN convolutional neural network, and initializing the training parameters in the network by using different small random numbers;

C. training a neural network of the rolling machine;

further, the step S4 specifically includes the following steps:

A. performing entry character segmentation on the license plate area, and segmenting the license plate area into individual character blocks;

B. carrying out character recognition;

the invention has the beneficial effects that:

1. under the condition that the natural environment factors are considered for positioning the license plate, the image correlation technology is utilized to enable the positioning effect to be more accurate, and the subsequent work is facilitated.

2. The corresponding threshold value is adjusted to accelerate the detection speed, so that the real-time performance is improved to some extent

3. The super-resolution processing technology of the image is added, so that the resolution and the resolution ratio of the image are improved, the subsequent character recognition work is facilitated, and the accuracy is improved.

Drawings

FIG. 1 is a flowchart of the whole license plate locating and identifying method of the present invention

FIG. 2 is a schematic diagram of the license plate detection process of the present invention

FIG. 3 is a schematic diagram of an enhanced SRCNN neural network according to the present invention

Detailed description of the invention

The invention will be further described with reference to the accompanying drawings in which:

the method comprises the following specific steps:

the method comprises the following steps: as shown in fig. 1, the license plate region is first detected. And making a data set, labeling by using LabelImg software, manually labeling the license plate region, processing the license plate region in a VOC data format, storing coordinate values of the related license plate region, and finally forming a corresponding xml file and a txt file for storage. (wherein the xml file stores the coordinate value of the license plate)

Step two: and correspondingly adjusting parameters of the YOLO network, wherein the parameters comprise the setting of a Makefile, cfg files of passacal data, tag names in a data directory and the like. Setting network training parameters, setting the number of training iterations to 50000, selecting a 'steps' mode by a learning rate strategy, setting the weight attenuation to 0.0005, and setting the batch to 64.

Step three: the YOLO network is trained. In the YOLO algorithm, when an image is acquired from a receiving end, the image is divided into a plurality of grids, each grid generates n bounding boxes for regression, each bounding box has 4 coordinate values and 1 confidence coefficient (5 parameters in total), and a plurality of category parameters are simultaneously provided, and detection can be performed through the confidence coefficient and the category probability. The YOLO model in this example consists of 24 convolutional layers and 2 fully-connected layers. For the detection task we arrange it to be implemented in the first 20 convolutional layers in the pre-processing, then add 4 convolutional layers, then 2 fully-connected layers. To make the result more accurate, the resolution of the input image needs to be increased, from 224 × 224 to 448 × 448. And finally, replacing a leak activation function with a maxout activation function, and carrying out normalization operation on all predicted values.

Step four: and correcting the candidate frame of the detection output by using a correction detector. The invention introduces a space transformation network, and the core of the network lies in the affine transformation of the added image. For a tilted license plate, first consider a fixed size imaginary square (m, n) around the center of the cell, and if the object probability of the cell is above a given detection threshold, then use the partial regression parameters to construct an affine matrix that transforms the imaginary rectangle into a tilted license plate region. Through the affine transformation matrix, a more accurate bounding box can be obtained.

Step five: and after a complete license plate area is obtained, processing the image by a super-resolution technology. The enhanced SRCNN network is used for improving the resolution of the picture, and the character recognition process is facilitated. As shown in fig. 3, the enhanced srnnn network consists of 6 convolutional layers, which are serially connected together in sequence. And continuously using convolution kernels with different sizes in the 3 rd to 5 th layers to extract features, and using convolution layers of 1 x 1 to realize a plurality of nonlinear mappings. Layer 6 reconstructs the mapped features using a 5 x 1 convolution kernel, generating a high resolution image. The parameters of this convolution enhanced SRCNN convolutional neural network are shown in Table 1:

table 1: convolution enhanced SRCNN convolutional neural network parameters

Step six: the method comprises the steps of carrying out gray level, binarization, outline extraction, external rectangle finding and block capturing on an image, realizing the purpose of character segmentation, and inputting the character into a trained ANN (Artificial neural networks) neural network for character recognition.

Claims (5)

1. A license plate positioning and identifying method based on a YOLO model is characterized by comprising the following steps:

s1: positioning a license plate region by adopting an improved deep learning YOLO algorithm;

s2: correcting the detection frame by using a correction detector;

s3: super-resolution technology for image of license plate area by using enhanced convolutional neural network SRCNN model

C, processing; s4: and performing optical character recognition by using a neural network.

2. The YOLO model-based license plate locating and recognizing method of claim 1, wherein S1 specifically comprises the following steps:

A. and constructing a YOLO convolution neural network and entering corresponding network parameter setting. In the YOLO algorithm, when an image is acquired from a receiving end, the image is divided into a plurality of grids, each grid generates n bounding boxes for regression, each bounding box has 4 coordinate values and 1 confidence coefficient (5 parameters in total), and a plurality of category parameters are simultaneously provided, and detection can be performed through the confidence coefficient and the category probability. The model consists of 24 convolutional layers and 2 fully-connected layers. For the detection task, the detection task is arranged to be realized in the first 20 convolutional layers in the preprocessing, and then 4 convolutional layers are added, and then 2 full-connection layers are added;

B. replacing a leak activation function with a maxout activation function, and performing normalization operation on all predicted values;

C. initializing the weight, wherein on one hand, the weight of the shared convolution layer is initialized by using an Imagenet classification training model, and the other layers are initialized by using zero-mean Gaussian distribution;

D. adjusting the threshold value to improve non-maximum value inhibition, thereby accelerating the screening speed of the bounding box;

E. training a convolutional neural network, setting different weights of loss during training, extracting features through a convolutional layer, inputting the features into a full-link layer for final prediction, and selecting one with the highest confidence level in a boundary box of each unit grid prediction by YOLO (regression).

3. The YOLO model-based license plate locating and recognizing method of claim 1, wherein S2 specifically comprises the following steps:

A. according to the comparison of the detection threshold values, an affine matrix for transforming the imaginary rectangle into the inclined license plate region is constructed;

B. an affine transformation of the image is performed.

4. The YOLO model-based license plate locating and recognizing method of claim 1, wherein S3 specifically comprises the following steps:

A. constructing an SRCNN convolutional neural network and entering corresponding network parameter setting, wherein the network is composed of 6 convolutional layers;

B. initializing the SRCNN convolutional neural network, and initializing the training parameters in the network by using different small random numbers;

C. and training a neural network of the rolling machine.

5. The YOLO model-based license plate locating and recognizing method of claim 1, wherein S4 specifically comprises the following steps:

A. performing entry character segmentation on the license plate area, and segmenting the license plate area into individual character blocks;

B. character recognition is performed.

Images