CN113159204A - License plate recognition model generation method, license plate recognition method and related components - Google Patents
License plate recognition model generation method, license plate recognition method and related components Download PDFInfo
- Publication number
- CN113159204A CN113159204A CN202110467812.7A CN202110467812A CN113159204A CN 113159204 A CN113159204 A CN 113159204A CN 202110467812 A CN202110467812 A CN 202110467812A CN 113159204 A CN113159204 A CN 113159204A
- Authority
- CN
- China
- Prior art keywords
- license plate
- image
- layer
- recognition model
- plate recognition
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000012549 training Methods 0.000 claims abstract description 63
- 238000012545 processing Methods 0.000 claims abstract description 34
- 238000010586 diagram Methods 0.000 claims abstract description 18
- 238000007781 pre-processing Methods 0.000 claims abstract description 16
- 238000005457 optimization Methods 0.000 claims abstract description 13
- 238000011176 pooling Methods 0.000 claims description 20
- 230000004913 activation Effects 0.000 claims description 19
- 238000010606 normalization Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 5
- 239000010410 layer Substances 0.000 description 126
- 230000008569 process Effects 0.000 description 13
- 238000004891 communication Methods 0.000 description 6
- 238000013528 artificial neural network Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000009471 action Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000002355 dual-layer Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computational Linguistics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Character Discrimination (AREA)
- Image Analysis (AREA)
Abstract
The application discloses a license plate recognition model generation method, a license plate recognition method and related components, which comprise the following steps: acquiring a license plate image, and preprocessing the license plate image to obtain a sample image, wherein the license plate in the license plate image is a double-layer license plate; constructing a sample training set based on the sample image, and training a blank model constructed based on a preset full convolution network and a connection time sequence classification by using the sample training set to obtain a trained license plate recognition model; the system comprises a preset full convolution network, a data processing unit and a data processing unit, wherein the preset full convolution network is used for processing a sample image to obtain a corresponding two-dimensional characteristic map, segmenting the two-dimensional characteristic map into an upper one-dimensional characteristic map and a lower one-dimensional characteristic map, and splicing the two one-dimensional characteristic maps in a serial mode to obtain a spliced characteristic map; and the connection time sequence classification is used for carrying out optimization training on the blank model based on the characteristic splicing diagram. The vehicle license plate recognition method and the vehicle license plate recognition system perform end-to-end recognition by using the vehicle license plate recognition model based on the preset full convolution network and the connection time sequence classification, have higher robustness for various double-layer vehicle license plate recognition, and improve the recognition efficiency.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a license plate recognition model generation method, a license plate recognition device, license plate recognition equipment and a storage medium.
Background
With the high-speed development of Chinese economy, the quantity of automobile reserves is continuously increased, the number of double-layer license plates is increased, so that the problems of road congestion and difficult parking are more serious, the existing license plate recognition technology is applied to scenes such as expressways, urban roadside roads, indoor and outdoor parking lots and the like, and the problem of how to quickly, accurately and robustly recognize various double-layer license plates is also a great challenge for the stable development of the automobile industry at present due to the problems of environmental influence, license plate damage and fouling, multiple types of double-layer license plates and the like.
Although the traditional double-layer license plate recognition method has certain guarantee on the recognition rate, the recognition process is too complicated, characters in the license plate need to be positioned and segmented firstly, then the characters are classified and recognized singly, and finally the license plate recognition result is obtained.
Disclosure of Invention
In view of the above, the present invention aims to provide a license plate recognition model generation method, a license plate recognition method and related components, which perform end-to-end recognition by using a license plate recognition model classified based on a connection timing sequence, have high robustness for various double-layer license plate recognition, and improve recognition efficiency. The specific scheme is as follows:
a first aspect of the present application provides a license plate recognition model generation method, including:
acquiring a license plate image, and preprocessing the license plate image to obtain a sample image; wherein the license plate in the license plate image is a double-layer license plate;
constructing a sample training set based on the sample images, and training a blank model constructed based on a preset full convolution network and a connection time sequence classification by using the sample training set to obtain a trained license plate recognition model; the preset full convolution network is used for processing the sample images in the training set to obtain corresponding two-dimensional feature maps, segmenting the two-dimensional feature maps into an upper one-dimensional feature map and a lower one-dimensional feature map, and splicing the two one-dimensional feature maps in a serial mode to obtain spliced feature maps corresponding to the two-dimensional feature maps; and the connection time sequence classification is used for carrying out optimization training on the blank model based on the characteristic splicing diagram.
Optionally, the preset full convolution network includes a convolution layer, a batch normalization layer, an activation layer, a pooling layer, and a Fire module.
Optionally, the processing, by the preset full convolution network, the sample image in the training set to obtain a corresponding two-dimensional feature map includes:
and sequentially inputting the sample images in the training set into one convolution layer, one batch of normalization layers and the activation layer structure, one pooling layer, two Fire modules and batch of normalization layers and the activation layer structure, one pooling layer, five Fire modules and batch of normalization layers and activation layer structures, one pooling layer and one convolution layer and activation layer structure for processing to obtain a corresponding two-dimensional feature map.
Optionally, the preprocessing the license plate image to obtain a sample image includes:
acquiring the position and the size of a license plate in the license plate image to obtain license plate information in the license plate image, and performing external expansion processing on the left and right boundaries of the license plate based on the license plate information;
setting the size of the license plate subjected to the external expansion processing as a preset size to obtain a sample image; and the preset size is the size of an input image of the license plate recognition model.
A second aspect of the present application provides a license plate recognition method, based on the license plate recognition model, including:
acquiring a license plate image to be recognized, and preprocessing the license plate image to be recognized to obtain a target license plate image; the license plate in the license plate image to be recognized is a double-layer license plate;
and inputting the target license plate image into the license plate recognition model so that the license plate recognition model can recognize the target license plate image.
Optionally, the inputting the target license plate image into the license plate recognition model so that after the license plate recognition model recognizes the target license plate image, the method further includes:
and decoding the recognition result output by the license plate recognition model by using a greedy algorithm to obtain the license plate information of the license plate image to be recognized.
A third aspect of the present application provides a license plate recognition model generation apparatus, including:
the data acquisition interface is used for acquiring a license plate image and preprocessing the license plate image to obtain a sample image; wherein the license plate in the license plate image is a double-layer license plate;
the training device is used for constructing a sample training set based on the sample images and training a blank model constructed based on a preset full convolution network and connection time sequence classification by using the sample training set so as to obtain a trained license plate recognition model; the preset full convolution network is used for processing the sample images in the training set to obtain corresponding two-dimensional feature maps, segmenting the two-dimensional feature maps into an upper one-dimensional feature map and a lower one-dimensional feature map, and splicing the two one-dimensional feature maps in a serial mode to obtain spliced feature maps corresponding to the two-dimensional feature maps; and the connection time sequence classification is used for carrying out optimization training on the blank model based on the characteristic splicing diagram.
A fourth aspect of the present application provides a license plate recognition apparatus, based on the aforementioned license plate recognition model, including:
the system comprises an acquisition module, a storage module and a display module, wherein the acquisition module is used for acquiring a license plate image to be recognized and preprocessing the license plate image to be recognized to obtain a target license plate image; the license plate in the license plate image to be recognized is a double-layer license plate;
and the recognition module is used for inputting the target license plate image into the license plate recognition model so that the license plate recognition model can recognize the target license plate image.
A fifth aspect of the present application provides an electronic device comprising a processor and a memory; the memory is used for storing a computer program which is loaded and executed by the processor to realize the license plate recognition model generation method and the license plate recognition method.
A sixth aspect of the present application provides a computer-readable storage medium, where computer-executable instructions are stored in the computer-readable storage medium, and when the computer-executable instructions are loaded and executed by a processor, the method for generating a license plate recognition model and the method for recognizing a license plate are implemented.
According to the method, a license plate image is obtained and preprocessed to obtain a sample image, wherein the license plate in the license plate image is a double-layer license plate. Secondly, a sample training set is constructed based on the sample images, and a blank model constructed based on a preset full convolution network and connection time sequence classification is trained by the sample training set to obtain a trained license plate recognition model; the system comprises a preset full convolution network, a data processing unit and a data processing unit, wherein the preset full convolution network is used for processing a sample image to obtain a corresponding two-dimensional characteristic map, segmenting the two-dimensional characteristic map into an upper one-dimensional characteristic map and a lower one-dimensional characteristic map, and splicing the two one-dimensional characteristic maps in a serial mode to obtain a spliced characteristic map; and the connection time sequence classification is used for carrying out optimization training on the blank model based on the characteristic splicing diagram. The vehicle license plate recognition method and the vehicle license plate recognition system perform end-to-end recognition by using the vehicle license plate recognition model based on the preset full convolution network and the connection time sequence classification, have higher robustness for various double-layer vehicle license plate recognition, and improve the recognition efficiency.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flowchart of a license plate recognition model generation method provided by the present application;
FIG. 2 is a block diagram of a specific Fire module provided in the present application;
FIG. 3 is a schematic diagram of a specific end-to-end license plate recognition network architecture provided in the present application;
FIG. 4 is a flowchart of a license plate recognition method provided by the present application;
fig. 5 is a schematic structural diagram of a license plate recognition model generation device provided in the present application;
fig. 6 is a schematic structural diagram of a license plate recognition device provided in the present application;
fig. 7 is a structural diagram of a license plate recognition model generation or license plate recognition electronic device provided by the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The existing double-layer license plate recognition method needs to position and divide characters in a license plate, then carries out classification recognition on the characters individually, and finally obtains a license plate recognition result. In order to overcome the technical defects, the application provides a license plate recognition model generation method and a license plate recognition method, the license plate recognition model classified based on a preset full convolution network and a connection time sequence is used for carrying out end-to-end recognition, the robustness for recognition of various double-layer license plates is high, and the recognition efficiency is improved.
Fig. 1 is a flowchart of a license plate recognition model generation method according to an embodiment of the present disclosure. Referring to fig. 1, the license plate recognition model generation method includes:
s11: acquiring a license plate image, and preprocessing the license plate image to obtain a sample image; and the license plate in the license plate image is a double-layer license plate.
In the embodiment, a license plate image is obtained first, and the license plate image is preprocessed to obtain a sample image for training a model, wherein a license plate in the license plate image is a double-layer license plate. In order to enable the trained license plate recognition model to have higher recognition accuracy, the quantity and quality of sample images need to be strictly controlled. Double-layer license plate images in different environments, different angles and different blurriness degrees are collected, so that the model can have better generalization performance, and the double-layer license plate images can be RGB color pictures containing various double-layer license plates downloaded from a network or the RGB color pictures containing the double-layer license plates automatically captured according to actual scenes. The pre-processing process of the double-layer license plate image may specifically be to first obtain a position and a size of a double-layer license plate in the double-layer license plate image to obtain double-layer license plate information in the double-layer license plate image, perform outward expansion processing on a left boundary and a right boundary of the double-layer license plate based on the double-layer license plate information, and then set the size of the double-layer license plate subjected to the outward expansion processing to a preset size to obtain a sample image, where the preset size is a size W × H of an input image of the license plate recognition model (W is a width of a picture, and H is a height of the picture). The method can be understood as a method that the position of a double-layer license plate in a double-layer license plate image can be detected by using the existing license plate detection model so as to obtain a double-layer license plate area, specifically, the size and the position of the double-layer license plate can be obtained according to license plate positioning, and random expansion is carried out on the left and right boundaries in a range from zero to one tenth of the width of the license plate based on license plate position information. In this embodiment, the license plate detection model is not limited, and further, the size of the license plate is corrected according to the position of the license plate and then fixed to the size of the input image of the model to be trained.
S12: constructing a sample training set based on the sample images, and training a blank model constructed based on a preset full convolution network and a connection time sequence classification by using the sample training set to obtain a trained license plate recognition model; the preset full convolution network is used for processing the sample images in the training set to obtain corresponding two-dimensional feature maps, segmenting the two-dimensional feature maps into an upper one-dimensional feature map and a lower one-dimensional feature map, and splicing the two one-dimensional feature maps in a serial mode to obtain spliced feature maps corresponding to the two-dimensional feature maps; and the connection time sequence classification is used for carrying out optimization training on the blank model based on the characteristic splicing diagram.
In this embodiment, a sample training set is constructed based on the sample image, where the sample training set includes the sample image and tag information corresponding to the sample image, and the tag information is a character string that can represent a license plate serial number of the sample image. And then, training a blank model which is constructed based on a preset full convolution network and a connection time sequence classification by using the sample training set to obtain a trained license plate recognition model. The Connection Timing Classification (CTC) is a tool for sequence modeling, and training samples do not need to be aligned, and specifically, a connection timing Classification Loss function (CTC Loss) is used for network training. In this embodiment, the connection timing sequence classification loss function is used to connect the feature splicing map for model optimization training. The connection time sequence is applied to the double-layer license plate recognition model in a classified mode, the trained model can support double-layer license plate recognition of various types of characters with variable length, with the help of an efficient convolution structure, the number of network parameters is less, the recognition efficiency is higher, and the problems that the traditional double-layer license plate recognition process is complicated, the robustness of a complex environment is low and the like are solved.
The preset full convolution neural network comprises a convolution layer, a Batch Normalization layer (BN), an activation layer, a pooling layer and a Fire module. The batch normalization layer is used for transforming a special function of the numerical value to form a normalized numerical value, the problem that the model training difficulty is increased due to deepening of the number of layers of the neural network is solved when the batch normalization layer is applied to the preset full-convolution neural network, network convergence can be accelerated when the whole network is added into the batch normalization layer, and meanwhile the generalization capability of the network model is increased. The activation function of the activation layer may be a Linear rectification function (ReLU), and the pooling layer is a maximum pooling layer. The Fire module is a special part of a lightweight network (squeezet), and as shown in fig. 2, the Fire module is composed of two layers, namely, a squeeze layer and an expanded layer, wherein the squeeze layer is a convolution layer of 1 × 1 convolution kernel, the expanded layer is a convolution layer of 1 × 1 convolution kernel and 3 × 3 convolution kernel, and in the expanded layer, feature maps obtained by 1 × 1 convolution kernel and 3 × 3 convolution kernel are subjected to concatation, so that features can be extracted efficiently, and the calculation amount is reduced. And processing the sample images in the sample training set by the preset full convolution network to obtain the corresponding two-dimensional characteristic image of the license plate image, namely the height of the preset full convolution network characteristic output layer is 2. The two-dimensional feature map is divided into an upper one-dimensional feature map and a lower one-dimensional feature map, namely, the feature layer is divided into two halves in the height direction, so that the upper part and the lower part are both one-dimensional feature maps, wherein the features in the two one-dimensional feature maps are in a mapping relation with the license plate information of two levels in a double-layer license plate, namely, the feature information in the one-dimensional feature map of the upper part is mapped to the features in the upper license plate, and the feature information in the one-dimensional feature map of the lower part is mapped to the features in the lower license plate. And splicing the two one-dimensional characteristic graphs in a series connection mode to obtain a spliced characteristic graph corresponding to the two-dimensional characteristic graph, and finally sending the spliced characteristic graph into a connection time sequence classification structure to carry out optimization training on the blank model.
This embodiment provides a specific structure of the preset full convolution network, which is shown in fig. 3. The preset full convolution neural network sequentially inputs the sample images in the training set into one convolution layer, one batch normalization layer + the activation layer structure, one pooling layer, two Fire modules + the batch normalization layer + the activation layer structure, one pooling layer, five Fire modules + the batch normalization layer + the activation layer structure, one pooling layer and one convolution layer + the activation layer structure for processing to obtain a corresponding two-dimensional feature map. Namely, the network input is an RGB image after detecting and correcting the size of a license plate, the RGB image passes through a convolutional layer (Conv1), a batch standardization layer (BN), an activation layer (RELU) and a Pooling layer, and 8 Fire modules are connected in series behind the convolutional layer, wherein the RGB image also comprises 4 maximum Pooling layers (Pooling), a convolutional layer (Conv2) and an activation layer (RELU), the height of the characteristic layer is 2 at the moment in Conv2+ RELU, the characteristic layer is divided into an upper part and a lower part in the height direction, the upper part and the lower part are spliced together from left to right, the RGB image is directly accessed to a CTC Loss during training, and an Adam optimization strategy is used for training. The preset full convolution network is only used as an implementation manner of this embodiment, and the specific structure of the preset neural network is not limited in this embodiment.
Therefore, in the embodiment of the application, the license plate image is firstly obtained and is preprocessed to obtain the sample image, wherein the license plate in the license plate image is a double-layer license plate. Secondly, a sample training set is constructed based on the sample images, and a blank model constructed based on a preset full convolution network and connection time sequence classification is trained by the sample training set to obtain a trained license plate recognition model; the system comprises a preset full convolution network, a data processing unit and a data processing unit, wherein the preset full convolution network is used for processing a sample image to obtain a corresponding two-dimensional characteristic map, segmenting the two-dimensional characteristic map into an upper one-dimensional characteristic map and a lower one-dimensional characteristic map, and splicing the two one-dimensional characteristic maps in a serial mode to obtain a spliced characteristic map; and the connection time sequence classification is used for carrying out optimization training on the blank model based on the characteristic splicing diagram. According to the embodiment of the application, the trained license plate recognition model has fewer network parameters, supports double-layer license plate recognition with characters being lengthened, can ensure fast and efficient recognition rate, effectively avoids the problems of character segmentation, complex process, low robustness and the like of the traditional double-layer license plate recognition method, and can be more suitable for various double-layer license plate recognition in complex environments.
Fig. 4 is a flowchart of a license plate recognition method provided in an embodiment of the present application, which is based on the license plate recognition model. Referring to fig. 4, the license plate recognition method includes:
s21: acquiring a license plate image to be recognized, and preprocessing the license plate image to be recognized to obtain a target license plate image; and the license plate in the license plate image to be recognized is a double-layer license plate.
In the embodiment, a license plate image to be recognized is obtained, and the license plate image to be recognized is preprocessed to obtain a target license plate image, wherein a license plate in the license plate image to be recognized is a double-layer license plate. The preprocessing process may refer to specific contents disclosed in the above embodiments, which are not described in detail in this embodiment.
S22: and inputting the target license plate image into the license plate recognition model so that the license plate recognition model can recognize the target license plate image.
In this embodiment, after the target license plate image is input to the license plate recognition model, the license plate recognition model may refer to the specific contents disclosed in the above embodiments for the process of recognizing the target license plate image, which is not described in detail in this embodiment. It should be noted that, in the actual recognition process, generally speaking, the license plate recognition model recognizes a series of character strings, and the character strings need to be converted into actual license plate numbers for convenient viewing, so that after the target license plate image is input to the license plate recognition model so that the license plate recognition model recognizes the target license plate image, the recognition result output by the license plate recognition model can be decoded by using Greedy algorithm (Greedy algorithm) to obtain the license plate information of the license plate image to be recognized, that is, the license plate information is decoded by using Greedy decode method, so as to obtain the final result of license plate recognition. Of course, the above decoding process may be integrated with the license plate recognition model, that is, the output of the last layer of the license plate recognition model is decoded to obtain the final result of the license plate recognition, and then the final result is output by the license plate recognition model. The method can be suitable for recognizing various types of double-layer license plates, including hongkong Australian double-layer license plates and overseas double-layer license plates with large character number change.
Therefore, the license plate image to be recognized is obtained first, and the license plate image to be recognized is preprocessed to obtain the target license plate image, wherein the license plate in the license plate image to be recognized is a double-layer license plate. And then inputting the target license plate image into the license plate recognition model so that the license plate recognition model can recognize the target license plate image. According to the embodiment of the application, the license plate recognition model based on the preset full convolution network and the connection time sequence classification is used for carrying out end-to-end recognition, so that the robustness for recognition of various double-layer license plates is high, and the recognition efficiency is improved.
Referring to fig. 5, an embodiment of the present application further discloses a license plate recognition model generation apparatus, which includes:
the data acquisition interface 11 is used for acquiring a license plate image and preprocessing the license plate image to obtain a sample image; wherein the license plate in the license plate image is a double-layer license plate;
the trainer 12 is used for constructing a sample training set based on the sample images and utilizing the sample training set to train a blank model constructed based on a preset full convolution network and a connection time sequence classification so as to obtain a trained license plate recognition model; the preset full convolution network is used for processing the sample images in the training set to obtain corresponding two-dimensional feature maps, segmenting the two-dimensional feature maps into an upper one-dimensional feature map and a lower one-dimensional feature map, and splicing the two one-dimensional feature maps in a serial mode to obtain spliced feature maps corresponding to the two-dimensional feature maps; and the connection time sequence classification is used for carrying out optimization training on the blank model based on the characteristic splicing diagram.
Therefore, in the embodiment of the application, the license plate image is firstly obtained and is preprocessed to obtain the sample image, wherein the license plate in the license plate image is a double-layer license plate. Secondly, a sample training set is constructed based on the sample images, and a blank model constructed based on a preset full convolution network and connection time sequence classification is trained by the sample training set to obtain a trained license plate recognition model; the system comprises a preset full convolution network, a data processing unit and a data processing unit, wherein the preset full convolution network is used for processing a sample image to obtain a corresponding two-dimensional characteristic map, segmenting the two-dimensional characteristic map into an upper one-dimensional characteristic map and a lower one-dimensional characteristic map, and splicing the two one-dimensional characteristic maps in a serial mode to obtain a spliced characteristic map; and the connection time sequence classification is used for carrying out optimization training on the blank model based on the characteristic splicing diagram. According to the embodiment of the application, the trained license plate recognition model has fewer network parameters, supports double-layer license plate recognition with characters being lengthened, can ensure fast and efficient recognition rate, effectively avoids the problems of character segmentation, complex process, low robustness and the like of the traditional double-layer license plate recognition method, and can be more suitable for various double-layer license plate recognition in complex environments.
In some specific embodiments, the data obtaining interface 11 specifically includes:
the positioning unit is used for acquiring the position and the size of a license plate in the license plate image to obtain license plate information in the license plate image, and performing external expansion processing on the left and right boundaries of the license plate based on the license plate information;
the setting unit is used for setting the size of the license plate subjected to the external expansion processing to be a preset size so as to obtain a sample image; and the preset size is the size of an input image of the license plate recognition model.
In some embodiments, the trainer 12 is specifically configured to sequentially input the sample images in the training set into one of the convolutional layers, one of the batch normalization layer + the active layer structure, one of the pooling layers, two of the Fire modules + the batch normalization layer + the active layer structure, one of the pooling layers, five of the Fire modules + the batch normalization layer + the active layer structure, one of the pooling layers, and one of the convolutional layers + the active layer structure to be processed, so as to obtain the two-dimensional feature map of the target license plate image.
Referring to fig. 6, an embodiment of the present application further discloses a license plate recognition apparatus, which includes, based on the license plate recognition model:
the acquisition module 21 is configured to acquire a license plate image to be recognized, and pre-process the license plate image to be recognized to obtain a target license plate image; the license plate in the license plate image to be recognized is a double-layer license plate;
and the recognition module 22 is configured to input the target license plate image to the license plate recognition model, so that the license plate recognition model recognizes the target license plate image.
Therefore, the license plate image to be recognized is obtained first, and the license plate image to be recognized is preprocessed to obtain the target license plate image, wherein the license plate in the license plate image to be recognized is a double-layer license plate. And then inputting the target license plate image into the license plate recognition model so that the license plate recognition model can recognize the target license plate image. According to the embodiment of the application, the license plate recognition model based on the connection time sequence classification is used for carrying out end-to-end recognition, so that the robustness for recognizing various double-layer license plates is high, and the recognition efficiency is improved.
In some embodiments, the license plate recognition device further includes:
and the decoding module is used for decoding the recognition result output by the license plate recognition model by using a greedy algorithm so as to obtain the license plate information of the license plate image to be recognized.
Further, the embodiment of the application also provides electronic equipment. FIG. 7 is a block diagram illustrating an electronic device 20 according to an exemplary embodiment, and the contents of the diagram should not be construed as limiting the scope of use of the present application in any way.
Fig. 7 is a schematic structural diagram of an electronic device 20 according to an embodiment of the present disclosure. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. The memory 22 is configured to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the license plate recognition model generation method and the related steps in the license plate recognition method disclosed in any of the foregoing embodiments. In addition, the electronic device 20 in the present embodiment may be specifically a portable computer.
In this embodiment, the power supply 23 is configured to provide a working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and a communication protocol followed by the communication interface is any communication protocol applicable to the technical solution of the present application, and is not specifically limited herein; the input/output interface 25 is configured to obtain external input data or output data to the outside, and a specific interface type thereof may be selected according to specific application requirements, which is not specifically limited herein.
In addition, the storage 22 is used as a carrier for resource storage, and may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon may include an operating system 221, a computer program 222, data 223, etc., and the storage may be a transient storage or a permanent storage.
The operating system 221 is used for managing and controlling each hardware device and the computer program 222 on the electronic device 20, so as to realize the operation and processing of the mass data 223 in the memory 22 by the processor 21, and may be Windows Server, Netware, Unix, Linux, and the like. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the license plate recognition model generation method and the license plate recognition method performed by the electronic device 20 disclosed in any of the foregoing embodiments. The data 223 may include dual-layer license plate image data collected by the electronic device 20.
Further, an embodiment of the present application further discloses a storage medium, in which a computer program is stored, and when the computer program is loaded and executed by a processor, the license plate recognition model generation method and the license plate recognition method disclosed in any of the foregoing embodiments are implemented.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The license plate recognition model generation method, the license plate recognition device, the license plate recognition equipment and the storage medium provided by the invention are introduced in detail, a specific example is applied in the text to explain the principle and the implementation mode of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A license plate recognition model generation method is characterized by comprising the following steps:
acquiring a license plate image, and preprocessing the license plate image to obtain a sample image; wherein the license plate in the license plate image is a double-layer license plate;
constructing a sample training set based on the sample images, and training a blank model constructed based on a preset full convolution network and a connection time sequence classification by using the sample training set to obtain a trained license plate recognition model; the preset full convolution network is used for processing the sample images in the training set to obtain corresponding two-dimensional feature maps, segmenting the two-dimensional feature maps into an upper one-dimensional feature map and a lower one-dimensional feature map, and splicing the two one-dimensional feature maps in a serial mode to obtain spliced feature maps corresponding to the two-dimensional feature maps; and the connection time sequence classification is used for carrying out optimization training on the blank model based on the characteristic splicing diagram.
2. The license plate recognition model generation method of claim 1, wherein the preset full convolution network comprises a convolution layer, a batch normalization layer, an activation layer, a pooling layer, and a Fire module.
3. The license plate recognition model generation method of claim 2, wherein the obtaining of the corresponding two-dimensional feature map by processing the sample images in the training set by the preset full convolution network comprises:
and sequentially inputting the sample images in the training set into one convolution layer, one batch of normalization layers and the activation layer structure, one pooling layer, two Fire modules and batch of normalization layers and the activation layer structure, one pooling layer, five Fire modules and batch of normalization layers and activation layer structures, one pooling layer and one convolution layer and activation layer structure for processing to obtain a corresponding two-dimensional feature map.
4. The method for generating the license plate recognition model according to claim 1, wherein the preprocessing the license plate image to obtain a sample image comprises:
acquiring the position and the size of a license plate in the license plate image to obtain license plate information in the license plate image, and performing external expansion processing on the left and right boundaries of the license plate based on the license plate information;
setting the size of the license plate subjected to the external expansion processing as a preset size to obtain a sample image; and the preset size is the size of an input image of the license plate recognition model.
5. A license plate recognition method based on the license plate recognition model of any one of claims 1 to 4, comprising:
acquiring a license plate image to be recognized, and preprocessing the license plate image to be recognized to obtain a target license plate image; the license plate in the license plate image to be recognized is a double-layer license plate;
and inputting the target license plate image into the license plate recognition model so that the license plate recognition model can recognize the target license plate image.
6. The license plate recognition method of claim 5, wherein the inputting the target license plate image into the license plate recognition model for the license plate recognition model to recognize the target license plate image further comprises:
and decoding the recognition result output by the license plate recognition model by using a greedy algorithm to obtain the license plate information of the license plate image to be recognized.
7. A license plate recognition model generation apparatus, characterized by comprising:
the data acquisition interface is used for acquiring a license plate image and preprocessing the license plate image to obtain a sample image; wherein the license plate in the license plate image is a double-layer license plate;
the training device is used for constructing a sample training set based on the sample images and training a blank model constructed based on a preset full convolution network and connection time sequence classification by using the sample training set so as to obtain a trained license plate recognition model; the preset full convolution network is used for processing the sample images in the training set to obtain corresponding two-dimensional feature maps, segmenting the two-dimensional feature maps into an upper one-dimensional feature map and a lower one-dimensional feature map, and splicing the two one-dimensional feature maps in a serial mode to obtain spliced feature maps corresponding to the two-dimensional feature maps; and the connection time sequence classification is used for carrying out optimization training on the blank model based on the characteristic splicing diagram.
8. A license plate recognition apparatus based on the license plate recognition model according to any one of claims 1 to 7, comprising:
the system comprises an acquisition module, a storage module and a display module, wherein the acquisition module is used for acquiring a license plate image to be recognized and preprocessing the license plate image to be recognized to obtain a target license plate image; the license plate in the license plate image to be recognized is a double-layer license plate;
and the recognition module is used for inputting the target license plate image into the license plate recognition model so that the license plate recognition model can recognize the target license plate image.
9. An electronic device, comprising a processor and a memory; wherein the memory is for storing a computer program that is loaded and executed by the processor to implement the method of any of claims 1 to 6.
10. A computer-readable storage medium storing computer-executable instructions which, when loaded and executed by a processor, implement the method of any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110467812.7A CN113159204A (en) | 2021-04-28 | 2021-04-28 | License plate recognition model generation method, license plate recognition method and related components |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110467812.7A CN113159204A (en) | 2021-04-28 | 2021-04-28 | License plate recognition model generation method, license plate recognition method and related components |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113159204A true CN113159204A (en) | 2021-07-23 |
Family
ID=76872070
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110467812.7A Pending CN113159204A (en) | 2021-04-28 | 2021-04-28 | License plate recognition model generation method, license plate recognition method and related components |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113159204A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113642586A (en) * | 2021-07-29 | 2021-11-12 | 深圳市捷顺科技实业股份有限公司 | License plate type identification method and device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106971185A (en) * | 2017-03-27 | 2017-07-21 | 深圳市丰巨泰科电子有限公司 | A kind of license plate locating method and device based on full convolutional network |
| CN109086722A (en) * | 2018-08-06 | 2018-12-25 | 汉王科技股份有限公司 | Mix licence plate recognition method, device, electronic equipment |
| CN110400275A (en) * | 2019-07-22 | 2019-11-01 | 中电健康云科技有限公司 | One kind being based on full convolutional neural networks and the pyramidal color calibration method of feature |
| CN110969164A (en) * | 2019-12-20 | 2020-04-07 | 湖南千视通信息科技有限公司 | Low-illumination imaging license plate recognition method and device based on deep learning end-to-end |
| CN111191663A (en) * | 2019-12-31 | 2020-05-22 | 深圳云天励飞技术有限公司 | License plate number recognition method and device, electronic equipment and storage medium |
| US20200193232A1 (en) * | 2016-12-23 | 2020-06-18 | Shenzhen Institute Of Advanced Technology | License plate recognition method, device thereof, and user equipment |
| CN111767927A (en) * | 2020-06-03 | 2020-10-13 | 上海交通大学 | Lightweight license plate recognition method and system based on full convolution network |
-
2021
- 2021-04-28 CN CN202110467812.7A patent/CN113159204A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200193232A1 (en) * | 2016-12-23 | 2020-06-18 | Shenzhen Institute Of Advanced Technology | License plate recognition method, device thereof, and user equipment |
| CN106971185A (en) * | 2017-03-27 | 2017-07-21 | 深圳市丰巨泰科电子有限公司 | A kind of license plate locating method and device based on full convolutional network |
| CN109086722A (en) * | 2018-08-06 | 2018-12-25 | 汉王科技股份有限公司 | Mix licence plate recognition method, device, electronic equipment |
| CN110400275A (en) * | 2019-07-22 | 2019-11-01 | 中电健康云科技有限公司 | One kind being based on full convolutional neural networks and the pyramidal color calibration method of feature |
| CN110969164A (en) * | 2019-12-20 | 2020-04-07 | 湖南千视通信息科技有限公司 | Low-illumination imaging license plate recognition method and device based on deep learning end-to-end |
| CN111191663A (en) * | 2019-12-31 | 2020-05-22 | 深圳云天励飞技术有限公司 | License plate number recognition method and device, electronic equipment and storage medium |
| CN111767927A (en) * | 2020-06-03 | 2020-10-13 | 上海交通大学 | Lightweight license plate recognition method and system based on full convolution network |
Non-Patent Citations (2)
| Title |
|---|
| 史建伟: "基于深度学习的车牌识别系统的设计与实现", 中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑 * |
| 周晓君等: "基于深度学习的车牌识别技术的研究与实现", 《中国优秀硕士学位论文全文数据库 工程科技Ⅱ辑》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113642586A (en) * | 2021-07-29 | 2021-11-12 | 深圳市捷顺科技实业股份有限公司 | License plate type identification method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021093435A1 (en) | Semantic segmentation network structure generation method and apparatus, device, and storage medium | |
| CN111027563A (en) | Text detection method, device and recognition system | |
| US20190340746A1 (en) | Stationary object detecting method, apparatus and electronic device | |
| JP7284352B2 (en) | Simultaneous real-time object detection and semantic segmentation system and method and non-transitory computer-readable medium | |
| CN110874618B (en) | OCR template learning method and device based on small sample, electronic equipment and medium | |
| CN113704531A (en) | Image processing method, image processing device, electronic equipment and computer readable storage medium | |
| CN111738036B (en) | Image processing method, device, equipment and storage medium | |
| CN112287896A (en) | Unmanned aerial vehicle aerial image target detection method and system based on deep learning | |
| CN112052855A (en) | License plate recognition method and device based on deep learning | |
| JP2023527615A (en) | Target object detection model training method, target object detection method, device, electronic device, storage medium and computer program | |
| CN113239753A (en) | Improved traffic sign detection and identification method based on YOLOv4 | |
| CN114820679A (en) | Image annotation method and device, electronic equipment and storage medium | |
| CN113159204A (en) | License plate recognition model generation method, license plate recognition method and related components | |
| CN114693963A (en) | Recognition model training and recognition method and device based on electric power data feature extraction | |
| CN113902793A (en) | End-to-end building height prediction method and system based on single vision remote sensing image and electronic equipment | |
| CN116992880A (en) | Building name identification method, device, electronic equipment and storage medium | |
| CN115588130A (en) | Cross-domain YOLO detection method based on domain self-adaptation | |
| CN113808142A (en) | Ground identifier identification method and device and electronic equipment | |
| CN112749293A (en) | Image classification method and device and storage medium | |
| CN113239931A (en) | Logistics station license plate recognition method | |
| CN111435537A (en) | Model training method and device and pose optimization method and device based on splicing map | |
| CN112163492B (en) | Long-time cross-scene optimization traffic object detection method, system and medium | |
| CN115063770B (en) | Lane line detection method, model training method, device, equipment and storage medium | |
| CN112052773B (en) | Unmanned aerial vehicle traffic dispersion intelligent broadcasting method and device based on image sensing | |
| CN111881778B (en) | Method, apparatus, device and computer readable medium for text detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |