CN114299486B - Embedded terminal real-time efficient license plate recognition method, device and medium - Google Patents
Embedded terminal real-time efficient license plate recognition method, device and medium Download PDFInfo
- Publication number
- CN114299486B CN114299486B CN202111647117.5A CN202111647117A CN114299486B CN 114299486 B CN114299486 B CN 114299486B CN 202111647117 A CN202111647117 A CN 202111647117A CN 114299486 B CN114299486 B CN 114299486B
- Authority
- CN
- China
- Prior art keywords
- residual block
- license plate
- layer
- shallow
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 35
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 claims abstract description 158
- 238000004364 calculation method Methods 0.000 claims abstract description 29
- 238000012549 training Methods 0.000 claims description 19
- 238000013527 convolutional neural network Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 10
- 230000004913 activation Effects 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 9
- 230000015654 memory Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 2
- 238000005286 illumination Methods 0.000 abstract description 5
- 238000012795 verification Methods 0.000 abstract description 3
- 238000004422 calculation algorithm Methods 0.000 description 11
- 238000012360 testing method Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 7
- 238000013135 deep learning Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 230000011218 segmentation Effects 0.000 description 4
- 238000001514 detection method Methods 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000011176 pooling Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000012821 model calculation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
Landscapes
- Character Discrimination (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method, equipment and medium for identifying license plates at an embedded end in real time and high efficiency, wherein the method comprises the following steps: reading a license plate image to be identified, transmitting the license plate image to be identified to a license plate identification network model, calculating layer by layer, and outputting a license plate identification result; the license plate recognition network model sequentially comprises a first residual block, a second residual block, a third residual block, a fourth residual block, a fifth residual block, a sixth residual block, a seventh residual block, an eighth residual block and an output layer; the first residual block is used for performing primary convolution calculation on the license plate image to be identified to obtain a first shallow license plate feature; the second residual block is used for carrying out convolution calculation on the first shallow license plate feature to obtain a second shallow license plate feature and the like; the embedded terminal real-time and efficient license plate recognition method provided by the invention has high real-time performance and is not easily influenced by illumination, license plates with various character numbers can be recognized, and the license plates can be seamlessly adapted to the embedded terminal through verification.
Description
Technical Field
The invention relates to the technical field of license plate recognition, in particular to a real-time and efficient license plate recognition method, device and medium of an embedded terminal.
Background
The license plate is used as one of the external significant identity information of the automobile, and the information such as the driving path, the type and the driver of the automobile can be obtained through the license plate. The license plate recognition application range is wider, and the license plate recognition method is widely used in the aspects of intelligent parking, expressway vehicle monitoring, city number limiting and the like.
At present, the adopted traditional license plate recognition algorithm has higher requirements on license plate picture quality, and more factors influencing license plate picture quality, such as light, distance and angle between a license plate and image acquisition equipment, and the like, so that the traditional license plate recognition algorithm cannot meet the recognition requirements of complex light, multi-color license plates, high recognition rate and high speed.
It can be understood that the traditional license plate recognition algorithm needs to separate characters in a license plate region, the segmentation algorithm comprises a character segmentation algorithm based on a template, a clustering algorithm character segmentation and the like, the method is seriously influenced by illumination, the subsequent recognition error is directly caused when the segmentation is inaccurate, and the accuracy rate of recognizing the license plate is low; then, machine learning algorithms svm, knn and other algorithms are used for identifying the segmented characters, and the character identification is used as the most important link in license plate identification, so that the accuracy rate of the character identification directly influences the accurate identification of the license plate; however, the machine learning algorithm is used for recognizing characters, so that the recognition rate is greatly improved.
With the appearance of deep learning, end-to-End recognition methods, such as crnn (full name: an End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition), lprnet (License Plate Recognition via Deep Neural Networks), and the like, appear, although the accuracy of license plate recognition based on deep learning is greatly improved, the model calculation complexity is larger, and phenomena that operators of partial layers are not supported when transplanting on An embedded End, such as lstm (Long short-term memory) layers in crnn and 3D pooling layers in lprnet are not supported by a deep learning reasoning frame rnn on a processor rk3399, the difficulty of realizing the layers by codes is large and the real-time performance is difficult to ensure, so that a deep learning algorithm with higher accuracy cannot be adapted to the embedded End, and the accuracy and the real-time performance of An embedded algorithm applied to complex light and multi-color license plate recognition are low.
Although the Chinese patent with the application number of CN201810736742.9 discloses a license plate recognition method based on a reduced ResNet residual network, the structure of the license plate recognition method is unchanged, the output of the full-connection layer of the last layer is 7 (seven characters of a blue plate), 7 neurons respectively correspond to each character in the blue plate license plate, once the model is trained well, only blue plates or yellow plates (7 characters) can be recognized, namely, the result obtained by recognizing green plates (8 characters) is still 7 characters, so that the license plate recognition method can only recognize license plates with fixed characters, cannot simultaneously recognize the blue plates, the yellow plates and the green plates, and cannot recognize the green plates with 8 characters.
Therefore, the accuracy and the real-time performance of the embedded end car plate identification still need to be further improved, and the study on the deep learning model suitable for the embedded end car plate identification is necessary.
In order to solve the above problems, an ideal technical solution is always sought.
Disclosure of Invention
The invention aims at overcoming the defects of the prior art, and provides a license plate identification method, device and medium with real-time and high efficiency at an embedded end.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
the first aspect of the invention provides a license plate recognition method with real-time and high efficiency at an embedded end, which comprises the following steps:
reading a license plate image to be identified, transmitting the license plate image to be identified to a license plate identification network model, calculating layer by layer, and outputting a license plate identification result;
the license plate recognition network model sequentially comprises a first residual block, a second residual block, a third residual block, a fourth residual block, a fifth residual block, a sixth residual block, a seventh residual block, an eighth residual block and an output layer; the first residual block is used for performing primary convolution calculation on the license plate image to be identified to obtain a first shallow license plate feature; the second residual block is used for carrying out convolution calculation on the first shallow license plate feature to obtain a second shallow license plate feature; the third residual block is used for carrying out convolution calculation on the second shallow license plate feature to obtain a third shallow license plate feature; the fourth residual block is used for carrying out convolution calculation on the third shallow license plate feature to obtain a fourth shallow license plate feature; the fifth residual block is used for carrying out convolution calculation on the fourth shallow license plate feature to obtain a fifth shallow license plate feature; the sixth residual block is used for carrying out convolution calculation on the fifth shallow license plate feature to obtain a first deep license plate feature; the seventh residual block is used for carrying out convolution calculation on the first deep license plate feature to obtain a second deep license plate feature; the eighth residual block is used for carrying out convolution calculation on the second deep license plate feature to obtain a third deep license plate feature; the output layer is used for carrying out convolution calculation on the third deep license plate feature to obtain a license plate recognition result;
the first residual block, the second residual block, the third residual block, the fourth residual block and the fifth residual block comprise a Conv2d layer, a BatchNorm2d layer, an activation function ReLU layer and a Basicblock layer of a convolutional neural network, the channel numbers of the first residual block, the second residual block, the third residual block, the fourth residual block and the fifth residual block are 64, 128, 256 and 512 in sequence, and the width parameters of the characteristic images output by the first residual block, the second residual block, the third residual block, the fourth residual block and the fifth residual block are 88, 43, 22 and 22 in sequence;
the sixth residual block and the seventh residual block comprise a convolutional neural network Conv2d layer, a BatchNorm2d layer, an activation function ReLU layer and a Dropout layer, the number of channels of the sixth residual block and the seventh residual block is 256 and 128 in sequence, and the width parameters of the output characteristic images of the sixth residual block and the seventh residual block are 20 and 18 in sequence;
the eighth residual block comprises a convolutional neural network Conv2d layer, a BatchNorm2d layer and an activation function ReLU layer, the number of channels of the eighth residual block is 66, and the width parameter of the output characteristic image of the eighth residual block is 18;
the output layer comprises a Conv2d layer of a convolutional neural network, the number of channels of the output layer is 66, and the step length of an output sequence of the output layer is 18.
The invention provides embedded-end real-time efficient license plate recognition equipment, which comprises a memory, a processor and embedded-end real-time efficient license plate recognition programs which are stored in the memory and can run on the processor, wherein when the embedded-end real-time efficient license plate recognition programs are executed by the processor, the steps of the embedded-end real-time efficient license plate recognition method are realized.
A third aspect of the present invention provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the embedded end real-time efficient license plate recognition method as described above.
The beneficial effects of the invention are as follows:
1) The license plate recognition network model training recognition license plate provided by the invention has feasibility, greatly improves recognition effect, has high instantaneity and is not easily influenced by illumination, the license plates with various character numbers can be recognized, can be seamlessly matched to an embedded terminal after verification, further improves the accuracy and instantaneity of embedded terminal license plate recognition in the industry, and has higher commercial value;
2) According to the invention, the license plate recognition network model can simultaneously recognize license plates with different character numbers through the sixth residual block, the seventh residual block, the eighth residual block, the output layer and the CTC decoding.
Drawings
FIGS. 1 and 2 are schematic diagrams of license plate recognition network models according to the present invention;
FIG. 3 is a flow chart of detection of an embodiment of the present invention;
FIG. 4 is a schematic diagram of a loss curve in the training process of the license plate recognition network model of the invention;
FIG. 5 is a graph of license plate recognition effect of a part of pictures on a test set according to the invention;
FIG. 6 is a schematic diagram of the result of the reasoning speed test of the license plate recognition network model of the invention;
fig. 7 is a diagram showing the recognition effect of transplanting the license plate recognition method of the present invention into an embedded system.
Detailed Description
The technical scheme of the invention is further described in detail through the following specific embodiments.
Example 1
As shown in fig. 1 to 3, an embedded end real-time efficient license plate recognition method comprises the following steps:
reading a license plate image to be identified, transmitting the license plate image to be identified to a license plate identification network model, calculating layer by layer, and outputting a license plate identification result;
the license plate recognition network model sequentially comprises a first residual block, a second residual block, a third residual block, a fourth residual block, a fifth residual block, a sixth residual block, a seventh residual block, an eighth residual block and an output layer; the first residual block is used for performing primary convolution calculation on the license plate image to be identified to obtain a first shallow license plate feature; the second residual block is used for carrying out convolution calculation on the first shallow license plate feature to obtain a second shallow license plate feature; the third residual block is used for carrying out convolution calculation on the second shallow license plate feature to obtain a third shallow license plate feature; the fourth residual block is used for carrying out convolution calculation on the third shallow license plate feature to obtain a fourth shallow license plate feature; the fifth residual block is used for carrying out convolution calculation on the fourth shallow license plate feature to obtain a fifth shallow license plate feature; the sixth residual block is used for carrying out convolution calculation on the fifth shallow license plate feature to obtain a first deep license plate feature; the seventh residual block is used for carrying out convolution calculation on the first deep license plate feature to obtain a second deep license plate feature; the eighth residual block is used for carrying out convolution calculation on the second deep license plate feature to obtain a third deep license plate feature; the output layer is used for carrying out convolution calculation on the third deep license plate feature to obtain a license plate recognition result;
the first residual block, the second residual block, the third residual block, the fourth residual block and the fifth residual block comprise a Conv2d layer, a BatchNorm2d layer, an activation function ReLU layer and a Basicblock layer of a convolutional neural network, the channel numbers of the first residual block, the second residual block, the third residual block, the fourth residual block and the fifth residual block are 64, 128, 256 and 512 in sequence, and the width parameters of the characteristic images output by the first residual block, the second residual block, the third residual block, the fourth residual block and the fifth residual block are 88, 43, 22 and 22 in sequence;
the sixth residual block and the seventh residual block comprise a convolutional neural network Conv2d layer, a BatchNorm2d layer, an activation function ReLU layer and a Dropout layer, the number of channels of the sixth residual block and the seventh residual block is 256 and 128 in sequence, and the width parameters of the output characteristic images of the sixth residual block and the seventh residual block are 20 and 18 in sequence;
the eighth residual block comprises a convolutional neural network Conv2d layer, a BatchNorm2d layer and an activation function ReLU layer, the number of channels of the eighth residual block is 66, and the width parameter of the output characteristic image of the eighth residual block is 18;
the output layer comprises a Conv2d layer of a convolutional neural network, the number of channels of the output layer is 66, and the step length of an output sequence of the output layer is 18.
Specifically, the size of the license plate image to be identified is configured to be 24943 so as to adapt to the input training and reasoning of the license plate image; wherein 24 represents the height of the license plate image to be identified, 94 represents the width of the license plate image to be identified, 3 represents the number of picture channels (BGR), as shown in fig. 1, after the license plate image to be identified passes through the first residual block, the number of picture channels (BGR) is stripped, so as to increase the detection speed. The output of the license plate recognition network model is configured to be 1866 dimensions, where 18 represents the step size of the output sequence and 66 represents the sum of the number of license plate provinces, english letters (excluding I and O), and numbers.
As shown in fig. 2, from the fifth residual block, the sixth residual block, the seventh residual block, the eighth residual block and the output layer, the height dimension of the feature map of each residual block is sequentially reduced from 4 to 2 to 1, and the width dimension of the feature map of each residual block is sequentially reduced from 22 to 20 to 18, so that the output layer outputs 1866 dimension data, and a foundation is laid for identifying license plates with different character numbers at the same time.
Further, the output layer outputs 1866 dimension data, and after each 66 dimension data passes through softmax, the position of the maximum value is recorded; sequentially storing the maximum value in all 66-dimensional data into the same one-dimensional array to obtain a first target array with the length of 18;
performing de-duplication processing on adjacent and same numbers in the first target array, and removing negative sample numbers in the first target array to obtain a second target array;
reading a preset license plate information comparison table, wherein the license plate information comparison table is { Beijing: 0, shanghai: 1, jin: 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,0, 31,1, 32,2, 3, 34,4, 35,5, 36,6, 37,7, 38,8, 39,9, 40, A, 41, B, 42, C, 43, D, 44, E, 45, F, 46, G, 47, H, 48, J, 49, K, 50, L, 51, M, 52, N, 53, P, 54, Q, 55, R, 57, S58, U, V, 62, V, 60, X, Y, 65, Z-65, a negative sign of the following the steps of 3, 8, 9, 35,5, 36,6, 37,7, 38,8, 39, 40, A, 41, B, 43, D, 44, E, 45, F, 46, G, 47, H, 48, J, 49, K, 51, M, 52, N, 53, P, 54, Q55, R57, S58, V, 62, V, 60, Z, 65, Z, Y, Z, Y, Z, L, Z, X, L, C, L, J, L, 4,4,;
and obtaining the license plate number corresponding to the license plate image to be identified according to the mapping relation between the second target array and the license plate information comparison table.
It can be understood that the output layer (Conv 2d layer of convolutional neural network) outputs 1866 dimension data, that is, 18 data 66, and each dimension 66 passes through softmax in the test reasoning stage, and records the position of the maximum value, that is, 18 positions of the maximum value, so as to obtain an array with the length of 18; for example, the first target array is [65,22,22,65,41,65,44,65,31,65,31,65,39,65,38,65,32,65]; only one adjacent and same number is reserved to obtain [65,22,65,41,65,44,65,31,65,31,65,39,65,38,65,32,65], then the negative sample number 65 is deleted to obtain a second target array [22,41,44,31,31,39,38,32], and a corresponding position is found in the license plate information comparison table; 22 in [22,41,44,31,31,39,38,32] corresponds to "Chuan", 41 corresponds to "A",44 corresponds to "D",31 corresponds to "0",39 corresponds to "8",38 corresponds to "7",32 corresponds to "1", and finally the license plate number is obtained: chuan AD00871;
it should be noted that the existence of adjacent and identical numbers for the second destination array [22,41,44,31,31,39,38,32] is permissible.
It should be noted that, through a large number of training test experiments, the license plate recognition network model provided by the invention has the advantages of feasibility, greatly improved recognition effect, high real-time performance, difficult influence of illumination, capability of recognizing multi-color license plates, and capability of performing conventional operator operation on all the license plates in the model, so that the license plates can be seamlessly adapted to an embedded terminal through verification, the accuracy and the real-time performance of recognition of the embedded terminal license plates are further improved in the industry, and the license plate recognition network model has higher commercial value.
Further, the convolution kernel size of the Conv2d layer of the convolution neural network of the fifth residual block is 33, and the convolution step length is 11; the convolution kernel size of a Conv2d layer of the convolution neural network of the sixth residual block is 3 multiplied by 3, and the convolution step length is 1 multiplied by 1; the convolution kernel size of the Conv2d layer of the convolutional neural network of the seven residual blocks is 2 multiplied by 3, and the convolution step length is 1 multiplied by 1; the convolution kernel size of a Conv2d layer of the convolution neural network of the eighth residual block is 1 multiplied by 1, and the convolution step length is 1 multiplied by 1; the convolution kernel size of the Conv2d layer of the convolutional neural network of the output layer is 1 multiplied by 1, and the convolution step length is 1 multiplied by 1.
It can be understood that the invention also specially designs the convolution kernel sizes of the sixth residual block, the seventh residual block and the eighth residual block comprising a Conv2d layer of the convolution neural network, and the characteristic map output by the corresponding residual block is adapted to the final dimension by changing the convolution kernel size (theoretically, the larger the convolution kernel is, the larger the step size is), so that the license plate with 7 characters or 8 characters can be identified simultaneously; therefore, the invention replaces the adaptive mean value pooling layer and all subsequent layers in the network structure of the resnet18 model with the sixth residual block, the seventh residual block, the eighth residual block and the output layer, so that the license plate recognition network model can recognize license plates with different numbers of characters at the same time.
The number of channels of the sixth residual block, the seventh residual block and the eighth residual block of the license plate recognition network model provided by the invention is sequentially halved, so that the dimension disaster of high-level semantic features is reduced, the final dimension is reduced to 66, and license plate characters on each time sequence step are predicted in a seamless connection manner on the dimension.
Further, the loss function used by the output layer is a CTC loss function, and the formula is as follows:
wherein D represents the entire training set, x i Representing the ith training sample picture in the training set, I i Representing a real label corresponding to an ith training sample picture in a training set, p (I i y i ) Representing an input as an ith training sample picture x i The output of the time network is y i y i Decoded by CTC to be the real label I i Is a probability of (2).
The invention is trained on a self-built picture data set, and the total number of pictures is 110000, including blue license plates, green license plates and yellow license plates. Wherein 100000 pictures are randomly divided for training, 10000 pictures are used for evaluating and testing, and a loss curve in the model training process is shown in fig. 4; the loss curve can show that the convergence rate is very high when the license plate recognition network model provided by the invention is used for training license plate recognition, and the model tends to be stable after 20 iterations. The license plate recognition effect of the partial pictures (blue, green and yellow in sequence from left to right) in the test set is shown in fig. 5, when the height and width of the license plate image to be recognized is 2494, the time for detecting and recognizing one license plate is about 3 milliseconds, fps is about 333, and the reasoning speed test result of the network model is shown in fig. 6. As can be seen from the results, whether the license plate recognition network model is a seven-character blue plate, a seven-character yellow plate or an 8-character green plate, the license plate recognition network model can be rapidly and accurately recognized.
Through a large number of experiments, the license plate recognition network model provided by the invention has a good license plate recognition test effect, and the accuracy rate on a test set is 99.35%; the reasoning speed is far and ultra-real time, the method has high commercial application value, is particularly suitable for embedded terminal transplanting, and particularly greatly improves the accuracy of license plate recognition of monitoring scenes.
The embedded terminal real-time and efficient license plate recognition method provided by the invention is transplanted to an embedded system for recognition effect, as shown in figure 7. Therefore, the embedded end real-time and efficient license plate recognition method provided by the invention has strong robustness, is not influenced by illumination and bad weather, and can be used for recognizing license plates with various character numbers.
Example 2
The embodiment provides an embedded-end real-time efficient license plate recognition device, which comprises a memory, a processor and an embedded-end real-time efficient license plate recognition program stored on the memory and capable of running on the processor, wherein the embedded-end real-time efficient license plate recognition program realizes the steps of the embedded-end real-time efficient license plate recognition method as in the embodiment 1 when being executed by the processor.
The present embodiment also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of embedded end real-time efficient license plate recognition as in embodiment 1.
In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the above-described division of modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated modules described above, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiment, or may be implemented by instructing related hardware by a computer program, where the computer program may be stored in a computer readable storage medium, and the computer program may implement the steps of each method embodiment described above when executed by a processor. The computer program comprises computer program code, and the computer program code can be in a source code form, an object code form, an executable file or some intermediate form and the like.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention and not for limiting the same; while the invention has been described in detail with reference to the preferred embodiments, those skilled in the art will appreciate that: modifications may be made to the specific embodiments of the present invention or equivalents may be substituted for part of the technical features thereof; without departing from the spirit of the invention, it is intended to cover the scope of the invention as claimed.
Claims (5)
1. The embedded end real-time efficient license plate recognition method is characterized by comprising the following steps of:
reading a license plate image to be identified, transmitting the license plate image to be identified to a license plate identification network model, calculating layer by layer, and outputting a license plate identification result;
the license plate recognition network model sequentially comprises a first residual block, a second residual block, a third residual block, a fourth residual block, a fifth residual block, a sixth residual block, a seventh residual block, an eighth residual block and an output layer; the first residual block is used for performing primary convolution calculation on the license plate image to be identified to obtain a first shallow license plate feature; the second residual block is used for carrying out convolution calculation on the first shallow license plate feature to obtain a second shallow license plate feature; the third residual block is used for carrying out convolution calculation on the second shallow license plate feature to obtain a third shallow license plate feature; the fourth residual block is used for carrying out convolution calculation on the third shallow license plate feature to obtain a fourth shallow license plate feature; the fifth residual block is used for carrying out convolution calculation on the fourth shallow license plate feature to obtain a fifth shallow license plate feature; the sixth residual block is used for carrying out convolution calculation on the fifth shallow license plate feature to obtain a first deep license plate feature; the seventh residual block is used for carrying out convolution calculation on the first deep license plate feature to obtain a second deep license plate feature; the eighth residual block is used for carrying out convolution calculation on the second deep license plate feature to obtain a third deep license plate feature; the output layer is used for carrying out convolution calculation on the third deep license plate feature to obtain a license plate recognition result;
the first residual block, the second residual block, the third residual block, the fourth residual block and the fifth residual block comprise a Conv2d layer, a BatchNorm2d layer, an activation function ReLU layer and a Basicblock layer of a convolutional neural network, the channel numbers of the first residual block, the second residual block, the third residual block, the fourth residual block and the fifth residual block are 64, 128, 256 and 512 in sequence, and the width parameters of the characteristic images output by the first residual block, the second residual block, the third residual block, the fourth residual block and the fifth residual block are 88, 43, 22 and 22 in sequence;
the sixth residual block and the seventh residual block comprise a convolutional neural network Conv2d layer, a BatchNorm2d layer, an activation function ReLU layer and a Dropout layer, the number of channels of the sixth residual block and the seventh residual block is 256 and 128 in sequence, and the width parameters of the output characteristic images of the sixth residual block and the seventh residual block are 20 and 18 in sequence;
the eighth residual block comprises a convolutional neural network Conv2d layer, a BatchNorm2d layer and an activation function ReLU layer, the number of channels of the eighth residual block is 66, and the width parameter of the output characteristic image of the eighth residual block is 18;
the output layer comprises a Conv2d layer of a convolutional neural network, the number of channels of the output layer is 66, and the step length of an output sequence of the output layer is 18;
the loss function used by the output layer is a CTC loss function, and the formula is as follows:
wherein D represents the entire training set, x i Representing the ith training sample picture in the training set, I i Representing a real label corresponding to an ith training sample picture in a training set, p (I i y i ) Representing an input as an ith training sample picture x i The output of the time network is y i y i Decoded by CTC to be the real label I i Is a probability of (2).
2. The embedded-end real-time efficient license plate recognition method according to claim 1, wherein the method is characterized by comprising the following steps of: the convolution kernel size of a Conv2d layer of the convolution neural network of the fifth residual block is 3 multiplied by 3, and the convolution step length is 1 multiplied by 1; the convolution kernel size of a Conv2d layer of the convolution neural network of the sixth residual block is 3 multiplied by 3, and the convolution step length is 1 multiplied by 1; the convolution kernel size of the Conv2d layer of the convolutional neural network of the seven residual blocks is 2 multiplied by 3, and the convolution step length is 1 multiplied by 1; the convolution kernel size of a Conv2d layer of the convolution neural network of the eighth residual block is 1 multiplied by 1, and the convolution step length is 1 multiplied by 1; the convolution kernel size of the Conv2d layer of the convolutional neural network of the output layer is 1 multiplied by 1, and the convolution step length is 1 multiplied by 1.
3. The embedded-end real-time efficient license plate recognition method according to claim 1, wherein the method is characterized by comprising the following steps of: the output layer outputs 18 x 66 dimension data, and after each 66 dimension data passes through softmax, the position of the maximum value is recorded; sequentially storing the maximum value in all 66-dimensional data into the same one-dimensional array to obtain a first target array with the length of 18;
performing de-duplication processing on adjacent and same numbers in the first target array, and removing negative sample numbers in the first target array to obtain a second target array;
reading a preset license plate information comparison table, wherein the license plate information comparison table is { Beijing: 0, shanghai: 1, jin: 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30,0, 31,1, 32,2, 3, 34,4, 35,5, 36,6, 37,7, 38,8, 39,9, 40, A, 41, B, 42, C, 43, D, 44, E, 45, F, 46, G, 47, H, 48, J, 49, K, 50, L, 51, M, 52, N, 53, P, 54, Q, 55, R, 57, S58, U, V, 62, V, 60, X, Y, 65, Z-65, a negative sign of the following the steps of 3, 8, 9, 35,5, 36,6, 37,7, 38,8, 39, 40, A, 41, B, 43, D, 44, E, 45, F, 46, G, 47, H, 48, J, 49, K, 51, M, 52, N, 53, P, 54, Q55, R57, S58, V, 62, V, 60, Z, 65, Z, Y, Z, Y, Z, L, Z, X, L, C, L, J, L, 4,4,;
and obtaining the license plate number corresponding to the license plate image to be identified according to the mapping relation between the second target array and the license plate information comparison table.
4. An embedded end real-time efficient license plate recognition device is characterized in that: the method comprises a memory, a processor and an embedded end real-time efficient license plate recognition program which is stored in the memory and can run on the processor, wherein the embedded end real-time efficient license plate recognition program realizes the steps of the embedded end real-time efficient license plate recognition method according to any one of claims 1-3 when the embedded end real-time efficient license plate recognition program is executed by the processor.
5. A computer-readable storage medium having stored thereon a computer program, characterized by: the steps of the embedded end real-time efficient license plate recognition method according to any one of claims 1-3 are realized when the computer program is executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111647117.5A CN114299486B (en) | 2021-12-30 | 2021-12-30 | Embedded terminal real-time efficient license plate recognition method, device and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111647117.5A CN114299486B (en) | 2021-12-30 | 2021-12-30 | Embedded terminal real-time efficient license plate recognition method, device and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114299486A CN114299486A (en) | 2022-04-08 |
| CN114299486B true CN114299486B (en) | 2024-04-16 |
Family
ID=80972023
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111647117.5A Active CN114299486B (en) | 2021-12-30 | 2021-12-30 | Embedded terminal real-time efficient license plate recognition method, device and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114299486B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110414451A (en) * | 2019-07-31 | 2019-11-05 | 深圳市捷顺科技实业股份有限公司 | It is a kind of based on end-to-end licence plate recognition method, device, equipment and storage medium |
| CN110688880A (en) * | 2018-07-06 | 2020-01-14 | 山东华软金盾软件股份有限公司 | License plate identification method based on simplified ResNet residual error network |
| CN111444917A (en) * | 2020-03-30 | 2020-07-24 | 合肥京东方显示技术有限公司 | License plate character recognition method and device, electronic equipment and storage medium |
| CN112232351A (en) * | 2020-11-09 | 2021-01-15 | 浙江工业职业技术学院 | License plate recognition system based on deep neural network |
-
2021
- 2021-12-30 CN CN202111647117.5A patent/CN114299486B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110688880A (en) * | 2018-07-06 | 2020-01-14 | 山东华软金盾软件股份有限公司 | License plate identification method based on simplified ResNet residual error network |
| CN110414451A (en) * | 2019-07-31 | 2019-11-05 | 深圳市捷顺科技实业股份有限公司 | It is a kind of based on end-to-end licence plate recognition method, device, equipment and storage medium |
| CN111444917A (en) * | 2020-03-30 | 2020-07-24 | 合肥京东方显示技术有限公司 | License plate character recognition method and device, electronic equipment and storage medium |
| WO2021196873A1 (en) * | 2020-03-30 | 2021-10-07 | 京东方科技集团股份有限公司 | License plate character recognition method and apparatus, electronic device, and storage medium |
| CN112232351A (en) * | 2020-11-09 | 2021-01-15 | 浙江工业职业技术学院 | License plate recognition system based on deep neural network |
Non-Patent Citations (2)
| Title |
|---|
| 使用多任务级联卷积神经网络进行车牌照识别;胡从坤;黄东军;;企业技术开发;20190201(第02期);全文 * |
| 基于改进LeNet-5的车牌识别算法;张荣梅;张琦;陈彬;;科学技术与工程;20200428(第12期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114299486A (en) | 2022-04-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108388896B (en) | License plate identification method based on dynamic time sequence convolution neural network | |
| CN109753913B (en) | Multi-mode video semantic segmentation method with high calculation efficiency | |
| CN108491836B (en) | Method for integrally identifying Chinese text in natural scene image | |
| CN109840523B (en) | Urban rail train number identification method based on image processing | |
| CN111444917A (en) | License plate character recognition method and device, electronic equipment and storage medium | |
| CN111191611B (en) | Traffic sign label identification method based on deep learning | |
| CN111582339B (en) | Vehicle detection and recognition method based on deep learning | |
| CN110032952B (en) | Road boundary point detection method based on deep learning | |
| CN113128507B (en) | License plate recognition method and device, electronic equipment and storage medium | |
| CN111027539A (en) | License plate character segmentation method based on spatial position information | |
| CN112232351A (en) | License plate recognition system based on deep neural network | |
| CN112149535A (en) | Lane line detection method and device combining SegNet and U-Net | |
| CN113537119B (en) | Transmission line connecting part detection method based on improved Yolov4-tiny | |
| CN111444916A (en) | License plate positioning and identifying method and system under unconstrained condition | |
| CN111881914B (en) | License plate character segmentation method and system based on self-learning threshold | |
| CN116524725B (en) | Intelligent driving traffic sign image data identification system | |
| CN114299486B (en) | Embedded terminal real-time efficient license plate recognition method, device and medium | |
| CN112528994A (en) | Free-angle license plate detection method, license plate identification method and identification system | |
| CN114037996A (en) | License plate recognition model training method, recognition method, device, equipment and medium | |
| CN115171001A (en) | Method and system for detecting vehicle on enhanced thermal infrared image based on improved SSD | |
| CN114882469A (en) | Traffic sign detection method and system based on DL-SSD model | |
| CN113159153A (en) | License plate recognition method based on convolutional neural network | |
| CN113343817A (en) | Unmanned vehicle path detection method and device for target area and medium | |
| CN110826564A (en) | Small target semantic segmentation method and system in complex scene image | |
| CN115578246B (en) | Non-aligned visible light and infrared mode fusion target detection method based on style migration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |