CN112580643A - License plate recognition method and device based on deep learning and storage medium - Google Patents
License plate recognition method and device based on deep learning and storage medium Download PDFInfo
- Publication number
- CN112580643A CN112580643A CN202011428529.5A CN202011428529A CN112580643A CN 112580643 A CN112580643 A CN 112580643A CN 202011428529 A CN202011428529 A CN 202011428529A CN 112580643 A CN112580643 A CN 112580643A
- Authority
- CN
- China
- Prior art keywords
- license plate
- image
- character
- probability distribution
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000013135 deep learning Methods 0.000 title claims abstract description 28
- 230000011218 segmentation Effects 0.000 claims abstract description 22
- 238000012549 training Methods 0.000 claims description 35
- 238000004422 calculation algorithm Methods 0.000 claims description 25
- 239000013598 vector Substances 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 14
- 238000001514 detection method Methods 0.000 claims description 8
- 239000000284 extract Substances 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 4
- 238000003709 image segmentation Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000009396 hybridization Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012015 optical character recognition Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
- G06V30/14—Image acquisition
- G06V30/148—Segmentation of character regions
- G06V30/153—Segmentation of character regions using recognition of characters or words
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Character Discrimination (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention discloses a license plate recognition method, a license plate recognition device, a storage medium and a terminal based on deep learning, wherein the method comprises the following steps: extracting a license plate image of a current vehicle, and performing character segmentation on the license plate image of the current vehicle to generate a plurality of character images; sequentially inputting a plurality of character images into a pre-trained license plate character recognition model, and outputting the probability distribution corresponding to each character image; acquiring the maximum probability value of each character image from the probability distribution corresponding to each character image; determining the subscript of the maximum probability value of each character image as the final category of each character image; the number of the license plate image is identified based on the final category of each character image. Therefore, by adopting the embodiment of the application, the probability distribution of each character in the license plate image is calculated through the trained network model, and the final category is determined through the probability distribution, so that the category of the character in the license plate can be accurately identified, and the identification precision of the license plate of the vehicle is further improved.
Description
Technical Field
The invention relates to the technical field of license plate recognition, in particular to a license plate recognition method and device based on deep learning and a storage medium.
Background
License plate identification is one of important components in modern intelligent traffic systems, and is very widely applied. The method is based on technologies such as digital image processing, mode recognition and computer vision, and analyzes vehicle images or video sequences shot by a camera to obtain a unique license plate number of each vehicle, so that the recognition process is completed. The hardware basis of the license plate recognition system generally includes a trigger device (for monitoring whether a vehicle enters a visual field), a camera device, a lighting device, an image acquisition device, a processing machine (such as a computer) for recognizing a license plate number, and the like. The software core comprises a license plate positioning algorithm, a license plate character segmentation algorithm, an optical character recognition algorithm, a sequence character string recognition algorithm and the like.
In the current technical scheme of vehicle identification, the algorithms for identifying the license plate can be divided into two types: the first algorithm firstly performs character segmentation on an input license plate image to obtain single license plate characters, and then performs classification and recognition on each license plate character. The second algorithm does not require segmentation of every character of the license plate. The algorithm inputs the whole license plate image and outputs the number sequence of the license plate. Although the algorithm has simple steps, the license plate has more types and unfixed length and line number, so that a universal character sequence recognition algorithm is needed, and the performance requirement of the algorithm is high. Because the two existing character recognition algorithms train the same classifier for all characters, all characters are classified by the same classifier in a testing stage, so that the input images can be classified into numbers or letters even if the input images are Chinese characters. Similarly, when the input image is non-Chinese character, the input image can also be recognized as Chinese character, thereby reducing the accuracy of license plate recognition.
Disclosure of Invention
The embodiment of the application provides a license plate recognition method and device based on deep learning and a storage medium. The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview and is intended to neither identify key/critical elements nor delineate the scope of such embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.
In a first aspect, an embodiment of the present application provides a license plate recognition method based on deep learning, where the method includes:
extracting a license plate image of a current vehicle, and performing character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
sequentially inputting a plurality of character images into a pre-trained license plate character recognition model, and outputting the probability distribution corresponding to each character image;
acquiring the maximum probability value of each character image from the probability distribution corresponding to each character image;
determining the subscript of the maximum probability value of each character image as the final category of each character image;
the number of the license plate image is identified based on the final category of each character image.
Optionally, extracting a license plate image of the current vehicle includes:
acquiring a whole vehicle image of a vehicle running direction and/or a vehicle reverse direction through a camera;
and (4) carrying out license plate segmentation on the whole vehicle image by adopting a license plate detection algorithm to generate a license plate image of the current vehicle.
Optionally, the pre-trained license plate character recognition model comprises a feature extraction layer and a full connection layer;
the method comprises the following steps of sequentially inputting a plurality of character images into a pre-trained license plate character recognition model, and outputting probability distribution corresponding to each character image, wherein the probability distribution comprises the following steps:
the feature extraction layer sequentially extracts features of the character images and generates a feature vector corresponding to each character image;
and the full-connection layer sequentially classifies the characteristic vector lines corresponding to each character image and outputs the probability distribution corresponding to each character image.
Optionally, the generating a pre-trained license plate character recognition model according to the following steps includes:
adopting a convolutional neural network to establish a license plate character recognition model;
collecting a plurality of images with license plates as training samples;
randomly acquiring a target image from a training sample, inputting the target image into a license plate character recognition model, and outputting the probability distribution of the target image;
inputting the probability distribution of the target image into a preset loss function for calculation, and outputting a loss value of the model;
and when the loss value of the model reaches a preset minimum value or when the iteration times during model training reach a preset maximum time, generating a pre-trained license plate character recognition model.
Optionally, when the loss value of the model reaches a preset minimum value or when the number of iterations in the model training reaches a preset maximum number, generating a pre-trained license plate character recognition model, including:
and when the loss value of the model does not reach the minimum and the iteration times during model training do not reach the maximum iteration times, continuing to execute the step of randomly acquiring a target image from the training sample and inputting the target image into the license plate character recognition model.
Optionally, randomly obtaining a target image from the training sample, inputting the target image into the license plate character recognition model, and outputting the probability distribution of the target image, including:
randomly acquiring a target image from the training sample;
the feature extraction layer performs feature extraction on the target image to generate a feature vector of the target image;
the fully-connected layer (matrix with the size of N x K) performs correlation classification on the feature vectors of the target images and preset dimension categories, and outputs the probability distribution of the target images; wherein N is the dimension of the feature, and K is the number of categories of the character;
wherein K is preferably 83; the probability distribution of the output can be expressed as y ∈ R83(ii) a The preset dimension categories are as follows: the first 47 dimensions of y are set to represent the probability distribution of the Chinese characters, and y is used47Represents; setting the 48 th-58 th dimension of y to represent the probability distribution of y on the number, and using y10Represents; setting the probability distribution over the numbers represented by the last 26 dimensions of y, with y26And (4) showing.
Optionally, the preset loss function calculation formula is:
where λ represents the weight of the constraint, yiIs the prediction probability value of the ith class, and z represents the type of the input image, wherein the type comprises Chinese characters, letters and numbers.
In a second aspect, an embodiment of the present application provides a license plate recognition device based on deep learning, where the license plate recognition device includes:
the image segmentation module is used for extracting a license plate image of the current vehicle and carrying out character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
the probability distribution output module is used for sequentially inputting a plurality of character images into a pre-trained license plate character recognition model and outputting the probability distribution corresponding to each character image;
the maximum probability value acquisition module is used for acquiring the maximum probability value of each character image from the probability distribution corresponding to each character image;
the final category determining module is used for determining the subscript of the maximum probability value of each character image as the final category of each character image;
and the number identification module is used for identifying the number of the license plate image based on the final category of each character image.
In a third aspect, embodiments of the present application provide a computer storage medium having stored thereon a plurality of instructions adapted to be loaded by a processor and to perform the above-mentioned method steps.
The technical scheme provided by the embodiment of the application can have the following beneficial effects:
in the embodiment of the application, a license plate recognition device firstly extracts a license plate image of a current vehicle, performs character segmentation on the license plate image of the current vehicle to generate a plurality of character images, sequentially inputs the character images into a pre-trained license plate character recognition model, outputs probability distribution corresponding to each character image, acquires a maximum probability value of each character image from the probability distribution corresponding to each character image, determines a subscript of the maximum probability value of each character image as a final class of each character image, and finally recognizes the number of the license plate image based on the final class of each character image. The probability distribution of each character in the license plate image is calculated through the trained network model, and the final category is determined through the probability distribution, so that the category to which the character in the license plate belongs can be accurately identified, and the identification precision of the license plate of the vehicle is further improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention, as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
Fig. 1 is a schematic flowchart of a license plate recognition method based on deep learning according to an embodiment of the present disclosure;
FIG. 2 is a schematic flow chart of license plate character recognition model training in a license plate recognition method based on deep learning according to an embodiment of the present disclosure;
fig. 3 is a schematic diagram of a license plate recognition device based on deep learning according to an embodiment of the present disclosure;
fig. 4 is a schematic structural diagram of a terminal according to an embodiment of the present application.
Detailed Description
The following description and the drawings sufficiently illustrate specific embodiments of the invention to enable those skilled in the art to practice them.
It should be understood that the described embodiments are only some embodiments of the invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention, as detailed in the appended claims.
In the description of the present invention, it is to be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The specific meanings of the above terms in the present invention can be understood in specific cases to those skilled in the art. In addition, in the description of the present invention, "a plurality" means two or more unless otherwise specified. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
In the technical scheme provided by the application, the probability distribution of each character in the license plate image is calculated through the trained network model, and the final category is determined through the probability distribution, so that the category of the character in the license plate can be accurately identified, the identification precision of the license plate of the vehicle is further improved, and the following adopts an exemplary embodiment for detailed description.
The license plate recognition method based on deep learning provided by the embodiment of the application will be described in detail below with reference to fig. 1. The method may be implemented in dependence on a computer program, and may be run on a von neumann architecture based license plate recognition device for deep learning. The computer program may be integrated into the application or may run as a separate tool-like application. The license plate recognition device based on deep learning in the embodiment of the application can be a user terminal, including but not limited to: personal computers, tablet computers, handheld devices, in-vehicle devices, wearable devices, computing devices or other processing devices connected to a wireless modem, and the like. The user terminals may be called different names in different networks, for example: user equipment, access terminal, subscriber unit, subscriber station, mobile station, remote terminal, mobile device, user terminal, wireless communication device, user agent or user equipment, cellular telephone, cordless telephone, Personal Digital Assistant (PDA), terminal equipment in a 5G network or future evolution network, and the like.
Referring to fig. 1, a schematic flow chart of a license plate recognition method based on deep learning is provided in an embodiment of the present application. As shown in fig. 1, the method of the embodiment of the present application may include the following steps:
s101, extracting a license plate image of a current vehicle, and performing character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
the license plate image is a license plate area image at the position where the license plate is mounted on the vehicle.
Generally, when extracting a license plate image, a camera is used for collecting a whole vehicle image in a vehicle driving direction and/or a vehicle reverse direction, and then a license plate detection algorithm is used for carrying out license plate segmentation on the whole vehicle image to generate a license plate image of a current vehicle. The acquired whole vehicle image of the vehicle driving direction and/or the vehicle reverse direction can be an image captured by a road camera in real time, and can also be a vehicle image obtained in other modes.
For example, the images may be images directly captured by a monitoring camera or a road overspeed detection camera, or may be vehicle images obtained after being processed in advance by a computer.
In a possible implementation mode, when a license plate detection algorithm is adopted to segment the license plate of a whole vehicle image, firstly, color detection is carried out on the whole vehicle image, and an area with the same color as a preset color is extracted as a candidate color area of the license plate. The color of the license plate may at least include blue, white, yellow, green and black, so the preset color may be set as blue, white, yellow, green and black. After the license plate candidate color region is extracted, filtering the license plate candidate region to generate a horizontal edge image of the license plate candidate region, then performing binarization processing on the horizontal edge image to generate pixel points of different pixel regions, finally acquiring pixel points with dense pixel hybridization from the pixel points of the different pixel regions, and determining the image region of the pixel points with dense pixel hybridization as the license plate image of the current vehicle.
Further, characters in the license plate image are subjected to character segmentation, and a plurality of character image areas corresponding to the license plate image are obtained.
In the embodiment of the application, when license plate recognition based on deep learning is performed, a whole vehicle image of a vehicle is firstly obtained, then a license plate region in the whole vehicle image is segmented through a license plate detection algorithm, and finally character segmentation is performed on the license plate region to generate a plurality of character images.
S102, sequentially inputting a plurality of character images into a pre-trained license plate character recognition model, and outputting probability distribution corresponding to each character image;
the pre-trained license plate character recognition model is a mathematical model for recognizing probability distribution of each character image in the license plate image, the probability distribution is generated after processing through a full connection layer (matrix with the size of N x K), wherein N is the feature dimension, K is the class number of characters, and the preferred value of K is 83; the probability distribution of the output can be expressed as y ∈ R83(ii) a The preset dimension categories are as follows: the first 47 dimensions of y are set to represent the probability distribution of the Chinese characters, and y is used47Represents; setting the 48 th-58 th dimension of y to represent the probability distribution of y on the number, and using y10Represents; setting the probability distribution over the numbers represented by the last 26 dimensions of y, with y26And (4) showing.
After setting probability distribution of different dimensions, when the training image is a Chinese character, the output distribution of the algorithm constraint network is y26And y10The smaller the probability of (b) is, the better the probability of (b), i.e., the letter and number is. When the training image is a letter, the output of the algorithm constraint network is distributed in y47And y10The smaller the probability of (c) is, the better. When the training image is a number, the algorithm constrains the output distribution of the network at y47And y26The smaller the probability of (c) is, the better. By means of the constraint, the problem that the algorithm wrongly identifies the Chinese characters, the letters and the numbers can be effectively avoided.
Generally, when a license plate character recognition model is trained, firstly, a convolutional neural network is adopted to create the license plate character recognition model, then, a plurality of images with license plates are collected as training samples, then, target images are randomly obtained from the training samples and input into the license plate character recognition model, the probability distribution of the target images is output, then, the probability distribution of the target images is input into a preset loss function for calculation, the loss value of the model is output, and finally, when the loss value of the model reaches a preset minimum value or when the iteration number during model training reaches a preset maximum number, a pre-trained license plate character recognition model is generated.
Further, when the loss value of the model does not reach the minimum value or the iteration frequency of the model does not reach the preset iteration frequency, the step of randomly acquiring a target image from the training sample and inputting the target image into the license plate character recognition model is continuously executed.
Specifically, when the model is trained, after a training image is input into a license plate character recognition model, firstly, a feature extraction layer performs feature extraction on a target image to generate a feature vector of the target image, then, a full connection layer (matrix with the size of N × K) performs associated classification on the feature vector of the target image and a preset dimension class, and probability distribution of the target image is output.
In a possible implementation manner, when the model is applied after training is finished, the plurality of character images generated by segmentation in step S101 are sequentially input into the trained license plate character recognition model, the features of the plurality of character images are sequentially extracted through the feature extraction layer of the model to generate a feature vector corresponding to each character image, the feature vectors corresponding to each character image are sequentially classified through the full-connection layer of the model, and the probability distribution corresponding to each character image is output.
S103, acquiring the maximum probability value of each character image from the probability distribution corresponding to each character image;
generally, after each character image is processed by a license plate character recognition model, a series of probability values corresponding to each character image can be obtained, and the probability values represent corresponding percentages under different types of labels.
Further, a maximum probability percentage is obtained from a series of probability values corresponding to each character image, and the label type of the maximum probability percentage can represent the final category of each character image.
S104, determining the subscript of the maximum probability value of each character image as the final category of each character image;
generally, when a license plate character recognition model is trained, in order to make the output of a network as close to a label vector as possible to realize the minimum classification error, the distinctive features of an image are further extracted. One of the loss functions commonly used in classification networks is the cross-entropy loss function. Let y denote the output probability distribution of the network and y' denote the true label of the data, then the cross entropy penalty for image classification is
loss＝-∑y′ilogyi
Wherein, y'iIs a true tag of class i, and yiIs the prediction probability value of the ith class. In the task of recognizing characters of a license plate, one image only possibly contains one character, so the task belongs to the problem of single-label classification. In such a problem, y' is represented by a one-hot vector (one-hot vector), then the loss function can be written in the form:
loss＝-logyk
where k denotes the class index to which the training sample belongs.
In the embodiment of the present application, let the vector z ∈ R3Indicating whether the input image belongs to a chinese character, a letter or a number. When the input image is a Chinese character, z is [1,0 ]]. When the input image is a letter, z is [0,1,0 ]]. When the input image is a number, z is [0,0,1 ]]. The formula for calculating the loss function proposed in this application can be obtained as follows:
For example, when the input training image is a Chinese character, the loss function of the algorithm is degraded to
If the values of the current algorithm are large in the alpha and numeric components, a large penalty will result. Through back propagation, the components of the algorithm on letters and numbers can be reduced, and therefore the aim of reducing false recognition is achieved.
S105, identifying the number of the license plate image based on the final category of each character image.
In one possible implementation, after the final category of each character image is determined, the number of the license plate image may be identified according to the final category of each character.
In the embodiment of the application, a license plate recognition device firstly extracts a license plate image of a current vehicle, performs character segmentation on the license plate image of the current vehicle to generate a plurality of character images, sequentially inputs the character images into a pre-trained license plate character recognition model, outputs probability distribution corresponding to each character image, acquires a maximum probability value of each character image from the probability distribution corresponding to each character image, determines a subscript of the maximum probability value of each character image as a final class of each character image, and finally recognizes the number of the license plate image based on the final class of each character image. The probability distribution of each character in the license plate image is calculated through the trained network model, and the final category is determined through the probability distribution, so that the category to which the character in the license plate belongs can be accurately identified, and the identification precision of the license plate of the vehicle is further improved.
Referring to fig. 2, a schematic flow chart of a license plate character recognition model training method in a license plate recognition method based on deep learning is provided for the embodiment of the present application. As shown in fig. 2, the method of the embodiment of the present application may include the following steps:
s201, establishing a license plate character recognition model by adopting a convolutional neural network;
s202, collecting a plurality of images with license plates as training samples;
s203, randomly acquiring a target image from the training sample, and performing feature extraction on the target image by a feature extraction layer to generate a feature vector of the target image;
s204, the fully-connected layer (matrix with the size of N x K) performs correlation classification on the feature vectors of the target image and preset dimension categories, and outputs probability distribution of the target image; wherein N is the dimension of the feature, and K is the number of categories of the character;
s205, inputting the probability distribution of the target image into a preset loss function for calculation, and outputting a loss value of the model;
and S206, when the loss value of the model reaches a preset minimum value or when the iteration frequency during model training reaches a preset maximum frequency, generating a pre-trained license plate character recognition model.
In a possible implementation manner, when the loss value of the model does not reach the minimum value, the step of randomly acquiring the target image from the training sample and inputting the target image into the license plate character recognition model is continuously executed for continuous training, and the training is stopped until the loss value of the model reaches the preset minimum value.
The following are embodiments of the apparatus of the present invention that may be used to perform embodiments of the method of the present invention. For details which are not disclosed in the embodiments of the apparatus of the present invention, reference is made to the embodiments of the method of the present invention.
Referring to fig. 3, a schematic structural diagram of a license plate recognition device based on deep learning according to an exemplary embodiment of the invention is shown. The license plate recognition device based on deep learning can be realized by software, hardware or a combination of the software and the hardware to be all or part of a terminal. The device 1 comprises an image segmentation module 10, a probability distribution output module 20, a maximum probability value acquisition module 30, a final class determination module 40 and a number identification module 50.
The image segmentation module 10 is configured to extract a license plate image of a current vehicle, and perform character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
the probability distribution output module 20 is configured to sequentially input a plurality of character images into a pre-trained license plate character recognition model, and output a probability distribution corresponding to each character image;
a maximum probability value obtaining module 30, configured to obtain a maximum probability value to which each character image belongs from a probability distribution corresponding to each character image;
a final category determining module 40, configured to determine a subscript of a maximum probability value to which each character image belongs as a final category of each character image;
a number recognition module 50 for recognizing the number of the license plate image based on the final category of each character image.
It should be noted that, when the license plate recognition device based on deep learning provided in the foregoing embodiment executes the license plate recognition method based on deep learning, only the division of the functional modules is taken as an example, and in practical applications, the function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the license plate recognition device based on deep learning provided by the embodiment and the license plate recognition method based on deep learning have the same concept, and the embodiment of the implementation process is detailed in the method embodiment and is not repeated herein.
The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.
In the embodiment of the application, a license plate recognition device firstly extracts a license plate image of a current vehicle, performs character segmentation on the license plate image of the current vehicle to generate a plurality of character images, sequentially inputs the character images into a pre-trained license plate character recognition model, outputs probability distribution corresponding to each character image, acquires a maximum probability value of each character image from the probability distribution corresponding to each character image, determines a subscript of the maximum probability value of each character image as a final class of each character image, and finally recognizes the number of the license plate image based on the final class of each character image. The probability distribution of each character in the license plate image is calculated through the trained network model, and the final category is determined through the probability distribution, so that the category to which the character in the license plate belongs can be accurately identified, and the identification precision of the license plate of the vehicle is further improved.
The invention also provides a computer readable medium, on which program instructions are stored, and the program instructions, when executed by a processor, implement the deep learning-based license plate recognition method provided by the above-mentioned method embodiments. The invention also provides a computer program product containing instructions, which when run on a computer causes the computer to execute the license plate recognition method based on deep learning of the above method embodiments.
Please refer to fig. 4, which provides a schematic structural diagram of a terminal according to an embodiment of the present application. As shown in fig. 4, terminal 1000 can include: at least one processor 1001, at least one network interface 1004, a user interface 1003, memory 1005, at least one communication bus 1002.
Wherein a communication bus 1002 is used to enable connective communication between these components.
The user interface 1003 may include a Display screen (Display) and a Camera (Camera), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.
The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.
The Memory 1005 may include a Random Access Memory (RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable medium. The memory 1005 may be used to store an instruction, a program, code, a set of codes, or a set of instructions. The memory 1005 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like; the storage data area may store data and the like referred to in the above respective method embodiments. The memory 1005 may optionally be at least one memory device located remotely from the processor 1001. As shown in fig. 4, a memory 1005, which is a type of computer storage medium, may include therein an operating system, a network communication module, a user interface module, and a deep learning based license plate recognition application.
In the terminal 1000 shown in fig. 4, the user interface 1003 is mainly used as an interface for providing input for a user, and acquiring data input by the user; the processor 1001 may be configured to call the license plate recognition application based on deep learning stored in the memory 1005, and specifically perform the following operations:
extracting a license plate image of a current vehicle, and performing character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
sequentially inputting a plurality of character images into a pre-trained license plate character recognition model, and outputting the probability distribution corresponding to each character image;
acquiring the maximum probability value of each character image from the probability distribution corresponding to each character image;
determining the subscript of the maximum probability value of each character image as the final category of each character image;
the number of the license plate image is identified based on the final category of each character image.
In one embodiment, when the processor 1001 extracts the license plate image of the current vehicle, the following operations are specifically performed:
acquiring a whole vehicle image of a vehicle running direction and/or a vehicle reverse direction through a camera;
and (4) carrying out license plate segmentation on the whole vehicle image by adopting a license plate detection algorithm to generate a license plate image of the current vehicle.
In an embodiment, when the processor 1001 sequentially inputs a plurality of character images into a pre-trained license plate character recognition model and outputs a probability distribution corresponding to each character image, the following operations are specifically performed:
the method comprises the following steps of sequentially inputting a plurality of character images into a pre-trained license plate character recognition model, and outputting probability distribution corresponding to each character image, wherein the probability distribution comprises the following steps:
the feature extraction layer sequentially extracts features of the character images and generates a feature vector corresponding to each character image;
and the full-connection layer sequentially classifies the feature vectors corresponding to the character images and outputs the probability distribution corresponding to each character image.
In the embodiment of the application, a license plate recognition device firstly extracts a license plate image of a current vehicle, performs character segmentation on the license plate image of the current vehicle to generate a plurality of character images, sequentially inputs the character images into a pre-trained license plate character recognition model, outputs probability distribution corresponding to each character image, acquires a maximum probability value of each character image from the probability distribution corresponding to each character image, determines a subscript of the maximum probability value of each character image as a final class of each character image, and finally recognizes the number of the license plate image based on the final class of each character image. The probability distribution of each character in the license plate image is calculated through the trained network model, and the final category is determined through the probability distribution, so that the category to which the character in the license plate belongs can be accurately identified, and the identification precision of the license plate of the vehicle is further improved.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware that is related to instructions of a computer program, and the program can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory or a random access memory.
The above disclosure is only for the purpose of illustrating the preferred embodiments of the present application and is not to be construed as limiting the scope of the present application, so that the present application is not limited thereto, and all equivalent variations and modifications can be made to the present application.
Claims (10)
1. A license plate recognition method based on deep learning is characterized by comprising the following steps:
extracting a license plate image of a current vehicle, and performing character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
sequentially inputting the character images into a pre-trained license plate character recognition model, and outputting the probability distribution corresponding to each character image;
acquiring the maximum probability value of each character image from the probability distribution corresponding to each character image;
determining the subscript of the maximum probability value of each character image as the final category of each character image;
and identifying the number of the license plate image based on the final category of each character image.
2. The method of claim 1, wherein the extracting the license plate image of the current vehicle comprises:
acquiring a whole vehicle image of a vehicle running direction and/or a vehicle reverse direction through a camera;
and (4) carrying out license plate segmentation on the whole vehicle image by adopting a license plate detection algorithm to generate a license plate image of the current vehicle.
3. The method of claim 1, wherein the pre-trained license plate character recognition model comprises a feature extraction layer and a full connection layer;
the sequentially inputting the character images into a pre-trained license plate character recognition model and outputting the probability distribution corresponding to each character image comprises the following steps:
the feature extraction layer sequentially extracts features of the character images and generates a feature vector corresponding to each character image;
and the full-connection layer sequentially classifies the feature vectors corresponding to the character images and outputs the probability distribution corresponding to each character image.
4. The method of claim 1, wherein generating a pre-trained license plate character recognition model comprises:
adopting a convolutional neural network to establish a license plate character recognition model;
collecting a plurality of images with license plates as training samples;
randomly acquiring a target image from the training sample, inputting the target image into the license plate character recognition model, and outputting the probability distribution of the target image;
inputting the probability distribution of the target image into a preset loss function for calculation, and outputting a loss value of a model;
and when the loss value of the model reaches a preset minimum value or when the iteration times during model training reach a preset maximum time, generating a pre-trained license plate character recognition model.
5. The method of claim 4, wherein generating a pre-trained license plate character recognition model when the loss value of the model reaches a preset minimum value or when the number of iterations in the model training reaches a preset maximum number comprises:
and when the loss value of the model does not reach the minimum and the iteration times during model training do not reach the maximum iteration times, continuing to execute the step of randomly acquiring a target image from the training sample and inputting the target image into the license plate character recognition model.
6. The method of claim 4, wherein the randomly obtaining a target image from the training sample, inputting the target image into the license plate character recognition model, and outputting a probability distribution of the target image comprises:
randomly acquiring a target image from the training sample;
the feature extraction layer performs feature extraction on the target image to generate a feature vector of the target image;
the fully-connected layer (matrix with the size of N x K) performs correlation classification on the feature vectors of the target images and preset dimension categories, and outputs the probability distribution of the target images; wherein N is the dimension of the feature, and K is the number of categories of the character;
wherein K is 83; the probability distribution of the output is y belongs to R83(ii) a Wherein the preset dimension categories are: the first 47 dimensions of y are set to represent the probability distribution of the Chinese characters, and y is used47Represents; setting the 48 th-58 th dimension of y to represent the probability distribution of y on the number, and using y10Represents; setting the probability distribution over the numbers represented by the last 26 dimensions of y, with y26And (4) showing.
7. The method of claim 4, wherein the predetermined loss function is calculated by:
where λ represents the weight of the constraint, yiIs the prediction probability value of the ith class, and z represents the type of the input image, wherein the type comprises Chinese characters, letters and numbers.
8. A license plate recognition device based on deep learning, the device comprising:
the image segmentation module is used for extracting a license plate image of a current vehicle and performing character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
the probability distribution output module is used for sequentially inputting the character images into a pre-trained license plate character recognition model and outputting the probability distribution corresponding to each character image;
a maximum probability value obtaining module, configured to obtain a maximum probability value to which each character image belongs from the probability distribution corresponding to each character image;
a final category determining module, configured to determine a subscript of a maximum probability value to which each character image belongs as a final category of each character image;
and the number recognition module is used for recognizing the number of the license plate image based on the final category of each character image.
9. A computer storage medium, characterized in that it stores a plurality of instructions adapted to be loaded by a processor and to perform the method steps according to any of claims 1-7.
10. A terminal, comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method steps of any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011428529.5A CN112580643B (en) | 2020-12-09 | License plate recognition method and device based on deep learning and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011428529.5A CN112580643B (en) | 2020-12-09 | License plate recognition method and device based on deep learning and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112580643A true CN112580643A (en) | 2021-03-30 |
| CN112580643B CN112580643B (en) | 2024-07-16 |
Family
ID=
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113177552A (en) * | 2021-05-27 | 2021-07-27 | 的卢技术有限公司 | License plate recognition method based on deep learning |
| CN113255671A (en) * | 2021-07-05 | 2021-08-13 | 浙江智慧视频安防创新中心有限公司 | Target detection method, system, device and medium for object with large length-width ratio |
| CN113435437A (en) * | 2021-06-24 | 2021-09-24 | 随锐科技集团股份有限公司 | Method and device for identifying state of switch on/off indicator and storage medium |
| CN113537342A (en) * | 2021-07-14 | 2021-10-22 | 浙江智慧视频安防创新中心有限公司 | Method and device for detecting object in image, storage medium and terminal |
| CN113592941A (en) * | 2021-08-02 | 2021-11-02 | 北京中交兴路信息科技有限公司 | Certificate image verification method and device, storage medium and terminal |
| CN113610770A (en) * | 2021-07-15 | 2021-11-05 | 浙江大华技术股份有限公司 | License plate recognition method, device and equipment |
| CN113627526A (en) * | 2021-08-10 | 2021-11-09 | 北京百度网讯科技有限公司 | Vehicle identification recognition method and device, electronic equipment and medium |
| CN116935373A (en) * | 2023-09-18 | 2023-10-24 | 珠海亿智电子科技有限公司 | Training method, device, equipment and storage medium for multi-type license plate recognition model |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100054539A1 (en) * | 2006-09-01 | 2010-03-04 | Sensen Networks Pty Ltd | Method and system of identifying one or more features represented in a plurality of sensor acquired data sets |
| CN106650721A (en) * | 2016-12-28 | 2017-05-10 | 吴晓军 | Industrial character identification method based on convolution neural network |
| CN106874907A (en) * | 2017-01-19 | 2017-06-20 | 博康智能信息技术有限公司北京海淀分公司 | A kind of method and device for setting up Car license recognition model |
| CN106934396A (en) * | 2017-03-09 | 2017-07-07 | 深圳市捷顺科技实业股份有限公司 | A kind of license number search method and system |
| CN110163206A (en) * | 2019-05-04 | 2019-08-23 | 苏州科技大学 | Licence plate recognition method, system, storage medium and device |
| CN111046891A (en) * | 2018-10-11 | 2020-04-21 | 杭州海康威视数字技术股份有限公司 | Training method of license plate recognition model, and license plate recognition method and device |
| CN111178370A (en) * | 2019-12-16 | 2020-05-19 | 深圳市华尊科技股份有限公司 | Vehicle retrieval method and related device |
| CN111310766A (en) * | 2020-03-13 | 2020-06-19 | 西北工业大学 | License plate identification method based on coding and decoding and two-dimensional attention mechanism |
| CN111563504A (en) * | 2020-07-16 | 2020-08-21 | 平安国际智慧城市科技股份有限公司 | License plate recognition method and related equipment |
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100054539A1 (en) * | 2006-09-01 | 2010-03-04 | Sensen Networks Pty Ltd | Method and system of identifying one or more features represented in a plurality of sensor acquired data sets |
| CN106650721A (en) * | 2016-12-28 | 2017-05-10 | 吴晓军 | Industrial character identification method based on convolution neural network |
| CN106874907A (en) * | 2017-01-19 | 2017-06-20 | 博康智能信息技术有限公司北京海淀分公司 | A kind of method and device for setting up Car license recognition model |
| CN106934396A (en) * | 2017-03-09 | 2017-07-07 | 深圳市捷顺科技实业股份有限公司 | A kind of license number search method and system |
| CN111046891A (en) * | 2018-10-11 | 2020-04-21 | 杭州海康威视数字技术股份有限公司 | Training method of license plate recognition model, and license plate recognition method and device |
| CN110163206A (en) * | 2019-05-04 | 2019-08-23 | 苏州科技大学 | Licence plate recognition method, system, storage medium and device |
| CN111178370A (en) * | 2019-12-16 | 2020-05-19 | 深圳市华尊科技股份有限公司 | Vehicle retrieval method and related device |
| CN111310766A (en) * | 2020-03-13 | 2020-06-19 | 西北工业大学 | License plate identification method based on coding and decoding and two-dimensional attention mechanism |
| CN111563504A (en) * | 2020-07-16 | 2020-08-21 | 平安国际智慧城市科技股份有限公司 | License plate recognition method and related equipment |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113177552A (en) * | 2021-05-27 | 2021-07-27 | 的卢技术有限公司 | License plate recognition method based on deep learning |
| CN113177552B (en) * | 2021-05-27 | 2024-04-26 | 的卢技术有限公司 | License plate recognition method based on deep learning |
| CN113435437A (en) * | 2021-06-24 | 2021-09-24 | 随锐科技集团股份有限公司 | Method and device for identifying state of switch on/off indicator and storage medium |
| CN113255671A (en) * | 2021-07-05 | 2021-08-13 | 浙江智慧视频安防创新中心有限公司 | Target detection method, system, device and medium for object with large length-width ratio |
| CN113537342A (en) * | 2021-07-14 | 2021-10-22 | 浙江智慧视频安防创新中心有限公司 | Method and device for detecting object in image, storage medium and terminal |
| CN113610770A (en) * | 2021-07-15 | 2021-11-05 | 浙江大华技术股份有限公司 | License plate recognition method, device and equipment |
| CN113592941A (en) * | 2021-08-02 | 2021-11-02 | 北京中交兴路信息科技有限公司 | Certificate image verification method and device, storage medium and terminal |
| CN113592941B (en) * | 2021-08-02 | 2023-09-12 | 北京中交兴路信息科技有限公司 | Certificate image verification method and device, storage medium and terminal |
| CN113627526A (en) * | 2021-08-10 | 2021-11-09 | 北京百度网讯科技有限公司 | Vehicle identification recognition method and device, electronic equipment and medium |
| CN113627526B (en) * | 2021-08-10 | 2023-07-25 | 北京百度网讯科技有限公司 | Vehicle identification recognition method and device, electronic equipment and medium |
| CN116935373A (en) * | 2023-09-18 | 2023-10-24 | 珠海亿智电子科技有限公司 | Training method, device, equipment and storage medium for multi-type license plate recognition model |
| CN116935373B (en) * | 2023-09-18 | 2024-03-12 | 珠海亿智电子科技有限公司 | Training method, device, equipment and storage medium for multi-type license plate recognition model |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107944450B (en) | License plate recognition method and device | |
| CN112434721A (en) | Image classification method, system, storage medium and terminal based on small sample learning | |
| CN111767882A (en) | Multi-mode pedestrian detection method based on improved YOLO model | |
| CN110807314A (en) | Text emotion analysis model training method, device and equipment and readable storage medium | |
| CN112381775A (en) | Image tampering detection method, terminal device and storage medium | |
| CN111739007B (en) | Endoscopic image recognition method, device, storage medium and apparatus | |
| CN112001373B (en) | Article identification method and device and storage medium | |
| CN108875727B (en) | The detection method and device of graph-text identification, storage medium, processor | |
| CN108596240B (en) | Image semantic segmentation method based on discriminant feature network | |
| CN112650868B (en) | Image retrieval method, device and storage medium | |
| CN112633159A (en) | Human-object interaction relation recognition method, model training method and corresponding device | |
| AG | Development of portable automatic number plate recognition (ANPR) system on Raspberry Pi | |
| CN112001362A (en) | Image analysis method, image analysis device and image analysis system | |
| CN111553241A (en) | Method, device and equipment for rejecting mismatching points of palm print and storage medium | |
| CN110659631B (en) | License plate recognition method and terminal equipment | |
| CN111401360B (en) | Method and system for optimizing license plate detection model, license plate detection method and system | |
| CN112329666A (en) | Face recognition method and device, electronic equipment and storage medium | |
| CN116977859A (en) | Weak supervision target detection method based on multi-scale image cutting and instance difficulty | |
| CN112580643B (en) | License plate recognition method and device based on deep learning and storage medium | |
| CN112580643A (en) | License plate recognition method and device based on deep learning and storage medium | |
| CN115984786A (en) | Vehicle damage detection method and device, terminal and storage medium | |
| CN115713669A (en) | Image classification method and device based on inter-class relation, storage medium and terminal | |
| CN113408571B (en) | Image classification method and device based on model distillation, storage medium and terminal | |
| CN115223157A (en) | Power grid equipment nameplate optical character recognition method based on recurrent neural network | |
| CN114972540A (en) | Target positioning method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |