CN112580643B - License plate recognition method and device based on deep learning and storage medium - Google Patents
License plate recognition method and device based on deep learning and storage medium Download PDFInfo
- Publication number
- CN112580643B CN112580643B CN202011428529.5A CN202011428529A CN112580643B CN 112580643 B CN112580643 B CN 112580643B CN 202011428529 A CN202011428529 A CN 202011428529A CN 112580643 B CN112580643 B CN 112580643B
- Authority
- CN
- China
- Prior art keywords
- license plate
- image
- character
- probability distribution
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 238000013135 deep learning Methods 0.000 title claims abstract description 28
- 238000009826 distribution Methods 0.000 claims abstract description 90
- 230000011218 segmentation Effects 0.000 claims abstract description 21
- 238000012549 training Methods 0.000 claims description 40
- 238000004422 calculation algorithm Methods 0.000 claims description 25
- 239000013598 vector Substances 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 19
- 238000000605 extraction Methods 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000001514 detection method Methods 0.000 claims description 8
- 239000000284 extract Substances 0.000 claims description 7
- 239000011159 matrix material Substances 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 5
- 238000003709 image segmentation Methods 0.000 claims description 4
- 238000013178 mathematical model Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000009396 hybridization Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012015 optical character recognition Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Abstract
The application discloses a license plate recognition method, a device, a storage medium and a terminal based on deep learning, wherein the method comprises the following steps: extracting a license plate image of a current vehicle, and performing character segmentation on the license plate image of the current vehicle to generate a plurality of character images; sequentially inputting a plurality of character images into a pre-trained license plate character recognition model, and outputting probability distribution corresponding to each character image; obtaining the maximum probability value of each character image from the probability distribution corresponding to each character image; determining the subscript of the maximum probability value to which each character image belongs as the final category of each character image; the number of the car license image is identified based on the final category of each character image. Therefore, by adopting the embodiment of the application, the probability distribution of each character in the license plate image is calculated through the trained network model, and the final category is determined through the probability distribution, so that the category of the character in the license plate can be accurately identified, and the identification precision of the license plate of the vehicle is further improved.
Description
Technical Field
The invention relates to the technical field of license plate recognition, in particular to a license plate recognition method and device based on deep learning and a storage medium.
Background
License plate identification is one of important components in modern intelligent traffic systems, and has very wide application. Based on digital image processing, pattern recognition, computer vision and other technologies, the method analyzes the vehicle image or video sequence shot by the camera to obtain a unique license plate number of each automobile, thereby completing the recognition process. The hardware base of the license plate recognition system generally comprises a triggering device (for monitoring whether a vehicle enters a visual field), a camera device, a lighting device, an image acquisition device, a processor (such as a computer) for recognizing a license plate number and the like. The software core comprises a license plate positioning algorithm, a license plate character segmentation algorithm, an optical character recognition algorithm, a sequence character string recognition algorithm and the like.
In the current vehicle recognition technical scheme, the algorithm of license plate recognition can be divided into two types: the first algorithm firstly carries out character segmentation on an input license plate image to obtain single license plate characters, and then carries out classification recognition on each license plate character. The second algorithm does not require segmentation of each character of the license plate. The algorithm inputs the whole license plate image and outputs the number sequence of the license plate. The algorithm has simple steps, but the number of license plates is more, the length and the number of lines are not fixed, a general character sequence recognition algorithm is needed, and the performance requirement on the algorithm is higher. Since the existing two character recognition algorithms train the same classifier for all characters, so that all characters are classified by the same classifier in the test stage, this will result in that even if the input image is a kanji, it is likely to be classified as a number or a letter. Similarly, when the input image is non-Chinese characters, the input image can be identified as Chinese characters, so that the accuracy of license plate identification is reduced.
Disclosure of Invention
The embodiment of the application provides a license plate recognition method and device based on deep learning and a storage medium. The following presents a simplified summary in order to provide a basic understanding of some aspects of the disclosed embodiments. This summary is not an extensive overview and is intended to neither identify key/critical elements nor delineate the scope of such embodiments. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.
In a first aspect, an embodiment of the present application provides a license plate recognition method based on deep learning, where the method includes:
Extracting a license plate image of a current vehicle, and performing character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
sequentially inputting a plurality of character images into a pre-trained license plate character recognition model, and outputting probability distribution corresponding to each character image;
obtaining the maximum probability value of each character image from the probability distribution corresponding to each character image;
Determining the subscript of the maximum probability value to which each character image belongs as the final category of each character image;
the number of the car license image is identified based on the final category of each character image.
Optionally, extracting the license plate image of the current vehicle includes:
Acquiring a whole vehicle image of the running direction and/or the reversing direction of the vehicle through a camera;
And (3) carrying out license plate segmentation on the whole vehicle image by adopting a license plate detection algorithm to generate a license plate image of the current vehicle.
Optionally, the pre-trained license plate character recognition model comprises a feature extraction layer and a full connection layer;
sequentially inputting a plurality of character images into a pre-trained license plate character recognition model, and outputting probability distribution corresponding to each character image, wherein the method comprises the following steps:
the feature extraction layer sequentially extracts features of the plurality of character images and generates feature vectors corresponding to each character image;
the full connection layer classifies the feature vector row corresponding to each character image in sequence and outputs probability distribution corresponding to each character image.
Optionally, generating the pre-trained license plate character recognition model according to the following steps includes:
Creating a license plate character recognition model by adopting a convolutional neural network;
collecting a plurality of images with license plates as training samples;
randomly acquiring a target image from a training sample, inputting the target image into a license plate character recognition model, and outputting probability distribution of the target image;
inputting probability distribution of the target image into a preset loss function for calculation, and outputting a loss value of the model;
And when the loss value of the model reaches a preset minimum value or the iteration number of the model training reaches a preset maximum number of times, generating a pre-trained license plate character recognition model.
Optionally, when the loss value of the model reaches a preset minimum value or when the iteration number of the model training reaches a preset maximum number, generating a pre-trained license plate character recognition model, including:
And when the loss value of the model does not reach the minimum and the iteration number of the model training does not reach the maximum iteration number, continuing to execute the step of randomly acquiring a target image from the training sample and inputting the target image into the license plate character recognition model.
Optionally, randomly acquiring a target image from the training sample, inputting the target image into the license plate character recognition model, and outputting probability distribution of the target image, including:
randomly acquiring a target image from the training sample;
the feature extraction layer performs feature extraction on the target image to generate a feature vector of the target image;
The full-connection layer (a matrix with the size of N x K) carries out association classification on the feature vector of the target image and a preset dimension class, and outputs probability distribution of the target image; wherein N is the dimension of the feature, K is the number of categories of the character;
Wherein K preferably has a value of 83; the probability distribution of the output may be expressed as y ε R 83; the preset dimension categories are as follows: setting the first 47 dimensions of y to represent the probability distribution of the y on Chinese characters, wherein y 47 is used for representing the probability distribution; setting the 48 th-58 th dimension of y to represent the probability distribution of y on numbers, which is represented by y 10; the last 26 dimensions of the set y represent the probability distribution over the numbers, denoted by y 26.
Optionally, the preset loss function calculation formula is: Wherein λ represents the weight of the constraint, y i is the predicted probability value of the i-th class, and z represents the type of the input image, and the type comprises Chinese characters, letters and numbers.
In a second aspect, an embodiment of the present application provides a license plate recognition device based on deep learning, where the device includes:
the image segmentation module is used for extracting a license plate image of the current vehicle and carrying out character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
the probability distribution output module is used for sequentially inputting a plurality of character images into a pre-trained license plate character recognition model and outputting probability distribution corresponding to each character image;
the maximum probability value acquisition module is used for acquiring the maximum probability value of each character image from probability distribution corresponding to each character image;
The final category determining module is used for determining the subscript of the maximum probability value of each character image as the final category of each character image;
and the number identification module is used for identifying the number of the license plate image based on the final category of each character image.
In a third aspect, embodiments of the present application provide a computer storage medium having stored thereon a plurality of instructions adapted to be loaded by a processor and to perform the above-described method steps.
The technical scheme provided by the embodiment of the application can have the following beneficial effects:
In the embodiment of the application, a license plate recognition device firstly extracts a license plate image of a current vehicle, performs character segmentation on the license plate image of the current vehicle to generate a plurality of character images, sequentially inputs the plurality of character images into a pre-trained license plate character recognition model, outputs probability distribution corresponding to each character image, acquires a maximum probability value of each character image from the probability distribution corresponding to each character image, determines a subscript of the maximum probability value of each character image as a final category of each character image, and finally recognizes the number of the license plate image based on the final category of each character image. The probability distribution of each character in the license plate image is calculated through the trained network model, and the final category is determined through the probability distribution, so that the category of the character in the license plate can be accurately identified, and the identification accuracy of the license plate of the vehicle is further improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.
Fig. 1 is a schematic flow chart of a license plate recognition method based on deep learning according to an embodiment of the present application;
fig. 2 is a schematic flow chart of training a license plate character recognition model in a license plate recognition method based on deep learning according to an embodiment of the present application;
Fig. 3 is a schematic diagram of a license plate recognition device based on deep learning according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of a terminal according to an embodiment of the present application.
Detailed Description
The following description and the drawings sufficiently illustrate specific embodiments of the invention to enable those skilled in the art to practice them.
It should be understood that the described embodiments are merely some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the invention as detailed in the accompanying claims.
In the description of the present invention, it should be understood that the terms "first," "second," and the like are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The specific meaning of the above terms in the present invention will be understood in specific cases by those of ordinary skill in the art. Furthermore, in the description of the present invention, unless otherwise indicated, "a plurality" means two or more. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
According to the technical scheme provided by the application, the probability distribution of each character in the license plate image is calculated through the trained network model, and the final category is determined through the probability distribution, so that the category of the character in the license plate can be accurately identified, the identification accuracy of the license plate of the vehicle is further improved, and the method is described in detail by adopting an exemplary embodiment.
The license plate recognition method based on deep learning provided by the embodiment of the application will be described in detail with reference to fig. 1. The method can be implemented by means of a computer program and can be run on a deep learning-based license plate recognition device based on a von neumann system. The computer program may be integrated in the application or may run as a stand-alone tool class application. The license plate recognition device based on deep learning in the embodiment of the application may be a user terminal, including but not limited to: personal computers, tablet computers, handheld devices, vehicle mounted devices, wearable devices, computing devices, or other processing devices connected to a wireless modem, and the like. User terminals may be called different names in different networks, for example: a user equipment, an access terminal, a subscriber unit, a subscriber station, a mobile station, a remote terminal, a mobile device, a user terminal, a wireless communication device, a user agent or user equipment, a cellular telephone, a cordless telephone, a Personal Digital Assistant (PDA), a terminal device in a 5G network or a future evolution network, etc.
Referring to fig. 1, a schematic flow chart of a license plate recognition method based on deep learning is provided in an embodiment of the present application. As shown in fig. 1, the method according to the embodiment of the present application may include the following steps:
s101, extracting a license plate image of a current vehicle, and performing character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
the license plate image is a license plate region image of a vehicle mounted license plate.
In general, when license plate images are extracted, firstly, a whole vehicle image in the running direction and/or the reversing direction of a vehicle is acquired through a camera, and then, a license plate detection algorithm is adopted to segment the whole vehicle image into license plate images of the current vehicle. The collected whole vehicle image of the running direction and/or the reversing direction of the vehicle can be an image captured in real time by a road camera, or can be a vehicle image obtained in other modes.
For example, the image may be an image directly captured by a monitoring camera or a road overspeed detection camera, or may be a vehicle image obtained by being preprocessed by a computer.
In one possible implementation manner, when the license plate detection algorithm is adopted to divide the license plate of the whole vehicle image, the color detection is firstly carried out on the whole vehicle image, and the region with the same color as the preset color is extracted as the license plate candidate color region. The color of the license plate at least comprises blue, white, yellow, green and black, so the preset color can be blue, white, yellow, green and black. After the license plate candidate color area is extracted, filtering the license plate candidate area to generate a horizontal edge image of the license plate candidate area, binarizing the horizontal edge image to generate pixel points of different pixel areas, finally obtaining pixel points with dense pixel hybridization from the pixel points of the different pixel areas, and determining the image area of the pixel points with dense pixel hybridization as the license plate image of the current vehicle.
Further, character segmentation is carried out on characters in the license plate image, and a plurality of character image areas corresponding to the license plate image are obtained.
In the embodiment of the application, when the license plate recognition based on deep learning is carried out, the whole vehicle image of the vehicle is firstly obtained, then the license plate region in the whole vehicle image is segmented through a license plate detection algorithm, and finally the license plate region is subjected to character segmentation to generate a plurality of character images.
S102, sequentially inputting a plurality of character images into a pre-trained license plate character recognition model, and outputting probability distribution corresponding to each character image;
The pre-trained license plate character recognition model is a mathematical model for recognizing probability distribution of each character image in a license plate image, the probability distribution is generated after being processed by a full-connection layer (matrix with the size of N x K), wherein N is the dimension of a feature, K is the category number of the character, and the optimal value of K is 83; the probability distribution of the output may be expressed as y ε R 83; the preset dimension categories are as follows: setting the first 47 dimensions of y to represent the probability distribution of the y on Chinese characters, wherein y 47 is used for representing the probability distribution; setting the 48 th-58 th dimension of y to represent the probability distribution of y on numbers, which is represented by y 10; the last 26 dimensions of the set y represent the probability distribution over the numbers, denoted by y 26.
After setting the probability distribution of different dimensions, when the training image is a Chinese character, the smaller the probability of the output distribution of the algorithm constraint network on y 26 and y 10 is, the better the probability on letters and numbers is. When the training image is a letter, the smaller the probability that the output of the algorithm constraint network is distributed over y 47 and y 10, the better. When the training image is a number, the smaller the probability that the output of the algorithm constraint network is distributed over y 47 and y 26, the better. Through the constraint, the problem that the algorithm erroneously recognizes Chinese characters, letters and numbers can be effectively avoided.
In general, when a license plate character recognition model is trained, firstly, a convolutional neural network is adopted to create the license plate character recognition model, then a plurality of images with license plates are collected to serve as training samples, then target images are randomly acquired from the training samples and input into the license plate character recognition model, probability distribution of the target images is output, probability distribution of the target images is input into a preset loss function to be calculated, loss values of the model are output, and finally, when the loss values of the model reach a preset minimum value or iteration times when the model is trained reach a preset maximum number of times, the pre-trained license plate character recognition model is generated.
Further, when the loss value of the model does not reach the minimum or the iteration number of the model does not reach the preset iteration number, the step of randomly acquiring the target image from the training sample and inputting the target image into the license plate character recognition model is continuously executed.
Specifically, during model training, after a training image is input into a license plate character recognition model, firstly, a feature extraction layer performs feature extraction on a target image to generate a feature vector of the target image, and then a full-connection layer (a matrix with the size of N x K) performs association classification on the feature vector of the target image and a preset dimension type to output probability distribution of the target image.
In one possible implementation manner, when the model is applied after training is finished, the plurality of character images generated by segmentation in the step S101 are sequentially input into a license plate character recognition model after training is finished, the features of the plurality of character images are sequentially extracted through a feature extraction layer of the model, feature vectors corresponding to each character image are generated, the feature vectors corresponding to each character image are sequentially classified through a full connection layer of the model, and probability distribution corresponding to each character image is output.
S103, obtaining the maximum probability value of each character image from probability distribution corresponding to each character image;
in general, after each character image is processed by the license plate character recognition model, a series of probability values corresponding to each character image can be obtained, and the probability values represent corresponding percentages under different types of labels.
Further, a maximum probability percentage is obtained from a series of probability values corresponding to each character image, and a tag type of the maximum probability percentage may represent a final category of each character image.
S104, determining the subscript of the maximum probability value of each character image as the final category of each character image;
Generally, in license plate character recognition model training, in order to make the output of the network as close as possible to the label vector to achieve the minimum classification error, distinguishing features of the image are extracted. One of the commonly used loss functions of classification networks is the cross entropy loss function. Let y denote the output probability distribution of the network, y' denote the true label of the data, then the cross entropy penalty of the image classification is
loss＝-∑y′ilogyi
Where y' i is the true label of class i and y i is the predicted probability value of class i. In license plate character recognition tasks, one image can only contain one character, so the task belongs to the single tag classification problem. In such a problem, y' is represented by a one-hot vector (one-hot vector), then the loss function can be written in the form:
loss＝-logyk
where k represents the category index to which the training sample belongs.
In the embodiment of the application, let the vector z epsilon R 3 represent whether the input image belongs to Chinese characters, letters or numbers. When the input image is a kanji, z= [1, 0]. When the input image is a letter, z= [0,1,0]. When the input image is a number, z= [0, 1]. Therefore, the loss function calculation formula provided by the application is as follows:
Where λ represents the weight of the constraint.
For example, when the input training image is a Chinese character, the loss function of the algorithm is degraded to
If the current algorithm has a relatively large value on the alpha and numeric components, a relatively large penalty will result. Through back propagation, the components of the algorithm on letters and numbers can be reduced, so that the aim of reducing false recognition is fulfilled.
S105, identifying the number of the license plate image based on the final category of each character image.
In one possible implementation, after determining the final category of each character image, the number of the car license image may be identified based on the final category of each character.
In the embodiment of the application, a license plate recognition device firstly extracts a license plate image of a current vehicle, performs character segmentation on the license plate image of the current vehicle to generate a plurality of character images, sequentially inputs the plurality of character images into a pre-trained license plate character recognition model, outputs probability distribution corresponding to each character image, acquires a maximum probability value of each character image from the probability distribution corresponding to each character image, determines a subscript of the maximum probability value of each character image as a final category of each character image, and finally recognizes the number of the license plate image based on the final category of each character image. The probability distribution of each character in the license plate image is calculated through the trained network model, and the final category is determined through the probability distribution, so that the category of the character in the license plate can be accurately identified, and the identification accuracy of the license plate of the vehicle is further improved.
Referring to fig. 2, a flowchart of a license plate character recognition model training method in a license plate recognition method based on deep learning is provided in an embodiment of the present application. As shown in fig. 2, the method according to the embodiment of the present application may include the following steps:
S201, a license plate character recognition model is established by adopting a convolutional neural network;
s202, collecting a plurality of images with license plates as training samples;
s203, randomly acquiring a target image from the training sample, and extracting features of the target image by a feature extraction layer to generate a feature vector of the target image;
S204, the full-connection layer (matrix with the size of N x K) carries out association classification on the feature vector of the target image and a preset dimension class, and outputs probability distribution of the target image; wherein N is the dimension of the feature, K is the number of categories of the character;
s205, inputting probability distribution of the target image into a preset loss function for calculation, and outputting a loss value of the model;
S206, when the loss value of the model reaches a preset minimum value or when the iteration number of the model training reaches a preset maximum number of times, generating a pre-trained license plate character recognition model.
In one possible implementation manner, when the loss value of the model does not reach the minimum value, continuing to perform the step of randomly acquiring the target image from the training sample and inputting the target image into the license plate character recognition model to continue training until the loss value of the model reaches the preset minimum value, and stopping training.
The following are examples of the apparatus of the present invention that may be used to perform the method embodiments of the present invention. For details not disclosed in the embodiments of the apparatus of the present invention, please refer to the embodiments of the method of the present invention.
Referring to fig. 3, a schematic structural diagram of a license plate recognition device based on deep learning according to an exemplary embodiment of the present invention is shown. The license plate recognition device based on deep learning can be realized into all or part of the terminal through software, hardware or a combination of the software and the hardware. The apparatus 1 comprises an image segmentation module 10, a probability distribution output module 20, a maximum probability value acquisition module 30, a final category determination module 40, and a number identification module 50.
The image segmentation module 10 is used for extracting a license plate image of a current vehicle and performing character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
The probability distribution output module 20 is configured to sequentially input a plurality of character images into a pre-trained license plate character recognition model, and output probability distribution corresponding to each character image;
A maximum probability value obtaining module 30, configured to obtain a maximum probability value to which each character image belongs from probability distributions corresponding to each character image;
a final category determining module 40, configured to determine, as a final category of each character image, a subscript of a maximum probability value to which each character image belongs;
The number recognition module 50 is used for recognizing the number of the license plate image based on the final category of each character image.
It should be noted that, when the license plate recognition device based on deep learning provided in the foregoing embodiment performs the license plate recognition method based on deep learning, only the division of the foregoing functional modules is used as an example, in practical application, the foregoing functional allocation may be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules, so as to complete all or part of the functions described above. In addition, the license plate recognition device based on deep learning provided in the above embodiment belongs to the same concept as the license plate recognition method based on deep learning, which embodies the detailed implementation process and is not described herein.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
In the embodiment of the application, a license plate recognition device firstly extracts a license plate image of a current vehicle, performs character segmentation on the license plate image of the current vehicle to generate a plurality of character images, sequentially inputs the plurality of character images into a pre-trained license plate character recognition model, outputs probability distribution corresponding to each character image, acquires a maximum probability value of each character image from the probability distribution corresponding to each character image, determines a subscript of the maximum probability value of each character image as a final category of each character image, and finally recognizes the number of the license plate image based on the final category of each character image. The probability distribution of each character in the license plate image is calculated through the trained network model, and the final category is determined through the probability distribution, so that the category of the character in the license plate can be accurately identified, and the identification accuracy of the license plate of the vehicle is further improved.
The invention also provides a computer readable medium, on which program instructions are stored, which when executed by a processor, implement the license plate recognition method based on deep learning provided by the above method embodiments. The invention also provides a computer program product containing instructions which, when run on a computer, cause the computer to perform the deep learning-based license plate recognition method of the above-described method embodiments.
Referring to fig. 4, a schematic structural diagram of a terminal is provided in an embodiment of the present application. As shown in fig. 4, terminal 1000 can include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, at least one communication bus 1002.
Wherein the communication bus 1002 is used to enable connected communication between these components.
The user interface 1003 may include a Display screen (Display) and a Camera (Camera), and the optional user interface 1003 may further include a standard wired interface and a wireless interface.
The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-FI interface), among others.
Wherein the processor 1001 may include one or more processing cores. The processor 1001 connects various parts within the entire electronic device 1000 using various interfaces and lines, and performs various functions of the electronic device 1000 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and invoking data stored in the memory 1005. Alternatively, the processor 1001 may be implemented in at least one hardware form of digital signal Processing (DIGITAL SIGNAL Processing, DSP), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 1001 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 1001 and may be implemented by a single chip.
The Memory 1005 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer readable medium (non-transitory computer-readable storage medium). The memory 1005 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 1005 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-described respective method embodiments, etc.; the storage data area may store data or the like referred to in the above respective method embodiments. The memory 1005 may also optionally be at least one storage device located remotely from the processor 1001. As shown in fig. 4, an operating system, a network communication module, a user interface module, and a license plate recognition application program based on deep learning may be included in a memory 1005 as one type of computer storage medium.
In terminal 1000 shown in fig. 4, user interface 1003 is mainly used for providing an input interface for a user, and acquiring data input by the user; and the processor 1001 may be configured to invoke the license plate recognition application program based on deep learning stored in the memory 1005, and specifically perform the following operations:
Extracting a license plate image of a current vehicle, and performing character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
sequentially inputting a plurality of character images into a pre-trained license plate character recognition model, and outputting probability distribution corresponding to each character image;
obtaining the maximum probability value of each character image from the probability distribution corresponding to each character image;
Determining the subscript of the maximum probability value to which each character image belongs as the final category of each character image;
the number of the car license image is identified based on the final category of each character image.
In one embodiment, the processor 1001, when executing the extraction of the license plate image of the current vehicle, specifically performs the following operations:
Acquiring a whole vehicle image of the running direction and/or the reversing direction of the vehicle through a camera;
And (3) carrying out license plate segmentation on the whole vehicle image by adopting a license plate detection algorithm to generate a license plate image of the current vehicle.
In one embodiment, the processor 1001, when executing the sequential input of a plurality of character images into a pre-trained license plate character recognition model and outputting a probability distribution corresponding to each character image, specifically executes the following operations:
sequentially inputting a plurality of character images into a pre-trained license plate character recognition model, and outputting probability distribution corresponding to each character image, wherein the method comprises the following steps:
the feature extraction layer sequentially extracts features of the plurality of character images and generates feature vectors corresponding to each character image;
And the full connection layer sequentially classifies the feature vectors corresponding to each character image and outputs probability distribution corresponding to each character image.
In the embodiment of the application, a license plate recognition device firstly extracts a license plate image of a current vehicle, performs character segmentation on the license plate image of the current vehicle to generate a plurality of character images, sequentially inputs the plurality of character images into a pre-trained license plate character recognition model, outputs probability distribution corresponding to each character image, acquires a maximum probability value of each character image from the probability distribution corresponding to each character image, determines a subscript of the maximum probability value of each character image as a final category of each character image, and finally recognizes the number of the license plate image based on the final category of each character image. The probability distribution of each character in the license plate image is calculated through the trained network model, and the final category is determined through the probability distribution, so that the category of the character in the license plate can be accurately identified, and the identification accuracy of the license plate of the vehicle is further improved.
Those skilled in the art will appreciate that implementing all or part of the above-described embodiment methods may be accomplished by computer programs to instruct related hardware, and the programs may be stored in a computer readable storage medium, which when executed may include the embodiment methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, or the like.
The foregoing disclosure is illustrative of the present application and is not to be construed as limiting the scope of the application, which is defined by the appended claims.
Claims (8)
1. A license plate recognition method based on deep learning, the method comprising:
Extracting a license plate image of a current vehicle, and performing character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
Sequentially inputting the plurality of character images into a pre-trained license plate character recognition model, and outputting probability distribution corresponding to each character image; the pre-trained license plate character recognition model is a mathematical model for recognizing probability distribution of each character image in license plate images, wherein the probability distribution is generated after being processed by a full-connection layer of a matrix with the size of N multiplied by K, N is the dimension of a feature, and K is the category number of the character;
Wherein, K is 83; the probability distribution of the output is ; Wherein, the preset dimension category is: setting upThe first 47 dimensions of (1) represent the probability distribution of the Chinese characters byA representation; setting up48 Th-58 th dimension of (C)Probability distribution in numbers byA representation; setting upThe last 26 dimensions of (a) represent the probability distribution over numbers byA representation;
Obtaining the maximum probability value of each character image from the probability distribution corresponding to each character image;
determining the subscript of the maximum probability value of each character image as the final category of each character image;
identifying the number of the license plate image based on the final category of each character image; wherein,
Generating a pre-trained license plate character recognition model according to the following steps:
Creating a license plate character recognition model by adopting a convolutional neural network;
collecting a plurality of images with license plates as training samples;
Randomly acquiring a target image from the training sample, inputting the target image into the license plate character recognition model, and outputting probability distribution of the target image;
inputting the probability distribution of the target image into a preset loss function for calculation, and outputting a loss value of a model; the preset loss function calculation formula is as follows: ; wherein, The weights of the constraints are represented as such,Is the predicted probability value for the i-th class,Representing the type of the input image, wherein the type comprises Chinese characters, letters and numbers;
And when the loss value of the model reaches a preset minimum value or the iteration number of the model training reaches a preset maximum number of times, generating a pre-trained license plate character recognition model.
2. The method of claim 1, wherein the extracting the license plate image of the current vehicle comprises:
Acquiring a whole vehicle image of the running direction and/or the reversing direction of the vehicle through a camera;
And (3) carrying out license plate segmentation on the whole vehicle image by adopting a license plate detection algorithm to generate a license plate image of the current vehicle.
3. The method according to claim 1, wherein the pre-trained license plate character recognition model comprises a feature extraction layer and a full connection layer;
The step of sequentially inputting the plurality of character images into a pre-trained license plate character recognition model and outputting probability distribution corresponding to each character image comprises the following steps:
the feature extraction layer sequentially extracts features of the plurality of character images and generates feature vectors corresponding to each character image;
And the full connection layer sequentially classifies the feature vectors corresponding to each character image and outputs probability distribution corresponding to each character image.
4. The method of claim 1, wherein the generating the pre-trained license plate character recognition model when the loss value of the model reaches a preset minimum value or when the number of iterations of the model training reaches a preset maximum number of iterations comprises:
And when the loss value of the model does not reach the minimum and the iteration number of the model training does not reach the maximum iteration number, continuing to execute the step of randomly acquiring a target image from the training sample and inputting the target image into the license plate character recognition model.
5. The method of claim 1, wherein randomly acquiring a target image from the training sample, inputting the target image into the license plate character recognition model, and outputting a probability distribution of the target image, comprises:
randomly acquiring a target image from the training sample;
the feature extraction layer performs feature extraction on the target image to generate a feature vector of the target image;
and carrying out association classification on the feature vector of the target image and a preset dimension class by a full-connection layer of the matrix with the size of N multiplied by K, and outputting probability distribution of the target image.
6. A license plate recognition device based on deep learning, the device comprising:
The image segmentation module is used for extracting a license plate image of a current vehicle and carrying out character segmentation on the license plate image of the current vehicle to generate a plurality of character images;
The probability distribution output module is used for sequentially inputting the plurality of character images into a pre-trained license plate character recognition model and outputting probability distribution corresponding to each character image; the pre-trained license plate character recognition model is a mathematical model for recognizing probability distribution of each character image in license plate images, wherein the probability distribution is generated after being processed by a full-connection layer of a matrix with the size of N multiplied by K, N is the dimension of a feature, and K is the category number of the character; wherein, K is 83; the probability distribution of the output is ; Wherein, the preset dimension category is: setting upThe first 47 dimensions of (1) represent the probability distribution of the Chinese characters byA representation; setting up48 Th-58 th dimension of (C)Probability distribution in numbers byA representation; setting upThe last 26 dimensions of (a) represent the probability distribution over numbers byA representation;
the maximum probability value acquisition module is used for acquiring the maximum probability value of each character image from the probability distribution corresponding to each character image;
The final category determining module is used for determining the subscript of the maximum probability value of each character image as the final category of each character image;
The number recognition module is used for recognizing the number of the license plate image based on the final category of each character image; wherein,
Generating a pre-trained license plate character recognition model according to the following steps:
Creating a license plate character recognition model by adopting a convolutional neural network;
collecting a plurality of images with license plates as training samples;
Randomly acquiring a target image from the training sample, inputting the target image into the license plate character recognition model, and outputting probability distribution of the target image;
inputting the probability distribution of the target image into a preset loss function for calculation, and outputting a loss value of a model; the preset loss function calculation formula is as follows: ; wherein, The weights of the constraints are represented as such,Is the predicted probability value for the i-th class,Representing the type of the input image, wherein the type comprises Chinese characters, letters and numbers;
And when the loss value of the model reaches a preset minimum value or the iteration number of the model training reaches a preset maximum number of times, generating a pre-trained license plate character recognition model.
7. A computer storage medium storing a plurality of instructions adapted to be loaded by a processor and to perform the method of any one of claims 1-5.
8. A terminal, comprising: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to perform the method according to any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011428529.5A CN112580643B (en) | 2020-12-09 | License plate recognition method and device based on deep learning and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011428529.5A CN112580643B (en) | 2020-12-09 | License plate recognition method and device based on deep learning and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112580643A CN112580643A (en) | 2021-03-30 |
| CN112580643B true CN112580643B (en) | 2024-07-16 |
Family
ID=
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110163206A (en) * | 2019-05-04 | 2019-08-23 | 苏州科技大学 | Licence plate recognition method, system, storage medium and device |
| CN111046891A (en) * | 2018-10-11 | 2020-04-21 | 杭州海康威视数字技术股份有限公司 | Training method of license plate recognition model, and license plate recognition method and device |
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111046891A (en) * | 2018-10-11 | 2020-04-21 | 杭州海康威视数字技术股份有限公司 | Training method of license plate recognition model, and license plate recognition method and device |
| CN110163206A (en) * | 2019-05-04 | 2019-08-23 | 苏州科技大学 | Licence plate recognition method, system, storage medium and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110807314A (en) | Text emotion analysis model training method, device and equipment and readable storage medium | |
| CN112447189A (en) | Voice event detection method and device, electronic equipment and computer storage medium | |
| CN111695609A (en) | Target damage degree determination method, target damage degree determination device, electronic device, and storage medium | |
| CN111462140B (en) | Real-time image instance segmentation method based on block stitching | |
| CN112132216B (en) | Vehicle type recognition method and device, electronic equipment and storage medium | |
| CN113408570A (en) | Image category identification method and device based on model distillation, storage medium and terminal | |
| AG | Development of portable automatic number plate recognition (ANPR) system on Raspberry Pi | |
| CN111444986A (en) | Building drawing component classification method and device, electronic equipment and storage medium | |
| CN112749653A (en) | Pedestrian detection method, device, electronic equipment and storage medium | |
| CN112001362A (en) | Image analysis method, image analysis device and image analysis system | |
| CN112016545A (en) | Image generation method and device containing text | |
| CN111694954B (en) | Image classification method and device and electronic equipment | |
| CN115713669B (en) | Image classification method and device based on inter-class relationship, storage medium and terminal | |
| CN112329666A (en) | Face recognition method and device, electronic equipment and storage medium | |
| CN112580643B (en) | License plate recognition method and device based on deep learning and storage medium | |
| CN115984786A (en) | Vehicle damage detection method and device, terminal and storage medium | |
| CN115223157A (en) | Power grid equipment nameplate optical character recognition method based on recurrent neural network | |
| CN113408571B (en) | Image classification method and device based on model distillation, storage medium and terminal | |
| CN112634382B (en) | Method and device for identifying and replacing images of unnatural objects | |
| CN115798004A (en) | Face card punching method and device based on local area, electronic equipment and medium | |
| CN112801960B (en) | Image processing method and device, storage medium and electronic equipment | |
| CN112580643A (en) | License plate recognition method and device based on deep learning and storage medium | |
| CN113515591A (en) | Text bad information identification method and device, electronic equipment and storage medium | |
| CN112561893A (en) | Picture matching method and device, electronic equipment and storage medium | |
| CN111931680A (en) | Vehicle weight recognition method and system based on multiple scales |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |