CN113343983B - License plate number recognition method and electronic equipment - Google Patents
License plate number recognition method and electronic equipment Download PDFInfo
- Publication number
- CN113343983B CN113343983B CN202110685026.4A CN202110685026A CN113343983B CN 113343983 B CN113343983 B CN 113343983B CN 202110685026 A CN202110685026 A CN 202110685026A CN 113343983 B CN113343983 B CN 113343983B
- Authority
- CN
- China
- Prior art keywords
- character
- image
- license plate
- queue
- character image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000012935 Averaging Methods 0.000 claims abstract description 20
- 230000011218 segmentation Effects 0.000 claims abstract description 15
- 238000003062 neural network model Methods 0.000 claims description 39
- 238000012549 training Methods 0.000 claims description 25
- 230000006870 function Effects 0.000 claims description 21
- 238000013145 classification model Methods 0.000 claims description 19
- 238000004891 communication Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 12
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 238000013527 convolutional neural network Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 7
- 238000003702 image correction Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 5
- 239000003086 colorant Substances 0.000 claims description 3
- 230000008569 process Effects 0.000 description 13
- 238000004422 calculation algorithm Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000012015 optical character recognition Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000012937 correction Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Character Discrimination (AREA)
- Traffic Control Systems (AREA)
Abstract
The embodiment of the invention provides a license plate number identification method and electronic equipment, and relates to the technical field of image identification. The embodiment of the invention comprises the following steps: and intercepting license plate images of the same vehicle from a plurality of image frames included in the video stream, and then performing character segmentation on each intercepted license plate image to obtain a plurality of character image queues. And extracting the characteristics of the character images included in each character image queue to obtain the character codes of the character images included in each character image queue. And aiming at each character image queue, carrying out bit-by-bit averaging on the character codes of the specified character images included in the character image queue to obtain the comprehensive character codes corresponding to the character image queue, and converting the comprehensive character codes corresponding to each character image queue into characters to obtain the vehicle license plate number. The license plate recognition can be more accurate.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a license plate number recognition method and electronic equipment.
Background
In order to manage the vehicle, it is necessary to identify the license plate number of the running vehicle. At present, a method for recognizing a license plate number includes recognizing a license plate number included in a vehicle picture based on an Optical Character Recognition (OCR) technique after the vehicle picture is collected.
However, under the influence of light angle or rain and snow weather, the acquired license plate image may be partially blocked or unclear, and the accuracy of license plate number identification is low in such a case.
Disclosure of Invention
The embodiment of the invention aims to provide a license plate number identification method and electronic equipment so as to improve the identification accuracy of a license plate. The specific technical scheme is as follows:
in a first aspect, the present invention provides a license plate number recognition method, including:
intercepting a license plate image of the same vehicle from a plurality of image frames included in a video stream;
performing character segmentation on each intercepted license plate image to obtain a plurality of character image queues; the same character image queue comprises character images at the same position in each license plate image;
extracting the characteristics of the character images included in each character image queue to obtain the character codes of the character images included in each character image queue;
aiming at each character image queue, carrying out bit-by-bit averaging on the character codes of the specified character images included in the character image queue to obtain a comprehensive character code corresponding to the character image queue;
and converting the comprehensive character code corresponding to each character image queue into characters to obtain the license plate number of the vehicle.
Optionally, the bit-wise averaging, for each character image queue, the character codes of the designated character images included in the character image queue to obtain the comprehensive character code corresponding to the character image queue includes:
aiming at each character image queue, averaging the character codes of all the character images included in the character image queue according to bits to obtain a comprehensive character code corresponding to the character image queue; alternatively, the first and second electrodes may be,
aiming at each character image queue, averaging the character codes of the continuous first preset number of character images included in the character image queue according to bits to obtain a comprehensive character code corresponding to the character image queue; alternatively, the first and second electrodes may be,
and for each character image queue, sampling character codes of a second preset number of character images from the character image queue at equal intervals, and averaging the character codes of the second preset number of character images according to bits to obtain a comprehensive character code corresponding to the character image queue.
Optionally, the converting the comprehensive character code corresponding to each character image queue into a character to obtain the license plate number of the vehicle includes:
classifying the comprehensive character codes corresponding to each character image queue to obtain character classification codes corresponding to each character image queue;
translating the character classification codes corresponding to each character image queue into characters through a character classification code comparison table to obtain the license plate number of the vehicle; wherein, the character classification code comparison table comprises character classification codes corresponding to characters.
Optionally, the performing character segmentation on each captured license plate image to obtain a plurality of character image queues includes:
carrying out image correction on each intercepted license plate image;
and performing character segmentation on each license plate image after image correction to obtain a plurality of character image queues.
Optionally, the image rectification of each captured license plate image includes:
and (3) carrying out the following processing on each license plate image:
identifying four vertexes of the license plate image;
carrying out perspective transformation on four vertexes of the license plate image and four vertexes of a standard license plate image;
and stretching or compressing the license plate image after perspective transformation to the size of the standard license plate image.
Optionally, the performing character segmentation on each intercepted license plate image to obtain a plurality of character image queues includes:
aiming at each intercepted license plate image, matching the license plate image with a license plate character position template, and intercepting a character image of each character position in the license plate image;
and aiming at each character position, generating a character image queue from the character image intercepted from the character position of each license plate image so as to obtain the character image queue corresponding to each character position.
Optionally, the extracting the features of the character images included in each character image queue to obtain the character codes of the character images included in each character image queue includes:
performing feature extraction on the character images included in each character image queue through a convolutional neural network part of a trained image classification model to obtain character codes of the character images included in each character image queue;
the classifying the comprehensive character codes corresponding to each character image to obtain the character classification codes corresponding to each character image includes:
and classifying the comprehensive character codes corresponding to each character image through a fully-connected neural network part of the trained image classification model to obtain the character classification codes corresponding to each character image.
Optionally, the image classification model is obtained by training in the following manner:
inputting sample character images in a sample character training set into a neural network model to obtain character classification codes output by the neural network model;
calculating a loss function value based on the character classification code output by the neural network model and the character classification code label of the sample character image, wherein the character classification code label is a character classification code corresponding to a character included in the sample character image;
judging whether the neural network model converges or not based on the loss function value;
if the neural network model does not converge, adjusting model parameters of the neural network model based on the loss function values, and returning to the step of inputting the sample character images in the sample character image training set into the neural network model;
and if the neural network model converges, taking the current neural network model as the image classification model.
Optionally, the sample character image training set is obtained by:
collecting a plurality of license plate images, cutting each license plate image into a plurality of sample character images, and generating a sample character image training set through the obtained sample character images; and/or the presence of a gas in the atmosphere,
selecting a plurality of characters which can be used as license plate numbers, setting the selected characters as preset fonts, adding background colors to the characters with the preset fonts, and generating character images;
and randomly adding interference noise points to the generated character image to obtain a sample character image, and generating a sample character image training set by obtaining the sample character image.
In a second aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor and the communication interface complete communication between the memory and the processor through the communication bus;
a memory for storing a computer program;
and the processor is used for realizing the steps of any license plate number identification method when the program stored in the memory is executed.
In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements any of the steps of the above license plate number identification method.
In a fourth aspect, an embodiment of the present invention further provides a computer program product containing instructions, which when run on a computer, causes the computer to execute any of the above license plate number recognition methods.
By adopting the technical scheme, although the partial area of the license plate is shielded due to rain and snow weather or light, the shielded area of the license plate can be changed in the driving process of the vehicle, so that the license plate images of the same vehicle can be captured from a plurality of image frames included in the video stream, and the shielded parts can be different even if the plurality of license plate images are partially shielded. In the embodiment of the application, each license plate image can be subjected to character segmentation to obtain a plurality of character image queues, and then the character images included in the character image queues are subjected to feature extraction to obtain the character codes of the character images included in each character image queue. Because the same character image queue comprises the character images at the same position in each license plate image, even if one character of a certain license plate image is shielded, the comprehensive character code corresponding to the character image queue is determined by carrying out bit-wise averaging on the character codes of the character images included in the same character image queue, the influence caused by the unclear or shielded character image can be eliminated, and the more accurate comprehensive character code can be obtained. And further, the comprehensive character codes corresponding to each character image queue are converted into characters, and the license plate number of the vehicle can be accurately obtained. Compared with the method for recognizing the license plate image through the OCR, the accuracy of the recognized license plate number can be improved.
Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other embodiments can be obtained according to the drawings without creative efforts.
Fig. 1 is a flowchart of a license plate number recognition method according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method for generating a training set of sample characters according to an embodiment of the present invention;
FIG. 3 is a flowchart of a method for training an image classification model according to an embodiment of the present invention;
fig. 4 is an exemplary schematic diagram of a process of obtaining a character classification code of a character image through an image classification model according to an embodiment of the present invention;
FIG. 5a is a flowchart of a method for generating a character image queue according to an embodiment of the present invention;
FIG. 5b is an exemplary schematic diagram including an image frame and a license plate sub-image of a vehicle according to an embodiment of the present invention;
FIG. 5c is an exemplary diagram of a character image queue provided by an embodiment of the present invention;
fig. 6 is a flowchart of a method for obtaining a character feature encoding queue according to an embodiment of the present invention;
FIG. 7 is a flowchart of another license plate number recognition method according to an embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
In order to improve the identification accuracy of the license plate, the embodiment of the invention provides a license plate number identification method which can be applied to electronic equipment, for example, the electronic equipment can be equipment with image processing capability, such as a server, a computer or a tablet computer. As shown in fig. 1, the method comprises the steps of:
s101, a license plate image of the same vehicle is intercepted from a plurality of image frames included in the video stream.
Alternatively, when a vehicle passes through a road intersection, a video including the vehicle may be shot by a monitoring camera of the road intersection or a multi-frame image including the vehicle may be shot. Alternatively, when a vehicle passes through a cell doorway, a video including the vehicle or a multi-frame image including the vehicle is taken by a monitoring camera at the cell doorway. Alternatively, the image frame including the vehicle may be obtained in other manners, which is not limited in this embodiment of the present invention.
When there are image frames containing different vehicles among a plurality of image frames included in the video stream, the image frames containing the same vehicle may be determined from the plurality of image frames by a preset object detection method. As an example, the preset target detection method may be a YOLO algorithm or a Single Shot multi box Detector (SSD) algorithm. Among them, the YOLO algorithm is called "You Only Look one" in its entirety, and is an object detection algorithm.
If a plurality of vehicles are included in a plurality of continuous image frames in the video stream, the license plate image of each vehicle can be intercepted, and the subsequent steps are respectively executed on the license plate image of each vehicle.
And S102, performing character segmentation on each intercepted license plate image to obtain a plurality of character image queues. And the same character image queue comprises character images at the same position in each license plate image.
Taking a car license plate as an example, the car license plate generally comprises 7 characters, and the character images of the 7 characters are respectively in 7 different character image queues. Assuming that 10 license plate images of the same vehicle are captured in S101, each character image queue includes 10 character images. For example, the first character image queue includes a first character image of each license plate image, the second character image queue includes a second character image of each license plate image, and so on.
S103, extracting the characteristics of the character images included in each character image queue to obtain the character codes of the character images included in each character image queue.
S104, aiming at each character image queue, carrying out bit-wise averaging on the character codes of the designated character images included in the character image queue to obtain the comprehensive character code corresponding to the character image queue.
And S105, converting the comprehensive character codes corresponding to each character image queue into characters to obtain the license plate number of the vehicle.
In one embodiment, the comprehensive character codes corresponding to each character image queue may be classified to obtain the character classification codes corresponding to each character image queue, and then the character classification codes corresponding to each character image queue are translated into characters through a character classification code comparison table to obtain the license plate number of the vehicle.
The character classification code comparison table comprises character classification codes corresponding to the characters. The character classification code comparison table can be preset.
After the character classification code corresponding to each character image queue is obtained, the character classification code corresponding to each character image queue can be searched from the character classification code comparison table, and then the character corresponding to the searched character classification code is obtained. And combining the acquired characters according to the sequence of the character image queue to obtain the license plate number of the vehicle. The character classification code comparison table comprises all characters which can be used as license plate numbers and character classification codes corresponding to all the characters.
Illustratively, the characters in the character classification code comparison table include: 26 capital letters of A to Z, numerals 0 to 9, and abbreviations of the respective provinces (e.g., beijing, hubei, wan, etc.). Optionally, the characters of the character classification code look-up table may also include characters used in a particular license plate, such as "learning" (characters used in a license plate of a learner-driven vehicle).
According to the license plate number identification method provided by the embodiment of the invention, although a part of the area of the license plate is shielded due to rain and snow weather or light, the shielded area of the license plate can be changed in the driving process of the vehicle, so that the license plate images of the same vehicle can be captured from a plurality of image frames included in the video stream, and the shielded parts can be different even if the plurality of license plate images are partially shielded. In the embodiment of the application, each license plate image can be subjected to character segmentation to obtain a plurality of character image queues, and then the character images included in the character image queues are subjected to feature extraction to obtain the character codes of the character images included in each character image queue. Because the same character image queue comprises the character images at the same position in each license plate image, even if one character of a certain license plate image is shielded, the comprehensive character code corresponding to the character image queue is determined by carrying out bit-wise averaging on the character codes of the character images included in the same character image queue, the influence caused by the unclear or shielded character image can be eliminated, and the more accurate comprehensive character code can be obtained. And further, the comprehensive character codes corresponding to each character image queue are converted into characters, and the license plate number of the vehicle can be accurately obtained. Compared with the method for recognizing the license plate image through the OCR, the accuracy of the recognized license plate number can be improved.
For the above S104, for each character image queue, the bit-wise averaging is performed on the character codes of the designated character images included in the character image queue, and the method for obtaining the comprehensive character code corresponding to the character image queue includes the following three methods:
in the first mode, aiming at each character image queue, the character codes of all the character images included in the character image queue are averaged according to bits, and the comprehensive character code corresponding to the character image queue is obtained.
In one embodiment, the character code of each character image is a matrix composed of numbers, and the matrixes corresponding to the character images included in the character image queue are averaged according to bits to obtain the comprehensive character code corresponding to the character image queue.
For example, the character image queue includes two character images, and the character of the character image 1 is encoded as [2,3,0;0,1,1;5,2,6], the character of the character image 2 is encoded as [3,2,1;1,0,1;4,1,7], averaging the two matrices according to bit to obtain the character code corresponding to the character image queue, as shown in formula (1):
and secondly, aiming at each character image queue, averaging the character codes of the continuous first preset number of character images included in the character image queue according to bits to obtain a comprehensive character code corresponding to the character image queue.
For example, a character image queue includes: character image 1, character image 2, character image 3, character image 4, and character image 5. The first preset number is 2, the character codes of the character image 1 and the character image 2 can be selected to be averaged bitwise. The character code of the character image 1 is F1= {2,3,0,0,1,1,5,2,6}, the character code of the character image 2 is F2= {3,2,1,1,0,1,4,1,7}, the two character codes are averaged bit by bit, and the resulting integrated character code corresponding to the character image queue is F = (F1 + F2)/2 = {2.5,2.5,0.5,0.5,0.5,1,4.5,1.5,6.5}.
And thirdly, for each character image queue, sampling character codes of a second preset number of character images from the character image queue at equal intervals, and averaging the character codes of the second preset number of character images according to bits to obtain a comprehensive character code corresponding to the character image queue.
For example, a character image queue includes: character image 1, character image 2, character image 3, character image 4, character image 5, character image 6, character image 7, character image 8, and character image 9. When the second preset number is 3, one character image may be selected every 2 character images, that is, the character image 1, the character image 4, and the character image 7 are selected. And averaging the character codes of the selected 3 character images according to bits to obtain the comprehensive character code corresponding to the character image queue.
It can be seen that, in the first mode, the comprehensive character code corresponding to the character image queue is determined based on the character codes of all the character images in the character image queue, and the determined comprehensive character code corresponding to the character image queue is more accurate.
The second mode and the third mode are based on the character codes of a part of character images in the character image queue, the comprehensive character codes corresponding to the character image queue are determined, the calculated amount is relatively small, the speed of determining the comprehensive character codes corresponding to the character image queue can be improved, and the calculation resources are saved.
In the embodiment of the application, under the condition that the number of the character images included in the character image queue is small, the sampling frequency of a camera for shooting the vehicle image is low, the difference between the character images is large, and in this case, the second mode can be adopted to improve the accuracy of determining the character codes of the character image queue.
Under the condition that the number of character images included in the character image queue is large, the sampling frequency of a camera for shooting the vehicle images is high, the difference between the character images is small, if only a plurality of character images adjacent in time are considered, the difference between the character images is small, and the function of mutual compensation is difficult to achieve. Therefore, a third mode can be adopted, and character images with relatively large differences can be selected by an equal-interval sampling mode, so that the accuracy of determining the character codes of the character image queues is improved.
Because the shooting angles of the license plate images are different, the distortion degrees of the license plate images are different, the image distortion influences the recognition of the image content, and the license plate images need to be corrected in order to improve the accuracy of the license plate recognition.
Based on this, in the above S102, the character segmentation is performed on each license plate image captured, so as to obtain a plurality of character image queues, which can be implemented as follows:
carrying out image correction on each intercepted license plate image;
and performing character segmentation on each license plate image after image correction to obtain a plurality of character image queues.
In the embodiment of the invention, the method for correcting the image of the license plate image comprises the following two steps:
the method comprises the following steps of firstly, carrying out the following processing on each license plate image: and identifying four vertexes of the license plate image, and then carrying out perspective transformation on the four vertexes of the license plate image and the four vertexes of the standard license plate image. And then the license plate image after perspective transformation is stretched or compressed to the size of a standard license plate image.
Optionally, a neural network model may be used to identify four vertices of the license plate image. The size of the standard license plate image may be preset. For example, the size of a standard license plate image is 440 mm × 140 mm.
And secondly, performing distortion removal on each license plate image, and stretching or compressing the license plate image to a preset size.
For example, the preset size may be 200 pixels × 80 pixels.
The embodiment of the invention corrects the license plate image, can reduce the influence of image distortion on license plate recognition, and improves the accuracy of license plate recognition.
In this embodiment of the present invention, in the above S102, performing character segmentation on each captured license plate image to obtain a plurality of character image queues, which may specifically be implemented as: and aiming at each intercepted license plate image, matching the license plate image with a license plate character position template, and intercepting the character image of each character position in the license plate image. And then generating a character image queue from the character image intercepted from the character position of each license plate image aiming at each character position so as to obtain the character image queue corresponding to each character position.
Optionally, the license plate character position template includes a plurality of character positions, each character position includes four corner positions, or each character position includes a center point position, a length, and a width.
In the embodiment of the invention, each character position corresponds to one character image queue, namely the positions of the character images included in each character image queue in the license plate image are the same.
Taking the car license plate as an example, assume that there are 6 license plate images, and each license plate image includes 7 characters. According to the sequence from left to right, the character image of the 1 st character of each license plate image in the 6 license plate images generates a character image queue 1, the character image of the 2 nd character of each license plate image generates a character image queue 2, and by analogy, the character image of the 7 th character of each license plate image generates a character image queue 7.
According to the embodiment of the invention, the character image at each character position in each license plate image generates a character image queue, and because the shooting angles of the license plate images are different and the shielding degrees are different, the character at one position is determined through a plurality of character images at the same position, so that the accuracy of determining the character at each position can be improved.
In an embodiment of the present invention, in step S103, performing feature extraction on the character images included in each character image queue to obtain the character codes of the character images included in each character image queue, where the method may be implemented as:
and performing feature extraction on the character images included in each character image queue through a convolutional neural network part of the trained image classification model to obtain character codes of the character images included in each character image queue.
Correspondingly, the above-mentioned classifying the comprehensive character code corresponding to each character image queue to obtain the character classification code corresponding to each character image queue can be implemented as follows:
and classifying the comprehensive character codes corresponding to each character image through a fully-connected neural network part of the trained image classification model to obtain the character classification codes corresponding to each character image.
In the embodiment of the invention, in order to obtain the trained image classification model, a sample character image training set needs to be obtained. Referring to fig. 2, a sample character image training set may be obtained by:
s201, collecting a plurality of license plate images, cutting each license plate image into a plurality of sample character images, and generating a sample character image training set through the obtained sample character images.
In the embodiment of the invention, the collected license plate image can be corrected firstly, so that the corrected license plate image is the size of the standard license plate image. And then, carrying out image segmentation on the license plate image to obtain a plurality of sample character images, wherein the size of each sample character image is the preset standard character image size. Each character image in the license plate image can be used as a sample character image.
In the embodiment of the present invention, only S201, only S202 and S203, or all of S201 to S203 may be performed.
S202, selecting a plurality of characters capable of being used as license plate numbers, setting the selected characters as preset fonts, adding background colors to the characters with the preset fonts, and generating character images.
The characters on the license plate are special characters developed by relevant departments to avoid counterfeiting of the license plate by other people, and the characters are obtained by improving a black body as a basic character. The preset font is a font similar to the special font. For example, the preset fonts include: black body, microsoft elegant black, chinese fine black, and the like.
The background color added to the characters with the preset font comprises a color which can be used as the background color of the license plate. For example, the background color added for each character may be: white, blue, yellow, green or black.
And S203, randomly adding interference noise points to the generated character image to obtain a sample character image, and generating a sample character image training set through the sample character image.
The embodiment of the invention can also set character classification code labels for each character image included in the sample character image training set, wherein the character classification codes are labeled as character classification codes corresponding to characters actually included in the character images.
Because the character image shot in the real scene may have interference, interference noise points are randomly added to the character image, and the character image shot in the real scene can be simulated. The neural network model is trained by using the character images, so that the image classification model obtained after training has the capability of identifying character images with interference, and the identification accuracy of the license plate number is improved.
In the embodiment of the present invention, referring to fig. 3, the image classification model is obtained by training through the following steps:
s301, inputting the sample character images in the sample character training set into a neural network model to obtain the character classification codes output by the neural network model.
In this embodiment, the neural network model may be AlexNet, vggtet, etc., where vggtet is a network constructed by a Visual Geometry Group (VGG) of a university. AlexNet is a network constructed by Hinton and his student Alex krishevsky.
The neural network model in the embodiment of the application comprises a convolutional neural network part and a fully-connected neural network part. As an example, as shown in fig. 4, the sample character image is the lowermost image containing "S" in fig. 4, the sample character image is input into a convolutional neural network to obtain a character code, and then the character code passes through a fully-connected neural network, and the fully-connected neural network can output a character classification code corresponding to the character code.
And S302, calculating a loss function value based on the character classification code output by the neural network model and the character classification code label of the sample character image.
The character classification code is marked as a character classification code corresponding to a character included in the sample character image.
Optionally, the loss function used for calculating the loss function value may be a cross-entropy loss function or other loss functions, which is not specifically limited in this embodiment of the present invention.
And S303, judging whether the neural network model converges or not based on the loss function value.
In one embodiment, the neural network model is determined to converge when the currently calculated loss function value is less than a preset threshold. And when the currently calculated loss function value is larger than or equal to a preset threshold value, determining that the neural network model does not converge.
In another embodiment, the neural network model is determined to converge when the difference between the currently calculated loss function value and the last calculated loss function value is less than a preset difference. And when the difference value between the current calculated loss function value and the last calculated loss function value is larger than or equal to a preset difference value, determining that the neural network model does not converge.
Alternatively, whether the neural network model converges may be determined in other manners, which is not specifically limited in the embodiment of the present invention.
S304, if the neural network model is not converged, adjusting the model parameters of the neural network model based on the loss function value, and returning to S301.
And adjusting model parameters of the neural network model, namely adjusting parameters of the convolutional neural network and the fully-connected neural network which are included in the neural network model.
S305, if the neural network model converges, taking the current neural network model as an image classification model.
The embodiment of the invention carries out supervised training on the neural network model based on the sample image training set, so that the character classification code corresponding to the output sample character image is closer to the sample character image label in the training process of the neural network model, and the accuracy of the image classification model obtained after training for recognizing the license plate number can be improved.
The license plate number recognition method provided by the embodiment of the application is described below with reference to specific scenes:
in one embodiment, the license plate image in each image frame may be processed in series to obtain a character image queue, see fig. 5a, the method comprising:
s501, a plurality of image frames including the vehicle a are acquired.
For example, one image frame including the vehicle a may be referred to as the uppermost image in fig. 5 b.
S502, respectively intercepting license plate sub-images in each image frame.
For example, referring to fig. 5b, the second image in fig. 5b is, in order from top to bottom: a license plate sub-image taken from the first image in fig. 5b (comprising the image frame of vehicle a).
S503, correcting and aligning each license plate sub-image.
For example, referring to fig. 5b, the third image in fig. 5b is the license plate image obtained by performing the correction alignment on the second image, in the order from the top.
S504, intercepting character images in each license plate sub-image after correction and alignment to obtain a plurality of character image queues.
As an example, assuming that the license plate sub-images in the two image frames, i.e., the license plate sub-image 1 and the license plate sub-image 2, are captured in S502, the character images in the two license plate sub-images may be captured in this step. For example, referring to fig. 5c, the first line image in fig. 5c is a character image included in the license plate sub-image 1, and the second line image is a character image included in the license plate sub-image 2.
The first column of images (i.e. two character images including "S") in the two lines of images is used as a character image queue, the second column of images (i.e. two character images including "G") in the two lines of images is used as a character image queue, and so on, the sixth column of images (i.e. two character images including "5") in the two lines of images is used as a character image queue, so that six character image queues are obtained.
And S505, judging whether the recognition is terminated. If yes, ending the process of generating the character image queue; if not, the process returns to S501.
Wherein, the condition for terminating the identification can be set according to the actual situation. As an example, the condition for terminating the recognition may be that the preset number Zhang Chepai sub-images corresponding to the same vehicle have been continuously recognized, or that the last image frame in the preset image frame sequence has been recognized.
Through the process shown in fig. 5a, i character image queues can be obtained, and then each character image queue can be subjected to feature coding in turn, as shown in fig. 6, the method includes:
s601, start, set i =1.
S602, extracting character images from the ith character image queue.
S603, extracting character images from the ith character image queue, performing feature extraction to obtain character codes, and putting the character codes into the ith character feature code queue.
The convolutional neural network part of the character image input image classification model extracted from the ith character image queue can be used for obtaining the character code output by the convolutional neural network part.
And S604, setting i = i +1, and judging whether to terminate the identification.
If yes, the procedure is terminated; if not, the process returns to S602.
Wherein the condition for terminating the recognition is i > number of character image queues.
Through the process shown in fig. 6, i character feature encoding queues can be obtained, and then the character codes included in each character feature encoding queue can be classified in series, as shown in fig. 7, the method includes:
s701, start, set i =1.
S702, extracting N character codes from the ith character feature code queue, and carrying out bit-wise averaging on the N character codes to obtain the character codes corresponding to the ith character feature code queue.
Wherein, continuous N character codes can be extracted from the i character feature code queue, or N character codes can be extracted at equal intervals. For example, if N =3, the 3 rd, 4 th, and 5 th character codes are extracted, or the 1 st, 3 th, and 5 th character codes are extracted.
And S703, inputting the character code corresponding to the ith character feature coding queue into the fully-connected neural network part of the image classification model to obtain the character classification code corresponding to the character code.
S704, converting the character classification code into a character by inquiring the character classification code comparison table.
S705, judging whether the last character feature coding queue is identified. If not, executing S706; if yes, outputting the license plate number.
S706, set i = i +1, and return to S702.
An embodiment of the present invention further provides an electronic device, as shown in fig. 8, which includes a processor 801, a communication interface 802, a memory 803, and a communication bus 804, where the processor 801, the communication interface 802, and the memory 803 complete mutual communication through the communication bus 804,
a memory 803 for storing a computer program;
the processor 801 is configured to implement the method steps in the above-described method embodiments when executing the program stored in the memory 803.
The communication bus mentioned in the electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown, but this does not mean that there is only one bus or one type of bus.
The communication interface is used for communication between the electronic equipment and other equipment.
The Memory may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.
The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.
In another embodiment of the present invention, a computer-readable storage medium is further provided, in which a computer program is stored, and the computer program, when executed by a processor, implements any of the steps of the above license plate number recognition method.
In another embodiment of the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform any of the license plate number recognition methods of the above embodiments.
In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the invention to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid State Disk (SSD)), among others.
It should be noted that, in this document, relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the electronic device embodiment and the readable storage medium embodiment, since they are substantially similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to part of the description of the method embodiment.
The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.
Claims (9)
1. A license plate number recognition method is characterized by comprising the following steps:
intercepting a license plate image of the same vehicle from a plurality of image frames included in a video stream;
performing character segmentation on each intercepted license plate image to obtain a plurality of character image queues; the same character image queue comprises character images at the same position in each license plate image;
extracting the characteristics of the character images included in each character image queue to obtain the character codes of the character images included in each character image queue;
aiming at each character image queue, carrying out bit-by-bit averaging on the character codes of the designated character images included in the character image queue to obtain a comprehensive character code corresponding to the character image queue;
converting the comprehensive character code corresponding to each character image queue into characters to obtain the license plate number of the vehicle;
the character segmentation is carried out on each intercepted license plate image to obtain a plurality of character image queues, and the method comprises the following steps:
aiming at each intercepted license plate image, matching the license plate image with a license plate character position template, and intercepting a character image of each character position in the license plate image;
and aiming at each character position, generating a character image queue from the character image intercepted from the character position of each license plate image so as to obtain the character image queue corresponding to each character position.
2. The method according to claim 1, wherein the bit-wise averaging, for each character image queue, the character codes of the designated character images included in the character image queue to obtain the comprehensive character code corresponding to the character image queue, includes:
aiming at each character image queue, averaging the character codes of all the character images in the character image queue according to bits to obtain a comprehensive character code corresponding to the character image queue; alternatively, the first and second electrodes may be,
aiming at each character image queue, averaging the character codes of the continuous first preset number of character images included in the character image queue according to bits to obtain a comprehensive character code corresponding to the character image queue; alternatively, the first and second electrodes may be,
and for each character image queue, sampling character codes of a second preset number of character images from the character image queue at equal intervals, and averaging the character codes of the second preset number of character images according to bits to obtain a comprehensive character code corresponding to the character image queue.
3. The method according to claim 1 or 2, wherein the converting the comprehensive character code corresponding to each character image queue into characters to obtain the license plate number of the vehicle comprises:
classifying the comprehensive character codes corresponding to each character image queue to obtain character classification codes corresponding to each character image queue;
translating the character classification codes corresponding to each character image queue into characters through a character classification code comparison table to obtain the license plate number of the vehicle; wherein, the character classification code comparison table comprises character classification codes corresponding to characters.
4. The method of claim 1, wherein the step of performing character segmentation on each intercepted license plate image to obtain a plurality of character image queues comprises:
carrying out image correction on each intercepted license plate image;
and performing character segmentation on each license plate image after image correction to obtain a plurality of character image queues.
5. The method of claim 4, wherein the image rectification of each intercepted license plate image comprises:
and (3) carrying out the following processing on each license plate image:
identifying four vertexes of the license plate image;
carrying out perspective transformation on four vertexes of the license plate image and four vertexes of a standard license plate image;
and stretching or compressing the license plate image after perspective transformation to the size of the standard license plate image.
6. The method according to claim 3, wherein the extracting the features of the character images included in each character image queue to obtain the character codes of the character images included in each character image queue comprises:
performing feature extraction on the character images included in each character image queue through a convolutional neural network part of a trained image classification model to obtain character codes of the character images included in each character image queue;
the classifying the comprehensive character codes corresponding to each character image to obtain the character classification codes corresponding to each character image includes:
and classifying the comprehensive character codes corresponding to each character image through a fully-connected neural network part of the trained image classification model to obtain the character classification codes corresponding to each character image.
7. The method of claim 6, wherein the image classification model is trained by:
inputting sample character images in a sample character training set into a neural network model to obtain character classification codes output by the neural network model;
calculating a loss function value based on the character classification code output by the neural network model and the character classification code label of the sample character image, wherein the character classification code label is a character classification code corresponding to a character included in the sample character image;
determining whether the neural network model converges based on the loss function value;
if the neural network model does not converge, adjusting model parameters of the neural network model based on the loss function values, and returning to the step of inputting the sample character images in the sample character image training set into the neural network model;
and if the neural network model converges, taking the current neural network model as the image classification model.
8. The method of claim 7, wherein the sample character image training set is obtained by:
collecting a plurality of license plate images, cutting each license plate image into a plurality of sample character images, and generating a sample character image training set through the obtained sample character images; and/or the presence of a gas in the gas,
selecting a plurality of characters capable of being used as license plate numbers, setting the selected characters as preset fonts, and adding background colors to the characters with the preset fonts to generate character images;
and randomly adding interference noise points to the generated character image to obtain a sample character image, and generating a sample character image training set by obtaining the sample character image.
9. An electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor and the communication interface are used for realizing mutual communication by the memory through the communication bus;
a memory for storing a computer program;
a processor for implementing the method steps of any of claims 1 to 8 when executing a program stored in the memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110685026.4A CN113343983B (en) | 2021-06-21 | 2021-06-21 | License plate number recognition method and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110685026.4A CN113343983B (en) | 2021-06-21 | 2021-06-21 | License plate number recognition method and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113343983A CN113343983A (en) | 2021-09-03 |
| CN113343983B true CN113343983B (en) | 2022-10-04 |
Family
ID=77477360
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110685026.4A Active CN113343983B (en) | 2021-06-21 | 2021-06-21 | License plate number recognition method and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113343983B (en) |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN201159903Y (en) * | 2008-03-14 | 2008-12-03 | 北京信路威科技发展有限公司 | License plate recognition device |
| JP5306124B2 (en) * | 2009-09-11 | 2013-10-02 | 株式会社東芝 | Number plate reading apparatus and number plate reading method |
| CN101761038B (en) * | 2009-12-31 | 2011-09-14 | 华南理工大学 | System and method for recognizing railway temporary speed-limiting sign |
| US9122953B2 (en) * | 2013-04-15 | 2015-09-01 | Xerox Corporation | Methods and systems for character segmentation in automated license plate recognition applications |
| CN104484672B (en) * | 2014-12-04 | 2018-05-08 | 深圳市平方科技股份有限公司 | Quick licence plate recognition method based on multiframe picture and autonomous learning |
| US10878298B2 (en) * | 2019-03-06 | 2020-12-29 | Adobe Inc. | Tag-based font recognition by utilizing an implicit font classification attention neural network |
-
2021
- 2021-06-21 CN CN202110685026.4A patent/CN113343983B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN113343983A (en) | 2021-09-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108885699B (en) | Character recognition method, device, storage medium and electronic equipment | |
| CN107609549B (en) | Text detection method for certificate image in natural scene | |
| CN110414507B (en) | License plate recognition method and device, computer equipment and storage medium | |
| US10817741B2 (en) | Word segmentation system, method and device | |
| CN109784342B (en) | OCR (optical character recognition) method and terminal based on deep learning model | |
| CN110647829A (en) | Bill text recognition method and system | |
| CN103824091B (en) | A kind of licence plate recognition method for intelligent transportation system | |
| CN110020692B (en) | Handwriting separation and positioning method based on print template | |
| CN107944427B (en) | Dynamic face recognition method and computer readable storage medium | |
| EP2605186B1 (en) | Method and apparatus for recognizing a character based on a photographed image | |
| CN110210297B (en) | Method for locating and extracting Chinese characters in customs clearance image | |
| TWI765469B (en) | System and method for license plate recognition and method for extracting character segments in a license plate | |
| CN109389115B (en) | Text recognition method, device, storage medium and computer equipment | |
| CN112329779A (en) | Method and related device for improving certificate identification accuracy based on mask | |
| CN112507782A (en) | Text image recognition method and device | |
| CN112052845A (en) | Image recognition method, device, equipment and storage medium | |
| CN111401171A (en) | Face image recognition method and device, electronic equipment and storage medium | |
| CN113344826A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN111507337A (en) | License plate recognition method based on hybrid neural network | |
| CN112215236A (en) | Text recognition method and device, electronic equipment and storage medium | |
| CN111582282A (en) | Text recognition method, device, equipment and storage medium | |
| RU2633182C1 (en) | Determination of text line orientation | |
| CN106897770B (en) | Method and device for establishing license plate recognition model | |
| CN110147785A (en) | Image-recognizing method, relevant apparatus and equipment | |
| CN113343983B (en) | License plate number recognition method and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220322 Address after: 430051 No. b1336, chuanggu startup area, taizihu cultural Digital Creative Industry Park, No. 18, Shenlong Avenue, Wuhan Economic and Technological Development Zone, Hubei Province Applicant after: Yikatong (Hubei) Technology Co.,Ltd. Address before: 430056 building B (qdxx-f7b), No.7 building, qiedixiexin science and Technology Innovation Park, South taizihu innovation Valley, Wuhan Economic and Technological Development Zone, Hubei Province Applicant before: HUBEI ECARX TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |