CN109447117B - Double-layer license plate recognition method and device, computer equipment and storage medium - Google Patents
Double-layer license plate recognition method and device, computer equipment and storage medium Download PDFInfo
- Publication number
- CN109447117B CN109447117B CN201811119253.5A CN201811119253A CN109447117B CN 109447117 B CN109447117 B CN 109447117B CN 201811119253 A CN201811119253 A CN 201811119253A CN 109447117 B CN109447117 B CN 109447117B
- Authority
- CN
- China
- Prior art keywords
- license plate
- image
- target
- layer
- double
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000003860 storage Methods 0.000 title claims abstract description 12
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 17
- 238000005520 cutting process Methods 0.000 claims abstract description 12
- 230000006870 function Effects 0.000 claims description 42
- 230000011218 segmentation Effects 0.000 claims description 34
- 238000004422 calculation algorithm Methods 0.000 claims description 21
- 238000003708 edge detection Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 16
- 238000007781 pre-processing Methods 0.000 claims description 13
- 238000012937 correction Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 230000005484 gravity Effects 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 2
- 230000005764 inhibitory process Effects 0.000 claims 1
- 238000012706 support-vector machine Methods 0.000 abstract description 21
- 239000010410 layer Substances 0.000 description 138
- 239000002356 single layer Substances 0.000 description 8
- 238000012545 processing Methods 0.000 description 7
- 230000001629 suppression Effects 0.000 description 7
- 230000008859 change Effects 0.000 description 6
- 229910052704 radon Inorganic materials 0.000 description 5
- SYUHGPGVQRZVTB-UHFFFAOYSA-N radon atom Chemical compound [Rn] SYUHGPGVQRZVTB-UHFFFAOYSA-N 0.000 description 5
- 238000010606 normalization Methods 0.000 description 4
- 238000001514 detection method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 238000005260 corrosion Methods 0.000 description 2
- 230000007797 corrosion Effects 0.000 description 2
- 239000002355 dual-layer Substances 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 238000000513 principal component analysis Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000009795 derivation Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000009977 dual effect Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 210000002569 neuron Anatomy 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/63—Scene text, e.g. street names
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
- G06V20/625—License plates
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Character Input (AREA)
- Image Analysis (AREA)
- Character Discrimination (AREA)
Abstract
The invention discloses a double-layer license plate recognition method, a double-layer license plate recognition device, computer equipment and a storage medium, wherein the double-layer license plate recognition method comprises the following steps: the initial license plate image is obtained, the initial license plate image is preprocessed, the number of license plate layers is judged by using a support vector machine, accurate recognition of double-layer license plates is guaranteed, if the number of license plate layers is double, each layer of license plate image is respectively slid by using a sliding window, each layer of license plate image is divided into a plurality of images containing partially overlapped sliding windows, characters in the license plate number repeatedly appear, the situation that partial characters in the license plate number cannot be recognized due to a manually divided character cutting template is avoided, the universality of double-layer license plate recognition is improved, meanwhile, the obtained sliding window images are input into a convolutional neural network for recognition, a plurality of corresponding recognition results are quickly obtained, and the efficiency of double-layer license plate recognition is improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method and apparatus for identifying a double-layer license plate, a computer device, and a storage medium.
Background
With the development of social economy, more and more automobiles are in road traffic or parking places, which brings many convenience to people's life, but the management of automobiles is becoming more and more complex. Such as vehicle charging and management, traffic flow detection, parking lot charging management, illegal vehicle monitoring, fake license plate vehicle identification, and the like.
Aiming at the problems, the main method adopted at present is to manage the vehicle by identifying the license plates, the current general license plates comprise single-layer license plates and double-layer license plates, the license plate identification technology adopted at present is to divide license plate images by directly and manually dividing the license plate images, and then to identify the divided images.
Disclosure of Invention
The embodiment of the invention provides a double-layer license plate recognition method, a double-layer license plate recognition device, computer equipment and a storage medium, which are used for solving the problems of low accuracy and weak universality of the double-layer license plate recognition at present.
A method for double-layer license plate recognition, comprising:
Acquiring an initial license plate image;
preprocessing the initial license plate image to obtain a target license plate image with the size of 2a multiplied by 2b pixels, wherein a and b are positive integers;
detecting the number of layers of the target license plate image by adopting a support vector machine to obtain a target number of layers, wherein the target number of layers comprises a single layer and a double layer;
if the target layer number is double, cutting the target license plate image into two temporary images with the size of 2a multiplied by b pixels, taking the temporary images with the size of 2a multiplied by b pixels of the lower layer as the lower layer image, cutting the temporary images with the size of 2a multiplied by b pixels of the upper layer, and obtaining the images with the size of a multiplied by b pixels of the central area as the upper layer image;
sliding operation is carried out in the upper layer image and the lower layer image from left to right respectively by adopting a sliding window with preset size of c multiplied by b pixels and taking 1 pixel as a step length, and a target license plate image in the sliding window range is obtained when each sliding is carried out and is used as a sliding window image, wherein c is a positive integer;
in the upper layer image, after sliding a-c times, stopping sliding operation of the upper layer to obtain a-c sliding window images, after sliding 2a-c times on the lower layer image, stopping sliding operation of the lower layer image to obtain 2a-c sliding window images, and obtaining 3a-2c sliding window images altogether;
Inputting 3a-2c sliding window images into a convolutional neural network for recognition to obtain 3a-2c recognition results, wherein each recognition result comprises a plurality of characters;
and forming a character set by all the characters in the 3a-2c recognition results, dividing the character set according to a preset dividing mode, acquiring target characters from each obtained sub-character set, and determining a target license plate number according to the target characters.
A dual layer license plate recognition device comprising:
the acquisition module is used for acquiring an initial license plate image;
the preprocessing module is used for preprocessing the initial license plate image to obtain a target license plate image with the size of 2a multiplied by 2b pixels, wherein a and b are positive integers;
the judging module is used for detecting the number of layers of the target license plate image by adopting a support vector machine to obtain a target number of layers, wherein the target number of layers comprises a single layer and a double layer;
the layering module is used for cutting the target license plate image into two temporary images with the size of 2a multiplied by b pixels if the target layer number is double, taking the temporary images with the size of 2a multiplied by b pixels of the lower layer as the lower layer image, cutting the temporary images with the size of 2a multiplied by b pixels of the upper layer, and obtaining the images with the size of a multiplied by b pixels of the central area as the upper layer image;
The sliding module is used for adopting a sliding window with preset size of c multiplied by b pixels, taking 1 pixel as a step length, respectively performing sliding operation from left to right in the upper layer image and the lower layer image, and acquiring a target license plate image in the range of the sliding window when sliding each time as a sliding window image, wherein c < a and c are positive integers;
the termination module is used for terminating the sliding operation of the upper layer after sliding for a-c times in the upper layer image to obtain a-c sliding window images, and terminating the sliding operation of the lower layer image after sliding for 2a-c times on the lower layer image to obtain 2a-c sliding window images to obtain 3a-2c sliding window images;
the recognition module is used for inputting 3a-2c sliding window images into the convolutional neural network for recognition to obtain 3a-2c recognition results, wherein each recognition result comprises a plurality of characters;
the segmentation module is used for forming a character set by all the characters in the 3a-2c recognition results, segmenting the character set according to a preset segmentation mode, acquiring target characters from each obtained sub-character set, and determining a target license plate number according to the target characters.
A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the above-described two-layer license plate recognition method when the computer program is executed by the processor.
A computer readable storage medium storing a computer program which when executed by a processor performs the steps of the above-described two-layer license plate recognition method.
According to the double-layer license plate recognition method, the device, the computer equipment and the storage medium, on one hand, the initial license plate image is obtained, the initial license plate image is preprocessed to obtain the target license plate image, the number of license plate layers is judged by using the support vector machine, accurate recognition of the double-layer license plate is guaranteed, if the number of license plate layers is double, each layer of license plate image is respectively slid by using the sliding window, so that each layer of license plate image is divided into a plurality of images comprising partially overlapped sliding windows, characters in the license plate number repeatedly appear, the situation that part of characters in the license plate number cannot be recognized due to the fact that the characters are cut by relying on manual division is avoided, and the universality of double-layer license plate recognition is improved; on the other hand, the obtained sliding window image is input into a convolutional neural network for recognition, a plurality of corresponding recognition results are obtained rapidly, and the double-layer license plate recognition efficiency is improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the description of the embodiments of the present invention will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of an application environment of a double-layer license plate recognition method according to an embodiment of the present invention;
FIG. 2 is a flowchart of an implementation of a double-layer license plate recognition method provided by an embodiment of the present invention;
fig. 3 is a flowchart illustrating implementation of step S30 in the double-layer license plate recognition method according to the embodiment of the present invention;
fig. 4 is a flowchart of implementation of step S20 in the double-layer license plate recognition method according to the embodiment of the present invention;
fig. 5 is a flowchart of implementation of step S21 in the double-layer license plate recognition method according to the embodiment of the present invention;
fig. 6 is a flowchart of implementation of step S80 in the double-layer license plate recognition method according to the embodiment of the present invention;
FIG. 7 is a schematic diagram of a double-layer license plate recognition device according to an embodiment of the present invention;
fig. 8 is a schematic diagram of a computer device according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, fig. 1 illustrates an application environment of a double-layer license plate recognition method according to an embodiment of the present invention. The double-layer license plate recognition method is applied to a license plate recognition scene aiming at the photographed license plate of the vehicle. The identification scene comprises a server and a client, wherein the server and the client are connected through a network, the client sends a photographed license plate image to the server, the server receives the license plate image sent by the client and identifies the license plate image, the client can be particularly but not exclusively an overspeed camera, a space network monitor, an electronic police, various personal computers, a notebook computer, a smart phone, a tablet personal computer and portable wearable equipment, and the server can be particularly realized by an independent server or a server cluster formed by a plurality of servers.
Referring to fig. 2, fig. 2 shows a double-layer license plate recognition method according to an embodiment of the present invention, and the method is applied to the server in fig. 1 for illustration, and is described in detail as follows:
s10: and acquiring an initial license plate image.
Specifically, after the client obtains the initial license plate image, the initial license plate image is sent to the server through a network transmission protocol, and the server receives the initial license plate image through the network transmission protocol.
The client can be monitoring equipment with shooting functions, such as an overspeed camera, a space net monitoring device, an electronic police device and the like, can directly shoot to obtain an initial license plate image, can also be intelligent terminal equipment, such as various personal computers, notebook computers, intelligent mobile phones or tablet computers and the like, and has the functions of storing the initial license plate image and performing network interaction with the server.
Among them, network transport protocols include, but are not limited to: internet control message protocol (Internet Control Message Protocol, ICMP), address resolution protocol (ARP Address Resolution Protocol, ARP), and file transfer protocol (File Transfer Protocol, FTP), among others.
S20: and preprocessing the initial license plate image to obtain a target license plate image with the size of 2ax2b pixels, wherein a and b are positive integers.
Specifically, due to the influence of factors such as shooting angle, distance, high-speed running of an automobile and the like, the acquired initial license plate image is low in quality, and the accuracy of direct identification is low, so that the initial license plate image needs to be preprocessed so as to reduce the influence caused by the factors, the accuracy of subsequent identification is improved, and a target license plate image with the size of 2a multiplied by 2b pixels is obtained after the initial license plate image is preprocessed.
Wherein the pretreatment includes, but is not limited to: image cropping, normalization, tilt correction, and the like.
Wherein, a and b are preset values, the basic unit is the width of 1 pixel, 2a is the horizontal side length, namely the width of 2a pixels is the horizontal side length, 2b is the vertical side length, namely the width of 2b pixels is the vertical side length, a and b are positive integers, and the specific values can be set according to the needs, and are not particularly limited herein.
Preferably, in the embodiment of the present invention, the value of a takes 70, and the value of b takes 28, that is, the target license plate image of 140×56 is finally obtained.
S30: and detecting the number of layers of the target license plate image by adopting a support vector machine to obtain the target number of layers, wherein the target number of layers comprises a single layer and a double layer.
Specifically, the number of layers of the target license plate image is detected by using a pre-trained support vector machine, so that the number of layers of the target license plate image is obtained and used as the target number of layers.
The support vector machine (Support Vector Machine, SVM) is a classifier, belongs to a supervised learning model related to a machine learning algorithm, constructs a hyperplane or infinite dimensional space, can be used for classification or regression, and in the embodiment of the invention, the pre-trained support vector machine can be used for identifying the layer number of a target license plate image.
S40: if the target layer number is two layers, the target license plate image is cut into two temporary images with the size of 2a multiplied by b pixels, the temporary images with the size of 2a multiplied by b pixels of the lower layer are used as the lower layer image, the temporary images with the size of 2a multiplied by b pixels of the upper layer are cut, and the images with the size of a multiplied by b pixels of the central area are obtained as the upper layer image.
Specifically, if the number of target layers obtained in step S30 is two, the target license plate image is cut into temporary images of which both the upper and lower sizes are 2a×b pixels, the temporary image of the lower layer of 2a×b pixels is taken as the lower layer image, the temporary image of the upper layer of 2a×b pixels is cut, and an image of a size of a×b of the central region is obtained as the upper layer image.
For example, in one embodiment, a has a value of 70, b has a value of 28, that is, the size of the target license plate image is 140×56 pixels, when the target layer number is two layers, the target license plate image is cut into two temporary images of 140×28 pixels, the lower temporary image is taken as a lower image, and an image with a central area of 70×28 pixels is cut out as an upper image.
S50: and adopting a sliding window with preset size of c multiplied by b pixels, taking 1 pixel as a step length, respectively performing sliding operation from left to right in an upper image and a lower image, and acquiring a target license plate image in the sliding window range when sliding each time as a sliding window image, wherein c < a and c are positive integers.
Specifically, through a preset sliding window with the size of c×b pixels, sliding operation is performed from left to right on the upper layer image and the lower layer image by taking 1 pixel as a step length, and each sliding operation is performed, a target license plate image in the range of the sliding window is obtained and is used as one sliding window image, so that after multiple sliding operations, a plurality of sliding windows have identical overlapped image parts, and the follow-up optimization of target characters is facilitated.
The width b of the sliding window is the same as the width of the target license plate image, and the length c can be set according to practical situations, which is not particularly limited herein.
Preferably, the value of c in the embodiment of the present invention is set to 28, that is, a square of 28×28 pixels is used as a sliding window for sliding.
It should be noted that, if the number of target layers detected in step S30 is a single layer, the sliding window method may also be used to slide the target license plate image in a window manner to obtain a plurality of sliding window images, so as to identify the sliding window images, or identify the sliding window images by using a universal license plate identification method, which is not particularly limited herein.
S60: in the upper layer image, after sliding a-c times, the sliding operation of the upper layer is stopped to obtain a-c sliding window images, after sliding 2a-c times on the lower layer image, the sliding operation of the lower layer image is stopped to obtain 2a-c sliding window images, and 3a-2c sliding window images are obtained in total.
Specifically, the length of the upper layer image is a, the length of the sliding window is b, the sliding step length is 1, the right end of the sliding window is overlapped with the right end of the target license plate image after a-c sliding, namely, the sliding window moves the rightmost end of the upper layer image, a total of a-c sliding window images are obtained at the moment, the length of the lower layer image is 2a, the length of the sliding window is b, the sliding step length is 1, the right end of the sliding window is overlapped with the right end of the lower layer image after a-c sliding, namely, the sliding window moves to the rightmost end of the lower layer image, 2a-c sliding window images are obtained at the moment, and 3a-2c sliding window images are obtained.
S70: 3a-2c sliding window images are input into a convolutional neural network for recognition, and 3a-2c recognition results are obtained, wherein each recognition result comprises a plurality of characters.
Specifically, 3a-2c sliding window images obtained in step S60 are used as input images, and are input into a trained convolutional neural network for recognition, 71 classifiers are preset in the fully connected layer of the convolutional neural network, each classifier corresponds to one preset character, and 71 preset characters are respectively: the method comprises the steps of inputting preset 35 Chinese characters, preset 26 capital English letters and preset 10 Arabic numbers into a full-connection layer after convolution operation of a sliding window image, and classifying and identifying by using preset 71 classifiers to obtain 3a-2c identification results.
It should be noted that each recognition result is one character or a plurality of characters of the 71 preset characters.
Wherein, the preset 35 Chinese characters comprise: beijing, jin, ji, jin, mongolian, liao, ji, black, hu, su, zhe, wan, min, gan, lu, yu, hu, xiang, yue, gui, qong, yu, chuan, qian, yun, tibetan, shan, gan, qing, tibetan, ning, xin, tai, gang and Australia.
Wherein, the preset 26 capital English letters comprise: A. b, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z.
Wherein, 10 Arabic numerals of presetting include: 0. 1, 2, 3, 4, 5, 6, 7, 8, 9.
The convolutional neural network (Convolutional Neural Network, CNN) is a feedforward neural network, and an artificial neuron of the convolutional neural network can respond to surrounding units in a part of coverage range, so that image processing can be performed quickly and efficiently.
S80: and forming a character set by the characters in the 3a-2c recognition results, dividing the character set according to a preset dividing mode, acquiring target characters from each obtained sub-character set, and determining a target license plate number according to the target characters.
Specifically, the characters in the 3a-2c recognition results form a character set, the character set is segmented according to a preset segmentation mode to obtain at least two sub-character sets, one target character is preferably selected from each sub-character set, and the target license plate number is obtained by combining each target character.
The preset segmentation mode may be to divide the character set equally according to the number of sub-character sets to be segmented, or divide the sub-character set according to a preset range or preset segmentation point, or set according to actual conditions, which is not limited specifically herein.
It should be noted that, because the license plate number is generally 7 characters, as a preferred mode, the embodiment of the invention divides the character set formed by 3a-2c recognition results into 7 sub-character sets according to a preset division mode.
In the embodiment, the initial license plate image is acquired, the initial license plate image is preprocessed, the target license plate image is obtained, the number of license plate layers is judged by using a support vector machine, accurate recognition of double-layer license plates is guaranteed, when the number of license plate layers is double-layer, each layer of license plate image is respectively slid by using a sliding window, each layer of license plate image is divided into a plurality of partially overlapped sliding window images, characters in the license plate number repeatedly appear, the situation that part of characters in the license plate number cannot be recognized due to a manually divided character cutting template is avoided, the universality of double-layer license plate recognition is improved, meanwhile, the obtained sliding window images are input into a convolutional neural network for recognition, a plurality of corresponding recognition results are quickly obtained, and the efficiency of double-layer license plate recognition is improved.
Based on the embodiment corresponding to fig. 2, a specific implementation method for detecting the number of layers of the target license plate image by using the support vector machine and obtaining the number of target layers mentioned in step S30 is described in detail below by using a specific embodiment.
Referring to fig. 3, fig. 3 shows a specific implementation flow of step S30 provided in the embodiment of the present invention, which is described in detail below:
s31: and extracting image vector features of the target license plate image.
Specifically, the image vector features of the target license plate image are extracted and used as the input features of a support vector machine.
The image vector features include, but are not limited to, color vector features, texture vector features, shape vector features and spatial relationship vector features, and preferably, the image vector features extracted by the embodiment of the invention are spatial relationship vector features, and the target layer number of the target license plate image is determined by inputting the spatial relationship vector features into a support vector machine.
Among them, extraction modes include, but are not limited to: principal component analysis (Principal Component Analysis, PCA) feature extraction algorithms, convolutional neural network based feature extraction and local binary pattern (Local Binary Pattern, LBP) feature extraction algorithms, and the like. The specific extraction mode can be set according to actual requirements, and is not limited herein.
S32: the function value of the classification function of the image feature is calculated using the following formula:
f(x)＝w T x+b
wherein f (x) is a classification function, w is a weight vector supporting a hyperplane of a vector machine, and w T The method is characterized in that the method is a transpose of a weight vector of a hyperplane, x is an image vector characteristic, and b is a preset offset value.
Specifically, the image vector features extracted in step S31 are input into a pre-trained support vector machine for calculation, and the function of the classification function corresponding to the image vector features is obtained by using the above formula in the support vector machineThe value, w is the weight vector of the hyperplane of the support vector machine T The transpose of the weight vector of the hyperplane is given, and b is the offset value of the support vector machine.
The support vector machine calculates the weight vector and the offset value of the hyperplane through kernel functions, different kernel functions are used, the calculated weight vector and offset value of the hyperplane are different, the identification accuracy is also different, and common kernel functions include but are not limited to: linear kernel functions, polynomial kernel functions, gaussian (Radial Basis Function, RBF) kernel functions, sigmoid kernel functions, and the like.
Preferably, the embodiment of the invention adopts the support vector machine based on the Gaussian kernel function, and the support vector machine based on the Gaussian kernel function has higher recognition accuracy no matter a large sample or a small sample, and has fewer kernel function parameters and stronger compatibility compared with other types.
S33: and if the function value is greater than or equal to 0, determining that the target layer number is double.
Specifically, if the function value calculated in step S32 is greater than or equal to 0, it is determined that the target layer number of the target license plate image is double.
If the function value calculated in step S32 is smaller than 0, it is determined that the target layer number of the target license plate image is a single layer, and the target license plate image of the single layer may be identified by using a general license plate identification algorithm or by using the identification method using a sliding window in the embodiment of the present invention.
In this embodiment, the image vector features of the target license plate image are extracted, and the image vector features are input into a preset support vector machine to calculate the function value of the classification function, so that the target layer number is judged according to the calculated function value, the automatic judgment of the target layer number is realized, and the accurate judgment of the double-layer license plate is ensured.
Based on the corresponding embodiment of fig. 2, a specific implementation method for preprocessing the initial license plate image mentioned in step S20 to obtain the target license plate image with the size of 2a×2b is described in detail below by using a specific embodiment.
Referring to fig. 4, fig. 4 shows a specific implementation flow of step S20 provided in the embodiment of the present invention, which is described in detail below:
s21: and acquiring the upper boundary of the license plate and the lower boundary of the license plate in the initial license plate image through an edge detection algorithm.
Specifically, due to the influence of factors such as shooting angles and distances, the shot license plate images generally comprise a plurality of non-license plate images outside the license plate areas, and the non-license plate images can interfere with subsequent license plate recognition, so that in order to improve the accuracy of subsequent license plate recognition, the upper boundary of the license plate in the initial license plate image and the lower boundary of the license plate are required to be found out through an edge checking algorithm, and the license plate range in the initial license plate image is determined.
The edge detection algorithm is an algorithm for performing edge detection, and edge detection is a fundamental problem in image processing and computer vision, and in the embodiment of the present invention, the purpose of edge detection is to identify points with obvious brightness change in license plate images, namely points of license plate boundaries, and significant changes in image attributes generally reflect important events and changes of the attributes, and the image attributes include but are not limited to: discontinuities in depth, surface direction discontinuities, material property changes, scene lighting changes, etc.
The edge of the image refers to the region where the gray level of the image changes sharply, and the change of the gray level of the image can be reflected by the gradient of the gray level distribution.
Common edge detection algorithms include, but are not limited to: sobel operator edge detection algorithm, laplace operator edge detection algorithm, lobez cross edge detection (Roberts Cross operator) algorithm, canny multi-level edge detection algorithm, and the like.
Preferably, the edge detection algorithm adopted by the embodiment of the invention is a Canny multi-stage edge detection algorithm.
It is worth to say that, according to the edge detection algorithm, the obtained upper boundary of the license plate and the lower boundary of the license plate are two segments. That is, the upper boundary of the obtained license plate comprises a left vertex and a right vertex, and the lower boundary of the obtained license plate comprises a left vertex and a right vertex.
S22: and determining a range image of the license plate according to the upper boundary of the license plate and the lower boundary of the license plate.
Specifically, the upper boundary of the license plate obtained in step S21 is connected with the lower boundary of the license plate, that is, the left vertex of the upper boundary of the license plate is connected with the left vertex of the lower boundary of the license plate, and the right vertex of the upper boundary of the license plate is connected with the right vertex of the lower boundary of the license plate, so that a quadrilateral is obtained, and an image within the range of the quadrilateral is used as a range image of the license plate.
S23: and performing inclination correction on the range image by using the radon transformation to obtain a corrected base image.
Specifically, due to the influence of shooting angles and distances, the acquired initial license plate image is inclined, and further the acquired range image is inclined, so that in order to improve the accuracy of subsequent license plate recognition, the range image is subjected to inclination correction through radon transformation, and a corrected basic image is obtained.
The radon transform is a method for obtaining a corrected image by determining an image inclination angle and correcting the image inclination angle by finding the angle when the maximum projection value is found through directional projection superposition.
S24: and cutting the basic image by taking the center of gravity of the basic image as the center to obtain a target license plate image with the size of a multiplied by b pixels.
Specifically, the base image obtained in step S23 is cut with the center of gravity of the base image as the center, a as the horizontal side length and b as the vertical side length, to obtain a rectangular target license plate image with a×b pixels.
Preferably, in the embodiment of the present invention, a is preset to 140, and b is preset to 28, i.e. the final target license plate image with 140×28 pixels is obtained.
For example, in one embodiment, the center of gravity of the base image is (82, 21), the horizontal side length is 140, and the vertical side length is 28, so that the upper left corner vertex coordinates (12, 7), the upper right corner vertex coordinates (152,7), the lower left corner vertex coordinates (12, 35), the lower right corner vertex coordinates (152, 35), a 140×28 pixel rectangle composed of the four vertices, and the base image is cut along the rectangle, so that the image within the rectangle range, that is, the target license plate image is obtained.
Preferably, after the target license plate image is obtained, the embodiment of the invention also carries out mean value removal and normalization on the target license plate image, and eliminates the difference between different dimension image data in the target license plate image.
The normalization is to normalize the image feature amplitude in the target license plate image to the same range, namely dividing the standard deviation of all the image features by each image feature, and taking the obtained result as the image feature after the normalization of the image features.
The mean value removing refers to that each dimension of the image features in the target license plate image is centered to 0, namely, the center point of the target license plate image is pulled back to the origin of the coordinate system.
In this embodiment, the upper boundary of the license plate and the lower boundary of the license plate in the initial license plate image are obtained through an edge detection algorithm, and then the range image of the license plate is determined according to the upper boundary of the license plate and the lower boundary of the license plate, and then the range image is subjected to inclination correction by using radon transformation to obtain a corrected basic image, and then the basic image is cut by taking the center of gravity of the basic image as the center to obtain a target license plate image with a size of a×b pixels, so that when the basic image is used for identification subsequently, identification errors caused by interference and inclination factors of images outside the range of the license plate are avoided, the quality of the target license plate image is enhanced, and the accuracy of identification of subsequent license plates is improved.
Based on the corresponding embodiment of fig. 4, a specific implementation method for acquiring the upper boundary of the license plate and the lower boundary of the license plate in the initial license plate image by the edge detection algorithm mentioned in step S21 is described in detail below by using a specific embodiment.
Referring to fig. 5, fig. 5 shows a specific implementation flow of step S21 provided in the embodiment of the present invention, which is described in detail below:
s211: and removing noise from the initial license plate image through Gaussian blur to obtain a denoising license plate image.
Specifically, the edge of the license plate image is a high-frequency signal, but the noise of the image is also concentrated in the high-frequency signal and is easily recognized as the edge by mistake, so that the noise of the image needs to be removed, and the interference of the noise of the image on the determined edge is avoided. In the embodiment of the invention, the Gaussian blur is adopted to remove noise of the initial license plate image, and the denoised license plate image is obtained.
The edge of the license plate image refers to an area with sharp change of gray level in the image at the juncture of the license plate area and the non-license plate area. The change in gray scale of the image can be reflected by the gradient of the gray scale distribution.
The noise of the image, that is, the noise of the image, refers to unnecessary or redundant interference information existing in the image data, and the noise seriously affects the quality of the image, so that the noise should be corrected before the image enhancement processing and the classification processing.
Among these, gaussian Blur (Gaussian blue), also known as Gaussian smoothing, is an image Blur filter that uses a normal distribution to calculate the transform for each pixel in an image, typically to reduce image noise and to reduce the level of detail.
It should be noted that, both the image edge and the noise are high-frequency signals, so that the radius selection of the gaussian blur is important, and too large a radius can easily prevent some weak edge points from being detected, and the specific setting of the radius can be adjusted according to the actual situation, which is not limited herein.
S212: and calculating gradient values in the horizontal direction and the vertical direction of the denoising license plate image by using a preset gradient operator to obtain an initial gradient value set.
Specifically, the edges of the image can point to different directions, and gradient values in the horizontal direction and the vertical direction of the denoising license plate image are calculated by using a preset operator, so that an initial gradient value set is obtained.
The digital image is a discrete point value spectrum, and can also be called a two-dimensional discrete function, the gradient of the image is the result of the derivation of the two-dimensional discrete function, and a gradient operator, namely a method for calculating the gradient.
Wherein, the preset gradient operator includes but is not limited to: sobel operator (Sobel operator), prewitt operator, roberts operator (Roberts operator), and Canny operator.
Preferably, the gradient operator adopted in the embodiment of the invention is a Canny operator.
S213: and carrying out edge refinement treatment on the initial gradient value set by adopting a non-maximum value suppression mode to obtain a gradient edge with a pixel width.
Specifically, edge refinement processing is performed on the initial gradient value set by adopting a non-maximum value suppression mode, so as to obtain a gradient edge with a pixel width.
Where non-maximum suppression (Non Maximum Suppression, NMS) is an element that suppresses non-maxima, it is understood that a maximum search is locally performed, helping to preserve the local maximum gradient while suppressing all other gradient values, meaning that only the sharpest locations in the gradient change are preserved.
For example, in a specific embodiment, in the vertical direction, a gradient value with a width of 4 pixels forms a part, and a non-maximum suppression mode is adopted in the part of the cover, and a pixel point with the maximum gradient value in the gradient value of the part is searched out and used as a gradient edge, so that edge refinement is realized.
S214: and filtering weak edge points in the gradient edge by using a preset double threshold value to obtain strong edge points in the gradient edge.
Specifically, edge pixels are distinguished by setting a double threshold, i.e., a high threshold and a low threshold. If the edge pixel point gradient value is greater than the high threshold, it is considered a strong edge point, if the edge gradient value is less than the high threshold, it is greater than the low threshold, it is marked as a weak edge point, and points less than the low threshold are suppressed, the strong edge point may be confirmed as a true edge, but the weak edge point may be a true edge, or may be caused by noise or color change, and in order to obtain an accurate result, the weak edge point is also suppressed in the embodiment of the present invention.
S215: and determining the upper boundary of the license plate and the lower boundary of the license plate according to the strong edge points.
Specifically, according to the strong edge points, the upper boundary of the license plate and the lower boundary of the license plate are obtained through image processing such as corrosion extension.
Wherein corrosion may eliminate individual ones of the strong edge points and extension may connect adjacent but unconnected strong edge points.
In this embodiment, noise is removed from an initial license plate image through gaussian blur, a denoised license plate image is obtained, gradient values in the horizontal direction and the vertical direction of the denoised license plate image are calculated by using a preset gradient operator, an initial gradient value set is obtained, further, edge refinement is performed on the initial gradient value set in a non-maximum value suppression mode, a gradient edge with a pixel width is obtained, weak edge points in the gradient edge are filtered by using a preset double threshold value, a strong edge point in the gradient edge is obtained, and then the upper boundary of a license plate and the lower boundary of the license plate are determined according to the strong edge point, so that the accuracy of license plate edge detection is improved, and the subsequent determination of a license plate range is facilitated.
On the basis of the corresponding embodiment of fig. 2, the specific implementation method for forming the characters in the 3a-2c recognition results mentioned in step S80 into a character set, dividing the character set according to a preset dividing manner, obtaining the target characters from each obtained sub-character set, and determining the target license plate number according to the target characters is described in detail below by using a specific embodiment.
Referring to fig. 6, fig. 6 shows a specific implementation flow of step S80 provided in the embodiment of the present invention, which is described in detail below:
s81: sequentially placing characters in the a-c recognition results into a character set X to obtain X= { X 1 ，x 2 ，...，x q Q is the total number of characters contained in a-c recognition results, x i For the ith character in character set X, i ε [1, q]Q is a positive integer.
Specifically, words contained in each recognition result are writtenThe symbols are sequentially put into a character set X with an initial value of empty to obtain X= { X 1 ，x 2 ，...，x q Q is the total number of characters contained in a-c recognition results, and xi is the ith character in character set X.
For example, in one embodiment, 112 recognition results are obtained, wherein the first 20 recognition results are: "," "I", "L", "O", "Shanghai" Shanghai "," Sha wherein the content in each double-quotation mark is a recognition result, each recognition result can be 1 character, or may be empty, or may be a plurality of characters, the 20 recognition results are sequentially stored into a character set X to obtain X= { "I", "L", "O", "Shanghai", shanghai ",.
S82: and determining k dividing points of the character set X according to k+1 preset ranges, wherein k is smaller than q, and k is a positive integer.
Specifically, after the character set X is obtained in step S81, the character set X needs to be segmented into a plurality of sub-character sets according to the number of target characters, and then a target character is determined according to each sub-character set.
In the embodiment of the invention, provided that k+1 ranges are preset according to the number k+1 of characters of the license plate to be recognized, k segmentation points in the character set X are determined according to the k+1 preset ranges.
The preset range is used to define a preferred range of each target character, for example, a first preset range is (0, 15), that is, the first recognition result to the 15 th recognition result are the first preset range.
Taking the example in step S81 as an example, if the first preset range is (0, 15), the obtained character set X= { "I", "L", "O", "Shanghai" Shanghai "," Shaw "," Sha then the 15 th recognition result is selected as a segmentation point, i.e. the position between the 15 th element "I" and the 16 th element "B" of the character set X is the first segmentation point.
It is noted that the preset range may include a fraction, for example, the second preset range is (15, 31.5), that is, the second preset range includes all characters from the 16 th recognition result to the 31 st recognition result, and the first 50% of characters of the second recognition result.
The target characters refer to characters in license plate number characters, and most license plate numbers are 7 target characters or 8 target characters currently, so that the number of target characters is usually 7 or 8, and the target characters can be set to other numerical values according to actual needs, and the number is not particularly limited.
S83: and performing character segmentation on q characters in the character set X by using k segmentation points to obtain k+1 sub-character sets.
Specifically, character segmentation is performed on characters in the character set X by using the obtained k segmentation points to obtain k+1 sub-character sets, namely k+1 preferred ranges.
The segmentation refers to taking two adjacent segmentation points as boundaries, taking all characters in the range of the two adjacent segmentation points as a sub-character set, taking all characters from a first character set to the segmentation point range as a first sub-character set for a first segmentation point, and taking all characters from a last segmentation point to a last character range as a last sub-character set for a last segmentation point.
S84: and obtaining the character with the largest occurrence number in each sub-character set as a target character to obtain k+1 target characters.
Specifically, counting the occurrence times of each character in each sub-character set aiming at each sub-character set, and taking the character with the largest occurrence times as the target character of the sub-character set to obtain k+1 target characters.
For example, in one embodiment, the first and second embodiments, one of the sub-character sets is { "B", "L", "B") "B", "O", "L", "B" }, the result is that the result contains 12 characters 'B', two characters 'L', and one character 'O', and the character 'B' is used as the target character of the sub-character set.
It should be noted that, in actual use of the license plate, the first target character of the license plate number is a Chinese character, but in recognition of the neural network, the Chinese character may be recognized as an uppercase english letter due to insufficiency of the Chinese character in part of the sliding image, and if the first target character is a non-chinese character, the chinese character with the largest occurrence number in the first sub-character set is obtained as the first target character.
S85: and combining the k+1 target characters according to the sequence of the k+1 preset ranges to obtain the target license plate number.
Specifically, the obtained k+1 target characters are ordered according to the order of the k+1 preset ranges, and the target license plate number is obtained.
For example, in a specific embodiment, there are 7 preset ranges, the target character of the sub-word character set corresponding to the first preset range is "Shanghai", the target character of the sub-word character set corresponding to the second preset range is "Shanghai", the target character of the sub-word character set corresponding to the third preset range is "2", the target character of the sub-word character set corresponding to the fourth preset range is "6", the target character of the sub-word character set corresponding to the fifth preset range is "A", the target character of the sub-word character set corresponding to the sixth preset range is "6", the target character of the sub-word character set corresponding to the seventh preset range is "Shanghai", and the target license plate number is "Shanghai B26A63" obtained by combining.
In this embodiment, the characters in the 3a-2c recognition results are sequentially placed in the character set X, k segmentation points of the character set X are determined according to k+1 preset ranges, and then the q characters in the character set X are segmented by using the k segmentation points to obtain k+1 sub-character sets, the character with the largest occurrence number in each sub-character set is obtained as a target character, k+1 target characters are obtained, then the k+1 target characters are combined according to the order of the k+1 preset ranges to obtain a target license number, so that the characters in the target license image can be obtained without segmentation of the target license image, the problem that part of characters cannot be recognized according to a preset template on the target vehicle image, the accuracy of the recognized characters is not high is avoided, and the accuracy and stability of license plate recognition are improved.
It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present invention.
In an embodiment, a double-layer license plate recognition device is provided, and the double-layer license plate recognition device corresponds to the double-layer license plate recognition method in the embodiment one by one. As shown in fig. 7, the double-layer license plate recognition device includes an acquisition module 10, a preprocessing module 20, a judgment module 30, a layering module 40, a sliding module 50, a termination module 60, a recognition module 70 and a segmentation module 80. The functional modules are described in detail as follows:
the acquisition module 10 is used for acquiring an initial license plate image;
the preprocessing module 20 is used for preprocessing the initial license plate image to obtain a target license plate image with the size of 2ax2b pixels, wherein a and b are positive integers;
and the judging module 30 is used for carrying out layer number detection on the target license plate image by adopting a support vector machine to obtain a target layer number, wherein the target layer number comprises a single layer and a double layer.
And the layering module 40 is configured to cut the target license plate image into two temporary images with the size of 2a×b pixels if the target layer number is two, take the temporary image with the size of 2a×b pixels of the lower layer as the lower layer image, cut the temporary image with the size of 2a×b pixels of the upper layer, and obtain an image with the size of a×b pixels of the central area as the upper layer image.
The sliding module 50 is configured to perform sliding operation from left to right on the upper layer image and the lower layer image by using a sliding window with a preset size of c×b pixels, and each sliding operation is performed on the upper layer image and the lower layer image by using 1 pixel as a step length, so as to obtain a target license plate image within the range of the sliding window as a sliding window image, where c < a and c are positive integers;
a termination module 60, configured to terminate the sliding operation on the upper layer after sliding a-c times on the upper layer image to obtain a-c sliding window images, and terminate the sliding operation on the lower layer after sliding 2a-c times on the lower layer image to obtain 2a-c sliding window images, so as to obtain 3a-2c sliding window images altogether;
the recognition module 70 is used for inputting 3a-2c sliding window images into the convolutional neural network for recognition to obtain 3a-2c recognition results, wherein each recognition result comprises a plurality of characters;
the segmentation module 80 is configured to divide the character set by using a preset segmentation method according to a character set formed by all the characters in the 3a-2c recognition results, obtain a target character from each obtained sub-character set, and determine a target license plate number according to the target character.
Further, the judging module 30 includes:
An extracting unit 31, configured to extract image vector features of the target license plate image;
a calculation unit 32 for calculating the function value of the classification function of the image feature using the following formula:
f(x)＝w T x+b
wherein f (x) is a classification function, w is a weight vector supporting a hyperplane of a vector machine, and w T The method is characterized in that the method is a transpose of a weight vector of a hyperplane, x is an image vector characteristic, and b is a preset offset value;
and a judging unit 33, configured to determine that the target layer number is double if the function value is greater than or equal to 0.
Further, the preprocessing module 20 includes:
the detecting unit 21 is configured to obtain an upper boundary of a license plate and a lower boundary of the license plate in the initial license plate image through an edge detection algorithm;
a determining unit 22, configured to determine a range image of the license plate according to an upper boundary of the license plate and a lower boundary of the license plate;
a correction unit 23 for performing tilt correction on the range image using radon transform to obtain a corrected base image;
and the clipping unit 24 is used for clipping the basic image by taking the gravity center of the basic image as the center to obtain a target license plate image with a size of a multiplied by b pixels.
Further, the detection unit 21 includes:
the denoising subunit 211 is configured to perform noise removal on the initial license plate image through gaussian blur, so as to obtain a denoised license plate image;
A calculating subunit 212, configured to calculate gradient values in the horizontal direction and the vertical direction of the denoised license plate image by using a preset gradient operator, so as to obtain an initial gradient value set;
a refinement subunit 213, configured to perform edge refinement on the initial gradient value set by using a non-maximum suppression manner, so as to obtain a gradient edge with a pixel width;
a filtering subunit 214, configured to filter weak edge points in the gradient edge by using a preset dual threshold value, so as to obtain strong edge points in the gradient edge;
the delimiting subunit 215 is configured to determine an upper boundary of the license plate and a lower boundary of the license plate according to the strong edge points.
Further, the segmentation module 80 includes:
a generating unit 81 for sequentially placing the characters in the 3a-2c recognition results into the character set X to obtain x= { X 1 ，x 2 ，...，x q Q is the total number of characters contained in 3a-2c recognition results, x i For the ith character in character set X, i ε [1, q]Q is a positive integer;
a selecting unit 82, configured to determine k dividing points of the character set X according to k+1 preset ranges, where k is a positive integer and k is < q;
a segmentation unit 83, configured to perform character segmentation on q characters in the character set X by using k segmentation points, so as to obtain k+1 sub-character sets;
A optimizing unit 84, configured to obtain, as target characters, characters with the largest occurrence number in each sub-character set, and obtain k+1 target characters;
and the combining unit 85 is configured to combine the k+1 target characters according to the order of the k+1 preset ranges, so as to obtain the target license plate number.
For specific limitations of the double-layer license plate recognition device, reference may be made to the above limitations of the double-layer license plate recognition method, and no further description is given here. All or part of the modules in the double-layer license plate recognition device can be realized by software, hardware and a combination thereof.
In one embodiment, a computer device is provided, which may be a server, and the internal structure of which may be as shown in fig. 8. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing the trained convolutional neural network model and the input initial license plate image. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program when executed by a processor implements a dual layer license plate recognition method.
In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the steps of the above-described embodiment double-layer license plate recognition method, such as steps S10 to S80 shown in fig. 2. Alternatively, the processor may implement the functions of each module/unit of the double-layer license plate recognition device in the above embodiment when executing the computer program, for example, the functions of the modules 10 to 80 shown in fig. 7. In order to avoid repetition, a description thereof is omitted.
In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, where the computer program when executed by a processor implements the method for identifying a double-layer license plate in the above method embodiment, or where the computer program when executed by a processor implements the functions of each module/unit in the double-layer license plate identifying device in the above device embodiment. In order to avoid repetition, a description thereof is omitted.
The above embodiments are only for illustrating the technical solution of the present invention, and not for limiting the same; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention.
Claims (6)
1. The double-layer license plate recognition method is characterized by comprising the following steps of:
acquiring an initial license plate image;
preprocessing the initial license plate image to obtain a target license plate image with the size of 2a multiplied by 2b pixels, wherein a and b are positive integers;
extracting image vector features of the target license plate image;
the function value of the classification function of the image vector feature is calculated using the following formula:
wherein,,
for the classification function, ++>
Weight vector for supporting hyperplane of vector machine, < +.>
Transpose of the weight vector for the hyperplane,/->
For the image vector feature,/->
Is a preset offset value;
if the function value is greater than or equal to 0, determining that the target layer number is double-layer;
if the target layer number is double, cutting the target license plate image into two temporary images with the size of 2a multiplied by b pixels, taking the temporary images with the size of 2a multiplied by b pixels of the lower layer as the lower layer image, cutting the temporary images with the size of 2a multiplied by b of the upper layer, and obtaining the images with the size of a multiplied by b pixels of the central area as the upper layer image;
sliding operation is carried out in the upper layer image and the lower layer image from left to right respectively by adopting a sliding window with preset size of c multiplied by b pixels and taking 1 pixel as a step length, and a target license plate image in the sliding window range is obtained when each sliding is carried out and is used as a sliding window image, wherein c is a positive integer;
In the upper layer image, after sliding a-c times, stopping sliding operation of the upper layer to obtain a-c sliding window images, after sliding 2a-c times on the lower layer image, stopping sliding operation of the lower layer image to obtain 2a-c sliding window images, and obtaining 3a-2c sliding window images altogether;
inputting 3a-2c sliding window images into a convolutional neural network for recognition to obtain 3a-2c recognition results, wherein each recognition result comprises a plurality of characters;
sequentially placing the characters in the 3a-2c recognition results into a character set X to obtainX= 
Wherein, the method comprises the steps of, wherein,qfor the total number of characters contained in 3a-2c recognition results,/o>
For the i-th character in character set X, -, is->
，qIs a positive integer;
according tok+1 preset ranges, determining the character set XkA plurality of dividing points, wherein,k<qand (2) andkis a positive integer;
usingkEach of the segmentation points is corresponding to the character set XqCharacter segmentation is carried out on each character to obtaink+1 sub-character sets;
acquiring the character with the largest occurrence number in each sub-character set as a target character to obtaink+1 of the target characters;
according tok+1 of said predetermined ranges, forkAnd combining +1 target characters to obtain the target license plate number.
2. The method of claim 1, wherein preprocessing the initial license plate image to obtain a target license plate image with a size of 2a×2b pixels comprises:
acquiring an upper boundary of a license plate and a lower boundary of the license plate in the initial license plate image through an edge detection algorithm;
determining a range image of the license plate according to the upper boundary of the license plate and the lower boundary of the license plate;
performing inclination correction on the range image by using Lato transformation to obtain a corrected basic image;
and cutting the basic image by taking the gravity center of the basic image as the center to obtain the target license plate image with the size of 2a multiplied by 2b pixels.
3. The method for identifying a double-layer license plate as claimed in claim 2, wherein the obtaining the upper boundary of the license plate and the lower boundary of the license plate in the initial license plate image by the edge detection algorithm comprises:
noise removal is carried out on the initial license plate image through Gaussian blur, and a denoising license plate image is obtained;
calculating gradient values of the denoising license plate image in the horizontal direction and the vertical direction by using a preset gradient operator to obtain an initial gradient value set;
Performing edge refinement treatment on the initial gradient value set in a non-maximum value inhibition mode to obtain a gradient edge with a pixel width;
filtering weak edge points in the gradient edge by using a preset double threshold value to obtain strong edge points in the gradient edge;
and determining the upper boundary of the license plate and the lower boundary of the license plate according to the strong edge points.
4. A double-layer license plate recognition device, characterized in that the double-layer license plate recognition device comprises:
the acquisition module is used for acquiring an initial license plate image;
the preprocessing module is used for preprocessing the initial license plate image to obtain a target license plate image with the size of 2a multiplied by 2b pixels, wherein a and b are positive integers;
the extraction unit is used for extracting the image vector characteristics of the target license plate image;
a calculation unit for calculating a function value of a classification function of the image vector feature using the following formula:
wherein,,
for the classification function, ++>
Weight vector for supporting hyperplane of vector machine, < +.>
Transpose of the weight vector for the hyperplane,/->
For the image vector feature,/->
Is a preset offset value;
the judging unit is used for determining that the target layer number is double if the function value is greater than or equal to 0;
The layering module is used for cutting the target license plate image into two temporary images with the size of 2a multiplied by b pixels if the target layer number is double, taking the temporary images with the size of 2a multiplied by b pixels of the lower layer as the lower layer image, cutting the temporary images with the size of 2a multiplied by b pixels of the upper layer, and obtaining the images with the size of a multiplied by b pixels of the central area as the upper layer image;
the sliding module is used for adopting a sliding window with a preset size of c multiplied by b, performing sliding operation from left to right in the upper layer image and the lower layer image respectively by taking 1 pixel as a step length, and acquiring a target license plate image in the range of the sliding window when sliding each time as a sliding window image, wherein c < a and c are positive integers;
the termination module is used for terminating the sliding operation of the upper layer after sliding for a-c times in the upper layer image to obtain a-c sliding window images, and terminating the sliding operation of the lower layer image after sliding for 2a-c times on the lower layer image to obtain 2a-c sliding window images to obtain 3a-2c sliding window images;
the recognition module is used for inputting 3a-2c sliding window images into the convolutional neural network for recognition to obtain 3a-2c recognition results, wherein each recognition result comprises a plurality of characters;
A generating unit, configured to sequentially put the characters in the 3a-2c recognition results into a character set X to obtain x=
Wherein q is the total number of characters contained in 3a-2c recognition results, +.>
For the i-th character in character set X, -, is->
Q is a positive integer;
a selecting unit, configured to determine k dividing points of the character set X according to k+1 preset ranges, where k is a positive integer and k is < q;
the segmentation unit is used for carrying out character segmentation on q characters in the character set X by using k segmentation points to obtain k+1 sub-character sets;
the optimization unit is used for acquiring the character with the largest occurrence frequency in each sub-character set as a target character to obtain k+1 target characters;
and the combination unit is used for combining the k+1 target characters according to the sequence of the k+1 preset ranges to obtain the target license plate number.
5. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the double layer license plate recognition method of any one of claims 1 to 3.
6. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the steps of the double layer license plate recognition method of any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811119253.5A CN109447117B (en) | 2018-09-25 | 2018-09-25 | Double-layer license plate recognition method and device, computer equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811119253.5A CN109447117B (en) | 2018-09-25 | 2018-09-25 | Double-layer license plate recognition method and device, computer equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109447117A CN109447117A (en) | 2019-03-08 |
| CN109447117B true CN109447117B (en) | 2023-06-30 |
Family
ID=65544415
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811119253.5A Active CN109447117B (en) | 2018-09-25 | 2018-09-25 | Double-layer license plate recognition method and device, computer equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109447117B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110210475B (en) * | 2019-05-06 | 2021-05-18 | 浙江大学 | License plate character image segmentation method based on non-binarization and edge detection |
| CN112686246B (en) * | 2019-10-18 | 2024-01-02 | 深圳市优必选科技股份有限公司 | License plate character segmentation method and device, storage medium and terminal equipment |
| CN111695563B (en) * | 2020-06-10 | 2022-07-05 | 北京筑梦园科技有限公司 | Single-layer and double-layer license plate recognition method, server and parking charging system |
| CN111950659B (en) * | 2020-09-01 | 2022-12-09 | 湖南国科微电子股份有限公司 | Double-layer license plate image processing method and device, electronic equipment and storage medium |
| CN112215224A (en) * | 2020-10-22 | 2021-01-12 | 深圳市平方科技股份有限公司 | Deep learning-based trailer number identification method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106529532A (en) * | 2016-11-07 | 2017-03-22 | 湖南源信光电科技有限公司 | License plate identification system based on integral feature channels and gray projection |
| CN107103317A (en) * | 2017-04-12 | 2017-08-29 | 湖南源信光电科技股份有限公司 | Fuzzy license plate image recognition algorithm based on image co-registration and blind deconvolution |
| WO2018028306A1 (en) * | 2016-08-11 | 2018-02-15 | 杭州海康威视数字技术股份有限公司 | Method and device for recognizing license plate number |
| CN108073928A (en) * | 2016-11-16 | 2018-05-25 | 杭州海康威视数字技术股份有限公司 | A kind of licence plate recognition method and device |
| CN108416348A (en) * | 2018-01-29 | 2018-08-17 | 重庆邮电大学 | Plate location recognition method based on support vector machines and convolutional neural networks |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9760789B2 (en) * | 2012-04-17 | 2017-09-12 | Conduent Business Services, Llc | Robust cropping of license plate images |
-
2018
- 2018-09-25 CN CN201811119253.5A patent/CN109447117B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018028306A1 (en) * | 2016-08-11 | 2018-02-15 | 杭州海康威视数字技术股份有限公司 | Method and device for recognizing license plate number |
| CN106529532A (en) * | 2016-11-07 | 2017-03-22 | 湖南源信光电科技有限公司 | License plate identification system based on integral feature channels and gray projection |
| CN108073928A (en) * | 2016-11-16 | 2018-05-25 | 杭州海康威视数字技术股份有限公司 | A kind of licence plate recognition method and device |
| CN107103317A (en) * | 2017-04-12 | 2017-08-29 | 湖南源信光电科技股份有限公司 | Fuzzy license plate image recognition algorithm based on image co-registration and blind deconvolution |
| CN108416348A (en) * | 2018-01-29 | 2018-08-17 | 重庆邮电大学 | Plate location recognition method based on support vector machines and convolutional neural networks |
Non-Patent Citations (1)
| Title |
|---|
| 基于多尺度模板匹配和部件模型的车牌字符分割方法;裴明涛等;《北京理工大学学报》;第第34卷卷(第第9期期);第961-965页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109447117A (en) | 2019-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109492642B (en) | License plate recognition method, license plate recognition device, computer equipment and storage medium | |
| CN109447117B (en) | Double-layer license plate recognition method and device, computer equipment and storage medium | |
| CN110414507B (en) | License plate recognition method and device, computer equipment and storage medium | |
| US20200184187A1 (en) | Feature extraction and matching for biometric authentication | |
| CN110569721A (en) | Recognition model training method, image recognition method, device, equipment and medium | |
| KR101848019B1 (en) | Method and Apparatus for Detecting Vehicle License Plate by Detecting Vehicle Area | |
| CN109472262A (en) | Licence plate recognition method, device, computer equipment and storage medium | |
| CN110660072B (en) | Method and device for identifying straight line edge, storage medium and electronic equipment | |
| CN112528868B (en) | Illegal line pressing judgment method based on improved Canny edge detection algorithm | |
| CN106981077A (en) | Infrared image and visible light image registration method based on DCE and LSS | |
| CN114049499A (en) | Target object detection method, apparatus and storage medium for continuous contour | |
| CN106446921A (en) | High-voltage power transmission line barrier identification method and apparatus | |
| CN109255792B (en) | Video image segmentation method and device, terminal equipment and storage medium | |
| CN109522768A (en) | The recognition methods of QR code and device | |
| CN111080542A (en) | Image processing method, image processing apparatus, electronic device, and storage medium | |
| WO2024016632A1 (en) | Bright spot location method, bright spot location apparatus, electronic device and storage medium | |
| Duan et al. | Real time road edges detection and road signs recognition | |
| CN108090425B (en) | Lane line detection method, device and terminal | |
| CN116434071B (en) | Determination method, determination device, equipment and medium for normalized building mask | |
| CN112184723B (en) | Image processing method and device, electronic equipment and storage medium | |
| CN113781413A (en) | Electrolytic capacitor positioning method based on Hough gradient method | |
| CN111915645B (en) | Image matching method and device, computer equipment and computer readable storage medium | |
| CN109871779B (en) | Palm print identification method and electronic equipment | |
| CN113159037B (en) | Picture correction method, device, computer equipment and storage medium | |
| CN115841632A (en) | Power transmission line extraction method and device and binocular ranging method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |