CN106096607A - A kind of licence plate recognition method - Google Patents

A kind of licence plate recognition method Download PDF

Info

Publication number
CN106096607A
CN106096607A CN201610408067.8A CN201610408067A CN106096607A CN 106096607 A CN106096607 A CN 106096607A CN 201610408067 A CN201610408067 A CN 201610408067A CN 106096607 A CN106096607 A CN 106096607A
Authority
CN
China
Prior art keywords
edge
edge group
license plate
posting
calculated
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201610408067.8A
Other languages
Chinese (zh)
Inventor
欧阳建权
唐欢容
李达
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiangtan University
Original Assignee
Xiangtan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiangtan University filed Critical Xiangtan University
Priority to CN201610408067.8A priority Critical patent/CN106096607A/en
Publication of CN106096607A publication Critical patent/CN106096607A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/63—Scene text, e.g. street names
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks

Abstract

The present invention provides a kind of licence plate recognition method, in conjunction with convolutional neural networks and viterbi algorithm, comprises the following steps: (1) image sparse；(2) combination edge group；(3) similarity of edge group is calculated；(4) weight w of each edge group is calculatedb；(5) scoring is calculated；(6) accurate License Plate frame is selected；(7) in step (6) to License Plate frame carry out convolution operation and pond operation obtains characteristic pattern；(8) use convolution god's network recognizer that characteristic pattern is scanned, obtain character string；(9) use viterbi algorithm that the character string in step (8) is calculated specific character sequence.The method can tackle the electronic image of complex condition shooting, improves the recognition accuracy of Vehicle License Plate Recognition System, and this technology is before character recognition, it is not necessary to split picture, it is to avoid the mistake caused owing to Character segmentation is inaccurate.

Description

A kind of licence plate recognition method
Technical field
The invention belongs to field of image recognition, particularly to a kind of licence plate recognition method based on convolutional neural networks.
Background technology
Along with popularizing on a large scale of automobile has also grown a series of unsafe factor to us while offering convenience.As Driver's disregard of law regulation is arbitrarily driven a vehicle, parking lack of standardization so that traffic safety cannot ensure.Country uses at present This problem tackled by electronic police, and wherein Car license recognition is a most important ring.
Vehicle License Plate Recognition System (License Plate Recognition System), it is therefore an objective to road traffic crossing electricity The car plate of the vehicle violating traffic law of sub-video camera shooting identifies automatically, and its primitive form is: by illegal violating the regulations The image input system of vehicle, system returns the number-plate number of this vehicle.Vehicle License Plate Recognition System can effectively reduce artificial work Measure, improve work efficiency.
Car license recognition is exactly extraction number-plate number information from a pictures.License plate recognition technology has and is widely applied very much model Enclose, such as high speed charge station, pay parking, traffic intersection management etc..The quality of picture is that to affect recognition accuracy main Factor, but due to some irresistible factor (as rained or when haze is more serious), or illumination between the lights When deficiency, the picture of video camera shooting is not fully up to expectations, and this is accomplished by us and captures this by a series of technological means A little difficult problems.
Convolutional neural networks (Convolutional Neural Network, CNN), is that a kind of outstanding image recognition is calculated Method, pulls out to obtain head in computer vision field, the large-scale image classification of this algorithm and large-scale image identification challenge match repeatedly Raise.Convolutional neural networks is a kind of deep neural network, and this algorithm is shared by receptive field (Receptive Field) and weights (Shared Weights) carries out convolution algorithm and pond computing, and reducing neutral net needs the parameter of training and keep image Translation invariance, reduces computation complexity, improves performance.
Existing Car license recognition flow process includes License Plate, Character segmentation, character recognition.In complex environment, video camera institute The picture of shooting often occurs that noise is excessive or picture distorts, and tilts, causes the character in image smudgy or tight Close connected, thus Character segmentation difficulty can be caused, therefore cause identifying accurately.
So, existing license plate recognition technology Shortcomings, it is necessary to improve.
Summary of the invention
In traditional algorithm of locating license plate of vehicle, the location of car plate is relied primarily on the feature of image, such as edge feature, face Color characteristic, textural characteristics etc..But these features are easy to be disturbed by the factor such as illumination or image background, cause extracting spy Levy difficulty, thus affect the accuracy rate of Vehicle License Plate Recognition System.In order to avoid the problems referred to above, the present invention carries based on convolutional neural networks Go out a kind of license plate locating method based on object detection algorithms.
The present invention is to provide a kind of combination convolutional neural networks and the licence plate recognition method of viterbi algorithm, and the method can be answered Electronic image to complex condition shooting, improves the recognition accuracy of Vehicle License Plate Recognition System, and this technology is in character recognition Before, it is not necessary to picture is split, thus avoids the mistake caused owing to Character segmentation is inaccurate.
At degree of depth learning areas, object detection is that the degree of depth learns one of absorbed direction, and the direction has breakthrough every year Progress, utilize the degree of depth learning algorithm object can not only be detected, moreover it is possible to object is carried out semantics recognition, the present invention is opened by this Send out, it is proposed that use the object detecting method in degree of depth learning algorithm to carry out License Plate.The present invention proposes a kind of new License plate locating method, first inputs image to be identified, then uses the Edge Boxes method proposed at list of references to generate The a series of candidate frame being probably license plate area, one convolutional neural networks filtration fraction non-license plate area candidate of retraining Frame, all comprises license plate area owing to there may be multiple candidate frame, therefore next needs to use non-maxima suppression algorithm to eliminate Unnecessary candidate frame, finally is adjusted navigating to license plate area to candidate frame.
In order to solve the problems referred to above, the technical scheme that the present invention provides is as follows:
A kind of licence plate recognition method, comprises the following steps:
(1) image sparse: use Structure Edge detector that input picture is calculated each pixel in image The calculating skirt response of point, rarefaction, obtain the image of a rarefaction；
(2) combination edge group: to the limit in the rarefaction image generated in step (1), with pixel for substantially counting Calculate unit, calculate the sum of orientation angle difference between neighbor pixel, if less than M, then it is assumed that these pixels similar Degree height, forms edge group, continues to calculate next pixel；If greater than M, then stop calculating, by calculated Pixel one edge group of composition；
(3) similarity of edge group is calculated: be designated as S by step (2) obtains all edge group, (si,sj)∈ Then S uses formula (1-1) to calculate si,sjSimilarity a (the s of two edge groupi, sj),
a(si,sj)=| cos (θi-θij)cos(θj-θij)|γ (1-1)
Wherein: θijRepresent xi,xjBetween angle, xi,xjFor si,sjAverage distance, θi,θjFor si,sjAverage angle, γ is used for adjusting the similarity between two edge group；
(4) weight w of each edge group is calculatedb: use sliding window to scan whole picture, often slide one After step obtains the Edge group in a window and their similarity, calculate each edge according to formula (1-2) The weight w of groupb；
w b ( s i ) = 1 - max T Π j | T | - 1 a ( t j , t j + 1 ) - - - ( 1 - 2 )
Wherein: T is the set of edge in each edge group, | T | represents the number of the edge in T, and j represents in T Edge sequence number, starts counting up from 0.tjAnd tj+1Represent edge, a (t adjacent in Tj, tj+1) represent tjAnd tj+1Between two edge Similarity；
(5) scoring is calculated: calculate the scoring h of posting according to formula (1-3)b, select the profile diagram of score Top-n.
h b = Σ i w b ( s i ) m i 2 ( b w + b h ) k - - - ( 1 - 3 )
Wherein: hbRepresent the scoring of posting, bwRepresent the width of posting, bhRepresent the height of location, miRepresent pixel Range value, i represents that the span of edge group in posting, i is the number of edge group, and k takes fixed value 1.5, uses Adjust the size of window；
(6) accurate License Plate frame is selected；
(7) in step (6) to License Plate frame region carry out convolution operation and pond operation obtains characteristic pattern；
(8) use convolution god's network recognizer that characteristic pattern is carried out slip scan, obtain character string；
(9) use viterbi algorithm that the character string in step (8) is calculated specific character sequence, this sequence It is Recognition of License Plate Characters result.
In the present invention, " si,sjMean place " be that in edge group, each edge and horizontal coordinate are shown in the flat of distance Average.“si,sjAverage angle " be the meansigma methods of the angle of each edge and horizontal coordinate in edge group.
In the present invention, the rarefaction described in step (1) uses NMS algorithm marginal information to carry out rarefaction.
In the present invention, described in step (2), M value is threshold value, is a fixing value, M=pi/2.
In the present invention, described in step (3), γ is generally 1.6-2.2, and such as 2.
In the present invention, in step (4): if wb=1, then this edge group is judged to contour of object in this window A part, calculates scoring for next step；If wb=0, then judge that this edge group is not belonging to contour of object in window, the most not For calculating scoring.
In the present invention, step (6) select accurate License Plate frame use non-maxima suppression method.
In the present invention, step (6) selects accurate License Plate frame, concretely comprises the following steps:
A) one convolutional neural networks model of training, filters out the profile diagram not comprising license plate area in step (5), obtains Comprise the posting of license plate area；
B) NMS algorithm is used to eliminate unnecessary posting；
C) posting adjusts, the posting each edge obtained in first expansion step b), then uses sobel in this region Operator carries out rim detection, obtains accurate License Plate frame.
In the present invention, posting each edge 5-40% obtained in expansion step b) in step c), preferably 10- 30%.More preferably 15-25.
In the present invention, the concrete operations of step (8) are: use convolution god's network recognizer of a fixed dimension to spy Levying figure one step-length is that 1 pixel carries out slip scan, obtains character string.
In the present invention, in order to obtain higher recall ratio, more profile diagrams, i.e. n can be selected to take greater value.n Value according to experiment effect adjust.Usually, n is 2-500, preferably 10-300, more elects 20-200 as, such as 40,60,80 or 100。
In the present invention, the thought of Edge Boxes method is based on such a intuition, if in a bounding box (Edge Boxes) if in contain the profile of some so this contain one in might mean that this bounding box Object.The research method of the present invention is to utilize marginal information (Edge), utilizes the profile number in bounding box, carries out object inspection Survey.Bounding box (Edge Boxes) method carries out the detailed step of object detection:
First it is the knot proposed according to document " Structured Forests for Fast Edge Detection " Structure edge algorithms, calculates the skirt response of each pixel, uses a Structured Edge detector to obtain limit Edge structure chart.Then use NMS (Non-Maximal Suppression) method, obtain the figure of a marginal information rarefaction Picture.
2., after rarefaction edge image obtained in the previous step, approximation edge line segment on the same line is put together Go into an edge group, use greedy algorithm to calculate the curvature on adjacent 8 limits, if direction change is less than a threshold value (pi/2), then be defined as an edge group these 8 limits.
3. calculate the similarity of edge group, it is assumed that the collection obtaining all edge group in previous step is combined into S, (si,sj) ∈ S then use formula (3-1) calculate si,sjThe similarity of two edge group, θ in formula (3-1)ijRepresent xi,xjBetween angle, xi,xjFor si,sjAverage distance, θi,θjFor si,sjAverage angle, γ be used for adjust two edge Similarity between group, general value is 2 in actual applications.
a(si,sj)=| cos (θi-θij)cos(θj-θij)|γ (0-1)
4. use sliding window to scan whole picture, often sliding move a step the Edge group that obtains in a window with And after their similarity, calculate the weight w of each edge group according to formula (3-2)b.If wb=1, then this Edge group is judged to a part for contour of object in this window, if wb=0, then judge that this edge group is not belonging to window Middle contour of object, wherein T is the path that contour of object edge group arranges in order.
w b ( s i ) = 1 - max T Π j | T | - 1 a ( t j , t j + 1 ) - - - ( 0 - 2 )
5. last, calculate scoring according to formula (3-3), select Top-n.
h b = Σ i w b ( s i ) m i 2 ( b w + b h ) k - - - ( 0 - 3 ) .
In the present invention, non-maxima suppression (Non-maximum suppression), i.e. search local maximum based on Greedy strategy, suppresses non-maximum.
In the present invention, bounding box (Bounding-Box) adjusts and uses following methods:
1. each edge of initial posting is expanded 20% (according to experiment test, expand 20% effect best)
2. use sobel operator to carry out vertical edge detection in the region that step 1 limits
3. the floor projection of the vertical edge characteristic pattern obtained according to step 2 obtains top and the bottom of license plate area.
4. obtain right boundary by upright projection.
5. rotate posting so that posting agrees with car plate.
In the present invention, viterbi algorithm is that application dynamic programming algorithm widely is in one drawing Find shortest path, the most often apply and shifting general according to observable state and state in hiding Markov model Rate solves most possible hidden state.For a specific HMM, viterbi algorithm is used to find life Become the most probable hidden state sequence of an observation sequence.Utilize the time invariance of probability, by avoiding calculating state to turn Change in figure the probability of each paths to reduce the complexity of problem.Viterbi algorithm saves one for each state Local probability.Local probability is that the path of instruction arrives certain shape probability of state.
As t=T, the local probability of these final states that viterbi algorithm is arrived is according to optimum (most probable) Path arrives this shape probability of state.Therefore, select maximum of which one, and hidden state path found out in backtracking, it is simply that The optimal solution of this problem.
Viterbi algorithm is not simply given for certain time point most probable hidden state of selection, but base Decision-making is done therefore, if there being the event of " extraordinary " to occur, for Viterbi in observation sequence in global sequence The result of algorithm also affects not quite.This is in characters on license plate string manipulation, when the receptive field of convolutional neural networks has detected part When the image information of adjacent character causes identifying mistake, this error message can be got rid of by viterbi algorithm, finds out correct road Footpath.
The present invention is by viterbi algorithms selection character string, if ciFor the weights of limit i, siIt is the start node of figure, diIt is last node of figure, there is also various nodes between which.Viterbi algorithm mainly does following work Making, each node has a cumulative weights vn.These accumulated values are that random order calculates, it is necessary to meet local in figure suitable Sequence is a directed acyclic graph.The weight initialization of start node is 0, remembers vstart=0.The weights of other nodes are by father node The carrying out of Weighted Recursive calculates.By these limits Un={ arc i with destination di=n}: When arriving last node, we can obtain a vend, it is minimum in the weights on all of limit.For terrible Node in viterbi path and limit, we recall these limits and node as follows.Open from last node Begin, nTLater node, recurrence finds minimum limitUntil arriving first node.This sequence is just It is intended to the character subsequence selected.
Compared with prior art, the present invention has a following Advantageous Effects:
1, a kind of license plate locating method based on Edge Boxes algorithm is proposed.
2, character recognition algorithm based on convolutional neural networks, convolutional neural networks can feature group from " low level " Close the learning feature to " higher level " such that it is able to the license plate image of shooting in reply complex environment.
3, the present invention uses character string recognizer, and this algorithm can be prevented effectively from existing license plate recognition technology due to word Symbol splits the inaccurate identification mistake caused.
Accompanying drawing explanation
Fig. 1 is one overall framework figure of the present invention.
Fig. 2 is the License Plate flow chart in the present invention.
Fig. 3 is the character string identification process figure in the present invention.
Fig. 4 is Viterbi Transformer training structure figure of the present invention.
Detailed description of the invention
In order to make those skilled in the art be better understood from the technical scheme of the application, implement below in conjunction with the application Accompanying drawing in example, carries out clear, complete description to the technical scheme in the embodiment of the present application.Should be appreciated that described herein Specific embodiment only in order to explain the present invention, be not intended to limit the present invention.
A kind of licence plate recognition method, comprises the following steps:
(1) image sparse: use Structure Edge detector that input picture is calculated each pixel in image The calculating skirt response of point, rarefaction, obtain the image of a rarefaction；
(2) combination edge group: to the limit in the rarefaction image generated in step (1), with pixel for substantially counting Calculate unit, calculate the sum of orientation angle difference between neighbor pixel, if less than M, then it is assumed that these pixels similar Degree height, forms edge group, continues to calculate next pixel；If greater than M, then stop calculating, by calculated Pixel one edge group of composition；
(3) similarity of edge group is calculated: be designated as S by step (2) obtains all edge group, (si,sj)∈ Then S uses formula (1-1) to calculate si,sjSimilarity a (the s of two edge groupi, sj),
a(si,sj)=| cos (θi-θij)cos(θj-θij)|γ (1-1)
Wherein: θijRepresent xi,xjBetween angle, xi,xjFor si,sjMean place, θi,θjFor si,sjAverage angle, γ is used for adjusting the similarity between two edge group；
(4) weight w of each edge group is calculatedb: use sliding window to scan whole picture, often slide one After step obtains the Edge group in a window and their similarity, calculate each edge according to formula (1-2) The weight w of groupb；
w b ( s i ) = 1 - max T Π j | T | - 1 a ( t j , t j + 1 ) - - - ( 1 - 2 )
Wherein: T is the set of edge in each edge group, | T | represents the number of the edge in T, and j represents in T Edge sequence number, starts counting up from 0.tjAnd tj+1Represent edge, a (t adjacent in Tj, tj+1) represent tjAnd tj+1Between two edge Similarity；
(5) scoring is calculated: calculate the scoring h of posting according to formula (1-3)b, select the profile diagram of score Top-n.
h b = Σ i w b ( s i ) m i 2 ( b w + b h ) k - - - ( 1 - 3 )
Wherein: hbRepresent the scoring of posting, bwRepresent the width of posting, bhRepresent the height of location, miRepresent pixel Range value, i represents that the span of edge group in posting, i is the number of edge group, and k takes fixed value 1.5, uses Adjust the size of window；
(6) accurate License Plate frame is selected；
(7) in step (6) to License Plate frame region carry out convolution operation and pond operation obtains characteristic pattern；
(8) use convolution god's network recognizer that characteristic pattern is carried out slip scan, obtain character string；
(9) use viterbi algorithm that the character string in step (8) is calculated specific character sequence, this sequence It is Recognition of License Plate Characters result.
In the present invention, the rarefaction described in step (1) uses NMS algorithm marginal information to carry out rarefaction.
In the present invention, described in step (2), M value is threshold value, is a fixing value, M=pi/2.
In the present invention, described in step (3), γ is 2.
In the present invention, in step (4): if wb=1, then this edge group is judged to contour of object in this window A part, calculates scoring for next step；If wb=0, then judge that this edge group is not belonging to contour of object in window, the most not For calculating scoring.
In the present invention, step (6) select accurate License Plate frame use non-maxima suppression method.
In the present invention, step (6) selects accurate License Plate frame, concretely comprises the following steps:
A) one convolutional neural networks model of training, filters out the profile diagram not comprising license plate area in step (5), obtains Comprise the posting of license plate area；
B) NMS algorithm is used to eliminate unnecessary posting；
C) posting adjusts, the posting each edge obtained in first expansion step b), then uses sobel in this region Operator carries out rim detection, obtains accurate License Plate frame.
In the present invention, posting each edge 5-40% obtained in expansion step b) in step c), preferably 10- 30%.More preferably 15-25.
In the present invention, the concrete operations of step (8) are: use convolution god's network recognizer of a fixed dimension to spy Levying figure one step-length is that 1 pixel carries out slip scan, obtains character string.
Embodiment 1
A kind of licence plate recognition method, comprises the following steps:
Step A, License Plate:
A1, use Structure Edge detector calculate the calculating limit of each pixel in image to input picture Edge responds, and then uses NMS algorithm marginal information to carry out rarefaction, obtains the image of a rarefaction；
A2, in A1 step generate rarefaction image in limit, with pixel for basic calculating unit, calculate adjacent picture The sum of the orientation angle difference between vegetarian refreshments, if the orientation angle difference between the point of two edges and more than (pi/2), by this A little limits are combined into edge group；
A3, the similarity of calculating edge group, it is assumed that in previous step, obtain all edge group be designated as S, (si,sj) ∈ S then use formula (3-1) calculate si,sjThe similarity of two edge group, θ in formula (1-1)ijRepresent xi,xjBetween angle, xi,xjFor si,sjMean place, θi,θjFor si,sjAverage angle, γ be used for adjust two edge Similarity between group, general value is 2 in actual applications.
a(si,sj)=| cos (θi-θij)cos(θj-θij)|γ (1-1)
A4, one sliding window of use scan whole picture, the often sliding Edge group obtained in a window that moves a step And after their similarity, calculate the weight w of each edge group according to formula (1-2)b.If wb=1, then this Edge group is judged to a part for contour of object in this window, if wb=0, then judge that this edge group is not belonging to window Middle contour of object, wherein T is the path that contour of object edge group arranges in order.
w b ( s i ) = 1 - max T Π j | T | - 1 a ( t j , t j + 1 ) - - - ( 1 - 2 )
T is the set of edge in each edge group, and | T | represents the number of the edge in T, and j represents edge sequence in T Number, start counting up from 0.tjAnd tj+1Represent edge, a (t adjacent in Tj, tj+1) represent tjAnd tj+1Similar between two edge Degree；
A5, last, calculate scoring according to formula (1-3), select the profile diagram of score Top-n.
h b = Σ i w b ( s i ) m i 2 ( b w + b h ) k - - - ( 1 - 3 )
Wherein: hbRepresent the scoring of posting, bwRepresent the width of posting, bhRepresent the height of location, miRepresent pixel Range value, i represents that the span of edge group in posting, i is the number of edge group, and k takes fixed value 1.5, uses Adjust the size of window；
A6, one convolutional neural networks model of training, filter out the profile diagram not comprising license plate area in A5 step, obtain Comprise the posting of license plate area.
A7, use NMS algorithm eliminate unnecessary posting.
A8, posting adjust, and first the posting each edge arrived in A7 step are expanded 20%, then make in this region Carry out rim detection with sobel operator, obtain accurate License Plate frame.
Step B, character string identification:
B1, in step A to license plate area carry out convolution operation and pond operation obtains characteristic pattern；
B2, convolution god's network recognizer of one fixed dimension of use are that 1 pixel is slided to characteristic pattern one step-length Scanning, obtains character string；
Obtaining specific character sequence in B3, the use viterbi algorithm character string to producing in B2 step, this sequence is i.e. For Recognition of License Plate Characters result.
Embodiment 2
Data set for experiment is mainly derived from the data set in open source projects EasyPR, and then I uses digital phase Machine is in rainy day, greasy weather, at dusk acquire 900 pictures respectively, and after original image is split binary conversion treatment and EasyPR number Combine according to training set.Whole data set has 49385 character pictures of picture, and totally 67 classes comprise numeral, capitalization English Letter and Chinese character, all of picture is all converted to gray level image and picture size is normalized to 24*24.
Experimental Hardware facility: tide server, model NF5240M3；CPU model: E5-2407V2；Internal memory: 8G.
Software environment: operating system: Ubuntu14.04；Degree of depth learning framework caffe；Matlab R2012b.
Use Edge Box Detector to detect car plate picture, generate candidate frame.Arrange Edge Box Detector's Step-length is 0.65, and the threshold value arranging Non-Maximum Suppression algorithm is 0.75.
Training cnn model, filters candidate frame.In CNN model, the size of input picture must be fixing, previous step The candidate frame generated is not of uniform size, needs the candidate frame region to extracting to be normalized.
NMS eliminates unnecessary posting.
Using top left co-ordinate and lower right corner coordinate to describe the position of posting, zero is in initial alignment frame Heart point.If the coordinate of original image is (x1, y1, x2, y2), wherein (x1, y1) represents top left co-ordinate, and (x2, y2) represents right Lower angular coordinate.The coordinate expanded after 20% by posting is (1.2*x1,1.2*y1,1.2*x2,1.2*y2).
Table 1 License Plate results contrast
As can be seen from the above table, the method in this paper experiment knot in recall ratio and precision ratio are better than Hsu method Really.In Hsu method use Expectation-Maximization (EM) algorithm in image edge feature cluster thus Car plate is positioned.EdgeBox algorithm used herein combines CNN grader and positions car plate, and EdgeBox calculates Method is also to detect car plate by marginal information in image, recall ratio difference in two ways little.Due to license plate area Middle context of methods also uses a CNN grader and filters non-license plate area, so the precision ratio of context of methods compares Hsu method There is obvious advantage.
Embodiment 3
CNN model training
Cnn experiment is all based in caffe framework and MATLAB completing, and license plate image to be identified is converted into gray-scale map Picture, is normalized into size 28*128 pixel size and the right and left respectively increases the limit of 10 pixels for retaining edge letter Breath.Details is described as follows:
Activation primitive use Relu function in propagated forward:
F (x)=max (0, x) (0-4)
Convolutional layer calculates:
x j l = f ( Σ i ∈ M j x i l - 1 * k i j l + b j l ) - - - ( 0 - 5 )
Wherein, Mj is characterized set of graphs；
Full articulamentum:
h w , b ( x ) = f ( Σ i = 1 n W i x i + b ) - - - ( 0 - 6 )
Sub sampling layer, the effect of sub sampling layer is the downsampled version generating a characteristic pattern, if there being N number of input figure Picture, then can produce N number of output image, and the input picture through down-sampling can be less than input picture.
x j l = f ( β j l d o w n ( x J l - 1 ) + b j l ) - - - ( 0 - 7 )
Wherein,Represent pond method, the maximum pond method used herein.
Viterbi Transformer trains:
Fig. 4 is Viterbi Transformer training structure figure, it is intended that use viterbi algorithm to find out optimum road Footpath, and this path just represents the order that this character string is correct.So our target is to adjust parameter by training so that just Really the weights in the path at character sequence place can minimize value.First we define a loss function, uses back propagation The gradient of algorithm counting loss function, uses Newton method to carry out regularized learning algorithm rate.As shown in Figure 4, we used in training The correct character string order of one path Selector labelling carries out propagated forward training, wherein CdforwThe propagation values of subpath, CforwFor the propagated forward value of whole figure, Edforw=Cdforw-Cforw, perfect condition is Edforw=0, so should use up in training Possible makes EdforwLevel off to 0.
Character recognition experiment herein is main by comparing context of methods and Hsu method and ANN at tri-subnumbers of AOLP According to the discrimination concentrated, experimental result is as shown in table 2 below.
Table 2 Recognition of License Plate Characters results contrast
Hsu method in list of references extracts character feature first by LBP, then uses LDA to classify compared to ANN side The experimental result odds ratio arrived that method carries out testing in AOLP data set is more apparent.But it is in this paper based on convolutional Neural The character recognition accuracy rate of network is higher than Hsu method, and analyzing reason is that the data in AOLP data set enumerate varying environment The picture of middle acquisition, single character feature extracting method is not enough to tackle changeable task, and convolutional neural networks can lead to Cross multiple low-dimensional that different disposal layer learns in the image feature to higher-dimension, so that model has higher identification ability.

Claims (9)

1. a licence plate recognition method, comprises the following steps:
(1) image sparse: use Structure Edge detector that input picture calculates each pixel in image Calculate skirt response, rarefaction, obtain the image of a rarefaction；
(2) combination edge group: to the limit in the rarefaction image generated in step (1), with pixel for basic calculating list Position, the sum of the orientation angle difference between calculating neighbor pixel, if less than M, then it is assumed that the similarity of these pixels is high, Composition edge group, continues to calculate next pixel；If greater than M, then stop calculating, the pixel that will have calculated Form an edge group；
(3) similarity of edge group is calculated: step (2) will obtain all edge groupIt is designated asS, (si,sj) ∈ S is right Rear use formula (1-1) calculates si,sjSimilarity a (the s of two edge groupi, sj),
a(si,sj)=| cos (θi-θij)cos(θj-θij)|γ (1-1)
Wherein: θijRepresent xi,xjBetween angle, xi,xjFor si,sjMean place, θi,θjFor si,sjAverage angle, γ use Adjust the similarity between two edge group；
(4) weight w of each edge group is calculatedb: use a sliding window to scan whole picture, often slide and move a step After Edge group in a window and their similarity, calculate each edge group according to formula (1-2) Weight wb；
w b ( s i ) = 1 - max T Π j | T | - 1 a ( t j , t j + 1 ) - - - ( 1 - 2 )
Wherein: T is the set of edge in each edge group, | T | represents the number of the edge in T, and j represents edge sequence in T Number, start counting up from 0.tjAnd tj+1Represent two adjacent edge, a (t in Tj, tj+1) represent tjAnd tj+1Between two edge Similarity；
(5) scoring is calculated: calculate the scoring h of posting according to formula (1-3)b, the profile diagram of selection score Top-n:
h b = Σ i w b ( s i ) m i 2 ( b w + b h ) k - - - ( 1 - 3 )
Wherein: hbRepresent the scoring of posting, bwRepresent the width of posting, bhRepresent the height of location, miRepresent the amplitude of pixel Value, i represents that the span of edge group in posting, i is the number of edge group, and k takes fixed value 1.5, is used for adjusting The size of whole window；
(6) accurate License Plate frame is selected；
(7) in step (6) to License Plate frame region carry out convolution operation and pond operation obtains characteristic pattern；
(8) use convolution god's network recognizer that characteristic pattern is carried out slip scan, obtain character string；
(9) using viterbi algorithm that the character string in step (8) is calculated specific character sequence, this sequence is Recognition of License Plate Characters result.
Method the most according to claim 1, it is characterised in that: the rarefaction described in step (1) uses NMS algorithm limit Edge information carries out rarefaction.
3. according to the method according to any one of claim 1-3, it is characterised in that: described in step (2), M value is threshold value, is One fixing value, M=pi/2.
4. according to the method according to any one of claim 1-3, it is characterised in that: described in step (3), γ is 2.
5. according to the method according to any one of claim 1-4, it is characterised in that: in step (4): if wb=1, then this Edge group is judged to a part for contour of object in this window, calculates scoring for next step；If wb=0, then judging should Edge group is not belonging to contour of object in window, then be not used in calculating scoring.
6. according to the method according to any one of claim 1-5, it is characterised in that: step (6) select accurate car plate fixed Position frame uses non-maxima suppression method.
7. according to the method according to any one of claim 1-5, it is characterised in that: step (6) select accurate car plate fixed Position frame, concretely comprises the following steps:
A) one convolutional neural networks model of training, filters out the profile diagram not comprising license plate area in step (5), is comprised The posting of license plate area；
B) NMS algorithm is used to eliminate unnecessary posting；
C) posting adjusts, the posting each edge obtained in first expansion step b), then uses sobel operator in this region Carry out rim detection, obtain accurate License Plate frame.
Method the most according to claim 7, it is characterised in that: the posting every obtained in expansion step b) in step c) Limit 5-40%, preferably 10-30%.More preferably 15-25%.
9. according to the method according to any one of claim 1-8, it is characterised in that: the concrete operations of step (8) are: use one Convolution god's network recognizer of individual fixed dimension is that 1 pixel carries out slip scan to characteristic pattern one step-length, obtains character string.
CN201610408067.8A 2016-06-12 2016-06-12 A kind of licence plate recognition method Pending CN106096607A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610408067.8A CN106096607A (en) 2016-06-12 2016-06-12 A kind of licence plate recognition method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610408067.8A CN106096607A (en) 2016-06-12 2016-06-12 A kind of licence plate recognition method

Publications (1)

Publication Number Publication Date
CN106096607A true CN106096607A (en) 2016-11-09

Family

ID=57228594

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610408067.8A Pending CN106096607A (en) 2016-06-12 2016-06-12 A kind of licence plate recognition method

Country Status (1)

Country Link
CN (1) CN106096607A (en)

Cited By (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106557768A (en) * 2016-11-25 2017-04-05 北京小米移动软件有限公司 The method and device is identified by word in picture
CN106650740A (en) * 2016-12-15 2017-05-10 深圳市华尊科技股份有限公司 License plate identification method and terminal
CN106845482A (en) * 2017-01-23 2017-06-13 南京邮电大学盐城大数据研究院有限公司 A kind of license plate locating method
CN106934396A (en) * 2017-03-09 2017-07-07 深圳市捷顺科技实业股份有限公司 A kind of license number search method and system
CN107229929A (en) * 2017-04-12 2017-10-03 西安电子科技大学 A kind of license plate locating method based on R CNN
CN107239778A (en) * 2017-06-09 2017-10-10 中国科学技术大学 The licence plate recognition method of efficiently and accurately
CN107256384A (en) * 2017-05-22 2017-10-17 汕头大学 A kind of card recognition and method of counting based on image and signal transacting
CN107369162A (en) * 2017-07-21 2017-11-21 华北电力大学（保定） A kind of generation method and system of insulator candidate target region
CN107564324A (en) * 2017-07-28 2018-01-09 山东大学 A kind of garage intelligent power saving method and system
CN107844750A (en) * 2017-10-19 2018-03-27 华中科技大学 A kind of water surface panoramic picture target detection recognition methods
CN107944457A (en) * 2017-11-23 2018-04-20 浙江清华长三角研究院 Drawing object identification and extracting method under a kind of complex scene
CN108133212A (en) * 2018-01-05 2018-06-08 东华大学 A kind of quota invoice amount identifying system based on deep learning
CN108221755A (en) * 2017-12-29 2018-06-29 青岛联合创智科技有限公司 A kind of intelligent space lock based on license plate recognition technology
CN108573254A (en) * 2017-03-13 2018-09-25 北京君正集成电路股份有限公司 The generation method of characters on license plate gray-scale map
CN108830188A (en) * 2018-05-30 2018-11-16 西安理工大学 Vehicle checking method based on deep learning
CN109711365A (en) * 2018-12-29 2019-05-03 佛山科学技术学院 A kind of vision SLAM winding detection method and device merging semantic information
CN109902692A (en) * 2019-01-14 2019-06-18 北京工商大学 A kind of image classification method based on regional area depth characteristic coding
CN109948612A (en) * 2019-03-19 2019-06-28 苏州怡林城信息科技有限公司 Detection method of license plate, storage medium and detection device based on convolutional network
CN109987102A (en) * 2018-01-03 2019-07-09 田文洪 A kind of method and apparatus of the High Precision Automatic identification driver's unsafe behaviors based on convolutional neural networks
WO2019174405A1 (en) * 2018-03-14 2019-09-19 台达电子工业股份有限公司 License plate identification method and system thereof
CN110378234A (en) * 2019-06-20 2019-10-25 合肥英威晟光电科技有限公司 Convolutional neural networks thermal imagery face identification method and system based on TensorFlow building
CN110659643A (en) * 2018-06-29 2020-01-07 杭州海康威视数字技术股份有限公司 Container number identification method and device
CN110956060A (en) * 2018-09-27 2020-04-03 北京市商汤科技开发有限公司 Motion recognition method, driving motion analysis method, device and electronic equipment
CN111444911A (en) * 2019-12-13 2020-07-24 珠海大横琴科技发展有限公司 Training method and device of license plate recognition model and license plate recognition method and device
CN111461128A (en) * 2020-03-31 2020-07-28 北京爱笔科技有限公司 License plate recognition method and device
CN111626354A (en) * 2020-05-27 2020-09-04 多伦科技股份有限公司 Clustering method based on task dependency applied to Internet of vehicles
CN112464938A (en) * 2020-12-18 2021-03-09 深圳赛安特技术服务有限公司 License plate detection and identification method, device, equipment and storage medium
CN114399921A (en) * 2022-01-19 2022-04-26 紫清智行科技(北京)有限公司 Method and system for detecting vacant parking spaces in underground parking lot
US20220284723A1 (en) * 2020-12-23 2022-09-08 Abbyy Development Inc. Zero-footprint image capture by mobile device

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104200207A (en) * 2014-09-16 2014-12-10 宁波熵联信息技术有限公司 License plate recognition method based on Hidden Markov models
CN104463220A (en) * 2014-12-19 2015-03-25 深圳市捷顺科技实业股份有限公司 License plate detection method and system
CN105160342A (en) * 2015-08-11 2015-12-16 成都数联铭品科技有限公司 HMM-GMM-based automatic word picture splitting method and system
CN105205448A (en) * 2015-08-11 2015-12-30 中国科学院自动化研究所 Character recognition model training method based on deep learning and recognition method thereof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104200207A (en) * 2014-09-16 2014-12-10 宁波熵联信息技术有限公司 License plate recognition method based on Hidden Markov models
CN104463220A (en) * 2014-12-19 2015-03-25 深圳市捷顺科技实业股份有限公司 License plate detection method and system
CN105160342A (en) * 2015-08-11 2015-12-16 成都数联铭品科技有限公司 HMM-GMM-based automatic word picture splitting method and system
CN105205448A (en) * 2015-08-11 2015-12-30 中国科学院自动化研究所 Character recognition model training method based on deep learning and recognition method thereof

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
C. LAWRENCE ZITNICK等: "Edge boxes: Locating object proposals from edges", 《ECCV2014》 *
HUI LI等: "Reading Car License Plates Using Deep Convolutiona Neural Networks and LSTMs", 《ARXIV:1601.05610V1 [CS.CV]》 *

Cited By (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106557768A (en) * 2016-11-25 2017-04-05 北京小米移动软件有限公司 The method and device is identified by word in picture
CN106557768B (en) * 2016-11-25 2021-07-06 北京小米移动软件有限公司 Method and device for recognizing characters in picture
CN106650740A (en) * 2016-12-15 2017-05-10 深圳市华尊科技股份有限公司 License plate identification method and terminal
CN106650740B (en) * 2016-12-15 2019-08-13 深圳市华尊科技股份有限公司 A kind of licence plate recognition method and terminal
CN106845482A (en) * 2017-01-23 2017-06-13 南京邮电大学盐城大数据研究院有限公司 A kind of license plate locating method
CN106934396A (en) * 2017-03-09 2017-07-07 深圳市捷顺科技实业股份有限公司 A kind of license number search method and system
CN108573254A (en) * 2017-03-13 2018-09-25 北京君正集成电路股份有限公司 The generation method of characters on license plate gray-scale map
CN108573254B (en) * 2017-03-13 2021-08-24 北京君正集成电路股份有限公司 License plate character gray scale image generation method
CN107229929A (en) * 2017-04-12 2017-10-03 西安电子科技大学 A kind of license plate locating method based on R CNN
CN107256384A (en) * 2017-05-22 2017-10-17 汕头大学 A kind of card recognition and method of counting based on image and signal transacting
CN107239778A (en) * 2017-06-09 2017-10-10 中国科学技术大学 The licence plate recognition method of efficiently and accurately
CN107239778B (en) * 2017-06-09 2020-01-03 中国科学技术大学 Efficient and accurate license plate recognition method
CN107369162B (en) * 2017-07-21 2020-07-10 华北电力大学（保定） Method and system for generating insulator candidate target area
CN107369162A (en) * 2017-07-21 2017-11-21 华北电力大学（保定） A kind of generation method and system of insulator candidate target region
CN107564324A (en) * 2017-07-28 2018-01-09 山东大学 A kind of garage intelligent power saving method and system
CN107844750A (en) * 2017-10-19 2018-03-27 华中科技大学 A kind of water surface panoramic picture target detection recognition methods
CN107844750B (en) * 2017-10-19 2020-05-19 华中科技大学 Water surface panoramic image target detection and identification method
CN107944457A (en) * 2017-11-23 2018-04-20 浙江清华长三角研究院 Drawing object identification and extracting method under a kind of complex scene
CN108221755A (en) * 2017-12-29 2018-06-29 青岛联合创智科技有限公司 A kind of intelligent space lock based on license plate recognition technology
CN109987102A (en) * 2018-01-03 2019-07-09 田文洪 A kind of method and apparatus of the High Precision Automatic identification driver's unsafe behaviors based on convolutional neural networks
CN108133212A (en) * 2018-01-05 2018-06-08 东华大学 A kind of quota invoice amount identifying system based on deep learning
CN108133212B (en) * 2018-01-05 2021-06-29 东华大学 Quota invoice amount recognition system based on deep learning
WO2019174405A1 (en) * 2018-03-14 2019-09-19 台达电子工业股份有限公司 License plate identification method and system thereof
US11443535B2 (en) 2018-03-14 2022-09-13 Delta Electronics, Inc. License plate identification method and system thereof
CN108830188B (en) * 2018-05-30 2022-03-04 西安理工大学 Vehicle detection method based on deep learning
CN108830188A (en) * 2018-05-30 2018-11-16 西安理工大学 Vehicle checking method based on deep learning
CN110659643A (en) * 2018-06-29 2020-01-07 杭州海康威视数字技术股份有限公司 Container number identification method and device
CN110659643B (en) * 2018-06-29 2022-08-26 杭州海康威视数字技术股份有限公司 Container number identification method and device
CN110956060A (en) * 2018-09-27 2020-04-03 北京市商汤科技开发有限公司 Motion recognition method, driving motion analysis method, device and electronic equipment
CN109711365A (en) * 2018-12-29 2019-05-03 佛山科学技术学院 A kind of vision SLAM winding detection method and device merging semantic information
CN109902692A (en) * 2019-01-14 2019-06-18 北京工商大学 A kind of image classification method based on regional area depth characteristic coding
CN109948612A (en) * 2019-03-19 2019-06-28 苏州怡林城信息科技有限公司 Detection method of license plate, storage medium and detection device based on convolutional network
CN110378234A (en) * 2019-06-20 2019-10-25 合肥英威晟光电科技有限公司 Convolutional neural networks thermal imagery face identification method and system based on TensorFlow building
CN111444911A (en) * 2019-12-13 2020-07-24 珠海大横琴科技发展有限公司 Training method and device of license plate recognition model and license plate recognition method and device
CN111444911B (en) * 2019-12-13 2021-02-26 珠海大横琴科技发展有限公司 Training method and device of license plate recognition model and license plate recognition method and device
CN111461128A (en) * 2020-03-31 2020-07-28 北京爱笔科技有限公司 License plate recognition method and device
CN111626354A (en) * 2020-05-27 2020-09-04 多伦科技股份有限公司 Clustering method based on task dependency applied to Internet of vehicles
CN111626354B (en) * 2020-05-27 2023-09-15 多伦科技股份有限公司 Clustering method applied to Internet of vehicles and based on task dependency
CN112464938A (en) * 2020-12-18 2021-03-09 深圳赛安特技术服务有限公司 License plate detection and identification method, device, equipment and storage medium
CN112464938B (en) * 2020-12-18 2024-04-12 深圳赛安特技术服务有限公司 License plate detection and identification method, device, equipment and storage medium
US20220284723A1 (en) * 2020-12-23 2022-09-08 Abbyy Development Inc. Zero-footprint image capture by mobile device
US11948385B2 (en) * 2020-12-23 2024-04-02 Abbyy Development Inc. Zero-footprint image capture by mobile device
CN114399921A (en) * 2022-01-19 2022-04-26 紫清智行科技(北京)有限公司 Method and system for detecting vacant parking spaces in underground parking lot

Similar Documents

Publication Publication Date Title
CN106096607A (en) A kind of licence plate recognition method
CN110163187B (en) F-RCNN-based remote traffic sign detection and identification method
CN109902677B (en) Vehicle detection method based on deep learning
CN105160309B (en) Three lanes detection method based on morphological image segmentation and region growing
Khammari et al. Vehicle detection combining gradient analysis and AdaBoost classification
CN110598690B (en) End-to-end optical character detection and recognition method and system
US8170340B2 (en) Device, method and computer program for identifying a traffic sign in an image
CN101673338B (en) Fuzzy license plate identification method based on multi-angle projection
CN102509091B (en) Airplane tail number recognition method
CN105260712B (en) A kind of vehicle front pedestrian detection method and system
CN111738055B (en) Multi-category text detection system and bill form detection method based on same
CN106022363B (en) A kind of Chinese text recognition methods suitable under natural scene
CN110097044A (en) Stage car plate detection recognition methods based on deep learning
CN109740584B (en) Automatic parking parking space detection method based on deep learning
Gu et al. Traffic sign detection in dual-focal active camera system
CN111582339B (en) Vehicle detection and recognition method based on deep learning
CN108009548A (en) A kind of Intelligent road sign recognition methods and system
CN105006003A (en) Random projection fern based real-time target tracking algorithm
CN105989334A (en) Monocular vision-based road detection method
KR101014125B1 (en) Method for detecting traffic sign board in intelligent vehicle and system for executing the method
Tian et al. Road marking detection based on mask R-CNN instance segmentation model
CN106529391B (en) A kind of speed limit road traffic sign detection of robust and recognition methods
CN108154144A (en) A kind of name of vessel character locating method and system based on image
CN111444916A (en) License plate positioning and identifying method and system under unconstrained condition
CN112418207A (en) Weak supervision character detection method based on self-attention distillation

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20161109