CN106778745A - A kind of licence plate recognition method and device, user equipment - Google Patents

A kind of licence plate recognition method and device, user equipment Download PDF

Info

Publication number
CN106778745A
CN106778745A CN201611207149.2A CN201611207149A CN106778745A CN 106778745 A CN106778745 A CN 106778745A CN 201611207149 A CN201611207149 A CN 201611207149A CN 106778745 A CN106778745 A CN 106778745A
Authority
CN
China
Prior art keywords
image
vehicle
license plate
convolutional neural
neural networks
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201611207149.2A
Other languages
Chinese (zh)
Inventor
王书强
曾德威
申妍燕
胡明辉
王永灿
杨岳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Priority to CN201611207149.2A priority Critical patent/CN106778745A/en
Publication of CN106778745A publication Critical patent/CN106778745A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/63—Scene text, e.g. street names
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The embodiment of the invention discloses a kind of licence plate recognition method and device, user equipment, the method includes：Vehicle image is obtained, and vehicle image input vehicle retrieval convolutional neural networks search vehicle image be whether there is into vehicle；If vehicle image has vehicle, license plate image is gathered on vehicle image, and license plate image input license number search convolutional neural networks search license plate image be whether there is into car plate；Then it is whether license plate image input tensor neutral net detection license plate image is abnormal；If without exception, identification image is then gathered on license plate image, output region numbering image, letter image and digital picture after judgement operation are carried out, then area number image, letter image and digital picture are input into convolutional neural networks to be respectively identified, export license plate recognition result.Licence plate recognition method provided in an embodiment of the present invention possess accurate classifying quality simultaneously network scale it is moderate, mobile device can be embedded into.

Description

A kind of licence plate recognition method and device, user equipment
Technical field
The present invention relates to image processing and artificial intelligence field, more particularly to a kind of licence plate recognition method and device, user Equipment.
Background technology
Convolutional neural networks (Convolutional Neural Network, CNN) are a kind of feedforward neural networks, it Artificial neuron can respond the surrounding cells in a part of coverage, have outstanding performance for large-scale image procossing.Convolution Neutral net avoids the complicated early stage pretreatment to image, can directly input original image, and basic structure is divided into two-layer, volume Lamination and pond layer.Convolutional layer is exported to preceding layer by multiple trainable wave filter groups and carries out linear Convolution, is produced special Levy mapping graph；Pond layer carries out pond computing to every group in Feature Mapping figure of four pixels, and space or characteristic type are carried out Polymerization.It is capable of identify that the feature extracted and consistency is distorted with displacement, scaling and other forms.
The preferable convolutional neural networks adjustable parameter of image recognition effect is larger, internal memory complexity and computation complexity It is higher, it is unfavorable in the middle of network application to mobile or embedded device；Meanwhile, distributed parallel computing etc. accelerates means pair In mobile or embedded device be unpractical.Convolutional neural networks have had more utilization in Car license recognition field, but sharp Car plate is recognized with convolutional neural networks and the scale that can accomplish to possess accurate classifying quality network simultaneously by system is moderate, can It is the problem for being currently needed for solving to be embedded into the mobile devices such as mobile phone.
The content of the invention
The embodiment of the invention discloses a kind of licence plate recognition method and device, user equipment, possesses accurately classifying quality The scale of network is moderate simultaneously, can be embedded into mobile device.
Embodiment of the present invention first aspect discloses a kind of licence plate recognition method, including：
Vehicle image is obtained, and vehicle image input vehicle retrieval convolutional neural networks are searched for into the vehicle image With the presence or absence of vehicle；
If the vehicle image has vehicle, license plate image is gathered on the vehicle image, and by the car plate Image input license number search convolutional neural networks search for the license plate image and whether there is car plate；
If the license plate image has car plate, license plate image input tensor neutral net is detected into the car plate Whether image is abnormal；
If without exception, identification image is gathered on the license plate image, and the identification image input is judged Neutral net judges area number, the letter and number on the identification image, and output region numbering image, letter image and Digital picture；
Respectively by the area number image, the letter image and the digital picture input area number identification convolution Neutral net, Letter identification convolutional neural networks and numeral identification convolutional neural networks carry out area number, letter and number Identification, and output region numbering, letter and number respectively；
According to area number, the letter and number of output, know by the form output license plate number of " area number alphanumeric " Other result.
As a kind of optional implementation method, in embodiment of the present invention first aspect, in methods described, the acquisition car Image, and vehicle image input vehicle retrieval convolutional neural networks are searched for into the vehicle image with the presence or absence of vehicle Step includes：
The vehicle image in video stream data is obtained, the vehicle image is represented with tensor pattern；
According to space switching network and convolutional neural networks by the vehicle image carry out multiple geometric space conversion correction, Process of convolution and pondization are processed, and obtain fisrt feature mapping graph；
The vectorization of fisrt feature mapping graph is exported into column vector, and connects the weights that multiple full articulamentums obtain full articulamentum Matrix；
The weight matrix of the full articulamentum is carried out into tensor row to decompose, first eigenvector is extracted；
The output layer that first eigenvector is input into the convolutional neural networks is carried out into classification to predict and optimize adjustable parameter, Search whether there is vehicle.
It is in the process, described by car in embodiment of the present invention first aspect as a kind of optional implementation method Board image input license number search convolutional neural networks are searched for the step of the license plate image whether there is car plate and included：
According to space switching network and convolutional neural networks by the license plate image carry out multiple geometric space conversion correction, Process of convolution and pondization are processed, and obtain second feature mapping graph；
The vectorization of second feature mapping graph is exported into column vector, and connects the weights that multiple full articulamentums obtain full articulamentum Matrix；
The weight matrix of the full articulamentum is carried out into tensor row to decompose, second feature vector is extracted；
The output layer of the second feature vector input convolutional neural networks is carried out into classification to predict and optimize adjustable parameter, Search whether there is car plate.
As a kind of optional implementation method, in embodiment of the present invention first aspect, the step of the optimization adjustable parameter It is rapid to be specially according to all adjustable parameters in direction of error propagation algorithm (Back-propagation, BP) optimization network, i.e., it is right The renewal of space conversion matrix, weights and skew, be the error returned according to later layer anti-pass carry out partial gradient calculate carry out 's.
It is in the process, described by car in embodiment of the present invention first aspect as a kind of optional implementation method Board image input tensor neutral net detects whether abnormal step includes the license plate image：
The license plate image is directly inputted and obtain its corresponding direct probability density in the tensor neutral net for training Than；
The critical direct probability for being obtained after the training by the direct probability density ratio with completion tensor neutral net again is close Degree ratio is compared, and is normal picture if the direct probability density ratio is more than the critical direct probability density ratio, no It is then abnormal image.
Embodiment of the present invention second aspect discloses a kind of license plate recognition device, including：
Data acquisition module, for the collection vehicle image in video streaming image；Car plate is gathered in the vehicle image Image；And identification image is gathered in the license plate image；
Vehicle search module, for vehicle image input vehicle retrieval convolutional neural networks to be searched for into the vehicle figure As whether there is vehicle；
Plate searching module, for license plate image input license number search convolutional neural networks to be searched for into the car plate figure As whether there is car plate；
Exception monitoring module, for license plate image input tensor neutral net to be checked into whether the license plate image is different Often；
Car license recognition module, for the identification image input to be judged into neutral net judges the ground on the identification image Area's numbering, letter and number, and output region numbering image, letter image and digital picture；Then the area is compiled respectively Number image, the letter image and the digital picture input area number identification convolutional neural networks, Letter identification convolution god Carry out the identification of area number, letter and number through network and numeral identification convolutional neural networks, and respectively output region numbering, Letter and number；Further according to area number, the letter and number of output, car plate is exported by the form of " area number alphanumeric " Number recognition result.
Used as a kind of optional implementation method, in embodiment of the present invention second aspect, the vehicle search module includes：
Vehicle image processing unit, for being carried out the vehicle image according to space switching network and convolutional neural networks Multiple geometric space conversion correction, process of convolution and pondization treatment, obtain fisrt feature mapping graph；
Weight matrix acquiring unit, connects entirely for the vectorization of fisrt feature mapping graph being exported into column vector, and connecting multiple Connect the weight matrix that layer obtains full articulamentum；
Tensor resolution unit, decomposes for the weight matrix of the full articulamentum to be carried out into tensor row, extracts fisrt feature Vector；
Classification predicting unit, the output layer for first eigenvector to be input into the convolutional neural networks carries out classifying pre- Adjustable parameter is surveyed and optimizes, search whether there is vehicle.
Used as a kind of optional implementation method, in embodiment of the present invention second aspect, the Plate searching module includes：
License plate image processing unit, for being carried out the license plate image according to space switching network and convolutional neural networks Multiple geometric space conversion correction, process of convolution and pondization treatment, obtain second feature mapping graph；
Weight matrix acquiring unit, connects entirely for the vectorization of second feature mapping graph being exported into column vector, and connecting multiple Connect the weight matrix that layer obtains full articulamentum；
Tensor resolution unit, decomposes for the weight matrix of the full articulamentum to be carried out into tensor row, extracts second feature Vector；
Classification predicting unit, for carrying out classifying pre- the output layer of the second feature vector input convolutional neural networks Adjustable parameter is surveyed and optimizes, search whether there is car plate.
Used as a kind of optional implementation method, in embodiment of the present invention second aspect, the exception monitoring module includes：
Tensor neutral net unit, corresponding direct probability density ratio is obtained for the license plate image according to input；
Comparing unit, it is critical for what is obtained after the training by the direct probability density ratio with completion tensor neutral net Direct probability density ratio is compared, if the direct probability density ratio is more than the critical direct probability density ratio, for Normal picture, is otherwise abnormal image.
The embodiment of the present invention third aspect discloses a kind of user equipment, including institute disclosed in embodiment of the present invention second aspect State license plate recognition device.
Compared with prior art, the embodiment of the present invention possesses following beneficial effect：
In the embodiment of the present invention, after obtaining vehicle image, by vehicle image input vehicle retrieval convolutional neural networks search The vehicle image whether there is vehicle；If vehicle image has vehicle, license plate image is gathered on vehicle image, and will License plate image input license number search convolutional neural networks search for the license plate image with the presence or absence of car plate；Then license plate image is input into Tensor neutral net detects whether the license plate image is abnormal；If without exception, identification image is gathered on the license plate image, It is and the identification image input that will be gathered judges that neutral net judges area number, the letter and number on the identification image and defeated Go out area number image, letter image and digital picture；Area number image, letter image and digital picture are input into respectively again Area number identification convolutional neural networks, Letter identification convolutional neural networks and numeral identification convolutional neural networks carry out regional volume Number, the identification of letter and number, and respectively output region numbering, letter and number；Finally according to area number, the letter of output And numeral, export license plate number recognition result by the form of " area number alphanumeric ".It can be seen that, implement the embodiment of the present invention and provide Licence plate recognition method possess accurate classifying quality simultaneously network scale it is moderate, mobile device can be embedded into.
Brief description of the drawings
Technical scheme in order to illustrate more clearly the embodiments of the present invention, below by to be used needed for embodiment Accompanying drawing is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the present invention, for ability For the those of ordinary skill of domain, on the premise of not paying creative work, can also obtain other attached according to these accompanying drawings Figure.
Fig. 1 is a kind of schematic flow sheet of licence plate recognition method disclosed in the embodiment of the present invention；
Fig. 2 is the schematic flow sheet of another licence plate recognition method disclosed in the embodiment of the present invention；
Fig. 3 is to the improved schematic diagram of traditional convolutional neural networks in the embodiment of the present invention；
Fig. 4 is the relation schematic diagram of weighting parameter scale and r values in the embodiment of the present invention；
Fig. 5 is a kind of structural representation of license plate recognition device disclosed in the embodiment of the present invention；
Fig. 6 is the structural representation of another license plate recognition device disclosed in the embodiment of the present invention；
Fig. 7 is a kind of structural representation of user equipment disclosed in the embodiment of the present invention.
Specific embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clear, complete Site preparation is described, it is clear that described embodiment is only a part of embodiment of the invention, rather than whole embodiments.Based on this Embodiment in invention, the every other reality that those of ordinary skill in the art are obtained under the premise of creative work is not made Example is applied, the scope of protection of the invention is belonged to.
It should be noted that the term " comprising " and " having " of the embodiment of the present invention and their any deformation, it is intended that Be cover it is non-exclusive include, for example, containing process, method, system, product or the equipment of series of steps or unit not Be necessarily limited to those steps or the unit clearly listed, but may include not list clearly or for these processes, side Method, product or other intrinsic steps of equipment or unit.
The embodiment of the invention discloses a kind of licence plate recognition method and device, user equipment, realize and possess accurate point Network size is moderate while class effect, can be embedded into mobile device.Accompanying drawing is below combined to be described in detail.
Embodiment one
Fig. 1 is referred to, Fig. 1 is a kind of schematic flow sheet of licence plate recognition method disclosed in the embodiment of the present invention.Such as Fig. 1 institutes Show, the licence plate recognition method may comprise steps of：
101st, vehicle image is obtained, and whether vehicle image is input into vehicle retrieval convolutional neural networks search vehicle image There is vehicle；
The first-class movement of electro-photographic or embedded device collection color video flow data, data are with tensor pattern storage table Show.First from video stream data, carried out time from the order in the upper left corner to the lower right corner on video streaming image with specific step-length Go through, obtain vehicle image, vehicle image is preprocessed to obtain pretreated vehicle image, and pretreated vehicle image enters Vehicle retrieval convolutional neural networks, check for vehicle.Specifically, first according to space switching network and convolutional Neural net Vehicle image is carried out multiple geometric space conversion correction, process of convolution and pondization treatment by network, i.e., first geometric space converts correction, Then the treatment of process of convolution, then pondization, then again from geometric space conversion correction start to process, so circulation is multiple, obtains the One Feature Mapping figure；Then the vectorization of fisrt feature mapping graph is exported into column vector, and connects multiple full articulamentums and connected entirely Connect the weight matrix of layer；The weight matrix of full articulamentum is carried out into tensor row again to decompose, first eigenvector is extracted；Finally by The output layer of one characteristic vector input convolutional neural networks carries out classification and predicts and optimize adjustable parameter, and search whether there is car .Wherein, optimize adjustable parameter the step of be specially according to direction of error propagation algorithm optimize network in all adjustable parameters, i.e., It is that the error returned according to later layer anti-pass carries out part to the weights in space conversion matrix, network and the renewal of skew What gradient calculation was carried out.
If the 102, vehicle image has vehicle, license plate image is gathered on vehicle image, and license plate image is input into License number search convolutional neural networks search license plate image whether there is car plate；
If vehicle image is adopted by there is vehicle after the search of vehicle retrieval convolutional neural networks on the vehicle image Collection license plate image, specifically, carrying out traversal collection from the order in the upper left corner to the lower right corner on vehicle image with specific step-length License plate image, license plate image is preprocessed to obtain pretreated license plate image, and pretreated license plate image is examined into car plate Rope convolutional neural networks, check for car plate.Specifically, first according to space switching network and convolutional neural networks by car Board image carries out multiple geometric space conversion correction, process of convolution and pondization treatment, i.e., first geometric space conversion correction, Ran Houjuan Product treatment, then pondization treatment, then again from geometric space conversion correction start to process, so circulation is multiple, obtains second feature Mapping graph；Then the vectorization of second feature mapping graph is exported into column vector, and connects multiple full articulamentums and obtain full articulamentum Weight matrix；The weight matrix of full articulamentum is carried out into tensor row again to decompose, second feature vector is extracted；Finally by second feature The output layer of vector input convolutional neural networks carries out classification and predicts and optimize adjustable parameter, and search whether there is car plate.Wherein, The step of optimization adjustable parameter, is specially according to all adjustable parameters in direction of error propagation algorithm optimization network, i.e., space is turned The weights changed in matrix, network and the renewal of skew, are that the error returned according to later layer anti-pass carries out partial gradient calculating Carry out.
103rd, it is whether license plate image input tensor neutral net detection license plate image is abnormal；
If license plate image after the search of license number search convolutional neural networks by having car plate, by license plate image input Whether amount neutral net detection license plate image is abnormal.Specifically, license plate image first to be directly inputted the tensor nerve net for training Its corresponding direct probability density ratio is obtained in network；Obtained after the training by direct probability density ratio with completion tensor neutral net again To critical direct probability density ratio be compared, if direct probability density ratio be more than critical direct probability density ratio, for Normal picture, is otherwise abnormal image.
If the 104, without exception, identification image is gathered on license plate image, and identification image input is judged into nerve Network judges area number, the letter and number on the identification image, and output region numbering image, letter image and digitized map Picture；
If license plate image gathers identification image, specifically, existing with specific step-length without abnormal on license plate image Order on license plate image from the upper left corner to the lower right corner is traveled through, collection identification image.Identification image input is judged into god again Judge area number, the letter and number on the identification image through network, and output region numbering image, letter image and numeral Image.Specifically, identification image obtains pretreated identification image by pretreatment, pretreated identification image enters one Individual simple judgement neutral net judged, the output of the judgement neutral net totally four class, respectively area number, letter, Numeral and other.If judged result is other, abandon currently recognizing image, judge next identification image；If judged result It is area number, the area number that identification image enters the Car license recognition stage is recognized；If judged result is alphabetical, identification image enters Enter the Letter identification in Car license recognition stage；If judged result is digital, identification image then enters the numeral knowledge in Car license recognition stage Not.
105th, respectively by area number image, letter image and digital picture input area number identification convolutional Neural net Network, Letter identification convolutional neural networks and numeral identification convolutional neural networks carry out the identification of area number, letter and number, and Difference output region numbering, letter and number；
License plate number cognitive phase, is made up of three parallel convolutional neural networks, is respectively, area number identification convolution god Through network, Letter identification convolutional neural networks and numeral recognize convolutional neural networks.
Area number is recognized：Area number image is pre-processed, pretreated area number image, pre- place is produced Area number image after reason carries out the identification of area number into area number identification convolutional neural networks.
Letter identification：Letter image is pre-processed, pretreated letter image, pretreated grapheme is produced Identification as carrying out letter into Letter identification convolutional neural networks.
Numeral identification：Digital picture is pre-processed, pretreated digital picture, pretreated grapheme is produced As entering the identification that numeral identification convolutional neural networks carry out numeral.
106th, according to area number, the letter and number of output, car plate is exported by the form of " area number alphanumeric " Number recognition result；
Finally according to three outputs of convolutional neural networks, know by the form output license plate number of " area number alphanumeric " Other result.
It should be noted that can be smoothing processing, denoising, light to the pretreatment of all kinds of images in the present embodiment According to normalized, enhancing treatment etc., according to being pre-processed the need for specific, it is not limited herein.
In the method described by Fig. 1, after obtaining vehicle image, vehicle image input vehicle retrieval convolutional neural networks are searched Suo Suoshu vehicle images whether there is vehicle；If vehicle image has vehicle, license plate image is gathered on vehicle image, and License plate image input license number search convolutional neural networks are searched for into the license plate image with the presence or absence of car plate；Then it is license plate image is defeated Enter tensor neutral net and detect whether the license plate image is abnormal；If without exception, identification figure is gathered on the license plate image Picture, and the identification image input that will be gathered judges that neutral net judges area number, the letter and number on the identification image, and Output region numbering image, letter image and digital picture；It is respectively that area number image, letter image and digital picture is defeated again Entering area number identification convolutional neural networks, Letter identification convolutional neural networks and numeral identification convolutional neural networks carries out area Numbering, the identification of letter and number, and output region numbering, letter and number respectively；Finally according to area number, the word of output Female and numeral, license plate number recognition result is exported by the form of " area number alphanumeric ".It can be seen that, implement the embodiment of the present invention and carry The licence plate recognition method of confession introduces space switching network and tensor resolution in convolutional neural networks, from classification accuracy and network The angle of scale, improves traditional convolutional network, and being identified to car plate by the multistage segmentation of multiple convolutional neural networks, Possess accurate classifying quality simultaneously network scale it is moderate, mobile device can be embedded into.
Embodiment two
Fig. 2 is referred to, Fig. 2 is the schematic flow sheet of another licence plate recognition method disclosed in the embodiment of the present invention.Such as Fig. 2 Shown, the licence plate recognition method may comprise steps of：
201st, video stream data is gathered；
The first-class movement of electro-photographic or embedded device collection color video flow data, data are with tensor pattern storage table Show.
202nd, the collection vehicle image from video streaming image；
From video stream data, carried out time from the order in the upper left corner to the lower right corner on video streaming image with specific step-length Go through, obtain vehicle image.
203rd, the pretreatment of vehicle image；
Vehicle image is preprocessed to obtain pretreated vehicle image, and pretreatment herein can be smoothing processing, go Make an uproar treatment, unitary of illumination treatment, enhancing treatment etc., according to the pretreatment being combined the need for specific, herein not It is limited.
204th, vehicle retrieval convolutional neural networks search whether there is vehicle；
Pretreated vehicle image enters vehicle retrieval convolutional neural networks, checks for vehicle.The present embodiment In convolutional neural networks be the introduction of the convolutional neural networks of space switching network and tensor resolution, from classification accuracy and net The angle of network scale improves traditional convolutional network.Specifically, first will according to space switching network and convolutional neural networks Vehicle image carries out multiple geometric space conversion correction, process of convolution and pondization treatment, i.e., first geometric space converts correction, then Process of convolution, then pondization treatment, then again from geometric space conversion correction start to process, so circulation is multiple, obtains first special Levy mapping graph；Then the vectorization of fisrt feature mapping graph is exported into column vector, and connects multiple full articulamentums and obtain full articulamentum Weight matrix；The weight matrix of full articulamentum is carried out into tensor row again to decompose, first eigenvector is extracted；It is finally special by first The output layer for levying vector input convolutional neural networks carries out classification and predicts and optimize adjustable parameter, and search whether there is vehicle.Its In, optimization adjustable parameter the step of be specially according to direction of error propagation algorithm optimize network in all adjustable parameters, i.e., to sky Between weights in transition matrix, right and skew renewal, be that the error returned according to later layer anti-pass carries out partial gradient Calculate what is carried out.
Vehicle image is represented with tensor pattern and directly inputs the input module of convolutional neural networks.Referring to Fig. 3 pairs The improvement of traditional convolutional neural networks is illustrated：
Image is represented with tensor pattern and directly inputs the input module of network.Specifically：
By taking MNIST handwritten numeral gray-scale maps as an example, scale 28 × 28, for convenience of subsequent step explanation, all samples are all passed through Cross 45 ° of rotate counterclockwise noises.So Z in the present embodiment(0)Scale be 28 × 28 × 1, wherein Z(l)Be with three rank tensors, L layers of output figure is represented, is also the input figure of l+1.First rank represents Z(l)Height, second-order represents Z(l)Width, the 3rd Rank represents Z(l)Figure layer.
Image carries out geometric space conversion correction into spatial alternation network.Specifically：
Input picture Z(0)Into positioning mixed-media network modules mixed-media study transformation matrix θ, the positioning network is a small-sized recurrence god Through network, i.e. convolutional neural networks or fully-connected network, do not limit herein.Each element of output transform matrix θ θij, the scale-dependent of transformation matrix θ is in the scale of input, for example, input is two-dimensional matrix, the scale of transformation matrix θ is 2 × 3, If three-dimensional matrice, the scale of transformation matrix θ is 3 × 4, by that analogy.
By output image Z(1)On coordinate system coordinate (xt,yt), wherein, " t " is the abbreviation of target, represents target image Z(1), i.e. output image is mapped to the coordinate system coordinate (x on input picture by transformation matrix θs,ys), wherein, " s " is The abbreviation of source, represents source images Z(0), i.e. input picture, generation sampling grid.The relation of target image and source images is such as Under：
The type of definition sampling core, sampling core k () of present invention definition, sampling core k () determines image interpolation type, The image interpolation type that sampling core defined in such as the present embodiment is determined is bilinear interpolation function, is not limited to herein double Linear interpolation function；To on source images sampling grid in (xs,ys) pixel at place enters row interpolation, and interpolation calculation is obtained Pixel value is delivered to output image Z(1)On coordinate (xt,yt) place finally obtains output image Z(1)。
Wherein, V in formulai c'sRepresent target image matrix VcI-th coordinate value (x of elementt,yt), together ReasonWithIt is source images UcI-th coordinate value (x of elements,ys), H, W represent the height and width of source images respectively, W ' and H ' points Not Biao Shi target image height and width.
Input picture Z(0)Correction chart is out renamed as Z from spatial alternation network(1), correction chart picture enter convolution module and Pond module, carries out process of convolution and pondization treatment respectively.Specifically：
In the present embodiment, feedforward convolution is effective convolution algorithm, and step-length is 1；2nd, 5 layer is convolutional layer, wherein Kernel(l)L layers of all convolution kernels are represented, the first second order represents the height and width of convolution kernel, and the 3rd rank represents figure layer, the 4th Rank represents the number of convolution kernel.The 3 of this example, 6 layers of expression pond layer, conventional pond type has maximum pondization and average pond.
By after multiple " geometric space conversion → convolution → pond ", by last Feature Mapping figure Z(6)Vectorization is exported Column vector, i.e., last Feature Mapping figure Z(6)Be rearranged for column vector.Specifically：In the present embodiment, last feature is reflected It is Z to penetrate figure(6), scale is 4 × 4 × 3, so the scale after vectorization is 48 × 1.
Then the weight matrix of full articulamentum carries out tensor row decomposition, is represented with TT-format, by the computing of full articulamentum Pattern is converted into tensor pattern by original vector pattern.Specifically：
In the present embodiment, full articulamentum Z(7)Node scale be 48 × 1, output layer Z(8)Node scale be 10 × 1, institute Scale with full articulamentum weight matrix is 10 × 48, and the reconfiguration scheme selected in the present embodiment is 2 × 5 and 6 × 8, sets TT- Rank is (1, r, 1), and weight matrix is mapped to the matrix (second-order tensor) of (2 × 6) × (5 × 8), is decomposed tensor row are carried out The TT-format for obtaining weight matrix represents that the TT-format of different r values correspondence different scales is represented, as shown in figure 4, original Scale when weight matrix is represented is 10 × 48=480, and TT-format is when representing, with weighting parameter scale and r values into Direct ratio, selects suitable r values to play the effect of reduction network parameter scale.
The high order tensor being mapped to when the input of full articulamentum is mapped into full articulamentum weight matrix tensor row decomposition In same order tensor, when the weight matrix tensor row of full articulamentum are decomposed such as in the present embodiment, matrix is mapped to 12 × 40 square Tensor row are carried out in battle array (second-order tensor) again to decompose, the input of full articulamentum is mapped in matrix (second-order tensor).So complete It is same order tensor (second-order tensor) that the output of articulamentum is also, and now output should be mapped into vectorial output, in order to under One full articulamentum is connected or in order to allow the result of output layer to be easy to understand entirely.
Further according to all adjustable parameters in direction of error propagation algorithm optimization network.Specifically：
It is L to define loss function (Loss function), and conventional loss function has cross entropy loss function, the difference of two squares Loss function etc..In this example, loss function is represented with alphabetical L, its concrete form is not limited, and is selected according to actual needs Select.
The error of output layer (the 8th layer in this example) is calculated firstWherein The error of k-th node of output layer is represented respectively, and net input is exported, desired output,Represent loss function L pairsAsk inclined Derivative.
Output layer in this example is also full articulamentum.Therefore during the error vector of output layer mapped back into same order tensor, lead to The full connection matrix that TT-format is represented is crossed, by Feedback error to next layer, because the error vector of output layer passes through After the full connection weight that TT-format is represented, the error transmitted is the form of same order tensor, so needing to map back vector Pattern, obtains the error vector of stack layer, i.e.,：
Vector x is mapped to same order tensor by wherein tensor (x) expressions,Represent TT-format tables The full connection weight shown, due in this example, full connection weight matrix is mapped in second-order tensor (matrix) in tensor row point Solution, so the full connection weight that the 8th layer of TT-format is represented isVec (X) is represented high order tensor X mappings It is vector, " ο " represents that by first multiplication (point-wise multiplication) f ' (x) representative functions f (x) leads letter to x's Number.So the TT-format full connection weights that represent are updated to：
b(8)new=b(8)+Δb(8),
Wherein, η represents learning rate, typically takes 0.001,Represent what l layers of TT-format was represented Full connection weight, " T " represents transposition, b(l)Represent l layers of offset parameter vector.
The error vector of the 7th layer of stack layer is mapped back the output of layer pond layer of chart-pattern, i.e., the 6th, i.e. δ(6)=ToMap (δ(7)), wherein ToMap (x) is represented and for vector x to be mapped back chart-pattern.
The 6th layer of Error Graph of pond layer is up-sampled into (upsample) according to pond type and is delivered to the 5th layer of convolutional layer, Upsample is pond inverse operation, if uniform pond (mean-pooling) then pooling layers of error averagely arrive its 4 In individual input, if maximum pond (max-pooling) then travels to error revert all in its 4 inputs.This example In pond layer using maximum pond layer, selection pond type is not restricted to maximum pond layer in the present embodiment.
δ(5)=upsample (δ(6))；
Then the convolution kernel weighting parameter of the 5th layer of convolutional layer is updated to：
I=1,2 ... .numel (A(4),3)；
Kernel(5)new=Kernel(5)+ΔKernel(5),
b(5)new=b(5)+Δb(5)；
Wherein, A(l)Represent that l layers of convolution is input into only, numel (A(l), 3) and represent the figure layer that l layers of convolution is input into only Number, rot90(X) represent and the element of matrix X is rotated clockwise 90 degree, Z(l)L layers of output figure is represented, is also l+1 Input figure, the first second order represents Z(l)Height and width, the 3rd rank represents Z(l)Figure layer；' valid ' represents convolution type to have Effect convolution.
The 5th layer of Error Graph of convolutional layer is delivered to the 4th sheaf space by full convolution algorithm (full convolution) Conversion layer：
Wherein numel (Kernel(l), 4) represent l layers be convolutional layer, the convolution kernel number of this layer；Convn represents convolution Computing；rot180(X) represent the element dextrorotation turnback of matrix X；' full ' represents that convolution type is full convolution.
4th layer be spatial alternation layer, it is necessary to regulation parameter for transformation matrix θ element, specifically：
Wherein θxθyThe first row and the second row element of correspondent transform matrix θ.Parameter in positioning network is according to traditional mistake Difference direction propagation algorithm undated parameter, does not repeat herein.
Above is introduce the citing of space switching network and tensor resolution in traditional convolutional neural networks, it is equally applicable In license number search convolutional neural networks, and identification convolutional neural networks.
205th, vehicle is judged whether；
Search whether there is vehicle, if it is, step 207 is performed, if it is not, then performing step 206.
206th, judge whether to travel through current static image；
If not existing vehicle in search Current vehicle image, then judge whether to have traveled through current static image, if It is then to perform step 201, if it is not, then performing step 202.
207th, license plate image is gathered；
If vehicle image is adopted by there is vehicle after the search of vehicle retrieval convolutional neural networks on the vehicle image Collection license plate image, specifically, carrying out traversal collection from the order in the upper left corner to the lower right corner on vehicle image with specific step-length License plate image.
208th, the pretreatment of license plate image；
License plate image is preprocessed to obtain pretreated license plate image, and pretreatment herein can be smoothing processing, go Make an uproar treatment, unitary of illumination treatment, enhancing treatment etc., according to being pre-processed the need for specific, be not limited herein.
209th, license number search convolutional neural networks search whether there is car plate；
Pretreated license plate image enters license number search convolutional neural networks, checks for car plate.The present embodiment In convolutional neural networks be the introduction of the convolutional neural networks of space switching network and tensor resolution, from classification accuracy and net The angle of network scale improves traditional convolutional network.Specifically, first will according to space switching network and convolutional neural networks License plate image carries out multiple geometric space conversion correction, process of convolution and pondization treatment, i.e., first geometric space converts correction, then Process of convolution, then pondization treatment, then again from geometric space conversion correction start to process, so circulation is multiple, obtains second special Levy mapping graph；Then the vectorization of second feature mapping graph is exported into column vector, and connects multiple full articulamentums and obtain full articulamentum Weight matrix；The weight matrix of full articulamentum is carried out into tensor row again to decompose, second feature vector is extracted；It is finally special by second The output layer for levying vector input convolutional neural networks carries out classification and predicts and optimize adjustable parameter, and search whether there is car plate.Its In, optimization adjustable parameter the step of be specially according to direction of error propagation algorithm optimize network in all adjustable parameters, i.e., to sky Between weights in transition matrix, network and skew renewal, be that the error returned according to later layer anti-pass carries out partial gradient Calculate what is carried out.
210th, car plate is judged whether；
Search whether there is car plate, if it is, step 213 is performed, if it is not, then performing step 211.
211st, judge whether to travel through current static image；
If not existing car plate in search license plate image, then judge whether to have traveled through current static image, if it is, Step 207 is performed, if otherwise performing step 212.
212nd, next vehicle image；
Whether there is car plate in next vehicle image of search.
213rd, license plate image input tensor neutral net is detected；
If there is car plate in search license plate image, license plate image input tensor neutral net is detected.Specifically 's：License plate image enters abnormality detection tensor neutral net and checks whether current license plate image is abnormal, if abnormal, gives up current vehicle Board image and output abnormality warning message, detect next license plate image；If normal, current license plate image is walked into Car license recognition Suddenly.
Abnormality detection in the present embodiment be based on tensor network and probability density than abnormality detection, specific training process It is as follows：
Precondition is：Assuming that in the presence of containing a small amount of abnormal sample set
First by sample setIt is divided into regular set standard { } and evaluate collection review { }, regular set Standard { } is consistent with the sample size of evaluate collection review { }, and wherein standard { } is comprising all samples in sample set Normal sample, evaluate collection review { } containing all of exceptional sample and part from standard { } sampling it is copied come it is normal , specifically, it is assumed that sample set contains the sample of 15 normal 3 exceptions, i.e., be present 3 exceptional samples in sample, then have in 18 samples Standard { 15 }, review { (3)+12 }, in wherein review { } 12 is random from 15 samples in standard { } 12 samples that sampling is obtained；
Then by the random one-to-one matching of sample in standard { } and review { }, data pair are obtainedTo sample according to standard, review's is sequentially input to tensor nerve to each data Mapping function f (x) is trained in network, it is output as the corresponding direct probability density ratios of input x；Complete the instruction of tensor neutral net After white silk, the sample in evaluate collection review { } is input in the tensor neutral net for training to obtain corresponding direct probability close Than p, the ratio ρ with reference to exception in sample set obtains the critical value p of critical direct probability density ratio to degreecritica l；
New samples is directly inputted again obtain its corresponding direct probability density ratio in the tensor neutral net for training pnewIf, pnew≥pcriticalIt is judged as normal sample, is otherwise considered as exceptional sample.
214th, exception is judged whether；
Judge whether license plate image is abnormal, if abnormal, perform step 215, direct output abnormality information is otherwise performed 216。
215th, output abnormality information；
216th, collection identification image；
If license plate image gathers identification image, specifically, existing with specific step-length without abnormal on license plate image Order on license plate image from the upper left corner to the lower right corner is traveled through, collection identification image.
217th, identification image input is judged that neutral net is judged；
Identification image input is judged that neutral net judges area number, the letter and number on the identification image, and it is defeated Go out area number image, letter image and digital picture.Specifically, identification image obtains pretreated identification by pretreatment Image, pretreated identification image simply judges that neutral net is judged into one, the judgement neutral net it is defeated Go out totally four class, respectively area number, letter, numeral and other.If judged result is other, abandon currently recognizing image, Judge next identification image；If judged result area number, the area number that identification image enters the Car license recognition stage is known Not；If judged result is alphabetical, identification image enters the Letter identification in Car license recognition stage；If judged result is digital, identification Image then enters the numeral identification in Car license recognition stage.
218th, the identification image is abandoned；
If recognition result is other, the identification image is abandoned.
219th, the pretreatment of area number image；
If recognition result is area number, area number image is pre-processed, produced pretreated area Numbering image.
220th, area number identification convolutional neural networks identification；
Pretreated area number image is carried out the knowledge of area number into area number identification convolutional neural networks Not.
221st, the pretreatment of letter image；
If recognition result is letter, letter image is pre-processed, produced pretreated letter image.
222nd, Letter identification convolutional neural networks identification；
Pretreated letter image is carried out the identification of letter into Letter identification convolutional neural networks.
223rd, the pretreatment of digital picture；
If recognition result is numeral, digital picture is pre-processed, produced pretreated digital picture.
224th, numeral identification convolutional neural networks identification；
Pretreated letter image is entered into the identification that numeral identification convolutional neural networks carry out numeral.
225th, recognition result is exported；
Finally according to three outputs of convolutional neural networks, know by the form output license plate number of " area number alphanumeric " Other result.
Wherein, in the method described by implementation Fig. 2, after the full articulamentum in convolutional neural networks introduces tensor resolution, Network is renamed as tensor neutral net, storage more new capital of the feedforward transmission, error back propagation and weights of its full articulamentum Change；Space switching network is re-introduced into the present embodiment on this basis in tensor neutral net, specifically, at each Embedded space modular converter before convolutional layer, useful space geometric transformation correction is carried out to convolution input, effectively alleviates convolutional layer The conversion limitation without space-invariance larger to input picture or Feature Mapping figure, improves convolution output space invariance Property, and then make the feature of extraction more representative, improve the classification degree of accuracy of output layer.Further, the computing of full articulamentum is adopted Carried out with the pattern of tensor, effectively alleviate the limitation of the spatial information loss that full connect band is come.Also, Vehicle License Plate Recognition System is used Multiple convolutional networks, multi-stage strategy realizes Car license recognition, and the complex process of Car license recognition is decomposed into multiple simple sub- mistakes Journey, i.e. vehicle detection, car plate detection, license plate number identification, license plate number identification is again including area number identification, Letter identification and numeral Identification, effectively increases the efficiency of Car license recognition.Furthermore, increase before Car license recognition based on direct probability density ratio and tensor god Through the abnormality detection of network, abnormal data is effectively accurately identified, Exception Filter data reduce unnecessary computing, improve system Efficiency.
Embodiment three
Fig. 5 is referred to, Fig. 5 is a kind of structural representation of license plate recognition device disclosed in the embodiment of the present invention.Such as Fig. 5 institutes Show, the license plate recognition device can include：
Data acquisition module 301, for the collection vehicle image in video streaming image, specifically, being existed with specific step-length Order on video streaming image from the upper left corner to the lower right corner is traveled through, and obtains vehicle image；Car plate is gathered in vehicle image Image, specifically, carrying out traversal collection car plate figure from the order in the upper left corner to the lower right corner on vehicle image with specific step-length Picture；And in license plate image gather identification image, specifically, with specific step-length on license plate image from the upper left corner to bottom right The order at angle is traveled through, collection identification image.
Vehicle search module 302, for the vehicle image input vehicle retrieval convolution god for gathering data acquisition module 301 Through web search, the vehicle image whether there is vehicle；Specifically, first will according to space switching network and convolutional neural networks Vehicle image carries out multiple geometric space conversion correction, process of convolution and pondization treatment, i.e., first geometric space converts correction, then Process of convolution, then pondization treatment, then again from geometric space conversion correction start to process, so circulation is multiple, obtains first special Levy mapping graph；Then the vectorization of fisrt feature mapping graph is exported into column vector, and connects multiple full articulamentums and obtain full articulamentum Weight matrix；The weight matrix of full articulamentum is carried out into tensor row again to decompose, first eigenvector is extracted；It is finally special by first The output layer for levying vector input convolutional neural networks carries out classification and predicts and optimize adjustable parameter, and search whether there is vehicle.Its In, optimization adjustable parameter the step of be specially according to direction of error propagation algorithm optimize network in all adjustable parameters, i.e., to sky Between weights in transition matrix, network and skew renewal, be that the error returned according to later layer anti-pass carries out partial gradient meter Carry out.
Plate searching module 303, for the license plate image input license number search convolution god for gathering data acquisition module 301 Through web search, the license plate image whether there is car plate；Specifically, first will according to space switching network and convolutional neural networks License plate image carries out multiple geometric space conversion correction, process of convolution and pondization treatment, i.e., first geometric space converts correction, then Process of convolution, then pondization treatment, then again from geometric space conversion correction start to process, so circulation is multiple, obtains second special Levy mapping graph；Then the vectorization of second feature mapping graph is exported into column vector, and connects multiple full articulamentums and obtain full articulamentum Weight matrix；The weight matrix of full articulamentum is carried out into tensor row again to decompose, second feature vector is extracted；It is finally special by second The output layer for levying vector input convolutional neural networks carries out classification and predicts and optimize adjustable parameter, and search whether there is car plate.Its In, optimization adjustable parameter the step of be specially according to direction of error propagation algorithm optimize network in all adjustable parameters, i.e., to sky Between weights in transition matrix, network and skew renewal, be that the error returned according to later layer anti-pass carries out partial gradient meter Carry out.
Exception monitoring module, for license plate image input tensor neutral net to be checked into whether the license plate image is abnormal；Tool Body, first license plate image is directly inputted and obtain its corresponding direct probability density ratio in the tensor neutral net for training；Again The critical direct probability density ratio obtained after training by direct probability density ratio with completion tensor neutral net is compared, such as Fruit direct probability density ratio is more than critical direct probability density ratio, then be normal picture, is otherwise abnormal image.
Car license recognition module, the identification image input for data acquisition module 301 to be gathered judges that neutral net judges Area number, letter and number on the identification image, and output region numbering image, letter image and digital picture；Specifically , the order with specific step-length from the upper left corner to the lower right corner on license plate image is traveled through, collection identification image.To know again Other image input judges that neutral net judges area number, the letter and number on the identification image, and output region numbering figure Picture, letter image and digital picture.Specifically, identification image obtains pretreated identification image by pretreatment, pre-process Identification image afterwards simply judges that neutral net is judged into one, the output of the judgement neutral net totally four class, point Not Wei area number, letter, numeral and other.If judged result is other, abandon currently recognizing image, judge next knowledge Other image；If judged result area number, the area number that identification image enters the Car license recognition stage is recognized；If judged result It is letter, identification image enters the Letter identification in Car license recognition stage；If judged result is digital, identification image then enters car plate The numeral identification of cognitive phase.Then respectively by the input area number identification of area number image, letter image and digital picture Convolutional neural networks, Letter identification convolutional neural networks and numeral identification convolutional neural networks carry out area number, alphabetical sum The identification of word, and output region numbering, letter and number respectively；Further according to area number, the letter and number of output, by " The form output license plate number recognition result of area numbering alphanumeric ".
Wherein, implement the license plate recognition device described by Fig. 5, space switching network is introduced in convolutional neural networks and is opened Amount is decomposed, and from classification accuracy and the angle of network size, improves traditional convolutional network, and by multiple convolutional neural networks Multistage segmentation is identified to car plate, and the scale for possessing accurate classifying quality network simultaneously is moderate, can be embedded into movement and set It is standby.
Example IV
Fig. 6 is referred to, Fig. 6 is the structural representation of another license plate recognition device disclosed in the embodiment of the present invention.Wherein, License plate recognition device shown in Fig. 6 is because the license plate recognition device shown in Fig. 5 optimizes what is obtained.
Further, in the license plate recognition device shown in Fig. 6, vehicle search module 302 includes：
Vehicle image processing unit 3021, for being carried out vehicle image according to space switching network and convolutional neural networks Multiple geometric space conversion correction, process of convolution and pondization treatment, obtain fisrt feature mapping graph；Specifically, first according to sky Between switching network and convolutional neural networks vehicle image is carried out into multiple geometric space conversion correction, process of convolution and Chi Huachu Reason, i.e., first geometric space conversion correction, then process of convolution, then pondization treatment, then again since geometric space conversion correction Treatment, so circulation are multiple, obtain fisrt feature mapping graph.
Weight matrix acquiring unit 3022, for the vectorization of fisrt feature mapping graph to be exported into column vector, and connects multiple Full articulamentum obtains the weight matrix of full articulamentum；
Tensor resolution unit 3023, decomposes for the weight matrix of full articulamentum to be carried out into tensor row, extracts fisrt feature Vector；
Classification predicting unit 3024, the output layer for first eigenvector to be input into the convolutional neural networks is divided Class is predicted and optimizes adjustable parameter, and search whether there is vehicle.
Further, Plate searching module 303 includes：
License plate image processing unit 3031, for being carried out license plate image according to space switching network and convolutional neural networks Multiple geometric space conversion correction, process of convolution and pondization treatment, obtain second feature mapping graph；Specifically, first according to sky Between switching network and convolutional neural networks license plate image is carried out into multiple geometric space conversion correction, process of convolution and Chi Huachu Reason, i.e., first geometric space conversion correction, then process of convolution, then pondization treatment, then again since geometric space conversion correction Treatment, so circulation are multiple, obtain second feature mapping graph.
Weight matrix acquiring unit 3032, for the vectorization of second feature mapping graph to be exported into column vector, and connects multiple Full articulamentum obtains the weight matrix of full articulamentum；
Tensor resolution unit 3033, decomposes for the weight matrix of full articulamentum to be carried out into tensor row, extracts second feature Vector；
Classification predicting unit 3034, for carrying out classifying pre- the output layer of second feature vector input convolutional neural networks Adjustable parameter is surveyed and optimizes, search whether there is car plate.
Further, exception monitoring module 304 includes：
Tensor neutral net unit 3041, corresponding direct probability density is obtained for the license plate image according to input Than；
Comparing unit 3042, it is critical for what is obtained after the training by direct probability density ratio with completion tensor neutral net Direct probability density ratio is compared, and is normal picture if direct probability density ratio is more than critical direct probability density ratio, Otherwise it is abnormal image.
Wherein, the license plate recognition device described by Fig. 6 is implemented, the full articulamentum in convolutional neural networks introduces tensor point Xie Hou, network is renamed as tensor neutral net, and the storage of the feedforward transmission, error back propagation and weights of its full articulamentum is more New capital changes；Space switching network is re-introduced into the present embodiment on this basis in tensor neutral net, specifically, Embedded space modular converter before each convolutional layer, useful space geometric transformation correction is carried out to convolution input, effectively alleviates volume The lamination conversion limitation without space-invariance larger to input picture or Feature Mapping figure, improves convolution output space not Denaturation, and then make the feature of extraction more representative, improve the classification degree of accuracy of output layer.Further, the computing of full articulamentum Carried out using the pattern of tensor, effectively alleviate the limitation of the spatial information loss that full connect band is come.Also, Vehicle License Plate Recognition System is adopted Use multiple convolutional networks, multi-stage strategy to realize Car license recognition, the complex process of Car license recognition is decomposed into multiple simple sons Process, i.e. vehicle detection, car plate detection, license plate number identification, license plate number identification is again including area number identification, Letter identification sum Word identification, effectively increases the efficiency of Car license recognition.Furthermore, increase before Car license recognition and be based on direct probability density ratio and tensor The abnormality detection of neutral net, effectively accurately identifies abnormal data, and Exception Filter data reduce unnecessary computing, improve system The efficiency of system.
Embodiment five
Fig. 7 is referred to, Fig. 7 is a kind of structural representation of user equipment disclosed in the embodiment of the present invention.Wherein, Fig. 7 institutes The user equipment for showing includes any one license plate recognition device of Fig. 5~Fig. 6.Implement the user equipment shown in Fig. 7, in convolutional Neural Space switching network and tensor resolution are introduced in network, from classification accuracy and the angle of network size, traditional convolution is improved Network, and being identified to car plate by the multistage segmentation of multiple convolutional neural networks, possess accurate classifying quality net simultaneously The scale of network is moderate, can be embedded into mobile device.
One of ordinary skill in the art will appreciate that all or part of step in the various methods of above-described embodiment is can Completed with instructing the hardware of correlation by program, the program can be stored in a computer-readable recording medium, storage Medium include read-only storage (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), programmable read only memory (Programmable Read-only Memory, PROM), erasable programmable is read-only deposits Reservoir (Erasable Programmable Read Only Memory, EPROM), disposable programmable read-only storage (One- Time Programmable Read-Only Memory, OTPROM), the electronics formula of erasing can make carbon copies read-only storage (Electrically-Erasable Programmable Read-Only Memory, EEPROM), read-only optical disc (Compact Disc Read-Only Memory, CD-ROM) or other disk storages, magnetic disk storage, magnetic tape storage or can For carrying or computer-readable any other medium of data storage.
Detailed Jie has been carried out to a kind of licence plate recognition method disclosed in the embodiment of the present invention and device, user equipment above Continue, specific case used herein is set forth to principle of the invention and implementation method, the explanation of above example is only It is to be used to help understand the method for the present invention and its core concept；Simultaneously for those of ordinary skill in the art, according to this hair Bright thought, be will change in specific embodiments and applications, and in sum, this specification content should not be managed It is limitation of the present invention to solve.

Claims (10)

1. a kind of licence plate recognition method, it is characterised in that including：
Vehicle image is obtained, and whether vehicle image input vehicle retrieval convolutional neural networks are searched for into the vehicle image There is vehicle；
If the vehicle image has vehicle, license plate image is gathered on the vehicle image, and by the license plate image Input license number search convolutional neural networks search for the license plate image and whether there is car plate；
If the license plate image has car plate, license plate image input tensor neutral net is detected into the license plate image It is whether abnormal；
If without exception, identification image is gathered on the license plate image, and the identification image input is judged into nerve Network judges area number, the letter and number on the identification image, and output region numbering image, letter image and numeral Image；
Respectively by the area number image, the letter image and the digital picture input area number identification convolutional Neural Network, Letter identification convolutional neural networks and numeral identification convolutional neural networks carry out the identification of area number, letter and number, And difference output region numbering, letter and number；
According to area number, the letter and number of output, by the form output license plate number identification knot of " area number alphanumeric " Really.
2. method according to claim 1, it is characterised in that the acquisition vehicle image, and the vehicle image is defeated Entering the step of vehicle retrieval convolutional neural networks search for the vehicle image with the presence or absence of vehicle includes：
The vehicle image in video stream data is obtained, the vehicle image is represented with tensor pattern；
The vehicle image is carried out by multiple geometric space conversion correction, convolution according to space switching network and convolutional neural networks Treatment and pondization treatment, obtain fisrt feature mapping graph；
The vectorization of fisrt feature mapping graph is exported into column vector, and connects the weights square that multiple full articulamentums obtain full articulamentum Battle array；
The weight matrix of the full articulamentum is carried out into tensor row to decompose, first eigenvector is extracted；
The output layer that first eigenvector is input into the convolutional neural networks is carried out into classification to predict and optimize adjustable parameter, is searched for With the presence or absence of vehicle.
3. method according to claim 2, it is characterised in that described that license plate image is input into license number search convolutional Neural net Network is searched for the step of the license plate image whether there is car plate to be included：
The license plate image is carried out by multiple geometric space conversion correction, convolution according to space switching network and convolutional neural networks Treatment and pondization treatment, obtain second feature mapping graph；
The vectorization of second feature mapping graph is exported into column vector, and connects the weights square that multiple full articulamentums obtain full articulamentum Battle array；
The weight matrix of the full articulamentum is carried out into tensor row to decompose, second feature vector is extracted；
The output layer of the second feature vector input convolutional neural networks is carried out into classification to predict and optimize adjustable parameter, is searched for With the presence or absence of car plate.
4. the method according to claim 2-3 any one, it is characterised in that specific the step of the optimization adjustable parameter Be according to direction of error propagation algorithm optimize network in all adjustable parameters, i.e., to space conversion matrix, weights and skew more Newly, be the error returned according to later layer anti-pass carry out partial gradient calculate carry out.
5. method according to claim 4, it is characterised in that described by license plate image input tensor neutral net detection institute Whether abnormal step includes to state license plate image：
The license plate image is directly inputted and obtain its corresponding direct probability density ratio in the tensor neutral net for training；
The critical direct probability density ratio for being obtained after the training by the direct probability density ratio with completion tensor neutral net again Be compared, if the direct probability density ratio be more than the critical direct probability density ratio, be normal picture, otherwise for Abnormal image.
6. a kind of license plate recognition device, it is characterised in that including：
Data acquisition module, for the collection vehicle image in video streaming image；License plate image is gathered in the vehicle image； And identification image is gathered in the license plate image；
Vehicle search module, be for vehicle image input vehicle retrieval convolutional neural networks to be searched for into the vehicle image It is no to there is vehicle；
Plate searching module, be for license plate image input license number search convolutional neural networks to be searched for into the license plate image It is no to there is car plate；
Exception monitoring module, for license plate image input tensor neutral net to be checked into whether the license plate image is abnormal；
Car license recognition module, for the identification image input to be judged into neutral net judges that the area on the identification image is compiled Number, letter and number, and output region numbering image, letter image and digital picture；Then respectively by the area number figure Picture, the letter image and the digital picture input area number identification convolutional neural networks, Letter identification convolutional Neural net Network and numeral identification convolutional neural networks carry out the identification of area number, letter and number, and output region numbering, letter respectively And numeral；Further according to area number, the letter and number of output, know by the form output license plate number of " area number alphanumeric " Other result.
7. device according to claim 6, it is characterised in that the vehicle search module includes：
Vehicle image processing unit, for being carried out repeatedly the vehicle image according to space switching network and convolutional neural networks Geometric space conversion correction, process of convolution and pondization treatment, obtain fisrt feature mapping graph；
Weight matrix acquiring unit, for the vectorization of fisrt feature mapping graph to be exported into column vector, and connects multiple full articulamentums Obtain the weight matrix of full articulamentum；
Tensor resolution unit, decomposes for the weight matrix of the full articulamentum to be carried out into tensor row, extracts first eigenvector；
Classification predicting unit, the output layer for first eigenvector to be input into the convolutional neural networks carries out classification prediction simultaneously Optimization adjustable parameter, search whether there is vehicle.
8. device according to claim 7, it is characterised in that the Plate searching module includes：
License plate image processing unit, for being carried out repeatedly the license plate image according to space switching network and convolutional neural networks Geometric space conversion correction, process of convolution and pondization treatment, obtain second feature mapping graph；
Weight matrix acquiring unit, for the vectorization of second feature mapping graph to be exported into column vector, and connects multiple full articulamentums Obtain the weight matrix of full articulamentum；
Tensor resolution unit, decomposes for the weight matrix of the full articulamentum to be carried out into tensor row, extracts second feature vector；
Classification predicting unit, for the output layer of the second feature vector input convolutional neural networks to be carried out into classification prediction simultaneously Optimization adjustable parameter, search whether there is car plate.
9. device according to claim 8, it is characterised in that the exception monitoring module includes：
Tensor neutral net unit, corresponding direct probability density ratio is obtained for the license plate image according to input；
Comparing unit, it is critical direct for what is obtained after the training by the direct probability density ratio with completion tensor neutral net Probability density ratio is compared, if the direct probability density ratio is more than the critical direct probability density ratio, for normal Image, is otherwise abnormal image.
10. a kind of user equipment, it is characterised in that including the Car license recognition described in claim 6~claim 9 any one Device.
CN201611207149.2A 2016-12-23 2016-12-23 A kind of licence plate recognition method and device, user equipment Pending CN106778745A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611207149.2A CN106778745A (en) 2016-12-23 2016-12-23 A kind of licence plate recognition method and device, user equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611207149.2A CN106778745A (en) 2016-12-23 2016-12-23 A kind of licence plate recognition method and device, user equipment

Publications (1)

Publication Number Publication Date
CN106778745A true CN106778745A (en) 2017-05-31

Family

ID=58920074

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611207149.2A Pending CN106778745A (en) 2016-12-23 2016-12-23 A kind of licence plate recognition method and device, user equipment

Country Status (1)

Country Link
CN (1) CN106778745A (en)

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107590492A (en) * 2017-08-28 2018-01-16 浙江工业大学 A kind of vehicle-logo location and recognition methods based on convolutional neural networks
CN107798697A (en) * 2017-10-26 2018-03-13 中国科学院深圳先进技术研究院 A kind of medical image registration method based on convolutional neural networks, system and electronic equipment
CN108073888A (en) * 2017-08-07 2018-05-25 中国科学院深圳先进技术研究院 A kind of teaching auxiliary and the teaching auxiliary system using this method
CN108229463A (en) * 2018-02-07 2018-06-29 众安信息技术服务有限公司 Character recognition method based on image
CN108229474A (en) * 2017-12-29 2018-06-29 北京旷视科技有限公司 Licence plate recognition method, device and electronic equipment
WO2019028592A1 (en) * 2017-08-07 2019-02-14 中国科学院深圳先进技术研究院 Teaching assistance method and teaching assistance system using said method
CN109829848A (en) * 2019-01-17 2019-05-31 柳州康云互联科技有限公司 A kind of system and method for Image space transformation neural network based in internet detection
CN110245664A (en) * 2019-06-26 2019-09-17 深兰科技（上海）有限公司 Licence plate recognition method
CN110321884A (en) * 2019-06-13 2019-10-11 贝式计算(天津)信息技术有限公司 Number the method and device of identification
CN110443245A (en) * 2019-08-14 2019-11-12 上海世茂物联网科技有限公司 Localization method, device and the equipment of a kind of license plate area under unrestricted scene
CN110533178A (en) * 2018-05-25 2019-12-03 杭州海康威视数字技术股份有限公司 A kind of neural network model training method, apparatus and system
CN110826567A (en) * 2019-11-06 2020-02-21 北京字节跳动网络技术有限公司 Optical character recognition method, device, equipment and storage medium
CN111353517A (en) * 2018-12-24 2020-06-30 杭州海康威视数字技术股份有限公司 License plate recognition method and device and electronic equipment
CN111461979A (en) * 2020-03-30 2020-07-28 招商局金融科技有限公司 Verification code image denoising and identifying method, electronic device and storage medium
US10777076B2 (en) 2018-09-19 2020-09-15 National Chung-Shan Institute Of Science And Technology License plate recognition system and license plate recognition method
CN111689099A (en) * 2020-06-22 2020-09-22 华东理工大学 ROS-based business building garbage recycling robot
CN112348177A (en) * 2019-07-05 2021-02-09 安徽寒武纪信息科技有限公司 Neural network model verification method and device, computer equipment and storage medium
CN112418168A (en) * 2020-12-10 2021-02-26 深圳云天励飞技术股份有限公司 Vehicle identification method, device, system, electronic equipment and storage medium
CN113159158A (en) * 2021-04-16 2021-07-23 西安建筑科技大学 License plate correction and reconstruction method and system based on generation countermeasure network
WO2022016925A1 (en) * 2020-07-22 2022-01-27 深圳云天励飞技术股份有限公司 Neural network computing device

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101344988A (en) * 2008-06-16 2009-01-14 上海高德威智能交通系统有限公司 Image acquisition and processing equipment and method, vehicle monitoring and recording system
CN102346847A (en) * 2011-09-26 2012-02-08 青岛海信网络科技股份有限公司 License plate character recognizing method of support vector machine
CN102663377A (en) * 2012-03-15 2012-09-12 华中科技大学 Character recognition method based on template matching
US8345921B1 (en) * 2009-03-10 2013-01-01 Google Inc. Object detection with false positive filtering
CN104268509A (en) * 2014-09-17 2015-01-07 深圳市捷顺科技实业股份有限公司 Method and system for detecting license plate of dump truck
CN104299006A (en) * 2014-07-23 2015-01-21 中国传媒大学 Vehicle license plate recognition method based on deep neural network
CN104298976A (en) * 2014-10-16 2015-01-21 电子科技大学 License plate detection method based on convolutional neural network
CN104809443A (en) * 2015-05-05 2015-07-29 上海交通大学 Convolutional neural network-based license plate detection method and system
CN105447457A (en) * 2015-11-15 2016-03-30 安徽清新互联信息科技有限公司 License plate character identification method based on adaptive characteristic
CN105844653A (en) * 2016-04-18 2016-08-10 深圳先进技术研究院 Multilayer convolution neural network optimization system and method
US20160267326A1 (en) * 2015-03-10 2016-09-15 Fringefy Ltd. Image abstraction system
CN106022232A (en) * 2016-05-12 2016-10-12 成都新舟锐视科技有限公司 License plate detection method based on deep learning

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101344988A (en) * 2008-06-16 2009-01-14 上海高德威智能交通系统有限公司 Image acquisition and processing equipment and method, vehicle monitoring and recording system
US8345921B1 (en) * 2009-03-10 2013-01-01 Google Inc. Object detection with false positive filtering
CN102346847A (en) * 2011-09-26 2012-02-08 青岛海信网络科技股份有限公司 License plate character recognizing method of support vector machine
CN102663377A (en) * 2012-03-15 2012-09-12 华中科技大学 Character recognition method based on template matching
CN104299006A (en) * 2014-07-23 2015-01-21 中国传媒大学 Vehicle license plate recognition method based on deep neural network
CN104268509A (en) * 2014-09-17 2015-01-07 深圳市捷顺科技实业股份有限公司 Method and system for detecting license plate of dump truck
CN104298976A (en) * 2014-10-16 2015-01-21 电子科技大学 License plate detection method based on convolutional neural network
US20160267326A1 (en) * 2015-03-10 2016-09-15 Fringefy Ltd. Image abstraction system
CN104809443A (en) * 2015-05-05 2015-07-29 上海交通大学 Convolutional neural network-based license plate detection method and system
CN105447457A (en) * 2015-11-15 2016-03-30 安徽清新互联信息科技有限公司 License plate character identification method based on adaptive characteristic
CN105844653A (en) * 2016-04-18 2016-08-10 深圳先进技术研究院 Multilayer convolution neural network optimization system and method
CN106022232A (en) * 2016-05-12 2016-10-12 成都新舟锐视科技有限公司 License plate detection method based on deep learning

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
王宇宁 等: "基于YOLO算法的车辆实时检测", 《武汉理工大学学报》 *

Cited By (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108073888A (en) * 2017-08-07 2018-05-25 中国科学院深圳先进技术研究院 A kind of teaching auxiliary and the teaching auxiliary system using this method
WO2019028592A1 (en) * 2017-08-07 2019-02-14 中国科学院深圳先进技术研究院 Teaching assistance method and teaching assistance system using said method
US11270526B2 (en) 2017-08-07 2022-03-08 Shenzhen Institutes Of Advanced Technology Chinese Academy Of Sciences Teaching assistance method and teaching assistance system using said method
CN107590492B (en) * 2017-08-28 2019-11-19 浙江工业大学 A kind of vehicle-logo location and recognition methods based on convolutional neural networks
CN107590492A (en) * 2017-08-28 2018-01-16 浙江工业大学 A kind of vehicle-logo location and recognition methods based on convolutional neural networks
CN107798697A (en) * 2017-10-26 2018-03-13 中国科学院深圳先进技术研究院 A kind of medical image registration method based on convolutional neural networks, system and electronic equipment
CN108229474A (en) * 2017-12-29 2018-06-29 北京旷视科技有限公司 Licence plate recognition method, device and electronic equipment
CN108229474B (en) * 2017-12-29 2019-10-01 北京旷视科技有限公司 Licence plate recognition method, device and electronic equipment
CN108229463A (en) * 2018-02-07 2018-06-29 众安信息技术服务有限公司 Character recognition method based on image
CN110533178A (en) * 2018-05-25 2019-12-03 杭州海康威视数字技术股份有限公司 A kind of neural network model training method, apparatus and system
US10777076B2 (en) 2018-09-19 2020-09-15 National Chung-Shan Institute Of Science And Technology License plate recognition system and license plate recognition method
CN111353517B (en) * 2018-12-24 2023-09-26 杭州海康威视数字技术股份有限公司 License plate recognition method and device and electronic equipment
CN111353517A (en) * 2018-12-24 2020-06-30 杭州海康威视数字技术股份有限公司 License plate recognition method and device and electronic equipment
CN109829848A (en) * 2019-01-17 2019-05-31 柳州康云互联科技有限公司 A kind of system and method for Image space transformation neural network based in internet detection
CN110321884A (en) * 2019-06-13 2019-10-11 贝式计算(天津)信息技术有限公司 Number the method and device of identification
CN110245664A (en) * 2019-06-26 2019-09-17 深兰科技（上海）有限公司 Licence plate recognition method
CN112348177A (en) * 2019-07-05 2021-02-09 安徽寒武纪信息科技有限公司 Neural network model verification method and device, computer equipment and storage medium
CN112348177B (en) * 2019-07-05 2024-01-09 安徽寒武纪信息科技有限公司 Neural network model verification method, device, computer equipment and storage medium
CN110443245B (en) * 2019-08-14 2022-02-15 上海世茂物联网科技有限公司 License plate region positioning method, device and equipment in non-limited scene
CN110443245A (en) * 2019-08-14 2019-11-12 上海世茂物联网科技有限公司 Localization method, device and the equipment of a kind of license plate area under unrestricted scene
CN110826567A (en) * 2019-11-06 2020-02-21 北京字节跳动网络技术有限公司 Optical character recognition method, device, equipment and storage medium
CN111461979A (en) * 2020-03-30 2020-07-28 招商局金融科技有限公司 Verification code image denoising and identifying method, electronic device and storage medium
CN111461979B (en) * 2020-03-30 2023-04-18 招商局金融科技有限公司 Verification code image denoising and identifying method, electronic device and storage medium
CN111689099A (en) * 2020-06-22 2020-09-22 华东理工大学 ROS-based business building garbage recycling robot
WO2022016925A1 (en) * 2020-07-22 2022-01-27 深圳云天励飞技术股份有限公司 Neural network computing device
CN112418168A (en) * 2020-12-10 2021-02-26 深圳云天励飞技术股份有限公司 Vehicle identification method, device, system, electronic equipment and storage medium
CN112418168B (en) * 2020-12-10 2024-04-02 深圳云天励飞技术股份有限公司 Vehicle identification method, device, system, electronic equipment and storage medium
CN113159158A (en) * 2021-04-16 2021-07-23 西安建筑科技大学 License plate correction and reconstruction method and system based on generation countermeasure network

Similar Documents

Publication Publication Date Title
CN106778745A (en) A kind of licence plate recognition method and device, user equipment
Goncharov et al. Disease detection on the plant leaves by deep learning
WO2018112900A1 (en) License plate recognition method and apparatus, and user equipment
CN110288018A (en) A kind of WiFi personal identification method merging deep learning model
CN110175671A (en) Construction method, image processing method and the device of neural network
CN109614981A (en) The Power System Intelligent fault detection method and system of convolutional neural networks based on Spearman rank correlation
Han et al. Convolutional neural network learning for generic data classification
Xiong et al. AI-NET: Attention inception neural networks for hyperspectral image classification
CN103345643B (en) A kind of Classifying Method in Remote Sensing Image
CN108010029A (en) Fabric defect detection method based on deep learning and support vector data description
CN105787557A (en) Design method of deep nerve network structure for computer intelligent identification
CN111783937A (en) Neural network construction method and system
CN110222760A (en) A kind of fast image processing method based on winograd algorithm
CN109711401A (en) A kind of Method for text detection in natural scene image based on Faster Rcnn
CN111523713A (en) Method and device for predicting residual oil saturation distribution in oil field
CN113191390B (en) Image classification model construction method, image classification method and storage medium
Tong et al. Detection of urban sprawl using a genetic algorithm-evolved artificial neural network classification in remote sensing: a case study in Jiading and Putuo districts of Shanghai, China
CN108205703A (en) Multi-input multi-output matrix average value pooling vectorization implementation method
CN110309835A (en) A kind of image local feature extracting method and device
CN107067410A (en) A kind of manifold regularization correlation filtering method for tracking target based on augmented sample
CN107301643A (en) Well-marked target detection method based on robust rarefaction representation Yu Laplce's regular terms
CN109978074A (en) Image aesthetic feeling and emotion joint classification method and system based on depth multi-task learning
CN109671055A (en) Pulmonary nodule detection method and device
CN101546389A (en) Primary direction neural network system
Pandhiani et al. Time series forecasting by using hybrid models for monthly streamflow data

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20170531