CN109086765B

CN109086765B - Licence plate recognition method, device, medium, server and automobile data recorder

Info

Publication number: CN109086765B
Application number: CN201810861038.6A
Authority: CN
Inventors: 李友增; 阮腾; 赵震
Original assignee: Beijing Didi Infinity Technology and Development Co Ltd
Current assignee: Beijing Didi Infinity Technology and Development Co Ltd
Priority date: 2018-08-01
Filing date: 2018-08-01
Publication date: 2019-09-17
Anticipated expiration: 2038-08-01
Also published as: CN109086765A

Abstract

This application provides licence plate recognition method, device, medium, server and automobile data recorders, are related to field of image recognition.Licence plate recognition method provided by the present application, using being known otherwise to license plate picture to be identified using Chinese Character Recognition neural network and non-Chinese Character Recognition neural network simultaneously, so that Car license recognition is more targeted, known otherwise compared in traditional technology using Chinese, English of the same neural network to license plate, method provided herein is in identification, the global feature of the minutia of Chinese and English, number is considered simultaneously, and then can be improved the accuracy of identification.

Description

Licence plate recognition method, device, medium, server and automobile data recorder

Technical field

This application involves field of image recognition, in particular to licence plate recognition method, device, medium, server and row Vehicle recorder.

Background technique

With being constantly progressive for information technology, license plate recognition technology is widely applied what all trades and professions all obtained, vehicle Park metered technology, suspect's car tracing technology is dependent on license plate recognition technology.

In the related technology, license plate recognition technology can be basically completed the identification to vehicle license under specified environment, but Existing license plate recognition technology still has certain deficiency.

Summary of the invention

The application's is designed to provide licence plate recognition method.

In a first aspect, the embodiment of the present application provides licence plate recognition method, comprising:

Obtain license plate picture to be identified；

License plate picture to be identified is identified using Chinese Character Recognition neural network and non-Chinese Character Recognition neural network respectively, Obtain the Chinese Character Recognition result and non-Chinese Character Recognition result of license plate；

According to Chinese Character Recognition result and non-Chinese Character Recognition as a result, generating license plate number information.

With reference to first aspect, the embodiment of the present application provides the first possible embodiment of first aspect, wherein step Suddenly license plate picture to be identified is identified using Chinese Character Recognition neural network and non-Chinese Character Recognition neural network respectively, obtains vehicle The Chinese Character Recognition result of board and non-Chinese Character Recognition result include:

By the first shared convolutional neural networks of license plate picture input Chinese Character Recognition neural network to be identified, to extract license plate The first eigenvector of picture to be identified；

By the full linking layer of Chinese Character Recognition of first eigenvector input Chinese Character Recognition neural network, to obtain Chinese Character Recognition knot Fruit；

First eigenvector is inputted to the full linking layer of non-Chinese Character Recognition of non-Chinese Character Recognition neural network, to obtain non-Chinese character Recognition result.

With reference to first aspect, the embodiment of the present application provides second of possible embodiment of first aspect, wherein step Suddenly to be identified to extract license plate by the first shared convolutional neural networks of license plate picture input Chinese Character Recognition neural network to be identified The first eigenvector of picture includes:

License plate picture sequence to be identified is inputted at least one feature extraction network in the first shared convolutional neural networks Group, the feature for making the feature vector of previous feature extraction group of networks output as next feature extraction group of networks and inputting to Amount, until the last one feature extraction group of networks exports first eigenvector；Feature extraction group of networks includes at least one layer of convolution Layer and at least two layers of residual error layer.

With reference to first aspect, the embodiment of the present application provides the third possible embodiment of first aspect, wherein step Suddenly license plate picture to be identified is identified using Chinese Character Recognition neural network and non-Chinese Character Recognition neural network respectively, obtains vehicle The Chinese Character Recognition result of board and non-Chinese Character Recognition result include:

License plate picture to be identified is inputted into the second shared convolutional neural networks, it is special with extract license plate picture to be identified second Levy vector；

By the first Chinese Character Recognition convolutional neural networks of second feature vector input Chinese Character Recognition neural network, to extract vehicle The third feature vector of board picture to be identified；

By the full linking layer of Chinese Character Recognition of third feature vector input Chinese Character Recognition neural network, to obtain Chinese Character Recognition knot Fruit；

Second feature vector is inputted to the first non-Chinese Character Recognition convolutional neural networks of non-Chinese Character Recognition neural network, to mention The fourth feature vector of pick-up board picture to be identified；

Fourth feature vector is inputted to the full linking layer of non-Chinese Character Recognition of non-Chinese Character Recognition neural network, to obtain non-Chinese character Recognition result.

With reference to first aspect, the embodiment of the present application provides the 4th kind of possible embodiment of first aspect, wherein

The number of dimensions for the feature vector that first Chinese Character Recognition convolutional neural networks are exported is rolled up with the first non-Chinese Character Recognition The ratio of the number of dimensions for the feature vector that product neural network is exported is greater than 1/4 less than 1/2；

The quantity of convolutional layer is less than the first non-Chinese Character Recognition convolutional neural networks in first Chinese Character Recognition convolutional neural networks The quantity of middle convolutional layer.

With reference to first aspect, the embodiment of the present application provides the 5th kind of possible embodiment of first aspect, wherein the Two shared convolutional neural networks, in the first Chinese Character Recognition convolutional neural networks and the first non-Chinese Character Recognition convolutional neural networks At least one is at least made of spaced convolutional layer and residual error layer.

With reference to first aspect, the embodiment of the present application provides the 6th kind of possible embodiment of first aspect, wherein obtains Pick-up board picture to be identified, comprising:

According to different interception ways, multiple candidate images are intercepted from the environment photo taken；

Using feature extraction network, the reference index of each candidate image is calculated separately；

Reference index is selected to meet the candidate image of preset requirement as license plate picture to be identified.

With reference to first aspect, the embodiment of the present application provides the 7th kind of possible embodiment of first aspect, wherein also Include:

Obtain the license plate picture sample by artificial mark Chinese character information；

It will be input to Chinese Character Recognition neural network without the license plate picture sample manually marked, obtains Chinese character neural network Recognition result；

Compare the difference by the license plate picture sample and Chinese character neural network recognization result that manually mark, obtains Chinese character knowledge Other error；

According to the parameter of Chinese Character Recognition error transfer factor Chinese Character Recognition neural network.

With reference to first aspect, the embodiment of the present application provides the 8th kind of possible embodiment of first aspect, wherein also Include:

Obtain the license plate picture sample by manually marking non-Chinese character information；

It will be input to Chinese Character Recognition neural network without the license plate picture sample manually marked, obtains non-Chinese character nerve net Network recognition result；

Compare the difference by the license plate picture sample and non-Chinese character neural network recognization result that manually mark, obtains the non-Chinese Word identification error；

According to the parameter of the non-Chinese Character Recognition neural network of non-Chinese Character Recognition error transfer factor.

Second aspect, the embodiment of the present application also provides a kind of license plate recognition devices, comprising:

First obtains module, for obtaining license plate picture to be identified；

Identification module, for be identified to license plate using Chinese Character Recognition neural network and non-Chinese Character Recognition neural network respectively Picture is identified, the Chinese Character Recognition result and non-Chinese Character Recognition result of license plate are obtained；

Generation module is used for according to Chinese Character Recognition result and non-Chinese Character Recognition as a result, generating license plate number information.

In conjunction with second aspect, the embodiment of the present application provides the first possible embodiment of second aspect, wherein knows Other module includes:

First input unit, for the first shared convolution of license plate picture input Chinese Character Recognition neural network to be identified is refreshing Through network, to extract the first eigenvector of license plate picture to be identified；

Second input unit, for linking the Chinese Character Recognition of first eigenvector input Chinese Character Recognition neural network entirely Layer, to obtain Chinese Character Recognition result；

Third input unit, for first eigenvector to be inputted to the full chain of non-Chinese Character Recognition of non-Chinese Character Recognition neural network Layer is connect, to obtain non-Chinese Character Recognition result.

In conjunction with second aspect, the embodiment of the present application provides second of possible embodiment of second aspect, wherein the One input unit includes:

First input subelement, for inputting in the first shared convolutional neural networks extremely license plate picture sequence to be identified A few feature extraction group of networks makes the feature vector of previous feature extraction group of networks output as next feature extraction net The feature vector of network group input, until the last one feature extraction group of networks exports first eigenvector；Feature extraction group of networks Including at least one layer of convolutional layer and at least two layers of residual error layer.

In conjunction with second aspect, the embodiment of the present application provides the third possible embodiment of second aspect, wherein knows Other module includes:

4th input unit, for license plate picture to be identified to be inputted the second shared convolutional neural networks, to extract license plate The second feature vector of picture to be identified；

5th input unit, for second feature vector to be inputted to the first Chinese Character Recognition convolution of Chinese Character Recognition neural network Neural network, to extract the third feature vector of license plate picture to be identified；

6th input unit, for linking the Chinese Character Recognition of third feature vector input Chinese Character Recognition neural network entirely Layer, to obtain Chinese Character Recognition result；

7th input unit, for second feature vector to be inputted to the first non-Chinese Character Recognition of non-Chinese Character Recognition neural network Convolutional neural networks, to extract the fourth feature vector of license plate picture to be identified；

8th input unit, for fourth feature vector to be inputted to the full chain of non-Chinese Character Recognition of non-Chinese Character Recognition neural network Layer is connect, to obtain non-Chinese Character Recognition result.

In conjunction with second aspect, the embodiment of the present application provides the 4th kind of possible embodiment of second aspect, wherein

In conjunction with second aspect, the embodiment of the present application provides the 5th kind of possible embodiment of second aspect, wherein the Two shared convolutional neural networks, in the first Chinese Character Recognition convolutional neural networks and the first non-Chinese Character Recognition convolutional neural networks At least one is at least made of spaced convolutional layer and residual error layer.

In conjunction with second aspect, the embodiment of the present application provides the 6th kind of possible embodiment of second aspect, wherein the One, which obtains module, includes:

Interception unit, for intercepting multiple candidate images from the environment photo taken according to different interception ways；

Computing unit calculates separately the reference index of each candidate image for using feature extraction network；

Selecting unit, for selecting reference index to meet the candidate image of preset requirement as license plate picture to be identified.

In conjunction with second aspect, the embodiment of the present application provides the 7th kind of possible embodiment of second aspect, wherein also Include:

Second obtains module, for obtaining the license plate picture sample by artificial mark Chinese character information；

First input module, for Chinese Character Recognition nerve net will to be input to without the license plate picture sample manually marked Network obtains Chinese character neural network recognization result；

First comparison module, for comparing license plate picture sample and Chinese character neural network recognization result by manually marking Difference, obtain Chinese Character Recognition error；

The first adjustment module, for the parameter according to Chinese Character Recognition error transfer factor Chinese Character Recognition neural network.

In conjunction with second aspect, the embodiment of the present application provides the 8th kind of possible embodiment of second aspect, wherein also Include:

Third obtains module, for obtaining the license plate picture sample by manually marking non-Chinese character information；

Second input module, for Chinese Character Recognition nerve net will to be input to without the license plate picture sample manually marked Network obtains non-Chinese character neural network recognization result；

Second comparison module, for comparing license plate picture sample and non-Chinese character neural network recognization knot by manually marking The difference of fruit obtains non-Chinese Character Recognition error；

Second adjustment module, for the parameter according to the non-Chinese Character Recognition neural network of non-Chinese Character Recognition error transfer factor.

The third aspect, the embodiment of the present application also provides a kind of non-volatile program codes that can be performed with processor Computer-readable medium, program code make processor execute first aspect either method.

Fourth aspect includes: processor, memory and bus, storage the embodiment of the present application also provides a kind of server Device, which is stored with, to be executed instruction, and when server operation, by bus communication between processor and memory, processor executes storage Stored in device such as first aspect either method.

5th aspect, the embodiment of the present application also provides a kind of automobile data recorder, the automobile data recorder include: camera, Processor, memory and bus, memory, which is stored with, to be executed instruction, when calculating equipment operation, camera and processor with By bus communication between memory, camera in vehicular motion for carrying out photograph taking；Processor executes storage Stored in device such as first aspect either method.

Licence plate recognition method provided by the embodiments of the present application, using simultaneously using Chinese Character Recognition neural network and the knowledge of non-Chinese character Other neural network knows otherwise license plate picture to be identified, so that Car license recognition is more targeted, compared to traditional skill Known otherwise in art using Chinese, English of the same neural network to license plate, method provided herein is being known When other, the global feature of the minutia of Chinese and English, number is considered simultaneously, and then can be improved the standard of identification Exactness.

To enable the above objects, features, and advantages of the application to be clearer and more comprehensible, preferred embodiment is cited below particularly, and cooperate Appended attached drawing, is described in detail below.

Detailed description of the invention

Technical solution in ord to more clearly illustrate embodiments of the present application, below will be to needed in the embodiment attached Figure is briefly described, it should be understood that the following drawings illustrates only some embodiments of the application, therefore is not construed as pair The restriction of range for those of ordinary skill in the art without creative efforts, can also be according to this A little attached drawings obtain other relevant attached drawings.

Fig. 1 shows a kind of basic flow chart of licence plate recognition method provided by the embodiment of the present application；

Fig. 2 shows in a kind of licence plate recognition method provided by the embodiment of the present application, the first shared convolutional Neural is used The schematic diagram of network；

Fig. 3 is shown in a kind of licence plate recognition method provided by the embodiment of the present application, uses the second shared convolutional Neural The schematic diagram of network；

Fig. 4 is shown in a kind of licence plate recognition method provided by the embodiment of the present application, uses the first shared volume of optimization The schematic diagram of product neural network；

Fig. 5 is shown in a kind of licence plate recognition method provided by the embodiment of the present application, uses the second shared volume of optimization The schematic diagram of product neural network；

Fig. 6 is shown in a kind of licence plate recognition method provided by the embodiment of the present application, uses basic convolutional neural networks Obtain the flow diagram of car plate detection result；

Fig. 7 shows a kind of optimized flow chart of licence plate recognition method provided by the embodiment of the present application；

Fig. 8 is shown in a kind of licence plate recognition method provided by the embodiment of the present application, by the way of the extraction of anchor point frame Extract the schematic diagram of candidate image；

Fig. 9 shows server provided by the embodiment of the present application；

Figure 10 shows automobile data recorder provided by the embodiment of the present application.

Specific embodiment

Below in conjunction with attached drawing in the embodiment of the present application, technical solutions in the embodiments of the present application carries out clear, complete Ground description, it is clear that described embodiments are only a part of embodiments of the present application, instead of all the embodiments.Usually exist The component of the embodiment of the present application described and illustrated in attached drawing can be arranged and be designed with a variety of different configurations herein.Cause This, is not intended to limit claimed the application's to the detailed description of the embodiments herein provided in the accompanying drawings below Range, but it is merely representative of the selected embodiment of the application.Based on embodiments herein, those skilled in the art are not being done Every other embodiment obtained under the premise of creative work out, shall fall in the protection scope of this application.

In the related technology, it has been widely used in every field using license plate recognition technology.It is common as electronics is received Take (ETC) system, is namely based on license plate recognition technology and carrys out work.Under normal circumstances, license plate recognition technology when in use can be with It is divided into three steps, is respectively:

It step 1, include the photo of license plate by camera acquisition；

Step 2, the mode that prospect of the application extracts, the image-region where extracting license plate in photo；

Step 3, the number in license plate is identified.

In general, the step of influencing Car license recognition accuracy mainly step 3.It is directed to step 3, is risen in license plate recognition technology The initial stage of step, technical staff are usually to be determined the image outline in the image-region where license plate by the way of edge analysis (image outline is exactly edge digital on license plate) out then by the way of edge analysis, determines that image outline is corresponding Which number is.The license plate recognition technology at initial stage is merely able to that identification is very single, is merely able to a certain special license plate of identification.

Then, nerual network technique has been dissolved into license plate recognition technology by technical staff, by the way of machine learning, As long as user provides enough samples, and passes through training appropriate, it will be able to so that the neural network model that training is completed There is stronger adaptability to various types of license plates.It is, sample used in training neural network model is all real The license plate on border picture to be identified, system can pick out the feature in these photos automatically, and then complete self-teaching.Use instruction When the neural network model perfected also can make up to a certain extent artificial trained, the problem of selected characteristic inaccuracy.

Present inventor has been carried out largely in the related technology using the method that nerual network technique carries out Car license recognition Test and analysis, the present inventors considered that in the related technology using nerual network technique carry out Car license recognition method still have Shortcoming.Mainly, when being trained in the related technology to neural network, the sample used is all a complete license plate, But existing English on a complete license plate, and have Chinese, there are also number, using the same neural network to including simultaneously this When the license plate of three kinds of texts is trained, neural network needs while being concerned about the characteristic of these three texts, so that Neural network becomes no specific aim, and trained neural network is also excessively complicated.In turn, using trained neural network pair When photo is identified, accuracy and recognition efficiency are all less desirable.

For this kind of situation, present inventor provides a kind of improved licence plate recognition method, as shown in fig. 6, showing The processing flow schematic diagram of this method, as shown in Figure 1, this method comprises the following steps:

Step S101 obtains license plate picture to be identified；

Step S102, respectively using Chinese Character Recognition neural network and non-Chinese Character Recognition neural network to license plate picture to be identified It is identified, obtains the Chinese Character Recognition result and non-Chinese Character Recognition result of license plate；

Step S103, according to Chinese Character Recognition result and non-Chinese Character Recognition as a result, generating license plate number information.

Wherein, license plate picture to be identified refer to by photographic equipment take include vehicle license (license plate) photograph Piece.In step S101, which can be system (the execution master of method provided herein Body) directly taken by its photographic equipment controlled, it is also possible to be clapped by other intelligent terminals to vehicle license System is sent to after taking the photograph, certainly, system can pre-process license plate picture to be identified before executing step S101, with Improve the specific aim identified using neural network to license plate.

Step S102 is the core content of this programme, is to have used two neural networks to license plate simultaneously in step S102 Picture to be identified is identified, to obtain recognition result.The two neural networks point that license plate picture to be identified is identified Be not Chinese Character Recognition neural network and non-Chinese Character Recognition neural network, wherein Chinese Character Recognition neural network mainly to Chinese character into The neural network of row identification, therefore, the possible output result of Chinese Character Recognition neural network should be entirely chinese character, or Person's overwhelming majority is all chinese character.It is corresponding, it should be all in the possible output result of non-Chinese Character Recognition neural network It is non-chinese character (such as English character and/or numerical character), or the overwhelming majority is all non-chinese character.The Chinese is designed in this way Word identifies neural network and non-Chinese Character Recognition neural network, can make Chinese Character Recognition neural network when training, more have Specific aim, so that Chinese Character Recognition neural network is to the more accurate of Chinese Character Recognition, so that non-Chinese Character Recognition neural network is to the non-Chinese Word identifies more accurate.Clarity by license plate picture to be identified is influenced, it is possible to there are unrecognized character, Therefore, " unclear " can also be increased in the possible output result of Chinese Character Recognition neural network to be exported as a result, class As, " unclear " can also be increased in the possible output result of non-Chinese Character Recognition neural network and exported as a result.

From the point of view of general, it is the first convolutional Neural respectively that Chinese Character Recognition neural network can be to be made of at least two parts Network and the full linking layer of Chinese Character Recognition, the effect of the first convolutional neural networks are to determine (generation) about license plate picture to be identified The feature vector of predetermined dimension, the effect of the full linking layer of Chinese Character Recognition be according to feature determined by the first convolutional neural networks to It measures and determines which Chinese character the Chinese character in license plate picture to be identified is specifically.

Specifically, the output process of the full linking layer of Chinese Character Recognition is understood that be realized by two processes, is true respectively A possibility that corresponding to fixed each first candidate result (result may be exported) probability and will likely property maximum probability first wait Result is selected to export as final result.Wherein it is determined that the step of probability of a possibility that corresponding to each first candidate result, it can Being determined according to feature vector: the probability of the first candidate result " capital " is 0.98, and the probability of the first candidate result " illiteracy " is 0.01, the probability of the first candidate result " river " is 0.01.In turn, step will likely property maximum probability the first candidate result make For in final result output, so that it may export the first candidate result " capital " as final result, that is, Chinese Character Recognition As a result it is " capital ".

By taking Chinese Car license recognition as an example, the result that Chinese Character Recognition neural network is exported at least has more than 30 Chinese characters, this More than 30 Chinese characters are the abbreviations (such as capital, river) in each province of China, there are also some special Chinese characters, such as " making ", " police ", into And this more than 30 Chinese characters can be considered that (Chinese Character Recognition result can only be for the possibility output result of Chinese Character Recognition neural network One in result may be exported).

The example of above-mentioned China's Car license recognition, Chinese Character Recognition result can only generally have (30-40) more than 30 to plant situation, corresponding , non-Chinese Character Recognition result generally also only has more than 30 kinds of situations (English alphabet quantity+digital numerical shares more than 30).

Corresponding with Chinese Character Recognition neural network, non-Chinese Character Recognition neural network is also possible at least be made of two parts , it is the second convolutional neural networks and the full linking layer of non-Chinese Character Recognition respectively, the effect of the second convolutional neural networks is to determine (life At) about license plate picture to be identified predetermined dimension feature vector, the effect of the non-full linking layer of Chinese Character Recognition is according to second Feature vector determined by convolutional neural networks determines non-chinese character (such as English character, number in license plate picture to be identified Character, Arabic character etc.) it is specifically which character.

Specifically, the output process of the full linking layer of non-Chinese Character Recognition is understood that be respectively by two processes realizations Determine probability a possibility that corresponding to each second candidate result (result may be exported) and will likely property maximum probability second Candidate result is exported as final result.Wherein it is determined that the step of probability of a possibility that corresponding to each second candidate result, It can be according to feature vector determination.For example, determining that the probability of obtained the second candidate result " A " is 0.98, second is candidate As a result the probability of " V " is 0.01, and the probability of the second candidate result " Z " is 0.01.In turn, step will likely property maximum probability Second candidate result is as in final result output, so that it may the second candidate result " A " exported as final result, Be just non-Chinese Character Recognition result be " A ".

By taking Chinese Car license recognition as an example, the non-possible output result of Chinese Character Recognition neural network can have 24 English alphabets (not having I and O in license plate) can also have 10 Arabic numerals (digital 0-9), can also be existing 24 English alphabets, again There are 10 Arabic numerals.

In scheme provided herein, Chinese Character Recognition neural network and non-Chinese Character Recognition nerve why have been used simultaneously Network identifies license plate picture to be identified, mainly the inventors of the present application found that the stroke of Chinese character is complicated, details compared with More, the stroke of English and number is relatively simple, and details is less.Neural network is when being trained for Chinese Character Recognition Guarantee recognition accuracy, neural network can focus more on the details of chinese character, in turn, if the same neural network can The existing Chinese of result can be exported, but have English or number if, then a neural network can not train enough targetedly Degree.In turn, use may export the existing Chinese of result, and have the neural network of English or number to identify license plate figure to be identified It is too low to will lead to recognition result accuracy for piece.As a result, in method provided herein, Chinese Character Recognition nerve net is used respectively Network and non-Chinese Character Recognition neural network identify license plate picture to be identified, can guarantee to a certain extent, the result of identification (Chinese Character Recognition result and non-Chinese Character Recognition result) is accurate.

Finally, in step s 103, Chinese Character Recognition result and non-Chinese Character Recognition result are synthesized a result (car plate detection As a result), and the result is exported.The result of output can be capital Q12345, wherein " capital " is using Chinese Character Recognition nerve net It is that network is identified as a result, " Q12345 " is the result identified using non-Chinese Character Recognition neural network.

Scheme provided herein when identifying to license plate, while having used Chinese Character Recognition neural network Chinese character and non-Chinese character are identified respectively with non-Chinese Character Recognition neural network, improve the accuracy of identification.

Specifically, improve treatment effeciency to optimize overall plan, as shown in Fig. 2, above-mentioned Chinese Character Recognition neural network and Non- Chinese Character Recognition neural network can be with some shared convolutional neural networks.It is, Chinese Character Recognition neural network includes Sequentially connected first shared convolutional neural networks and the full linking layer of Chinese Character Recognition are (that is, the first shared convolutional neural networks is defeated Result, which is capable of providing, out gives Chinese Character Recognition full linking layer)；Non- Chinese Character Recognition neural network includes sequentially connected first shared volume Product neural network and the full linking layer of non-Chinese Character Recognition are (that is, the output result of the first shared convolutional neural networks is capable of providing to non- The full linking layer of Chinese Character Recognition)；First shared convolutional neural networks are by Chinese Character Recognition neural network and non-Chinese Character Recognition nerve net The convolutional neural networks (effect of the convolution log on is to extract first eigenvector) that network shares, Chinese Character Recognition links entirely Layer and the full linking layer of non-Chinese Character Recognition are different full linking layer.

In turn, in method provided herein, step S102 be may include steps of:

Step 1021, license plate picture to be identified is inputted to the first shared convolutional neural networks of Chinese Character Recognition neural network, To extract the first eigenvector of license plate picture to be identified；

Step 1022, by the full linking layer of Chinese Character Recognition of first eigenvector input Chinese Character Recognition neural network, to obtain Chinese Character Recognition result；

Step 1023, first eigenvector is inputted to the full linking layer of non-Chinese Character Recognition of non-Chinese Character Recognition neural network, with Obtain non-Chinese Character Recognition result.

Wherein, there is no absolute execution sequencing, the two steps can be held simultaneously for step 1022 and step 1023 Row, can also successively execute.The trigger condition of the two steps is to obtain first eigenvector.

In the above method, first eigenvector is whole features for describing license plate picture to be identified, this part is special Sign is directly respectively supplied to the full linking layer of Chinese Character Recognition and the full linking layer of non-Chinese Character Recognition, so that the two full linking layers foundations the One feature vector generates corresponding result.Wherein, the possible output result of the full linking layer of Chinese Character Recognition is chinese character, non- The possible output result of the full linking layer of Chinese Character Recognition is non-chinese character.

After being trained, the full linking layer of Chinese Character Recognition can pay close attention to portion related with Chinese character in first eigenvector automatically Point, the non-full linking layer of Chinese Character Recognition can pay close attention to part related with non-Chinese character in first eigenvector automatically, to be respectively completed Chinese character and non-Chinese Character Recognition.

Chinese Character Recognition neural network and non-Chinese Character Recognition neural network share the part for extracting feature vector, simplify system Framework improves whole computational efficiency.

Preferably, the maximum value of the convolution kernel of the convolutional layer in the first shared convolutional neural networks is (5,5).Ordinary circumstance Under, having at least two sequentially connected convolutional layers in the first shared convolutional neural networks, (first convolutional layer is used to export centre As a result feature vector, second convolutional layer are used to export the first eigenvector of final result), the convolution of the two convolutional layers Core all can be (5*5)；The convolution kernel of either at least one layer of convolutional layer is (5*5)；The either convolution of first convolutional layer Core is (5*5), in general, the convolution kernel of forward convolutional layer is greater than or equal to volume rearward in the first shared convolutional neural networks The convolution kernel of lamination.Present inventor tested several different convolution kernels, for example, using the convolution kernel of (7*7), will lead to Model parameter is more, and complexity is big, and convergence is slower when training；Using the convolution kernel of (3*3), then susceptibility is poor, and it is comprehensive to measure, And after being tested, it is believed that use the convolution kernel of (5*5) more appropriate.

Preferably, the shared 6-12 convolutional layer in the first shared convolutional neural networks, and the convolution of forward convolutional layer Core is greater than the convolution kernel of convolutional layer rearward.It is furthermore preferred that sharing 10 convolutional layers in the first shared convolutional neural networks, (this 10 A convolutional layer is sequentially connected).When designing convolutional neural networks, usually there are two standards to constrain research and development people jointly Member, the two standards are accuracy in computation and computational efficiency respectively, are in the case of certain, convolutional layer is more, obtained result meeting It is more more accurate, but after the quantity of convolutional layer reaches some numerical value, being further added by convolutional layer just only can reduce computational efficiency, without Accuracy can be improved.Therefore, the number of plies of convolutional layer is not easy excessive or very few, by the test of present inventor, by Convolutional layer in one shared convolutional neural networks be set as 10 layers it is proper.

More specifically, for example, being provided with 10 volumes altogether according to sequence from front to back in the first shared convolutional neural networks Lamination, is the first convolutional layer, the tenth convolutional layer of the second convolutional layer ... respectively, and the convolution kernel of first the-the five convolutional layer of convolutional layer is (5,5), the convolution kernel of the 6th to the tenth convolutional layer are (3,3).

It in other words, include the first convolution group and the second convolution group being sequentially arranged in the first shared convolutional neural networks, the One convolution group and the second convolution group are made of at least two convolutional layers being sequentially arranged, the convolutional layer in the first convolution group Convolution kernel size it is all the same, the size of the convolution kernel of each convolutional layer in the second convolution group is all the same, and, the first convolution The convolution kernel of convolutional layer is bigger than the convolution kernel of the convolutional layer in the second convolution group in group.More specifically, convolution in the first convolution group The convolution kernel of layer is (5,5), and the convolution kernel of convolutional layer is (3,3) in the second convolution group.

On the basis of having used the first shared convolutional neural networks to carry out the extraction of first eigenvector, the present application People thinks, if simple increase convolutional layer, will lead to that feature extraction efficiency is too low, and feature extraction accuracy will soon reach Saturation state, especially when the clarity of license plate picture to be identified is not high enough.

Be directed to this kind of situation the present inventors considered that, can in the first shared convolutional neural networks simultaneously using volume Lamination and residual error layer carry out feature extraction, to improve feature extraction efficiency, and accelerate the convergence rate of model training.

It is, as shown in figure 4, in scheme provided herein, the first shared convolutional neural networks preferably at least by Spaced convolutional layer and residual error layer are constituted；In turn, adjacent convolutional layer and residual error layer are just capable of forming a feature extraction Group of networks, entirety can be improved to the progress feature extraction of license plate picture to be identified to a certain extent by extracting group of networks by this feature Efficiency.

In turn, in method provided herein, step 1021 includes:

Wherein, the quantity of convolutional layer and residual error layer does not require in feature extraction group of networks, but in general residual error layer The minimum unit used is 2 layers, and the minimum unit that convolutional layer uses is 1 layer.For example, in feature extraction group of networks, it can sequence It is provided with level 1 volume lamination, 2 layers of residual error layer and level 1 volume lamination；Level 2 volume lamination, 2 layers of residual error layer and 1 layer can also be sequentially set with Convolutional layer；Level 1 volume lamination, 2 layers of residual error layer, level 1 volume lamination and 2 layers of residual error layer can also be sequentially set with.

In the following, with feature extraction group of networks by 1 layer of first convolutional layer, 2 layer of first residual error layer and 1 layer of second convolutional layer institute group At come the course of work that illustrates feature extraction group of networks:

Step 1, the first median feature vector is input to the first convolutional layer, obtains the second median feature vector；

Step 2, the second median feature vector is input to the first residual error layer, obtains third median feature vector；

Step 3, third median feature vector is input to the second convolutional layer, obtains the 4th median feature vector；Wherein, One median feature vector is exported by previous feature extraction network, and the 4th median feature vector is for next spy Sign extracts network inputs, if it is first feature extraction group of networks that this feature, which extracts group of networks, the first intermediate features to Amount should be changed to license plate picture to be identified；If it is the last one feature extraction group of networks that this feature, which extracts group of networks, the 4th Median feature vector should be changed to first eigenvector.

Such as explanation hereinbefore, in scheme provided herein, due to used simultaneously Chinese Character Recognition neural network and Non- Chinese Character Recognition neural network identifies license plate, improves the precision of identification, also, in Chinese Character Recognition neural network and In the case that non-Chinese Character Recognition neural network shares the first shared convolutional neural networks progress feature extraction, it can reduce whole It calculates cost and improves computational efficiency.

But present inventor is during further test and analysis, has been found that Chinese Character Recognition neural network and non- Chinese Character Recognition neural network shares completely the part of feature extraction, (convolutional Neural net during carrying out feature vector and determining The main function of network is feature vector corresponding to determining license plate picture to be identified), since feature extraction is without specific aim, meeting Cause the extraction accuracy of feature vector not ideal enough.Main cause is Chinese and number, English feature extraction mode are not It is identical, relatively fixed and phase when the position that every kind of character (Chinese character and numerical character, English character) occupies in license plate Poor little (these three characters are all to be arranged side by side on license plate, and the area of three kinds of characters is essentially identical), but Chinese is often Normal stroke is more, and details is more, and in training process, algorithm usually needs to retain more details letter when extracting Chinese feature Breath, could distinguish different Chinese, therefore the characteristic information that Chinese extracts biases toward part.But English and digital pen To draw relatively easy, the characteristic information for needing to extract is partial to Global Information, for example, number 3 and number 8, letter b, if extracted Then 8 and B may also be considered as 3 to local message.

Because of this reason, causes algorithm when extracting Chinese character, English character and numerical character, have occurred " competition ", it is, sharing completely the same feature extraction net in Chinese Character Recognition neural network and non-Chinese Character Recognition neural network In the case where network (the first shared convolutional neural networks), when extracting feature vector, need to look after Chinese character, English character is looked after, in such cases, the feature vector (giving the feature vector that full linking layer uses) finally extracted is It is dissatisfactory, it is unfavorable for subsequent two full linking layers and is respectively processed, in turn, and in order to solve this problem, the application institute In the method for offer, Chinese Character Recognition neural network and non-Chinese Character Recognition neural network are improved.

It is, as shown in figure 3, Chinese Character Recognition neural network include the sequentially connected second shared convolutional neural networks, First Chinese Character Recognition convolutional neural networks and the full linking layer of Chinese Character Recognition are (that is, the output result of the second shared convolutional neural networks It is capable of providing to the first Chinese Character Recognition convolutional neural networks, the output result of the first Chinese Character Recognition convolutional neural networks is capable of providing Give Chinese Character Recognition full linking layer)；Non- Chinese Character Recognition neural network includes the sequentially connected second shared convolutional neural networks, first Non- Chinese Character Recognition convolutional neural networks and the full linking layer of non-Chinese Character Recognition are (that is, the second shared convolutional neural networks are also capable of providing To the first non-Chinese Character Recognition convolutional neural networks, the output result of the first non-Chinese Character Recognition convolutional neural networks is capable of providing to non- The full linking layer of Chinese Character Recognition)；First Chinese Character Recognition convolutional neural networks and the first non-Chinese Character Recognition convolutional neural networks are different Convolutional neural networks；The full linking layer of non-Chinese Character Recognition and the full linking layer of Chinese Character Recognition are different full linking layer.

Under such mode, Chinese Character Recognition neural network and non-Chinese Character Recognition neural network have still shared a part of convolution mind It is used to extract essential characteristic through network (the second shared convolutional neural networks), but other than shared convolutional neural networks, the Chinese Word identification neural network and non-Chinese Character Recognition neural network are also respectively provided with independent convolutional neural networks (the first Chinese Character Recognition Convolutional neural networks and the first non-Chinese Character Recognition convolutional neural networks) extract more targeted feature.Specifically, second Shared convolutional neural networks are the common trait for extracting chinese character and non-chinese character, the first Chinese Character Recognition convolution mind It is for carrying out feature extraction for chinese character through network, the first non-Chinese Character Recognition convolutional neural networks are for for non- Chinese character (English character, numerical character) carries out feature extraction.

In turn, step S102 can be executed as follows:

Step 1024, license plate picture to be identified is inputted into the second shared convolutional neural networks, to extract license plate figure to be identified The second feature vector of piece；

Step 1025, by the first Chinese Character Recognition convolutional Neural net of second feature vector input Chinese Character Recognition neural network Network, to extract the third feature vector of license plate picture to be identified；

Step 1026, by the full linking layer of Chinese Character Recognition of third feature vector input Chinese Character Recognition neural network, to obtain Chinese Character Recognition result；

Step 1027, second feature vector is inputted to the first non-Chinese Character Recognition convolutional Neural of non-Chinese Character Recognition neural network Network, to extract the fourth feature vector of license plate picture to be identified；

Step 1028, fourth feature vector is inputted to the full linking layer of non-Chinese Character Recognition of non-Chinese Character Recognition neural network, with Obtain non-Chinese Character Recognition result.

It is, the common trait of chinese character and non-chinese character is first carried out using the first shared convolutional neural networks Extract, obtain it is common as a result, and then common result that will extract be separately input to the first Chinese Character Recognition convolutional Neural Network and the first non-Chinese Character Recognition convolutional neural networks are suitble to really with obtaining the third feature vector sum for being adapted to determine Chinese character The fourth feature vector of fixed non-Chinese character is known finally, third feature vector sum fourth feature vector is separately input to Chinese character respectively In infull linking layer and the full linking layer of non-Chinese Character Recognition, to obtain Chinese Character Recognition result and non-Chinese Character Recognition result.

Further, by the experiment of inventor and summary, it is believed that, what the first Chinese Character Recognition convolutional neural networks were exported The dimension of feature vector can the dimension of feature vector that is exported of Chinese Character Recognition convolutional neural networks more non-than first it is low, guaranteeing In the case where accuracy, the number of dimensions for the feature vector that the first Chinese Character Recognition convolutional neural networks are exported and the first non-Chinese character The ratio of the number of dimensions for the feature vector that identification convolutional neural networks are exported can be less than 1/2, and greater than 1/4.More Specifically, the dimension for the feature vector that the first Chinese Character Recognition convolutional neural networks are exported can be 2048, the first non-Chinese character is known The dimension for the feature vector that other convolutional neural networks are exported can be 6144.

Preferably, the quantity of convolutional layer is refreshing less than the first non-Chinese Character Recognition convolution in the first Chinese Character Recognition convolutional neural networks Quantity through convolutional layer in network.For example, the first Chinese Character Recognition convolutional neural networks are made of 3 layers of convolutional layer, the first non-Chinese character Identification convolutional neural networks are made of 5 layers of convolutional layer.

It is above-mentioned to be adopted as Chinese Character Recognition neural network and non-Chinese Character Recognition neural network and be arranged to share convolutional neural networks (the Two shared convolutional neural networks), and be respectively set independent convolutional neural networks (the first non-Chinese Character Recognition convolutional neural networks and First Chinese Character Recognition convolutional neural networks) scheme can largely improve whole recognition accuracy.But with using The increase of number, present inventor's discovery, due to camera, or the reason of shooting license plate environment, certain journey The clarity that will lead to license plate on degree is not high enough.For this kind of situation, present inventor is through overtesting, it is believed that can pass through Increase the mode of residual error layer to solve the problems, such as this.

From the point of view of general, residual error layer can be added to the second shared convolutional neural networks, the first Chinese Character Recognition convolutional Neural net In at least one neural network in network and the first non-Chinese Character Recognition convolutional neural networks.

I.e., it is possible to be that the second shared convolutional neural networks and the first Chinese Character Recognition convolutional neural networks are at least set by interval The convolutional layer and residual error layer set are formed；It can also be the first Chinese Character Recognition convolutional neural networks and the first non-Chinese Character Recognition volume Product neural network is at least made of spaced convolutional layer and residual error layer；It is also possible to the second shared convolution nerve net Network, the first Chinese Character Recognition convolutional neural networks and the first non-Chinese Character Recognition convolutional neural networks are at least by spaced convolution Layer and residual error layer are formed.

As shown in figure 5, it is non-to show the second shared convolutional neural networks, the first Chinese Character Recognition convolutional neural networks and first The Chinese Character Recognition convolutional neural networks at least example as composed by spaced convolutional layer and residual error layer.

In such cases, Chinese Character Recognition neural network includes sequentially connected 5th convolutional neural networks, the knowledge of the first Chinese character Other convolutional neural networks and the full linking layer of Chinese Character Recognition；Non- Chinese Character Recognition neural network includes the sequentially connected second shared convolution Neural network, the first non-Chinese Character Recognition convolutional neural networks and the full linking layer of non-Chinese Character Recognition；First Chinese Character Recognition convolutional Neural Network and the first non-Chinese Character Recognition convolutional neural networks are different convolutional neural networks；The full linking layer of non-Chinese Character Recognition and Chinese character Identify that full linking layer is different full linking layer；Second shared convolutional neural networks, the first Chinese Character Recognition convolutional neural networks and First non-Chinese Character Recognition convolutional neural networks are at least made of spaced convolutional layer and residual error layer.

After increasing residual error layer in convolutional neural networks, the order of accuarcy identified to license plate picture to be identified is obtained Promotion is arrived, also, there has also been very big promotions for the speed of the trained model.There are identical mapping mainly inside residual error layer, A part of primitive character can be retained when extracting feature, this can not lose spy for fuzzy image to a certain extent Sign.

When the second shared convolutional neural networks are by successively tactic first convolutional layer, the first residual error layer, the second convolution When layer, the second residual error layer and formed third convolutional layer, the process for extracting feature be may include steps of:

Step 1, license plate picture to be identified is inputted to the first convolutional layer, makes the first intermediate features of output of the first convolutional layer Vector；

Step 2, the first median feature vector is inputted to the first residual error layer, the first residual error layer is made to export the second intermediate features Vector；

Step 3, the second median feature vector is inputted to the second convolutional layer, the second convolutional layer is made to export third intermediate features Vector；

Step 4, third median feature vector is inputted to the second residual error layer, the second residual error layer is made to export the 4th intermediate features Vector；

Step 5, the 4th median feature vector is inputted to third convolutional layer, the second convolutional layer is made to export the 5th intermediate features Vector；

5th median feature vector then should be respectively to the first Chinese Character Recognition convolutional neural networks and the first non-Chinese Character Recognition Convolutional neural networks input, to carry out subsequent feature extraction.

When actual use, being not generally possible to whole region in the photo that shoots by camera is all license plate Part, in the photo alternatively taken by camera, the region where license plate is one for occupying whole photos Point, in turn, in order to improve the accuracy of identification, preferably first the photo that camera is shot is pre-processed.

As shown in fig. 7, step S101 includes the following steps:

S1011 intercepts multiple candidate images from the environment photo taken according to different interception ways；

S1012 calculates separately the reference index of each candidate image using feature extraction network；

S1013 selects reference index to meet the candidate image of preset requirement as license plate picture to be identified.

Step S1011 mainly extracts corresponding image by the way of the extraction of anchor point frame from environment photo.Specifically For, in step S1011, environment photo is usually the original photo shot by video camera, in the original photo, license plate Shared region is usually smaller, if this environment photo is directly put into Chinese Character Recognition neural network and non-Chinese character It is identified in identification neural network, may result in the accuracy decline of identification.And then firstly, it is necessary to using anchor point frame The mode that (anchor box) is extracted, intercepts out candidate image from environment photo, and the candidate image of interception should be multiple, and Multiple candidate images should be distributed across on the different location in environment photo, to accomplish uniformly to extract as far as possible, avoid omission (can be overlapped between different candidate images).Meanwhile the size of candidate image should also be as suffering restraints, it cannot be excessive, also not Can be too small, candidate image is excessive, then will lead to the region in candidate image where license plate and remain unchanged very little, candidate image is too small, then Will lead to candidate image can not intercept out entire license plate, this can bring error to subsequent identification process.Under normal circumstances, it waits The size of image is selected to can be (128*128).

Specifically, one anchor point can be placed every 32 length for the picture of a 1600*640 size, Respectively on the basis of each anchor point, 3 various sizes of anchor box are placed with (as shown in figure 8, anchor point is Dot in Fig. 8, anchor box, which is three of upper left in figure, the box of overlapping), then the picture may finally give birth to (2400 candidate figures can be truncated at the candidate anchor point frame of (1280/32) * (640/32) * 3=40*20*3=2400 Picture).

In step S1012, full linking layer (fully Connect) can be used, one is done to each candidate image of generation The prediction of a 4+2 dimension, that is, determine corresponding reference index for each candidate image, reference index have dx, dy, dh, Dw, score1 and socre2；Wherein, (dx, dy) indicates that centre deviation, (dh, dw) indicate high wide deviation, and score1 characterizes the frame Non-targeted probability, score2 characterize the probability that the frame is target.Under normal circumstances, in score2 > score1, then it represents that right It include license plate in the candidate image answered, in turn, so that it may using the candidate image as license plate picture to be identified, and pass through Chinese character Identification neural network and non-Chinese Character Recognition neural network are further identified.It is, in step S1013, Ke Yixuan The candidate anchor point frame of score2 > score1 is selected as license plate picture to be identified.

In scheme provided herein, two networks have been used altogether, are to extract license plate from environment photo to wait knowing respectively Feature extraction network and Text region network (including Chinese Character Recognition neural network and the non-Chinese Character Recognition nerve net of other picture Network).In the following, briefly being introduced the training process of this feature extraction network and Text region network.

The training process of feature extraction network includes the following steps:

Step 11, according to different interception ways, multiple candidate images are intercepted from the environment photo taken；

Step 12, using feature extraction network, the reference index of each candidate image is calculated separately；

Step 13, reference index is selected to meet the candidate image of preset requirement as feature extraction picture；

Step 14, comparative feature extracts position and the location error of the picture manually marked of picture, obtains error amount；

Step 15, feature extraction network will be adjusted according to error amount.

Wherein, step 11 and step 12 are similar to step S1011 and step S1012 respectively, be all interception candidate image and Calculate the reference index of each candidate image.In step 13, select the satisfactory candidate image of reference index as there is feature The picture (feature extraction picture) that network is extracted is extracted, and then calculates the position of feature extraction picture and manually marks Picture position between difference, obtain error amount, finally according to error amount adjust feature extraction network.Then, it is also necessary to Repeat the above steps 11-15, until error amount is sufficiently small or error amount is 0, so that it may export this feature and extract network.

The training process of feature extraction network can be divided into the process and the non-Chinese character of training of trained Chinese Character Recognition neural network Identify the process of neural network.

Specifically, the process of training Chinese Character Recognition neural network includes the following steps:

Step 21, the license plate picture sample by artificial mark Chinese character information is obtained；

Step 22, it will be input to Chinese Character Recognition neural network without the license plate picture sample manually marked, obtains Chinese character Neural network recognization result；

Step 23, compare the difference by the license plate picture sample and Chinese character neural network recognization result that manually mark, obtain To Chinese Character Recognition error；

Step 24, according to the parameter of Chinese Character Recognition error transfer factor Chinese Character Recognition neural network.

Wherein, the license plate picture sample by the Chinese character information manually marked, which refers to manually having marked out, carrys out license plate figure Actual chinese character (such as " capital ", " Shanghai ") in piece, which is exactly standard results.Step 22 is using not The Chinese Character Recognition neural network that training is completed identifies license plate picture sample, to obtain Chinese character neural network recognization result (for example Chinese character neural network recognization result may be " illiteracy ") then can be by comparing standard results and Chinese character in step 23 The error of neural network recognization result adjusts the parameter of Chinese Character Recognition neural network, so that Chinese Character Recognition adjusted is neural The parameter of network can more accurately identify chinese character.

Corresponding, the process of the non-Chinese Character Recognition neural network of training includes the following steps:

Step 31, the license plate picture sample by manually marking non-Chinese character information is obtained；

Step 32, it will be input to Chinese Character Recognition neural network without the license plate picture sample manually marked, obtains the non-Chinese Word neural network recognization result；

Step 33, compare the difference by the license plate picture sample and non-Chinese character neural network recognization result that manually mark, Obtain non-Chinese Character Recognition error；

Step 34, according to the parameter of the non-Chinese Character Recognition neural network of non-Chinese Character Recognition error transfer factor.

Wherein, the license plate picture sample by the non-Chinese character information manually marked, which refers to manually having marked out, carrys out license plate Actual non-chinese character (such as " F ", " 2 ") in picture, which is exactly standard results.Step 22 is to make License plate picture sample is identified with the non-Chinese Character Recognition neural network that training is not completed, is known with obtaining non-Chinese character neural network Other result (for example non-Chinese character neural network recognization result may be " Z ") then can be by comparing standard results in step 23 The parameter that non-Chinese Character Recognition neural network is adjusted with the error of non-Chinese character neural network recognization result, so that adjusted non- The parameter of Chinese Character Recognition neural network can more accurately identify non-chinese character.

It corresponds to the above method, present invention also provides a kind of license plate recognition devices, comprising:

First obtains module, for obtaining license plate picture to be identified；

Preferably, identification module includes:

Preferably, the first input unit includes:

Preferably, identification module includes:

Preferably, the number of dimensions and the first non-Chinese character for the feature vector that the first Chinese Character Recognition convolutional neural networks are exported The ratio of the number of dimensions for the feature vector that identification convolutional neural networks are exported is greater than 1/4 less than 1/2；

Preferably, the second shared convolutional neural networks, the first Chinese Character Recognition convolutional neural networks and the first non-Chinese Character Recognition At least one of convolutional neural networks are at least made of spaced convolutional layer and residual error layer.

Preferably, the first acquisition module includes:

Preferably, further includes:

It corresponds to the above method, present invention also provides a kind of non-volatile program generations that can be performed with processor The computer-readable medium of code, said program code make the processor execute licence plate recognition method.

As shown in figure 9, for server schematic diagram provided by the embodiment of the present application, the server 90 include: processor 91, Memory 92 and bus 93, memory 92, which is stored with, to be executed instruction, when server operation, between processor 91 and memory 92 Communicated by bus 93, processor 91 execute memory 92 in store licence plate recognition method the step of.

In view of method provided herein is more suitable for the technology scene identified for fuzzy license plate, therefore, Meanwhile the usual clarity of photo obtained captured by automobile data recorder be all it is relatively low, therefore, method provided herein It is particularly suitable for using in automobile data recorder, that is, the driving that is revised as of being adapted to property of method provided herein is remembered Record licence plate recognition method used in instrument.

It is corresponding, it is automobile data recorder schematic diagram provided by the embodiment of the present application, the automobile data recorder as shown in Figure 10 1000 include: camera 1004, processor 1001, memory 1002 and bus 1003, and memory 1002, which is stored with, to be executed instruction, When automobile data recorder operation, communicated between processor 1001 and memory 1002 by bus 1003, camera 1004 is used for Photograph taking is carried out in vehicular motion；Processor 1001 executes the step of the licence plate recognition method stored in memory 1002 Suddenly.

It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product It is stored in a computer readable storage medium.Based on this understanding, the technical solution of the application is substantially in other words The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a People's computer, server or network equipment etc.) execute each embodiment the method for the application all or part of the steps. And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic or disk.

The above, the only specific embodiment of the application, but the protection scope of the application is not limited thereto, it is any Those familiar with the art within the technical scope of the present application, can easily think of the change or the replacement, and should all contain Lid is within the scope of protection of this application.Therefore, the protection scope of the application shall be subject to the protection scope of the claim.

Claims

1. a kind of licence plate recognition method characterized by comprising

Obtain license plate picture to be identified；

License plate picture to be identified is identified using Chinese Character Recognition neural network and non-Chinese Character Recognition neural network respectively, Obtain the Chinese Character Recognition result and non-Chinese Character Recognition result of the license plate；

According to the Chinese Character Recognition result and non-Chinese Character Recognition as a result, generating license plate number information,

Wherein step uses Chinese Character Recognition neural network and non-Chinese Character Recognition neural network to license plate picture to be identified respectively It is identified, the Chinese Character Recognition result and non-Chinese Character Recognition result for obtaining the license plate include:

License plate picture to be identified is inputted to the first shared convolutional neural networks of the Chinese Character Recognition neural network, to extract The first eigenvector of license plate picture to be identified；

The first eigenvector is inputted to the full linking layer of Chinese Character Recognition of the Chinese Character Recognition neural network, to obtain the Chinese Word recognition result；

The first eigenvector is inputted to the full linking layer of non-Chinese Character Recognition of the non-Chinese Character Recognition neural network, to obtain State non-Chinese Character Recognition as a result,

Wherein the full linking layer of the Chinese Character Recognition can pay close attention to automatically in the first eigenvector with license plate figure to be identified The related part of chinese character in piece, and the full linking layer of non-Chinese Character Recognition can pay close attention to automatically the fisrt feature to Part related with the non-chinese character in license plate picture to be identified in amount, the non-chinese character include English character and At least one of in numerical character.

2. the method according to claim 1, wherein license plate picture to be identified is inputted the Chinese character by step Identify neural network the first shared convolutional neural networks, include: to extract the first eigenvector of license plate picture to be identified

License plate picture sequence to be identified is inputted at least one feature extraction network in the described first shared convolutional neural networks Group, the feature for making the feature vector of previous feature extraction group of networks output as next feature extraction group of networks and inputting to Amount, until the last one feature extraction group of networks exports first eigenvector；Feature extraction group of networks includes at least one layer of convolution Layer and at least two layers of residual error layer.

3. the method according to claim 1, wherein step uses Chinese Character Recognition neural network and non-Chinese character respectively Identification neural network identifies that the Chinese Character Recognition result and non-Chinese character for obtaining the license plate are known to license plate picture to be identified Other result includes:

License plate picture to be identified is inputted into the second shared convolutional neural networks, with extract the second feature of license plate picture to be identified to Amount；

The second feature vector is inputted to the first Chinese Character Recognition convolutional neural networks of the Chinese Character Recognition neural network, to mention Take the third feature vector of license plate picture to be identified；

The third feature vector is inputted to the full linking layer of Chinese Character Recognition of the Chinese Character Recognition neural network, to obtain Chinese character knowledge Other result；

Second feature vector is inputted to the first non-Chinese Character Recognition convolutional neural networks of the non-Chinese Character Recognition neural network, to mention The fourth feature vector of pick-up board picture to be identified；

The fourth feature vector is inputted to the full linking layer of non-Chinese Character Recognition of the non-Chinese Character Recognition neural network, it is non-to obtain Chinese Character Recognition result.

4. according to the method described in claim 3, it is characterized in that,

The number of dimensions for the feature vector that first Chinese Character Recognition convolutional neural networks are exported and the first non-Chinese Character Recognition convolution mind The ratio of the number of dimensions of the feature vector exported through network is greater than 1/4 less than 1/2；

The quantity of convolutional layer in the first non-Chinese Character Recognition convolutional neural networks less than rolling up in first Chinese Character Recognition convolutional neural networks The quantity of lamination.

5. according to the method described in claim 3, it is characterized in that, the second shared convolutional neural networks, the first Chinese character are known At least one of other convolutional neural networks and the first non-Chinese Character Recognition convolutional neural networks are at least by spaced convolutional layer It is formed with residual error layer.

6. the method according to claim 1, wherein obtaining license plate picture to be identified, comprising:

7. the method according to claim 1, wherein further include:

It will be input to Chinese Character Recognition neural network without the license plate picture sample manually marked, obtains Chinese character neural network recognization As a result；

Compare the difference by the license plate picture sample and the Chinese character neural network recognization result that manually mark, obtains the Chinese Word identification error；

According to the parameter of Chinese Character Recognition neural network described in the Chinese Character Recognition error transfer factor.

8. the method according to claim 1, wherein further include:

It will be input to Chinese Character Recognition neural network without the license plate picture sample manually marked, obtains non-Chinese character neural network and know Other result；

Compare the difference by the license plate picture sample and the non-Chinese character neural network recognization result that manually mark, obtains Non- Chinese Character Recognition error；

The parameter of non-Chinese Character Recognition neural network according to non-Chinese Character Recognition error transfer factor.

9. a kind of license plate recognition device characterized by comprising

First obtains module, for obtaining license plate picture to be identified；

Identification module, for be identified to the license plate using Chinese Character Recognition neural network and non-Chinese Character Recognition neural network respectively Picture is identified, the Chinese Character Recognition result and non-Chinese Character Recognition result of the license plate are obtained；

Generation module, for according to the Chinese Character Recognition result and non-Chinese Character Recognition as a result, generate license plate number information,

Wherein the identification module includes:

First input unit, for license plate picture to be identified to be inputted to the first shared volume of the Chinese Character Recognition neural network Product neural network, to extract the first eigenvector of license plate picture to be identified；

Second input unit, for the first eigenvector to be inputted to the full chain of Chinese Character Recognition of the Chinese Character Recognition neural network Layer is connect, to obtain the Chinese Character Recognition result；

Third input unit, for the first eigenvector to be inputted to the non-Chinese Character Recognition of the non-Chinese Character Recognition neural network Full linking layer, to obtain the non-Chinese Character Recognition as a result,

10. device according to claim 9, which is characterized in that the first input unit includes:

First input subelement, for inputting in the described first shared convolutional neural networks extremely license plate picture sequence to be identified A few feature extraction group of networks makes the feature vector of previous feature extraction group of networks output as next feature extraction net The feature vector of network group input, until the last one feature extraction group of networks exports first eigenvector；Feature extraction group of networks Including at least one layer of convolutional layer and at least two layers of residual error layer.

11. device according to claim 9, which is characterized in that identification module includes:

4th input unit waits knowing for license plate picture to be identified to be inputted the second shared convolutional neural networks to extract license plate The second feature vector of other picture；

5th input unit, for the second feature vector to be inputted to the first Chinese Character Recognition of the Chinese Character Recognition neural network Convolutional neural networks, to extract the third feature vector of license plate picture to be identified；

6th input unit, for the third feature vector to be inputted to the full chain of Chinese Character Recognition of the Chinese Character Recognition neural network Layer is connect, to obtain Chinese Character Recognition result；

7th input unit, for second feature vector to be inputted to the first non-Chinese Character Recognition of the non-Chinese Character Recognition neural network Convolutional neural networks, to extract the fourth feature vector of license plate picture to be identified；

8th input unit, for the fourth feature vector to be inputted to the non-Chinese Character Recognition of the non-Chinese Character Recognition neural network Full linking layer, to obtain non-Chinese Character Recognition result.

12. device according to claim 11, which is characterized in that

13. device according to claim 11, which is characterized in that the second shared convolutional neural networks, the first Chinese character Identify at least one of convolutional neural networks and the first non-Chinese Character Recognition convolutional neural networks at least by spaced convolution Layer and residual error layer are formed.

14. device according to claim 9, which is characterized in that first, which obtains module, includes:

15. device according to claim 9, which is characterized in that further include:

First input module is obtained for will be input to Chinese Character Recognition neural network without the license plate picture sample manually marked To Chinese character neural network recognization result；

First comparison module, for the license plate picture sample by manually marking and the Chinese character neural network recognization As a result difference obtains Chinese Character Recognition error；

The first adjustment module, the parameter for the Chinese Character Recognition neural network according to the Chinese Character Recognition error transfer factor.

16. device according to claim 9, which is characterized in that further include:

Second input module is obtained for will be input to Chinese Character Recognition neural network without the license plate picture sample manually marked To non-Chinese character neural network recognization result；

Second comparison module is known for the license plate picture sample by manually marking and the non-Chinese character neural network The difference of other result obtains non-Chinese Character Recognition error；

Second adjustment module, the parameter for the non-Chinese Character Recognition neural network according to non-Chinese Character Recognition error transfer factor.

17. a kind of computer-readable medium for the non-volatile program code that can be performed with processor, which is characterized in that described Program code makes the processor execute -8 any the method according to claim 1.

18. a kind of server includes: processor, memory and bus, memory, which is stored with, to be executed instruction, when server is run When, by bus communication between processor and memory, processor execute stored in memory it is -8 any according to claim 1 The method.

19. a kind of automobile data recorder, which includes: camera, processor, memory and bus, memory storage It executes instruction, when calculating equipment operation, camera and processor pass through bus communication, camera use between memory In carrying out photograph taking in vehicular motion；Processor executes -8 any institute according to claim 1 stored in memory State method.