CN110163199A

CN110163199A - Licence plate recognition method, license plate recognition device, car license recognition equipment and medium

Info

Publication number: CN110163199A
Application number: CN201811159787.0A
Authority: CN
Inventors: 戴雨辰; 董鑫; 郭晓威; 余宗桥; 张睿欣; 姚雨石
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2018-09-30
Filing date: 2018-09-30
Publication date: 2019-08-23

Abstract

Disclose a kind of licence plate recognition method, license plate recognition device, car license recognition equipment and medium.The licence plate recognition method includes: to pre-process to vehicle image, obtains the license plate type of license plate image to be identified and the license plate image to be identified；In the case where the license plate type of the license plate image to be identified indicates multirow license plate, type conversion is carried out for the license plate image to be identified of the multirow license plate type, obtains the license plate image of uniline license plate type；The license plate image of the uniline license plate type is identified, license plate number is obtained.By the way that multirow license plate is converted to uniline license plate, the identification for multirow license plate can be simply and efficiently realized.

Description

Licence plate recognition method, license plate recognition device, car license recognition equipment and medium

Technical field

This disclosure relates to intelligent transportation field, relate more specifically to a kind of licence plate recognition method, license plate recognition device, license plate Identify equipment and medium.

Background technique

As deep learning continues deeply civilian and commercial kitchen area, vehicle identification is as the pass in intelligent transportation system Key technology is carrying out vehicle detection, especially in the detection identification of a variety of license plates, is also faced with higher requirement.It is based on The various type of license plate type and the greatest differences of uniline license plate and multirow license plate, the prior art when identifying multirow license plate, according to Rely multiline text detection technique, detection is oriented every a line of multirow license plate, then known line by line using uniline identification model Result is not obtained.

However, under such identification method, need to mark out every a line of multirow vehicle, mark amount is big and labor intensive Resource；Also, it need to call multiple identification model that could complete the identification to multirow license plate, time-consuming；In addition, if in multirow license plate Certain a line be missed, then will lead to recognition failures, the robustness of recognizer is lower.

It is called therefore, it is necessary to a kind of repetition that can reduce mark workload, reduce identification model and there is higher robust The licence plate recognition method of property.

Summary of the invention

In view of the above problems, present disclose provides a kind of licence plate recognition method, device, equipment and media.Utilize the disclosure The licence plate recognition method of offer can efficiently identify a variety of license plate types, and consuming is largely indexed when solving duplicate rows Car license recognition Human resources and repeatedly call identification model cause handle time longer problem, improve detection accuracy and efficiency, and the party Method has good robustness.

According to the one side of the disclosure, a kind of licence plate recognition method is provided, comprising: vehicle image is pre-processed, Obtain the license plate type of license plate image to be identified and the license plate image to be identified；In the vehicle of the license plate image to be identified In the case that board type indicates multirow license plate, type conversion is carried out for the license plate image to be identified of the multirow license plate type, Obtain the license plate image of uniline license plate type；The license plate image of the uniline license plate type is identified, license plate number is obtained.

According to another aspect of the present disclosure, a kind of license plate recognition device is additionally provided, comprising: preprocessing module is configured To pre-process to vehicle image, the license plate type of license plate image to be identified and the license plate image to be identified is obtained；Class Type conversion module is configured as in the case where the license plate type of the license plate image to be identified indicates multirow license plate, for institute The license plate image to be identified for stating multirow license plate type carries out type conversion, obtains the license plate image of uniline license plate type；And vehicle Board identification module is configured as identifying the license plate image of the uniline license plate type, obtains license plate number.

According to another aspect of the present disclosure, a kind of car license recognition equipment is additionally provided, wherein the equipment includes processor And memory, the memory include one group of instruction, one group of instruction knows the license plate when being executed by the processor Other equipment executes operation, and the operation includes: to pre-process to vehicle image, obtain obtaining license plate image to be identified and The license plate type of the license plate image to be identified；The case where the license plate type of the license plate image to be identified indicates multirow license plate Under, type conversion is carried out for the license plate image to be identified of the multirow license plate type, obtains the license plate figure of uniline license plate type Picture；And the license plate image of the uniline license plate type is identified, obtain license plate number.

According to another aspect of the present disclosure, a kind of computer readable storage medium is additionally provided, which is characterized in that deposit thereon Computer-readable instruction is contained, executes method as described above when executing described instruction using computer.

Using the method for the Car license recognition that the disclosure provides, can complete well based on vehicle image, to different type The license plate process that is detected, pre-processed, identified particularly can simply and efficiently realize the identification for duplicate rows license plate Journey.

Detailed description of the invention

It, below will be to required use in embodiment description in order to illustrate more clearly of the technical solution of the embodiment of the present disclosure Attached drawing be briefly described, it should be apparent that, the accompanying drawings in the following description is only some embodiments of the present disclosure, for this For the those of ordinary skill of field, without making creative work, it can also be obtained according to these attached drawings other Attached drawing.The following drawings is not drawn by actual size equal proportion scaling deliberately, it is preferred that emphasis is shows the purport of the disclosure.

Fig. 1 shows the illustrative flow chart of licence plate recognition method according to an embodiment of the present disclosure；

Fig. 2 shows being pre-processed in accordance with an embodiment of the present disclosure to vehicle image, obtain license plate image to be identified and The exemplary process diagram of the license plate type of license plate image to be identified；

Fig. 3 shows a kind of illustrative methods for carrying out Slant Rectify to license plate image to be identified according to the embodiment of the present disclosure Flow chart；

Fig. 4 A shows another example that Slant Rectify is carried out to license plate image to be identified according to the embodiment of the present disclosure The flow chart of property method；

Fig. 4 B shows another example that Slant Rectify is carried out to license plate image to be identified according to the embodiment of the present disclosure The network structure of property method；

Fig. 5, which is shown, carries out type conversion to the license plate image to be identified of multirow license plate type in accordance with an embodiment of the present disclosure Obtain the exemplary process diagram of the license plate image of uniline license plate type；

Fig. 6 A is shown to be identified to obtain license plate to the license plate image of uniline license plate type in accordance with an embodiment of the present disclosure The exemplary process diagram of number；

Fig. 6 B shows special by character string of the Recognition with Recurrent Neural Network to the multiple column in accordance with an embodiment of the present disclosure Sign is handled to obtain the exemplary process diagram of license plate number；

Fig. 7, which is shown, carries out the exemplary of result extraction for the differentiation result of neural network in accordance with an embodiment of the present disclosure Flow chart；

Using the schematic flow chart of disclosed method when Fig. 8 shows across camera car tracing；

Fig. 9 shows the illustrative block diagram of license plate recognition device according to an embodiment of the present disclosure；

Figure 10 shows the illustrative block diagram of car license recognition equipment according to an embodiment of the present disclosure.

Specific embodiment

The technical solution in the embodiment of the present disclosure is clearly and completely described below in conjunction with attached drawing, it is clear that Ground, described embodiment are only the section Example of the disclosure, instead of all the embodiments.Implemented based on the disclosure Example, every other embodiment obtained by those of ordinary skill in the art without making creative efforts also belong to The range of disclosure protection.

As shown in the application and claims, unless context clearly prompts exceptional situation, " one ", "one", " one The words such as kind " and/or "the" not refer in particular to odd number, may also comprise plural number.It is, in general, that term " includes " only prompts to wrap with "comprising" Include clearly identify the step of and element, and these steps and element do not constitute one it is exclusive enumerate, method or apparatus The step of may also including other or element.

Although the application is made that various references to the certain module in system according to an embodiment of the present application, however, Any amount of disparate modules can be used and be operated on user terminal and/or server.The module is only illustrative , and disparate modules can be used in the different aspect of the system and method.

Flow chart used herein is used to illustrate operation performed by system according to an embodiment of the present application.It should Understand, before or operation below not necessarily accurately carry out in sequence.On the contrary, as needed, it can be according to inverted order Or various steps are handled simultaneously.It is also possible to during other operations are added to these, or it is a certain from the removal of these processes Step or number step operation.

Fig. 1 shows the exemplary process diagram of the licence plate recognition method 100 according to the embodiment of the present disclosure.

Firstly, in step s101, pre-processed to vehicle image, obtain license plate image to be identified and it is described to Identify the license plate type of license plate image.

The vehicle image can be through the real-time captured image of road camera, or be also possible in advance with its other party The vehicle image that formula obtains.The embodiment of the present disclosure is not limited by the source of vehicle image and acquisition modes.For example, can serve as reasons The image that monitoring camera or road overspeed detection camera are directly shot, or can be and obtained after computer pre-processes The vehicle image arrived.

The license plate image to be identified is the image based on determined by license plate position in vehicle image, such as can be based on vehicle Memorial tablet is set to intercept vehicle image and be obtained.License plate position is location information of the license plate in vehicle image, the position Information can be indicated with the pixel coordinate of license plate.

The license plate type of the license plate image to be identified may include domestic or existing in the world a variety of license plates comprising But be not limited to uniline indigo plant board, uniline yellow card, uniline person who is not a member of any political party, the black board of uniline, duplicate rows license plate, motorcycle license plate, tricycle license plate etc.. It will be appreciated that the licence plate recognition method of the embodiment of the present disclosure is not limited to these license plate types, those skilled in the art are based on this It is open, the licence plate recognition method of the embodiment of the present disclosure easily can be applied to remaining license plate type.

In step s 102, in the case where the license plate type of the license plate image to be identified indicates multirow license plate, for The license plate image to be identified of the multirow license plate type carries out type conversion, obtains the license plate image of uniline license plate type.At this In step, the license plate image of multirow license plate type can be converted to the license plate of uniline license plate type by the license plate type based on acquisition The license plate image (hereinafter, referred to as duplicate rows license plate image) of duplicate rows license plate type can be particularly converted to uniline vehicle by image The license plate image (hereinafter, referred to as uniline license plate image) of board type.

It, in step s 103, will be further to the uniline license plate class after being converted to uniline license plate image via type The license plate image of type is identified, license plate number is obtained.The identification process for example can be complete by existing uniline identification model At identification, wherein since duplicate rows license plate image has carried out type conversion, only need to call a uniline identification mould in identification process Type reduces repetition and calls brought time loss, enhances the high efficiency of identification process.In addition, the identification process can also According to actual needs, to choose extraction and feature sequence of the realization such as convolutional neural networks, Recognition with Recurrent Neural Network for vehicle license plate characteristic The identification of column can also complete the identification for license plate content by Character segmentation and individual character identification step.

Fig. 2 shows pre-processing to vehicle image according to the embodiment of the present disclosure, obtain license plate image to be identified and The exemplary process diagram of the license plate type of license plate image to be identified.

Referring to Fig. 2, it is based on vehicle image by depth convolutional neural networks and obtains license plate image to be identified and to be identified The license plate type of license plate image.

Firstly, extracting characteristics of image in step S201 for vehicle image via the first convolutional neural networks, identifying license plate Position obtains license plate image to be identified.For example, being handled using depth convolutional neural networks (CNN) for image, at this In the process, the characteristic pattern after extraction is divided into 10*10 pixel region or 8*8 pixel region first, each pixel region is made The multiple characteristics of pixel in each region to be determined, base are further extracted by convolutional neural networks for region to be determined In extracted feature, region to be determined is classified, and has been divided into license plate portion and without license plate portion, has been finally obtained the position of license plate Area information can be based further on the position later and vehicle pictures are intercepted, obtain license plate image to be identified.

After the extraction for completing license plate image to be identified, by step S202, via the second convolutional neural networks for vehicle Image zooming-out characteristics of image obtains the license plate type of license plate image to be identified.It wherein can for example choose another depth convolutional Neural Network (CNN) handles multiple regions to be determined on vehicle image, and convolutional neural networks herein realize license plate class When type feature extraction, it can be obtained by relevant informations such as the comprehensive color extracted in determinating area, shape, pattern, text distributions The multiple characteristics of pixel in determinating area are based on extracted feature, determine that it is uniline or multirow, and identify its license plate face Color.Alternatively, neural network can also support machine (SVM) to classify it by preset vector, it is divided into different License plate type, by contrasting the license plate with the license plate type prestored, identification obtains its license plate type.

The process can be more specifically described, for example, when selected depth convolutional neural networks be based on sliding window algorithm for When road violation vehicle image carries out vehicle type recognition, license plate area can be extracted for obtained license plate image to be identified The color of interior pixel, texture information, and the text characteristic distributions based on license plate area are based on above-mentioned two parts information, can be for The license plate type of license plate image to be identified judged, in addition, can based on license plate Text region as a result, including the number of text Amount and distributing position further help the differentiation for realizing the type for license plate.

Process synthesis Text region result and license plate characteristics of image itself identify, are beneficial to through various information Differentiate the accuracy improved for license plate type identification.

It is described referring to Fig. 2 and obtains license plate image using the first convolutional neural networks and utilize and the first convolution nerve net The second different convolutional neural networks of network obtain the embodiment of the license plate type of license plate image to be identified.However, the disclosure is implemented Example is without being limited thereto, extracts characteristics of image for vehicle image for example, can use third convolutional neural networks, while obtaining wait know The license plate type of other license plate image and license plate image to be identified.In the embodiments of the present disclosure, not to obtaining license plate image to be identified Any restrictions are made with the quantity of the convolutional neural networks of the license plate type of license plate image to be identified.

More specifically, the third convolutional neural networks can be for multiple regions to be determined on vehicle image at Reason, extracting includes location information, whether there is or not the multiple characteristics of license board information, character and color and dimensions information etc., and the further spy Sign can support machine (SVM) or other sorters to carry out processing differentiation by multiple vectors, therefore can be via the convolutional Neural net The processing of network while the location information and type information for obtaining license plate in vehicle image.

It will be appreciated that above-mentioned given embodiment is only that the process for illustrating to extract vehicle position information and type information, it is right Realize that characteristic quantity and means are with no restrictions based on judgement in it, it can be using for example real via Text region (OCR) merely The type identification of existing vehicle, merely via size color realize vehicle type identification, via it is both above-mentioned combine it is obtained As a result comprehensive realization type of vehicle, which differentiates or further adds other features such as gray scale, vehicle context information, is formed MULTIPLE COMPOSITE identification process realize the identification for type of vehicle, any restrictions are not made to it herein.

In addition, the first, second, third convolutional neural networks described in above-described embodiment can be selected based on actual needs It takes, for example, by using depth convolutional neural networks, neural network with friendship and than other algorithm phases such as algorithm, non-maximum value restrainable algorithms It is formed by again in conjunction with the network of formation, or using it to be integrated with fully-connected network or other integrated treatment network architectures Close neural network.Any restrictions are not made to it herein.

In addition, being carried out using the training data in existing picture library for the first, second, third convolutional neural networks Training, wherein training data may be, for example, the vehicle image obtained under monitoring scene or via the pretreated vehicle figure of computer Picture.Further, the image in existing picture library can be selected, translated to expand training data, improve convolutional Neural net The data training burden of network, further enhances its performance.

According to the embodiment of the present disclosure, it can also include carrying out to license plate image to be identified that vehicle image, which is pre-processed, Slant Rectify.

Fig. 3 shows a kind of exemplary side that Slant Rectify is carried out to license plate image to be identified according to the embodiment of the present disclosure Method.

Firstly, filtering the background information of license plate image to be identified in step S301, which can for example pass through neural network Feature extraction is completed specific region and is chosen, or the rejecting for background can be realized via image procossing.Pass through abovementioned steps License plate image to be identified after being cut further in step s 302, the inclination angle of license plate is detected by straight-line detection Degree.The straight-line detection process can be detected for example by Hough transformation or be realized by integral transformation (such as radon transformation), and phase is based on The tilt angle of current license plate is calculated in the formula answered.Further, based on acquired tilt angle, in step S303 In, in the case where the license plate sloped degree meets predetermined condition, Slant Rectify is carried out to the license plate image to be identified.Example Such as, license plate is rotated to horizontal direction by rotation transformation, the horizontal license plate after being corrected.

Fig. 4 A shows another example that Slant Rectify is carried out to license plate image to be identified according to the embodiment of the present disclosure Property method.Fig. 4 B shows network structure used by Fig. 4 A method.Firstly, in step S401, passing through convolution referring to Fig. 4 A Neural network extracts the characteristics of image of license plate image to be identified.Specifically, the step for example can be via shown in Fig. 4 B Convolutional neural networks input license plate image to be identified in input layer, via the convolutional layer of convolutional neural networks and the place of pond layer Reason, can obtain the license plate image to be identified after image characteristics extraction in output end.Further, it is based on described image feature License plate image to be identified after extraction is handled extracted characteristics of image by fully-connected network in step S402, Obtain the position of current license plate key point.According to the embodiment of the present disclosure, the license plate key point for example can be four of license plate Angle point, offset of the position of obtained license plate key point by license plate key point relative to pre-set image origin indicate.So And according to the embodiment of the present disclosure, the license plate key point can also be four angle points and central point or can be vehicle of license plate The midpoint on four sides of four angle points and license plate of board or can be four angle points of license plate and four of central point and license plate The midpoint on side can also be surveyed as needed according to other standards and choose the license plate key point, and the embodiment of the present disclosure does not close license plate The position of key point and selection mode make limitation.

Operation above-mentioned steps S402 can be more specifically described, when carrying out the positioning of license plate key point position, such as can By fully-connected network shown in figure, it is based on preset image origin, the coordinate of each crucial angle point is returned, is obtained Offset to each angle point relative to image origin.As described in Fig. 4 B, such as image can be arranged xy reference axis, setting figure Upper left angle point is origin as in, and positive direction of the x-axis is to extend in the horizontal direction towards on the right side of image from origin, positive direction of the y-axis be from Origin extends vertically towards below image, then is based on the reference axis and origin position, available license plate upper left angle point A, upper right angle point B, lower-left angle point C, bottom right angle point D relative to the coordinate of origin be followed successively by (0,20), (110,0), (10,80) and (120,60), the coordinate characterize the position of license plate key point on this image.

Behind the position for having obtained current license plate key point, next in S403, the position based on current license plate key point It sets, Slant Rectify is carried out to the license plate image to be identified, obtains horizontal license plate image.Wherein, in some embodiments, it is based on License plate is converted in the position of current key point to be rotated license plate to horizontal direction via rotation transformation, after obtaining correction Horizontal license plate image.In further embodiments, the position based on current license plate key point, can be via perspective transform, by vehicle Board maps to horizontal direction, the horizontal license plate after being corrected.It is based on known current key point position and preset target License plate by original evolution is horizontal position, completed for license plate to be identified via perspective transform by license plate key point position The correcting process of image.

Above-mentioned steps S403 can be more specifically described, such as the inclination for realizing license plate image to be identified is being changed by transformation In the case where correction, referring to the rotation transformation process in Fig. 4 B, it can be incited somebody to action based on the position of obtained key point A, B, C, D Its line two-by-two, obtains the straight line AB and CD in horizontal direction.It further, can be by the horizontal axis of itself and license plate image, i.e., The horizontal line parallel with x-axis axis is compared, and obtains its angle relative to horizontal axis, such as AB and process A point and and x The angle of axis parallel horizontal axes is 30 degree, and CD is 33 degree with the angle by C point and the horizontal axis parallel with x-axis, then may be used License plate area composed by its key point is rotated 33 degree along towards x-axis direction around image origin, obtains bottom edge and horizontal direction Parallel horizontal license plate.

In addition, for example in the case where realizing the Slant Rectify of license plate image to be identified by perspective transform, it is possible, firstly, to Perspective transformation matrix is calculated referring to the position for presetting license plate key point based on the position of current license plate key point.It is specific and Speech, the formula of transformation matrix are as follows:

Wherein x, y are target position, and u, v are the position of current license plate key point.A matrix is transformation matrix in figure, is led to The position for substituting into current key point position and target license plate key point is crossed, can seek obtaining transformation matrix.

Specifically, referring to the perspective transform process in Fig. 4 B, such as the target position difference of target critical point A, B, C, D For A ' (0,0), B ' (120,0), C ' (0,80), D ' (120,80), current location be respectively (0,20), (110,0), (10, 80) with (120,60), then formula is substituted into, transformation matrix A can be acquired, then, the transformation matrix can be based on, on license plate All the points application transformation matrix, seek its corresponding position under dbjective state, realize the perspective transform for license plate accordingly, Horizontal license plate after being corrected.

Based on above-mentioned non-limiting embodiment, gives and Slant Rectify method is carried out to license plate image to be identified, go forward side by side one Step ground, based on different embodiments, during being corrected based on key point for license plate image to be identified, for obtaining Correction is carried out for license plate image to be identified after license plate key point and gives rotation transformation and the two different sides of perspective transform Method, it should be understood that the application is not limited to the above method, in fact, can also take other methods, such as Selection Center point carries out Image rotation, such as the correction via translation transformation and the realization of the complex transformation of rotation transformation for license plate image to be identified Process.

In some embodiments, it is operated executing Slant Rectify described in embodiment as above for license plate image to be identified Before may also include the process that gradient judgement is carried out for the license plate in license plate image to be identified, be less than or wait in license plate sloped degree In the case where preset threshold, the position of current license plate key point is directly exported；It is greater than the feelings of preset threshold in license plate sloped degree Under condition, Slant Rectify is carried out for license plate image to be identified.

Wherein in the license plate image to be identified license plate gradient, indicate current obtained license plate relative to horizontal line Gradient, tilt angle can be used and be indicated.Such as can be used license plate hemline and horizontal angle, top line with Horizontal angle characterizes, or the angle for the top line or hemline and vertical line that can also pass through license plate characterizes.

For example, using it is above-mentioned realized based on license plate key point license plate image to be identified is corrected in the case where, based on working as The position of license plate key point, converts license plate in preceding license plate image to be identified, and obtaining horizontal license plate before can be for it Gradient is judged, is formed by top line in license plate key point line and horizontal angle is greater than preset threshold and vehicle Board key point line is formed by the case that hemline and horizontal angle be greater than preset threshold, or works as license plate key point Line is formed by top line and license plate key point line is formed by hemline one of both and horizontal angle is greater than in advance If in the case where threshold value, then position based on current license plate key point converts license plate image to be identified, obtains level License plate image.Wherein preset threshold for example may be designed as 5 degree or 10 degree.

It can be more specifically described, such as when it is 5 degree that preset threshold, which is arranged, if license plate key point line is formed by top Sideline and horizontal angle, that is, straight line AB and by A point and to be parallel to the horizontal angle of x-axis axis be 10 degree is big In 5 degree；And license plate key point line is formed by hemline and horizontal angle, that is, straight line CD with by C point and being parallel to The horizontal angle of x-axis axis is 3 degree, when less than 5 degree, then license plate image to be identified is carried out described in step S403 Correcting process.If license plate key point line is formed by top line and horizontal angle, that is, straight line AB and by A point and flat Row is 3 degree in the horizontal angle of x-axis axis, and license plate key point line is formed by hemline and horizontal angle, That is, straight line CD and by C point and to be parallel to the horizontal angle of x-axis axis be 2 degree, two straight line AB, CD and horizontal line at this time Angle be respectively less than 5 degree, then the conversion of correction described in S403 is not executed it, directly exports current license plate image to be identified ?.

In addition, can equally complete step S302 institute for example during realizing license plate correction based on straight-line detection After the tilt angle detection stated, which is compared with preset threshold, if it is less than or equal to preset threshold, no Correcting process described in subsequent step S303 is executed, only when it is greater than preset threshold, knowledge is treated using step S303 execution The correcting process of other license plate image.

Such judgment step can increase the license plate antidote 300 based on straight-line detection and the license plate based on key point is rectified The flexibility ratio of correction method 400, that is, when image deviation angle is lower, avoids and introduce new direction position influence because of correction, Pretreated step is simplified simultaneously, improves efficiency；When image deviation angle is larger, realized by correction for image Calibration obtains horizontal license plate, is convenient for subsequent treatment process.

After completing preprocessing process, in S102, multirow license plate is indicated in the license plate type of the license plate image to be identified In the case where, type conversion is carried out for the license plate image to be identified of the multirow license plate type, obtains uniline license plate type License plate image.

Fig. 5 shows the feelings of the license plate type instruction multirow license plate in license plate image to be identified according to the embodiment of the present disclosure Under condition, is converted to the license plate image to be identified progress type of multirow license plate type showing for the license plate image of uniline license plate type Example property flow chart.

Referring to the flow diagram, in the method that the license plate image to be identified for multirow license plate type carries out type conversion In 500, it is necessary first to judged by license plate type of the step S501 to license plate image to be identified, then, such as step S502, In the case where the license plate type of license plate image to be identified is multirow license plate, then for the multirow license plate by splicing at conversion Reason, obtains the license plate image of uniline license plate type；It is not more in the license plate type of the license plate image to be identified such as step S503 It drives a vehicle in the case where board, directly outputs it the license plate image as uniline license plate type.That is, being obtained to based on feature extraction A variety of license plate types judged, if it is multirow license plate, then further carry out type conversion to it, and as it is uniline Various other types such as yellow card, uniline indigo plant board, the black board of uniline, uniline person who is not a member of any political party, then be directly entered subsequent identification module, and nothing Type conversion need to be carried out to it.

As described above, in step S502, in the case where the license plate type of license plate image to be identified is multirow license plate, then For the multirow license plate by splicing conversion processing, the license plate image of uniline license plate type is obtained.Below with the multirow vehicle Board be duplicate rows license plate for, be illustrated.It will be appreciated that the multirow license plate splicing conversion processing of the embodiment of the present disclosure be not limited only to it is double Driving board, but can be adapted for any multirow license plate.

Firstly, by step S5021, based on the dimensions of duplicate rows license plate, in the license plate to be identified of duplicate rows license plate type The position of upper and lower level cut-off rule is determined on image.Pass through the upper and lower level distributing position of duplicate rows license plate and occupied height and face Product, determines cut-off rule, which can be well by upper and lower level cutting without will affect in upper and lower level in corresponding data and character Hold.The process can be more specifically described, when acquired current license plate is to meet People's Republic of China's public safety industry mark The duplicate rows license plate of the size of quasi- GA36-2014 People's Republic of China (PRC) automotive number plate duplicate rows license plate then can for example be based on its institute It is arranged by cut-off rule according to the ratio of 6:14 as the horizontal line apart from top line 60mm in defined upper and lower level size；Or it can By cut-off rule according to the ratio of 4:11, it is set for the horizontal line apart from top line 58mm.

After completing the setting for cut-off rule, further, in step S5022, along identified cut-off rule cutting The license plate image to be identified of the duplicate rows license plate type, obtains upper layer license plate image and lower layer's license plate image.In this course, Based on the feature of the double-deck license plate, the processing of blank redundancy can be removed to it, such as current license plate is to meet China's motor vehicle The license plate of duplicate rows license plate number plate size then further can be away from its left and right edges 20mm for the upper layer license plate after cutting Place is arranged perpendicular to horizontal vertical divider, the upper layer license plate is further cut along the vertical divider, is gone Except extra blank background portion, the upper layer license plate image of more high quality is obtained.

The cutting for license plate is being completed, after obtaining upper layer and lower layer's license plate image, in step S5023, is being based on Preset rules carry out size change over to the upper layer license plate image, the upper layer license plate image after obtaining size change over.In some realities It applies in example, which can be realized by perspective transform, such as by preset target position, for complete in the license plate of upper layer Portion's point carries out perspective transform, so that arriving that treated, upper layer license plate image is identical as lower layer's license plate length and width dimensions.At other In embodiment, which can be realized by scaling, i.e., reach its height or width by simple tension upper layer license plate It is set to obtain size identical with lower layer's license plate to fixed length or via biaxial tension, equal proportion scaling.Wherein preset rules are intended to The transformation that size is carried out for upper layer license plate makes it possible to well splice with lower layer license plate and spliced character size size is use up May be consistent, it will not be affected for subsequent identification process.

Operation above-mentioned steps S5023 can be more specifically described, such as size change over is taken to realize for upper layer license plate Processing, it is contemplated that the height of upper and lower level license plate is different after cutting, respectively 60mm and 110mm, at the same in view of both word The difference of body size and spacing, then can preset rules be fixed upper layer license plate height be 110mm carry out license plate amplification, same to time control The length-width ratio of upper layer license plate processed is in the range of being less than or equal to 0.375 times that does not scale preceding upper layer license plate length-width ratio.Based on this Preset rules transformation, on the one hand the size of adjustable license plate up and down, splices, while passing through strict control convenient for subsequent license plate Length-width ratio has been evaded in equal proportion amplification process, is stretched and is deformed as height increases caused character, is convenient for subsequent progress Character recognition.

It, will treated upper layer license plate image and lower layer's vehicle in step S5024 after completing for the conversion of upper layer license plate Board image carries out horizontal direction splicing, the license plate image of the uniline license plate type after being converted.In this process, based on motor-driven The splicing sequence of upper layer license plate and lower layer's license plate may be selected in the rule of vehicle.Specifically, according to the rule of China's motor vehicle duplicate rows license plate Model sequentially splices treated upper layer license plate with lower layer license plate, then can in the case where completely remaining its sequence and content, The transformation of to be identified license plate image of the good license plate image to be identified for realizing duplicate rows license plate type to uniline license plate type.

It will be appreciated that above-described embodiment is intended to illustrate that the license plate image to be identified by multirow license plate type is converted to uniline license plate The process of the license plate image to be identified of type, the multirow license plate type are not limited to duplicate rows license plate described in embodiment, It may include other a variety of multirow license plate types such as three driving boards.

The above process extracts license plate image feature to be identified, and using setting cut-off rule, voluntarily divides then It is spliced to form the mode of the license plate image to be identified of uniline license plate type, is reduced in the mark extraction process of position for manpower Consumption simplify the processing of multirow license plate while convenient for the subsequent identification completed via Car license recognition for multirow license plate The step of, repetition when reducing processing for identification model is called, and is further improved for multirow license plate type wait know The treatment effeciency of other license plate image.

It completes based on the license plate image and license plate type, type conversion is carried out for license plate, obtains uniline license plate After process, in S103, the license plate image of the uniline license plate type is identified, license plate number is obtained.

Fig. 6 A is shown to be identified to obtain to the license plate image of the uniline license plate type in accordance with an embodiment of the present disclosure The exemplary process diagram of the process 600 of license plate number.

Referring to Fig. 6 A, first in step s 601, by convolutional neural networks from the license plate figure of the uniline license plate type License plate image feature is extracted as in.Wherein the convolutional neural networks can be for example with the common convolution kernel of 3*3 convolutional layer, can also Think the combination that the convolutional neural networks and the convolutional neural networks with 1*1 convolution kernel of convolution kernel are separated with 3*3 depth. Specifically, the convolutional neural networks for separating convolution kernel with 3*3 depth can be used in said combination convolutional neural networks Convolution is carried out to each channel in the license plate image of uniline license plate type, obtains corresponding feature extraction result；With 1*1 volumes The convolutional neural networks of product core are then used for after the convolution in all channels, the convolution results in all channels are carried out linear Combination, that is, play the role of liter dimension or a dimensionality reduction for the convolution results in all channels.Such compound convolutional neural networks are compared In common convolution kernel neural network above-mentioned, the complexity of convolution process can be largely reduced, and further promote convolution The overall performance of neural network.

After completing the image characteristics extraction of license plate image of uniline license plate type, further, it is based on step S602, is pressed Character string feature is divided according to column, obtains the character string feature of multiple column.That is, will be during image characteristics extraction, convolution The obtained whole image feature of neural network, is divided according to different column.

For example, in the neural network characteristics figure obtained after image zooming-out process, such as 512*16*1,512 are characterized figure Port number, 16 are characterized figure width, and 1 is characterized figure height, and the character sequence in its region then can be got to feature graph region Column feature, the character string feature can be identified by column vector, may be, for example, the character string feature vector group of 16 column, for each The obtained character feature vector of image-region is the character string feature vector group of 512*1.Then in this step, finally may be used Obtain 16 character feature sequences divided based on column.

Character string feature for obtained multiple column further in step S603, passes through Recognition with Recurrent Neural Network The character string feature of the multiple column is handled, license plate number is obtained.Recognition with Recurrent Neural Network herein may include but It is not limited to shot and long term memory network (LSTM), two-way shot and long term memory network (Bi-LSTM), deep layer Recognition with Recurrent Neural Network (DRNN) And its with other neural networks are compound is formed by complex neural network, the present invention is not limited to used Recognition with Recurrent Neural Network Type.

Fig. 6 B is shown to be handled by character string feature of the Recognition with Recurrent Neural Network to the multiple column, obtains license plate The exemplary process diagram of number.

Referring to Fig. 6 B, when being handled using neural network it, first via step S6031, by the compound spy Sign vector is sent to the input terminal of selected neural network.That is, the character feature sequence of aforementioned obtained multiple column is passed The input terminal for transporting to neural network, is handled it convenient for neural network.

Thereafter, it in step S6032, is handled via character string feature of the neural network for the multiple column, It is background or respective symbols by the character string feature decision in each column.It is right herein based on the recognition mode in neural network In the character string feature inputted, via the processing of neural network, its character string feature is identified as background or word by setting Symbol, exports as a result.If it is blank or meaningless lines, it is identified as background, if it is character, identifies corresponding word Symbol output.

The above process can be specifically described, such as take two-way length memory network in short-term, by the vehicle of content " capital AF2036 " Board is divided into 16 column, and the column character string feature that each column 512 is tieed up is inputted into two-way length memory network (long-short) in short-term, then base To the processing of layer and reversed layer before neural network, the result of final output can be " ' background ', ' capital ', ' background ', ' A ', ' back Scape ', ' F ', ' background ', ' 2 ', ' background ', ' 0 ', ' background ', ' 3 ', ' background ', ' 6 ', ' background ', ' background ' ".

After completing identification, it can be based on step S6033, based on differentiation as a result, obtaining license plate number.In some embodiments, Step S6033 may include that result extraction process and error reject process.It is neural network based defeated in result extraction process Out, the result of Car license recognition is extracted.Such as the Repeating Field in license plate recognition result can be verified and be deleted, it obtains Delete verification after as a result, and " background " in recognition result can be deleted, obtain the significant character sequence of vehicle Column.Can also be carried out after result extraction process error reject process, such as can for the character not in characters on license plate, such as " % ", " $ " etc. are deleted.

Fig. 7, which is shown, carries out the exemplary of result extraction according to the differentiation result for neural network of the embodiment of the present disclosure Flow chart.

Referring to the flow diagram, result extracting method 700 herein, it is contemplated that character string feature is identified by column vector When, the column may overlap, thus it is obtained as a result, may in adjacent column to cause Recognition with Recurrent Neural Network to detect Occur identical as a result, the problem of causing character recognition to repeat.It is considered based on such, accidentally result is being carried out for recognition result In extracting method 700, the recognition result of Recognition with Recurrent Neural Network is obtained via step S701 first, thereafter, in step S702, purport Every two adjacent column verify in for recognition result, and whether the recognition result for verifying adjacent column is identical.If adjacent Column recognition result it is different, then use step S704, retain the recognition result of the adjacent column, be not processed.If adjacent column Recognition result it is identical, then by step S703, further judge whether the identical recognition result is character, knowledge if they are the same Other result is character, then uses step S706, retain the recognition result of the adjacent column, be not processed；If the identical identification knot Fruit is character, then by step S705, identical characters which is identified and be a character.

For example, if recognition result be " ' background ', ' Soviet Union ', ' background ', ' A ', ' A ', ' background ', ' Y ', ' Y ', ' background ', ' 0 ', ' 3 ', ' 3 ', ' 8 ', ' 9 ', ' 9 ', ' background ' ", then after carrying out result extraction process, duplicate character is merged, institute Obtained license plate number is " Soviet Union AY0389 ".Via the result extraction process, may be implemented for neural network recognization result Screening and amendment evade identification content and repeat to repeat with the symbol for obviously not meeting license plate specification, improve license plate recognition result Accuracy.

Fig. 8 is shown in across camera realization car tracing using the schematic flow of the method for the embodiment of the present disclosure Figure.

Referring to Fig. 8, next by by the implement scene of across camera realization car tracing for embodiment of the present disclosure institute The method of stating is further described through, in the case where across camera realization car tracing, based on containing for current street capture The image of vehicle body is primarily based on step S801 via method described in the embodiment of the present disclosure, vehicle image is carried out pre- Processing, obtains the license plate type of license plate image to be identified and the license plate image to be identified.The process can be used via convolution Neural network for vehicle image carry out feature extraction, obtain about the vehicle license plate image to be identified and it is described to Identify the license plate type of license plate image.As shown in Figure 8, judge to obtain in the license plate image to be identified license plate type as " duplicate rows Yellow card " simultaneously obtains its license plate image.

Thereafter, based on obtained license plate type, in step S802, it is based on obtained license plate image to be identified, it can be into One step carries out Slant Rectify process for the license plate image to be identified, firstly, identifying four angles of license plate in step S8021 Point in step S8022, based on angle point A, B, C, the D identified, obtains the top line of current license plate thereafter as key point AB and its hemline CD, seeks itself and horizontal angle respectively, and as shown in the figure, the top line and horizontal angle are 25 Degree, hemline and horizontal angle are 20 degree.Thereafter, its gradient is judged based on step S8023, is corrected herein In the process, preset threshold is 10 degree, therefore relatively knows that the tilt angle of current vehicle is greater than preset threshold, is tilted to it Correcting process.It can be based on perspective transform through step S8024 after it and be mapped to horizontal position.

Thereafter, in step S803, the license plate type of the current license plate image to be identified obtained according to differentiation is duplicate rows vehicle Board determines cut-off rule based on the dimensions of duplicate rows license plate in license plate image and divides its edge further via step S8031 Cutting is that upper layer license plate image and lower layer's license plate image carry out size change over to it by step S8032 thereafter to secant up and down, So that the adjoining dimensions of upper layer license plate and lower layer's license plate, are easy to splice.Finally in step S8033, for the upper layer after conversion License plate splices it with lower layer's license plate, the license plate image of the uniline license plate type after being converted.

Finally, the license plate image of the uniline license plate type is inputted into the input terminal into neural network via step S804, In step S8041, feature extraction is carried out to it via neural network, thereafter in step S8042, divided by column sequence signature, The character string feature of multiple column is obtained, later via step S8043, its each column is determined as background or character, is known Other result " ' background ', ' Soviet Union ', ' background ', ' A ', ' background ', ' Y ', ' Y ', ' background ', ' 0 ', ' background ', ' 3 ', ' background ', Result after ' 6 ', ' background ', ' 8 ', ' background ' " differentiates further can carry out error rejecting processing by step S8044, reject The wherein repeat character (RPT) between adjacent column finally obtains the license plate number " Soviet Union for vehicle body in the street monitoring image AY0368”。

Based on this as a result, can judge it with the vehicle of tracking is required, the association between the two is differentiated, it can be to city Political affairs traffic management department or the publication in time of interested individual across camera track as a result, to may be based on the differentiation further true The wheelpath or escape route for determining vehicle, further deploy to ensure effective monitoring and control of illegal activities in tracing process convenient for traffic control department.This method can be answered well During deploying to ensure effective monitoring and control of illegal activities for across the camera tracking of vehicle or vehicle, improves efficiency and reduced via the conversion process of multirow license plate and known The time-consuming of other process.

Fig. 9 shows the exemplary block diagram of the license plate recognition device according to the embodiment of the present disclosure.

License plate recognition device 900 as shown in Figure 9 includes: preprocessing module 910, type conversion module 920 and license plate Identification module 930.

The preprocessing module 910 is configured as pre-processing vehicle image, obtain license plate image to be identified and The license plate type of the license plate image to be identified.

The type conversion module 920 is configured to indicate multirow license plate in the license plate type of the license plate image to be identified In the case of, type conversion is carried out for the license plate image to be identified of the multirow license plate type, obtains the vehicle of uniline license plate type Board image.

The Car license recognition module 930 is configured as identifying the license plate image of the uniline license plate type, obtain License plate number.

Wherein the vehicle image can be through the real-time captured image of road camera, or be also possible in advance with The vehicle image that other modes obtain.The embodiment of the present disclosure is not limited by the source of vehicle image and acquisition modes.For example, can With the image directly to be shot by monitoring camera or road overspeed detection camera, or it can be and locate in advance by computer The vehicle image obtained after reason.

The license plate image to be identified is the image based on determined by license plate position in vehicle image, can be based on license plate position It sets, vehicle image is intercepted and is obtained.Wherein license plate position is location information of the license plate in vehicle image, the position Information can be indicated with the pixel coordinate of license plate.

In preprocessing module, process as shown in Figure 2 can be performed, via the first convolutional neural networks for vehicle image Characteristics of image is extracted, license plate position is identified, obtains license plate image to be identified.For example, using depth convolutional neural networks (CNN) Image is handled, in this process, the characteristic pattern after extraction is divided into 10*10 pixel region or 8*8 first Pixel region, each pixel region is as region to be determined, further, it is to be determined to extract each by convolutional neural networks The multiple characteristics of pixel in region are based on extracted feature, and region to be determined is classified, and have been divided into license plate portion and without vehicle Board portion finally obtains the position area information of license plate.

After the extraction for completing license plate image, by step S202, vehicle image is mentioned via the second convolutional neural networks Characteristics of image is taken, identifies the license plate type of the license plate image to be identified.It wherein can for example choose another depth convolutional Neural net Network (CNN) handles multiple regions to be determined on vehicle image, and convolutional neural networks herein realize license plate type When feature extraction, it can be sentenced by relevant informations such as the comprehensive color extracted in determinating area, shape, pattern, text distributions Determine the multiple characteristics of the pixel in region, is based on extracted feature, determines that it is uniline or multirow, and identify its license plate face Color.Alternatively, neural network can also support machine (SVM) to classify it by preset vector, it is divided into different License plate type, by contrasting the license plate with the license plate type prestored, identification obtains its license plate type.

The process can be more specifically described, for example, when selected depth convolutional neural networks be based on sliding window algorithm for When road violation vehicle image carries out vehicle type recognition, license plate area can be extracted for obtained license plate image to be identified The color of interior pixel, texture information, and the text characteristic distributions based on license plate area are based on above-mentioned two parts information, can be for The license plate type of license plate image to be identified judges, in addition, can based on license plate Text region as a result, include text quantity and Distributing position further helps the differentiation for realizing the license plate type for license plate image to be identified.

Process synthesis Text region result and license plate characteristics of image itself identify, are beneficial to through various information Differentiate the accuracy for improving the license plate type identification for license plate image to be identified.

However, the embodiment of the present disclosure is without being limited thereto, for example, preprocessing module can use third convolutional neural networks for Vehicle image extracts characteristics of image, while obtaining the license plate type of license plate image to be identified and the license plate image to be identified. In the embodiments of the present disclosure, any restrictions are not made to the quantity for the convolutional neural networks for obtaining license plate image and license plate type.

More specifically, the third convolutional neural networks can be for multiple regions to be determined on vehicle image at Reason, extracts that it includes position, whether there is or not the multiple characteristics such as license plate, character and color and dimensions, and further this feature can be by multiple Vector supports that machine (SVM) or other sorters carry out processing differentiation, thus can via the convolutional neural networks processing simultaneously Obtain the position of license plate and type in vehicle image.

Preprocessing module 910 shown in Fig. 9 may also include Slant Rectify module, can be performed as shown in Figure 3 or Figure 4 Slant Rectify process carries out slant correction to the license plate image to be identified.

Further, multirow license plate be duplicate rows license plate when, in type conversion module 920 include License Plate Segmentation module 921, Size change over module 922 and license plate splicing module 923.

The License Plate Segmentation module 921 is configured as the dimensions based on duplicate rows license plate, duplicate rows license plate type to Identify the position that upper and lower level cut-off rule is determined on license plate image.Its executable mistake as shown in step S5021 and S5022 in Fig. 5 Journey, upper and lower level distributing position and occupied height and area by duplicate rows license plate, determines cut-off rule, so that the cut-off rule can Well by upper and lower level cutting without will affect corresponding data and character content in upper and lower level.When acquired current license plate is symbol Close the ruler of People's Republic of China's industry standards of public safety GA36-2014 People's Republic of China (PRC) automotive number plate duplicate rows license plate Very little duplicate rows license plate, then for example can be based on the upper and lower level size of its defined, by cut-off rule according to the ratio of 6:14, it, which is arranged, is Horizontal line apart from top line 60mm；Or can be by cut-off rule according to the ratio of 4:11, it is apart from top line 59mm's that it, which is arranged, Horizontal line.

After completing the setting for cut-off rule, along the license plate to be identified of identified cut-off rule cutting duplicate rows license plate type Image obtains upper layer license plate image and lower layer's license plate image.In this course, the feature based on the double-deck license plate, can to its into The processing of row removal blank redundancy, such as current license plate is the license plate for meeting China's motor vehicle duplicate rows license plate number plate size, then into One step, the upper layer license plate after cutting can be arranged at away from its left and right edges 20mm perpendicular to horizontal vertical point Secant further cuts the upper layer license plate along the vertical divider, removes extra blank background portion, obtain more For the upper layer license plate image of high quality.

The cutting for license plate is being completed, after obtaining upper layer and lower layer's license plate image, in size change over module 922 In, preset rules are based on, size change over will be carried out to the upper layer license plate image, the upper layer license plate figure after obtaining size change over Picture.In some embodiments, which can be realized by perspective transform, such as by preset target position, for upper Whole points in layer license plate carry out perspective transforms, so that arrive that treated upper layer license plate image and lower layer's license plate length and width dimensions phase Together.In further embodiments, which can be realized by size change over, i.e., upper layer license plate is made by simple tension Its height or width reach fixed length or so that it is obtained size identical with lower layer's license plate via biaxial tension, equal proportion scaling.Its Middle preset rules are intended to carry out upper layer license plate the transformation of size, make it possible to well to splice with lower layer license plate and spliced Character size size is as consistent as possible, will not affect for subsequent identification process.

Operation performed by the size change over module 922 can be more specifically described, such as take size change over realization pair In the processing of upper layer license plate, it is contemplated that the height of upper and lower level license plate is different after cutting, and respectively 60mm and 110mm consider simultaneously To the font size of the two and the difference of spacing, then can preset rules be fixed upper layer license plate height be 110mm carry out license plate Amplification, while the length-width ratio for controlling upper layer license plate is being less than or equal to 0.375 times of model for not scaling preceding upper layer license plate length-width ratio In enclosing.It is converted based on the preset rules, on the one hand the size of adjustable license plate up and down, splices, simultaneously convenient for subsequent license plate By strict control length-width ratio, evade in equal proportion amplification process, stretched and deform as height increases caused character, Convenient for subsequent carry out character recognition.

After completing for the conversion of upper layer license plate, in license plate splicing module 923, will execute will treated upper layer license plate Image and lower layer's license plate image carry out the operation of horizontal direction splicing, the license plate image of the uniline license plate type after being converted. In this process, the splicing sequence of upper layer license plate and lower layer's license plate may be selected in the rule based on motor vehicle.Specifically, according to China Treated upper layer license plate is sequentially spliced with lower layer license plate, then can completely remain it by the specification of motor vehicle duplicate rows license plate It is good to realize that the license plate image to be identified of multirow license plate type, which is carried out type, is converted to uniline in the case where sequence and content The license plate image of license plate type.

Based on above-mentioned module, in type conversion module, license plate image feature is extracted, and using setting segmentation Line voluntarily divides the mode for being then spliced to form uniline license plate, reduces the disappearing for manpower in the mark extraction process of position Consumption, while the identification for duplicate rows license plate is completed via a Car license recognition convenient for subsequent, simplify the step of the double-deck license plate processing Suddenly, repetition when reducing processing for identification model is called, and the treatment effeciency for duplicate rows license plate is further improved.

Process as shown in Figure 5 can be performed in Car license recognition module 930, may include that license plate image feature mentions further Modulus block 931 is configured to extract license plate image feature from the license plate image of uniline license plate type by convolutional neural networks. Wherein the convolutional neural networks can be for example with the common convolution kernel of 3*3 convolutional layer, or has 3*3 depth separable The combination of the convolutional neural networks of convolution kernel and the convolutional neural networks with 1*1 convolution kernel.Specifically, being rolled up in said combination In product neural network, the convolutional neural networks for separating convolution kernel with 3*3 depth can be used for the license plate to uniline license plate type Each channel of image carries out convolution, obtains corresponding feature extraction result；Convolutional neural networks with 1*1 convolution kernel are then used In after the convolution in all channels, linear combination is carried out for the convolution results in all channels, that is, for all channels Convolution results play the role of liter dimension or a dimensionality reduction.Such compound convolutional neural networks are compared to common convolution kernel nerve above-mentioned Network can largely reduce the complexity of convolution process, and further promote the overall performance of convolutional neural networks.

It may also include sequence signature division module in Car license recognition module 930, be configured to divide character string spy according to column Sign, obtains the character string feature of multiple column, which will be during image characteristics extraction, and convolutional neural networks are obtained Character string feature is divided according to different column.

Further, it may also include sequence signature processing module 933 in Car license recognition module 930, be configured to by following Ring neural network handles the character string feature of the multiple column, obtains license plate number.Such as the word for being inputted Sequence signature is accorded with, via the processing of neural network, its character string feature is identified as background or character by setting, defeated as a result Out.If it is blank or meaningless lines, it is identified as background, if it is character, identifies that respective symbols export.Herein Recognition with Recurrent Neural Network can include but is not limited to shot and long term memory network (LSTM), two-way shot and long term memory network (Bi-LSTM), Deep layer Recognition with Recurrent Neural Network (DRNN) and its with other neural networks are compound is formed by complex neural network, the present invention is not limited to The type of used Recognition with Recurrent Neural Network.

Figure 10 shows the exemplary block diagram of car license recognition equipment according to an embodiment of the present disclosure.

Car license recognition equipment 950 as shown in Figure 10 can be implemented as one or more dedicated or general computer systems Module or component, such as PC, laptop, tablet computer, mobile phone, personal digital assistant (personal Digital assistance, PDA), smart glasses, smart watches, intelligent finger ring, intelligent helmet and any intelligent and portable set It is standby.Wherein, car license recognition equipment 950 may include at least one processor 960 and memory 970.

Wherein, at least one described processor is for executing program instructions.The memory 970 is in car license recognition equipment In 950 can program storage unit in different forms and data storage element exist, such as hard disk, read-only memory (ROM), random access memory (RAM) it can be used to storage processor processing and/or executes Car license recognition use in the process Various data files and processor performed by possible program instruction.Although being not shown, Car license recognition is set Standby 950 can also include an input output assembly, support car license recognition equipment 950 and other assemblies (such as image capture device 980) input/output data stream between.Car license recognition equipment 950 can also send and receive letter from network by communication port Breath and data.

In some embodiments, one group of instruction that the memory 970 is stored by the processor 960 execute when, The car license recognition equipment 950 is set to execute operation, the operation includes: to pre-process to vehicle image, obtains vehicle to be identified The license plate type of board image and the license plate image to be identified；It is more in the license plate type instruction of the license plate image to be identified It drives a vehicle in the case where board, type conversion is carried out for the license plate image to be identified of the multirow license plate type, obtains uniline license plate The license plate image of type；And the license plate image of the uniline license plate type is identified, obtain license plate number.

In some embodiments, the multirow license plate is duplicate rows license plate, and the car license recognition equipment 950 executes: based on double The dimensions of driving board, determines the position of upper and lower level cut-off rule on the license plate image to be identified of duplicate rows license plate type；Along institute The license plate image to be identified of duplicate rows license plate type described in determining cut-off rule cutting, obtains upper layer license plate image and lower layer's license plate figure Picture；Size change over is carried out to the upper layer license plate image, the upper layer license plate image after obtaining size change over；And by size change over Upper layer license plate image and lower layer's license plate image afterwards carries out horizontal direction splicing, the license plate of the uniline license plate type after being converted Image.

In some embodiments, car license recognition equipment 950 can receive the image outside the car license recognition equipment 950 Equipment vehicle image collected is acquired, and in the above-described licence plate recognition method of received image data execution, realization The function of the license plate recognition device of text description.

It can be, for example, road camera or monitoring device that described image, which acquires equipment,.

Although processor 960, memory 970 are rendered as individual module in Figure 10, those skilled in the art can be managed Solution, above equipment module may be implemented as individual hardware device, can also be integrated into one or more hardware devices.Only It can be realized the principle of disclosure description, the specific implementation of different hardware devices should not be used as limitation disclosure protection The factor of range.

According to another aspect of the present disclosure, a kind of non-volatile computer readable storage medium is additionally provided, is deposited thereon Computer-readable instruction is contained, foregoing method can be executed when executing described instruction using computer.

It is existing in the form of program part in technology is considered code and/or related data can be performed " product " or " product ", is participated in or is realized by computer-readable medium.Tangible, permanent storage medium can wrap Include memory or memory used in any computer, processor or similar devices or relevant module.For example, various partly lead Body memory, tape drive, disc driver can provide the equipment of store function similar to any for software.

All softwares or in which a part there may come a time when to be communicated by network, such as internet or other communication networks Network.Software can be loaded into another from a computer equipment or processor by such communication.Such as: from car license recognition equipment A server or host computer be loaded onto a computer environment hardware platform or other realize systems computer Environment, or the system of similar functions relevant to information required for Car license recognition is provided.Therefore, another kind can transmit software The medium of element is also used as physical connection between local devices, such as light wave, electric wave, electromagnetic wave etc., by cable, The realizations such as optical cable or air are propagated.It, can also for the physical medium such as similar devices such as cable, wireless connection or optical cable of carrier wave To be considered as the medium for carrying software.Unless limiting tangible " storage " medium, other indicate to calculate usage herein The term of machine or machine " readable medium " all indicates the medium participated in during processor executes any instruction.

The application has used particular words to describe embodiments herein.Such as " first/second embodiment ", " one implements Example ", and/or " some embodiments " mean a certain feature relevant at least one embodiment of the application, structure or feature.Cause This, it should be highlighted that and it is noted that " embodiment " or " an implementation referred to twice or repeatedly in this specification in different location Example " or " alternate embodiment " are not necessarily meant to refer to the same embodiment.In addition, in one or more embodiments of the application Certain features, structure or feature can carry out combination appropriate.

In addition, it will be understood by those skilled in the art that the various aspects of the application can be by several with patentability Type or situation are illustrated and described, the combination or right including any new and useful process, machine, product or substance Their any new and useful improvement.Correspondingly, the various aspects of the application can completely by hardware execute, can be complete It is executed, can also be executed by combination of hardware by software (including firmware, resident software, microcode etc.).Hardware above is soft Part is referred to alternatively as " data block ", " module ", " engine ", " unit ", " component " or " system ".In addition, the various aspects of the application The computer product being located in one or more computer-readable mediums may be shown as, which includes computer-readable program Coding.

Unless otherwise defined, all terms (including technical and scientific term) used herein have leads with belonging to the present invention The identical meanings that the those of ordinary skill in domain is commonly understood by.It is also understood that those of definition term such as in usual dictionary The meaning consistent with their meanings in the context of the relevant technologies should be interpreted as having, without application idealization or The meaning of extremely formalization explains, unless being clearly defined herein.

The above is the description of the invention, and is not considered as limitation ot it.Notwithstanding of the invention several Exemplary embodiment, but those skilled in the art will readily appreciate that, before without departing substantially from teaching and advantage of the invention Many modifications can be carried out to exemplary embodiment by putting.Therefore, all such modifications are intended to be included in claims institute In the scope of the invention of restriction.It should be appreciated that being the description of the invention above, and it should not be considered limited to disclosed spy Determine embodiment, and the model in the appended claims is intended to encompass to the modification of the disclosed embodiments and other embodiments In enclosing.The present invention is limited by claims and its equivalent.

Claims

1. a kind of licence plate recognition method, comprising:

Vehicle image is pre-processed, the license plate type of license plate image to be identified and the license plate image to be identified is obtained；

In the case where the license plate type of the license plate image to be identified indicates multirow license plate, for the multirow license plate type License plate image to be identified carries out type conversion, obtains the license plate image of uniline license plate type；And

The license plate image of the uniline license plate type is identified, license plate number is obtained.

2. licence plate recognition method as described in claim 1, wherein the multirow license plate is duplicate rows license plate, to multirow license plate class The license plate image to be identified of type carries out type and is converted to the license plate image of uniline license plate type

Based on the dimensions of duplicate rows license plate, upper and lower level cut-off rule is determined on the license plate image to be identified of duplicate rows license plate type Position；

Along the license plate image to be identified of duplicate rows license plate type described in identified cut-off rule cutting, upper layer license plate image is obtained under Layer license plate image；

Size change over is carried out to the upper layer license plate image, the upper layer license plate image after obtaining size change over；And

Upper layer license plate image after size change over is subjected to horizontal direction splicing with lower layer's license plate image, the uniline after being converted The license plate image of license plate type.

3. licence plate recognition method as described in claim 1, wherein pre-processed to vehicle image, obtain license plate to be identified The license plate type of image and the license plate image to be identified includes:

Characteristics of image is extracted for vehicle image via the first convolutional neural networks, obtains license plate image to be identified, and via Second convolutional neural networks extract characteristics of image for vehicle image, obtain the license plate type of the license plate image to be identified；Or Person

Characteristics of image is extracted for vehicle image via third convolutional neural networks, obtains license plate image to be identified and described The license plate type of license plate image to be identified.

4. licence plate recognition method as described in claim 1, wherein pre-processed to vehicle image further include:

Determine license plate sloped degree in the license plate image to be identified；And

In the case where the license plate sloped degree meets predetermined condition, Slant Rectify is carried out to the license plate image to be identified.

5. licence plate recognition method as claimed in claim 4, wherein carry out Slant Rectify packet to the license plate image to be identified It includes:

The characteristics of image of the license plate image to be identified is extracted by convolutional neural networks；

Extracted characteristics of image is handled by fully-connected network, obtains the position of current license plate key point；And

Based on the position of current license plate key point, slant correction is carried out to the license plate image to be identified；

Wherein, the license plate key point is four angle points of license plate, the position of license plate key point by license plate key point relative to The offset of pre-set image origin indicates.

6. licence plate recognition method as described in claim 1, wherein identified to the license plate image of the uniline license plate type Include: to obtain license plate number

License plate image feature is extracted from the license plate image of the uniline license plate type by convolutional neural networks；

Character string feature is divided according to column, obtains the character string feature of multiple column；And

It is handled by character string feature of the Recognition with Recurrent Neural Network to the multiple column, obtains license plate number.

7. licence plate recognition method as claimed in claim 6, wherein by Recognition with Recurrent Neural Network to the character sequence of the multiple column Column feature is handled, and is obtained license plate number and is included:

The compound characteristics vector is sent to the input terminal of neural network；

It is handled via character string feature of the neural network for the multiple column, by the character string feature in each column It is determined as background or respective symbols；And

Based on differentiation as a result, obtaining license plate number.

8. a kind of license plate recognition device, comprising:

Preprocessing module is configured as pre-processing vehicle image, obtains license plate image to be identified and described to be identified The license plate type of license plate image；

Type conversion module is configured as in the case where the license plate type of the license plate image to be identified indicates multirow license plate, Type conversion is carried out for the license plate image to be identified of the multirow license plate type, obtains the license plate image of uniline license plate type； And

Car license recognition module is configured as identifying the license plate image of the uniline license plate type, obtains license plate number.

9. license plate recognition device as claimed in claim 8, wherein the multirow license plate is duplicate rows license plate, the type conversion Module includes:

License Plate Segmentation module is configured as the dimensions based on duplicate rows license plate, in the license plate figure to be identified of duplicate rows license plate type As upper determining upper and lower level cut-off rule position and along the license plate to be identified of duplicate rows license plate type described in identified cut-off rule cutting Image obtains upper layer license plate image and lower layer's license plate image；

Size change over module is configured as carrying out size change over to the upper layer license plate image, the upper layer after obtaining size change over License plate image；And

License plate splicing module is configured as the upper layer license plate image and lower layer's license plate image progress horizontal direction after size change over Splicing, the license plate image of the uniline license plate type after being converted.

10. license plate recognition device as claimed in claim 8, wherein the Car license recognition module includes:

License plate image characteristic extracting module is configured as the license plate image by convolutional neural networks from the uniline license plate type Middle extraction license plate image feature；

Sequence signature division module is configured as dividing character string feature according to column, obtains the character string feature of multiple column； And

Sequence signature processing module is configured as through Recognition with Recurrent Neural Network to the character string feature of the multiple column Reason, obtains license plate number.

11. license plate recognition device as claimed in claim 8, wherein the preprocessing module further include:

Slant Rectify module is configured to determine that license plate sloped degree in the license plate image to be identified, and inclines in the license plate In the case that gradient meets predetermined condition, Slant Rectify is carried out to the license plate image to be identified.

12. a kind of car license recognition equipment, wherein the equipment includes processor and memory, the memory includes one group and refers to It enables, one group of instruction makes the car license recognition equipment execute operation when being executed by the processor, and the operation includes:

In the case where the license plate type of the license plate image to be identified indicates multirow license plate, for the multirow license plate type License plate image to be identified carries out type conversion, obtains the license plate image of uniline license plate type；

13. car license recognition equipment as claimed in claim 12, wherein the multirow license plate is duplicate rows license plate, one group of instruction The car license recognition equipment is set to execute following operation to obtain the license plate figure of uniline license plate type when being executed by the processor Picture:

Size change over is carried out to the upper layer license plate image, the upper layer license plate image after obtaining size change over；

14. car license recognition equipment as claimed in claim 12, wherein one group of instruction makes when being executed by the processor The car license recognition equipment executes following operation to obtain license plate number:

15. a kind of computer readable storage medium, which is characterized in that be stored thereon with computer-readable instruction, counted when utilizing Calculation machine executes method described in any one of the claims 1-7 when executing described instruction.