CN105117724A

CN105117724A - License plate positioning method and apparatus

Info

Publication number: CN105117724A
Application number: CN201510458607.9A
Authority: CN
Inventors: 马华东; 傅慧源; 周沫; 车广富
Original assignee: Beijing University of Posts and Telecommunications
Current assignee: Beijing University of Posts and Telecommunications
Priority date: 2015-07-30
Filing date: 2015-07-30
Publication date: 2015-12-02
Anticipated expiration: 2035-07-30
Also published as: CN105117724B

Abstract

The invention provides a license plate positioning method and apparatus. The method comprises the following steps of inputting the Haar feature of a video frame image to a first classifier to acquire a first type license plate image information sequence; calculating and screening the maximum value of first type license plate similarity values and judging whether the maximum value is greater than a preset first similarity threshold; if yes, positioning at least one effective license plate image, otherwise rotating the video frame image and inputting the rotated image to the first classifier to acquire multiple second type license plate image information sequences; calculating and screening the maximum second type license plate similarity value and judging whether the maximum value is greater than a preset second similarity threshold; if yes, positioning at least one effective license plate image, otherwise inputting the LBP feature of the video frame image to a second classifier to acquire a third type license plate image information sequence; and positioning at least one effective license plate image on the basis of the maximum value of the third type license plate similarity values. By means of the method, the license plate positioning accuracy can be improved.

Description

A kind of license plate locating method and device

Technical field

The present invention relates to technical field of computer vision, particularly relate to a kind of license plate locating method and device.

Background technology

Along with Digital Image Processing, the reaching its maturity of pattern-recognition and artificial intelligence technology, intelligent transportation system (ITS) has become the trend of 21 century Transportation Development gradually.License plate recognition technology is the basis realizing intelligent transportation system, and License Plate is link very important in Car license recognition.In the complex scene of the non-traffic block ports such as community, campus, crossroad or multilane crossing, license plate area in video frame images captured by monitoring camera has obvious difference usually in position, angle, size sharpness, in addition, the background area in the video frame images taken under non-traffic block port is mixed and disorderly.

In the vehicle license location technique of current existing non-traffic block port, certain feature in video frame images captured under usually extracting non-traffic block port, such as, using the feature of the single features such as gray-scale value or color as positioning licence plate.

Obviously, these unfavorable factors of license plate area in video frame images captured under non-traffic block port increase difficulty to the accurate positioning licence plate in later stage, therefore, when adopting certain single features to carry out positioning licence plate in above-mentioned prior art, usually occur locating unsuccessfully or undetected situation.

Summary of the invention

The object of the embodiment of the present invention is to provide a kind of license plate locating method and device, to realize accurate positioning licence plate, reduces and locates unsuccessfully or undetected situation.

For achieving the above object, the embodiment of the invention discloses a kind of license plate locating method, described method comprises:

Extract the Ha Er Haar feature of the video frame images of car plate to be positioned, and described Haar feature is inputed in the first sorter set up in advance, obtain the first kind license plate image information sequence corresponding with described video frame images, wherein, the positional information of the first kind candidate license plate image searched in video frame images is comprised in described first kind license plate image information sequence, described first sorter is: based on the Ha Er Haar feature for each image in the first predetermined sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained,

Based on the positional information of described first kind candidate license plate image, determine the first kind candidate license plate image in described video frame images;

Calculate the first kind car plate similar value of described first kind candidate license plate image, and filter out the maximal value in first kind car plate similar value;

Judge whether the maximal value in the first kind car plate similar value that screening obtains is greater than the first default similarity threshold:

When judged result is for being greater than, according to the first effective car plate locating rule preset and the first kind car plate similar value calculated, at least one effective license plate image is navigated to from described first kind candidate license plate image, when judged result is for being not more than, according to the image rotation rule preset, carry out N time to described frequency two field picture to rotate, the video frame images rotating rear gained is each time inputed to again in described first sorter, obtain N number of Equations of The Second Kind license plate image information sequence, based on the positional information of the Equations of The Second Kind candidate license plate image in described Equations of The Second Kind license plate image information sequence, determine the Equations of The Second Kind candidate license plate image in described video frame images, calculate the Equations of The Second Kind car plate similar value of each Equations of The Second Kind candidate license plate image corresponding to Equations of The Second Kind license plate image information sequence, and the maximal value filtered out in the Equations of The Second Kind car plate similar value of each Equations of The Second Kind candidate license plate image corresponding to Equations of The Second Kind license plate image information sequence, and then, in maximal value from each Equations of The Second Kind car plate similar value that screening obtains, determine the target Equations of The Second Kind car plate similar value that numerical value is maximum, judge whether described target Equations of The Second Kind car plate similar value is greater than the second default similarity threshold, if be greater than, according to the second effective car plate locating rule preset and the Equations of The Second Kind car plate similar value calculated, from the Equations of The Second Kind license plate image information sequence corresponding to described target Equations of The Second Kind car plate similar value for Equations of The Second Kind candidate license plate image navigate at least one effective license plate image, if be not more than, then

Continue the local binary patterns LBP feature extracting described video frame images, and described LBP feature is inputed in the second sorter set up in advance, obtain the three class license plate image information sequence corresponding with described video frame images, wherein, the positional information of the 3rd class candidate license plate image searched in video frame images is comprised in described 3rd class license plate image information sequence, described second sorter is: based on the local binary patterns LBP feature for each image in the second predetermined sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained,

Based on the positional information of described 3rd class candidate license plate image, determine the 3rd class candidate license plate image in described video frame images;

Calculate the 3rd class car plate similar value of described 3rd class candidate license plate image, and filter out the maximal value in the 3rd class car plate similar value;

Whether the maximal value judging to screen the 3rd class car plate similar value obtained is greater than default third phase is seemingly spent threshold value; When judged result is for being greater than, according to the 3rd effective car plate locating rule preset and the 3rd class car plate similar value calculated, at least one effective license plate image is navigated to from described 3rd class candidate license plate image, when judged result is for being not more than, determine by described first sorter and the second sorter no-fix to effective license plate image.

Preferably, calculate the process of the first kind car plate similar value of arbitrary first kind candidate license plate image, comprising:

Binary conversion treatment is carried out to first kind candidate license plate image;

Opening operation morphological transformation is carried out to the first kind candidate license plate image obtained after binary conversion treatment;

By Sobel algorithm, vertical filtering process is carried out to the first kind candidate license plate image after opening operation morphological transformation, obtain the first kind VG (vertical gradient) image of the first kind candidate license plate image after opening operation morphological transformation;

By Canny algorithm, rim detection is carried out to described first kind VG (vertical gradient) image, obtain first kind edge image;

Carry out six deciles for the edge extracted in described first kind edge image according to vertical direction, calculate pixel saltus step average in the horizontal direction in marginal portion in each decile;

Calculate described saltus step mean of mean, the described mean value calculated is defined as the first kind car plate similar value of described first kind candidate license plate image.

Preferably, first effective car plate locating rule that described basis is preset and the first kind car plate similar value calculated, navigate at least one effective license plate image, comprising from described first kind candidate license plate image:

First kind candidate license plate image corresponding to maximal value in described first kind car plate similar value is defined as the effective license plate image navigated to;

Or

By in described first kind car plate similar value, the first kind candidate license plate image corresponding to first kind car plate similar value being greater than the first default locating threshold is defined as navigating to effective license plate image.

Preferably, describedly determine that described method also comprises by after described first sorter and the second sorter no-fix to effective license plate image:

Described Haar feature is inputed in the 3rd sorter set up in advance, obtain the four class license plate image information sequence corresponding with described video frame images, wherein, the positional information of the 4th class candidate license plate image searched in video frame images is comprised in described 4th class license plate image information sequence, described 3rd sorter is: based on the Ha Er Haar feature for each image in the 3rd predetermined sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained;

Based on the positional information of described 4th class candidate license plate image, determine the 4th class candidate license plate image in described video frame images;

Calculate the 4th class car plate similar value of described 4th class candidate license plate image, and filter out the maximal value in the 4th class car plate similar value;

Judge whether the maximal value in the 4th class car plate similar value that screening obtains is greater than the 4th default similarity threshold; When judged result is for being greater than, according to the 4th effective car plate locating rule preset and the 4th class car plate similar value calculated, at least one effective license plate image is navigated to from described 4th class candidate license plate image, when judged result is for being not more than, determine by described first sorter, the second sorter and the 3rd sorter no-fix to effective license plate image.

Preferably, describedly determine that described method also comprises by after described first sorter, the second sorter and the 3rd sorter no-fix to effective license plate image:

Described Haar feature is inputed in the 4th sorter set up in advance, obtain the five class license plate image information sequence corresponding with described video frame images, wherein, the positional information of the 5th class candidate license plate image searched in video frame images is comprised in described 5th class license plate image information sequence, described 4th sorter is: based on the Ha Er Haar feature for each image in the 4th sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained;

Based on the positional information of described 5th class candidate license plate image, determine the 5th class candidate license plate image in described video frame images;

Calculate the 5th class car plate similar value of described 5th class candidate license plate image, and filter out the maximal value in the 5th class car plate similar value;

Judge whether the maximal value in the 5th class car plate similar value that screening obtains is greater than the 5th default similarity threshold; When judged result is for being greater than, according to the 5th effective car plate locating rule preset and the 5th class car plate similar value calculated, at least one effective license plate image is navigated to from described 5th class candidate license plate image, when judged result is for being not more than, determine that no-fix arrives effective license plate image.

For achieving the above object, the embodiment of the invention discloses a kind of license plate positioning device, described device comprises:

First car plate sequence obtains module, for extracting the Ha Er Haar feature of the video frame images of car plate to be positioned, and described Haar feature is inputed in the first sorter set up in advance, obtain the first kind license plate image information sequence corresponding with described video frame images, wherein, the positional information of the first kind candidate license plate image searched in video frame images is comprised in described first kind license plate image information sequence, described first sorter is: based on the Ha Er Haar feature for each image in the first predetermined sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained,

First license plate image determination module, for the positional information based on described first kind candidate license plate image, determines the first kind candidate license plate image in described video frame images;

First car plate similarity calculation module, for calculating the first kind car plate similar value of described first kind candidate license plate image, and filters out the maximal value in first kind car plate similar value;

Whether the first car plate similarity judge module, be greater than the first default similarity threshold for the maximal value judging to screen in the first kind car plate similar value that obtains; When judged result is for being greater than, according to the first effective car plate locating rule preset and the first kind car plate similar value calculated, at least one effective license plate image is navigated to from described first kind candidate license plate image, when judged result is for being not more than, triggers the second car plate sequence and obtaining module;

Described second car plate sequence obtains module, for regular according to the image rotation preset, carry out N time to described frequency two field picture to rotate, the video frame images rotating rear gained is each time inputed in described first sorter again, obtains N number of Equations of The Second Kind license plate image information sequence;

Second license plate image determination module, for the positional information based on the Equations of The Second Kind candidate license plate image in described Equations of The Second Kind license plate image information sequence, determines the Equations of The Second Kind candidate license plate image in described video frame images;

Second car plate similarity calculation module, for calculating the Equations of The Second Kind car plate similar value of the Equations of The Second Kind candidate license plate image corresponding to each Equations of The Second Kind license plate image information sequence, and the maximal value filtered out in the Equations of The Second Kind car plate similar value of each Equations of The Second Kind candidate license plate image corresponding to Equations of The Second Kind license plate image information sequence, and then, in maximal value from each Equations of The Second Kind car plate similar value that screening obtains, determine the target Equations of The Second Kind car plate similar value that numerical value is maximum;

Second car plate similarity judge module, for judging whether described target Equations of The Second Kind car plate similar value is greater than the second default similarity threshold, if be greater than, according to the second effective car plate locating rule preset and the Equations of The Second Kind car plate similar value calculated, from the Equations of The Second Kind license plate image information sequence corresponding to described target Equations of The Second Kind car plate similar value for Equations of The Second Kind candidate license plate image navigate at least one effective license plate image, if be not more than, then triggered the 3rd car plate sequence and obtain module;

Described 3rd car plate sequence obtains module, for continuing the local binary patterns LBP feature extracting described video frame images, and described LBP feature is inputed in the second sorter set up in advance, obtain the three class license plate image information sequence corresponding with described video frame images, wherein, the positional information of the 3rd class candidate license plate image searched in video frame images is comprised in described 3rd class license plate image information sequence, described second sorter is: based on the local binary patterns LBP feature for each image in the second predetermined sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained,

3rd license plate image determination module, for the positional information based on described 3rd class candidate license plate image, determines the 3rd class candidate license plate image in described video frame images;

3rd car plate similarity calculation module, for calculating the 3rd class car plate similar value of described 3rd class candidate license plate image, and filters out the maximal value in the 3rd class car plate similar value;

3rd car plate similarity judge module, whether the maximal value for judging to screen the 3rd class car plate similar value obtained is greater than default third phase is seemingly spent threshold value; When judged result is for being greater than, according to the 3rd effective car plate locating rule preset and the 3rd class car plate similar value calculated, at least one effective license plate image is navigated to from described 3rd class candidate license plate image, when judged result is for being not more than, determine by described first sorter and the second sorter no-fix to effective license plate image.

Preferably, described first car plate similarity calculation module, for:

Preferably, described first car plate similarity judge module comprises the first License Plate submodule for navigating at least one effective license plate image from described first kind candidate license plate image according to the first effective car plate locating rule preset and the first kind car plate similar value calculated;

Described first License Plate submodule, specifically for:

First kind candidate license plate image corresponding to maximal value in described first kind car plate similar value is defined as the effective license plate image navigated to; Or by described first kind car plate similar value, the first kind candidate license plate image corresponding to first kind car plate similar value being greater than the first default locating threshold is defined as navigating to effective license plate image.

Preferably, described device also comprises:

4th car plate sequence obtains module, for described Haar feature being inputed in the 3rd sorter set up in advance, obtain the four class license plate image information sequence corresponding with described video frame images, wherein, the positional information of the 4th class candidate license plate image searched in video frame images is comprised in described 4th class license plate image information sequence, described 3rd sorter is: based on the Ha Er Haar feature for each image in the 3rd predetermined sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained;

4th license plate image determination module, for the positional information based on described 4th class candidate license plate image, determines the 4th class candidate license plate image in described video frame images;

4th car plate similarity calculation module, for calculating the 4th class car plate similar value of described 4th class candidate license plate image, and filters out the maximal value in the 4th class car plate similar value;

Whether the 4th car plate similarity judge module, be greater than the 4th default similarity threshold for the maximal value judging to screen in the 4th class car plate similar value that obtains; When judged result is for being greater than, according to the 4th effective car plate locating rule preset and the 4th class car plate similar value calculated, at least one effective license plate image is navigated to from described 4th class candidate license plate image, when judged result is for being not more than, determine by described first sorter, the second sorter and the 3rd sorter no-fix to effective license plate image.

Preferably, described device also comprises:

5th car plate sequence obtains module, for described Haar feature being inputed in the 4th sorter set up in advance, obtain the five class license plate image information sequence corresponding with described video frame images, wherein, the positional information of the 5th class candidate license plate image searched in video frame images is comprised in described 5th class license plate image information sequence, described 4th sorter is: based on the Ha Er Haar feature for each image in the 4th sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained;

5th license plate image determination module, for the positional information based on described 5th class candidate license plate image, determines the 5th class candidate license plate image in described video frame images;

5th car plate similarity calculation module, for calculating the 5th class car plate similar value of described 5th class candidate license plate image, and filters out the maximal value in the 5th class car plate similar value;

Whether the 5th car plate similarity judge module, be greater than the 5th default similarity threshold for the maximal value judging to screen in the 5th class car plate similar value that obtains; When judged result is for being greater than, according to the 5th effective car plate locating rule preset and the 5th class car plate similar value calculated, at least one effective license plate image is navigated to from described 5th class candidate license plate image, when judged result is for being not more than, determine that no-fix arrives effective license plate image.

A kind of license plate locating method that the embodiment of the present invention provides and installation method and device, by the sorter of multiple series connection of design multistage screening mechanism, extract the comprehensive characteristics of multiple structural feature of the image in the video frame images taken by camera, substantially increase the License Plate failure or undetected situation that easily occur when utilizing single features positioning licence plate, simultaneously, when after the Ha Er Haar feature that video frame images input is extracted inputs to the first sorter, no-fix is to the candidate license plate image satisfied condition, further multiple rotary is carried out to this video frame images, postrotational image is inputed to the first sorter again, the location failure or undetected situation that bring because of image taking angle can be improved like this, obviously, by the search of sorter layer by layer to the candidate license plate image in this video frame images, improve the possibility of accurate positioning licence plate, reduce and locate unsuccessfully or the probability of undetected situation.Certainly, arbitrary product of the present invention is implemented or method must not necessarily need to reach above-described all advantages simultaneously.

Figure explanation

In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.

The schematic flow sheet of a kind of license plate locating method that Fig. 1 provides for the embodiment of the present invention;

Effective license plate image that two width that Fig. 2 provides for the embodiment of the present invention navigate to;

The schematic flow sheet of the another kind of license plate locating method that Fig. 3 provides for the embodiment of the present invention;

The schematic flow sheet of the another kind of license plate locating method that Fig. 4 provides for the embodiment of the present invention;

Result schematic diagram after the Hough transformation that Fig. 5 (a) and Fig. 5 (b) provide for the embodiment of the present invention;

Result schematic diagram after the impurity elimination binary conversion treatment that Fig. 6 (a)-Fig. 6 (d) provides for the embodiment of the present invention;

The structural representation of a kind of license plate positioning device that Fig. 7 provides for the embodiment of the present invention.

Embodiment

Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.

The schematic flow sheet of a kind of license plate locating method that Fig. 1 provides for the embodiment of the present invention, the method can comprise the following steps:

Step S101: the Ha Er Haar feature extracting the video frame images of car plate to be positioned, and described Haar feature is inputed in the first sorter set up in advance, obtain the first kind license plate image information sequence corresponding with described video frame images.

Wherein, the positional information of the first kind candidate license plate image searched in video frame images is comprised in first kind license plate image information sequence.

The image searched in the video frame images that " candidate license plate image " in the present invention obtains for sorter from monitor video, it should be noted that, be likely the image of car plate in candidate license plate image, is not also likely the image of car plate.In addition, if the candidate license plate image navigated to through subsequent step is the image of car plate, illustrate that sorter location is correct; If through subsequent step no-fix to candidate license plate image, and when not being the image of car plate really in this candidate license plate image searched out, also illustrate that sorter location is correct, otherwise, show that sorter is located unsuccessfully.

It should be noted that, in " the license plate image information sequence " mentioned in the present invention, comprise the positional information of the candidate license plate image searched in video frame images, such as, the image-region represented by pixel coordinate that license plate image is residing in video frame images.In addition, the positional information of the whole candidate license plate image searched by sorter is contained in license plate image information sequence.

Wherein, the first sorter is: based on the Ha Er Haar feature for each image in the first predetermined sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained.

In a kind of specific implementation of the present invention, the first sorter can be set up in the following way: first, choose the positive negative sample for training classifier, wherein, license plate image (positive sample) 1000 width, non-license plate image (negative sample) 10000 width; Then, the Haar feature of every width image in positive negative sample is proposed; Finally, input in the Adaboost sorter training program that OpenCV carries run extracting the Haar feature that obtains, the parameter such as false alarm rate (that is: non-license plate image being positioned to license plate image), hit rate according to set up sorter optimizes set up sorter.

It should be noted that, Adaboost sorter can be used to carry out off-line training for a kind of feature of license plate image, acquisition can the sorter of positioning licence plate, and the sorter utilizing Adaboost algorithm to train has certain feasibility and reliability in License Plate.In addition, in the training or process of establishing of sorter, preferably make the sample image of training large as much as possible, sample image is diversified as much as possible, and like this, the efficiency of sorter set up in actual applications is better.

Also it should be noted that, the above mentioned sorter trained based on Adaboost algorithm is only citing, the present invention does not need the specific algorithm to setting up sorter to limit, certainly other can also be had to set up the mode of sorter, such as, based on the sorter method for building up of support vector machines, any possible implementation all can be applied to the present invention, and those skilled in the art need to determine specifically to adopt which kind of mode to set up sorter according to actual conditions.

Step S102: based on the positional information of described first kind candidate license plate image, determines the first kind candidate license plate image in described video frame images.

Step S103: the first kind car plate similar value calculating described first kind candidate license plate image, and filter out the maximal value in first kind car plate similar value.

In a kind of specific implementation of the present invention, calculate the process of the first kind car plate similar value of arbitrary first kind candidate license plate image, can comprise the following steps:

A: binary conversion treatment is carried out to first kind candidate license plate image;

B: opening operation morphological transformation is carried out to the first kind candidate license plate image obtained after binary conversion treatment;

C: carry out vertical filtering process to the first kind candidate license plate image after opening operation morphological transformation by Sobel algorithm, obtains the first kind VG (vertical gradient) image of the first kind candidate license plate image after opening operation morphological transformation;

D: carry out rim detection to described first kind VG (vertical gradient) image by Canny algorithm, obtains first kind edge image;

E: carry out six deciles according to vertical direction for the edge extracted in described first kind edge image, calculates pixel saltus step average in the horizontal direction in marginal portion in each decile;

Wherein, owing to being detected by step D in the edge image that obtains, the pixel value of the pixel that marginal portion is is 255, and the pixel value of the pixel of non-edge part is 0.Therefore, when the arbitrary height obtained after step e six decile, according to during from head to this row pixel of the order traversal of tail, just there will be the checker of black-white point, a saltus step is just designated as when the pixel value of continuous two pixels is different, add up this row transition times, just can obtain pixel saltus step average in the horizontal direction in marginal portion in each decile.

F: calculate described saltus step mean of mean, is defined as the first kind car plate similar value of described first kind candidate license plate image by the described mean value calculated.

It should be noted that, the process of the first kind car plate similar value of above-mentioned calculating arbitrary first kind candidate license plate image is only citing, the embodiment of the present invention does not need to limit the computation process of car plate similar value, and any possible implementation all can be applied to the present invention.

Step S104: judge whether the maximal value in the first kind car plate similar value that screening obtains is greater than the first default similarity threshold, when sentence read result is for being greater than, perform step S105, otherwise perform step S106.

Preferably, the first similarity threshold can comprise: for single car plate, and this threshold value is set to 13; For double car plate, this threshold value is set to 12.It should be noted that, here set threshold value is only a kind of optimal way of the present invention, the present invention does not need to limit the concrete numerical value of the first similarity threshold, as choosing which kind of threshold value, needs those skilled in the art to determine according to actual conditions.

Step S105: according to the first effective car plate locating rule preset and the first kind car plate similar value calculated, at least one effective license plate image is navigated to from described first kind candidate license plate image, see effective license plate image that Fig. 2, Fig. 2 navigate to for two width that the embodiment of the present invention provides.

In a kind of specific implementation of the present invention, the first kind candidate license plate image corresponding to the maximal value in first kind car plate similar value can be defined as the effective license plate image navigated to.

In another kind of specific implementation of the present invention, can by described first kind car plate similar value, the first kind candidate license plate image corresponding to first kind car plate similar value being greater than the first locating threshold is defined as navigating to effective license plate image.

Concrete, the first locating threshold preset can be set to a certain multiple of the maximal value max in first kind car plate similar value, such as: max*70%.Certainly, the present invention does not need specifically to limit this multiple, and those skilled in the art need to determine according to practical situations.

For example, suppose: first kind car plate similar value is arranged respectively according to descending order: a=99%, b=95%, c=93%, d=89% and e=85%, the first default locating threshold is 90%.

Obviously, according to the method mentioned in the first implementation above-mentioned, exactly first kind candidate license plate image corresponding for a is defined as the effective license plate image navigated to;

Obviously, according to the method mentioned in above-mentioned the second implementation, first kind candidate license plate image corresponding to a, b and c exactly similar value being greater than 90% is defined as the effective license plate image navigated to.

It should be noted that, above-mentioned two kinds of cited specific implementations are only citings, the specific implementation that certain the present invention also has other possible, the present invention does not need to limit the specific implementation of the effective license plate image in location, and those skilled in the art need to determine according to the concrete condition in practical application.

Step S106: according to the image rotation rule preset, N time is carried out to described frequency two field picture and rotates, the video frame images rotating rear gained is each time inputed in described first sorter again, obtains N number of Equations of The Second Kind license plate image information sequence.

In a kind of specific implementation of the present invention, the image rotation rule preset can be: rotated according to clockwise direction in the scope of [-20 °, 20 °] by video frame images, and each rotation 2 °, obtain the video frame images that a width is new.

It should be noted that, because the angle of the license plate image in video frame images captured under non-traffic block port scene is very different, so there will be no-fix to license plate image or situation license plate image being judged into by accident non-license plate image in the position fixing process of license plate image, therefore, scheme in the present invention rotates video frame images in these cases, search out the angle of the most applicable location, to make the first sorter of setting up in advance for the excessive license plate image accurate positioning as much as possible in angle of inclination.Also it should be noted that, do not need in the present invention to limit the allowed band of video frame images rotation and the concrete numerical value of each rotated angle, those skilled in the art need reasonably to set according to the concrete condition in practical application.

Step S107: based on the positional information of the Equations of The Second Kind candidate license plate image in described Equations of The Second Kind license plate image information sequence, determines the Equations of The Second Kind candidate license plate image in described video frame images.

Step S108: the Equations of The Second Kind car plate similar value calculating each Equations of The Second Kind candidate license plate image corresponding to Equations of The Second Kind license plate image information sequence, and the maximal value filtered out in the Equations of The Second Kind car plate similar value of each Equations of The Second Kind candidate license plate image corresponding to Equations of The Second Kind license plate image information sequence, and then, in maximal value from each Equations of The Second Kind car plate similar value that screening obtains, determine the target Equations of The Second Kind car plate similar value that numerical value is maximum.

Step S109: judge whether described target Equations of The Second Kind car plate similar value is greater than the second default similarity threshold, if be greater than, performs step S110, otherwise performs step S11.

Step S110: according to the second effective car plate locating rule preset and the Equations of The Second Kind car plate similar value that calculates, from the Equations of The Second Kind license plate image information sequence corresponding to described target Equations of The Second Kind car plate similar value for Equations of The Second Kind candidate license plate image navigate at least one effective license plate image.

It should be noted that, the method for step S107-S110 and step S102-S105 is similar, repeats no more herein.

Step S111: continue the local binary patterns LBP feature extracting described video frame images, and described LBP feature is inputed in the second sorter set up in advance, obtain the three class license plate image information sequence corresponding with described video frame images.

Wherein, the positional information of the 3rd class candidate license plate image searched in video frame images is comprised in the 3rd class license plate image information sequence.

Wherein, the second sorter is: based on the local binary patterns LBP feature for each image in the second predetermined sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained.Compared with the first sorter, the sample image set selected when training or setting up this second sorter is different, and the feature of the sample image set of extracting is different.

It should be noted that, mention that the sorter trained based on Adaboost algorithm is only citing above, the present invention does not need the specific algorithm to setting up sorter to limit, certainly other can also be had to set up the mode of sorter, such as, based on the sorter method for building up of support vector machines, any possible implementation all can be applied to the present invention, and those skilled in the art need to determine specifically to adopt which kind of mode to set up sorter according to actual conditions.

Step S112: based on the positional information of described 3rd class candidate license plate image, determines the 3rd class candidate license plate image in described video frame images.

Step S113: the 3rd class car plate similar value calculating described 3rd class candidate license plate image, and filter out the maximal value in the 3rd class car plate similar value.

Step S114: whether the maximal value judging to screen the 3rd class car plate similar value obtained is greater than default third phase is seemingly spent threshold value; When judged result is for being greater than, according to the 3rd effective car plate locating rule preset and the 3rd class car plate similar value calculated, at least one effective license plate image is navigated to from described 3rd class candidate license plate image, when judged result is for being not more than, determine by described first sorter and the second sorter no-fix to effective license plate image.

It should be noted that, step S112-S114 and step S102-S104 is similar, repeats no more herein.

The embodiment of the present invention is by the sorter of multiple series connection of design multistage screening mechanism, extract the comprehensive characteristics of multiple structural feature of the image in the video frame images taken by camera, substantially increase the License Plate failure or undetected situation that easily occur when utilizing single features positioning licence plate, simultaneously, when after the Ha Er Haar feature that video frame images input is extracted inputs to the first sorter, no-fix is to the candidate license plate image satisfied condition, further multiple rotary is carried out to this video frame images, postrotational image is inputed to the first sorter again, the location failure or undetected situation that bring because of image taking angle can be improved like this, obviously, by the search of sorter layer by layer to the candidate license plate image in this video frame images, improve the possibility of accurate positioning licence plate, reduce and locate unsuccessfully or the probability of undetected situation.

See Fig. 3, in order to improve the accuracy of license plate image location, on the basis of the license plate locating method shown in Fig. 1, can also comprise the following steps:

Step S115: described Haar feature inputed in the 3rd sorter set up in advance, obtains the four class license plate image information sequence corresponding with described video frame images.

Wherein, the positional information of the 4th class candidate license plate image searched in video frame images is comprised in the 4th class license plate image information sequence.

Wherein, the 3rd sorter is: based on the Ha Er Haar feature for each image in the 3rd predetermined sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained.Compared with the first sorter, the sample image set selected when training or setting up the 3rd sorter is different, and the feature of the sample image set of extracting is identical.

In addition, in the training or process of establishing of sorter, preferably make the sample image of training large as much as possible, sample image is diversified as much as possible, and like this, the efficiency of sorter set up in actual applications is better.

In a kind of specific implementation of the present invention, the 3rd sorter can be set up in the following way: first, choose the positive negative sample for training classifier, wherein, license plate image (positive sample) 3000 width, non-license plate image (negative sample) 10000 width; Then, the Haar feature of every width image in positive negative sample is proposed; Finally, input in the Adaboost sorter training program that OpenCV carries run extracting the Haar feature that obtains, the parameter such as false alarm rate (that is: non-license plate image being positioned to license plate image), hit rate according to set up sorter optimizes set up sorter.

Step S116: based on the positional information of described 4th class candidate license plate image, determines the 4th class candidate license plate image in described video frame images.

Step S117: the 4th class car plate similar value calculating described 4th class candidate license plate image, and filter out the maximal value in the 4th class car plate similar value.

Step S118: judge whether the maximal value in the 4th class car plate similar value that screening obtains is greater than the 4th default similarity threshold; When judged result is for being greater than, according to the 4th effective car plate locating rule preset and the 4th class car plate similar value calculated, at least one effective license plate image is navigated to from described 4th class candidate license plate image, when judged result is for being not more than, determine by described first sorter, the second sorter and the 3rd sorter no-fix to effective license plate image.

It should be noted that, step S116-S118 and step S102-S104 is similar, repeats no more herein.

Further, see Fig. 4, on the basis of the license plate locating method shown in Fig. 3, can also comprise the following steps:

Step S119: described Haar feature inputed in the 4th sorter set up in advance, obtains the five class license plate image information sequence corresponding with described video frame images.

Wherein, the positional information of the 5th class candidate license plate image searched in video frame images is comprised in the 5th class license plate image information sequence.

Wherein, the 4th sorter is: based on the Ha Er Haar feature for each image in the 4th sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained.Compared with the first sorter, the sample image set selected when training or setting up the 4th sorter is different, and the feature of the sample image set of extracting is identical.

In addition, in the training or process of establishing of sorter, preferably make the quantity of the sample image of training large as much as possible, sample image is diversified as much as possible, and like this, the efficiency of sorter set up in actual applications is better.

In a kind of specific implementation of the present invention, the 4th sorter can be set up in the following way: first, choose the positive negative sample for training classifier, wherein, license plate image (positive sample) 2000 width, and the positive sample in this license plate image is all double license plate image, non-license plate image (negative sample) 10000 width; Then, the Haar feature of every width image in positive negative sample is proposed; Finally, input in the Adaboost sorter training program that OpenCV carries run extracting the Haar feature that obtains, the parameter such as false alarm rate (that is: non-license plate image being positioned to license plate image), hit rate according to set up sorter optimizes set up sorter.

Step S120: based on the positional information of described 5th class candidate license plate image, determines the 5th class candidate license plate image in described video frame images.

Step S121: the 5th class car plate similar value calculating described 5th class candidate license plate image, and filter out the maximal value in the 5th class car plate similar value.

Step S122: judge whether the maximal value in the 5th class car plate similar value that screening obtains is greater than the 5th default similarity threshold; When judged result is for being greater than, according to the 5th effective car plate locating rule preset and the 5th class car plate similar value calculated, at least one effective license plate image is navigated to from described 5th class candidate license plate image, when judged result is for being not more than, determine that no-fix arrives effective license plate image.

It should be noted that, step S120-S122 and step S102-S104 is similar, repeats no more herein.

In a kind of specific embodiment of the present invention, on the basis of the method for the License Plate shown in Fig. 1, thin localization process can also be carried out to the effective license plate image navigated to further, specifically can comprise the following steps:

(1) impurity elimination binary conversion treatment is carried out to the effective license plate image navigated to, only comprised effective license plate image of car plate part;

(2) Hough transformation Hough transform is utilized to carry out the rim detection of license plate area to the effective license plate image after impurity elimination, and extract the edge of this license plate area, result schematic diagram after the Hough transformation provided for the embodiment of the present invention see Fig. 5, Fig. 5 (a) and Fig. 5 (b);

(3) according to the edge of the license plate area extracted, license plate area is calculated at level and/or vertical direction pitch angle;

(4) based on the pitch angle calculated, level correction is carried out to the image of license plate area and mistake cuts correction process, obtain the license plate image after correcting;

(5) vertical gradient process is carried out to the license plate image in the vertical direction after obtained correction, determine the up-and-down boundary of the license plate image after correction, meanwhile, according to the right boundary more tentatively determining the license plate image after correction with predetermined threshold value;

(6) utilize flooding approach to remove edge fragment in four borders up and down determined, process is optimized to four borders up and down of determined license plate image.

Wherein, " impurity elimination binary conversion treatment " mentioned in the present invention, mainly can comprise the following steps:

1) judge whether the body color corresponding to this license plate image is light color according to the size of the pixel value sum of the pixel of the top area in the gray-scale map of the license plate image after widening.

In a kind of specific implementation of the present invention, will can determine that the license plate image on four borders is up and down widened the car plate height of 0.125 times respectively and widens the car plate width of 0.125 times in the lateral direction on upward and downward; Setting widen after the gray-scale value sum of the 0.125 times of height component image in license plate image top, if when to be greater than this area image size be all 50% of gray-scale value sum in Pure white pixels point 255 situation, be judged as light vehicle body, otherwise be judged as dark vehicle body.

2) according to obtained body color, binary conversion treatment is carried out to the effective license plate image corresponding to this license plate image, obtain binary image A, see Fig. 6 (a).

In a kind of concrete implementation of the present invention, can comprise:

In step 1) in when being judged as light vehicle body, choose the channel S of coloured image in hsv color space of the effective license plate image corresponding to this license plate image, binary conversion treatment carried out to the effective license plate image corresponding to this channel S;

In step 1) in when being judged as dark vehicle body, need to be handled as follows in conjunction with car plate color: if blue car plate, the effective license plate image chosen corresponding to the channel B in RGB color space carries out binary conversion treatment, if yellow car plate or white number plate, choose G, R passage merging treatment in RGB color space, and the effective license plate image after being combined carries out binary conversion treatment.

3) choose the b passage in the Lab color space of the coloured image of the effective license plate image corresponding to this license plate image, and carry out binary conversion treatment, obtain binary image B, see Fig. 6 (b).

4) by step 2) in the binary image A that obtains and step 3) in be in correspondence position in the binary image B that obtains pixel compare, if the pixel value of the pixel of correspondence position is consistent, then the pixel value of pixel corresponding in binary image A is set to 255.

5) calculation procedure 4) process after binary image A in the ratio shared by blank pixel point, when this ratio is greater than default blank threshold, determine to replace binary image A to carry out subsequent treatment with binary image B, see Fig. 6 (c), otherwise still carry out subsequent treatment with binary image A.

Preferably, the blank threshold preset can be set to 90%.Certainly, the present invention does not need to limit the concrete numerical value of this threshold value, and those skilled in the art need reasonably to arrange according to the concrete condition in practical application.

6) by step 5) in the binary image determined utilize flooding approach to remove the miscellaneous region of non-interconnected in this image, see Fig. 6 (d), complete the thin localization process to the effective license plate image navigated to.

The structural representation of a kind of license plate positioning device that Fig. 7 provides for the embodiment of the present invention, this device can comprise with lower module:

First car plate sequence obtains module 201, for extracting the Ha Er Haar feature of the video frame images of car plate to be positioned, and described Haar feature is inputed in the first sorter set up in advance, obtain the first kind license plate image information sequence corresponding with described video frame images.

First license plate image determination module 202, for the positional information based on described first kind candidate license plate image, determines the first kind candidate license plate image in described video frame images.

First car plate similarity calculation module 203, for calculating the first kind car plate similar value of described first kind candidate license plate image, and filters out the maximal value in first kind car plate similar value.

In a kind of specific implementation of the present invention, the first car plate similarity calculation module 203, may be used for:

Whether the first car plate similarity judge module 204, be greater than the first default similarity threshold for the maximal value judging to screen in the first kind car plate similar value that obtains; When judged result is for being greater than, trigger the first License Plate submodule 205, otherwise trigger the second car plate sequence acquisition module 206.

First License Plate submodule 205, for according to the first effective car plate locating rule preset and the first kind car plate similar value calculated, navigates at least one effective license plate image from described first kind candidate license plate image.

Wherein, the first License Plate submodule 205, specifically for:

Second car plate sequence obtains module 206, for regular according to the image rotation preset, carry out N time to described frequency two field picture to rotate, the video frame images rotating rear gained is each time inputed in described first sorter again, obtains N number of Equations of The Second Kind license plate image information sequence.

Second license plate image determination module 207, for the positional information based on the Equations of The Second Kind candidate license plate image in described Equations of The Second Kind license plate image information sequence, determines the Equations of The Second Kind candidate license plate image in described video frame images.

Second car plate similarity calculation module 208, for calculating the Equations of The Second Kind car plate similar value of the Equations of The Second Kind candidate license plate image corresponding to each Equations of The Second Kind license plate image information sequence, and the maximal value filtered out in the Equations of The Second Kind car plate similar value of each Equations of The Second Kind candidate license plate image corresponding to Equations of The Second Kind license plate image information sequence, and then, in maximal value from each Equations of The Second Kind car plate similar value that screening obtains, determine the target Equations of The Second Kind car plate similar value that numerical value is maximum.

Second car plate similarity judge module 209, for judging whether described target Equations of The Second Kind car plate similar value is greater than the second default similarity threshold, if be greater than, triggers the second License Plate submodule 210, otherwise triggers the 3rd car plate sequence acquisition module 211.

Second License Plate submodule 210, for according to the second effective car plate locating rule preset and the Equations of The Second Kind car plate similar value that calculates, from the Equations of The Second Kind license plate image information sequence corresponding to described target Equations of The Second Kind car plate similar value for Equations of The Second Kind candidate license plate image navigate at least one effective license plate image.

3rd car plate sequence obtains module 211, for continuing the local binary patterns LBP feature extracting described video frame images, and described LBP feature is inputed in the second sorter set up in advance, obtain the three class license plate image information sequence corresponding with described video frame images.

Wherein, the second sorter is: based on the local binary patterns LBP feature for each image in the second predetermined sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained.

3rd license plate image determination module 212, for the positional information based on described 3rd class candidate license plate image, determines the 3rd class candidate license plate image in described video frame images;

3rd car plate similarity calculation module 213, for calculating the 3rd class car plate similar value of described 3rd class candidate license plate image, and filters out the maximal value in the 3rd class car plate similar value;

3rd car plate similarity judge module 214, whether the maximal value for judging to screen the 3rd class car plate similar value obtained is greater than default third phase is seemingly spent threshold value; When judged result is for being greater than, according to the 3rd effective car plate locating rule preset and the 3rd class car plate similar value calculated, at least one effective license plate image is navigated to from described 3rd class candidate license plate image, when judged result is for being not more than, determine by described first sorter and the second sorter no-fix to effective license plate image.

In a kind of specific embodiment of the present invention, in order to improve the accuracy of license plate image location, on the basis of the license plate positioning device shown in Fig. 7, can also comprise with lower module:

4th car plate sequence obtains module, for described Haar feature being inputed in the 3rd sorter set up in advance, obtains the four class license plate image information sequence corresponding with described video frame images.

Wherein, the 3rd sorter is: based on the Ha Er Haar feature for each image in the 3rd predetermined sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained.

4th license plate image determination module, for the positional information based on described 4th class candidate license plate image, determines the 4th class candidate license plate image in described video frame images.

4th car plate similarity calculation module, for calculating the 4th class car plate similar value of described 4th class candidate license plate image, and filters out the maximal value in the 4th class car plate similar value.

In another kind of specific embodiment of the present invention, this device can further include with lower module:

5th car plate sequence obtains module, for described Haar feature being inputed in the 4th sorter set up in advance, obtains the five class license plate image information sequence corresponding with described video frame images.

Wherein, the 4th sorter is: based on the Ha Er Haar feature for each image in the 4th sample image set extracted, and adopt adaptive boosting algorithm Adaboost algorithm to train the sorter obtained.

5th license plate image determination module, for the positional information based on described 5th class candidate license plate image, determines the 5th class candidate license plate image in described video frame images.

5th car plate similarity calculation module, for calculating the 5th class car plate similar value of described 5th class candidate license plate image, and filters out the maximal value in the 5th class car plate similar value.

For system or device embodiment, because it is substantially similar to embodiment of the method, so description is fairly simple, relevant part illustrates see the part of embodiment of the method.

It should be noted that, in this article, the such as relational terms of first and second grades and so on is only used for an entity or operation to separate with another entity or operational zone, and not necessarily requires or imply the relation that there is any this reality between these entities or operation or sequentially.And, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thus make to comprise the process of a series of key element, method, article or equipment and not only comprise those key elements, but also comprise other key elements clearly do not listed, or also comprise by the intrinsic key element of this process, method, article or equipment.When not more restrictions, the key element limited by statement " comprising ... ", and be not precluded within process, method, article or the equipment comprising described key element and also there is other identical element.

One of ordinary skill in the art will appreciate that all or part of step realized in said method embodiment is that the hardware that can carry out instruction relevant by program has come, described program can be stored in computer read/write memory medium, here the alleged storage medium obtained, as: ROM/RAM, magnetic disc, CD etc.

The foregoing is only preferred embodiment of the present invention, be not intended to limit protection scope of the present invention.All any amendments done within the spirit and principles in the present invention, equivalent replacement, improvement etc., be all included in protection scope of the present invention.

Claims

1. a license plate locating method, is characterized in that, described method comprises:

2. method according to claim 1, is characterized in that, calculates the process of the first kind car plate similar value of arbitrary first kind candidate license plate image, comprising:

3. method according to claim 1, it is characterized in that, first effective car plate locating rule that described basis is preset and the first kind car plate similar value calculated, navigate at least one effective license plate image, comprising from described first kind candidate license plate image:

Or

4. method according to claim 1, is characterized in that, describedly determines that described method also comprises by after described first sorter and the second sorter no-fix to effective license plate image:

5. method according to claim 4, is characterized in that, describedly determines that described method also comprises by after described first sorter, the second sorter and the 3rd sorter no-fix to effective license plate image:

6. a license plate positioning device, is characterized in that, described device comprises:

7. device according to claim 6, is characterized in that, described first car plate similarity calculation module, for:

8. device according to claim 6, it is characterized in that, described first car plate similarity judge module comprises the first License Plate submodule for navigating at least one effective license plate image from described first kind candidate license plate image according to the first effective car plate locating rule preset and the first kind car plate similar value calculated;

Described first License Plate submodule, specifically for:

9. device according to claim 6, is characterized in that, described device also comprises:

10. device according to claim 9, is characterized in that, described device also comprises: