CN106845479B - Small-size license plate detection method based on color contrast rectangle features - Google Patents

Small-size license plate detection method based on color contrast rectangle features Download PDF

Info

Publication number
CN106845479B
CN106845479B CN201710021867.9A CN201710021867A CN106845479B CN 106845479 B CN106845479 B CN 106845479B CN 201710021867 A CN201710021867 A CN 201710021867A CN 106845479 B CN106845479 B CN 106845479B
Authority
CN
China
Prior art keywords
license plate
color
rectangular
features
contrast
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710021867.9A
Other languages
Chinese (zh)
Other versions
CN106845479A (en
Inventor
刘春生
常发亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong University
Original Assignee
Shandong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong University filed Critical Shandong University
Priority to CN201710021867.9A priority Critical patent/CN106845479B/en
Publication of CN106845479A publication Critical patent/CN106845479A/en
Application granted granted Critical
Publication of CN106845479B publication Critical patent/CN106845479B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/63—Scene text, e.g. street names
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/62—Text, e.g. of license plates, overlay texts or captions on TV images
    • G06V20/625—License plates

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a small-size license plate detection method based on color contrast rectangular characteristics, which comprises the steps of generating a pyramid image set by using a scaling algorithm for an image to be detected; calculating the difference value of the weighted sum of the pixel values of all rectangular blocks of each color channel, determining the rectangular characteristics of contrast colors, and generating a weak classifier of a cascade classifier structure containing license plates of different sizes; judging whether a license plate exists in the detection area step by step in a predetermined size in a pyramid image set by using a multi-feature cascade structure; further judging whether a license plate exists or not based on the HOG characteristics and a detection method of the SVM classifier; and calibrating the position of the license plate, converting the position into the original image according to the image of the pyramid and the reduction scale, and determining the position and the size of the detected license plate. The method can better represent the remarkable color characteristics of the license plate, can achieve high detection rate, and has good robustness.

Description

Small-size license plate detection method based on color contrast rectangle features
Technical Field
The invention relates to a small-size license plate detection method based on color contrast rectangle characteristics.
Background
In recent decades, the detection and identification of license plates has been applicable in some fields, such as: detecting and identifying the license plate of the vehicle against the traffic regulations, detecting and identifying the license plate of the parking lot and the like. The shooting quality of the license plate in the shot image is one of the main influence factors influencing the detection and identification method, the detection and identification of the clear license plate are well solved and have a plurality of successful applications, however, under the complex monitoring large scene, the detection of the small-size license plate is still a very challenging problem, and the difficult points to be overcome are as follows: firstly, the edge and content information of the small-size license plate are fuzzy, and feature extraction is not easy to perform; secondly, in the face of a complex and large background, a small-size license plate needs to be searched in a large-resolution image, which is a very time-consuming process.
The fast detection system based on the AdaBoost algorithm, the cascade mechanism and the Haar-like characteristics, which is proposed by Viola, is better applied to the detection of the license plate, but is hard to be competent for the detection of the small-size license plate. In the field of license plate detection, the detection framework of Viola has been successfully applied to a plurality of license plate detection systems, but the speed and the recognition rate of the detection framework are all to be improved, and the problem of small-size license plate detection is not involved. In the detection methods based on AdaBoost, there are typical signboards designed by using the AdaBoost method and Haar-like features such as Dlagnekov and the like, and signboards designed by using overall and local features such as Zhang and the like, and these methods cannot detect small-size license plates and have a slow operation speed when facing large-resolution images.
Disclosure of Invention
The invention aims to solve the problems and provides a small-size license plate detection method based on contrast color rectangular features.
In order to achieve the purpose, the invention adopts the following technical scheme:
a small-size license plate detection method based on color contrast rectangle features comprises the following steps:
(1) setting a scaling ratio, and generating a pyramid image set by using an input image to be detected through a scaling algorithm;
(2) calculating the difference value of the weighted sum of the pixel values of all rectangular blocks of each color channel, determining color contrast rectangular features, generating a classifier which establishes a cascade classifier structure containing license plates of different sizes, and judging whether license plates exist in the detection area step by step in a predetermined size in a pyramid image set by using a multi-feature cascade structure;
(3) judging whether a license plate exists or not based on the HOG characteristics and a detection method of an SVM classifier;
(4) and calibrating the position of the license plate, converting the position into the original image according to the image of the pyramid and the reduction scale, and determining the position and the size of the detected license plate.
In the step (1), the input image is scaled by a scaling ratio s, and the image to be measured is scaled to s, s of the size of the original image2,...,snAnd (5) generating a pyramid image set with the layer number n + 1.
In the step (2), in the off-line training process of the contrast color rectangular feature cascade detector, calculating the difference value of the weighted sum of the pixel values of all rectangular blocks of each color channel, determining the contrast color rectangular feature, generating corresponding weak classifiers, and then constructing strong classifiers capable of detecting the license plate and excluding the background area by using a plurality of weak classifiers; in the online detection process, a detector is cascaded by using a rectangular characteristic with a contrast color, and whether a license plate exists in the detection area is judged step by step in a predetermined size in a pyramid image set.
In the step (2), based on the characteristic that the license plate has obvious contrasting colors such as blue, white and the like, the color characteristic of the license plate is extracted by using the contrasting color design rectangular characteristic.
In the step (2), the N color channels respectively calculate an integral map, the integral map is used to calculate the sum of the rectangular block pixel values of each channel, and then the difference value of the weighted sum of all the rectangular block pixel values of each color channel is calculated to obtain the contrast color rectangular feature.
In the step (2), the method for calculating the sum γ of pixel values of a rectangular block of the color channel is a difference between the sum of values of the channel integral map at the upper left and lower right of the rectangular block and the sum of coordinate values of the channel integral map at the upper right and lower left of the rectangle.
In the step (2), weak classifiers used for learning and training of an AdaBoost algorithm are generated by contrasting color rectangular features, and different weak classifiers are weighted and added by the AdaBoost algorithm to obtain a strong classifier with a strong classification function.
In the step (3), the feature is formed by calculating and counting the histogram of the gradient direction of the local area of the image, the gradient feature of the image is extracted by using the HOG feature with certain overlap, and a linear or non-linear kernel function SVM classifier is used.
The invention has the beneficial effects that:
(1) the contrast color rectangular feature designed by the invention can better represent the obvious color feature of the license plate by designing the rectangular feature through contrast, and solves the problem that the traditional Haar-like feature is a gray level rectangular feature, and the gray level feature of the traditional Haar-like feature cannot express the obvious color feature of the license plate.
(2) The license plates with different sizes in the high-resolution image can be rapidly detected, the high detection rate can be achieved, and the robustness is good.
Drawings
FIG. 1 is a schematic drawing of a rectangular feature of contrasting colors of the present invention;
FIG. 2 is a schematic diagram of the multi-feature cascade classifier structure of the present invention.
The specific implementation mode is as follows:
the invention is further described with reference to the following figures and examples.
The traditional Haar-like characteristic is a gray rectangular characteristic, the gray characteristic of which cannot express the remarkable color characteristic of the license plate, the 'contrast color rectangular characteristic' designed by the invention can better express the remarkable color characteristic of the license plate by designing the rectangular characteristic through contrast, and the basic principle is as follows: the license plate has obvious contrasting colors such as blue, white and the like, and the color features of the license plate can be effectively extracted by designing the rectangular features with the contrasting colors.
The method comprises the following specific steps:
the integral map is calculated for each of the N color channels, x, y are the x-axis and y-axis coordinates of the integral map, z represents the color channel, and the coordinate point on the upper left of (x, y) is represented as (x ', y'), the integral map I (x, y, z) is calculated as follows,
Figure BDA0001208526510000031
where P (x ', y', z) is a pixel value of the input image P at the coordinate point (x ', y') of the z color channel.
The sum y of the pixel values of a rectangular block of the z-channel is calculated,
γ＝I(x3,y3,z)+I(x0,y0,z)-I(x1,y1,z)-I(x2,y2,z) (6)
wherein, I (x)i,yiZ) is the z-channel integral plot at xi,yiThe value of (c).
(x0,y0),(x1,y1),(x2,y2),(x3,y3) The coordinate values of the rectangular block at the upper left, upper right, lower left and lower right. The "contrast color rectangle feature" is calculated as follows,
f＝∑w(i)·r(i)-∑w(j)·r(j)(7)
wherein, Σ w(i)·r(i)Sum Σ w(j)·r(j)In, r(i)And r(j)Is the sum of the pixel values of the i and j color channel tiles, w(i)And w(j)Are respectively r(i)And r(j)By weight of (E), Σ w(i)·r(i)Sum Σ w(j)·r(j)Is a weighted sum of the sums of all tile pixel values belonging to the i and j color channels, respectively.
The weak classifier of the "contrast color rectangle feature" is denoted as hh：
Figure BDA0001208526510000032
Wherein, thetahIs the threshold obtained during the AdaBoost training process, and σ ∈ { +1, -1} is a polarity parameter.
The AdaBoost method is a machine learning method, which can obtain weak classifiers capable of distinguishing positive and negative samples by training positive and negative training samples, the weak classifiers have weak classification capability and can only obtain classification results with higher accuracy than 50%, but a strong classifier with strong classification function is obtained by weighting and adding different weak classifiers, and the strong classifier can effectively distinguish the positive and negative samples.
The strong classifier after AdaBoost learning is Hh：
Figure BDA0001208526510000041
Wherein, whIs hhThe corresponding weight.
Since the small-size license plate has the problems of fuzzy edge, unobvious features and the like, most background windows can be eliminated by detecting the color matching rectangular feature cascade detector, but the obtained detection window still contains partial background, and therefore, the HOG feature and SVM classifier classification method is further used for judging whether the detection window is the license plate or the background.
On the basis of the detection of the colorimetric rectangular feature cascade detector, a detection method based on HOG features and an SVM classifier is designed, and whether a license plate exists or not is further judged and detected. The HOG feature, also called Histogram of oriented Gradient (Histogram of oriented) feature, is a feature descriptor used for object detection in computer vision and image processing. It constructs features by calculating and counting the histogram of gradient direction of local area of image. HOG features in combination with SVM classifiers have been widely used in image recognition. The patent extracts the HOG features with certain overlap, and under the minimum detection window of 50 multiplied by 14, the extracted HOG features are 1944-dimensional and have the parameters as follows: one block unit is 10 × 4, each small block cell in the block is 5 × 2, step is 5 × 2, and directions are 9 different directions. The extracted HOG features are classified by a linear or nonlinear kernel function SVM classifier, so that a good detection effect is achieved.
In the detection process, the whole detection steps are as follows:
(1) generating a pyramid image set by using a scaling algorithm for an input image to be detected, taking the scaling as s, wherein s is equal to or more than 0.87 and is equal to or less than 0.95 under the condition of detecting a small-size license plate, and scaling the input image into s, s of the size of an original image according to the scaling s2,...,snAnd (5) generating a pyramid image set with the layer number n + 1.
(2) In the off-line training process of the contrast color rectangular feature cascade detector, calculating the difference value of the weighted sum of the pixel values of all rectangular blocks of each color channel, determining the contrast color rectangular feature, generating corresponding weak classifiers, and then constructing a strong classifier capable of detecting the license plate and excluding the background area by using a plurality of weak classifiers; in the online detection process, a color-contrast rectangular feature cascade detector is used for judging whether a license plate exists in the detection area step by step in a predetermined size in a pyramid image set; in the detection process of the small-size license plate, the experience value range of the stepping b is that b is more than or equal to 5 and less than or equal to 10. Since the small-size license plate has the problems of fuzzy edge, unobvious features and the like, most background windows can be eliminated by detecting the color matching rectangular feature cascade detector, but the obtained detection window still contains partial background, and therefore, the HOG feature and SVM classifier classification method is further used for judging whether the detection window is the license plate or the background.
(3) And in the off-line training process of the HOG characteristic and the SVM classifier, the HOG + SVM classifier is obtained by training the license plate and the background image sample. In the detection process, a detection window obtained by a contrast color rectangular feature cascade detector is used as input, HOG features of the detection window are extracted, an SVM classifier trained offline is used for carrying out secondary classification (license plate and background), and whether the detection window is the license plate or the background is judged;
(4) and calibrating the position of the license plate, converting the position into the original image according to the image of the pyramid and the reduction scale, and determining the position and the size of the detected license plate. Calibrating the position of the license plate in the pyramid scaled image to be (x, y) and the size of the license plate to be (w, h); and converting the image into an original image according to the reduction ratio s of the positioned pyramid image, and determining the position (x/s, y/s) and the size (w/s, h/s) of the detected license plate.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive efforts by those skilled in the art based on the technical solution of the present invention.

Claims (3)

1. A small-size license plate detection method based on color contrast rectangle features is characterized by comprising the following steps: the method comprises the following steps:
(1) setting a scaling ratio, and generating a pyramid image set by using an input image to be detected through a scaling algorithm;
(2) calculating the difference value of the weighted sum of the pixel values of all rectangular blocks of each color channel, determining color contrast rectangular features, generating a classifier which establishes a cascade classifier structure containing license plates of different sizes, and judging whether license plates exist in a detection area step by step in a predetermined size in a pyramid image set by using a multi-feature cascade structure;
(3) judging whether a license plate exists or not based on the HOG characteristics and a detection method of an SVM classifier;
(4) calibrating the position of the license plate, converting the position into an original image according to the image of the pyramid and the reduction scale, and determining the position and the size of the detected license plate;
in the step (1), the input image is scaled by a scaling ratio s, and the image to be measured is scaled to s, s of the size of the original image2,...,snMultiplying, generating a pyramid image set with the number of layers being n + 1;
in the step (2), in the off-line training process of the contrast color rectangular feature cascade detector, calculating the difference value of the weighted sum of the pixel values of all rectangular blocks of each color channel, determining the contrast color rectangular feature, generating corresponding weak classifiers, and then constructing strong classifiers capable of detecting the license plate and excluding the background area by using a plurality of weak classifiers; in the online detection process, a color-contrast rectangular feature cascade detector is used for judging whether a license plate exists in a detection area step by step in a predetermined size in a pyramid image set;
in the step (2), based on the characteristic that the license plate has obvious blue and white contrast colors, the color characteristic of the license plate is extracted by using the contrast color design rectangular characteristic;
in the step (2), respectively calculating an integral map by the N color channels, calculating the sum of the pixel values of the rectangular blocks of each channel by using the integral map, and further calculating the difference value of the weighted sum of the sums of the pixel values of all the rectangular blocks of each color channel to obtain the rectangular features of the contrast color;
in the step (3), in the detection process, a detection window obtained by a contrast color rectangular feature cascade detector is used as input, HOG features of the detection window are extracted, an SVM classifier of off-line training is used for carrying out secondary classification, whether the detection window is a license plate or a background is judged, features are formed by calculating and counting a gradient direction histogram of a local area of an image, gradient features of the detection window are extracted by using the overlapped HOG features, and a linear or non-linear kernel function SVM classifier is used.
2. The method for detecting the small-size license plate based on the rectangle features with the contrasting colors as claimed in claim 1, which is characterized in that: in the step (2), the method for calculating the sum γ of pixel values of a rectangular block of the color channel is a difference between the sum of values of the channel integral map at the upper left and lower right of the rectangular block and the sum of coordinate values of the channel integral map at the upper right and lower left of the rectangle.
3. The method for detecting the small-size license plate based on the rectangle features with the contrasting colors as claimed in claim 1, which is characterized in that: in the step (2), weak classifiers used for learning and training of an AdaBoost algorithm are generated by contrasting color rectangular features, and different weak classifiers are weighted and added by the AdaBoost algorithm to obtain a strong classifier with a strong classification function.
CN201710021867.9A 2017-01-12 2017-01-12 Small-size license plate detection method based on color contrast rectangle features Active CN106845479B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710021867.9A CN106845479B (en) 2017-01-12 2017-01-12 Small-size license plate detection method based on color contrast rectangle features

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710021867.9A CN106845479B (en) 2017-01-12 2017-01-12 Small-size license plate detection method based on color contrast rectangle features

Publications (2)

Publication Number Publication Date
CN106845479A CN106845479A (en) 2017-06-13
CN106845479B true CN106845479B (en) 2020-03-27

Family

ID=59123494

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710021867.9A Active CN106845479B (en) 2017-01-12 2017-01-12 Small-size license plate detection method based on color contrast rectangle features

Country Status (1)

Country Link
CN (1) CN106845479B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111199225B (en) * 2018-11-20 2023-10-31 浙江宇视科技有限公司 License plate calibration method and device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102073854A (en) * 2011-01-14 2011-05-25 天津大学 Color license plate positioning method
CN103093250A (en) * 2013-02-22 2013-05-08 福建师范大学 Adaboost face detection method based on new Haar- like feature
CN103400121A (en) * 2013-08-06 2013-11-20 河海大学 License plate locating method based on colorful binary image
CN103761529A (en) * 2013-12-31 2014-04-30 北京大学 Open fire detection method and system based on multicolor models and rectangular features
CN105551264A (en) * 2015-12-25 2016-05-04 中国科学院上海高等研究院 Speed detection method based on license plate characteristic matching

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102073854A (en) * 2011-01-14 2011-05-25 天津大学 Color license plate positioning method
CN103093250A (en) * 2013-02-22 2013-05-08 福建师范大学 Adaboost face detection method based on new Haar- like feature
CN103400121A (en) * 2013-08-06 2013-11-20 河海大学 License plate locating method based on colorful binary image
CN103761529A (en) * 2013-12-31 2014-04-30 北京大学 Open fire detection method and system based on multicolor models and rectangular features
CN105551264A (en) * 2015-12-25 2016-05-04 中国科学院上海高等研究院 Speed detection method based on license plate characteristic matching

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于多特征联合的车牌定位算法研究;刘涞;《中国优秀硕士学位论文全文数据库 信息科技辑》;20160715(第07期);正文第3、8、34、36-38、47、57-61页，图2-1，图5-1 *

Also Published As

Publication number Publication date
CN106845479A (en) 2017-06-13

Similar Documents

Publication Publication Date Title
CN109241985B (en) Image identification method and device
CN111310645B (en) Method, device, equipment and storage medium for warning overflow bin of goods accumulation
CN107038416B (en) Pedestrian detection method based on binary image improved HOG characteristics
WO2020248513A1 (en) Ocr method for comprehensive performance test
CN109918971B (en) Method and device for detecting number of people in monitoring video
CN107103317A (en) Fuzzy license plate image recognition algorithm based on image co-registration and blind deconvolution
CN109919002B (en) Yellow stop line identification method and device, computer equipment and storage medium
CN102819733B (en) Rapid detection fuzzy method of face in street view image
CN114972922B (en) Coal gangue sorting and identifying method, device and equipment based on machine learning
WO2016045215A1 (en) Fingerprint image quality determination method and device
US8655060B2 (en) Night-scene light source detecting device and night-scene light source detecting method
WO2023279966A1 (en) Multi-lane-line detection method and apparatus, and detection device
CN105740751A (en) Object detection and identification method and system
CN105095835A (en) Pedestrian detection method and system
CN104050684A (en) Video moving object classification method and system based on on-line training
Wali et al. Shape matching and color segmentation based traffic sign detection system
CN106485639A (en) The method and apparatus differentiating forged certificate picture
Lin et al. A traffic sign recognition method based on deep visual feature
CN105809092A (en) Population target detection method and device thereof
Wu et al. Automatic gear sorting system based on monocular vision
CN106845479B (en) Small-size license plate detection method based on color contrast rectangle features
CN106778913B (en) Fuzzy license plate detection method based on pixel cascade characteristic
CN117765287A (en) Image target extraction method combining LWR and density clustering
Piérard et al. A probabilistic pixel-based approach to detect humans in video streams
CN104463910B (en) High-speed motion target extraction method based on motion vector

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant