CN109993171B - License plate character segmentation method based on multiple templates and multiple proportions - Google Patents

License plate character segmentation method based on multiple templates and multiple proportions Download PDF

Info

Publication number
CN109993171B
CN109993171B CN201910183838.1A CN201910183838A CN109993171B CN 109993171 B CN109993171 B CN 109993171B CN 201910183838 A CN201910183838 A CN 201910183838A CN 109993171 B CN109993171 B CN 109993171B
Authority
CN
China
Prior art keywords
license plate
character
template
training
row
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910183838.1A
Other languages
Chinese (zh)
Other versions
CN109993171A (en
Inventor
解梅
李思琦
秦国义
易鑫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Electronic Science and Technology of China
Original Assignee
University of Electronic Science and Technology of China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Electronic Science and Technology of China filed Critical University of Electronic Science and Technology of China
Priority to CN201910183838.1A priority Critical patent/CN109993171B/en
Publication of CN109993171A publication Critical patent/CN109993171A/en
Application granted granted Critical
Publication of CN109993171B publication Critical patent/CN109993171B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/245—Classification techniques relating to the decision surface
    • G06F18/2451—Classification techniques relating to the decision surface linear, e.g. hyperplane
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/148—Segmentation of character regions
    • G06V30/153—Segmentation of character regions using recognition of characters or words

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Character Input (AREA)
  • Character Discrimination (AREA)
  • Traffic Control Systems (AREA)

Abstract

The invention belongs to the field of image processing and pattern recognition, and provides a license plate character segmentation method based on multiple templates and multiple proportions, which is used for overcoming the problems that the existing character segmentation method is single in license plate type, cannot segment fuzzy or stained characters and the like, and realizing accurate character segmentation of license plates of multiple types. Based on the idea that a license plate standard template guides character segmentation, a multi-scale template is adopted on the basis, a single-double-row judgment model, a multi-class template sliding scoring mechanism and a judgment auxiliary model are added, and an optimal segmentation result selection strategy is obtained; in conclusion, the invention provides a license plate character segmentation method based on multiple templates and multiple proportions, which can realize accurate segmentation of multiple types of license plates and fuzzy and stained license plates.

Description

License plate character segmentation method based on multiple templates and multiple proportions
Technical Field
The invention belongs to the field of image processing and pattern recognition, and mainly relates to a license plate character segmentation method based on multiple templates and multiple proportions, which is used for carrying out character segmentation on license plates of various categories.
Background
The intelligent transportation is an indispensable link for building an intelligent city, and the license plate recognition system is an important component in the intelligent transportation; the license plate recognition algorithm generally comprises 3 parts, namely license plate positioning, character segmentation and character recognition, wherein the license plate character segmentation occupies an important position in the license plate recognition algorithm, and the license plate character recognition depends on a better license plate character segmentation result.
In a real environment, 7 types of common license plates are respectively shown in fig. 1 to 7; the 1 st type is a normal 7-digit license plate, and specific license plates include a small-sized automobile license plate, a large-sized automobile front license plate, a license plate of a leadership, a hong Kong entry and exit license plate, a Macau entry and exit license plate and a coach car license plate as shown in FIG. 1; the 2 nd type license plate is a police car license plate, as shown in figure 2; the 3 rd type license plate is a new energy license plate, and is shown in fig. 3; the 4 th type of license plate is a license plate of a embassy, as shown in FIG. 4; the 5 th type of license plate is a police license plate, as shown in figure 5; the 6 th category is the rear license plate of the large automobile, and as shown in fig. 6, the specific license plate comprises a trailer rear license plate and a large automobile rear license plate; category 7 is low speed front license plate, as shown in fig. 7; the templates of the license plates of each type are the same, the 1 st type to the 5 th type are single-row license plates, and the 6 th type and the 7 th type are double-row license plates.
The license plate character segmentation method adopting the traditional method mainly comprises template matching, and the license plate image segmentation method by utilizing the standard license plate template has higher requirements on the characters: (1) the left and right boundaries of the input license plate need to be accurately positioned, (2) the license plate cannot be blurred or the characters are stained, and (3) the license plate is single in type; how to accurately segment characters of a plurality of types of license plates is a difficult problem to be solved urgently.
Disclosure of Invention
The invention aims to solve the problems and provides a license plate character segmentation method based on multiple templates and multiple proportions, which is used for accurately segmenting various license plates in a real environment; the method is not only suitable for various license plates, but also has better robustness, and can effectively segment the blurred and stained license plates.
In order to achieve the purpose, the invention adopts the technical scheme that:
a license plate character segmentation method based on multiple templates and multiple proportions comprises the following steps:
step 1, training single-double-row license plate judgment model
Taking a single-line license plate as a positive sample of training and a double-line license plate as a negative sample of training, extracting characteristics of the training samples, training a two-classification model based on a support vector machine based on the characteristics, and taking the two-classification model as a single-line and double-line license plate judgment model;
step 2, training scoring model
Taking the character images as positive samples of training and the non-character images as negative samples of training, extracting character features from the training samples, and training a scoring model based on a support vector machine based on the character features to serve as a scoring model;
step 3, training and distinguishing auxiliary model
Taking the first and last characters of the type 1 license plate, the type 2 license plate and the type 3 license plate as positive samples of training, taking an image in a character segmentation frame before the first character of the type 1 license plate and the type 2 license plate and taking an image in a character segmentation frame after the last character as negative samples; extracting character features from the training samples, and training a two-classification model based on a support vector machine based on the character features to serve as a discrimination auxiliary model;
step 4, constructing templates of various types and proportions
Aiming at a single-row license plate, constructing a multi-scale template: the 7 character boxes are represented from left to right as follows:
(0,0,αi·w,h)，(αi·w,0,αi·w,h)，(αi(L-5w),0,αi·w,h)，(αi(L-4w),0,αi·w,h)，(αi(L-3w),0,αi·w,h)，(αi(L-2w),0,αi·w,h)，(αi(L-w),0,αi·w,h)；
wherein h is the height of the standard template, w is the width of the character frame, L is the width of the standard template, alpha0,α1,…,αi,…,αn-1,αnRepresenting n scaling factors;
aiming at the double-row license plate template, constructing a multi-scale template:
first row: from left to right in sequence: (alphai·w2,0,αi·w1,h1)，(αi·(L-w2-w1),0,αi·w1,h1)，
A second row: from left to right in sequence: (0, h-h)2,αi·w2,h2)，(αi·w2,h-h2,αi·w2,h2)，(2(αi·w2),h-h2,αi·w2,h2)，(3(αi·w2),h-h2,αi·w2,h2)，(4(αi·w2),h-h2,αi·w2,h2)；
Wherein h is1Is the height, h, of the character box in the first line2Is the height, w, of the character box in the second line1Is the width, w, of the character box in the first line2Is the width of the character box in the second line;
step 5, adopting the single-double-row license plate judgment model obtained by training in the step 1 to judge the single-double rows of license plates to be segmented; if the license plate is a single-row license plate, performing step 6; if the number plate is a double-row number plate, performing a step 8;
step 6, sliding scoring of the multi-class multi-scale templates of the single-row license plate
Step 6.1, calculating a normalization factor:
Figure BDA0001992180820000031
the license plate template is normalized by adopting the normalization factor, and the normalized size is as follows:
Figure BDA0001992180820000032
wherein, rows is the height of the input license plate image;
step 6.2, sliding scoring:
templates of the ith scale for class p templates
Figure BDA0001992180820000033
p is 1,2,3,4,5, and the sliding range is
Figure BDA0001992180820000034
Figure BDA0001992180820000035
Representing normalized templates
Figure BDA0001992180820000036
Cols is the width of the input license plate image;
as a template
Figure BDA0001992180820000037
When the image slides to the coordinate m, taking out a plurality of images segmented by the template, extracting the characteristics of the images, grading the images by adopting the grading model trained in the step 2, and calculating to obtain an average score
Figure BDA0001992180820000038
Selecting the highest score of the multi-proportion template in each type of template
Figure BDA0001992180820000039
Step 7, selecting the optimal segmentation result of the single-row license plate
Selecting a maximum value:
Figure BDA00019921808200000310
according to the above
Figure BDA00019921808200000311
Selecting an optimal segmentation result:
step 7-1-1, if pmaxFurther judgment is made as 1
Figure BDA00019921808200000312
Taking a character after the last segmentation frame of the corresponding segmentation resultIf the frame exceeds the test image, if so, the license plate to be segmented is a type 1 license plate, and the license plate is output
Figure BDA00019921808200000313
Corresponding segmentation results, otherwise, 7-1-2 is carried out;
step 7-1-2, judging by adopting a discrimination auxiliary model
Figure BDA00019921808200000314
After the last segmentation frame of the corresponding segmentation result, whether the content in a character frame is a character is selected; if yes, the license plate to be divided is a 3 rd type license plate, and output
Figure BDA00019921808200000315
Corresponding segmentation results; otherwise, carrying out 7-1-3;
step 7-1-3, adopting a discrimination auxiliary model to further judge
Figure BDA00019921808200000316
If the 1 st bit of the corresponding segmentation result is a character, outputting
Figure BDA00019921808200000317
Corresponding segmentation result, otherwise, outputting
Figure BDA00019921808200000318
Corresponding segmentation results;
step 7-2-1, if pmaxFurther judge 2
Figure BDA00019921808200000319
Taking a character frame before the first segmentation frame of the corresponding segmentation result to determine whether the character frame exceeds the test image, if so, outputting the license plate to be segmented as a type 2 license plate
Figure BDA0001992180820000041
Corresponding segmentation results, otherwise, 7-2-2 is carried out;
step 7-2-2, adopting a discrimination auxiliary modelJudgment of
Figure BDA0001992180820000042
Before the first segmentation frame of the corresponding segmentation result, whether the content in a character frame is a character is selected; if yes, the license plate to be divided is a 3 rd type license plate, and output
Figure BDA0001992180820000043
Corresponding segmentation results; otherwise, output
Figure BDA0001992180820000044
Corresponding segmentation results;
step 7-3-1, if pmaxWhen the output is 3,4 and 5, the output is obtained
Figure BDA0001992180820000045
Corresponding segmentation results;
step 8, sliding scoring of multi-class multi-scale templates of double-row license plates
Step 8.1, firstly, normalizing the size of the template to the actual size of the license plate: the normalized sizes are:
Figure BDA0001992180820000046
step 8.2, sliding scoring, i proportion template for p class template
Figure BDA0001992180820000047
And (4) obtaining the highest score of the multi-scale template in each type of template by adopting the same sliding scoring process in the step 6.2 when the p is 6 and 7
Figure BDA0001992180820000048
Step 9, selecting an optimal segmentation result for the double-row license plate:
selecting a maximum value:
Figure BDA0001992180820000049
according to the above
Figure BDA00019921808200000410
Correspondingly outputting a segmentation result: p is a radical ofmax6,7, output
Figure BDA00019921808200000411
And (4) corresponding segmentation results.
The invention has the beneficial effects that:
the invention provides a license plate character segmentation method based on multiple templates and multiple proportions, which is based on the idea of guiding character segmentation by a license plate standard template, and adds a single-double-row judgment model, a multi-class template sliding scoring mechanism, a judgment auxiliary model and a multiple proportion template on the basis to obtain an optimal segmentation result selection strategy; the accurate segmentation of the multi-class license plate and the fuzzy stained license plate can be realized.
Drawings
Fig. 1 to 7 are schematic diagrams of types 1 to 7 of license plates commonly used in a real environment in sequence.
FIG. 8 is a flow chart of a license plate character segmentation method with multiple templates and multiple proportions according to the present invention.
FIG. 9 is a diagram of an optimal classification surface of a support vector machine.
FIG. 10 is a schematic diagram of the feature extraction process in step 1 of the present invention.
Fig. 11 is a schematic diagram of the number plate style of a single row of license plates issued by a country.
FIG. 12 is a diagram illustrating a standard template for a single-row license plate in an embodiment.
FIG. 13 is a template diagram showing multiple scales of a single row license plate in an implementation.
FIG. 14 is a schematic diagram of the number plate style of a state issuing double-row license plate.
FIG. 15 is a diagram illustrating a standard template for a double-row license plate according to an embodiment.
Fig. 16 is a schematic diagram of a type 2 license plate template segmenting a type 2 license plate and a type 3 license plate in the embodiment.
Fig. 17 is a schematic diagram illustrating a type 1 license plate template segmenting a type 1 license plate and a type 3 license plate in the embodiment.
FIG. 18 is a schematic diagram of class 1-3 standard license plate templates and license plate images in the embodiment.
FIGS. 19 to 25 are sequentially graphs showing the results of the type 1 to type 7 license plate segmentation in the examples.
FIG. 26 is a diagram illustrating the segmentation results of the fuzzy type 1 license plate in the example.
FIG. 27 is a graph showing the segmentation results of the stained type 1 license plate in the example.
FIG. 28 is a diagram illustrating the segmentation results of the fuzzy 6 th type license plate in the example.
FIG. 29 is a graph of the segmentation results of the stained type 6 license plate in the example.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples.
The invention provides a license plate character segmentation method based on multiple templates and multiple proportions, which comprises 6 steps of training a single-double-row license plate judgment model, training a scoring model, constructing templates with multiple types and multiple proportions, sliding scoring the templates with multiple types and multiple proportions, training a discrimination auxiliary model and selecting an optimal segmentation result. The classification models used in the invention are all support vector machines, the support vector machines are two classification models, the basic model of the support vector machines is a linear classifier with the maximum interval defined on a feature space, and the maximum interval makes the support vector machines different from a perception machine; the support vector machine also includes kernel skills that make it a substantially non-linear classifier, and the learning strategy of the support vector machine is interval maximization, as shown in fig. 9.
In this embodiment, the specific steps are as follows:
1. training single-double-row license plate judging model
1) Collecting and making a data set, wherein a single-row license plate is used as a positive sample for training, and a double-row license plate is used as a negative sample for training;
2) extracting features, as shown in fig. 10;
2-1) converting the RGB image into a gray image,
2-2) scale-normalizing the grayscale image to 140 x 44,
2-3) extracting HOG characteristics from the normalized image,
a HOG feature, namely a Histogram of Oriented Gradient (HOG), which is a feature descriptor for describing local textures of an image, wherein the extracted HOG feature is 80 dimensions; the specific step of extracting the HOG features in this embodiment is to divide the normalized grayscale character image into 8 blocks as shown in fig. 10, where each block includes 770 pixel points, calculate the gradient of each pixel, and then record the directional gradient features of a single block with a histogram of 10 bins, where each bin represents the range of a gradient direction (angle), and its value is obtained by superimposing the gradient amplitude values of the pixel points satisfying the corresponding gradient direction;
2-4) extracting vertical projection characteristics and horizontal projection characteristics from the normalized image, wherein the vertical projection characteristics have 140 dimensions and the horizontal projection characteristics have 44 dimensions;
2-5) combining the HOG feature, the vertical projection feature and the horizontal projection feature together to form a 264-dimensional new feature;
3) training a two-classification model based on a support vector machine by using the extracted new features, and judging whether the input license plate is a single-row license plate or a double-row license plate;
2. training scoring model
1) Collecting and making a data set, taking the character images as positive samples of training, and taking the non-character images as negative samples of training;
2) extracting features, where the step of extracting features is the same as in step 1;
2-1) converting the RGB image into a gray image,
2-2) scale-normalizing the grayscale image to 16 x 32,
2-3) extracting HOG features from the normalized image, wherein the extracted HOG features are 80-dimensional,
2-4) extracting vertical projection characteristics and horizontal projection characteristics from the normalized image, wherein the vertical projection characteristics are 16-dimensional, the horizontal projection characteristics are 32-dimensional,
2-5) combining the HOG features, the vertical projection features, and the horizontal projection features together to form a new feature of 128 dimensions, denoted as x,
2-6) training a scoring model based on a support vector machine by using the extracted novel character features x, recording the model as R (x), and recording the extracted character features as x for any test imageiThe score is marked as R (x) by the scoring modeli)，R(xi) Actually representing the probability that the image is a character;
3. form board for constructing various types and proportions
According to the number plate style issued by the country to each type of number plate, a number plate template with standard size can be constructed; after the standard license plate template is determined, the coordinates and the size of the rectangular frames are also determined;
for example, for a single-row license plate type 1, the number plate style published by the country is as shown in fig. 11, from which a standard template as shown in fig. 12 can be constructed, wherein h is 90 for the height of the standard template, w is 57 for the width of the character frame, and L is 421 for the width of the standard template, and then 7 character frames can be represented as (0,0, w, h), (w,0, w, h), (L-5w,0, w, h), (L-4w,0, w, h), (L-3w,0, w, h), (L-2w,0, w, h), (L-w,0, w, h); the 4 parameters of each rectangular frame are sequentially the coordinates of a point x at the upper left corner, the coordinates of a point y at the upper left corner, the width w of the rectangular frame and the height h of the rectangular frame from left to right;
although the aspect ratio of the license plate characters is fixed, the aspect ratio of the characters of the license plate image changes due to the problem of the shooting angle of a camera in reality; therefore, if only one template, the standard template shown in fig. 12, is used, the best segmentation result cannot be achieved; therefore, license plate templates with various character aspect ratios need to be constructed according to standard templates, a plurality of proportionality coefficients are usually selected at equal intervals between 0.5 and 1.5 (the specific selection number is determined according to the number of step lengths and the equal interval selection needs to be determined through experiments, and the comprehensive precision and the efficiency are jointly determined), and it is assumed that n proportionality coefficients (alpha) are selected for the type 1 license plate templates0,α1,…,αi,…,αn-1,αn) As shown in fig. 13, the 7 character boxes of the ith license plate template from left to right can be represented as:
(0,0,αi·w,h)，(αi·w,0,αi·w,h)，(αi(L-5w),0,αi·w,h)，(αi(L-4w),0,αi·w,h)，(αi(L-3w),0,αi·w,h)，(αi(L-2w),0,αi·w,h)，(αi(L-w),0,αi·w,h)；
the pattern of the double-row license plate is shown in fig. 14, and thus a standard template shown in fig. 15 can be constructed, wherein h-185 is the height and h of the standard template160 is the height h of the 2 character boxes in the first line 2110 is the height of the 5 character boxes in the second row, w 180 is the width, w, of the first line character box 2100 is the width of the second line of character frame, and L421 is the width of the standard template; the 2 character boxes of the first row can be represented as (w) from left to right2,0,w1,h1)，(L-w2-w1,0,w1,h1) The 5 character boxes of the second row can be represented from left to right as:
(0,h-h2,w2,h2)，(w2,h-h2,w2,h2)，(2w2,h-h2,w2,h2)，(3w2,h-h2,w2,h2)，(4w2,h-h2,w2,h2)；
similarly, according to the method for acquiring the multi-scale template from the single-row license plate, a plurality of double-row license plate templates with different aspect ratios can be obtained, and the 7 character frames of the ith license plate template can be represented as:
first row: from left to right in sequence: (alphai·w2,0,αi·w1,h1)，(αi·(L-w2-w1),0,αi·w1,h1)
A second row: from left to right in sequence: (0, h-h)2,αi·w2,h2)，(αi·w2,h-h2,αi·w2,h2)，(2(αi·w2),h-h2,αi·w2,h2)，(3(αi·w2),h-h2,αi·w2,h2)，(4(αi·w2),h-h2,αi·w2,h2)；
4. Adopting the single-double-row license plate judgment model obtained by training in the step 1 to judge the single-double rows of license plates to be segmented; if the license plate is a single-row license plate, performing the step 5; if the number plate is a double-row number plate, performing the step 6;
5. single-line license plate multi-class multi-scale template sliding scoring
Inputting a license plate image with the width of cols and the height of rows; firstly, normalizing the size of a template to the actual size of a license plate, and specifically operating:
calculating a normalization factor:
Figure BDA0001992180820000081
the license plate template is normalized by adopting the normalization factor:
the normalized sizes are:
Figure BDA0001992180820000082
thus, the sizes and relative positions of all character frames in the actual size can be determined, and then the normalized template slides from left to right on the input license plate image and is scored;
templates of the ith scale for class p templates
Figure BDA0001992180820000083
p is 1,2,3,4,5, and the sliding range is
Figure BDA0001992180820000084
Figure BDA0001992180820000085
Representing normalized templates
Figure BDA0001992180820000086
After balancing the accuracy and speed of the segmentation, a suitable sliding step length step can be selectedSliding; when in use
Figure BDA0001992180820000087
When the image is slid to the coordinate m, 7 images obtained by dividing the template are extracted (if p is 3, there are 8 divided images, the principle is the same), and the 7 images are subjected to feature extraction, and the 7 features are expressed as:
Figure BDA0001992180820000088
then, the 7 images are scored by using the scoring model in step 2, so that:
Figure BDA0001992180820000089
the average of these 7 scores was then scored as the average score
Figure BDA00019921808200000810
Then, a set is used for representing the scores of templates with any kind and any proportion at any position
Figure BDA00019921808200000811
When the license plate templates of different types and different proportions are scored, the highest score in each type of template is selected and recorded as
Figure BDA0001992180820000091
Denotes the ith class templatemaxSliding the seed ratio to the coordinate mmaxThe time division result is the highest score in the template; only the segmentation results corresponding to the highest scores of the license plate templates of different categories are compared to obtain the final optimal segmentation result, and how to obtain the final optimal segmentation result is introduced in step 7;
6. double-row license plate multi-class multi-scale template sliding scoring
Firstly, normalizing the size of the template to the actual size of the license plate: the normalized sizes are:
Figure BDA0001992180820000092
templates of the ith scale for class p templates
Figure BDA0001992180820000099
And p is 6 and 7, and the same sliding scoring process in the step 5 is adopted to obtain a set
Figure BDA0001992180820000093
And selecting the maximum value
Figure BDA0001992180820000094
7. Selecting the optimal segmentation result
Through the steps, whether the license plate to be detected is a single-row license plate or a double-row license plate can be judged, and the highest score of each type of template for sliding of the license plate to be segmented can be obtained
Figure BDA0001992180820000095
And respectively selecting maximum values for the single-row license plate and the double-row license plate:
single row:
Figure BDA0001992180820000096
double rows:
Figure BDA0001992180820000097
in the identification process aiming at the single-row license plate, if the license plate is scored as
Figure BDA0001992180820000098
The temporal segmentation result cannot be output as an optimal segmentation result; when the score of the 2 nd template is the highest, the license plate at the moment is not necessarily the 2 nd license plate, but may also be the 3 rd license plate, as shown in fig. 16; if the segmentation result of the 2 nd type template is directly taken and output, the optimal segmentation result cannot be obtained,the optimal segmentation result can be obtained only by taking the license plate templates of the same type to carry out sliding scoring on the license plates of the same type; when the score of the type 1 template is the highest, the license plate at this time is not necessarily the type 1 license plate, and may be the type 3 license plate, as shown in fig. 17; similarly, if the segmentation result of the type 3 template is directly taken and output, the optimal segmentation result cannot be obtained; when the scores of other types of templates are the highest, outputting the template segmentation result with the highest score to obtain the optimal segmentation result; this occurs because the three categories of cards are too similar in style, as shown in fig. 18, while the other categories of vehicles have more different styles. Therefore, the method introduces a discrimination auxiliary model to select the optimal segmentation result; the discriminant auxiliary model training process is as follows:
1) collecting and making a data set, taking the first and last characters of the type 1 license plate, the type 2 license plate and the type 3 license plate as positive samples of training, taking an image in a character segmentation frame before the first character of the type 1 license plate and the type 2 license plate and taking an image in a character segmentation frame after the last character as negative samples;
2) extracting character features from the training samples, wherein the character features are the same as the features used for training the scoring model, and the process is not repeated;
3) training a two-classification model based on a support vector machine by using character characteristics as a discrimination auxiliary model;
the optimal segmentation result is selected according to the strategy of the invention, and the flow is shown in fig. 8, which specifically comprises the following steps:
1) license plate to be divided into single-line license plates
1-1-1, if pmaxFurther judgment is made as 1
Figure BDA0001992180820000101
After the last segmentation frame of the corresponding segmentation result, taking a character frame to determine whether the character frame exceeds the test image, if so, outputting the license plate to be segmented as a type 1 license plate as shown in FIG. 17
Figure BDA0001992180820000102
Corresponding segmentation results, otherwise, performing 1-1-2;
1-1-2, judging by adopting a discrimination auxiliary model
Figure BDA0001992180820000103
After the last segmentation frame of the corresponding segmentation result, whether the content in a character frame is a character is selected; if yes, the license plate to be divided is a type 3 license plate, as shown in FIG. 16, that is, outputting
Figure BDA0001992180820000104
Corresponding segmentation results; otherwise, 1-1-3 is carried out;
1-1-3, adopting a discrimination auxiliary model to further judge
Figure BDA0001992180820000105
If the 1 st bit of the corresponding segmentation result is a character, outputting
Figure BDA0001992180820000106
Corresponding segmentation result, otherwise, outputting
Figure BDA0001992180820000107
Corresponding segmentation results;
1-2-1, if pmaxFurther judge 2
Figure BDA0001992180820000108
Taking another character frame before the first segmentation frame of the corresponding segmentation result to determine whether the character frame exceeds the test image, if so, outputting the license plate to be segmented as the type 2 license plate as shown in FIG. 15
Figure BDA0001992180820000109
Corresponding segmentation results, otherwise, performing 1-2-1;
1-2-1, judging by adopting a discrimination auxiliary model
Figure BDA00019921808200001010
First bit of corresponding division resultBefore the frame is divided, whether the content in a character frame is a character or not is selected; if yes, the license plate to be divided is a type 3 license plate, as shown in FIG. 16, that is, outputting
Figure BDA00019921808200001011
Corresponding segmentation results; otherwise, output
Figure BDA00019921808200001012
Corresponding segmentation results;
1-3, if pmaxWhen the output is 3,4 and 5, the output is obtained
Figure BDA00019921808200001013
Corresponding segmentation results;
2) the license plate to be tested is a double-row license plate
pmaxWhen the output is 6,7, the output is
Figure BDA00019921808200001014
And (4) corresponding segmentation results.
The license plate character segmentation method based on multiple templates and multiple proportions is adopted to perform character segmentation on multiple license plate images to be segmented, the result is shown in fig. 19-25 which are sequentially a type 1-7 license plate segmentation result graph, fig. 26 is a fuzzy type 1 license plate segmentation result graph, fig. 27 is an stained type 1 license plate segmentation result graph, fig. 28 is a fuzzy type 6 license plate segmentation result graph, and fig. 29 is a stained type 6 license plate segmentation result graph; as can be seen from the figure, the method can realize the accurate segmentation of the multi-class license plates and the fuzzy stained license plates.
While the invention has been described with reference to specific embodiments, any feature disclosed in this specification may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise; all of the disclosed features, or all of the method or process steps, may be combined in any combination, except mutually exclusive features and/or steps.

Claims (1)

1. A license plate character segmentation method based on multiple templates and multiple proportions comprises the following steps:
step 1, training single-double-row license plate judgment model
Taking a single-line license plate as a positive sample of training and a double-line license plate as a negative sample of training, extracting characteristics of the training samples, training a two-classification model based on a support vector machine based on the characteristics, and taking the two-classification model as a single-line and double-line license plate judgment model;
step 2, training scoring model
Taking the character images as positive samples of training and the non-character images as negative samples of training, extracting character features from the training samples, and training a scoring model based on a support vector machine based on the character features to serve as a scoring model;
step 3, training and distinguishing auxiliary model
Taking the first and last characters of the type 1 license plate, the type 2 license plate and the type 3 license plate as positive samples of training, taking an image in a character segmentation frame before the first character of the type 1 license plate and the type 2 license plate and taking an image in a character segmentation frame after the last character as negative samples; extracting character features from the training samples, and training a two-classification model based on a support vector machine based on the character features to serve as a discrimination auxiliary model;
step 4, constructing templates of various types and proportions
Aiming at a single-row license plate, constructing a multi-scale template: the 7 character boxes are represented from left to right as follows:
(0,0,αi·w,h)，(αi·w,0,αi·w,h)，(αi(L-5w),0,αi·w,h)，(αi(L-4w),0,αi·w,h)，(αi(L-3w),0,αi·w,h)，(αi(L-2w),0,αi·w,h)，(αi(L-w),0,αi·w,h)；
wherein h is the height of the standard template, w is the width of the character frame, L is the width of the standard template, alpha0,α1,…,αi,…,αn-1,αnRepresenting n scaling factors;
aiming at the double-row license plate template, constructing a multi-scale template:
first row: from left to right in sequence: (alphai·w2,0,αi·w1,h1)，(αi·(L-w2-w1),0,αi·w1,h1)，
A second row: from left to right in sequence: (0, h-h)2,αi·w2,h2)，(αi·w2,h-h2,αi·w2,h2)，(2(αi·w2),h-h2,αi·w2,h2)，(3(αi·w2),h-h2,αi·w2,h2)，(4(αi·w2),h-h2,αi·w2,h2)；
Wherein h is1Is the height, h, of the character box in the first line2Is the height, w, of the character box in the second line1Is the width, w, of the character box in the first line2Is the width of the character box in the second line;
step 5, adopting the single-double-row license plate judgment model trained in the step 1 to judge the single-double rows of license plates to be segmented; if the license plate is a single-row license plate, performing step 6; if the number plate is a double-row number plate, performing a step 8;
step 6, sliding scoring of the multi-class multi-scale templates of the single-row license plate
Step 6.1, calculating a normalization factor:
Figure FDA0001992180810000021
the license plate template is normalized by adopting the normalization factor, and the normalized size is as follows:
Figure FDA0001992180810000022
wherein, rows is the height of the input license plate image;
step 6.2, sliding scoring:
templates of the ith scale for class p templates
Figure FDA0001992180810000023
A sliding range of
Figure FDA0001992180810000024
Figure FDA0001992180810000025
Representing normalized templates
Figure FDA0001992180810000026
Cols is the width of the input license plate image;
as a template
Figure FDA0001992180810000027
When the image slides to the coordinate m, taking out a plurality of images segmented by the template, extracting the characteristics of the images, grading the images by adopting the grading model trained in the step 2, and calculating to obtain an average score
Figure FDA0001992180810000028
Selecting the highest score of the multi-proportion template in each type of template
Figure FDA0001992180810000029
Step 7, selecting the optimal segmentation result of the single-row license plate
Selecting a maximum value:
Figure FDA00019921808100000210
according to the above
Figure FDA00019921808100000211
Selecting an optimal segmentation result:
step 7-1-1, if pmaxFurther judgment is made as 1
Figure FDA00019921808100000212
Taking a character frame after the last segmentation frame of the corresponding segmentation result to determine whether the character frame exceeds the test image, if so, outputting the license plate to be segmented as a type 1 license plate
Figure FDA00019921808100000213
Corresponding segmentation results, otherwise, 7-1-2 is carried out;
step 7-1-2, judging by adopting a discrimination auxiliary model
Figure FDA00019921808100000214
After the last segmentation frame of the corresponding segmentation result, whether the content in a character frame is a character is selected; if yes, the license plate to be divided is a 3 rd type license plate, and output
Figure FDA00019921808100000215
Corresponding segmentation results; otherwise, carrying out 7-1-3;
step 7-1-3, adopting a discrimination auxiliary model to further judge
Figure FDA00019921808100000216
If the 1 st bit of the corresponding segmentation result is a character, outputting
Figure FDA00019921808100000217
Corresponding segmentation result, otherwise, outputting
Figure FDA00019921808100000218
Corresponding segmentation results;
step 7-2-1, if pmaxFurther judge 2
Figure FDA00019921808100000219
Taking a character frame before the first segmentation frame of the corresponding segmentation result to determine whether the character frame exceeds the test image, if so, outputting the license plate to be segmented as a type 2 license plate
Figure FDA0001992180810000031
Corresponding segmentation results, otherwise, 7-2-2 is carried out;
step 7-2-2, judging by adopting a discrimination auxiliary model
Figure FDA0001992180810000032
Before the first segmentation frame of the corresponding segmentation result, whether the content in a character frame is a character is selected; if yes, the license plate to be divided is a 3 rd type license plate, and output
Figure FDA0001992180810000033
Corresponding segmentation results; otherwise, output
Figure FDA0001992180810000034
Corresponding segmentation results;
step 7-3-1, if pmaxWhen the output is 3,4 and 5, the output is obtained
Figure FDA0001992180810000035
Corresponding segmentation results;
step 8, sliding scoring of multi-class multi-scale templates of double-row license plates
Step 8.1, firstly, normalizing the size of the template to the actual size of the license plate: the normalized sizes are:
Figure FDA0001992180810000036
step 8.2, sliding scoring, i proportion template for p class template
Figure FDA0001992180810000037
The same sliding scoring process in the step 6.2 is adopted to obtain the highest score of the multi-proportion template in each type of templates
Figure FDA0001992180810000038
Step 9, selecting an optimal segmentation result for the double-row license plate:
selecting a maximum value:
Figure FDA0001992180810000039
according to the above
Figure FDA00019921808100000310
Correspondingly outputting a segmentation result: p is a radical ofmax6,7, output
Figure FDA00019921808100000311
And (4) corresponding segmentation results.
CN201910183838.1A 2019-03-12 2019-03-12 License plate character segmentation method based on multiple templates and multiple proportions Active CN109993171B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910183838.1A CN109993171B (en) 2019-03-12 2019-03-12 License plate character segmentation method based on multiple templates and multiple proportions

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910183838.1A CN109993171B (en) 2019-03-12 2019-03-12 License plate character segmentation method based on multiple templates and multiple proportions

Publications (2)

Publication Number Publication Date
CN109993171A CN109993171A (en) 2019-07-09
CN109993171B true CN109993171B (en) 2022-05-03

Family

ID=67130565

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910183838.1A Active CN109993171B (en) 2019-03-12 2019-03-12 License plate character segmentation method based on multiple templates and multiple proportions

Country Status (1)

Country Link
CN (1) CN109993171B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111914845A (en) * 2020-07-02 2020-11-10 浙江大华技术股份有限公司 Character layering method and device in license plate and electronic equipment
CN112598005A (en) * 2020-12-24 2021-04-02 深圳市信义科技有限公司 Character segmentation method for license plate of armed police, storage medium and computer equipment

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103207998A (en) * 2012-12-24 2013-07-17 电子科技大学 License plate character segmentation method based on support vector machine
CN103268489A (en) * 2013-05-29 2013-08-28 电子科技大学 Motor vehicle plate identification method based on sliding window searching
CN104715252A (en) * 2015-03-12 2015-06-17 电子科技大学 License plate character segmentation method with combination of dynamic template and pixel points
CN105184317A (en) * 2015-08-29 2015-12-23 电子科技大学 License plate character segmentation method based on SVM classification
CN105447457A (en) * 2015-11-15 2016-03-30 安徽清新互联信息科技有限公司 License plate character identification method based on adaptive characteristic
CN106384113A (en) * 2016-11-07 2017-02-08 湖南源信光电科技有限公司 Odd-and-even-line license plate character dividing method based on projection and template matching
CN106709530A (en) * 2017-01-17 2017-05-24 中国科学院上海高等研究院 License plate recognition method based on video
CN108537226A (en) * 2018-03-06 2018-09-14 北京邮电大学 A kind of licence plate recognition method and device
CN108734170A (en) * 2018-05-25 2018-11-02 电子科技大学 Registration number character dividing method based on machine learning and template

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9996749B2 (en) * 2015-05-29 2018-06-12 Accenture Global Solutions Limited Detecting contextual trends in digital video content

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103207998A (en) * 2012-12-24 2013-07-17 电子科技大学 License plate character segmentation method based on support vector machine
CN103268489A (en) * 2013-05-29 2013-08-28 电子科技大学 Motor vehicle plate identification method based on sliding window searching
CN104715252A (en) * 2015-03-12 2015-06-17 电子科技大学 License plate character segmentation method with combination of dynamic template and pixel points
CN105184317A (en) * 2015-08-29 2015-12-23 电子科技大学 License plate character segmentation method based on SVM classification
CN105447457A (en) * 2015-11-15 2016-03-30 安徽清新互联信息科技有限公司 License plate character identification method based on adaptive characteristic
CN106384113A (en) * 2016-11-07 2017-02-08 湖南源信光电科技有限公司 Odd-and-even-line license plate character dividing method based on projection and template matching
CN106709530A (en) * 2017-01-17 2017-05-24 中国科学院上海高等研究院 License plate recognition method based on video
CN108537226A (en) * 2018-03-06 2018-09-14 北京邮电大学 A kind of licence plate recognition method and device
CN108734170A (en) * 2018-05-25 2018-11-02 电子科技大学 Registration number character dividing method based on machine learning and template

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
A new method for license plate characters recognition based on sliding window search;Ying-Jia Bu 等;《2013 IEEE 11th International Conference on Dependable, Autonomic and Secure Computing》;20140626;304-307 *
Multi-Modal Multi-Scale Deep Learning for Large-Scale Image Annotation;Yulei Niu 等;《arxiv平台在线公开:arxiv.org/pdf/1709.01220.pdf》;20181019;1-11 *
低质量监控视频中的车牌识别技术研究与实现;李宇琦;《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》;20190215(第02期);I140-235 *
车牌定位与字符分割算法的研究与实现;鲁飞;《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》;20110715(第07期);I138-607 *

Also Published As

Publication number Publication date
CN109993171A (en) 2019-07-09

Similar Documents

Publication Publication Date Title
CN105046196B (en) Front truck information of vehicles structuring output method based on concatenated convolutional neutral net
CN109101924B (en) Machine learning-based road traffic sign identification method
CN105354568A (en) Convolutional neural network based vehicle logo identification method
CN106650731B (en) Robust license plate and vehicle logo recognition method
CN103310211B (en) A kind ofly fill in mark recognition method based on image procossing
CN105787466B (en) A kind of fine recognition methods and system of type of vehicle
CN105261109A (en) Identification method of prefix letter of banknote
CN106709530A (en) License plate recognition method based on video
CN111539330B (en) Transformer substation digital display instrument identification method based on double-SVM multi-classifier
CN107886539B (en) High-precision gear visual detection method in industrial scene
CN108009567B (en) Automatic excrement character distinguishing method combining image color and HOG and SVM
CN113205026B (en) Improved vehicle type recognition method based on fast RCNN deep learning network
CN109993171B (en) License plate character segmentation method based on multiple templates and multiple proportions
CN107330365A (en) Traffic sign recognition method based on maximum stable extremal region and SVM
CN115393861B (en) Method for accurately segmenting handwritten text
Shah et al. OCR-based chassis-number recognition using artificial neural networks
CN110443235B (en) Intelligent paper test paper total score identification method and system
CN105354533A (en) Bag-of-word model based vehicle type identification method for unlicensed vehicle at gate
CN110689003A (en) Low-illumination imaging license plate recognition method and system, computer equipment and storage medium
CN113191348A (en) Template-based text structured extraction method and tool
CN115082922A (en) Water meter digital picture processing method and system based on deep learning
CN108734170B (en) License plate character segmentation method based on machine learning and template
CN110766009A (en) Tail plate identification method and device and computer readable storage medium
CN110766001B (en) Bank card number positioning and end-to-end identification method based on CNN and RNN
CN113158778A (en) SAR image target detection method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant