CN114663731B - Training method and system of license plate detection model, and license plate detection method and system - Google Patents
Training method and system of license plate detection model, and license plate detection method and system Download PDFInfo
- Publication number
- CN114663731B CN114663731B CN202210574127.9A CN202210574127A CN114663731B CN 114663731 B CN114663731 B CN 114663731B CN 202210574127 A CN202210574127 A CN 202210574127A CN 114663731 B CN114663731 B CN 114663731B
- Authority
- CN
- China
- Prior art keywords
- license plate
- classification
- prediction
- loss
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012549 training Methods 0.000 title claims abstract description 123
- 238000001514 detection method Methods 0.000 title claims abstract description 95
- 238000000034 method Methods 0.000 title claims abstract description 50
- 238000004364 calculation method Methods 0.000 claims abstract description 37
- 230000008569 process Effects 0.000 claims abstract description 11
- 238000002372 labelling Methods 0.000 claims description 16
- 239000011159 matrix material Substances 0.000 claims description 9
- 230000003068 static effect Effects 0.000 claims description 9
- 238000000605 extraction Methods 0.000 claims description 5
- 238000004458 analytical method Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 238000013461 design Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 14
- 238000004590 computer program Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000010187 selection method Methods 0.000 description 1
- 238000010008 shearing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a training method and a system of a license plate detection model, and a license plate detection method and a system, wherein the training method comprises the steps of generating error loss based on point loss values of all positive sample points in an iterative training process, and feeding back and updating a pre-training model based on the error loss; the step of calculating the point loss value of the positive sample point is: extracting a prediction label corresponding to the positive sample point; generating a classification loss value of the positive sample point under each license plate category based on the prediction label and the classification label corresponding to each license plate category; extracting corresponding license plate categories as target categories based on the classification loss values; obtaining the confidence coefficient that the positive sample point belongs to the target class, and taking the confidence coefficient as the corresponding loss weight; and performing loss calculation based on the prediction labels and the loss weight and the classification label corresponding to each target class to generate corresponding point loss values. The design of the error loss can improve the fault tolerance rate of the wrong mark data, so that the robustness of the obtained license plate detection model is improved.
Description
Technical Field
The invention relates to the field of image recognition, in particular to a license plate detection model training method and system and a license plate detection method and system.
Background
The license plate detection comprises the detection of license plate types, wherein the license plate types comprise yellow plates, white plates, black plates, green plates, blue plates and the like, and the blue plates are the most common license plates;
nowadays, the collected images are often manually labeled to indicate the position and the type of the license plate, the conditions of label error and label missing inevitably exist in the process, and the performance of the trained license plate detection model is influenced by wrong labeling data.
Disclosure of Invention
Aiming at the defect that wrong labeling data can influence the performance of a trained license plate detection model in the prior art, the invention provides a training technology of the license plate detection model and also provides a technology for carrying out license plate detection on the license plate detection model obtained by training by using the training technology;
in order to solve the technical problem, the invention is solved by the following technical scheme:
a training method of a license plate detection model comprises the steps of generating error loss based on point loss values of all positive sample points in an iterative training process, and feeding back and updating a pre-training model based on the error loss; the input of the pre-training model is a sample image, and the output is classified prediction data;
the classified prediction data comprises a plurality of positive sample points and prediction labels corresponding to the positive sample points, wherein the prediction labels comprise confidence coefficients of the corresponding positive sample points belonging to various license plate categories;
according to the method and the device, the point loss values of all the positive sample points are calculated, the obtained point loss values are summed to obtain the error loss, the network parameters of the pre-training model are adjusted by using the error loss, the fault tolerance rate of the pre-training model to the false mark and the missed mark is improved, the stability of network training is improved, and finally the network robustness is improved.
The step of calculating the point loss value of the positive sample point is:
extracting a prediction label corresponding to the positive sample point;
generating a classification loss value of the positive sample point under each license plate category based on the prediction label and the classification label corresponding to each license plate category;
extracting corresponding license plate categories as target categories based on the classification loss values;
obtaining the confidence coefficient that the positive sample point belongs to the target class, and taking the confidence coefficient as a corresponding loss weight;
and performing loss calculation based on the prediction labels and the loss weight and the classification label corresponding to each target class to generate corresponding point loss values.
The pre-training model is a model trained to a convergence state in advance, after a network is in the convergence state, the classification loss value of a positive sample point under each license plate category is calculated, two categories with the highest classification loss values are extracted as target categories, the confidence coefficient of the positive sample point belonging to each license plate category can represent the output of network inference prediction, the confidence coefficient of the positive sample point belonging to the target categories is used as loss weight, the point loss value corresponding to the positive sample point is calculated, corresponding error loss is generated, and the penalty of the network for the prediction of wrong labeling data is reduced through the error loss.
As an implementation mode, before generating the classification loss value of the positive sample point under each license plate category, obtaining weight data, and based on the weight data, the prediction labels, and the classification labels corresponding to each license plate category, generating the classification loss value of the positive sample point under each license plate category;
the method for acquiring the weight data comprises the following steps:
classifying and counting the number of sample images in the current iterative training based on the license plate categories to obtain the number of sample images corresponding to each license plate category;
and generating dynamic weights corresponding to the license plate types one by one based on the number of the sample images to obtain weight data, wherein the dynamic weights corresponding to the license plate types are inversely related to the number of the sample images.
As an implementable manner, the specific step of extracting the corresponding license plate category as the target category based on the classification loss value is as follows:
and taking N license plate categories with the largest classification loss values as target categories, wherein N is more than or equal to 1 and less than M, and M represents the total number of the license plate categories.
As an implementable manner, the point loss value E is calculated by the formula:
where N represents the number of target classes,k n representing the loss weight corresponding to the nth object class,Y n indicating the class label corresponding to the nth object class,
a prediction label representing a target sample point.
As an implementation manner, the obtaining manner of the pre-training model includes:
carrying out iterative training on a preset license plate detection network based on a sample image until a preset iteration termination condition is met, and obtaining a pre-training model, wherein the iterative training comprises the following steps:
inputting a sample image into the license plate detection network, and outputting classification prediction data containing a plurality of prediction labels by the license plate detection network, wherein the classification prediction data comprises a plurality of sample points and prediction labels corresponding to the sample points, the sample points comprise positive sample points and negative sample points, the prediction labels are used for indicating confidence degrees that the corresponding sample points belong to classification categories, and the classification categories comprise backgrounds and license plate categories;
acquiring weight labels and acquiring labeling labels which are in one-to-one correspondence with the prediction labels;
and generating corresponding classification prediction loss based on the weight label, the prediction label and the labeling label, and performing feedback updating on the pre-training model based on the classification prediction loss.
As an implementation manner, the step of obtaining the weight label is:
judging whether the current iteration times exceed a preset stage time threshold;
when the weight of the current frame does not exceed the preset weight threshold, extracting a preset static weight matrix as a weight label;
when exceeding:
classifying and counting the number of sample images in the current iterative training based on the license plate categories to obtain the number of sample images corresponding to each license plate category;
and generating dynamic weights corresponding to the license plate types one by one based on the number of the sample images, and generating weight labels based on the dynamic weights.
As an implementation manner, the obtaining manner of the annotation tag includes:
acquiring annotation data corresponding to the sample image, wherein the annotation data comprises position information and category information of a corresponding license plate;
and determining the classification category of the sample point based on the labeling data, and taking the classification label corresponding to the classification category as the labeling label of the corresponding sample point.
The invention also provides a training system of the license plate detection model, which is used for carrying out iterative training on the pre-training model based on the sample image to obtain the license plate detection model, and the training system comprises a prediction module, a calculation module and an updating module;
the prediction module is used for inputting sample images into the pre-training model, outputting corresponding classification prediction data by the pre-training model, wherein the classification prediction data comprises a plurality of positive sample points and prediction labels corresponding to the positive sample points, and the prediction labels comprise confidence coefficients of the corresponding positive sample points belonging to the license plate categories;
the calculation module is used for calculating the point loss value of each positive sample point based on the classification label of each license plate category and the prediction label corresponding to each positive sample point, and is also used for generating error loss based on the point loss value;
the updating module is used for performing feedback updating on the pre-training model based on the error loss;
wherein the calculation module comprises:
the first extraction unit is used for extracting a prediction label corresponding to the positive sample point;
the first calculation unit is used for generating a classification loss value of the positive sample point under each license plate category based on the prediction label and the classification label corresponding to each license plate category;
the screening unit is used for extracting corresponding license plate categories as target categories based on the classification loss values;
the second extraction unit is used for acquiring the confidence coefficient that the positive sample point belongs to the target class, and taking the confidence coefficient as the corresponding loss weight;
and the second calculation unit is used for performing loss calculation on the basis of the prediction label and the loss weight and the classification label corresponding to each target class to generate a corresponding point loss value.
The invention also provides a license plate detection method, which comprises the following steps:
acquiring an image to be detected, inputting the image to be detected into a license plate detection model, outputting corresponding classification prediction data by the license plate detection model, and training the license plate detection model by adopting any one of the training methods to obtain the license plate detection model;
and generating and outputting a corresponding detection result based on the classification prediction data.
The invention also provides a license plate detection system, which comprises:
the detection module is used for acquiring an image to be detected, inputting the image to be detected into a license plate detection model, outputting corresponding classified prediction data by the license plate detection model, and training the license plate detection model by adopting any one of the training methods to obtain the license plate detection model;
and the analysis module is used for generating and outputting a corresponding detection result based on the classification prediction data.
Due to the adoption of the technical scheme, the invention has the remarkable technical effects that:
1. according to the method, the error loss is calculated, and the pre-training model is subjected to feedback updating by utilizing the error loss, so that the influence of error mark and missing mark data on the pre-training model in the marking process is reduced, and the robustness of the pre-training model is improved.
2. According to the invention, through the design of the dynamic weight corresponding to each license plate category, the influence of sample unbalance on classification loss can be reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a point loss value calculation method according to the present invention;
FIG. 2 is a schematic diagram of the module connection of a training system for a license plate detection model according to the present invention;
fig. 3 is a block diagram of the computing block 200 of fig. 2.
Detailed Description
The present invention will be described in further detail with reference to examples, which are illustrative of the present invention and are not to be construed as being limited thereto.
Embodiment 1, a training method of a license plate detection model, comprising the following steps:
s100, acquiring training data:
the training data comprises a sample image and marking data corresponding to the sample image, wherein the marking data comprises position information and category information of a corresponding license plate, the position information is a license plate frame in the embodiment, the category information is a classification label indicating the category of the license plate, and the category of the license plate comprises a blue plate, a green plate, a yellow plate, a black plate and a white plate;
in this embodiment, the classification label is a one-hot code, and the classification label further includes a label indicating a background, that is, the classification category includes a background and various license plate categories.
The obtaining mode of the sample image in this embodiment includes:
acquiring an image in a natural scene to obtain an original image; for example, images collected by a monitoring camera include a monitoring camera for road monitoring and a monitoring camera for parking lot monitoring;
preprocessing an original image to obtain a plurality of sample images, wherein the preprocessing mode comprises random scaling, random shearing, image enhancement and normalization processing, and specifically comprises the following steps:
randomly scaling the original image to adapt to license plates of different sizes;
randomly cutting the scaled original image based on a preset resolution (256 × 256) to obtain a plurality of image blocks, and enabling the image blocks not containing the complete license plate image to serve as negative samples to increase negative sample learning of a network;
carrying out data enhancement operations such as Gaussian blur, brightness, turnover, Cutout and the like on the obtained image block randomly to obtain an enhanced image block;
and carrying out normalization processing on each enhanced image block to obtain a sample image.
S200, obtaining a pre-training model;
the pre-training model is a license plate detection model which is obtained by training based on any one of the existing disclosed training methods and can identify the license plate type;
the pre-training model can be a license plate detection model obtained by training based on a construction method disclosed in a patent publication with the publication number of CN111310850A, for example;
s300, performing iterative training on the pre-training model obtained in the step S200 by using the training data obtained in the step S100 to obtain a license plate detection model; the iterative training process is as follows:
s310, inputting the sample images of the current batch into the pre-training model, and outputting corresponding classification prediction data by the pre-training model;
the classification prediction data comprises a hot spot diagram and classification prediction data which correspond to each other, wherein the hot spot diagram comprises a plurality of sample points, the sample points are mapped with pixel points of a sample image, the classification prediction data comprises prediction labels which correspond to the sample points one by one, and the prediction labels are used for indicating the probability that the sample points belong to each classification class.
S320, calculating a point loss value based on the classification label of each license plate category and the prediction label, and generating error loss based on the point loss value;
the sample points are divided into positive sample points and negative sample points, the point loss value is an error loss value corresponding to each positive sample point, and the error loss is the sum of the loss values of the points;
the positive sample points are sample points for which the label indicates the license plate category, for example:
determining the region of the license plate in the sample image and the license plate type of the license plate based on the labeling data, and taking the classification label corresponding to the license plate type as the labeling label of the sample point corresponding to all pixel points in the region;
the hard positive sample can also be obtained based on the positive and negative hard sample mining method disclosed in the patent publication No. CN111310850A, and the hard positive sample is used as a positive sample point in the present application to perform error loss calculation.
The selection method of the positive sample point can be set by a person skilled in the art according to actual conditions, and the embodiment does not limit the method in detail.
Referring to fig. 1, the step of calculating the point loss value of the positive sample point is:
s321, extracting a prediction label corresponding to the positive sample point;
s322, generating a classification loss value of the positive sample point under each license plate category based on the prediction label and the classification label corresponding to each license plate category;
respectively taking each license plate type as a real type of the positive sample point, and calculating to obtain a classification loss value of the positive sample point under the corresponding license plate type based on the dynamic weight of the corresponding license plate type and the confidence coefficient that the corresponding positive sample point belongs to the license plate type;
for example, the cross entropy loss function is used to calculate the loss values of the prediction label of the positive sample point and the label corresponding to the blue tile, and the obtained loss values are weighted by using the dynamic weight corresponding to the blue tile, so as to obtain the classification loss value of the positive sample point under the blue tile.
S323, extracting corresponding license plate categories as target categories based on the classification loss values;
for example, a license plate category of which the classification loss value exceeds a preset threshold value may be selected as a target category, one or more license plate categories of which the classification loss value is the largest may also be selected as a target category, and N license plate categories of which the classification loss value is the largest are selected as target categories, where N is greater than or equal to 1 and less than M, and M represents the total number of the license plate categories, where N is 2 in this embodiment;
that is, the license plate categories are arranged in the order of the classification loss values calculated in step S322 from large to small, and the first 2 license plate categories are extracted as the target categories of the current positive sample points.
S324, obtaining a confidence coefficient that the positive sample point belongs to the target class, and taking the confidence coefficient as a corresponding loss weight;
s325, performing loss calculation based on the prediction labels and the loss weights and the classification labels corresponding to the target classes to generate corresponding point loss values, wherein the calculation formula of the point loss value E is as follows;
where N represents the number of target classes,k n representing the loss weight corresponding to the nth object class,Y n indicating the classification label corresponding to the nth object class,
a prediction label representing a target sample point.
And S330, updating the feedback of the pre-training model based on the error loss.
The training mode disclosed by the application is that on the basis of the existing disclosed training method of the vehicle detection model, the error loss is increased, and the pre-training model is fed back and updated by using the error loss, so that the influence of error mark and missing mark data on the pre-training model in the marking process is reduced, and the robustness of the pre-training model is improved.
Data detected by a target usually needs to be marked artificially, and situations such as missing marks or wrong marks are inevitable; the network loss mutation can be caused in the training process due to the mislabeling, so that the network training is unstable, and the learning of positive samples is influenced; although the network convergence can be guaranteed to be normal without dynamic correction, the confidence coefficient of partial positive samples is influenced to be lower by wrong labeling labels finally, so that the overall generalization capability of the network is not optimal, and the robustness of the network is lower;
the implementation fine-tunes the network parameters of the pre-training model based on the error loss, that is, a small learning rate (in the embodiment, the learning rate is 0.0001) is adopted, and the target category is dynamically adjusted according to the output of the network, so that the network corrects the error label through the error loss.
Further, step S321 further includes a step of extracting weight data;
in this embodiment, the manner of obtaining the weight data includes:
classifying and counting the number of sample images in the current iterative training based on the license plate categories to obtain the number of sample images corresponding to each license plate category;
based on the number of each sample image, generating dynamic weights corresponding to the license plate types one to obtain weight data, wherein the dynamic weights corresponding to the license plate types are inversely related to the number of the sample images, and specifically the method comprises the following steps:
judging whether the number of each sample image is greater than 0, and updating the number of the sample images by using a preset image number (such as 1) when the number of the sample images is less than or equal to 0;
secondly, calculating a first parameter corresponding to each license plate type based on the number of the sample images, wherein the calculation formula is as follows:
wherein,effective m representing a first parameter corresponding to the mth license plate type, wherein M is more than or equal to 1 and less than or equal to M, and M represents the total number of types of the license plate type; theta is a constant and takes a value of 0.99,a m and the number of sample images corresponding to the mth license plate category is represented.
Thirdly, calculating the category weight of the corresponding license plate category based on the first parameter, wherein the calculation formula is as follows:
wherein,per m and representing the category weight corresponding to the mth license plate category.
Calculating the dynamic weight of each license plate category based on the weight of each category, wherein the calculation formula is as follows:
wherein,weight m and representing the dynamic weight corresponding to the mth license plate category, wherein M represents the total category number of the license plate categories, and x represents multiplication operation.
Further, in step S322, based on the weight data extracted in step S321, the prediction labels, and the classification labels corresponding to the license plate categories, a classification loss value of the positive sample point under each license plate category is generated;
the calculation formula of the classification loss value is as follows:
wherein,loss m the classification loss value corresponding to the mth license plate class of the positive sample point is represented, M represents the total number of the license plate classes,weight m representing the dynamic weight corresponding to the mth license plate category,Y m the classification label corresponding to the mth license plate category is shown,
a prediction label representing the positive sample point.
Embodiment 2, on the basis of embodiment 1, a method for obtaining a pre-training model is added, and the rest is the same as embodiment 1;
the pre-training model is obtained by performing iterative training on a preset license plate detection network based on the training data obtained in the step S100 until a preset iteration termination condition is met, and obtaining a pre-training model;
the license plate detection network can adopt a first convolutional neural network disclosed in a patent publication with a publication number of CN 111310850A;
the iteration termination condition is that the iteration times reach the preset iteration termination times.
The iterative training comprises the following steps:
s210, inputting the sample image into the license plate detection network, and outputting corresponding classification prediction data by the license plate detection network;
the classification prediction data comprises a hot spot diagram and classification prediction data which correspond to each other, wherein the hot spot diagram comprises a plurality of sample points, the sample points are mapped with pixel points of a sample image, the classification prediction data comprises prediction labels which correspond to the sample points one by one, the prediction labels are used for indicating the probability that the sample points belong to each classification category, and the classification categories comprise a background and each license plate category;
the license plate detection network, the pre-training model and the license plate detection model belong to different training stages of the same network, so that input data and output data of the license plate detection network, the pre-training model and the license plate detection model are consistent.
S220, obtaining weight labels and obtaining label labels corresponding to the prediction labels one by one;
s221, obtaining a weight label, wherein the weight label is a weight matrix and comprises weight coefficients corresponding to various classification categories (backgrounds and various classification categories);
the method comprises the following specific steps:
judging whether the current iteration times exceed a preset stage time threshold value;
the stage time threshold is smaller than the iteration termination threshold, which is 180 in this embodiment and 200 in this embodiment;
secondly, when the weight is not exceeded, extracting a preset static weight matrix as a weight label;
that is, the weight coefficients corresponding to the background and each license plate category are preset fixed values, and a person skilled in the art can set a static weight matrix according to actual needs, in this embodiment, each weight coefficient in the static weight matrix is 1.
Thirdly, when the license plate exceeds the preset license plate category, calculating a weight coefficient corresponding to each license plate category, and generating a weight label based on each obtained dynamic weight;
the weight coefficient corresponding to the background in the weight label is a preset fixed value 1, and the weight coefficient corresponding to each license plate type is a dynamic weight obtained through calculation;
the method specifically comprises the following steps:
classifying and counting the number of sample images in the current iterative training based on the license plate categories to obtain the number of sample images corresponding to each license plate category;
and generating dynamic weights corresponding to the license plate types one by one based on the number of the sample images, and generating weight labels based on the dynamic weights.
S220, generating corresponding classification prediction loss based on the weight label, the prediction label and the labeling label;
categorizing predictive lossesE Loss_c The calculation formula of (2) is as follows:
wherein weight represents a weight matrix, which includes a weight coefficient 1 corresponding to the background and a dynamic weight corresponding to each license plate category,
a label representing the ith sample point,
a prediction tag representing the ith sample point.
And S230, performing feedback updating on the pre-training model based on the classification prediction loss.
In the license plate detection scene, the number of sample images corresponding to the license plate category is unbalanced, for example, a blue plate is the most common license plate, the number of sample images corresponding to the blue plate is also large, while a white plate and a black plate are rare, and the number of sample images is small; the number of sample images corresponding to various license plate categories is unbalanced, so that the detection effect is unsatisfactory, and the error rate is high.
In the prior art, the problem of unbalanced samples is often solved by the following three ways:
the first method is to oversample a small amount of sample data, for example, oversampling is performed on an image block corresponding to a black card, and this method may cause a part of the small amount of sample data to appear repeatedly, and the trained model has an overfitting risk.
And secondly, under-sampling multi-sample data, for example, under-sampling image blocks corresponding to blue tiles, the scheme can cause part of the data to be lost, and the trained model only learns part of the overall mode.
Setting fixed weight in advance according to the number of samples, namely configuring static weight data, wherein the scheme is suitable for scenes that the number of labels is fixed in the training process of data and the like;
however, in a license plate detection scene, operations such as scaling and random cropping are generally required to be performed on targets so as to adapt to the fact that an image acquired in an actual detection scene contains a plurality of license plates and the images are characterized by different sizes of the license plates based on the principle of large and small numbers, the random scaling and cropping operations cause the number of uncertain targets in a sample image, a scheme of configuring static weight data influences network learning on positive samples, the attention of a few targets is not paid, the attention of multiple samples is excessive, and the problem that the attention of the targets during the whole network training period is unclear is caused.
Aiming at the problems, the embodiment trains the license plate detection network in sections;
a first stage (when the iteration number does not exceed a stage number threshold), wherein a static weight matrix is used for loss calculation;
and in the second stage (when the iteration times exceed the stage time threshold), in each iteration training process, calculating the weight corresponding to each license plate type according to the classification condition of the adopted sample images, and dynamically adjusting the weight along with the difference of the sample images adopted in each iteration training so as to make the network training target more definite.
Embodiment 3, a training system of a license plate detection model, configured to perform iterative training on a pre-training model based on a sample image to obtain a license plate detection model, as shown in fig. 2, the training system includes a prediction module 100, a calculation module 200, and an update module 300;
the prediction module 100 is configured to input a sample image into the pre-training model, and output corresponding classification prediction data by the pre-training model, where the classification prediction data includes a plurality of positive sample points and prediction labels corresponding to the positive sample points, and the prediction labels include confidence coefficients that the corresponding positive sample points belong to each license plate category;
the calculation module 200 is configured to calculate a point loss value of each positive sample point based on the classification label of each license plate category and the prediction label corresponding to each positive sample point, and further configured to generate an error loss based on the point loss value;
the updating module 300 is configured to perform feedback updating on the pre-training model based on the error loss;
referring to fig. 3, the calculation module 200 includes:
a first extraction unit 210, configured to extract a prediction label corresponding to a positive sample point;
a first calculating unit 220, configured to generate a classification loss value of the positive sample point under each license plate category based on the prediction label and a classification label corresponding to each license plate category;
a screening unit 230, configured to extract a corresponding license plate category as a target category based on the classification loss value;
a second extracting unit 240, configured to obtain a confidence that the positive sample point belongs to the target class, and use the confidence as a corresponding loss weight;
a second calculating unit 250, which performs loss calculation based on the prediction labels and the loss weights and classification labels corresponding to the target classes to generate corresponding point loss values;
the third calculating unit 260 calculates and obtains the corresponding error loss based on the obtained loss value of each point.
Further, the calculation module 200 further includes a weight calculation unit;
the weight calculation unit:
the license plate classification and statistics device is used for classifying and counting the number of sample images in current iterative training based on license plate classes to obtain the number of sample images corresponding to each license plate class;
and the license plate classification generating unit is also used for generating dynamic weights corresponding to the license plate classifications one by one based on the number of each sample image to obtain weight data, wherein the dynamic weights corresponding to the license plate classifications are inversely related to the number of the sample images.
The first extracting unit 210 is further configured to extract the weight data;
the first calculating unit 220 is further configured to generate a classification loss value of the positive sample point under each license plate category based on the weight data, the prediction labels, and the classification labels corresponding to each license plate category.
Further, in training the pre-training model:
the prediction module 100 is configured to input a sample image into the license plate detection network, and the license plate detection network outputs classified prediction data including a plurality of prediction labels, where the classified prediction data includes a plurality of sample points and prediction labels corresponding to the sample points, where the sample points include positive sample points and negative sample points, the prediction labels are used to indicate confidence levels that the corresponding sample points belong to classification categories, and the classification categories include a background and license plate categories;
a first extracting unit 210, configured to extract a weight label and a label that corresponds to the prediction label one to one; a first calculating unit 220, configured to generate a corresponding classification prediction loss based on the weight label, the prediction label and the labeling label;
the updating module 300 is further configured to perform feedback updating on the pre-training model based on the classification prediction loss.
Further, the calculation module 200 further includes a count judgment unit;
the counting judgment unit is used for recording the current iteration times and judging whether the current iteration times exceed a preset stage time threshold;
the weight calculation unit is used for calculating the dynamic weight corresponding to each license plate type when the current iteration times exceed the stage time threshold value and generating a weight label;
the first extracting unit 210 is configured to extract a preset static weight matrix as a weight label when the current iteration number does not exceed the stage number threshold, and is further configured to extract the weight label generated by the weight calculating unit when the current iteration number exceeds the stage number threshold.
This embodiment is an apparatus embodiment corresponding to method embodiment 1 and method embodiment 2, and since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Embodiment 4, a license plate detection method, comprising the steps of:
acquiring an image to be detected, inputting the image to be detected into a license plate detection model, and outputting corresponding classification prediction data by the license plate detection model, wherein the license plate detection model is obtained by training by adopting the training method of any one of embodiment 1 or embodiment 2;
and generating and outputting a corresponding detection result based on the classification prediction data.
Embodiment 5, a license plate detection system includes:
the system comprises a detection module, a license plate detection module and a license plate detection module, wherein the detection module is used for acquiring an image to be detected, inputting the image to be detected into a license plate detection model, and outputting corresponding classification prediction data by the license plate detection model, and the license plate detection model is obtained by training by adopting the training method in any one of embodiment 1 or embodiment 2;
and the analysis module is used for generating and outputting a corresponding detection result based on the classification prediction data.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that:
while preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
In addition, it should be noted that the specific embodiments described in the present specification may differ in the shape of the components, the names of the components, and the like. All equivalent or simple changes of the structure, the characteristics and the principle of the invention which are described in the patent conception of the invention are included in the protection scope of the patent of the invention. Various modifications, additions and substitutions for the specific embodiments described may be made by those skilled in the art without departing from the scope of the invention as defined in the accompanying claims.
Claims (9)
1. A training method of a license plate detection model is characterized by comprising the steps of generating error loss based on point loss values of all positive sample points in an iterative training process, and feeding back and updating a pre-training model based on the error loss;
the input of the pre-training model is a sample image, and the output of the pre-training model is classified prediction data;
the classified prediction data comprises a plurality of positive sample points and prediction labels corresponding to the positive sample points, wherein the prediction labels comprise confidence coefficients of the corresponding positive sample points belonging to various license plate categories;
the step of calculating the point loss value of the positive sample point is:
extracting a prediction label corresponding to the positive sample point;
generating a classification loss value of the positive sample point under each license plate category based on the prediction label and the classification label corresponding to each license plate category;
extracting corresponding license plate categories as target categories based on the classification loss values;
obtaining the confidence coefficient that the positive sample point belongs to the target class, and taking the confidence coefficient as a corresponding loss weight;
performing loss calculation based on the prediction labels and the loss weight and the classification label corresponding to each target class to generate corresponding point loss values;
the method for acquiring the weight data comprises the following steps:
classifying and counting the number of sample images in the current iterative training based on the license plate categories to obtain the number of sample images corresponding to each license plate category;
and generating dynamic weights corresponding to the license plate types one by one based on the number of the sample images to obtain weight data, wherein the dynamic weights corresponding to the license plate types are inversely related to the number of the sample images.
2. The method for training the license plate detection model according to claim 1, wherein the specific steps of extracting the corresponding license plate class as the target class based on the classification loss value are as follows:
and taking N license plate categories with the largest classification loss values as target categories, wherein N is more than or equal to 1 and less than M, and M represents the total number of the license plate categories.
3. The training method of the license plate detection model according to claim 2, wherein the calculation formula of the point loss value E is as follows:
where N represents the number of target classes,k n indicating the loss weight corresponding to the nth object class,Y n indicating the classification label corresponding to the nth object class,
a prediction label representing a target sample point.
4. The method for training the license plate detection model according to any one of claims 1 to 3, wherein the pre-training model is obtained in a manner that:
carrying out iterative training on a preset license plate detection network based on a sample image until a preset iteration termination condition is met, and obtaining a pre-training model, wherein the iterative training comprises the following steps:
inputting a sample image into the license plate detection network, and outputting classified prediction data containing a plurality of prediction labels by the license plate detection network, wherein the classified prediction data comprises a plurality of sample points and the prediction labels corresponding to the sample points, the sample points comprise positive sample points and negative sample points, the prediction labels are used for indicating the confidence degrees that the corresponding sample points belong to various classification categories, and the classification categories comprise backgrounds and license plate categories;
acquiring weight labels and acquiring labeling labels corresponding to the prediction labels one by one;
and generating corresponding classification prediction loss based on the weight label, the prediction label and the labeling label, and performing feedback updating on the pre-training model based on the classification prediction loss.
5. The training method of the license plate detection model of claim 4, wherein the step of obtaining the weight label is as follows:
judging whether the current iteration times exceed a preset stage time threshold;
when the weight of the current frame does not exceed the preset weight threshold, extracting a preset static weight matrix as a weight label;
when exceeding:
classifying and counting the number of sample images in the current iterative training based on the license plate categories to obtain the number of sample images corresponding to each license plate category;
and generating dynamic weights corresponding to the license plate types one by one based on the number of the sample images, and generating weight labels based on the dynamic weights.
6. The training method of the license plate detection model of claim 4, wherein the obtaining manner of the label comprises:
acquiring annotation data corresponding to the sample image, wherein the annotation data comprises position information and category information of a corresponding license plate;
and determining the classification category of the sample point based on the labeling data, and taking the classification label corresponding to the classification category as the labeling label of the corresponding sample point.
7. A training system of a license plate detection model is characterized by being used for carrying out iterative training on a pre-trained model based on a sample image to obtain the license plate detection model, and comprising a prediction module, a calculation module and an updating module;
the prediction module is used for inputting sample images into the pre-training model, outputting corresponding classification prediction data by the pre-training model, wherein the classification prediction data comprises a plurality of positive sample points and prediction labels corresponding to the positive sample points, and the prediction labels comprise confidence coefficients of the corresponding positive sample points belonging to the license plate categories;
the calculation module is used for calculating the point loss value of each positive sample point based on the classification label of each license plate category and the prediction label corresponding to each positive sample point, and is also used for generating error loss based on the point loss value;
the updating module is used for performing feedback updating on the pre-training model based on the error loss;
wherein the calculation module comprises:
the first extraction unit is used for extracting a prediction label corresponding to the positive sample point;
the first calculation unit is used for generating a classification loss value of the positive sample point under each license plate category based on the prediction label and the classification label corresponding to each license plate category;
the screening unit is used for extracting corresponding license plate categories as target categories based on the classification loss values;
the second extraction unit is used for acquiring the confidence coefficient that the positive sample point belongs to the target class, and taking the confidence coefficient as the corresponding loss weight;
the second calculation unit is used for performing loss calculation on the basis of the prediction labels and the loss weights and the classification labels corresponding to the target classes to generate corresponding point loss values;
a weight calculation unit:
the license plate classification and statistics device is used for classifying and counting the number of sample images in current iterative training based on license plate classes to obtain the number of sample images corresponding to each license plate class;
and the license plate classification generating unit is also used for generating dynamic weights corresponding to the license plate classifications one by one based on the number of each sample image to obtain weight data, wherein the dynamic weights corresponding to the license plate classifications are inversely related to the number of the sample images.
8. A license plate detection method is characterized by comprising the following steps:
acquiring an image to be detected, inputting the image to be detected into a license plate detection model, and outputting corresponding classification prediction data by the license plate detection model, wherein the license plate detection model is obtained by training by adopting the training method of any one of claims 1 to 6;
and generating and outputting a corresponding detection result based on the classification prediction data.
9. A license plate detection system, comprising:
the detection module is used for acquiring an image to be detected, inputting the image to be detected into a license plate detection model, and outputting corresponding classified prediction data by the license plate detection model, wherein the license plate detection model is a license plate detection model obtained by training by adopting the training method of any one of claims 1 to 6;
and the analysis module is used for generating and outputting a corresponding detection result based on the classification prediction data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210574127.9A CN114663731B (en) | 2022-05-25 | 2022-05-25 | Training method and system of license plate detection model, and license plate detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210574127.9A CN114663731B (en) | 2022-05-25 | 2022-05-25 | Training method and system of license plate detection model, and license plate detection method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114663731A CN114663731A (en) | 2022-06-24 |
| CN114663731B true CN114663731B (en) | 2022-09-20 |
Family
ID=82038378
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210574127.9A Active CN114663731B (en) | 2022-05-25 | 2022-05-25 | Training method and system of license plate detection model, and license plate detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114663731B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116137061B (en) * | 2023-04-20 | 2023-08-01 | 北京睿芯通量科技发展有限公司 | Training method and device for quantity statistical model, electronic equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112597995A (en) * | 2020-12-02 | 2021-04-02 | 浙江大华技术股份有限公司 | License plate detection model training method, device, equipment and medium |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111047507B (en) * | 2019-11-29 | 2024-03-26 | 北京达佳互联信息技术有限公司 | Training method of image generation model, image generation method and device |
| CN111160411B (en) * | 2019-12-11 | 2023-09-29 | 东软集团股份有限公司 | Classification model training method, image processing method, device, medium and equipment |
| CN111508019A (en) * | 2020-03-11 | 2020-08-07 | 上海商汤智能科技有限公司 | Target detection method, training method of model thereof, and related device and equipment |
| CN112906502B (en) * | 2021-01-29 | 2023-08-01 | 北京百度网讯科技有限公司 | Training method, device, equipment and storage medium of target detection model |
-
2022
- 2022-05-25 CN CN202210574127.9A patent/CN114663731B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112597995A (en) * | 2020-12-02 | 2021-04-02 | 浙江大华技术股份有限公司 | License plate detection model training method, device, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114663731A (en) | 2022-06-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111310850B (en) | License plate detection model construction method and system, license plate detection method and system | |
| CN109272509B (en) | Target detection method, device and equipment for continuous images and storage medium | |
| CN109740689B (en) | Method and system for screening error labeling data of image semantic segmentation | |
| CN112508975A (en) | Image identification method, device, equipment and storage medium | |
| CN107516102B (en) | Method, device and system for classifying image data and establishing classification model | |
| CN109740553B (en) | Image semantic segmentation data screening method and system based on recognition | |
| CN112307919B (en) | Improved YOLOv 3-based digital information area identification method in document image | |
| CN111325067B (en) | Illegal video identification method and device and electronic equipment | |
| CN109858327B (en) | Character segmentation method based on deep learning | |
| CN112906794A (en) | Target detection method, device, storage medium and terminal | |
| CN112529210A (en) | Model training method, device and computer readable storage medium | |
| CN111597845A (en) | Two-dimensional code detection method, device and equipment and readable storage medium | |
| CN112465031B (en) | Data classification method, device and computer readable storage medium | |
| CN111310826A (en) | Method and device for detecting labeling abnormity of sample set and electronic equipment | |
| CN111340041A (en) | License plate recognition method and device based on deep learning | |
| CN114663731B (en) | Training method and system of license plate detection model, and license plate detection method and system | |
| CN113705294A (en) | Image identification method and device based on artificial intelligence | |
| CN111008953A (en) | Method and device for controlling quality of digital pathological image | |
| CN115731422A (en) | Training method, classification method and device of multi-label classification model | |
| CN115082922A (en) | Water meter digital picture processing method and system based on deep learning | |
| CN112804522B (en) | Method and device for detecting abnormal conditions of camera | |
| CN112784494B (en) | Training method of false positive recognition model, target recognition method and device | |
| CN111401360B (en) | Method and system for optimizing license plate detection model, license plate detection method and system | |
| CN113283388A (en) | Training method, device and equipment of living human face detection model and storage medium | |
| CN116137061A (en) | Training method and device for quantity statistical model, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Training method and system of license plate detection model, license plate detection method and system Effective date of registration: 20221211 Granted publication date: 20220920 Pledgee: Hangzhou Fuyang Sub branch of Zheshang Bank Co.,Ltd. Pledgor: Hangzhou xiongmai integrated circuit technology Co.,Ltd. Registration number: Y2022330003571 |
|
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 311422 4th floor, building 9, Yinhu innovation center, 9 Fuxian Road, Yinhu street, Fuyang District, Hangzhou City, Zhejiang Province Patentee after: Zhejiang Xinmai Microelectronics Co.,Ltd. Address before: 311400 4th floor, building 9, Yinhu innovation center, No.9 Fuxian Road, Yinhu street, Fuyang District, Hangzhou City, Zhejiang Province Patentee before: Hangzhou xiongmai integrated circuit technology Co.,Ltd. |