CN114663731A - Training method and system of license plate detection model, and license plate detection method and system - Google Patents
Training method and system of license plate detection model, and license plate detection method and system Download PDFInfo
- Publication number
- CN114663731A CN114663731A CN202210574127.9A CN202210574127A CN114663731A CN 114663731 A CN114663731 A CN 114663731A CN 202210574127 A CN202210574127 A CN 202210574127A CN 114663731 A CN114663731 A CN 114663731A
- Authority
- CN
- China
- Prior art keywords
- license plate
- classification
- prediction
- loss
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Abstract
The invention discloses a training method and a system of a license plate detection model, and a license plate detection method and a system, wherein the training method comprises the steps of generating error loss based on point loss values of all positive sample points in an iterative training process, and feeding back and updating a pre-training model based on the error loss; the step of calculating the point loss value of the positive sample point is: extracting a prediction label corresponding to the positive sample point; generating a classification loss value of the positive sample point under each license plate category based on the prediction label and the classification label corresponding to each license plate category; extracting corresponding license plate categories as target categories based on the classification loss values; obtaining the confidence coefficient that the positive sample point belongs to the target category, and taking the confidence coefficient as the corresponding loss weight; and performing loss calculation based on the prediction labels and the loss weight and the classification label corresponding to each target category to generate a corresponding point loss value. The design of the error loss can improve the fault tolerance rate of the wrong mark data, so that the robustness of the obtained license plate detection model is improved.
Description
Technical Field
The invention relates to the field of image recognition, in particular to a training method and a training system for a license plate detection model and a license plate detection method and a license plate detection system.
Background
The license plate detection comprises the detection of license plate types, wherein the license plate types comprise yellow plates, white plates, black plates, green plates, blue plates and the like, and the blue plates are the most common license plates;
nowadays, the collected images are often manually labeled to indicate the position and the type of the license plate, the conditions of label error and label missing inevitably exist in the process, and the performance of the trained license plate detection model is influenced by wrong labeling data.
Disclosure of Invention
Aiming at the defect that wrong labeling data can influence the performance of a trained license plate detection model in the prior art, the invention provides a training technology of the license plate detection model and also provides a technology for carrying out license plate detection on the license plate detection model obtained by training by using the training technology;
in order to solve the technical problems, the invention is solved by the following technical scheme:
a training method of a license plate detection model comprises the steps of generating error loss based on point loss values of all positive sample points in an iterative training process, and feeding back and updating a pre-training model based on the error loss; the input of the pre-training model is a sample image, and the output is classified prediction data;
the classified prediction data comprises a plurality of positive sample points and prediction labels corresponding to the positive sample points, wherein the prediction labels comprise confidence coefficients of the corresponding positive sample points belonging to various license plate categories;
according to the method and the device, the point loss values of all the positive sample points are calculated, the obtained point loss values are summed to obtain the error loss, the network parameters of the pre-training model are adjusted by using the error loss, the fault tolerance rate of the pre-training model to the false mark and the missed mark is improved, the stability of network training is improved, and finally the network robustness is improved.
The step of calculating the point loss value of the positive sample point is:
extracting a prediction label corresponding to the positive sample point;
generating a classification loss value of the positive sample point under each license plate category based on the prediction label and the classification label corresponding to each license plate category;
extracting corresponding license plate categories as target categories based on the classification loss values;
obtaining the confidence coefficient that the positive sample point belongs to the target class, and taking the confidence coefficient as a corresponding loss weight;
and performing loss calculation based on the prediction labels and the loss weight and the classification label corresponding to each target class to generate corresponding point loss values.
The pre-training model is a model trained to a convergence state in advance, after a network is in the convergence state, the classification loss value of a positive sample point under each license plate category is calculated, two categories with the highest classification loss values are extracted as target categories, the confidence coefficient of the positive sample point belonging to each license plate category can represent the output of network inference prediction, the confidence coefficient of the positive sample point belonging to the target categories is used as loss weight, the point loss value corresponding to the positive sample point is calculated, corresponding error loss is generated, and the penalty of the network for the prediction of wrong labeling data is reduced through the error loss.
As an implementation mode, before generating the classification loss value of the positive sample point under each license plate category, obtaining weight data, and based on the weight data, the prediction labels, and the classification labels corresponding to each license plate category, generating the classification loss value of the positive sample point under each license plate category;
the method for acquiring the weight data comprises the following steps:
classifying and counting the number of sample images in the current iterative training based on the license plate categories to obtain the number of sample images corresponding to each license plate category;
and generating dynamic weights corresponding to the license plate types one by one based on the number of the sample images to obtain weight data, wherein the dynamic weights corresponding to the license plate types are inversely related to the number of the sample images.
As an implementable manner, the specific step of extracting the corresponding license plate category as the target category based on the classification loss value is as follows:
and taking N license plate categories with the largest classification loss values as target categories, wherein N is more than or equal to 1 and less than M, and M represents the total number of the license plate categories.
As an implementable manner, the point loss value E is calculated by the formula:
where N represents the number of target classes,k n indicating the loss weight corresponding to the nth object class,Y n indicating the classification label corresponding to the nth object class,
a prediction label representing a target sample point.
As an implementation manner, the obtaining manner of the pre-training model includes:
carrying out iterative training on a preset license plate detection network based on a sample image until a preset iteration termination condition is met, and obtaining a pre-training model, wherein the iterative training comprises the following steps:
inputting a sample image into the license plate detection network, and outputting classification prediction data containing a plurality of prediction labels by the license plate detection network, wherein the classification prediction data comprises a plurality of sample points and prediction labels corresponding to the sample points, the sample points comprise positive sample points and negative sample points, the prediction labels are used for indicating confidence degrees that the corresponding sample points belong to classification categories, and the classification categories comprise backgrounds and license plate categories;
acquiring weight labels and acquiring labeling labels which are in one-to-one correspondence with the prediction labels;
and generating corresponding classification prediction loss based on the weight label, the prediction label and the labeling label, and performing feedback updating on the pre-training model based on the classification prediction loss.
As an implementation manner, the step of obtaining the weight label is:
judging whether the current iteration times exceed a preset stage time threshold;
when the weight of the current frame does not exceed the preset weight threshold, extracting a preset static weight matrix as a weight label;
when exceeding:
classifying and counting the number of sample images in the current iterative training based on the license plate categories to obtain the number of sample images corresponding to each license plate category;
and generating dynamic weights corresponding to the license plate types one by one based on the number of the sample images, and generating weight labels based on the dynamic weights.
As an implementation manner, the obtaining manner of the annotation tag includes:
acquiring annotation data corresponding to the sample image, wherein the annotation data comprises position information and category information of a corresponding license plate;
and determining the classification category of the sample point based on the labeling data, and taking the classification label corresponding to the classification category as the labeling label of the corresponding sample point.
The invention also provides a training system of the license plate detection model, which is used for carrying out iterative training on the pre-training model based on the sample image to obtain the license plate detection model, and the training system comprises a prediction module, a calculation module and an updating module;
the prediction module is used for inputting sample images into the pre-training model, outputting corresponding classification prediction data by the pre-training model, wherein the classification prediction data comprises a plurality of positive sample points and prediction labels corresponding to the positive sample points, and the prediction labels comprise confidence coefficients of the corresponding positive sample points belonging to the license plate categories;
the calculation module is used for calculating the point loss value of each positive sample point based on the classification label of each license plate category and the prediction label corresponding to each positive sample point, and is also used for generating error loss based on the point loss value;
the updating module is used for performing feedback updating on the pre-training model based on the error loss;
wherein the calculation module comprises:
the first extraction unit is used for extracting a prediction label corresponding to the positive sample point;
the first calculation unit is used for generating a classification loss value of the positive sample point under each license plate category based on the prediction label and the classification label corresponding to each license plate category;
the screening unit is used for extracting corresponding license plate categories as target categories based on the classification loss values;
the second extraction unit is used for acquiring the confidence coefficient that the positive sample point belongs to the target class, and taking the confidence coefficient as the corresponding loss weight;
and the second calculation unit is used for performing loss calculation on the basis of the prediction label and the loss weight and the classification label corresponding to each target class to generate a corresponding point loss value.
The invention also provides a license plate detection method, which comprises the following steps:
acquiring an image to be detected, inputting the image to be detected into a license plate detection model, outputting corresponding classification prediction data by the license plate detection model, and training the license plate detection model by adopting any one of the training methods to obtain the license plate detection model;
and generating and outputting a corresponding detection result based on the classification prediction data.
The invention also provides a license plate detection system, which comprises:
the detection module is used for acquiring an image to be detected, inputting the image to be detected into a license plate detection model, outputting corresponding classified prediction data by the license plate detection model, and training the license plate detection model by adopting any one of the training methods to obtain the license plate detection model;
and the analysis module is used for generating and outputting a corresponding detection result based on the classification prediction data.
Due to the adoption of the technical scheme, the invention has the remarkable technical effects that:
1. according to the method, the error loss is calculated, and the pre-training model is subjected to feedback updating by utilizing the error loss, so that the influence of error mark and missing mark data on the pre-training model in the marking process is reduced, and the robustness of the pre-training model is improved.
2. According to the invention, through the design of the dynamic weight corresponding to each license plate category, the influence of sample unbalance on classification loss can be reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a point loss value calculation method according to the present invention;
FIG. 2 is a schematic diagram of the module connection of a training system for a license plate detection model according to the present invention;
fig. 3 is a block diagram of the computing block 200 of fig. 2.
Detailed Description
The present invention will be described in further detail with reference to examples, which are illustrative of the present invention and are not to be construed as being limited thereto.
Embodiment 1, a training method of a license plate detection model, comprising the following steps:
s100, acquiring training data:
the training data comprises a sample image and marking data corresponding to the sample image, wherein the marking data comprises position information and category information of a corresponding license plate, the position information is a license plate frame in the embodiment, the category information is a classification label indicating the category of the license plate, and the category of the license plate comprises a blue plate, a green plate, a yellow plate, a black plate and a white plate;
in this embodiment, the classification label is a one-hot code, and the classification label further includes a label indicating a background, that is, the classification category includes a background and various license plate categories.
The obtaining mode of the sample image in this embodiment includes:
acquiring an image in a natural scene to obtain an original image; for example, images collected by a monitoring camera include a monitoring camera for road monitoring and a monitoring camera for parking lot monitoring;
preprocessing an original image to obtain a plurality of sample images, wherein the preprocessing mode comprises random scaling, random shearing, image enhancement and normalization processing, and specifically comprises the following steps:
randomly scaling the original image to adapt to license plates of different sizes;
randomly cutting the scaled original image based on a preset resolution (256 × 256) to obtain a plurality of image blocks, and enabling the image blocks not containing the complete license plate image to serve as negative samples to increase negative sample learning of a network;
carrying out data enhancement operations such as Gaussian blur, brightness, turnover, Cutout and the like on the obtained image block randomly to obtain an enhanced image block;
and carrying out normalization processing on each enhanced image block to obtain a sample image.
S200, obtaining a pre-training model;
the pre-training model is a license plate detection model which is obtained by training based on any one of the existing disclosed training methods and can identify the license plate type;
the pre-training model can be a license plate detection model obtained by training based on a construction method disclosed in a patent publication with the publication number of CN111310850A, for example;
s300, performing iterative training on the pre-training model obtained in the step S200 by using the training data obtained in the step S100 to obtain a license plate detection model; the iterative training process is as follows:
s310, inputting the sample images of the current batch into the pre-training model, and outputting corresponding classification prediction data by the pre-training model;
the classification prediction data comprises a hot spot diagram and classification prediction data which correspond to each other, wherein the hot spot diagram comprises a plurality of sample points, the sample points are mapped with pixel points of a sample image, the classification prediction data comprises prediction labels which correspond to the sample points one by one, and the prediction labels are used for indicating the probability that the sample points belong to each classification class.
S320, calculating a point loss value based on the classification label of each license plate category and the prediction label, and generating error loss based on the point loss value;
the sample points are divided into positive sample points and negative sample points, the point loss value is an error loss value corresponding to each positive sample point, and the error loss is the sum of the loss values of the points;
the positive sample points are sample points for which the label indicates the license plate category, for example:
determining the region of the license plate in the sample image and the license plate type of the license plate based on the labeling data, and taking the classification label corresponding to the license plate type as the labeling label of the sample point corresponding to all pixel points in the region;
the hard positive sample can also be obtained based on the positive and negative hard sample mining method disclosed in the patent publication No. CN111310850A, and the hard positive sample is used as a positive sample point in the present application to perform error loss calculation.
The selection method of the positive sample point can be set by a person skilled in the art according to actual conditions, and the embodiment does not limit the method in detail.
Referring to fig. 1, the step of calculating the point loss value of the positive sample point is:
s321, extracting a prediction label corresponding to the positive sample point;
s322, generating a classification loss value of the positive sample point under each license plate category based on the prediction label and the classification label corresponding to each license plate category;
respectively taking each license plate type as a real type of the positive sample point, and calculating to obtain a classification loss value of the positive sample point under the corresponding license plate type based on the dynamic weight of the corresponding license plate type and the confidence coefficient that the corresponding positive sample point belongs to the license plate type;
for example, the cross entropy loss function is used to calculate the loss values of the prediction label of the positive sample point and the label corresponding to the blue tile, and the obtained loss values are weighted by using the dynamic weight corresponding to the blue tile, so as to obtain the classification loss value of the positive sample point under the blue tile.
S323, extracting corresponding license plate categories as target categories based on the classification loss values;
for example, a license plate category of which the classification loss value exceeds a preset threshold value may be selected as a target category, one or more license plate categories of which the classification loss value is the largest may also be selected as a target category, and N license plate categories of which the classification loss value is the largest are selected as target categories, where N is greater than or equal to 1 and less than M, and M represents the total number of the license plate categories, where N is 2 in this embodiment;
that is, the license plate categories are arranged in the order of the classification loss values calculated in step S322 from large to small, and the first 2 license plate categories are extracted as the target categories of the current positive sample points.
S324, obtaining a confidence coefficient that the positive sample point belongs to the target class, and taking the confidence coefficient as a corresponding loss weight;
s325, performing loss calculation based on the prediction labels and the loss weights and the classification labels corresponding to the target classes to generate corresponding point loss values, wherein the calculation formula of the point loss value E is as follows;
where N represents the number of target classes,k n indicating the loss weight corresponding to the nth object class,Y n indicating the classification label corresponding to the nth object class,
a prediction label representing a target sample point.
And S330, updating the feedback of the pre-training model based on the error loss.
The training mode disclosed by the application is that on the basis of the existing disclosed training method of the vehicle detection model, the error loss is increased, and the pre-training model is fed back and updated by using the error loss, so that the influence of error mark and missing mark data on the pre-training model in the marking process is reduced, and the robustness of the pre-training model is improved.
Data detected by a target usually needs to be marked artificially, and situations such as missing marks or wrong marks are inevitable; the network loss mutation can be caused in the training process due to the mislabeling, so that the network training is unstable, and the learning of positive samples is influenced; although the network convergence can be guaranteed to be normal without dynamic correction, the confidence coefficient of partial positive samples is influenced to be lower by wrong labeling labels finally, so that the overall generalization capability of the network is not optimal, and the robustness of the network is lower;
the implementation fine-tunes the network parameters of the pre-training model based on the error loss, that is, a small learning rate (in the embodiment, the learning rate is 0.0001) is adopted, and the target category is dynamically adjusted according to the output of the network, so that the network corrects the error label through the error loss.
Further, step S321 further includes a step of extracting weight data;
in this embodiment, the manner of obtaining the weight data includes:
classifying and counting the number of sample images in the current iterative training based on the license plate categories to obtain the number of sample images corresponding to each license plate category;
based on the number of each sample image, generating dynamic weights corresponding to the license plate types one to obtain weight data, wherein the dynamic weights corresponding to the license plate types are inversely related to the number of the sample images, and specifically the method comprises the following steps:
judging whether the number of each sample image is greater than 0, and updating the number of the sample images by using a preset image number (such as 1) when the number of the sample images is less than or equal to 0;
secondly, calculating a first parameter corresponding to each license plate type based on the number of the sample images, wherein the calculation formula is as follows:
wherein the content of the first and second substances,effective m representing a first parameter corresponding to the mth license plate type, wherein M is more than or equal to 1 and less than or equal to M, and M represents the total number of types of the license plate type; theta is a constant and takes a value of 0.99,a m and representing the number of sample images corresponding to the mth license plate type.
Thirdly, calculating the category weight of the corresponding license plate category based on the first parameter, wherein the calculation formula is as follows:
wherein, the first and the second end of the pipe are connected with each other,per m and representing the category weight corresponding to the mth license plate category.
Calculating the dynamic weight of each license plate category based on the weight of each category, wherein the calculation formula is as follows:
wherein the content of the first and second substances,weight m and representing the dynamic weight corresponding to the mth license plate category, wherein M represents the total category number of the license plate categories, and x represents multiplication operation.
Further, in step S322, based on the weight data extracted in step S321, the prediction labels, and the classification labels corresponding to the license plate categories, a classification loss value of the positive sample point under each license plate category is generated;
the calculation formula of the classification loss value is as follows:
wherein, the first and the second end of the pipe are connected with each other,loss m the classification loss value corresponding to the mth license plate class of the positive sample point is represented, M represents the total number of the license plate classes,weight m represents the dynamic weight corresponding to the mth license plate category,Y m represents the classification label corresponding to the mth license plate category,
a prediction label representing the positive sample point.
Embodiment 2, on the basis of embodiment 1, a method for obtaining a pre-training model is added, and the rest is the same as embodiment 1;
the pre-training model is obtained by performing iterative training on a preset license plate detection network based on the training data obtained in the step S100 until a preset iteration termination condition is met, and obtaining a pre-training model;
the license plate detection network can adopt a first convolutional neural network disclosed in a patent publication with a publication number of CN 111310850A;
the iteration termination condition is that the iteration times reach the preset iteration termination times.
The iterative training comprises the following steps:
s210, inputting the sample image into the license plate detection network, and outputting corresponding classification prediction data by the license plate detection network;
the classification prediction data comprises a hot spot diagram and classification prediction data which correspond to each other, wherein the hot spot diagram comprises a plurality of sample points, the sample points are mapped with pixel points of a sample image, the classification prediction data comprises prediction labels which correspond to the sample points one by one, the prediction labels are used for indicating the probability that the sample points belong to each classification category, and the classification categories comprise a background and each license plate category;
the license plate detection network, the pre-training model and the license plate detection model belong to different training stages of the same network, so that input data and output data of the license plate detection network, the pre-training model and the license plate detection model are consistent.
S220, obtaining weight labels and obtaining label labels corresponding to the prediction labels one by one;
s221, obtaining a weight label, wherein the weight label is a weight matrix and comprises weight coefficients corresponding to all classification categories (backgrounds and all classification categories);
the method comprises the following specific steps:
judging whether the current iteration times exceed a preset stage time threshold value;
the stage time threshold is smaller than the iteration termination threshold, which is 180 in this embodiment and 200 in this embodiment;
secondly, when the weight is not exceeded, extracting a preset static weight matrix as a weight label;
that is, the weight coefficients corresponding to the background and each license plate category are preset fixed values, and a person skilled in the art can set a static weight matrix according to actual needs, in this embodiment, each weight coefficient in the static weight matrix is 1.
Thirdly, when the license plate exceeds the preset license plate category, calculating a weight coefficient corresponding to each license plate category, and generating a weight label based on each obtained dynamic weight;
the weight coefficient corresponding to the background in the weight label is a preset fixed value 1, and the weight coefficient corresponding to each license plate type is a dynamic weight obtained through calculation;
the method specifically comprises the following steps:
classifying and counting the number of sample images in the current iterative training based on the license plate categories to obtain the number of sample images corresponding to each license plate category;
and generating dynamic weights corresponding to the license plate types one by one based on the number of the sample images, and generating weight labels based on the dynamic weights.
S220, generating corresponding classification prediction loss based on the weight label, the prediction label and the labeling label;
categorizing predictive lossesE Loss_c The calculation formula of (2) is as follows:
wherein weight represents a weight matrix, which includes a weight coefficient 1 corresponding to the background and a dynamic weight corresponding to each license plate category,
a label representing the ith sample point,
a prediction label representing the ith sample point.
And S230, performing feedback updating on the pre-training model based on the classification prediction loss.
In the license plate detection scene, the number of sample images corresponding to the license plate category is unbalanced, for example, a blue plate is the most common license plate, the number of sample images corresponding to the blue plate is also large, while a white plate and a black plate are rare, and the number of sample images is small; the number of sample images corresponding to various license plate categories is unbalanced, so that the detection effect is unsatisfactory, and the error rate is high.
In the prior art, the problem of unbalanced samples is often solved by the following three ways:
the first method is to oversample a small amount of sample data, for example, oversampling is performed on an image block corresponding to a black card, and this method may cause a part of the small amount of sample data to appear repeatedly, and the trained model has an overfitting risk.
And secondly, under-sampling multi-sample data, for example, under-sampling image blocks corresponding to blue tiles, the scheme can cause part of data to be lost, and the trained model only learns part of the overall pattern.
Setting fixed weight in advance according to the number of samples, namely configuring static weight data, wherein the scheme is suitable for scenes that the number of labels is fixed and the like in the training process of data;
however, in a license plate detection scene, operations such as scaling and random cropping are generally required to be performed on targets so as to adapt to the fact that an image acquired in an actual detection scene contains a plurality of license plates and the images are characterized by different sizes of the license plates based on the principle of large and small numbers, the random scaling and cropping operations cause the number of uncertain targets in a sample image, a scheme of configuring static weight data influences network learning on positive samples, the attention of a few targets is not paid, the attention of multiple samples is excessive, and the problem that the attention of the targets during the whole network training period is unclear is caused.
Aiming at the problems, the embodiment trains the license plate detection network in sections;
a first stage (when the iteration times do not exceed a stage time threshold), wherein a static weight matrix is used for loss calculation;
and in the second stage (when the iteration times exceed the stage time threshold), in each iteration training process, calculating the weight corresponding to each license plate type according to the classification condition of the adopted sample images, and dynamically adjusting the weight along with the difference of the sample images adopted in each iteration training so as to make the network training target more definite.
Embodiment 3 discloses a training system for a license plate detection model, which is configured to iteratively train a pre-training model based on a sample image to obtain a license plate detection model, and as shown in fig. 2, the training system includes a prediction module 100, a calculation module 200, and an update module 300;
the prediction module 100 is configured to input a sample image into the pre-training model, and output corresponding classification prediction data by the pre-training model, where the classification prediction data includes a plurality of positive sample points and prediction labels corresponding to the positive sample points, and the prediction labels include confidence coefficients that the corresponding positive sample points belong to each license plate category;
the calculating module 200 is configured to calculate a point loss value of each positive sample point based on the classification label of each license plate category and the prediction label corresponding to each positive sample point, and is further configured to generate an error loss based on the point loss value;
the updating module 300 is configured to perform feedback updating on the pre-training model based on the error loss;
referring to fig. 3, the calculation module 200 includes:
a first extraction unit 210, configured to extract a prediction label corresponding to a positive sample point;
a first calculating unit 220, configured to generate a classification loss value of the positive sample point in each license plate category based on the prediction label and a classification label corresponding to each license plate category;
a screening unit 230, configured to extract a corresponding license plate category as a target category based on the classification loss value;
a second extracting unit 240, configured to obtain a confidence that the positive sample point belongs to the target class, and use the confidence as a corresponding loss weight;
a second calculating unit 250, which performs loss calculation based on the prediction labels and the loss weights and classification labels corresponding to the target classes to generate corresponding point loss values;
the third calculating unit 260 calculates and obtains the corresponding error loss based on the obtained loss value of each point.
Further, the calculation module 200 further includes a weight calculation unit;
the weight calculation unit:
the license plate classification and statistics device is used for classifying and counting the number of sample images in current iterative training based on license plate classes to obtain the number of sample images corresponding to each license plate class;
and the license plate classification generating unit is further used for generating dynamic weights corresponding to the license plate classifications one to one based on the number of each sample image to obtain weight data, wherein the dynamic weights corresponding to the license plate classifications are inversely related to the number of the sample images.
The first extracting unit 210 is further configured to extract the weight data;
the first calculating unit 220 is further configured to generate a classification loss value of the positive sample point under each license plate category based on the weight data, the prediction labels, and the classification labels corresponding to each license plate category.
Further, in training the pre-training model:
the prediction module 100 is configured to input a sample image into the license plate detection network, and the license plate detection network outputs classified prediction data including a plurality of prediction labels, where the classified prediction data includes a plurality of sample points and prediction labels corresponding to the sample points, where the sample points include positive sample points and negative sample points, the prediction labels are used to indicate confidence levels that the corresponding sample points belong to classification categories, and the classification categories include a background and license plate categories;
a first extracting unit 210, configured to extract a weight label and a label corresponding to the prediction label; a first calculating unit 220, configured to generate a corresponding classification prediction loss based on the weight label, the prediction label and the labeling label;
the updating module 300 is further configured to perform feedback updating on the pre-training model based on the classification prediction loss.
Further, the calculation module 200 further includes a count judgment unit;
the counting judgment unit is used for recording the current iteration times and judging whether the current iteration times exceed a preset stage time threshold;
the weight calculation unit is used for calculating the dynamic weight corresponding to each license plate type when the current iteration number exceeds the stage number threshold value and generating a weight label;
the first extracting unit 210 is configured to extract a preset static weight matrix as a weight label when the current iteration number does not exceed the stage number threshold, and is further configured to extract the weight label generated by the weight calculating unit when the current iteration number exceeds the stage number threshold.
This embodiment is an apparatus embodiment corresponding to method embodiment 1 and method embodiment 2, and since it is basically similar to the method embodiment, the description is relatively simple, and for the relevant points, reference may be made to the partial description of the method embodiment.
Embodiment 4, a license plate detection method, comprising the steps of:
acquiring an image to be detected, inputting the image to be detected into a license plate detection model, and outputting corresponding classification prediction data by the license plate detection model, wherein the license plate detection model is a license plate detection model obtained by training by adopting the training method in any one of embodiment 1 or embodiment 2;
and generating and outputting a corresponding detection result based on the classification prediction data.
Embodiment 5, a license plate detection system includes:
the system comprises a detection module, a license plate detection module and a recognition module, wherein the detection module is used for acquiring an image to be detected and inputting the image to be detected into a license plate detection model, the license plate detection model outputs corresponding classification prediction data, and the license plate detection model is trained by the training method in any one of embodiment 1 and embodiment 2;
and the analysis module is used for generating and outputting a corresponding detection result based on the classification prediction data.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, terminal devices (systems), and computer program products according to the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing terminal to cause a series of operational steps to be performed on the computer or other programmable terminal to produce a computer implemented process such that the instructions which execute on the computer or other programmable terminal provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that:
while preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
In addition, it should be noted that the specific embodiments described in the present specification may be different in terms of the parts, the shapes of the components, the names of the components, and the like. All equivalent or simple changes of the structure, the characteristics and the principle of the invention which are described in the patent conception of the invention are included in the protection scope of the patent of the invention. Various modifications, additions and substitutions for the specific embodiments described may be made by those skilled in the art without departing from the scope of the invention as defined in the accompanying claims.
Claims (10)
1. A training method of a license plate detection model is characterized by comprising the steps of generating error loss based on point loss values of all positive sample points in an iterative training process, and feeding back and updating a pre-training model based on the error loss;
the input of the pre-training model is a sample image, and the output is classified prediction data;
the classified prediction data comprises a plurality of positive sample points and prediction labels corresponding to the positive sample points, wherein the prediction labels comprise confidence coefficients of the corresponding positive sample points belonging to various license plate categories;
the step of calculating the point loss value of the positive sample point is:
extracting a prediction label corresponding to the positive sample point;
generating a classification loss value of the positive sample point under each license plate category based on the prediction label and the classification label corresponding to each license plate category;
extracting corresponding license plate categories as target categories based on the classification loss values;
obtaining the confidence coefficient that the positive sample point belongs to the target class, and taking the confidence coefficient as a corresponding loss weight;
and performing loss calculation based on the prediction labels and the loss weight and the classification label corresponding to each target class to generate corresponding point loss values.
2. The training method of the license plate detection model according to claim 1, wherein before generating the classification loss value of the positive sample point under each license plate category, weight data is further obtained, and based on the weight data, the prediction label and the classification label corresponding to each license plate category, the classification loss value of the positive sample point under each license plate category is generated;
the method for acquiring the weight data comprises the following steps:
classifying and counting the number of sample images in the current iterative training based on the license plate categories to obtain the number of sample images corresponding to each license plate category;
and generating dynamic weights corresponding to the license plate types one by one based on the number of the sample images to obtain weight data, wherein the dynamic weights corresponding to the license plate types are inversely related to the number of the sample images.
3. The training method of the license plate detection model according to claim 1, wherein the specific steps of extracting the corresponding license plate category as the target category based on the classification loss value are as follows:
and taking N license plate categories with the largest classification loss values as target categories, wherein N is more than or equal to 1 and less than M, and M represents the total number of the license plate categories.
4. The method for training the license plate detection model of claim 3, wherein the calculation formula of the point loss value E is as follows:
where N represents the number of target classes,k n indicating the loss weight corresponding to the nth object class,Y n indicating the classification label corresponding to the nth object class,
a prediction label representing a target sample point.
5. The training method of the license plate detection model according to any one of claims 1 to 4, wherein the obtaining mode of the pre-training model comprises:
carrying out iterative training on a preset license plate detection network based on a sample image until a preset iteration termination condition is met, and obtaining a pre-training model, wherein the iterative training comprises the following steps:
inputting a sample image into the license plate detection network, and outputting classified prediction data containing a plurality of prediction labels by the license plate detection network, wherein the classified prediction data comprises a plurality of sample points and the prediction labels corresponding to the sample points, the sample points comprise positive sample points and negative sample points, the prediction labels are used for indicating the confidence degrees that the corresponding sample points belong to various classification categories, and the classification categories comprise backgrounds and license plate categories;
acquiring weight labels and acquiring labeling labels which are in one-to-one correspondence with the prediction labels;
and generating corresponding classification prediction loss based on the weight label, the prediction label and the labeling label, and performing feedback updating on the pre-training model based on the classification prediction loss.
6. The training method of the license plate detection model of claim 5, wherein the step of obtaining the weight label is as follows:
judging whether the current iteration times exceed a preset stage time threshold;
when the weight of the current frame does not exceed the preset weight threshold, extracting a preset static weight matrix as a weight label;
when exceeding:
classifying and counting the number of sample images in the current iterative training based on the license plate categories to obtain the number of sample images corresponding to each license plate category;
and generating dynamic weights corresponding to the license plate types one by one based on the number of the sample images, and generating weight labels based on the dynamic weights.
7. The training method of the license plate detection model of claim 5, wherein the label obtaining manner comprises:
acquiring annotation data corresponding to the sample image, wherein the annotation data comprises position information and category information of a corresponding license plate;
and determining the classification category of the sample point based on the labeling data, and taking the classification label corresponding to the classification category as the labeling label of the corresponding sample point.
8. A training system of a license plate detection model is characterized by being used for carrying out iterative training on a pre-trained model based on a sample image to obtain the license plate detection model, and comprising a prediction module, a calculation module and an updating module;
the prediction module is used for inputting sample images into the pre-training model, outputting corresponding classification prediction data by the pre-training model, wherein the classification prediction data comprises a plurality of positive sample points and prediction labels corresponding to the positive sample points, and the prediction labels comprise confidence coefficients of the corresponding positive sample points belonging to the license plate categories;
the calculation module is used for calculating a point loss value of each positive sample point based on the classification label of each license plate category and the prediction label corresponding to each positive sample point, and is also used for generating an error loss based on the point loss value;
the updating module is used for performing feedback updating on the pre-training model based on the error loss;
wherein the calculation module comprises:
the first extraction unit is used for extracting a prediction label corresponding to the positive sample point;
the first calculation unit is used for generating a classification loss value of the positive sample point under each license plate category based on the prediction label and the classification label corresponding to each license plate category;
the screening unit is used for extracting corresponding license plate categories as target categories based on the classification loss values;
the second extraction unit is used for acquiring the confidence coefficient that the positive sample point belongs to the target class, and taking the confidence coefficient as the corresponding loss weight;
and the second calculation unit is used for performing loss calculation on the basis of the prediction label and the loss weight and the classification label corresponding to each target class to generate a corresponding point loss value.
9. A license plate detection method is characterized by comprising the following steps:
acquiring an image to be detected, inputting the image to be detected into a license plate detection model, and outputting corresponding classification prediction data by the license plate detection model, wherein the license plate detection model is obtained by training by adopting the training method of any one of claims 1 to 7;
and generating and outputting a corresponding detection result based on the classification prediction data.
10. A license plate detection system, comprising:
the detection module is used for acquiring an image to be detected, inputting the image to be detected into a license plate detection model, and outputting corresponding classified prediction data by the license plate detection model, wherein the license plate detection model is a license plate detection model obtained by training by adopting the training method of any one of claims 1 to 7;
and the analysis module is used for generating and outputting a corresponding detection result based on the classification prediction data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210574127.9A CN114663731B (en) | 2022-05-25 | 2022-05-25 | Training method and system of license plate detection model, and license plate detection method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210574127.9A CN114663731B (en) | 2022-05-25 | 2022-05-25 | Training method and system of license plate detection model, and license plate detection method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114663731A true CN114663731A (en) | 2022-06-24 |
| CN114663731B CN114663731B (en) | 2022-09-20 |
Family
ID=82038378
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210574127.9A Active CN114663731B (en) | 2022-05-25 | 2022-05-25 | Training method and system of license plate detection model, and license plate detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114663731B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116137061A (en) * | 2023-04-20 | 2023-05-19 | 北京睿芯通量科技发展有限公司 | Training method and device for quantity statistical model, electronic equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111047507A (en) * | 2019-11-29 | 2020-04-21 | 北京达佳互联信息技术有限公司 | Training method of image generation model, image generation method and device |
| CN111160411A (en) * | 2019-12-11 | 2020-05-15 | 东软集团股份有限公司 | Classification model training method, image processing method, device, medium, and apparatus |
| CN112597995A (en) * | 2020-12-02 | 2021-04-02 | 浙江大华技术股份有限公司 | License plate detection model training method, device, equipment and medium |
| CN112906502A (en) * | 2021-01-29 | 2021-06-04 | 北京百度网讯科技有限公司 | Training method, device and equipment of target detection model and storage medium |
| WO2021179498A1 (en) * | 2020-03-11 | 2021-09-16 | 上海商汤智能科技有限公司 | Target detection method, method for training model therefor, apparatus, and electronic device |
-
2022
- 2022-05-25 CN CN202210574127.9A patent/CN114663731B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111047507A (en) * | 2019-11-29 | 2020-04-21 | 北京达佳互联信息技术有限公司 | Training method of image generation model, image generation method and device |
| CN111160411A (en) * | 2019-12-11 | 2020-05-15 | 东软集团股份有限公司 | Classification model training method, image processing method, device, medium, and apparatus |
| WO2021179498A1 (en) * | 2020-03-11 | 2021-09-16 | 上海商汤智能科技有限公司 | Target detection method, method for training model therefor, apparatus, and electronic device |
| CN112597995A (en) * | 2020-12-02 | 2021-04-02 | 浙江大华技术股份有限公司 | License plate detection model training method, device, equipment and medium |
| CN112906502A (en) * | 2021-01-29 | 2021-06-04 | 北京百度网讯科技有限公司 | Training method, device and equipment of target detection model and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| YUANZHOUHAN CAO ET AL.: "Estimating Depth From Monocular Images as Classification Using Deep Fully Convolutional Residual Networks", 《IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY》 * |
| 李海: "基于卷积神经网络的目标识别算法研究", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116137061A (en) * | 2023-04-20 | 2023-05-19 | 北京睿芯通量科技发展有限公司 | Training method and device for quantity statistical model, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114663731B (en) | 2022-09-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109272509B (en) | Target detection method, device and equipment for continuous images and storage medium | |
| CN111310850B (en) | License plate detection model construction method and system, license plate detection method and system | |
| CN109740689B (en) | Method and system for screening error labeling data of image semantic segmentation | |
| CN107516102B (en) | Method, device and system for classifying image data and establishing classification model | |
| CN109858327B (en) | Character segmentation method based on deep learning | |
| CN112307919B (en) | Improved YOLOv 3-based digital information area identification method in document image | |
| CN109740553B (en) | Image semantic segmentation data screening method and system based on recognition | |
| CN112529210A (en) | Model training method, device and computer readable storage medium | |
| CN112906794A (en) | Target detection method, device, storage medium and terminal | |
| CN109189965A (en) | Pictograph search method and system | |
| CN111310826A (en) | Method and device for detecting labeling abnormity of sample set and electronic equipment | |
| CN113344826A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN114663731B (en) | Training method and system of license plate detection model, and license plate detection method and system | |
| CN115082922A (en) | Water meter digital picture processing method and system based on deep learning | |
| CN111008953A (en) | Method and device for controlling quality of digital pathological image | |
| CN112784494B (en) | Training method of false positive recognition model, target recognition method and device | |
| CN111597845A (en) | Two-dimensional code detection method, device and equipment and readable storage medium | |
| CN111401360B (en) | Method and system for optimizing license plate detection model, license plate detection method and system | |
| CN111340041A (en) | License plate recognition method and device based on deep learning | |
| CN114445716B (en) | Key point detection method, key point detection device, computer device, medium, and program product | |
| CN115953744A (en) | Vehicle identification tracking method based on deep learning | |
| CN115439850A (en) | Image-text character recognition method, device, equipment and storage medium based on examination sheet | |
| CN115424250A (en) | License plate recognition method and device | |
| CN114187445A (en) | Method and device for recognizing text in image, electronic equipment and storage medium | |
| CN113283388A (en) | Training method, device and equipment of living human face detection model and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: Training method and system of license plate detection model, license plate detection method and system Effective date of registration: 20221211 Granted publication date: 20220920 Pledgee: Hangzhou Fuyang Sub branch of Zheshang Bank Co.,Ltd. Pledgor: Hangzhou xiongmai integrated circuit technology Co.,Ltd. Registration number: Y2022330003571 |
|
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 311422 4th floor, building 9, Yinhu innovation center, 9 Fuxian Road, Yinhu street, Fuyang District, Hangzhou City, Zhejiang Province Patentee after: Zhejiang Xinmai Microelectronics Co.,Ltd. Address before: 311400 4th floor, building 9, Yinhu innovation center, No.9 Fuxian Road, Yinhu street, Fuyang District, Hangzhou City, Zhejiang Province Patentee before: Hangzhou xiongmai integrated circuit technology Co.,Ltd. |