CN110363228A - Noise label correcting method - Google Patents

Abstract

The present invention provides a noise label re-labeling method, comprising the following steps: Step 1, use a base classifier to classify the observed samples and estimate the noise rate, and identify noise label data; Step 2, use the base classifier to classify the noise label samples Re-labeling is performed to obtain a clean sample dataset with the noisy label samples corrected.

Description

Noise Label Correction Method

technical field

The invention relates to a data mining technology, in particular to a noise label correction method.

Background technique

Traditional supervised learning classification problems usually assume that the labels of the dataset are complete, that is, noise-free correct labels exist for each dataset sample. However, in the real world, due to the randomness of the label labeling process, the sample labels are easily polluted by noise, resulting in inaccurate sample labels. The generation of noisy data is usually related to the way in which the dataset is obtained. For example, in the process of labeling the original data, the amount of sample data provided to the labeler is not enough to cause the labeler to misclassify the sample, or because the classification process itself is a subjective process or the labeler's expertise is not enough to ensure the correct classification sex. Various popular data annotation platforms are also one of the sources of noisy data. These annotation platforms utilize a large number of registered users to realize crowdsourced data annotation work. For example, Amazon's Amazon Mechanical Turk, Datatang, JD.com and other data service platforms. However, the data labels obtained in this way are not completely in line with the real situation due to the professional limitations or personal differences of the annotators, and different annotators may have different views on the same sample, resulting in different labeling results for the same sample. . The noise in the dataset can be divided into feature noise and label noise according to the location of the noise. Generally, the noise in the label has a greater impact on the model performance than the noise in the feature (Mirylenka K, Giannakopoulos G, Do L M, et al. On classifier behavior in the presence of mislabeling noise [J]. Data Mining and Knowledge Discovery, 2017). In binary classification, a PU (Positive-unlabeled) learning problem is proposed based on the characteristics of noise distribution in positive and negative datasets (Khetan A, Lipton Z C, Anandkumar A. Learning From Noisy Singly-labeled Data [J]. 2017 ). PU learning refers to a binary classification task in which only a portion of the positive training samples in the dataset are labeled and other samples are unlabeled. For the PU learning problem, all unlabeled samples can be regarded as negative samples. In this way, the PU learning problem is transformed into a noisy binary classification problem. The existence of noisy label data will not only have a serious negative impact on the classification accuracy of the classifier model, but also increase the complexity of the classifier. Therefore, it is of great research significance and application value to design a classification learning algorithm that adapts to noisy label data.

For classification problems with noisy labels, Frénay, B summarized a variety of solutions, including noise cleaning algorithms, noise label robust methods and noise label modeling methods (Frenay B, Verleysen M. Classification in the Presence of Label Noise: A Survey[J].IEEE Transactionson Neural Networks and Learning Systems, 2014). The noise label robust method uses the model's own ability to adapt to noise, and different models have different sensitivity to label noise. A classifier that is insensitive to label noise needs to be selected for learning. For example, in the empirical risk minimization problem of binary classification, a loss function is used to measure the loss of misclassification, and a classifier is learned by minimizing the minimum loss of the samples. Common losses are 0-1 losses. For uniform label noise, 0-1 loss and least squares loss are robust against noisy labels. And for other loss functions that are not anti-noise labels even in the case of uniform noise distribution, such as 1) exponential loss 2) logarithmic loss 3) hinge loss. Most learning algorithms in machine learning are not completely immune to noisy labels and only work well when the training data is disturbed by a small amount of label noise. With the development of deep learning, neural networks are often used in image classification problems to solve the problem of noisy label images. For example, Mnih proposed to incorporate the noise model into the neural network, but it only considers binary classification, and assumes that the noise belongs to symmetric label noise (MnihV, Hinton G. Learning to Label Aerial Images from Noisy Data [C]//International Conference on Machine Learning. 2013).

Solving noisy label learning problems using a noise cleaning strategy typically requires two steps: (1) estimating the noise rate and (2) using the noise rate and prediction. In order to estimate the noise rate, Scott et al. established a lower bound method for estimating the inversion noise rate and (Blanchard G, Flaska M, Handy G, et al. Classification with Asymmetric LabelNoise: Consistency and Maximal Denoising [J]. Journal of Machine Learning Research , 2013). However, the unbounded function obtained by this method may not converge. After adding additional assumptions, Scott (2015) proposed a time-efficient noise rate estimation method, but the estimation performance was poor (Scott C.A RateofConvergence for Mixture Proportion Estimation,with,Application to LearningfromNoisy Labels[J].2015) . Liu Tao rewrites and modifies the loss function through the importance weights, but the rewritten weights are derived from the predicted probability, so they may be sensitive to inaccurate estimates (Liu T, Tao D. Classification with Noisy Labels by Importance Reweighting [J] .IEEE Transactions on Pattern Analysis & Machine Intelligence, 2014). Natarajan (2013) did not propose a method for estimating noise but regarded the noise rate as a parameter optimized in the cross-validation process (Natarajan N, Dhillon I S, Ravikumar P K, et al. Learning with Noisy Labels [C]//International Conference on NeuralInformation Processing Systems. Curran Associates Inc. 2013). Natarajan proposes two methods to modify the loss function. The first method constructs an unbiased estimator of the correct distribution from the noise distribution, but the estimator may still be non-convex even when the original loss function is convex. The second approach builds a label-dependent loss function such that for a 0-1 loss, the minimum risk of Nat13 is equal to the risk of the correct distribution. Northcutt proposes the concept of learning with confident examples, calculates the equivalent variable value according to the classification probability of the noise data by the base classifier, and deletes the part to be identified according to the size of the prediction result of the base classifier for each sample is a sample of noisy label data, the process is called pruning by rank (Northcutt C G, Wu T, Chuang I L. Learning with Confident Examples: Rank Pruning for Robust Classification with NoisyLabels [J]. 2017).

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a noise label correction method.

The technical scheme for realizing the object of the present invention is: a noise label correction method, comprising the following steps:

Step 1, use the base classifier to predict the sample to obtain the sample prediction probability, take the expected value of the predicted probability of all samples in the positive example set and the negative example set respectively as the lower bound threshold and the upper bound threshold, and use the lower bound threshold and the upper bound threshold to judge the true value of the observed sample. label, identify noisy label data;

Step 2, use the base classifier to re-label the noise label samples to obtain a clean sample data set after the noise label samples have been corrected; wherein

In step 2, for the binary classification result, after identifying the noise label samples, sort the samples in ascending order according to the predicted probability value of each sample in the base classifier, and relabel the labels of the first a samples in the observed positive sample set. is 0; in the observed negative sample set, the sample labels are relabeled to 1;

In step 2, for the multi-class classification result, according to the classification result matrix predicted by the base classifier for all sample data, the probability matrix is used to relabel the label of the sample as the label to which the predicted probability is the largest except the current label.

Further, the specific steps of step 1 include:

Step 1.1, the base classifier predicts the sample to obtain the sample prediction probability g(x)=P(s=1|x); set

The noise rate ρ ₁ =P(s=0|y=1) represents the probability that a sample whose true label is 1 is mislabeled as 0,

represents the number of samples with observed label 1 and true label 1,

represents the number of samples with observed label 0 and true label 1,

represents the number of samples with observed label 1 and true label 0,

Indicates the number of samples whose observed label is 0 and the true label is 0;

Step 1.2, use the classification result of the base classifier Judging the true label of the sample: Use the lower threshold LB _y=1 to determine whether the true label of the sample is 1. When the prediction result of the observed sample on the base classifier g(x) is greater than the lower threshold, let the true label of the observed sample be 1; When the prediction result of the observed sample on the base classifier is less than the upper bound threshold UB _y=0 , set the true label of the observed sample to be 0.

Step 1.3, Calculate

in, is the observation positive sample set, In order to observe the negative sample set, the thresholds of the previous session and the next session are respectively set as the expected value of the classification probability g(x) of the positive and negative samples on the base classifier:

Step 1.4, Calculate an estimate of the noise rate and

Step 1.5, by Bayes' theorem, derive the value of the inverted noise rate from the estimated value of the noise rate

Step 1.6, set represents the number of samples whose true label is 0 in the observed positive sample set, Indicates the number of samples whose true label is 1 in the observed negative sample set, and sorts the samples in ascending order according to the predicted value of the base classifier g(x) for each sample; in the observed positive sample set middle, front The samples are regarded as noise label samples in the positive sample set; in the observation negative sample set middle, after The samples are regarded as noise label samples in the negative sample set.

Further, in step 2, the specific process of obtaining a clean sample data set after the noise label sample is corrected for the binary classification situation is as follows:

After identifying the noise label samples, the samples are sorted in ascending order according to the predicted probability value of each sample in the base classifier g(x)=P(s=1|x). In the observed positive sample set in the front The labels of the samples are relabeled as 0; in the observation negative sample set in, after sample labels are relabeled to 1;

Positive sample set after relabeling and negative sample set They are respectively expressed as:

in, Indicates the first g(x) value in the observed positive sample set small g(x) values, Indicates the first g(x) value of the observed negative sample set Large g(x) values.

Further, in the case of multi-class classification in step 2, the label re-labeling of the noise samples is used to obtain a clean sample data set after the noise label samples have been corrected. The specific process is as follows:

When the base classifier predicts all sample data, it needs to record the probability that the sample belongs to each category, and obtain the classification result matrix psx={p _ij |i∈N,j∈K}, psx is an N×K probability matrix, where N is the number of samples, K is the number of label types, among them, the probability value represents the classification result matrix of the base classification for all samples, and the i-th row of the matrix p _i =(p _i1 ,p _i2 ,,,p _ik ) indicates that the sample x _i is in The probability that the base classifier f(x) belongs to various labels, and the value p _ij represents the probability that the sample x _i belongs to the class k _j ;

When the sample x is determined to be a noise label, the probability matrix psx is used to relabel the label of x as the label to which the predicted probability is the largest except the current label:

y _i ^relabel =k _max (k _max =arg max psx _i )

Among them, k _max is the label category to which the maximum probability of the sample _xi in the classification probability of the base classifier, except for the original noise label _si of the sample, belongs.

Compared with the prior art, the present invention has the following advantages: (1) A general solution is proposed for noise label learning, which is suitable for any form of classifier; (2) The noise sample recognition rate is high, and all sample information is fully utilized to improve classification (3) The algorithm is suitable for binary classification and multi-class classification problems.

The present invention will be further described below with reference to the accompanying drawings.

Description of drawings

FIG. 1 is a schematic flow chart of the method of the present invention.

Figure 2 is a schematic diagram of the process of identifying noisy label data based on a base classifier.

Figure 3 is a schematic diagram of the re-labeling process of noise label samples.

Detailed ways

Combined with Figure 1, a method of classifying the observed samples and estimating the noise rate using the base classifier identifies the noise label data. The process is as follows:

Step 1, combined with Figure 2, use the base classifier to classify the observed samples and estimate the noise rate, and identify the noise label data. The process is as follows:

Step 1.1, the base classifier clf predicts the sample clf.fit(X,s), and obtains the sample prediction probability g(x)=P(s=1|x). The base classifier can choose any existing classification algorithm, as long as the predicted probability of the sample can be obtained.

For the noise rate ρ ₁ =P(s=0|y=1), it represents the probability that the sample with the true label of 1 is incorrectly labeled as 0, that is, the proportion of the number of samples with the observed label of 0 in the sample set with the correct label of 1. The number of samples in each case is represented by the following variables: Indicates that the observed label is 1 and the real label is 1; Indicates that the observed label is 0 and the real label is 1; Indicates that the observed label is 1 and the real label is 0; Indicates that the observed label is 0 and the true label is 0.

Step 1.2, because the true distribution of the sample is unknown, the classification result of the base classifier is used Determine the true label of the sample. Use the lower bound threshold LB _y=1 to determine whether the true label of the sample is 1. When the prediction result of the observed sample on the base classifier g(x) is greater than the lower bound threshold, it can be assumed that the true label of the observed sample is 1. Also use the upper bound UB _y=0 to determine whether the true label of the observed sample is 0.

Step 1.3, Calculate

in, is the observation positive sample set, For the observation of the negative sample set, the threshold is set as the expected value of the classification probability g(x)=P(s=1|x) of the positive and negative samples on the base classifier:

Step 1.4, Calculate an estimate of the noise rate and The process is as follows:

In step 1.5, the value of the inverted noise rate is derived from the estimated value of the noise rate by Bayes' theorem:

where p _s1 =P(s=1) represents the number of positive samples in the observed sample set. Since the inversion noise rate represents the probability that the true label is 0 or 1 in the positive and negative samples of observations, so Indicates the number of samples whose true label is 0 in the observed positive sample set, that is, the number of noise samples in the observed positive sample set. Similarly, Indicates the number of samples whose true label is 1 in the observed negative sample set, that is, the number of noise samples in the observed negative sample set. Finally, according to the predicted value of each sample base classifier g(x), the samples are sorted in ascending order, and the observed positive sample set is middle, front The samples are regarded as the noise label samples in the positive sample set, and the negative samples are observed in the negative sample set. middle, after The samples are regarded as noise label samples in the negative sample set.

Step 2, in conjunction with Figure 3, use the base classifier classification result to re-label the noise label samples, and obtain a clean sample data set after the noise label samples have been corrected. The specific process is as follows:

Step 2.1, for the binary classification case. After identifying the noise label samples, the samples are sorted in ascending order according to the predicted probability value of each sample in the base classifier g(x)=P(s=1|x). In the observed positive sample set in the front The labels of the samples are relabeled as 0; in the observation negative sample set in, after Each sample label is relabeled to 1. Positive sample set after relabeling and negative sample set They are respectively expressed as:

in Indicates the first g(x) value in the observed positive sample set small g(x) values, Indicates the first g(x) value of the observed negative sample set Large g(x) values.

Step 2.2, for the multi-class classification case. In the case of multi-class classification, the total number of sample labels is more than two. At this time, the label re-labeling of noise samples needs to consider which label the sample most likely belongs to and assign the label. The labels for re-labeling noise samples need to be selected according to the classification results of all samples by the base classifier. Therefore, when the base classifier predicts all sample data, it is necessary to record the probability that the sample belongs to each category, and the final result is a classification result matrix psx={p _ij |i∈N,j∈K}, psx is an N×K The probability matrix of (where N is the number of samples and K is the number of label types), where the probability value represents the classification result matrix of the base classification for all samples, the i-th row of the matrix p _i = (p _i1 ,p _i2 ,,,p _ik ) represents the probability that the sample _xi belongs to various labels under the base classifier f(x), and the value p _ij represents the probability that the sample _xi belongs to the class k _j . After the sample x is determined to be a noise label, the probability matrix psx is used to relabel the label of x as the label that belongs to when the predicted probability is the largest except the current label. That is, for the noisy label sample _xi , its relabeled label is:

y _i ^relabel =k _max (k _max =arg max psx _i )

where k _max is the label category to which the maximum probability of the sample _xi in the classification probability of the base classifier, except for the original noise label _si of the sample, belongs. The data obtained after the final relabeling is the correct data set after correcting the noise labels.

Claims (4)

1. A noise tag correction method, comprising the steps of:

step 1, predicting a sample by using a base classifier to obtain a sample prediction probability, respectively taking prediction probability expectation values of all samples in a positive example set and a negative example set as a lower bound threshold and an upper bound threshold, judging a real label of an observed sample by using the lower bound threshold and the upper bound threshold, and identifying noise label data;

step 2, re-labeling the noise label sample by using a base classifier to obtain a clean sample data set after the noise label sample is corrected; wherein

After the binary classification result is identified as the noise label samples, sorting the samples in an ascending order according to the prediction probability value of each sample in the base classifier, and re-labeling the labels of the previous a samples as 0 in the observation of the positive sample set; after observing the negative sample setEach sample label is re-labeled as 1;

and 2, as for the multi-class classification result, according to a classification result matrix obtained by predicting all sample data by the base classifier, re-labeling the label of the sample as the label which belongs to the sample except the current label when the prediction probability is maximum by using the probability matrix.

2. The method according to claim 1, wherein the specific steps of step 1 include:

step 1.1, the base classifier predicts a sample to obtain a sample prediction probability g (x) P (s 1| x); is provided with

Noise rate ρ₁P (s-0 | y-1) represents the probability that a sample with a true label of 1 is incorrectly labeled as 0,

representing the number of samples for which the observation label is 1 and the true label is 1,

representing the number of samples with an observation tag of 0 and a true tag of 1,

representing the number of samples for which the observation label is 1 and the true label is 0,

represents the number of samples with an observation label of 0 and a true label of 0;

step 1.2, classification results using base classifiersJudging the real label of the sample: using lower bound thresholds LB_y＝1Judging whether the real label of the sample is 1, and setting the real label of the observation sample to be 1 when the prediction result of the observation sample on a base classifier g (x) is greater than the lower bound threshold value; when the prediction result of the observation sample on the base classifier is less than the upper bound threshold UB_y＝0When the actual label of the observation sample is set to 0.

Step 1.3, calculate

Wherein,in order to observe the set of positive examples samples,for observing the negative sample set, the upper and lower threshold values are respectively set as the expected values of the classification probability g (x) of the positive and negative samples on the base classifier:

step 1.4, calculating the estimated value of the noise rateAnd

step 1.5, deducing the value of the reversal noise rate according to the estimated value of the noise rate by Bayes' theorem

Step 1.6, settingThe number of samples with a true tag of 0 in the observation sample set,representing the number of samples with a real label of 1 in the observation negative sample set, and sorting the samples in an ascending order according to the predicted value of each sample base classifier g (x); sample set of observation right caseMiddle and frontTaking the sample as a noise label sample in the normal sample set; sample set of negative examples under observationMiddle and back rowIndividual samples are considered as noise labeled samples in the negative sample set.

3. The method according to claim 2, wherein the specific process of obtaining the clean sample data set after the noise label samples are modified for the binary classification case in step 2 is:

after the noise label samples are identified, the samples are sorted in an ascending order according to the predicted probability value of each sample in the base classifier g (x) P (s 1| x). Sample set of observation right caseIn front ofThe label of each sample is re-labeled as 0; sample set of negative examples under observationIn, afterEach sample label is re-labeled as 1;

re-labeled positive sample setAnd negative sample setRespectively expressed as:

wherein,representing the g (x) value in the sample set of the observation positive caseThe small value of g (x) is,sample set g (x) values representing negative observationsLarge g (x) values.

4. The method according to claim 2, wherein, for the multi-class classification condition in step 2, the label re-labeling of the noise sample is adopted to obtain a clean sample data set after the noise label sample is modified, and the specific process is as follows:

when predicting all sample data, the base classifier needs to record the probability that the sample belongs to each class, and a classification result matrix psx is obtained, wherein the probability is p_ijI belongs to N, j belongs to K, psx is a N multiplied by K probability matrix, where N is the number of samples and K is the number of label categories, where the probability value represents the classification result matrix of the base classification to all the samples, and the ith row p of the matrix_i＝(p_i1,p_i2,,,p_ik) Represents a sample x_iProbability, value p, of belonging to classes of labels under base classifier f (x)_ijRepresents a sample x_iBelong to k_jThe probability of a class;

when sample x is determined to be a noise label, label x is re-labeled with the probability matrix psx as the label to which the prediction probability is the highest except the current label:

y_i ^relabel＝k_max(k_max＝arg max psx_i)

wherein k is_maxIs a sample x_iRemoving original noise label s of the sample in classification probability of a base classifier_iThe label category to which the outer probability maximum belongs.