CN114638336A - Unbalanced learning focusing on strange samples - Google Patents

Unbalanced learning focusing on strange samples Download PDF

Info

Publication number
CN114638336A
CN114638336A CN202111606351.3A CN202111606351A CN114638336A CN 114638336 A CN114638336 A CN 114638336A CN 202111606351 A CN202111606351 A CN 202111606351A CN 114638336 A CN114638336 A CN 114638336A
Authority
CN
China
Prior art keywords
model
sample
samples
loss function
strange
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111606351.3A
Other languages
Chinese (zh)
Other versions
CN114638336B (en
Inventor
胡祝华
赵瑶池
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hainan University
Original Assignee
Hainan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hainan University filed Critical Hainan University
Priority to CN202111606351.3A priority Critical patent/CN114638336B/en
Publication of CN114638336A publication Critical patent/CN114638336A/en
Application granted granted Critical
Publication of CN114638336B publication Critical patent/CN114638336B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/04—Inference or reasoning models
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Abstract

The imbalance learning method focusing on the strange samples can be used for imbalance learning and reasoning of a classification model based on a deep neural network. The method takes the network logic output value of the sample as an index value of the familiarity degree of the model to the sample, and considers the sample with a lower logic value as an unfamiliar sample of the model. Specifically, in the model training process, the loss function in the invention is a cost sensitive loss function based on the sample logit value: there may be an instance level and a category level. The invention can improve the aggregation characteristics of the class samples in training and reduce the negative influence of annotation errors on unbalanced learning. In the reasoning process of the model, the invention adopts a migration strategy to realize the reasoning of the model: firstly, obtaining an offset parameter in the optimal classifier on the basis of the offset of the logit on the verification set, and then reasoning on the test set according to the offset parameter. The migration strategy can be used for correcting classification errors of the model caused by migration of the model in the reasoning process.

Description

Unbalanced learning focusing on strange samples
Technical Field
The invention relates to machine learning, in particular to an imbalance learning method focusing on strange samples, which can be used for imbalance learning and reasoning of a classification model based on a deep neural network.
Background
Class imbalance refers to the existence of a quantitative imbalance between different classes in the data. In the past few years, as machine learning has continued to evolve, deep artificial neural networks have made great progress. Deep neural network models are always trained on well-designed data sets, which are usually distributed balanced. However, in the real world, class imbalance data is ubiquitous in real training scenarios due to the different frequencies of occurrence of objects, events and behaviors. If no correction mechanism exists in the training process, the unbalanced characteristics in the data set can cause the recognition performance of the deep neural network model on the subclasses to be low. With the wide application of deep learning, how to learn a high-performance model from data sets with unbalanced classes or long tail distribution becomes an urgent problem to be solved.
At present, some methods for solving the imbalance-like problem exist, and can be divided into a data-based method and an algorithm-based method. In data-based approaches, one common way is to resample the training data by over-sampling small classes, under-sampling large classes, or a combination thereof, but this will change the distribution of the data. Another way is a set of classifiers, each trained from a different sample of the original data set. In recent years, in order to better learn the distribution of the data set, researchers have also expanded the data set by way of sample generation, thereby improving the performance of the model. In algorithm-based approaches, cost-sensitive loss functions are often widely adopted to solve the problem of unbalanced learning by directly or indirectly emphasizing/suppressing the importance of small/large classes by introducing a cost (weight) for each class or each sample. Compared with data-based methods, such methods are intuitive, easier to implement, and widely used in deep learning. In a widely used cost sensitive loss function, the cost is based on the loss or class probability of the sample. However, these methods only utilize the classification features between classes, discard the clustering features inside the classes, and introduce the error of labeling into the computation of the cost sensitive loss function, which easily leads to suboptimal models. In addition, in a task of dense labeling of samples, such as an image segmentation task in computer vision, due to the workload and the tediousness of labeling samples, the labeled contour is often deviated from the real contour by several pixels, so that a large number of wrongly labeled samples are generated in the vicinity of the target contour. In the existing cost-sensitive loss function based on loss or class probability, these labeling errors can be introduced into the calculation of the cost-sensitive loss function, which affects the improvement of the model performance in the training process.
Disclosure of Invention
1. The imbalance learning method focusing on the strange samples can be used for imbalance learning and reasoning of a classification model based on a deep neural network. The method takes the network logic output value of the sample as an index value of the familiarity degree of the model to the sample, and considers the sample with a lower logic value as an unfamiliar sample of the model. Specifically, in the training process of the model, the loss function is a cost sensitive loss function based on the sample logit value; in the inference process of the model, a migration strategy is adopted to realize the inference of the model: firstly, obtaining an offset parameter in the optimal classifier on the basis of the offset of the logit on the verification set, and then reasoning on the test set according to the offset parameter.
The idea of focusing on unfamiliar sample learning is adopted, namely the traditional Cross Entropy loss function is improved into an example-level cost-sensitive loss function (IFSL) focusing on unfamiliar samples, and the traditional balanced Cross Entropy loss function (Cross Entropy Based on Class Frequency) Based on Class Frequency cost is improved into a Class-level cost-sensitive loss function (CFSL) focusing on unfamiliar samples.
2. The formula for calculating the cost sensitive loss function IFSL with the example level focused on strange samples is
Figure BDA0003434069860000021
Wherein x isiFor the ith sample input, θ is the depth network model parameter, piFor the probability output of the ith sample on the network model, pitProbability output for true class of i-th sample, yiIs the label of the i-th sample,
Figure BDA0003434069860000025
lo on the network model for the ith samplegit output linear transformation to [0,255 ]]Value in area, μtFor a logit-based cost sensitive loss function, defined as:
Figure BDA0003434069860000022
wherein, T epsilon (0,1) represents the definition range of the unfamiliar samples, and mu epsilon (1,2) represents the focusing degree of the unfamiliar samples.
3. The formula for calculating the cost-sensitive loss function CFSL with category level focusing on strange samples is
Figure BDA0003434069860000023
Wherein xi,θ,yi,
Figure BDA0003434069860000024
And mutThe meaning of (a) is as described in the summary of the invention 2.
4. The model inference procedure using the migration strategy is:
1) the optimal offset parameters for the classifiers in the model are first obtained using equation (4).
Figure BDA0003434069860000031
Wherein, H (·) is the performance index of the model on the verification set, and p ═ { p ·)0,p1As the prediction probability, y ═ y is defined as shown in formula (5)0,y1Is the label value, b ═ b0,b1Is an offset function, defined as shown in equation (6).
Figure BDA0003434069860000032
Wherein f iscFor the value of the sample's location on class c,
Figure BDA0003434069860000033
in the formula (4), the first and second groups,
Figure BDA0003434069860000034
the optimal delta is the delta which enables the model performance to reach the optimal delta on the verification set.
2) The prediction class is inferred in the classifier of the model using equation (7).
Figure BDA0003434069860000035
Wherein, CpreFor prediction class, pcThe calculation formula is shown in (5) for the probability in the category c, and δ is the offset value obtained in step 1).
The invention has the beneficial effects that:
the imbalance learning method focusing on the strange samples is used for imbalance learning and reasoning of the classification model based on the deep neural network, and can improve the aggregation characteristics of the samples in the class and reduce the negative influence of annotation errors on the imbalance learning in the training process, so that the classification performance of the training model is improved. In addition, the inference strategy based on the locality migration can correct the classification error of the model caused by the migration of the model in the inference process.
The existing deep neural network model is usually trained on a balanced distributed data set, and the classification performance is low for subclasses. With the development of the in-place application of deep learning technology in the real world, the unbalanced data ubiquitous in the real world increasingly becomes a bottleneck limiting the deep learning performance and application. Aiming at the problem, the invention improves the existing unbalanced learning method and has wide application prospect.
Additional aspects and advantages of the invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.
Drawings
FIG. 1 is a drawing of the abstract of the present invention;
FIG. 2 is a graph of performance as a function of offset parameter for this example 1;
FIG. 3 is the reasoning result of the model trained using different cost-sensitive loss functions on the test set in this embodiment 1;
FIG. 4 is a significant box generated by the model generated using CFSL and BCE _ F in this example 2;
fig. 5 is a division result using the offset policy and the conventional policy in this embodiment 2.
Detailed description of the preferred embodiments
The concept, specific steps and technical effects of the present invention will be clearly and completely described below in conjunction with the embodiments and fig. 2 to 5 to fully understand the objects, features and effects of the present invention. It is to be understood that the described embodiments are merely exemplary of the present invention, and that functional, methodological, or structural equivalents or substitutions that are described by those of ordinary skill in the art based on the described embodiments are within the scope of the present invention.
1. The imbalance learning method focusing on the strange samples can be used for imbalance learning and reasoning of a classification model based on a deep neural network. The method takes the network output logic value of the sample as an index value of the familiarity degree of the model to the sample, and considers the sample with a lower logic value as an unfamiliar sample of the model. Specifically, in the training process of the model, the loss function is a cost sensitive loss function based on the sample logit value; in the inference process of the model, a migration strategy is adopted to realize the inference of the model: firstly, obtaining an offset parameter in the optimal classifier on the basis of the offset of the logit on the verification set, and then reasoning on the test set according to the offset parameter.
Since the cost sensitive loss function is divided into an example-level function and a category-level function, the cost sensitive loss function focusing on strange sample learning in the invention is divided into an example-level method and a category-level method. By focusing on the idea of unfamiliar sample learning, the traditional cross entropy loss function is improved into an example-level cost-sensitive loss function (IFSL) focusing on unfamiliar samples; a traditional balanced Cross Entropy loss function (BCE _ F) Based on Class Frequency cost is improved into a Class-level cost sensitive loss function (CFSL) focused on strange samples.
2. The formula for calculating the cost sensitive loss function IFSL with the example level focused on strange samples is
Figure BDA0003434069860000041
Wherein x isiFor the ith sample input, θ is the depth network model parameter, piFor the probability output of the ith sample on the network model, pitProbability output for true class of i-th sample, yiIs the label of the i-th sample,
Figure BDA0003434069860000051
linear transformation of the logit output on the network model to [0,255 ] for the ith sample]Value in area, μtFor a logit-based cost sensitive loss function, defined as:
Figure BDA0003434069860000052
wherein, T epsilon (0,1) represents the definition range of the unfamiliar samples, and mu epsilon (1,2) represents the focusing degree of the unfamiliar samples.
3. The formula for calculating the cost-sensitive loss function CFSL with category level focusing on strange samples is
Figure BDA0003434069860000053
Wherein xi,θ,yi,
Figure BDA0003434069860000054
And mutHave the same meaning asAs described in summary of the invention 2.
4. The model inference procedure using the migration strategy is:
1) first, the optimal parameters of the classifier in the model are obtained using equation (4).
Figure BDA0003434069860000055
Wherein, H (·) is the performance index of the model on the verification set, and p ═ { p ·)0,p1As the prediction probability, y ═ y is defined as shown in formula (5)0,y1Is the label value, b ═ b0,b1Is the offset function, defined as shown in equation 6.
Figure BDA0003434069860000056
Wherein f iscFor the value of the sample's logit on class c,
Figure BDA0003434069860000057
in the formula (4), the first and second groups of the chemical reaction are shown in the specification,
Figure BDA0003434069860000059
the optimal delta is the delta which enables the model performance to reach the optimal delta on the verification set.
2) The prediction class is inferred in the classifier of the model using equation (7).
Figure BDA0003434069860000058
Wherein, CpreFor the prediction class, pcThe calculation formula is shown in (5) for the probability in the category c, and δ is the offset value obtained in step 1).
The invention is implemented in two typical visual tasks, one is a semantic segmentation task, and the other is a salient object detection subtask for image salient instance segmentation. Since the example-level cost sensitive method and the category-level cost sensitive method have different application scenarios, the IFSL in the present disclosure is implemented in the semantic segmentation task, as described in embodiment 1. The CFSL in this summary is implemented in the significance target detection subtask of significance example segmentation, as described in example 2.
Example 1:
image semantic segmentation is to assign a semantic label to each pixel in an image, and has wide application, such as scene understanding and automatic driving. The image semantic segmentation task in this embodiment is implemented on the PASCAL VOC 2012 augmentent variant. The pascalloc 2012 augmenter is the most popular data set for image semantic segmentation. It contains 10582 images for training and 1449 images for testing ([1] Evargham, M., et al, The past visual object class challenge: A retroactive, Int J ComputVision, 2015.[2] Hariharan, B., et al, magnetic controls from change detectors in 2011International Conference on Computer Vision (CVPR), 2011.). Because the original data set is class balanced, the present embodiment makes a class unbalanced variant of the original data set. It consists of the most challenging classes of chairs and non-chairs, with the frequency ratio of the small class to the large class being about 1: 100.
depeplav 3+ is a popular semantic segmentation network used as the base network in the experiments of this example. This example compares IFSL with the current state-of-the-art losses, such as CE, BCE _ C, BCE _ F, FL, PF, BENS, and CFL (3. Chen, L.C., et al., Encoder-decoder with associated segment selection for segment identification, In: Proceedings of the European connection component, 20158. 4. Fan, R., et al., S4. network: Single stage sample analysis, In: Proceedings of the IEEE/F connection component and Pattern Recognition (CVPR),2019. 5. Mobil, M.J., feed conversion component, CVP. video and Pattern Recognition (CVPR), IEEE _ 5. J., M.M., M.E., feedback segment, and/or C., In: real-time version, real-version of the I.P.C., real-time version, real-time, real-time, real-, 2021.[7] Cui, Y., et al., Class-based lost based on effective number of samples, In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Registration (CVPR),2019.[8] ZHao, Y., et al., Constrained-focal-based lost for registration of fields, IEEEAccess, 2019). Among these methods, BCE _ C, BCE _ F, PF, CFL, and BENS belong to class-level methods, and the rest belong to example-level methods. For each comparison method, the present embodiment searches for the best parameters to obtain the best performance. This embodiment adopts MIoU (Mean Intersection over Unit of the predicted set and the ground-route set) as the performance evaluation index of semantic segmentation.
Table 1 shows the evaluation index on the test set after using the IFSL and the comparison method to obtain the model using the basic network training. Of all the methods, IFSL performs best.
TABLE 1 MIoU comparison (%)% of IFSL and comparative methods on PASCAL VOC 2012AUGMENT
Figure BDA0003434069860000071
This example performed a number of experiments to analyze the effect of hyper-parameters in IFSL on model performance. The present embodiment sets the values of T to 0.15, 0.20, and 0.25, respectively. For each value of T, the present embodiment is respectivelyμMIoU was tested at 1.2, 1.4, 1.6 and 1.8. T andμthe optimum combination of (a) and the corresponding properties are shown in table 2. It can be seen that when T is 0.2 (meaning that 20% of the whole sample is considered as strange sample) andμthe best performance is achieved when the ratio is 1.6. In addition, from Table 2, it can be observed that a rule, when the value of T is large (the strange sample ratio is high),μshould be set to a relatively small value (the strange sample should be less important). The rule accords with the intuitive thinking of people.
TABLE 2 Effect of hyper-parameters in IFSL on model Performance
Figure BDA0003434069860000072
Since the data distribution of the PASCAL VOC 2012 augmentent variant is severely unbalanced, the model trained by using the IFSL still has a bias, and the present embodiment adopts a biased inference strategy to correct the classification error of the model. This example first randomly splits the test set of the PASCAL VOC 2012 augmentent variant into two equal parts, one serving as the validation set and the other serving as the new test set. The model performance was then tested on the validation set using different values of δ, and the MIoU versus δ curve is shown in fig. 2. MIoU is a concave function of δ and is obtained according to equation (4)
Figure BDA0003434069860000073
Finally, the performance of the conventional inference strategy and the offset inference strategy of the present invention are compared on a new test set. With the conventional inference strategy, the MIoU of the model is 70.4%, while with the offset strategy, the MIoU of the model reaches 71.0%, which is improved by 0.6 percentage points.
FIG. 3 shows the classification results on a test set for models trained using different cost sensitive loss functions. For CE, some object pixels, especially the boundary pixels of the object (see columns 1 and 4), are wrongly classified as background pixels because they are not given enough attention during the training process. For FL, the subclass is under-represented, also appearing to misclassify the target pixel as a background pixel (see columns 1 and 4). These phenomena also occur for CFL and BENS, in particular BENS. For IFSL, only slight misclassifications occur. Furthermore, the segmentation result can be further improved using an offset strategy, as shown in the last line.
Example 2:
salient instance segmentation segments the salient region and identifies each instance, with a wide range of applications, such as accurate image editing and weakly supervised segmentation. Significant instance partitioning includes two consecutive subtasks: salient object detection and salient segmentation. In the saliency segmentation phase, the samples used for training are class-balanced, but in the saliency target detection phase, there is an extreme class imbalance between saliency boxes and non-saliency boxes (class-frequency ratio of about 1: 1000). Furthermore, there are a large number of recommendation boxes that have some significance, but are labeled as non-significance boxes. This makes the distribution of non-salient blocks very complex. The embodiment performs the unbalanced learning of the deep neural network model on the saliency target detection subtask, and improves the classification between the saliency frame and the non-saliency frame by using the CFSL in the invention, thereby improving the saliency instance segmentation.
The Salient example Segmentation in this example uses the ILSO dataset ([9] G.Li, et al, instant-Level salt Object Segmentation, in: the proceedings of the IEEE International Conference on Computer Vision, 2017.). The present embodiment uses S4net [4] as the base network. Following the standard COCO index, this example uses the maps (average precision) to evaluate the proposed method and reports map0.5 and map0.7.
This embodiment compares CFSL with existing cost-sensitive methods, including BCE _ F, FL, OHEM, PF, BENS, and CFL ([4] Fan, R., et al, S4net: Single stage tolerance segment, In: Proceedings IEEE/CVF Congression on Computer Vision and Pattern Registration (CVPR),2019.[6] Du, J., et al, Parameter-fragment for use In a Class-based and estimated deletion In image classification, IEEE Transactions on Neural Networks and Learning Systems,2021.[7] Cui, Y., Class-fragment for use In a Class-based feedback system, CVF 2019, J., and C., IEEE transaction for use In a Class-sample feedback system, and CVF 2018. J., CVF, J., and C., CVF, J.S, J., a, et al, Training region-based object detectors with online hard execution in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2016. They are the most advanced imbalance method learning methods in the task of object detection or image classification. Where FL is an example level method, and OHEM can be considered a combination of an example level method and a class level method, and others belong to the class level method. For the existing method, the present embodiment searches the entire parameter space for the best hyper-parameter to obtain the best performance.
Embodiments because of the extreme class imbalance of the data for training the significance detection model, the imbalance ratio is about 1: 1000, this makes the model trained after unbalanced training still biased. The present embodiment uses an offset inference strategy in the inference process and compares it with the results of a conventional inference strategy.
TABLE 3 comparison of the performance of CFSL and existing methods
Method FL OHEM PF BCE_F CFL BENS CFSL
mAP0.5 87.5 86.3 81.7 86.3 85.7 86.6 87.6
mAP0.7 61.5 63.0 58.0 63.6 63.0 64.4 64.5
First, this example quantitatively compares CFSL with existing methods, as shown in table 3. Among all the methods, CFSL performs best. The CFSL improved the performance of map0.5 and map0.7 by 1.3 and 0.9 percentage points, respectively, compared to its basic method BCE _ F. The advantages of CFSL can be attributed to the improvement in the in-class polymerization characteristics. Using CFSL, better saliency features that are more consistent with human visual laws are learned into the model, so that a more accurate score can be provided for the saliency box. Finally, the segmentation performance of the significant instance segmentation can be improved.
Then, this example qualitatively compares CFSL with its basic method BCE _ F. Figure 4 shows the significant boxes produced by the model generated using these two loss functions. In each image, n (real numbers of salient objects) real salient frames are marked with white frames, and n prediction frames are marked with green and blue frames in the order of prediction probability, as shown in the first line. The BCE _ F method gives high scores for some severely incomplete frames (see the frame in the first image and the blue frame in the second image), erroneously determines some non-significant frames as significant frames (see images 3 and 4), and misses some significant frames (see men in image 3 and cups in the bottom right corner of image 4). In contrast, the CFSL method reduces the frequency of occurrence of these phenomena.
To analyze the effect of the hyper-parameters in CFSL on the model performance, this example performed experiments on the ILSO dataset. Through experimentation, strange samples were found to be mainly from the main group. Therefore, in this embodiment, the values of the parameter T in the formula (2) are set to 0.00075,0.001, 0.00125 and 0.0015, which makes the number of strange samples approximately equal to the number of few classes of samples, avoiding the overwhelming effect of the majority classes. In addition, the parameters in the formula (2) are also adjusted in the embodimentμValues of (a) are 1.2, 1.5 and 1.8.
T andμthe optimal combination of (d) and the corresponding mAp0.5 and mAp0.7 performance are shown in Table 4. It can be seen that when T is equal to 0.001 (i.e., the number of strange samples is approximately equal to the number of few classes of samples) andμat 1.6, the performance is optimal. This embodiment may also observe that when the value of T is relatively large,μshould be set to a relatively small value, similar to the experimental results of the IFSL. Furthermore, at T andμin the best combination of (see the third row),μthe value of (d) is 1.5, which is also similar to the results of the IFSL.
TABLE 4 Effect of the superparameters in CFSL on model Performance (%)
T Optimization ofμ mAP0.5 mAP0.7
0.00075 1.5 85.9 64.3
0.00100 1.5 87.6 64.5
0.00125 1.2 86.0 63.8
0.00150 1.2 86.9 64.7
Due to extreme class imbalance (about 1000:1) in the object detection phase, the model obtained by imbalance learning still has bias. The present embodiment uses an offset strategy in the inference process to correct misclassifications of the model. Since the mAP value in the significance instance segmentation is calculated over 20 predicted significance instances, the true number of significance instances is always less than it. Therefore, this embodiment uses the F1-score metric as H in equation (4) and obtains the result on the validation set
Figure BDA0003434069860000101
The embodiment then uses the optimal parameters on the test set
Figure BDA0003434069860000102
F1-Score for the test model.
This example compares the results of the segmentation of the offset strategy and the conventional strategy on the ILSO dataset, a qualitative and quantitative comparison of which is shown in table 5 and fig. 5. As can be seen from table 5, the migration strategy in the present invention greatly improves the segmentation performance on the test set compared to the conventional inference strategy. As can be seen from fig. 5, a large number of non-salient blocks are erroneously determined as salient blocks using a conventional inference strategy. In contrast, using the migration strategy of the present invention, such classification errors can be greatly reduced, and thus the segmentation result can be improved.
Table 5F 1_ Score comparison using offset policy and conventional policy inference
Figure BDA0003434069860000111
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. The details of the embodiments are not to be interpreted as limiting the scope of the invention, and any obvious changes, such as equivalent alterations, simple substitutions and the like, based on the technical solution of the invention, can be interpreted without departing from the spirit and scope of the invention.

Claims (4)

1. The imbalance learning method focusing on the strange samples can be used for imbalance learning and reasoning of a classification model based on a deep neural network. The method takes the network logic output value of the sample as an index value of the familiarity degree of the model to the sample, and considers the sample with a lower logic value as an unfamiliar sample of the model. Specifically, in the training process of the model, the loss function is a cost sensitive loss function based on the sample logit value; in the inference process of the model, a migration strategy is adopted to realize the inference of the model: firstly, obtaining an offset parameter in the optimal classifier on the basis of the offset of the logit on the verification set, and then reasoning on the test set according to the offset parameter.
The idea of focusing on strange sample learning is adopted, namely, a traditional Cross Entropy loss function is improved into an example-level cost-sensitive loss function (IFSL) focusing on strange samples, and a traditional balance Cross Entropy loss function (BCE _ F) Based on Class Frequency cost is improved into a Class-level cost-sensitive loss function (CFSL) focusing on strange samples.
2. The method of claim 1, wherein: the example-level cost sensitive loss function IFSL focused on strange samples is calculated by the formula
Figure FDA0003434069850000011
Wherein x isiFor the ith sample input, θ is the depth network model parameter, piFor the probability output of the ith sample on the network model, pitProbability output for true class of i-th sample, yiIs the label of the i-th sample,
Figure FDA0003434069850000012
linear transformation of the logit output on the network model to [0,255 ] for the ith sample]Value in area, μtFor a logit-based cost sensitive loss function, defined as:
Figure FDA0003434069850000013
wherein T epsilon (0,1) represents the definition range of the strange samples, and mu epsilon (1,2) represents the focusing degree of the strange samples.
3. The method of claim 1, wherein: the cost sensitive loss function CFSL focused on strange samples at category level is calculated by the formula
Figure FDA0003434069850000014
Wherein xi,θ,yi,
Figure FDA0003434069850000015
And mutThe meaning of (A) is as defined in claim 2.
4. The method of claim 1, the inference step using an offset strategy is:
1) the optimal offset parameters for the classifiers in the model are first obtained using equation (4).
Figure FDA0003434069850000016
Wherein, H (·) is the performance index of the model on the verification set, and p ═ { p ·)0,p1As the prediction probability, y ═ y is defined as shown in formula (5)0,y1Is the label value, b ═ b0,b1Is the offset function, defined as shown in equation 6.
Figure FDA0003434069850000021
Wherein f iscFor the value of the sample's logit on class c,
Figure FDA0003434069850000022
in the formula (4), the first and second groups,
Figure FDA0003434069850000023
the optimal delta is the delta which enables the model performance to reach the optimal delta on the verification set.
2) The prediction class is inferred in the classifier of the model using equation (7).
Figure FDA0003434069850000024
Wherein, CpreFor prediction class, pcThe calculation formula is shown in (5) for the probability in the category c, and δ is the offset value obtained in step 1).
CN202111606351.3A 2021-12-26 2021-12-26 Unbalanced learning focused on strange samples Active CN114638336B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111606351.3A CN114638336B (en) 2021-12-26 2021-12-26 Unbalanced learning focused on strange samples

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111606351.3A CN114638336B (en) 2021-12-26 2021-12-26 Unbalanced learning focused on strange samples

Publications (2)

Publication Number Publication Date
CN114638336A true CN114638336A (en) 2022-06-17
CN114638336B CN114638336B (en) 2023-09-22

Family

ID=81946584

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111606351.3A Active CN114638336B (en) 2021-12-26 2021-12-26 Unbalanced learning focused on strange samples

Country Status (1)

Country Link
CN (1) CN114638336B (en)

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050289089A1 (en) * 2004-06-28 2005-12-29 Naoki Abe Methods for multi-class cost-sensitive learning
CN107545275A (en) * 2017-07-27 2018-01-05 华南理工大学 The unbalanced data Ensemble classifier method that resampling is merged with cost sensitive learning
WO2019033636A1 (en) * 2017-08-16 2019-02-21 哈尔滨工业大学深圳研究生院 Method of using minimized-loss learning to classify imbalanced samples
CN109800810A (en) * 2019-01-22 2019-05-24 重庆大学 A kind of few sample learning classifier construction method based on unbalanced data
US20190266731A1 (en) * 2018-02-26 2019-08-29 Abc Fintech Co., Ltd. Image segmentation method and device
CN110689544A (en) * 2019-09-06 2020-01-14 哈尔滨工程大学 Method for segmenting delicate target of remote sensing image
CN111178897A (en) * 2019-12-18 2020-05-19 浙江大学 Cost-sensitive dynamic clustering method for performing rapid feature learning on unbalanced data
US10970650B1 (en) * 2020-05-18 2021-04-06 King Abdulaziz University AUC-maximized high-accuracy classifier for imbalanced datasets
CN113222035A (en) * 2021-05-20 2021-08-06 浙江大学 Multi-class imbalance fault classification method based on reinforcement learning and knowledge distillation
CN113554653A (en) * 2021-06-07 2021-10-26 之江实验室 Semantic segmentation method for long-tail distribution of point cloud data based on mutual information calibration
KR20210145923A (en) * 2020-05-26 2021-12-03 동아대학교 산학협력단 A deep-learning system based small learning data-set and method of deep-learning used it

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050289089A1 (en) * 2004-06-28 2005-12-29 Naoki Abe Methods for multi-class cost-sensitive learning
CN107545275A (en) * 2017-07-27 2018-01-05 华南理工大学 The unbalanced data Ensemble classifier method that resampling is merged with cost sensitive learning
WO2019033636A1 (en) * 2017-08-16 2019-02-21 哈尔滨工业大学深圳研究生院 Method of using minimized-loss learning to classify imbalanced samples
US20190266731A1 (en) * 2018-02-26 2019-08-29 Abc Fintech Co., Ltd. Image segmentation method and device
CN109800810A (en) * 2019-01-22 2019-05-24 重庆大学 A kind of few sample learning classifier construction method based on unbalanced data
CN110689544A (en) * 2019-09-06 2020-01-14 哈尔滨工程大学 Method for segmenting delicate target of remote sensing image
CN111178897A (en) * 2019-12-18 2020-05-19 浙江大学 Cost-sensitive dynamic clustering method for performing rapid feature learning on unbalanced data
US10970650B1 (en) * 2020-05-18 2021-04-06 King Abdulaziz University AUC-maximized high-accuracy classifier for imbalanced datasets
KR20210145923A (en) * 2020-05-26 2021-12-03 동아대학교 산학협력단 A deep-learning system based small learning data-set and method of deep-learning used it
CN113222035A (en) * 2021-05-20 2021-08-06 浙江大学 Multi-class imbalance fault classification method based on reinforcement learning and knowledge distillation
CN113554653A (en) * 2021-06-07 2021-10-26 之江实验室 Semantic segmentation method for long-tail distribution of point cloud data based on mutual information calibration

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
SALMAN H. KHAN ET AL.: "Cost-Sensitive Learning of Deep Feature Representations From Imbalanced Data", 《 IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS》, vol. 29, no. 8, pages 3573 - 3587, XP011687373, DOI: 10.1109/TNNLS.2017.2732482 *
ZEJU LI ET AL.: "Overfitting of Neural Nets Under Class Imbalance: Analysis and Improvements for Segmentation", 《MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION – MICCAI 2019》, pages 402 - 410 *
ZHE WANG ET AL.: "Cost-sensitive Fuzzy Multiple Kernel Learning for imbalanced problem", 《NEUROCOMPUTING》, vol. 366, pages 178 - 193, XP085842803, DOI: 10.1016/j.neucom.2019.06.065 *
平瑞等: "高度不平衡数据的代价敏感随机森林分类算法", 《模式识别与人工智能》, vol. 33, no. 3, pages 249 - 257 *
张驰名等: "基于深度学习的胸部常见病变诊断方法", 《计算机工程》, vol. 46, no. 7, pages 306 - 320 *
朱泽民等: "基于通道注意力和代价敏感改进图像分割的矿区环境遥感监测", 《矿业研究与开发》, vol. 41, no. 8, pages 183 - 191 *
谭洁帆等: "基于卷积神经网络和代价敏感的不平衡图像分类方法", 《计算机应用》, vol. 38, no. 7, pages 1862 - 1871 *

Also Published As

Publication number Publication date
CN114638336B (en) 2023-09-22

Similar Documents

Publication Publication Date Title
Ru et al. Learning affinity from attention: End-to-end weakly-supervised semantic segmentation with transformers
CN110276765B (en) Image panorama segmentation method based on multitask learning deep neural network
EP3767536A1 (en) Latent code for unsupervised domain adaptation
CN111160469B (en) Active learning method of target detection system
CN110322445B (en) Semantic segmentation method based on maximum prediction and inter-label correlation loss function
US11295240B2 (en) Systems and methods for machine classification and learning that is robust to unknown inputs
CN111428733A (en) Zero sample target detection method and system based on semantic feature space conversion
Zheng et al. Improvement of grayscale image 2D maximum entropy threshold segmentation method
CN111488911B (en) Image entity extraction method based on Mask R-CNN and GAN
CN112232395B (en) Semi-supervised image classification method for generating countermeasure network based on joint training
CN117011563B (en) Road damage inspection cross-domain detection method and system based on semi-supervised federal learning
WO2015146113A1 (en) Identification dictionary learning system, identification dictionary learning method, and recording medium
CN114299362A (en) Small sample image classification method based on k-means clustering
CN114333062B (en) Pedestrian re-recognition model training method based on heterogeneous dual networks and feature consistency
CN113033410B (en) Domain generalization pedestrian re-recognition method, system and medium based on automatic data enhancement
CN110705713A (en) Domain specific feature alignment method based on generation of countermeasure network
CN111797935B (en) Semi-supervised depth network picture classification method based on group intelligence
Kumar et al. Unsupervised fusion weight learning in multiple classifier systems
CN114638336A (en) Unbalanced learning focusing on strange samples
CN109063732B (en) Image ranking method and system based on feature interaction and multi-task learning
US11328179B2 (en) Information processing apparatus and information processing method
CN114724015A (en) Target detection method for reducing labeling requirements based on active domain adaptive learning
CN114595695A (en) Self-training model construction method for few-sample intention recognition system
CN114792114A (en) Unsupervised domain adaptation method based on black box under multi-source domain general scene
CN113313213A (en) Data set processing method for accelerating training of target detection algorithm

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant