LU504988B1 - A method for classifying undersampled integrated imbalance data based on fuzzy c-means clustering - Google Patents

Abstract

The present invention relates to the field of integrated learning technology, particularly to a method for classifying integrated imbalance data with subsampling based on fuzzy C-mean clustering, comprising the following steps: subsampling based on fuzzy C-mean -clustering; integrated learning training; integrated learning testing; and experimental analysis; The advantageous effect is: the integrated sub-sample classification method for imbalanced data based on fuzzy C-means clustering proposed by the present invention is by fuzzy C-means clustering (FCM). a clustering method that constructs a fuzzy matrix based on the attribute characteristics of the research object itself and determines the classification relationship based on the calculated membership degree. The fuzzy C-means clustering algorithm has the advantages of easy classification and high classification accuracy. From the perspective of data preprocessing, the fuzzy C-means clustering algorithm is used to subsample the unbalanced data to obtain the balanced data, and the built-in learning algorithm is used to improve the classification accuracy of some classes.

Description

A method for classifying integrated disequilibrium data using LU504988

Subsampling based on fuzzy c-means clustering

Technical part

The present invention relates to the field of integrated learning technology, particularly to a method for classifying integrated imbalance data

Subsampling based on fuzzy c-means clustering.

Technology in the background

Classification is one of the main research directions in the field of machine learning. Traditional classification algorithms can solve the problem of balanced data classification well, and the overall accuracy of classification as

Use evaluation criteria.

In the existing technology, the number of samples varies between different

Classes in practice are often unbalanced, e.g. E.g. in medical diagnosis, predicting software errors, monitoring credit card fraud transactions, etc

Sentiment analysis, customer churn prediction, etc., and in these areas one tends to give more preference to the classification results of sampling a few classes

to pay attention.

However, since it is more likely that the samples of a minority class from

If a classifier is misclassified, misclassification has a significant impact on the accuracy of the classifier. Let's take the binary classification problem as an example:

Assuming that the ratio of the number of samples between classes in the data set is 1:99, then the classifier tends if the overall classification accuracy is as

criterion is used, tends to belong to the majority class with a larger number of

Samples, and although the calculated accuracy reaches 99%, is such

Classification result obviously meaningless. Therefore, it is particularly important to develop an optimized classification algorithm for classifying imbalanced data.

Content of the invention

The aim of the present invention is to provide a method for classifying undersampled integrated imbalance data based on fuzzy C-means.

To provide clustering to solve the problems raised in the prior art above.

In order to achieve the above objective, the present invention provides the following technical solution: a method for classifying undersampled integrated

Disequilibrium data based on fuzzy C-means clustering, where the

Procedure includes the following steps:

subsampling based on fuzzy C-means clustering;

Integration learning training;

testing integration learning;

Experimental analysis.

Preferably, fuzzy C-means clustering based under-sampling includes the following specific operations:

Subsampling the majority class samples for clustering using the

FCM algorithm,Setting the value of k to the number of samples of the minority class, k=nı, k samples obtained from clustering centers instead of the original data samples of the

Majority class consist,and obtain a balanced data set consisting of a similaré:/504988

Number of data samples of the two classes exists.

Preferably, fuzzy C-means clustering based undersampling also includes the following specific operations:

Training a classifier on the obtained balanced dataset and

Conducting multiple classification experiments; dividing the set of data samples into a set of minority class samples and a set of majority class samples; clustering the majority class samples and forming a balanced data set with the centers of the obtained clusters and all minority class samples; After subsampling, it is ensured that the number of samples of the majority class is the same as the number of samples of the

Minority class is balanced, and the number of clusters of majority class samples is set to the number of minority class samples.

Preferably, the integrated learning training process includes the following:

If there is no strong dependency between the base classifiers, this will

Classification model generated in parallel, there is a strong dependency between the

Base classifiers, the classification model must be generated serially. The parallel

Generation of the classification model is typically represented by the bagging algorithm and the serial generation of the classification model by the boosting algorithm.

Preferably, the specific operation of the integrated learning test includes:

Selecting Decision-Stump with KNN as the base classifier in the integrated

Learning algorithm

Preferably, the specific operations of the experimental analysis include:

Selecting multiple unbalanced data sets for Matlab simulation;

Conclusion that the minority class samples are imbalanced

Dataset P and the majority class samples are N;

Using the confusion matrix, a series of evaluation indices are created for the

Overall accuracy, the correctness rate, the rate for checking all classes, the

Minority class accuracy, correctness verification rate, F-measure and AUC are defined, with the overall accuracy of the classifier calculated according to Equation (1): . TP+IN

TP+IN FPL FS (1)

The verification rate of the classifier, also known as subclass accuracy or positive accuracy, is calculated as shown in equation (2):

CnTF

TER = Reval = Mid = SSDET

TP + FN (2)

The discovery rate of the classifier, which is also the discovery rate of the small class of

Sampling is calculated as shown in equation (3):

SE

Precision = TE

TP + FE (3)

The F-measure is an evaluation criterion aimed at classifying unbalanced

Data sets aligned as shown in Equation (4): LU504988

N (led Vx Recall « Precision # * m SEE = SS Spa &° x Recall + Precision (4) b is a coefficient for regulating precision and recall, which usually takes the value 1, i.e. Fl, and becomes the F1 value for evaluating the unbalanced

Classification performance used.

The advantageous effects of the present invention compared to the prior art

Technology are:

The present invention proposes an integrated classification method for imbalanced data based on fuzzy C-means clustering through fuzzy

C-means clustering (FuzzyC-Means clustering, FCM), a clustering method that uses a

Fuzzy matrix constructed based on the attribute characteristics of the research object itself and the classification relationship based on the calculated

Degree of membership determined. The fuzzy C-means clustering algorithm has the advantages of easy classification and high classification accuracy. From the perspective of the

Data preprocessing, the fuzzy C-mean clustering algorithm is used to subsample the unbalanced data to obtain the balanced data, and the built-in learning algorithm is used to improve the classification accuracy of some classes. The basic idea is to use the fuzzy C-means clustering algorithm to subsample the majority class samples in the unbalanced data set, so that the balanced data set can consist of all cluster centroid samples and all minority class samples. To classify the balanced

Data set, a built-in learning algorithm based on bagging is used, which ultimately leads to better classification performance.

Description of the attached drawings

Figure 1 shows a diagram of the distribution of data features before clustering of the present invention;

Figure 2 shows a diagram of the distribution of data features after clustering of the present invention;

Figure 3 shows a flowchart of the implementation of bagging in the sense presented here

Invention.

Detailed description

In order to more clearly understand the purpose of the present invention, the technical solutions for a clear and complete description, and the advantages are as follows

Embodiments of the present invention are described in detail in connection with the accompanying

Drawings described. It should be understood that the specific embodiments described herein are part of, not all, the embodiments of the present invention

Embodiments, only for the explanation of the embodiments of the present invention, and is not used to limit the embodiments of the present invention, all other embodiments that can be made by the ordinary person skilled in the art, without creative work under the premise of protecting the present Invention is part of that

Scope of the present invention.

Referring to Figures 1 to 3, the present invention provides a technical

Solution: a method for classifying undersampled integrated imbalanced data based on fuzzy C-means clustering, where ah 4504988

Method includes the following steps: (1) The subsampling process based on the fuzzy C-means clustering ECFCM

Algorithm has two main processes: First, the samples are taken from the majority class

Using the FCM algorithm clustered and subsampled, the k value is set to the number of

Samples of the minority class (i.e. 1kn) are set, and the k cluster centers form the samples in place of the original data samples of the majority class. Therefore, the obtained balanced data set consists of two similar numbers of data samples. Secondly, the

Classifier trained on the obtained balanced data set and there are several

Classification experiments conducted. Compared with random under-sampling, under-sampling based on fuzzy C-mean clustering can reduce the number of samples for the majority class while reducing the information loss of the samples for the

Avoid majority class as much as possible. The under-sampling process is described as follows:

Input: Unbalanced data sets

Output: balanced data sets f°

Initialization: =, dy =, k=0

For x'ed oy =0 ded Ui

Else dy = dy U ix’)

Endif k = card{d" ds = fom Clustertk, a, } d'=d' Udy

EndFor

The above pseudocode describes the process of under-sampling based on the

Fuzzy C-mean clustering. With the under-sampling process, the data set is first divided into one

set of minority and a set of majority samples; then the majority samples are clustered, and the centers of the resulting clusters and all

Minority samples form a balanced data set. To ensure that the

The number of samples of the majority class is balanced with the number of samples of the minority class, the number of clusters of the majority class is set to the number of samples of the minority class.

Although the number of samples of the majority class varies according to the fuzzy C-means

Clustering reduces, the spatial distribution of the data does not change, and the spatial

Distribution of data before and after clustering is shown in Figures 1 and 2. Beiht/504988

Under-sampling based on fuzzy C-means clustering, the data between classes is switched from unbalanced to balanced using the clustering method, while the characteristics of the spatial distribution of the data remain unchanged. The 5 built-in classification process is based on the built-in learning algorithm to learn the rules for the balanced data and build a classification model. (2) Integrated learning training process learning there are mainly two types, namely, there is no strong dependency between base classifiers that need to generate classification models in parallel, as well as there is a strong dependency between base classifiers that need to be generated serially, to classify the model, the first is typically by the

Bagging algorithm is represented, and the latter is typically represented by the boosting

Algorithm represented. Since the boosting algorithm leads to some practical problems

Over-fitting problem, it can provide worse classification results than a single classifier, while the bagging algorithm can better avoid such problems, so in this paper, the bagging algorithm is chosen as the classifier for integrated learning.

Bagging implementation steps are: given a base classifier and T

Training sets, each training set consists of the first data set (the total number of samples for N) in the random nnN (63.2%) ~x of the sample composition, base classifier

Training T rounds to T prediction function Ft, with T prediction function each on the

Test set prediction, and then in accordance with the majority of voting method to produce the final prediction results. The steps of bagging implementation are shown in Figure 3. (3) Integration of learning testing process

Assuming that the computational complexity of the base classifier is O(m) and the

The computational complexity of sampling and voting/averaging is O(s).

Computational complexity of Bagging is about t(O(m)+O(s)), and O(s) is generally small, so t is a small constant, and thus one can see that Bagging is a more efficient integrated

learning process is. In addition, bagging is based on the self-sampling method, and the

Advantage of self-sampling is reflected in the following: Since each base classifier uses only about 63.2% of the samples in the initial training set, the other 36.8% of the samples are used as the validation set for the out-of-package estimation of the

Generalization performance used.

If the base classifier is a decision tree, the out-of-package

Sampling is used to estimate the posterior probability of each node

Decision tree estimation and processing nodes with null

Training samples to help. Among the classic classification algorithms, KNN is one

Lower time complexity algorithm commonly used in integrated bagging learning. Based on the above considerations, Decision-Stump and

ANN chosen as the base classifiers for the integrated learning algorithm.

When reviewing the ECFCM algorithm, to ensure the objectivity of the

Experiment used the method of 10-fold cross-validation, i.e. the data of

Data set will be randomly divided into 10 equal parts, 9 of which will be

Training set and 1 as test set are used to conduct the experiments in order, and the average of the results of the corresponding 10 experiments becomes a14/504988

Final result of 10-fold cross validation used. (4) Experimental analysis 4.1 Experimental data set

To evaluate the effectiveness of the ECFCM algorithm in processing data with a lower imbalance rate (imbalance rate less than 5) and a larger

To test imbalance rate (imbalance rate above 5), four sets of

Disequilibrium data sets, “UCI” and KEEL®, were selected for the Matlab simulation (see

Table 1);

Table 1: 4 unbalanced data sets 2000

Data set source pes Aubaeeene Late

Spambase UCI 4 601 57 IntogerKral 1.54 ionosphere LN 351 34 Integer/Real 1.786

Segments? KEEL 2308 19 Reals 5.02

Hate? KEEL $214 Real 11.58 4.2 Experimental Valuation Index

The convention in the unbalanced dataset minority class samples as P,

Majority class samples as N, construct the confusion matrix as shown in Table 2:

Table 2: Binary confusion matrix

Positive category TP FN

Reverse category Fp TN

TN refers to the original majority class, and the prediction is still the majority class. Based on the confusion matrix, a number of

Evaluation criteria such as overall accuracy (Acc), correctness rate (TPR),

Recall, MIA, Precision, F-Measure, AUC, etc. defined;

The overall accuracy of the classifier is calculated as shown in equation (1):

TP+IN £8 Lié +. SOU EN (1)

The verification rate of the classifier, also known as subclass accuracy or positive accuracy, is calculated as shown in equation (2):

J LU504988

TPR = Recall = Mid = 7

TEA

The discovery rate of the classifier, which is also the discovery rate of the small class of

Sampling is calculated as shown in equation (3):

Precision = AE

FEL FR (3)

The F-measure is an evaluation criterion aimed at classifying unbalanced

datasets, as shown in equation (4): _ {1+ & Vo Recall x Precision & x Recall + Precision (4) b is the coefficient that regulates precision and recall and usually takes the value 1, i.e. F1 , and the value of F1 is used for evaluating the unbalanced classification performance;

The ROC curve is a two-dimensional graph with the false positive rate (FP) as x-

axis and the true rate (TP) as the y-axis, and the area under the ROC curve is represented by AUC, which is often used to evaluate the performance of the classifier, so in this work the AUC value as one of the indicators used for evaluating the classifiers; 4.3 Experimental platform

The experimental environment is a 64-bit Mac OS Sierra operating system, 8GB RAM,

Intel Corei5 processor, with MatlabR2017a for simulation experiments, data preprocessing with

ClassificationLearner application package. 4.4 Results analysis

Through simulation experiments, the classification performance of the decision stump

Algorithm, the KNN algorithm, the Bagging algorithm based on Decision-Stump and the Bagging algorithm based on KNN were compared. The first two algorithms belong to the traditional classification algorithms, the latter two to the integrated ones

Classification algorithms based on base classifiers. The experimental ones

Results are shown in Tables 3 to 6. The black numbers in the table indicate the experimental results with the best performance index;

Table 3 Comparison of the performance indices of the experiments with the Spambase dataset

TTS

LT NE

Tecision-Brenp GT GER OXF

KENN AN {13} GS

Unbalanced posing (Devision-Stanp) 095 099 0.90

Bagging (ANN) {SI {EST eg _

Dogisicn-Stumg 180 108 ER

Balanced CAN 9.54 fd {1,534

ECPOM Bagging (Deckslon-Stump} 180 1880 am

Haggine {RAS} $.92 8.98 3h92

Table 4 Comparison of individual performance metrics for experiments on the data set

Ionosphere ee ess ss Oï a es

Decision-Sremp It BH HASH

KKK 087 883 05

Lnbalanced Bagginp {Dection-Siuem 8.03 4.58 BR

Hagging (KNN} 282 196 83

Decision Stamp 4.94 &.95 HE

Balanced CORE is 8.98 8.98

ECTUM Bagging {Decipen-Rlumpr 0 1.80 HA

Hagging (KNN} 4.93 8.95 8233

Table 5 Comparison of individual performance metrics for experiments on the data set

Segment0 co Ii NN oo ,.,

Motto Bin de (98 88

Unaus Zewo ge N BN R 2 . 83 #83 3 BE

Bagging (Deciston-Sump 096 LE gas ee Bag RNIN TO 096

Pillow Siena 1 Lie ss

Balanced KNN #25 1.69 4.96

BOFCM Bazgme (Devisson-Shangy 1.98 its 138

DB (RN A BRS ORE

Table 6 Comparison of individual performance metrics for experiments on the dataset

Glass2

Deeps Alsi eee a A

DegisionNtuny Use 867 ded ; KNN to 20862 887

UBAUSESWOBER ping (Docision-Stumpi 0.93 088 0.86

Bagging (KENNY SOS AR

Peciston-Stenyy 834 882

Balanced RMN UNE SE | A86

ECFCM Hagging (Decision Stump} 497 1080 888

Bagging (KNK) gas 8863 055

From Table 3 to Table 6, the average improvement in Acc, AUC and F1 value of the ECFCM algorithm is calculated for balanced and unbalanced data, respectively, and the change in each index is expressed as a percentage (+ represents improvement, - for deterioration), as shown in Table 7;

Table 7: Improvement in classification performance of ECFCM algorithm (%)

Spanbasg +8.75 +453 +7.84 lonosphere +249 +463 +182

Segment £3.36 +0.52 -3.58

Glass? — 84 +1384 +8 0%

Avg 47 +3 8% +2.33

When using the ECFCM algorithm for classification, the Acc,

Algorithm AUC and F1 values on average up to 5.75% (Spambase), 13.84% (Glass2) and 7.54% (Spambase). For the SegmentO and Glass2 data sets, the Acc-

Values of the algorithm by 0.26% and 9.84%, respectively. The analysis shows that this is due to the imbalanced rate of the two datasets, 6.02 and 11.59, respectively. The larger the imbalance rate of the data set, the more valid samples are lost in the under-

Sampling lost in most classes, i.e. H. the loss of data samples at the edge of the

Classification affects the classification result, resulting in a reduction

classification accuracy leads;

Compared to the four sets of unbalanced data, the ECFCM improves

algorithm increased the AUC and F1 values by an average of 5.88% and 2.73% for the four sets of balanced data. On the balanced data set, the Bagging(Decision-

Stump) algorithm performed better on the three metrics than the other three algorithms. It can be seen that the Bagging (Decision-Stump) algorithm is better for the classification of

Minority samples are suitable. At the same time, the F1 values of the Decision-Stump and Bagging(Decision-Stump) algorithms are higher than their respective F1 values in the unbalanced data set, indicating that the ECFCM algorithm is able to ensure the completeness of the

Testing and improving the accuracy of testing of minority samples, i.e. H. it is more effective in classifying minority samples; LU504988

In summary, the ECFCM algorithm can effectively improve the comprehensive performance of classification of imbalanced data, especially when the

Imbalance rate of the imbalanced data is less than 5 and that it is the

Classification performance of a few classes of samples effectively improved, which makes the

Proves feasibility and effectiveness of the ECFCM algorithm in classifying imbalanced data.

Although embodiments of the present invention have been shown and described, it will be apparent to those skilled in the art that various changes, modifications,

Substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is limited by the appended claims and their equivalents.

Claims (6)

Claims LU504988 1. A method for classifying integrated imbalance data with subsampling based on fuzzy C-means clustering, characterized in that: the method comprises the steps of: subsampling based on fuzzy C-means clustering; training integration learning; testing integration learning; experimental analysis. 2. A method for classifying integrated imbalance data with undersampling based on fuzzy C-means clustering according to claim 1, characterized in that the undersampling based on fuzzy C-means clustering comprises the following specific operations: Subsampling the samples the majority class by clustering using the FCM algorithm, setting the value of k to the number of minority class samples, k=n1, k samples consisting of cluster centers instead of the original majority class data samples, and obtaining a balanced data set consisting of a similar number of data samples from the two classes. 3. A method for classifying integrated imbalance data with subsampling based on fuzzy C-means clustering according to claim 2, characterized in that subsampling based on fuzzy C-means clustering further comprises the following specific operations: training a classifier on the obtained balanced data set and conducting multiple classification experiments; dividing the set of data samples into a set of minority class samples and a set of majority class samples; clustering the majority class samples and forming a balanced data set with the obtained cluster centroids and all minority class samples; After subsampling, to ensure that the number of majority class samples is balanced with the number of minority class samples, the number of clusters of the majority class samples is set to the number of minority class samples. 4. A method for classifying undersampled integrated imbalance data based on fuzzy C-means clustering according to claim 1, characterized in that the specific operation of the integrated learning training method includes: There is no strong dependency between base classifiers, the classification model becomes parallel generated, and there is a strong dependency between base classifiers, the classification model needs to be generated serially, the parallel generation of the classification model is typically represented by the bagging algorithm, and the serial generation of the classification model is typically represented by the boosting algorithm. 5. A method for classifying undersampled integrated imbalance data based on fuzzy C-means clustering according to claim 1, characterized in that the specific operation of the integrated learning test comprises: selecting decision stump with KNN as a base classifier in which integrated learning algorithm. 6. A method for classifying integrated disequilibrium data with Subsampling based on fuzzy C-means clustering according to claim 1, characterized in that the specific operation of the experimental analysis comprises: Selecting a variety of unbalanced data sets for Matlab simulation; Conclude that the minority class samples in the unbalanced Dataset P and the majority class samples are N; According to the confusion matrix, a set of evaluation indices for overall accuracy, correctness rate, check-all rate, minority class accuracy, correctness-check rate, F-measure and AUC are defined, and the overall accuracy of the classifier is as shown in equation (1) calculated: TFA EN doo – EEE DPA IN + FP FN (1) The verification rate of the classifier, also known as subclass accuracy or positive accuracy, is calculated as shown in equation (2): TPR = Recall m Mid = pr TED The discovery rate of the classifier, which is also the discovery rate of the small class of samples, is calculated as shown in equation (3): "FE? Precision = EL PPAFP (3) The F-measure is an evaluation criterion aimed at the classification of unbalanced data sets, as shown in equation (4): N {1+ x Recall « Precision bw Recall + Precision (4) b is the coefficient that regulates precision and recall and usually takes the value 1, i.e. F1, and the value of F1 is used for evaluating the imbalanced classification performance.