CN110210529A - A kind of feature selection approach based on binary quanta particle swarm optimization - Google Patents

Abstract

The invention discloses a kind of feature selection approach based on binary quanta particle swarm optimization.Feature correlation analysis is carried out using maximum information coefficient, then feature selecting processing is carried out by improved BQPSO algorithm, carries out classification accuracy verifying using SVM later.Gene expression profile the experimental results showed that, based on improved BQPSO algorithm carry out feature selecting be a kind of practicable method.The present invention mainly improves the binary quanta particle colony optimization algorithm of standard, and the calculating of local attraction's has used the mode based on complete learning strategy, while introducing the variation thought of genetic algorithm to increase the diversity of population.Experiment shows that better classification accuracy can be obtained using improved BQPSO algorithm progress feature selecting.

Description

A Feature Selection Method Based on Binary Quantum Particle Swarm Algorithm

technical field

The invention belongs to the technical field of data mining, and relates to a feature selection method based on binary quantum particle swarm algorithm.

Background technique

In classification problems, data sets usually contain tens of thousands of features, including those related, irrelevant and redundant features. Due to the excessively large data set, it may even reduce the classification performance, which will cause the "curse of dimensionality" . Reducing the dimensionality of a data set through feature selection is one of the ways of data dimensionality reduction.

Feature selection occupies a very important position in the field of pattern recognition, and has very high research value. On the one hand, feature selection can effectively reduce the amount of data to be processed and reduce computing overhead; on the other hand, feature selection can eliminate non-critical interference features, reduce the correlation between features, and enhance the effectiveness of features.

Currently, there are feature selection methods based on filtering, wrapping, and embedding. The wrapper method uses a classifier to evaluate the generated subset of features. Filtering, on the other hand, evaluates subsets of features in terms of their information content and statistical measures. In general, wrapping methods give better results than filtering methods, but are more computationally intensive. The classifier construction process of the embedding method is also a feature selection process. How to design an effective feature selection method is an important problem facing high-dimensional data.

Contents of the invention

The purpose of the present invention is to propose a feature selection method based on binary quantum particle swarm algorithm in view of the existing demand for feature selection of high-dimensional and small-sample data. This method uses the maximal information coefficient (MIC for short) (see DN, R., et al.'s paper Detecting novelassociations in large data sets. Science (New York, N.Y.), 2011.334 (6062)) for data preprocessing , delete weakly correlated features, and then perform feature selection operations through the improved Binary Quantum Particle Swarm Optimization (BQPSO) algorithm, and then use SVM to verify the classification accuracy of the selected features to maintain a high accuracy rate.

Concrete steps of the present invention are as follows:

Step 1: Input public dataset;

Step 2: Use the maximum information coefficient MIC to calculate the correlation between the characteristics of each data field and the class label, set the correlation less than the threshold as the weakly correlated feature, and delete the weakly correlated feature;

Use the maximum information coefficient MIC to calculate the correlation between each feature and class mark, specifically:

Among them, X is the sample feature, Y is the class label, and B takes the 0.6 or 0.55 power of the total amount of data.

Step 3: For strongly correlated features, use the mutation idea of genetic algorithm and binary quantum particle swarm algorithm to select the optimal feature subset;

Specifically:

1) Initialize the population;

2) Calculate the fitness value of each particle in the population according to the fitness function, and compare it with the previous local optimal value, if f(x _i )<f(pbest _i ), then pbest _i =x _i , otherwise not renew;

3) Calculate the population optimal value gbest, and calculate the average optimal value mbest;

4) Calculate the local attractor p _i , and calculate the new position update probability pr of the particle;

5) Update the value of _xi according to the function Transf(p _i , pr);

6) Screen out the particles with poor fitness values, and use the mutation idea of genetic algorithm to mutate the particles with poor fitness values with the probability of Pm, thereby increasing the diversity of particle swarms;

7) Judging whether the termination condition is met, if not, then return to Step4), otherwise enter the next step;

8) Output the optimal feature subset;

where the fitness function is:

Among them, w _A is the weight of SVM classification accuracy rate, w _F is the weight of the number of features related to the class standard strength, sum(chrom) refers to the number of features related to the class standard strength, Acc is the classification accuracy rate based on the selected features, mic_c is obtained by calculating the correlation between features and class labels through the maximum linear coefficient MIC; mic_f is calculated by calculating the correlation between features and features through the maximum information coefficient MIC;

Step 4: Use the support vector machine algorithm to verify and evaluate the validity of the selected feature subset.

Beneficial effects of the present invention: the present invention mainly improves the standard binary quantum particle swarm optimization algorithm. The calculation of the local attractor uses a method based on a complete learning strategy, and at the same time introduces the variation idea of the genetic algorithm to increase the diversity of the particle swarm sex. Experiments show that using the improved BQPSO algorithm for feature selection can get better classification accuracy.

Description of drawings

Fig. 1 is the general flowchart of algorithm of the present invention;

Fig. 2 is the binary quantum particle swarm algorithm flowchart of the present invention;

Fig. 3 is a feature subset obtained by the present invention for the Lymphoma lymphoma data set, and the classification accuracy is obtained by a Support Vector Machine (SVM).

Detailed ways

As shown in Figure 1, a feature selection method based on binary quantum particle swarm algorithm, the specific steps are as follows:

Step 1. Input the public dataset Lymphoma, where the number of samples is 45, the number of features is 4026, the number of negative samples is 22, and the number of positive samples is 23.

Step 2. Using the maximum information coefficient (MIC) to calculate the correlation between all features and class labels. The calculation method of MIC is shown in formula (1) (2).

Step 3. The features are sorted according to the relevance of the MIC value, and some weakly correlated features are deleted according to the set threshold.

Step 4. Use the binary particle swarm optimization algorithm to search and optimize the remaining features to obtain the optimal feature subset. The specific algorithm flow chart is shown in Figure 2.

In the BQPSO algorithm, there is no concept of velocity and trajectory, only the concept of particle position and distance between particles. The distance between two particles is represented by the Hamming distance. In QPSO, p _i is the local attractor of the calculation population, the value of p _id is between pbest _id and gbest _d , p _i =(p _i1 ,p _i2 ,...p _iD ) is located between pbest _i and gbest In the supermatrix at both ends of the diagonal, the distance from p _i to pbest _i or gbest must be less than the length of the diagonal, that is, the following inequality must be satisfied:

|p _i -pbest _i |≤|pbest _i -gbest| (3)

|p _i -gbest|≤|pbest _i -gbest| (4)

Through the calculation of the local attractor p _i , the population can be diversified and jump out of the local search area of the particle. In the BQPSO algorithm, the generation method of p _i is different from that of the QPSO algorithm, and a new offspring is generated through the random intersection of each bit in the parent pbest _i and gbest.

With the deepening of the PSO algorithm iteration, the particles tend to converge prematurely and fall into the local optimal solution. In order to solve this problem, the mutation idea of genetic algorithm is introduced, and some particles with poor fitness are mutated with the probability of Pm, so as to increase the diversity of the particle swarm and prevent the particles from falling into the local optimal solution prematurely.

This method hopes to obtain higher classification accuracy while selecting fewer features. Therefore, the fitness function of the design algorithm is formula (3):

Among them, sum(chrom) refers to the number of selected features for each population, and Acc is the accuracy rate obtained by classifying according to the selected features. This method uses a binary classifier SVM to classify and model samples according to the feature subset of each population, and uses fitness to evaluate the effect. The fitness function makes the number of selected features as small as possible while making the classification error rate as low as possible. The process of selecting features by binary quantum particle swarm algorithm is as follows:

1) Initialize the population.

2) Calculate the fitness value of each particle in the population according to the fitness function, and compare it with the previous local optimal value, if f(x _i )<f(pbest _i ), then pbest _i =x _i , otherwise Not updated.

3) Calculate the population optimal value gbest, and calculate the average optimal value mbest.

4) Calculate the local attractor p _i , and calculate the update probability pr of the particle's new position.

5) Update the value of _xi according to the function Transf(p _i , pr).

6) Screen out the particles with poor fitness value, and use the mutation idea of genetic algorithm to mutate the particles with poor fitness value with the probability of Pm, so as to increase the diversity of particle swarms.

7) Judging whether the termination condition is satisfied, if not, return to Step4, otherwise enter the next step.

8) Output the optimal chromosome, that is, the optimal 01 string, where 0 indicates that the feature is not selected, and 1 indicates that the feature is selected.

Step 5. The above four steps are repeated multiple times to obtain the selected feature subset. Each resulting subset of features is validated using ten-fold cross-validation. Schematic diagram of the classification accuracy comparison between the improved BQPSO algorithm and the BQPSO algorithm obtained through support vector machine classification modeling (see Figure 3).

Claims (2)

1. a feature selection method based on binary quantum particle swarm algorithm, is characterized in that: the concrete steps of this method are as follows: Step 1: Input public dataset; Step 2: Use the maximum information coefficient MIC to calculate the correlation between the characteristics of each data field and the class label, set the correlation less than the threshold as the weakly correlated feature, and delete the weakly correlated feature; Step 3: For strongly correlated features, use the mutation idea of genetic algorithm and binary quantum particle swarm algorithm to select the optimal feature subset; Specifically: 1) Initialize the population; 2) Calculate the fitness value of each particle in the population according to the fitness function, and compare it with the previous local optimal value, if f(x _i )<f(pbest _i ), then pbest _i =x _i , otherwise not renew; 3) Calculate the population optimal value gbest, and calculate the average optimal value mbest; 4) Calculate the local attractor p _i , and calculate the new position update probability pr of the particle; 5) Update the value of _xi according to the function Transf(p _i , pr); 6) Screen out the particles with poor fitness values, and use the mutation idea of genetic algorithm to mutate the particles with poor fitness values with the probability of Pm, thereby increasing the diversity of particle swarms; 7) Judging whether the termination condition is met, if not, then return to Step4), otherwise enter the next step; 8) Output the optimal feature subset; where the fitness function is: Among them, w _A is the weight of SVM classification accuracy rate, w _F is the weight of the number of features related to the class standard strength, sum(chrom) refers to the number of features related to the class standard strength, Acc is the classification accuracy rate based on the selected features, mic_c is obtained by calculating the correlation between features and class labels through the maximum linear coefficient MIC; mic_f is calculated by calculating the correlation between features and features through the maximum information coefficient MIC; Step 4: Use the support vector machine algorithm to verify and evaluate the validity of the selected feature subset. 2. a kind of feature selection method based on binary quantum particle swarm algorithm according to claim 1, is characterized in that: use maximum information coefficient MIC to calculate the correlation of each feature and class mark, Specifically: Among them, X is the sample feature, Y is the class label, and B takes the 0.6 or 0.55 power of the total amount of data.