CN111460161A - A method for extracting topic-related genes from unsupervised text for unbalanced large datasets - Google Patents

Abstract

The invention discloses an unsupervised text topic-related gene extraction method for unbalanced large data sets. Factor analysis and density peak algorithm are used to obtain clusters of high-dimensional sample sets, and unlabeled samples are marked; the average local density and Information entropy is used to improve the feature selection method based on the CHI statistical matrix, and the feature expression degree of low-density and small sample clusters is strengthened; the fast fixed-point algorithm based on negative entropy is used to analyze the high-order statistical correlation between multi-dimensional data and extract independent hidden features. Contains subject eigengenes and completes the removal of high-order redundancy between components. There is no need to use large-scale labeled samples for training, which can effectively avoid the pre-definition of sample category relationships and feature structures; overcome the impact of over-sampling or under-sampling methods on the category distribution of the original unbalanced dataset. By revising the feature category structure, the performance of the CHI statistical selection method is improved; it also achieves effective feature dimensionality reduction while maintaining the ability to identify the sample set.

Description

A method for extracting topic-related genes from unsupervised text for unbalanced large datasets

technical field

The invention belongs to the technical field of data interpretation and topic discovery in natural language processing, and in particular relates to an unsupervised text topic-related gene extraction method for unbalanced large data sets.

Background technique

As society gradually enters the era of "big data", people acquire more and more information through web pages, microblogs, forums, etc., while spending less and less time reading and organizing information. Accurately analyzing the subject of information has become an effective means to realize big data understanding and value discovery, and its application fields cover many aspects such as Internet public opinion monitoring and early warning, network harmful information filtering, and sentiment analysis. When processing data in these fields, it is often necessary to deal with a large number of high-dimensional data with redundant or irrelevant features, which greatly reduces the efficiency and performance of learning algorithms. One link also directly affects the efficiency and accuracy of model construction and analysis.

At present, feature extraction can be divided into two categories: supervised and unsupervised, according to different categories of information. In the process of text content analysis, no matter what kind of category is adopted, it is necessary to use the Vector Space Model to represent the text as a vector space composed of a certain number of feature words, which inevitably causes two problems in practical applications. :

① The distribution of sample categories (clusters) in the data set is not balanced, and as a metric function for the quality evaluation of feature subsets, whether it is correlation analysis and similarity analysis based on independence; or Euclidean distance, Mahalanobis distance based on distance distance; even the most widely used methods such as mutual information and information gain based on information entropy all adopt the same or similar distribution of sample categories (clusters) in the dataset, so that most of the determined features come from categories (clusters). ) The number (density) is dominant in the "big class", and none or very few parts come from the non-dominant "small class", resulting in the most discriminative feature subset selected, which cannot accurately reflect the real information in the entire sample space , reducing the performance of subsequent learning methods to solve practical problems;

② "Big data" makes the objects to be processed become more complex, and the data dimension shows an explosive growth. Facing the ultra-high-dimensional data set, it not only means huge memory requirements, but also means high computing costs. In these high-dimensional feature spaces, there is a strong correlation between a large number of feature points, resulting in the introduction of a lot of redundancy and even noise, which makes the generalization ability of the feature items selected by the traditional method deteriorate sharply. The phenomenon of "empty space" also makes the problem of multivariate density estimation very difficult. How to extract the essential features of things from the complex representational information, that is, to find out the independent and hidden potential information, to complete the removal of high-order redundancy, to extract the complete and independent theme-related gene data, and to improve the feature items. The generalization ability is more important.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide an unsupervised text subject-related gene extraction method for unbalanced large data sets in view of the deficiencies in the above-mentioned prior art, which effectively avoids the pre-definition of sample category relationships and feature structures, and Overcome the impact of oversampling or undersampling on the class distribution of the original imbalanced dataset.

The present invention adopts following technical scheme:

An unsupervised text topic-related gene extraction method for imbalanced large datasets, including the following steps:

S1. Use factor analysis to reduce the dimension of the high-dimensional samples in the unlabeled sample set, and output the feature index matrix of the sample set;

S2. For each sample represented by a common factor, analyze the local density and the distance to the point with higher local density, draw a decision diagram, and use the fast search and density peak discovery algorithm to perform exploratory clustering on the dimensionality-reduced sample set , obtain C clustering divisions of n samples, and output the clustering divisions of the sample set;

S3. Use information entropy and average local density to improve the χ ² statistic, construct a sample feature distribution matrix based on the weighted χ ² statistic, weight the features and sample categories in the sample set χ ² statistic, and pass the weighted χ ² Statistics constructs a new statistical matrix to represent the weighted probability distribution of features in different categories and the same category, and performs feature selection to obtain feature subsets T=t ₁ , t ₂ ,...t _p ;

S4. Use the fast fixed-point algorithm based on negative entropy to analyze the high-order statistical correlation between the data in the multi-dimensional feature subset T=t ₁ , t ₂ , ... t _p , extract independent feature genes, and complete the high-order redundancy between components. remove.

Specifically, step S1 is specifically:

S101. Assume that the sample set X includes n samples x ₁ , x ₂ , . . . , x _n , and each sample x _i is composed of m characteristic indicators, denoted as X=(x _ij ) _n×m =(X ₁ ,X ₂ ,...,X _m ), perform KMO test on the correlation between samples, and jump to step S102 when the KMO statistic is greater than 0.5, otherwise jump to step S106;

S102. Calculate the eigenvalues and eigenvectors of the covariance matrix Σ=(h _ij ) _m×m of the sample set X ₁ , X ₂ ,...,X _m , and determine according to the percentage of the sum of eigen roots in the sum of all eigen roots the number of common factors;

S103, calculating the factor load matrix, when there is no obvious difference in the load of each factor on different characteristic indexes, jump to step S104, otherwise jump to step S105;

S104, using the orthogonal rotation method to rotate the factor loading matrix;

S105. Evaluate the load of the characteristic index in the factor load matrix in the corresponding common factor, and retain the maximum load value;

S106, output the feature index matrix of the sample set X.

Further, in step S106, each sample x _i is composed of a finite sample set XΔ composed of u characteristic index factors, and the characteristic index matrix X* of n samples is specifically:

表示第i个样本的第j个特性指标因子，i＝1,2,…,n；j＝1,2,…,u，。in,

Indicates the j-th characteristic index factor of the i-th sample, i=1,2,...,n; j=1,2,...,u,.

Specifically, step S2 is specifically:

之间的距离Sim(i,j)；S201. Using the adjusted cosine similarity, calculate the similarity between samples to define the variable d _ij , and calculate any two data points

The distance between Sim(i,j);

的局部密度

和该点到具有更高局部密度点的距离

S202. Select an appropriate cutoff distance to calculate any data point in X ^*

the local density of

and the distance from that point to the point with higher local density

为横轴，以

为纵轴，绘制决策图；S203. According to the local density of all the sample points and the distance value from the point with higher local density, to

is the horizontal axis, with

As the vertical axis, draw a decision diagram;

和

的簇中心点及噪声点；S204. Use the decision diagram to mark the sample set

and

The cluster center point and noise point of ;

S205: Allocate the remaining points to obtain C cluster divisions of the n samples, and output the cluster divisions of the sample set, which are used as the basis for the next analysis.

之间的相似度Sim(i,j)定义如下：Further, in step S201, the sample

The similarity between Sim(i,j) is defined as follows:

u为对象的属性数，截断距离d_c是以数据点x_i为圆心，以d_c为半径，ρ_i累计个数满足|X|×2％。Among them, i,j=1,2,...,n,

u is the number of attributes of the object, the cut-off distance d _c takes the data point x _i as the center of the circle, and d _c as the radius, and the cumulative number of ρ _i satisfies |X|×2%.

Specifically, step S3 is specifically:

S301, using the information entropy value of the feature and the sample category (cluster) to weight the χ ² statistic of the sample set;

S302, construct a new statistical matrix K by using the weighted χ ² statistic, and the rows and columns in K are respectively represented as weighted probability distributions of features in different categories (clusters) and the same category (clusters);

及

S303. Select each row _ti in the statistical matrix K in turn, and search for the

and

将t_i转化成对应的隶属度μ_ij，并构造新类别向量

b_ij为t_i中，按照降序排列的隶属度μ_ij；S304, pass

Convert t _i to the corresponding degree of membership μ _ij , and construct a new category vector

b _ij is the membership degree μ _ij in descending order in _ti ;

S305. Calculate the sum of contributions of the feature t _i to various types of offers;

S306. Calculate the cumulative variance contribution rate;

时，得到特征子集T＝t₁,t₂,…t_p。S307. Repeat steps S303 to S306,

When , the feature subset T=t ₁ , t ₂ ,...t _p is obtained.

Further, in step S301, in the χ ² statistic, the feature t and the sample category c _i are weighted, and the weighted χ ² statistic is defined as Wχ ² (t, c _i ), and the weight is defined as the feature t. and the information entropy value of the sample category c _i , specifically:

Wχ ² (t, c _i ) is expressed as

为样本类别c_i中样本点的平均局部密度，C＝{c₁,c₂,…,c_k}表示样本类别集合；

的定义为

c_i.rep表示簇c_i中的样本点。Among them, p(t|c _i ) is the occurrence probability of feature t in sample category c _i , p(c _i ) is the probability of occurrence of sample category c _i , p(t, c _i ) is the occurrence probability of sample category c _i the probability of feature t,

is the average local density of sample points in sample category c _i , C={c ₁ ,c ₂ ,...,c _k } represents the set of sample categories;

is defined as

c _i .rep represents the sample points in cluster c _i .

Further, in step S302, the statistical matrix K is expressed as:

Among them, the row and column represent the weighted probability distribution of features in different categories and the same category, respectively.

Specifically, step S4 is specifically:

S401. Center the feature subset T=t ₁ , t ₂ , . . . t _p to make the mean value 0;

进行白化得到z；S402, to the centralized feature subset

Perform whitening to get z;

S403, select the number m of independent components to be estimated, and set i=1;

S404, select an initialization (can be randomly selected) vector w _i with unit norm;

函数g为非二次型函数G的导数；S405, update

The function g is the derivative of the non-quadratic function G;

S406. Standardize w _i , w _i ←w _i /||w _i ||;

S407, if it has not converged, return to step S405;

S408, let i←i+1, if i≤m, return to step S404.

Compared with the prior art, the present invention at least has the following beneficial effects:

The present invention is oriented to the unsupervised text topic related gene extraction method for unbalanced large data sets, does not need to use large-scale labeled samples for training, can effectively avoid the pre-definition of sample category relationships and feature structures, and has more practical value: most of the The samples obtained by crawling methods are not labeled with categories, which makes it difficult to effectively implement traditional supervised topic discovery methods. The present invention is based on an unsupervised feature extraction method, so there is no such limitation; it overcomes the influence of the over-sampling or under-sampling method on the class distribution of the original unbalanced data set. Through the modification of the feature category structure, the real information in the sample space is accurately reflected, and it has stronger generalization in the face of unbalanced large data sets; the present invention realizes effective features while maintaining the sample set identification ability. Dimensionality reduction, further reducing noise word interference, weakening the "empty space" phenomenon of high-dimensional data space, and reducing uncertainty in sample analysis.

Furthermore, factor analysis method is used to find the optimal low-dimensional base to describe the original high-dimensional vector space, which provides the possibility for the density peak algorithm to quickly find the sample clusters of large-scale data sets.

Further, the clustering algorithm of the density peak is guided by the neighborhood similarity of the sample points to realize the clustering and automatic labeling of the unlabeled text set.

Further, the average local density and information entropy are introduced into the weight definition of feature items, so as to construct the discrimination matrix of feature items to sample categories (clusters), so as to eliminate the defects of traditional methods for feature selection of unbalanced sample sets.

Further, the independent component analysis method (ICA) is used to find out the implicit information components that are independent of each other by analyzing the high-order correlation between multi-dimensional statistical data, so as to achieve a more accurate selection of comprehensive and true reflections in the unbalanced large data set. The optimal feature subset of text topic information to improve the performance of text classification and recognition.

To sum up, the present invention focuses on unsupervised text feature extraction, and studies how to select a stable and strong generalization ability of a subset of text topic-related genes, thereby reducing the feature dimension of the vector space and enhancing the category (cluster) representation of feature words. ability to improve the classification and recognition effect.

The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments.

Description of drawings

Fig. 1 is the overall flow chart of the unsupervised text subject-related gene extraction method for unbalanced large data sets of the present invention;

Figure 2 is a flow chart of the sample feature analysis process;

Figure 3 is a flow chart of the sample clustering process;

Fig. 4 is a flow chart of feature selection process;

Figure 5 is a flow chart of the subject gene extraction process;

6 is a schematic diagram of the normalized mutual information value of each algorithm under the selection of different numbers of features of the present invention, wherein, (a) is the normalized mutual information (%) of each algorithm on the Sohu News Data (SogouCS) 20151022 corpus, ( b) is the normalized mutual information (%) of each algorithm on the Reuter-21578 corpus.

Detailed ways

The present invention provides an unsupervised text topic-related gene extraction method oriented to unbalanced large data sets, which adopts factor analysis and density peak algorithm to obtain clusters of high-dimensional sample sets, and labels unlabeled samples accordingly; Local density and information entropy are used to improve the feature selection method based on CHI statistical matrix, so as to strengthen the feature expression of low-density and small-sample clusters; the fast fixed-point algorithm (FastICA) based on negative entropy is used to analyze high-order inter-dimensional data. Statistical correlations are used to extract independent implicit theme eigengenes and complete the removal of high-order redundancy between components. This method does not require the use of large-scale labeled samples for training, which can effectively avoid the pre-definition of sample category relationships and feature structures; Impact. Through the modification of the feature category structure, the performance of the CHI statistical selection method is greatly improved; it also achieves effective feature dimension reduction while maintaining the sample set identification ability.

Referring to FIG. 1, a method for extracting related genes from unsupervised text topics oriented to unbalanced large data sets of the present invention includes the following steps:

S1. Use factor analysis to reduce the dimension of the high-dimensional samples in the unlabeled sample set, and output the feature index matrix of the sample set;

Factor analysis is performed on the original feature variables of the sample set, and a small number of "abstract" variables (ie common factors) are selected to replace the original feature variables, so as to reduce the correlation of sample features and reduce the dimension. The specific process is shown in Figure 2:

S101. Perform KMO test on the correlation between samples, and jump to step S102 when the KMO statistic is greater than 0.5, otherwise jump to step S106;

Suppose the sample set X contains n samples x ₁ , x ₂ ,...,x _n , each sample x _i is composed of m characteristic indicators, denoted as X=(x _ij ) _n×m =(X ₁ ,X ₂ , ..., X _m );

The KMO (Kaiser Meyer Olkin, KMO) test is used to determine the degree of correlation between the samples X ₁ , X ₂ , ..., X _m to determine the necessity of factor analysis. The closer the KMO statistic is to 0, the weaker the correlation of X ₁ , X ₂ ,…,X _m is; the closer the KMO statistic is to 1, the stronger the correlation of X ₁ , X ₂ ,…, X _m is.

Usually, when the KMO statistic is greater than 0.5, it is practical to perform factor analysis.

S102. Calculate the eigenroot and eigenvector of the sample set X ₁ , X ₂ ,...,X _m covariance matrix Σ=(h _ij ) _m×m , and according to the percentage of the sum of its eigen roots in the sum of all eigen roots, Determine the number of common factors;

From the characteristic equation of Σ |Σ-λI|=0, the characteristic root of the covariance matrix can be obtained as λ ₁ ≥λ ₂ ≥…≥λ _p ≥0, and the corresponding unit eigenvectors are T ₁ ,T ₂ ,…,T _p ;

In addition, according to the processing principle in practical problems, the first u eigenvalues and eigenvectors are taken, so that the sum of their eigenvalues accounts for more than 85% of the sum of all eigenvalues, so as to determine the number of common factors;

S103, calculating the factor load matrix, when there is no obvious difference in the load of each factor on different characteristic indexes, jump to step S104, otherwise jump to step S105;

The factor loading matrix is calculated using the eigenroots and eigenvectors of Σ as follows:

S104, using the orthogonal rotation method to rotate the factor loading matrix;

If there is no significant difference in the loading of each factor on different characteristic indicators, the factor loading matrix needs to be rotated. Usually, the orthogonal rotation method is used to rotate the factor loading matrix, and the rotated factor loading matrix A' is obtained as follows:

Perform b _ip =Max{b _i1 ,b _i2 ,...,b _iu }, i=1,2,...,m, p∈{1,2,...,u} on the rotated factor loading matrix A' row vector The operation of retaining the maximum load value b _ip of the characteristic index X _i in the u factors in the matrix A', the matrix is obtained as follows:

A ^* =(b' _ij ) _m×u

Among them, i=1,2,...,m; j=1,2,...,u;

S105. Evaluate the load of the characteristic index in the factor load matrix in the corresponding common factor, and retain its maximum load value;

S106, output the feature index matrix of the sample set, which is used as the basis for the next step of analysis.

The sample set X is simplified to include n samples through the above operation process, and each sample _xi is a finite sample set X ^Δ composed of u characteristic index factors, thus constructing the characteristic index matrix of n samples as follows:

表示第i个样本的第j个特性指标因子，i＝1,2,…,n；j＝1,2,…,u。in,

Indicates the jth characteristic index factor of the ith sample, i=1,2,...,n; j=1,2,...,u.

S2. Use fast search and density peak discovery algorithms to perform exploratory clustering on the dimensionality-reduced sample set;

For each sample represented by a common factor, analyze its local density and the distance to a point with a higher local density, draw a decision diagram, and generate a clustering partition of the sample set. The specific process is shown in Figure 3:

S201. Calculate the similarity between any two row vectors in the feature index matrix of the sample set;

之间的距离Sim(i,j)，样本

之间的相似度Sim(i,j)定义如下：Using the adjusted cosine similarity, calculate the similarity between samples to define the variable d _ij , calculate any two data points

distance between Sim(i,j), samples

The similarity between Sim(i,j) is defined as follows:

u为对象的属性数，截断距离d_c是以数据点x_i为圆心，以d_c为半径，ρ_i累计个数满足|X|×2％Among them, i,j=1,2,...,n,

u is the number of attributes of the object, the truncation distance d _c takes the data point x _i as the center of the circle, and d _c as the radius, the cumulative number of ρ _i satisfies |X|×2%

的局部密度

和该点到具有更高局部密度点的距离

S202. Select an appropriate cutoff distance to calculate any data point in X*

the local density of

and the distance from that point to the point with higher local density

数据点x_i到局部密度比它大且聚类最近的数据点x_j的距离为

其中，

d_ij为不同的数据点之间的距离，d_c为截断距离(超参数)。Let the local density of data point x _i be

The distance from a data point x _i to a data point x _j whose local density is greater than it and the cluster is closest to is

in,

d _ij is the distance between different data points, and d _c is the cutoff distance (hyperparameter).

为横轴，以

为纵轴，绘制决策图；S203. According to the local density of all the sample points and the distance value from the point with higher local density, to

is the horizontal axis, with

As the vertical axis, draw a decision diagram;

和

的簇中心点及噪声点；S204. Use the decision diagram to mark the sample set

and

The cluster center point and noise point of ;

Calculate ρ _{i and δ i of any data point xi, mark C data points of ρ i and δ i as} the center point of the cluster after sorting, and assign the remaining data points to its nearest neighbors and the data with higher density than it The cluster where the point is located.

S205: Allocate the remaining points to obtain C cluster divisions of the n samples, and output the cluster divisions of the sample set, which are used as the basis for the next analysis.

S3. Use information entropy and average local density to improve the χ ² statistic, construct a sample feature distribution matrix based on the weighted χ ² statistic, and weight the features and sample categories (clusters) in the sample set χ ² statistic, and the weights Defined as the information entropy value of the feature and the sample category (cluster), a new statistical matrix is constructed by the weighted χ ² statistic to represent the weighted probability distribution of the feature in different categories (clusters) and the same category (clusters). This is the basis for feature selection;

The specific process is shown in Figure 4:

S301, using the information entropy value of the feature and the sample category (cluster) to weight the χ ² statistic of the sample set;

In the χ ² statistic, the feature t and the sample category (cluster) c _i are weighted, and the weighted χ ² statistic is defined as Wχ ² (t, c _i ), and its weight is defined as the feature t and the sample category The information entropy value of (cluster) c _i , i.e.

Wχ ² (t, c _i ) is expressed as

为样本类别(簇)c_i中样本点的平均局部密度，C＝{c₁,c₂,…,c_k}表示样本类别(簇)集合。

的定义为

c_i.rep表示簇c_i中的样本点。Among them, p(t|c _i ) is the occurrence probability of feature t in the sample category (cluster) c _i , p(c _i ) is the occurrence probability of the sample category (cluster) c _i , and p(t, c _i ) is The probability of feature t appearing in sample category (cluster) c _i ,

is the average local density of sample points in the sample category (cluster) c _i , and C={c ₁ , c ₂ , . . . , c _k } represents the set of sample categories (clusters).

is defined as

c _i .rep represents the sample points in cluster c _i .

S302, construct a new statistical matrix K by using the weighted χ ² statistic, and the rows and columns in K are respectively represented as weighted probability distributions of features in different categories (clusters) and the same category (clusters);

Use the weighted χ ² statistic to construct a statistical matrix K, that is, K is expressed as

Among them, the rows and columns represent the weighted probability distribution of features in different categories (clusters) and the same category (clusters), respectively.

及

S303. Select each row _ti in the statistical matrix K in turn, and search for the

and

将t_i转化成对应的隶属度μ_ij，并构造新类别(簇)向量

b_ij为t_i中，按照降序排列的隶属度μ_ij；S304, pass

Convert t _i to the corresponding degree of membership μ _ij , and construct a new category (cluster) vector

b _ij is the membership degree μ _ij in descending order in _ti ;

S305. Calculate the sum of contributions provided by the feature t _i to each category;

具体为：The sum of the contributions provided by the feature t _i to each category

Specifically:

S306. Calculate the cumulative variance contribution rate;

具体为：Cumulative variance contribution rate

Specifically:

时，得到特征子集T＝t₁,t₂,…t_p。S307. Repeat steps S303 to S306,

When , the feature subset T=t ₁ , t ₂ ,...t _p is obtained.

S4. Using the fast fixed point algorithm (FastICA) based on negative entropy, analyze the high-order statistical correlation between the data in the multi-dimensional feature subset T=t ₁ , t ₂ , ... t _p , extract independent feature genes and complete the high-order correlation between components order redundancy removal.

Please refer to Figure 5. The specific process of subject-related gene extraction is as follows:

S401. Center the feature subset T=t ₁ , t ₂ , . . . t _p to make the mean value 0;

进行白化得到z；S402, to the centralized feature subset

Perform whitening to get z;

S403, select the number m of independent components to be estimated, and set i=1;

S404, select an initialization (can be randomly selected) vector w _i with unit norm;

函数g为非二次型函数G的导数；S405, update

The function g is the derivative of the non-quadratic function G;

S406. Standardize w _i , w _i ←w _i /||w _i ||;

S407, if it has not converged, return to step S405;

S408, let i←i+1, if i≤m, return to step S404.

Extracting theme-related genes is a key step in the preprocessing of industrial big data and social big data. By discovering mutually independent implicit information components, it is possible to more accurately select data that comprehensively and truly reflect text theme information in unbalanced big data sets. The optimal feature subset can significantly improve the recognition performance of the classifier.

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. The components of the embodiments of the invention generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations. Thus, the following detailed description of the embodiments of the invention provided in the accompanying drawings are not intended to limit the scope of the invention as claimed, but are merely representative of selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In order to further test the practical application effect of the present invention, the simulation experiment selects two corpora of Sohu news data (SogouCS) 20151022 and Reuter-21578, adopts k-means clustering algorithm, and analyzes the clustering result category information and original category information of each algorithm The normalized mutual information to measure the effectiveness of the algorithm. Since the number of clusters needs to be clearly given during k-means clustering, in order to reduce the influence of K value selection on the method, the number of clusters K of the method of the present invention and the comparison method is set to the number of categories contained in each data label, that is, 20, 10, 12. Figures 6(a) and (b) show the normalized mutual information values of each algorithm under the selection of different number of features.

It can be seen from Figures (a) and (b) that the method for extracting topic-related genes from unsupervised texts for unbalanced large data sets proposed by the present invention has obvious advantages over the other four algorithms. Figures (a) and (b) ), the method of the present invention can quickly achieve a better effect when the number of features is small. Therefore, using the algorithm proposed by the present invention for unsupervised feature selection has better effect than the general unsupervised feature selection algorithm.

To sum up, the present invention is an unsupervised text topic related gene extraction method for unbalanced large data sets, which does not require large-scale labeled samples for training, avoids pre-defining category relationships and related features, and overcomes sample categories. The problem of weak generalization ability of the model caused by the unbalanced distribution. Based on the text clustering method of finding peaks by fast search, the text feature distribution matrix of weighted χ ² statistic is constructed by using information entropy, avoiding the use of over-sampling or under-sampling methods to make the class distribution of the original unbalanced data set. The performance of the CHI statistical selection method is greatly improved by modifying the distribution of feature categories. Finally, the fast fixed-point algorithm based on negative entropy (FastICA) is used to extract independent implicit information components between multi-dimensional data. Feature dimensionality reduction in the case of discrimination ability.

In addition, for the industrial big data and social big data brought by informatization, feature dimensionality reduction is a key step in the preprocessing of these big data. The idea of extracting genes related to text topics proposed by the present invention will play a more important role in these big data fields. How to better adapt to the data processing needs of these fields is the research work to be carried out in the future.

The above content is only to illustrate the technical idea of the present invention, and cannot limit the protection scope of the present invention. Any changes made on the basis of the technical solution according to the technical idea proposed by the present invention all fall within the scope of the claims of the present invention. within the scope of protection.

Claims (9)

1. The method for extracting related genes of unsupervised text topics for unbalanced large data sets, is characterized in that, comprises the following steps: S1. Use factor analysis to reduce the dimension of the high-dimensional samples in the unlabeled sample set, and output the feature index matrix of the sample set; S2. For each sample represented by a common factor, analyze the local density and the distance to the point with higher local density, draw a decision diagram, and use the fast search and density peak discovery algorithm to perform exploratory clustering on the dimensionality-reduced sample set , obtain C clustering divisions of n samples, and output the clustering divisions of the sample set; S3. Use information entropy and average local density to improve the χ ² statistic, construct a sample feature distribution matrix based on the weighted χ ² statistic, weight the features and sample categories in the sample set χ ² statistic, and pass the weighted χ ² Statistics constructs a new statistical matrix to represent the weighted probability distribution of features in different categories and the same category, and performs feature selection to obtain feature subsets T=t ₁ , t ₂ ,...t _p ; S4. Use the fast fixed-point algorithm based on negative entropy to analyze the high-order statistical correlation between the data in the multi-dimensional feature subset T=t ₁ , t ₂ , ... t _p , extract independent feature genes, and complete the high-order redundancy between components. remove. 2. the method for extracting unsupervised text subject-related genes for unbalanced large data sets according to claim 1, is characterized in that, step S1 is specifically: S101. Assume that the sample set X includes n samples x ₁ , x ₂ , . . . , x _n , and each sample x _i is composed of m characteristic indicators, denoted as X=(x _ij ) _n×m =(X ₁ ,X ₂ ,...,X _m ), perform KMO test on the correlation between samples, and jump to step S102 when the KMO statistic is greater than 0.5, otherwise jump to step S106; S102. Calculate the eigenvalues and eigenvectors of the covariance matrix Σ=(h _ij ) _m×m of the sample set X ₁ , X ₂ ,...,X _m , and determine according to the percentage of the sum of eigen roots in the sum of all eigen roots the number of common factors; S103, calculating the factor load matrix, when there is no obvious difference in the load of each factor on different characteristic indexes, jump to step S104, otherwise jump to step S105; S104, using the orthogonal rotation method to rotate the factor loading matrix; S105. Evaluate the load of the characteristic index in the factor load matrix in the corresponding common factor, and retain the maximum load value; S106, output the feature index matrix of the sample set X. 3. The method for extracting unsupervised text topic-related genes for unbalanced large data sets according to claim 2, wherein in step S106, each sample x _i is composed of a limited sample set X consisting of u characteristic index factors ^Δ , the characteristic index matrix X ^* of n samples is specifically:

Wherein, x _ij ^* represents the j-th characteristic index factor of the i-th sample, i=1, 2,...,n; j=1, 2,...,u,. 4. The method for extracting unsupervised text subject-related genes for unbalanced large data sets according to claim 1, wherein step S2 is specifically: S201. Using the adjusted cosine similarity, calculate the similarity between samples to define the variable d _ij , and calculate any two data points

The distance between Sim(i,j); S202. Select an appropriate cutoff distance to calculate any data point in X ^*

the local density of

and the distance from that point to the point with higher local density

S203. According to the local density of all the sample points and the distance value from the point with higher local density, to

is the horizontal axis, with

As the vertical axis, draw a decision diagram; S204. Use the decision diagram to mark the sample set

and

The cluster center point and noise point of ; S205: Allocate the remaining points to obtain C cluster divisions of the n samples, and output the cluster divisions of the sample set, which are used as the basis for the next analysis. 5. The method for extracting unsupervised text topic-related genes for unbalanced large data sets according to claim 4, wherein in step S201, the sample

The similarity between Sim(i,j) is defined as follows:

Among them, i,j=1,2,...,n,

u is the number of attributes of the object, the cut-off distance d _c takes the data point x _i as the center of the circle, and d _c as the radius, and the cumulative number of ρ _i satisfies |X|×2%. 6. The method for extracting unsupervised text subject-related genes for unbalanced large data sets according to claim 1, wherein step S3 is specifically: S301, using the information entropy value of the feature and the sample category (cluster) to weight the χ ² statistic of the sample set; S302, construct a new statistical matrix K by using the weighted χ ² statistic, and the rows and columns in K are respectively represented as weighted probability distributions of features in different categories (clusters) and the same category (clusters); S303. Select each row _ti in the statistical matrix K in turn, and search for the

and

S304, pass

Convert t _i to the corresponding degree of membership μ _ij , and construct a new category vector

b _ij is the membership degree μ _ij in descending order in _ti ; S305. Calculate the sum of contributions of the feature t _i to various types of offers; S306. Calculate the cumulative variance contribution rate; S307. Repeat steps S303 to S306,

When , the feature subset T=t ₁ , t ₂ ,...t _p is obtained. 7. The method for extracting unsupervised text topic-related genes for unbalanced large data sets according to claim 6, wherein in step S301, in the χ ² statistic, the feature t and the sample category c _i are weighted , and the weighted χ ² statistic is defined as Wχ ² (t, c _i ), and the weight is defined as the information entropy value of the feature t and the sample category c _i , specifically: Wχ ² (t, c _i ) is expressed as

Among them, p(t|c _i ) is the occurrence probability of feature t in sample category c _i , p(c _i ) is the probability of occurrence of sample category c _i , p(t, c _i ) is the occurrence probability of sample category c _i the probability of feature t,

is the average local density of sample points in sample category c _i , C={c ₁ ,c ₂ ,...,c _k } represents the set of sample categories;

is defined as

c _i .rep represents the sample points in cluster c _i . 8. The method for extracting unsupervised text topic-related genes for unbalanced large data sets according to claim 6, wherein in step S302, the statistical matrix K is expressed as:

Among them, the row and column represent the weighted probability distribution of features in different categories and the same category, respectively. 9. The method for extracting unsupervised text subject-related genes for unbalanced large data sets according to claim 1, wherein step S4 is specifically: S401. Center the feature subset T=t ₁ , t ₂ , . . . t _p to make the mean value 0; S402, to the centralized feature subset

Perform whitening to get z; S403, select the number m of independent components to be estimated, let i=1; S404, select an initialization (can be randomly selected) vector w _i with unit norm; S405, update

The function g is the derivative of the non-quadratic function G; S406. Standardize w _i , w _i ←w _i /||w _i ||; S407, if it has not converged, return to step S405; S408, let i←i+1, if i≤m, return to step S404.

Images