CN111476100B - Data processing method, device and storage medium based on principal component analysis - Google Patents

Abstract

Embodiments of the present invention relate to the field of software defect prediction, and disclose a data processing method, device and computer-readable storage medium based on principal component analysis. The method includes: performing dimensionality reduction processing on initial sample data to obtain a preset dimension. Sample data; obtain multiple features of the sample data, and calculate the correlation between each feature and a preset category, wherein the preset category is one of multiple categories of the sample data; remove Among the features whose correlation degree is less than the preset correlation degree among the plurality of features, the remaining features are used as the identification features of the sample data. The data processing method, device and computer-readable storage medium based on principal component analysis provided by the present invention can remove redundant features in sample data and obtain sample data with high discriminability, thereby improving prediction efficiency.

Description

Data processing method, device and storage medium based on principal component analysis

Technical field

Embodiments of the present invention relate to the field of data processing, and in particular to a data processing method, device and computer-readable storage medium based on principal component analysis.

Background technique

Information entropy is a measure of the amount of information required to eliminate uncertainty, that is, the amount of information that unknown events may contain. An event or a system is, to be precise, a random variable, which has a certain degree of uncertainty. The uncertainty of some random variables is very high. To eliminate this uncertainty, a lot of information needs to be introduced. The measurement of this much information is expressed by "information entropy". The more information that needs to be introduced to eliminate uncertainty, the higher the information entropy, and vice versa. If a situation is very certain, there is little need to introduce information, so the information entropy is very low. According to the information entropy formula given by Shannon, for any random variable The more equal the probabilities of various randomnesses in the system, the greater the information entropy, and vice versa.

The inventor found that there are at least the following problems in the prior art: analyzing the characteristics of the sample data according to the above formula results in a large number of redundant features, resulting in low prediction efficiency of the model trained using the sample data.

Contents of the invention

The purpose of the embodiments of the present invention is to provide a data processing method, device and computer-readable storage medium based on principal component analysis, which can remove redundant features in sample data and obtain sample data with high discriminability, thereby improving prediction efficiency.

In order to solve the above technical problems, embodiments of the present invention provide a data processing method based on principal component analysis, including:

Perform dimensionality reduction processing on the initial sample data to obtain sample data with preset dimensions; obtain multiple features of the sample data, and calculate the correlation between each feature and the preset category, where the preset category is the One of the multiple categories of the sample data; remove the features whose correlation degree is less than the preset correlation degree among the multiple features, and use the remaining features as the identification features of the sample data.

An embodiment of the present invention also provides a data processing device based on principal component analysis, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores data that can be The instructions executed by the at least one processor are executed by the at least one processor so that the at least one processor can execute the above-mentioned data processing method based on principal component analysis.

Embodiments of the present invention also provide a computer-readable storage medium that stores a computer program. When the computer program is executed by a processor, the above-mentioned data processing method based on principal component analysis is implemented.

Compared with the existing technology, the embodiment of the present invention performs dimensionality reduction processing on the initial sample data to obtain sample data with preset dimensions, so as to facilitate the calculation of subsequent steps, reduce the calculation amount of subsequent steps, and thereby improve The efficiency of the data processing method; by obtaining multiple features of the sample data and calculating the correlation between each feature and the preset category. Since the preset category is one of the multiple categories of the sample data, through this In this way, it is possible to know which features among the multiple features of the sample data are redundant features based on the similarity; by removing the features whose correlation degree is less than the preset correlation among the multiple features, the remaining features are used as the redundant features of the sample data. The discriminant feature can obtain sample data with high discriminability, which makes the budget of the training model using the sample data faster, thereby improving the prediction efficiency.

In addition, after removing the features whose correlation degree is less than the preset correlation degree among the plurality of features, it also includes: sorting the remaining features in the order of the correlation degree from high to low; Divided into N feature segments, each feature segment includes M features, and N and M are both integers greater than 1; determine whether there are M feature segments with all features greater than the preset threshold. When determining whether there are, Remove the features with the smallest similarity among the feature segments.

In addition, performing dimensionality reduction processing on the initial sample data specifically includes: converting the initial sample data into a data matrix; calculating the covariance matrix of the data matrix, and performing eigendecomposition on the covariance matrix, Obtain the eigenvalues of the covariance matrix and the eigenvectors corresponding to the eigenvalues; obtain a projection matrix based on the eigenvalues and the eigenvectors, and reduce the dimension of the initial sample data to the projection matrix corresponding dimensions.

In addition, obtaining a projection matrix based on the eigenvalues and the eigenvector specifically includes: arranging the eigenvectors in rows from top to bottom into a matrix, wherein the larger the eigenvalue corresponding to the eigenvector, the greater the eigenvalue. The feature vector is located in the preceding row of the matrix; the first k rows are taken to form the projection matrix, where k is an integer greater than 1.

In addition, before calculating the covariance matrix of the data matrix, it also includes: performing zero-meaning processing on each row of the data matrix; and calculating the covariance matrix of the data matrix specifically includes: calculating zero-meaning The covariance matrix of the processed data matrix.

In addition, the correlation between the feature and the preset category is calculated through the following formula: Si=[X ^T ×Y+X` <sup>T</sup> ×Y+X <sup>T</sup> ×Y`+X` <sup>T</sup> ×Y`]+[2×(IG (X|L))-(H(X)+H(L))]+[2×(IG(Y|L))-(H(Y)+H(L))]; where Si is Described similarity; ^X and Y are two different characteristics of the sample data; X` is the representation of X in different dimensions, Y` is the representation of Y in different dimensions; L is the preset category; [ X` <sup>T</sup> ×Y+X <sup>T</sup> ×Y`+X` <sup>T</sup> ×Y`] represents the discriminant correlation between the representation of X in different dimensions and the representation of Y in different dimensions; [2×(IG(X|L))- (H(X)+H(L))]+[2×(IG(Y|L))-(H(Y)+H(L))] represents the correlation between X, Y and the default category Spend.

In addition, the correlation between the feature and the preset category is calculated through the following formula: Si=[X ^T ×Y+X` <sup>T</sup> ×Y+X <sup>T</sup> ×Y`+X` <sup>T</sup> ×Y`]+λ×[2× (IG(X|L))-(H(X)+H(L))]+[2×(IG(Y|L))-(H(Y)+H(L))]; where, Si is the similarity; X and Y are two different features of the sample data; X` is the representation of X in different dimensions, Y` is the representation of Y ⁱⁿ different dimensions; L is the preset category; [ Y+X` <sup>T</sup> ×Y+X <sup>T</sup> ×Y`+X` <sup>T</sup> ×Y`] represents the discriminant correlation between the representation of X in different dimensions and the representation of Y in different dimensions; [2×(IG(X|L) )-(H(X)+H(L))]+[2×(IG(Y|L))-(H(Y)+H(L))] represents the difference between X and Y respectively and the default category The degree of correlation; λ is the equilibrium constant.

In addition, the initial sample data is image sample data.

Description of the drawings

One or more embodiments are exemplified by the pictures in the corresponding drawings. These illustrative illustrations do not constitute limitations to the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings are not intended to be limited to scale.

Figure 1 is a flow chart of a data processing method based on principal component analysis provided according to the first embodiment of the present invention;

Figure 2 is a flow chart of a data processing method based on principal component analysis provided according to a second embodiment of the present invention;

Figure 3 is a schematic structural diagram of a data processing device based on principal component analysis provided according to the third embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, each implementation mode of the present invention will be described in detail below with reference to the accompanying drawings. However, those of ordinary skill in the art will understand that in each embodiment of the present invention, many technical details are provided to enable readers to better understand the present invention. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solution claimed by the present invention can also be implemented.

The first embodiment of the present invention relates to a data processing method based on principal component analysis. The specific process is shown in Figure 1, including:

S101: Perform dimensionality reduction processing on the initial sample data to obtain sample data with preset dimensions.

Specifically, before performing dimensionality reduction on the initial sample data, the initial sample data to be processed will be obtained in advance. The dimension reduction processing of the initial sample data described in this embodiment specifically includes: converting the initial sample data into a data matrix; calculating the covariance matrix of the data matrix, and characterizing the covariance matrix. Decompose to obtain the eigenvalues of the covariance matrix and the eigenvectors corresponding to the eigenvalues; obtain the projection matrix according to the eigenvalues and the eigenvectors, and reduce the dimension of the initial sample data to the The dimension corresponding to the projection matrix. Obtaining a projection matrix based on the eigenvalues and the eigenvector specifically includes: arranging the eigenvectors in rows from top to bottom into a matrix, wherein the larger the eigenvalue corresponding to the eigenvector, the greater the eigenvalue. The vector is located in the forward row of the matrix; the first k rows are taken to form the projection matrix, where k is an integer greater than 1. For example, there are 8 rows in a matrix of eigenvectors arranged from top to bottom, indicating that there are 8 eigenvectors in total. If the eigenvalue corresponding to a certain eigenvector is the largest among the eigenvalues corresponding to these 8 eigenvectors, then the feature The vector is in the first row of the matrix, and so on.

It is worth mentioning that in order to reduce the error of the data matrix and avoid the impact of noise data in the data matrix on the final analysis results, before calculating the covariance matrix of the data matrix, it also includes: Each row is subjected to zero-meaning processing; the calculation of the covariance matrix of the data matrix specifically includes: calculating the covariance matrix of the data matrix after zero-meaning processing.

It can be understood that this implementation method uses the PCA method to reduce the dimension of the initial sample data. It should be noted that in the existing technology, multidimensional scaling analysis (MDS) is usually used to reduce the dimensionality of data samples. MDS is a dimensionality reduction method that mines hidden structural information in the data by analyzing similar data. Usually, the similarity measure is represented by the Euclidean distance measure. Therefore, the purpose of the MDS algorithm is to map the data samples to a low-dimensional space while retaining the distance between data samples as much as possible, thereby reducing the dimension of the samples. MDS is a classic method that theoretically maintains Euclidean distance. MDS was first mainly used for data visualization. Since the center of the low-dimensional representation obtained by MDS is at the origin, it can be said to maintain the inner product. That is, distances in high-dimensional space are approximated by inner products in low-dimensional space. In the classic MDS method, the distance in high-dimensional space generally uses Euclidean distance. Multidimensional scaling analysis (MDS) and principal component analysis (PCA) are both data dimensionality reduction techniques, but they are different in the direction of optimization. The input of PCA is the original vector of n-dimensional space, and the data is projected to the projection direction with the largest covariance, so the characteristics of the data are basically preserved during the dimensionality reduction process. The input of MDS is the pairwise distance between points, and the output of MDS is the projection of the distance-preserved point in two or three dimensions.

In short, PCA minimizes the sample dimension, which can preserve the covariance of the data. MDS minimizes the sample dimension and can save the distance between data points. If the data covariance and the Euclidean distance between high-dimensional data points are consistent, they are the same; if the distance measures are different, then the two methods are different. Obviously, MDS has its limitations, and PCA can make up for it as an alternative method, with a wider range of applications. Moreover, the input of PCA is the original vector of n-dimensional space, so compared with MDS, it simplifies the algorithm and reduces the complexity of the algorithm in terms of input. The most important thing is that the PCA method is widely used in dimensionality reduction and preprocessing of data in terms of software defects, and the effect is better than MDS.

In order to facilitate understanding, the algorithm process of the PCA method is explained in detail below:

Suppose there are N image training samples in total, which are simply expressed as x _k ∈X (k=1,...,N) _. , the column vector dimension obtained by expanding the image matrix of each data is n. The average sample of all image training samples is expressed by:

The average sample of the i-th (i=1,...,c) class of training samples is expressed as follows:

The specific process of the principal component analysis method is: first, read the database, and expand each read-in two-dimensional data image data into a one-dimensional vector. Each type of image sample can select a certain number of images based on the generated random matrix. The images constitute the training sample set, and the rest constitute the test sample set. The next step is to calculate the generating matrix of the KL orthogonal transformation. The generating matrix can be represented by the overall divergence matrix S _T of the training sample, or by the inter-class divergence matrix S _B of the training sample. The divergence matrix is represented by the training set Generated, here represented by the overall divergence matrix S _T , defined as:

The generating matrix Σ can be expressed as: Σ=S _T S _T ^T

Then perform eigenvalue decomposition, calculate the eigenvalues and eigenvectors of the generated matrix Σ, sort the eigenvalues from large to small, and retain the first m largest eigenvalues and the eigenvectors corresponding to these m eigenvalues. Thus, the projection matrix from high-dimensional space to low-dimensional space is obtained, and the characteristic subspace is constructed. That is to say, the PCA method using KL transformation aims to find a set of optimal projection vectors that satisfy the criterion function:

The next step is to find the best projection vector, that is, the unit vector w that maximizes the above criterion function. Its physical meaning is: in the direction represented by the projection vector w, the overall degree of dispersion of the feature vector obtained after the image vector is projected is the largest. That is, the distance between each sample of the image data and the average sample of the overall training samples is the largest. Because the optimal projection vector calculated above is the unit eigenvector corresponding to the maximum eigenvalue of the overall divergence matrix S _T. When there are a large number of sample categories, only a single optimal projection direction is not enough to fully represent the characteristics of all image samples. Therefore, it is necessary to find a set of criteria functions that can maximize The best projection vector group w ₁ , w ₂ ,...,w _m that can satisfy the standard orthogonal condition. The optimal projection matrix is represented by the optimal projection vector group, that is, P=[w ₁ , w ₂ ,..., w _m ].

Next, the training samples and test samples are respectively projected into the feature subspace calculated above. After each data image is projected into the above feature subspace, it will correspond to a point in the subspace. Similarly, any point in the feature subspace can also find its corresponding data image. The points obtained by the projection of the data image in these feature subspaces are called "eigenfaces". As the name suggests, the "eigenface" method refers to the method of data recognition through K-L orthogonal transformation.

Finally, all the test image samples transformed into the feature subspace after the above vector projection are compared with the training image samples to determine the category of the data image sample to be identified. This is the classification of the test samples, and it is necessary to select the appropriate Classifier and dissimilarity test formulas.

S102: Obtain multiple features of the sample data and calculate the correlation between each feature and the preset category.

Specifically, this implementation can calculate the correlation between the feature and the preset category through the following formula: Si=[X ^T ×Y+X` <sup>T</sup> ×Y+X <sup>T</sup> ×Y`+X` <sup>T</sup> ×Y`] +[2×(IG(X|L))-(H(X)+H(L))]+[2×(IG(Y|L))-(H(Y)+H(L))] ; Among them, Si is the similarity; X and Y are two different characteristics of the sample data; X` is the representation of X in different dimensions, Y` is the representation of Y in different dimensions; L is the preset category; [X ^T ×Y+X` <sup>T</sup> ×Y+X <sup>T</sup> ×Y`+X` <sup>T</sup> ×Y`] represents the discriminant correlation between the representation of X in different dimensions and the representation of Y in different dimensions; [2×(IG( X|L))-(H(X)+H(L))]+[2×(IG(Y|L))-(H(Y)+H(L))] means that Assume the correlation between categories.

It is worth mentioning that in order to balance the calculation items of the first part and the second part, a balance parameter λ needs to be added. Therefore, this embodiment can also calculate the correlation between the feature and the preset category through the following formula:

Si＝[X ^T ×Y+X` <sup>T</sup> ×Y+X <sup>T</sup> ×Y`+X` <sup>T</sup> ×Y`]+λ×[2×(IG(X|L))-(H(X)+H( L))]+[2×(IG(Y|L))-(H(Y)+H(L))]; where Si is the similarity; X and Y are two different features of the sample data ;X` is the representation of <sup>X</sup> in different dimensions, Y` is the representation of Y ⁱⁿ different ^dimensions ; L is the preset category; [X ^T ×Y`] represents the discriminant correlation between the representation of X in different dimensions and the representation of Y in different dimensions; [2×(IG(X|L))-(H(X)+H(L))]+[2 ×(IG(Y|L))-(H(Y)+H(L))] represents the correlation between X and Y and the preset category; λ is the equilibrium constant.

It can be understood that compared with the information entropy formula in the prior art, the formula in this embodiment expands the calculation method of only sample features in the original method, so that the identification between any two different features between the same sample Feature selection categories have high correlation. The first square bracket represents the discriminant correlation between any two different features of the same sample and their representation in different dimensions, and through the constraint of this calculation, a more intuitive and higher correlation can be obtained better Features, and the original method cannot intuitively calculate features with higher correlation. It only considers the correlation of sample features, but ignores the more important feature correlations between sample features. Through this calculation, we can Make full use of the connection between different features of the same sample, including relevant features and redundant features. Obviously, if the value calculated by the first square bracket is larger, it means that the correlation between the sample features is higher, and it can be obtained Its related features, and vice versa, if the value calculated by the first square bracket is smaller, it means the higher the redundancy between the sample features, and its redundant features can be effectively removed, making the sample features more discriminative. . The calculation of the following two square brackets indicates the similarity between two different features of the same sample and the categorical variable. Similarly, the larger the value, the higher the similarity between the feature and the category, that is, the higher the correlation, and vice versa. The smaller the value, the lower the similarity between the feature and the category, that is, the lower the correlation.

S103: Remove features whose correlation degree is less than the preset correlation degree among multiple features, and use the remaining features as discriminating features of the sample data.

Specifically, the size of the preset correlation degree can be set according to actual needs, and this embodiment does not specifically limit this. This implementation is based on principal component analysis sample data. The basic idea of principal component analysis is to extract the main features of the high-dimensional data space and maintain most of the information of the original high-dimensional data, so that the high-dimensional data can be stored in a lower-dimensional space. are processed in feature space. K-L transform is the basis of principal component analysis. It is an optimal orthogonal transformation based on the target statistical characteristics. Its purpose is to find a linear projection transformation through which the new feature components are orthogonal or irrelevant, and in order to make the energy of the data more concentrated. , it is required that the characteristic component after projection reconstruction and the original input sample have the smallest error in the sense of minimum mean square. This results in a low-dimensional approximate representation of the original sample, which can better compress the original data. Using K-L transformation for data recognition, the classic eigenface method (Eigenfaces) was proposed, which formed the basis of the subspace learning method. In short, a set of eigenface images is obtained from the input data training images through principal component analysis, and then any data image is given, so that each data image can be linearly represented by this set of eigenface images, that is Weighted linear combinations of eigenface images obtained by computing principal component analysis.

The essence of principal component analysis is to calculate the covariance matrix and diagonalize it. It can be assumed that all data images are in a linear low-dimensional space, and all data images are linearly separable in this low-dimensional space. Then the principal component analysis method is used for data feature identification. The specific method is to Perform K-L transformation, transform the high-dimensional image input space to obtain a new set of orthogonal bases, filter the transformed orthogonal bases according to certain methods according to certain conditions, eliminate some redundant vectors, and retain those feature identifications A powerful vector can be used to generate a low-dimensional data subspace, which is the characteristic face subspace of the data. The key to using the principal component analysis method to reduce the spatial dimension of the input data is to find the projection method that best represents the original data to "reduce noise" and eliminate "redundant" dimensions, so that while reducing the dimension, the original input data can be guaranteed The most important features are not lost. In the covariance matrix, you only need to select those dimensions with relatively large energy (eigenvalues), and discard the others with relatively low energy. This way, you can retain the important feature information in the input image data, and discard the ones that are not beneficial to data recognition. other parts.

In order to facilitate understanding, the following is a specific example of how to process sample data in this implementation:

Input: training sample set: X=[X1, X2,...,Xc], where Xi=(F1, F2,..., Fm, L), k<m, i=1...m.

Dimensions of PCA data dimensionality reduction: k

Correlation threshold (preset correlation): β

1) Form the original data into a data matrix by columns, and expand each read-in two-dimensional data image data into a one-dimensional vector.

2) Zero-mean each row of the data matrix (representing an attribute field), that is, subtract the mean of this row.

3) Find the covariance matrix.

4) Perform eigenvalue decomposition to find the eigenvalues and corresponding eigenvectors of the covariance matrix.

5) Arrange the eigenvectors into a matrix from top to bottom in rows according to the size of the corresponding eigenvalues, and take the first k rows to form the sample projection matrix.

6) Reduce the dimensionality of the data to the dimension corresponding to the projection matrix, that is, k. X' = PX is the data after dimensionality reduction to k dimensions. The obtained sample set after dimensionality reduction is expressed as X'=[X'1, X'2,...,X'c], where =1...m.

7) Let i=1 to k, j=1 to k (i≠j) loop, and calculate Si=ISU(Fi, Fi’, Fj, Fj’, L).

8) Sort Si from large to small.

9) Use the first g features in the sequence as features of the new sample to obtain the sample set X”=[X”1, X”2,…,X”c], where , Fg, L), g<k, i=1...m.

10) Perform correlation analysis on each pair of features from back to front, remove specified features greater than β, and obtain the final sample Y.

Output: sample set Y.

Compared with the existing technology, the embodiment of the present invention performs dimensionality reduction processing on the initial sample data to obtain sample data with preset dimensions, so as to facilitate the calculation of subsequent steps, reduce the calculation amount of subsequent steps, and thereby improve The efficiency of the data processing method; by obtaining multiple features of the sample data and calculating the correlation between each feature and the preset category. Since the preset category is one of the multiple categories of the sample data, through this In this way, it is possible to know which features among the multiple features of the sample data are redundant features based on the similarity; by removing the features whose correlation degree is less than the preset correlation among the multiple features, the remaining features are used as the redundant features of the sample data. The discriminant feature can obtain sample data with high discriminability, which makes the budget of the training model using the sample data faster, thereby improving the prediction efficiency.

The second embodiment of the present invention relates to a data processing method based on principal component analysis. The second embodiment is further improved on the basis of the first embodiment. The specific improvements are: in the second embodiment , after removing the features whose correlation degree is less than the preset correlation degree among the plurality of features, it also includes: sorting the remaining features according to the order of the correlation degree from high to low; dividing the sorted remaining features into is N feature segments, where each feature segment includes M features, and N and M are both integers greater than 1; determine whether there are M feature segments with all features greater than the preset threshold, and when determining that they exist, remove The feature with the smallest similarity among the feature segments. In this way, redundant features in the sample data can be further reduced, further improving the prediction efficiency.

The specific process of this implementation is shown in Figure 2, including:

S201: Perform dimensionality reduction processing on the initial sample data to obtain sample data with preset dimensions.

S202: Obtain multiple features of the sample data, and calculate the correlation between each feature and the preset category.

S203: Remove features whose correlation degree is less than the preset correlation degree among multiple features.

S204: Sort the multiple features after removing the features whose correlation degree is less than the preset correlation degree in order from high to low correlation, and divide the remaining features after sorting into N feature segments.

S205: Determine whether there are M feature segments whose features are all greater than the preset threshold. When it is determined that there are M feature segments, remove the feature with the smallest similarity among the feature segments.

Regarding the above steps S204 to S205, specifically, the threshold correlation method is used to remove redundant features of the sample data. The threshold correlation method uses the correlation between features to identify redundant features. In actual software measurement, there is a nonlinear relationship, so ISU is still chosen here to calculate the correlation between a pair of features. The threshold correlation method uses pre- Let β (i.e., the preset threshold) be used as the critical value of correlation. After removing the features whose correlation degree is less than the preset correlation degree among multiple features, correlation analysis is performed on the remaining features from back to front. All features that are greater than the critical value are For a pair of features, the lower feature is removed from the sample set, and so on. The reason why the correlation analysis is performed from back to front is because the discriminability of features that are smaller than the preset correlation among multiple features becomes more and more discriminating from back to front, so the correlation analysis is performed from back to front. When encountering two features with a correlation greater than the β value, the less discriminating features can be removed first, thereby retaining the more discriminating features.

S206: Use the remaining features as discriminating features of the sample data.

Steps S201 to S203 and S206 in this embodiment are similar to steps S101 to S103 in the first embodiment. In order to avoid duplication, they will not be described again here.

Compared with the existing technology, the embodiment of the present invention performs dimensionality reduction processing on the initial sample data to obtain sample data with preset dimensions, so as to facilitate the calculation of subsequent steps, reduce the calculation amount of subsequent steps, and thereby improve The efficiency of the data processing method; by obtaining multiple features of the sample data and calculating the correlation between each feature and the preset category. Since the preset category is one of the multiple categories of the sample data, through this In this way, it is possible to know which features among the multiple features of the sample data are redundant features based on the similarity; by removing the features whose correlation degree is less than the preset correlation among the multiple features, the remaining features are used as the redundant features of the sample data. The discriminant feature can obtain sample data with high discriminability, which makes the budget of the training model using the sample data faster, thereby improving the prediction efficiency.

The third embodiment of the present invention relates to a data processing device based on principal component analysis, as shown in Figure 3, including:

at least one processor 301; and,

A memory 302 communicatively connected to at least one processor 301; wherein,

The memory 302 stores instructions that can be executed by at least one processor 301, and the instructions are executed by at least one processor 301, so that at least one processor 301 can execute the above-mentioned data processing method based on principal component analysis.

The memory 302 and the processor 301 are connected using a bus. The bus may include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors 301 and the memory 302 together. The bus may also connect various other circuits together such as peripherals, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be described further herein. The bus interface provides the interface between the bus and the transceiver. A transceiver may be one element or may be multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. The data processed by the processor 301 is transmitted on the wireless medium through the antenna. Furthermore, the antenna also receives the data and transmits the data to the processor 301.

Processor 301 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory 302 may be used to store data used by the processor 301 when performing operations.

The fourth embodiment of the present invention relates to a computer-readable storage medium storing a computer program. The above method embodiments are implemented when the computer program is executed by the processor.

That is, those skilled in the art can understand that all or part of the steps in the methods of the above embodiments can be completed by instructing relevant hardware through a program. The program is stored in a storage medium and includes several instructions to cause a device ( It may be a microcontroller, a chip, etc.) or a processor (processor) that executes all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program code.

Those of ordinary skill in the art can understand that the above-mentioned embodiments are specific examples for realizing the present invention, and in practical applications, various changes can be made in form and details without departing from the spirit and spirit of the present invention. scope.

Claims (8)

1. A data processing method based on principal component analysis, comprising:

performing dimension reduction processing on initial sample data to obtain sample data with preset dimensions, wherein the initial sample data are image sample data, and the sample data are characteristic face images;

acquiring a plurality of characteristics of the sample data, and calculating the correlation degree of each characteristic and a preset category, wherein the preset category is one category of a plurality of categories of the sample data;

removing the features with the correlation degree smaller than the preset correlation degree from the plurality of features, and taking the remaining features as identification features of the sample data;

the step of performing dimension reduction processing on the initial sample data to obtain sample data with preset dimensions comprises the following steps:

performing data feature recognition processing on the image sample data based on a principal component analysis method to acquire each feature face image, wherein each feature face image is used for linearly representing any one of the image sample data;

after removing the feature with the correlation degree smaller than the preset correlation degree, the method further includes:

ranking the remaining features in the order of the correlation from high to low;

dividing the sequenced residual features into N feature segments, wherein each feature segment comprises M features, and N, M is an integer greater than 1;

judging whether there are feature segments with the correlation degree of M features being greater than a preset threshold value, and removing the feature with the minimum correlation degree in the feature segments when judging that there are the feature segments.

2. The method for processing data based on principal component analysis according to claim 1, wherein the performing the dimension reduction processing on the initial sample data specifically comprises:

converting the initial sample data into a data matrix;

calculating a covariance matrix of the data matrix, and carrying out feature decomposition on the covariance matrix to obtain a feature value of the covariance matrix and a feature vector corresponding to the feature value;

and obtaining a projection matrix according to the characteristic values and the characteristic vectors, and reducing the dimension of the initial sample data to the dimension corresponding to the projection matrix.

3. The method for processing data based on principal component analysis according to claim 2, wherein the obtaining a projection matrix according to the eigenvalues and the eigenvectors specifically includes:

arranging the feature vectors into a matrix from top to bottom according to rows, wherein the larger the feature value corresponding to the feature vector is, the more forward the feature vector is positioned in the matrix;

and taking the first k rows to form the projection matrix, wherein k is an integer greater than 1.

4. A data processing method based on principal component analysis according to claim 2 or 3, further comprising, before calculating the covariance matrix of the data matrix:

zero-equalizing each row of the data matrix;

the calculating the covariance matrix of the data matrix specifically comprises the following steps:

and calculating a covariance matrix of the data matrix after zero-mean processing.

5. The principal component analysis-based data processing method according to claim 1, wherein the correlation of features with the preset categories is calculated by the following formula:

Si＝[X ^T ×Y+X` <sup>T</sup> ×Y+X <sup>T</sup> ×Y`+X` <sup>T</sup> ×Y`]+[2×(IG(X|L))-(H(X)+H(L))]+[2×

(IG(Y|L))-(H(Y)+H(L))]；

wherein Si is the degree of correlation; x, Y are two different features of the sample data; x 'is the representation of X in different dimensions, Y' is the representation of Y in different dimensions; l is the preset category; [ X ] ^T ×Y+X` <sup>T</sup> ×Y+X <sup>T</sup> ×Y`+X` <sup>T</sup> ×Y`]An authentication correlation of the representation of X in different dimensions with the representation of Y in different dimensions; [2× (IG (X|L)) - (H (X) +H (L))]+[2×(IG(Y|L))-(H(Y)+H(L))]The correlation between X, Y and the preset categories respectively is represented.

6. The principal component analysis-based data processing method according to claim 1, wherein the correlation of features with the preset categories is calculated by the following formula:

Si＝[X ^T ×Y+X` <sup>T</sup> ×Y+X <sup>T</sup> ×Y`+X` <sup>T</sup> ×Y`]+λ×[2×(IG(X|L))-(H(X)+H(L))]+[2×

(IG(Y|L))-(H(Y)+H(L))]；

wherein Si is the degree of correlation; x, Y are two different features of the sample data; x 'is the representation of X in different dimensions, Y' is the representation of Y in different dimensions; l is the preset category; [ X ] ^T ×Y+X` <sup>T</sup> ×Y+X <sup>T</sup> ×Y`+X` <sup>T</sup> ×Y`]An authentication correlation of the representation of X in different dimensions with the representation of Y in different dimensions; [2× (IG (X|L)) - (H (X) +H (L))]+[2×(IG(Y|L))-(H(Y)+H(L))]Representing the correlation degree between X, Y and the preset categories respectively; lambda is the equilibrium constant.

7. A data processing apparatus based on principal component analysis, comprising: at least one processor; the method comprises the steps of,

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the principal component analysis-based data processing method of any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the principal component analysis-based data processing method according to any one of claims 1 to 6.