WO2022178977A1

WO2022178977A1 - Unsupervised data dimensionality reduction method based on adaptive nearest neighbor graph embedding

Info

Publication number: WO2022178977A1
Application number: PCT/CN2021/090827
Authority: WO
Inventors: 王靖宇; 谢方园; 聂飞平; 李学龙
Original assignee: 西北工业大学
Priority date: 2021-02-26
Filing date: 2021-04-29
Publication date: 2022-09-01
Also published as: CN112836672A; CN112836672B

Abstract

The present invention relates to the fields of image recognition and classification and pattern recognition, and relates to an unsupervised data dimensionality reduction method based on adaptive nearest neighbor graph embedding. The method comprises: preprocessing data; constructing a nearest neighbor graph and initializing same; and optimizing an objective function by means of alternating iteration. The present invention further provides a face recognition method based on the data dimensionality reduction method, comprising: performing dimensionality reduction on a face image to obtain a projection matrix and low-dimensional data, and clustering the low-dimensional data by using an unsupervised clustering algorithm to obtain clustering centers of categories; and taking, according to the Euclidean distances between an image to be classified and the clustering centers, a clustering center having the minimum Euclidean distance, the category of the clustering center being the category of a new face image. Face recognition is performed in a low-dimensional space, so that the amount of data storage can be reduced, the amount of data calculation can be reduced, the calculation efficiency can be improved, and finally, the real-time performance and recognition precision of a face recognition technology can be improved.

Description

A Dimensionality Reduction Method for Unsupervised Data Based on Adaptive Neighbor Graph Embedding

technical field

The invention relates to an unsupervised data dimension reduction method based on self-adaptive nearest neighbor graph embedding, and belongs to the fields of image recognition, classification and pattern recognition.

Background technique

Data dimensionality reduction technology is an important research topic in the field of image classification and pattern recognition. Under the background of big data, the amount of raw data directly obtained in practical application scenarios is huge. Handle the requirements of the hardware platform. Data dimensionality reduction is to perform feature extraction and dimension reduction processing on the original high-dimensional data. While ensuring that the dimensionality-reduced data still retains most of the information contained in the original data, the dimensionality of the data is reduced as much as possible to improve data storage and processing. efficiency, reducing the requirements for hardware and subsequent data processing algorithms. Because data dimensionality reduction can reduce the data dimension and required storage space, save model training and calculation time, and improve the accuracy of subsequent applied algorithms, data dimensionality reduction technology has been widely used in pattern recognition, computer vision, hyperspectral image processing and other fields. After the data is processed by dimensionality reduction, the amount of data is greatly reduced, which can improve the speed and accuracy of subsequent data classification.

Recently, the unsupervised dimensionality reduction method based on graph embedding has attracted attention. The dimensionality reduction method based on graph embedding regards sample points as graph nodes, and the weight value between nodes represents the distance between nodes. Perform dimensionality reduction. However, the traditional dimensionality reduction method based on graph embedding needs to construct the nearest neighbor graph in advance. The quality of the nearest neighbor graph construction is directly related to the effect after dimensionality reduction.

Shanhua Zhan et al. ("Robust Sparse Local Preserving Projection for Adaptive Graph Embedding", Computer Engineering and Design, 2020, 41(08): 2296-2301.) proposed a robust sparse localization for adaptive embedding A projection-preserving dimensionality reduction method that integrates graph learning and dimensionality reduction learning into a joint learning framework. Although the proposed model takes into account comprehensive factors such as sparsity and robustness, there are too many parameters in the model and the model is redundant, which cannot balance the relationship between parameters and performance. The selection of parameters has an important impact on the performance of the model, resulting in The model is more difficult in practical application.

At present, in the field of image recognition, due to the high dimensionality of the data, it is difficult to recognize or classify the process, resulting in a slow recognition or classification. Dimensionality reduction is a technique that can convert multi-dimensional indicators into a small number of comprehensive indicators, and is generally used as a preprocessing step. At present, for face recognition systems, the commonly used dimensionality reduction technology is principal component analysis technology, namely PCA. Through PCA technology, useful information can be extracted and feature dimensionality reduction can be performed to obtain a low-dimensional feature space. In the low-dimensional feature space, the speed and accuracy of classification and recognition are effectively improved, but because the PCA technology only maximizes the variance of the data after dimensionality reduction, that is, only the overall information of the data is considered, and the local structure between the data is not considered. , so the classification accuracy is limited.

SUMMARY OF THE INVENTION

technical problem to be solved

Aiming at the defect that the current neighbor graph construction method and the data dimension reduction algorithm are separated from each other, resulting in insignificant dimensionality reduction effect, resulting in low efficiency and low accuracy of face recognition later, the present invention proposes a method based on adaptive neighbor graph embedding. Supervised data dimensionality reduction method and face recognition method based on data dimensionality reduction.

Technical solutions

An unsupervised data dimensionality reduction method based on adaptive nearest neighbor graph embedding, characterized in that the steps are as follows:

Step 1: Data Preprocessing

Principal component analysis PCA on raw data

Perform preprocessing to obtain a data matrix X∈R ^d×n , where n is the number of sample points, d' is the dimension of the sample point, and d is the dimension of the sample point after PCA processing;

Step 2: Build the nearest neighbor graph and initialize it

According to the data matrix X∈R ^d×n , construct the nearest neighbor graph G=(X,S), where G represents the constructed nearest neighbor graph, X represents the node set in the nearest neighbor graph, and S represents the distance relationship between nodes. The elements S _ij represent the distance between the ith node and the jth node; the weight matrix S is obtained by minimizing the following problem:

Among them, the power exponent factor r is used to adjust the size of the weight, x _i ∈R ^d×1 , i=1,2,...,n is the i-th column vector of the matrix X, that is, the i-th sample point coordinate;

Step 3: Alternately iteratively optimize the objective function

set the projection matrix

d ₁ is the dimension of the low-dimensional space, and the projection matrix W maps the data from the d-dimensional space to the d ₁ <<d-dimensional space; in order to ensure that the data after projection is statistically irrelevant, add the constraint W ^T S _t W=I ,

is the global divergence matrix,

is the data matrix,

is the identity matrix,

is a matrix whose elements are all 1; the objective function is as follows:

Among them, the denoising matrix

represents the approximate matrix for noise removal in the subspace,

is the transpose of the ith row vector of F, and λ is the regularization parameter;

The first term in the objective function can be simplified as follows:

in

is a Laplacian matrix with dimension n×n;

is the degree matrix, which is a diagonal matrix,

The dimension is n×n; S ^r is the similarity matrix, each element of which is the power of r of each element in the weight matrix S, and the dimension is n×n; then the objective function can be simplified to the following formula:

The above objective function is solved by the alternate iteration method, and the projection matrix W is obtained, and the data matrix after data dimension reduction is Y=W ^T X.

Preferably: the data in step 1 is a face image or a hyperspectral image.

Preferably: in step 2, r is taken as 1.1.

A face recognition method based on a data dimension reduction method, characterized in that the dimension reduction method is used to reduce the dimension of a face image to obtain a projection matrix and low-dimensional data, and an unsupervised clustering algorithm is used to cluster the low-dimensional data. After the camera collects a new face image, it uses the obtained projection matrix to reduce the dimension of the new image to obtain the low-dimensional projection coordinates, and calculate the low-dimensional projection coordinates and each cluster. The Euclidean distance between the cluster centers and the cluster center with the smallest Euclidean distance is taken, then the category to which the cluster center belongs is the category of the new face image.

beneficial effect

An unsupervised data dimensionality reduction method for self-adaptive nearest neighbor graph embedding proposed by the present invention has the following beneficial effects:

(1) A new method for constructing the nearest neighbor graph is proposed through the invention step 1, which avoids the sensitivity to noise when the traditional k-nearest neighbor graph is constructed. This method of constructing the nearest neighbor graph can not only be used in the data dimensionality reduction algorithm, but also can be extended to other algorithms that need to construct the nearest neighbor graph, such as clustering.

(2) Through the invention step 3, the learning of the neighbor graph and the learning of the projection matrix in the data dimensionality reduction are combined into a framework, and the construction of the neighbor graph is continuously updated in the subspace, and finally a reasonable neighbor graph can be obtained. The construction method It can adaptively find reasonable neighbor graphs and is suitable for different types of data sets.

(3) The present invention proposes a face recognition method based on graph embedding dimension reduction. In the dimensionality reduction processing step, the optimal neighbor graph is constructed through the continuous update of the neighbor graph, so as to better maintain the local structure of the data, and at the same time, the overall information of the data is taken into account to obtain a low-dimensional image containing more effective features. data. Performing face recognition in low-dimensional space can reduce the amount of data storage, reduce the amount of data calculation, improve computing efficiency, and ultimately improve the real-time performance and recognition accuracy of face recognition technology.

Description of drawings

Figure 1 Flowchart of dimensionality reduction method

Fig. 2 Flow chart of face recognition method based on dimensionality reduction method

Detailed ways

The present invention will now be further described in conjunction with the embodiments and accompanying drawings:

The present invention is based on an unsupervised data dimensionality reduction method embedded in an adaptive nearest neighbor graph, and its basic flowchart is shown in Figure 1, and its specific steps are as follows:

Step 1: Data preprocessing. The original data matrix is X'∈R ^d'×n , where n is the number of sample points, and d' is the dimension of the sample points. Since there is inevitably a null space in the original space, the principal component analysis (Principal Component Analysis, PCA) to preprocess the raw data. Principal component analysis is to decompose the eigenvalues of the covariance matrix of the data. The larger the eigenvalue, the more useful information it contains when the corresponding eigenvector is selected as the projection matrix. If the eigenvectors corresponding to the first d largest eigenvalues are selected, it satisfies

The value of 95%-99%, that is, the energy of maintaining 95%-99% of the original data, making the subsequent algorithm faster. The obtained data matrix is X∈R ^d×n , where d is the dimension of the sample point after PCA processing.

Step 2: Build the nearest neighbor graph and initialize it. According to the data matrix X∈R ^d×n , construct the nearest neighbor graph G=(X,S), where G represents the constructed nearest neighbor graph, X represents the node set in the nearest neighbor graph, and S represents the distance relationship between nodes. The elements S _ij represent the distance between the ith node and the jth node. The weight matrix S is created by minimizing the following problem:

Among them, the power exponent factor r is used to adjust the size of the weight, the empirical value is 1.1, x _i ∈ R ^d×1 , i=1,2,...,n is the ith column vector of the matrix X, that is, the ith column vector The coordinates of the i sample points. This formula shows that the weight matrix is measured by calculating the distance between the sample points in the high-dimensional space. The smaller the distance between the sample points, the larger the element value in the corresponding weight matrix, that is, the two sample points are neighbors The greater the possibility of , and vice versa, the smaller the element value in the weight matrix.

Step 3: Alternately iteratively optimize the objective function. set the projection matrix

d ₁ is the dimension of the low-dimensional space, and the projection matrix W maps the data from the d-dimensional space to the d ₁ <<d-dimensional space. In order to ensure that the data after projection are not correlated in a statistical sense, add constraints W ^T S _t W=I, S _t ∈ R ^n×n is the global divergence matrix,

X∈R ^d×n is the data matrix, I∈R ^n×n is the identity matrix, and 1∈R ^n×1 is the matrix whose elements are all 1. The objective function is as follows:

Among them, the denoising matrix

represents the approximate matrix for noise removal in the subspace,

is the transpose of the ith row vector of F, and λ is the regularization parameter, and its value is generally larger.

The first term in the objective function can be simplified as follows,

in

is a Laplacian matrix with dimension n×n;

is the degree matrix, which is a diagonal matrix,

The dimension is n×n; S ^r is the similarity matrix, each element of which is the power of r of each element in the weight matrix S, and the dimension is n×n. Then the objective function can be simplified to the following formula:

The objective function is solved by alternate iteration method. Fix S, solve F and W, then fix F and W, solve S, and use the obtained S as the initial value S ₀ to iterate again. The solution steps are as follows:

Step 3.1: Fix S, solve F and W.

Fixed S, because the objective function has no constraints on F, so the partial derivative of the objective function with respect to F is 0, and the optimization function at this time is

Taking the partial derivative of the above formula with respect to F, and setting the equation to 0, the following formula can be obtained.

Available

F=PX ^T W (7)

in

is a positive definite real symmetric matrix. It can be seen here that when λ is larger,

Close to 0, then F=X ^T W is the data matrix in the subspace to remove noise. Substituting F into the objective function, the following formula can be obtained, and the following formula is used to solve W.

Among them, M=X(IP)X ^T , which is a positive definite real symmetric matrix, and the objective function is the following formula

Using the Lagrange multiplier method to solve the above formula, we can obtain W as a matrix composed of the eigenvectors corresponding to the first d' smallest eigenvalues of (S _t ) ^-1 M. After the optimal value of W is solved, Substitute into (7) to obtain the optimal value of F.

Step 3.2: Fix W, find S from F.

After passing step 3.1, the optimal W and F are obtained. After fixing these two parameters, the objective function at this time is

Observe the above formula and find that it is independent for any sample i, i=1,2,...,n. For each i, the objective function can be written as follows.

Among them, 1∈Rn ^×1 is a vector whose elements are all 1.

Using the Lagrangian method to solve the above equation,

in,

η is the Lagrange multiplier, and the optimal solution of s _ij (i≠j) can be obtained as

It can be seen from the above formula that η should take a positive value. According to the KKT condition, β _i ≥ 0, β _i s _i =0. In the weight matrix S, s _ii =0 is defined. Therefore, when i≠j, β _ij =0, and s _ij can be calculated by (13) at this time; when i = j, s _ij =0.

In addition, if s _i 1=1, then there is the following formula

desirable

After η is determined, observing equation (15), it can be obtained that when the distance between the two sample points is small, the value of s _ij is larger, otherwise, the smaller it is, which is consistent with the above basic assumption.

So far, S is updated, and the next iterative operation is performed again until the algorithm converges. After the solution is completed, the projection matrix W can be obtained, and the data matrix after data dimension reduction is Y=W ^T X.

The specific embodiments of the present invention are described below with reference to FIG. 3 for an example of an actual face recognition method, but the technical content of the present invention is not limited to the described scope.

The present invention proposes a face recognition method based on dimensionality reduction data, comprising the following steps:

Step 1: Build a face database, collect face images for recognition, and perform data preprocessing. Assuming that the number of face images is _n and the size is 32×32, each image can be elongated into a vector with a dimension of 1024 according to the gray value of the face image, and the original data is preprocessed by PCA to retain the original The energy of the data is 95%, and the dimension is 273, then the data matrix X∈R ^273×n .

Step 2: Build the nearest neighbor graph and initialize it. According to the data matrix X∈R ^d×n , construct the nearest neighbor graph G=(X,S), where G represents the constructed nearest neighbor graph, X represents the node set in the nearest neighbor graph, and S represents the distance relationship between nodes. The elements S _ij represent the distance between the ith node and the jth node. The initial weight matrix S is obtained by minimizing the following problem:

Step 3: Alternately iteratively optimize the objective function. Assuming that the dimension of the low-dimensional space is 30, the projection matrix W∈R ^273×30 . In order to ensure that the data after projection are uncorrelated in a statistical sense, add constraints W ^T S _t W=I, S _t ∈ R ^n×n is the global divergence matrix, the expression is

It contains the overall information of the data. X∈R ^273×n is the data matrix, I∈R ^n×n is the identity matrix, and 1∈R ^n×1 is the matrix whose elements are all 1. The objective function is as follows:

Among them, the denoising matrix F∈R ^n×30 represents the approximate matrix for denoising noise in the subspace, f _i ∈ R ^30×1 is the transpose of the ith row vector of F, λ is the regularization parameter, and its value is generally larger.

The first term in the objective function can be simplified as follows,

in

is a Laplacian matrix with dimension n×n;

is the degree matrix, which is a diagonal matrix,

The solution method of the objective function is the alternate iteration method. In step (2), S ₀ has been obtained, then fix S = S ₀ , solve F and W, then fix F and W, solve S, and take the obtained S as the initial The value _S0 is iterated again. The solution steps are as follows:

Step 3.1: Fix S, solve for F and W.

Fixed S, because the objective function has no constraint on F, the partial derivative of the objective function with respect to F is 0, and the optimization function at this time is

We can use W to express F, we get

F=PX ^T W (7)

in

Close to 0, then F≈X ^T W is the data matrix with noise removed after data dimension reduction. Substituting F into the objective function, the following formula can be obtained, and the following formula is used to solve W.

Using the Lagrange multiplier method to solve the above formula, we can obtain W as a matrix composed of the eigenvectors corresponding to the first 30 smallest eigenvalues of (S _t ) ^-1 M. When the optimal value of W is solved, substitute it into (7), the optimal value of F can be obtained.

Step 3.2: Fix W, find S from F.

After fixing W and F, the objective function at this time is

in,

Using the Lagrangian method to solve the above equation, the Lagrangian function is as follows

in,

In addition, if s _i 1=1, then there is the following formula

desirable

After η is determined, observe equation (15), it can be obtained that when the distance between the two sample points is small, the value of s _ij is larger, otherwise, the smaller the value is, which is consistent with the above basic assumption, and the update of S is completed. , the value of the objective function is calculated at this time. If the absolute value of the difference between the objective function values generated by the two iterations satisfies a certain precision (for example, 10 ^-6 ), the iteration is stopped, and the final W _opt and S _opt are obtained. After the solution is completed, the projection matrix W _opt ∈ R ^273×30 can be obtained, and the data matrix after data dimension reduction is Y=W _opt ^T X∈R ^30×n . The low-dimensional data is clustered by unsupervised clustering algorithm, and the cluster centers of each category are obtained.

Step 4: After the camera collects a new face image, use the obtained projection matrix W _opt to perform dimension reduction processing on the new image, obtain low-dimensional projection coordinates, and calculate the Euclidean relationship between the low-dimensional projection coordinates and each cluster center. The distance and the cluster center with the smallest Euclidean distance are taken, then the category to which the cluster center belongs is the category of the new face image.

The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited to this. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed by the present invention. Modifications or substitutions should be included within the protection scope of the present invention.

Claims

An unsupervised data dimensionality reduction method based on adaptive nearest neighbor graph embedding, characterized in that the steps are as follows:

Step 1: Data Preprocessing

Principal component analysis PCA on raw data
Perform preprocessing to obtain a data matrix X∈R d×n , where n is the number of sample points, d' is the dimension of the sample point, and d is the dimension of the sample point after PCA processing;

Step 2: Build the nearest neighbor graph and initialize it

According to the data matrix X∈R d×n , construct the nearest neighbor graph G=(X,S), where G represents the constructed nearest neighbor graph, X represents the node set in the nearest neighbor graph, and S represents the distance relationship between nodes. The elements S ij represent the distance between the ith node and the jth node; the weight matrix S is obtained by minimizing the following problem:

Among them, the power exponent factor r is used to adjust the size of the weight, x i ∈R d×1 , i=1,2,...,n is the i-th column vector of the matrix X, that is, the i-th sample point coordinate;

Step 3: Alternately iteratively optimize the objective function

set the projection matrix
d 1 is the dimension of the low-dimensional space, and the projection matrix W maps the data from the d-dimensional space to the d 1 <<d-dimensional space; in order to ensure that the data after projection is statistically irrelevant, add the constraint W T S t W=I ,
is the global divergence matrix,
is the data matrix,
is the identity matrix,
is a matrix whose elements are all 1; the objective function is as follows:

Among them, the denoising matrix
represents the approximate matrix for noise removal in the subspace,
is the transpose of the ith row vector of F, and λ is the regularization parameter;

The first term in the objective function can be simplified as follows:

in
is a Laplacian matrix with dimension n×n;
is the degree matrix, which is a diagonal matrix,
The dimension is n×n; S r is the similarity matrix, each element of which is the r power of each element in the weight matrix S, and the dimension is n×n; then the objective function can be simplified to the following formula:

The above objective function is solved by the alternate iteration method, and the projection matrix W is obtained, and the data matrix after data dimension reduction is Y=W T X.
The method for unsupervised data dimensionality reduction based on adaptive nearest neighbor graph embedding according to claim 1, wherein the data in step 1 is a face image or a hyperspectral image.
The unsupervised data dimensionality reduction method based on adaptive nearest neighbor graph embedding according to claim 1 is characterized in that in step 2, r is taken as 1.1.
A face recognition method based on the data dimensionality reduction method according to claim 1, characterized in that the dimensionality reduction method according to claim 1 is used to reduce the dimension of the face image to obtain a projection matrix and low-dimensional data, and the low-dimensional The data is clustered by an unsupervised clustering algorithm to obtain the clustering centers of various categories; when the camera collects a new face image, the new image is dimensionally reduced using the obtained projection matrix to obtain the low-dimensional projection coordinates. Calculate the Euclidean distance between the low-dimensional projection coordinates and each cluster center, and take the cluster center with the smallest Euclidean distance, then the category to which the cluster center belongs is the category of the new face image.