CN112508117A - Self-adaptive multi-view dimension reduction method and device based on graph embedding - Google Patents

Abstract

The adaptive multi-view dimensionality reduction method and device based on graph embedding can fully explore the correlation of data between different views, and do not ignore the relationship between data and data within a single view, and can highlight the importance of some features. The method includes: (1) embedding high-dimensional data into low-dimensional space by means of graph embedding; (2) calculating the distance between different samples from the same view to measure the similarity of different samples; (3) sharing a common view through different views (4) measure the similarity to explore the relationship between different views; (5) obtain the projection matrix of each view, and compare the projection matrix with the original data. Multiply to get the final dimensionality reduction result.

Description

Adaptive multi-view dimensionality reduction method and device based on graph embedding

technical field

The invention relates to the technical fields of data mining, machine learning and pattern recognition, in particular to an adaptive multi-view dimensionality reduction method based on graph embedding, and an adaptive multi-view dimensionality reduction device based on graph embedding.

Background technique

Dimensionality reduction is a commonly used basic method for processing high-dimensional data. Its purpose is to reduce the feature dimension of the data from the original dimension to the specified dimension, so that the data after dimension reduction can still maintain the local structure between the data, and similar data. The more similar the data, the larger the gap between the dissimilar data. In the past few decades, many classical dimensionality reduction methods (such as PCA dimensionality reduction, LLE dimensionality reduction, CCA dimensionality reduction, etc.) have been proposed, and have achieved great success in data mining and image processing.

In today's society, with the wide application of technologies such as cameras and sensors, data usually consists of different information sources, modalities or features. For example, the same news can be reported in different languages; pictures can be described by features such as HOG, GIST and LBP; a person can be photographed by cameras from different angles, and these different features or angles can describe different feature information of the data, all These are called multi-view data.

The main method currently used for machine learning is to reduce the dimensionality of the data represented by a single view. This method is called single-view dimensionality reduction (or traditional dimensionality reduction), because it does not consider relevant information from different views, which is easy to cause The problem of partial generalization, so it cannot truly reflect the full picture of the data. For multi-view data, the dimensionality reduction performance is improved by using the compatible and complementary information of each view to comprehensively reflect the whole data.

With the advent of the era of big data, we have increasingly advanced data collection and storage capabilities, but massive data will lead to the overload of scientific information. Dimensionality reduction for multi-view data has attracted more and more attention from researchers. It reduces multi-view data without any annotations to a very low dimension, enabling existing methods to process data while saving memory usage. space. It plays an important role in information retrieval, biological data analysis and medical diagnosis.

The purpose of multi-view dimensionality reduction is to consider the information of different views, and finally obtain the low-dimensional representation of the multi-view high-dimensional data by fusing the information of each view, so that the original structural relationship can be maintained even when the data dimension is reduced. At present, most of the existing methods to solve the multi-view dimensionality reduction problem directly use the single-view dimensionality reduction method to directly reduce the dimensionality of the multi-view data, without considering the correlation and complementarity between the views. PCA is the most classic dimensionality reduction method. The flexible multi-view collaborative dimensionality reduction proposed by Zhang Changqing et al. can directly process multi-view data and use the Hilbert-Schmidt independence criterion to enhance the correlation between perspectives. It has attracted much attention. This method can directly obtain the projection matrix of each view to reduce the dimension of each view, and can achieve a good effect. However, the current multi-view dimensionality reduction methods have two shortcomings: 1) The correlation of data between different views is not fully explored. 2) Exploring the correlation between data from different views but ignoring the relationship between data and data within a single view. 3) Most dimensionality reduction methods cannot highlight the importance of some features.

SUMMARY OF THE INVENTION

In order to overcome the defects of the prior art, the technical problem to be solved by the present invention is to provide an adaptive multi-view dimensionality reduction method based on graph embedding, which can fully explore the correlation of data between different views without ignoring single view. The relationship between internal data and data can highlight the importance of some features.

The technical scheme of the present invention is: this adaptive multi-view dimensionality reduction method based on graph embedding includes the following steps:

(1) Embed high-dimensional data into low-dimensional space by means of graph embedding;

(2) Measure the similarity of different samples by calculating the distance between different samples with the same view;

(3) Explore the relationship between samples between different views by sharing a same similarity matrix from different views;

(4) Measure similarity, so as to explore the relationship between different views;

(5) The projection matrix of each view is obtained, and the final dimension reduction result is obtained by multiplying the projection matrix with the original data.

The invention firstly embeds high-dimensional data into low-dimensional space by means of graph embedding; secondly, the similarity of different samples is measured by calculating the distance between different samples of the same view; and then the same similarity matrix is shared by different views to fully explore different The relationship between the samples between the views; finally, the similarity is measured to explore the relationship between different views; finally, the projection matrix of each view is obtained, and the final dimension reduction result is obtained by multiplying the projection matrix and the original data, so It can fully explore the correlation of data between different views, and does not ignore the relationship between data and data within a single view, which can highlight the importance of some features.

Also provided is an adaptive multi-view dimensionality reduction device based on graph embedding, the device comprising:

Embedding module, which embeds high-dimensional data into low-dimensional space by means of graph embedding;

A distance calculation module, which measures the similarity of different samples by calculating the distance between different samples with the same view;

Shared module, which explores the relationship between samples between different views by sharing a same similarity matrix from different views;

A measurement module, which measures similarity to explore the relationship between different views;

Iterative module, which obtains the projection matrix of each view, and obtains the final dimension reduction result by multiplying the projection matrix with the original data.

Description of drawings

FIG. 1 shows a flow chart of an adaptive multi-view dimensionality reduction method based on graph embedding according to the present invention.

Detailed ways

As shown in Figure 1, this adaptive multi-view dimensionality reduction method based on graph embedding includes the following steps:

(1) Embed high-dimensional data into low-dimensional space by means of graph embedding;

(2) Measure the similarity of different samples by calculating the distance between different samples with the same view;

(3) Explore the relationship between samples between different views by sharing a same similarity matrix from different views;

(4) Measure similarity, so as to explore the relationship between different views;

(5) The projection matrix of each view is obtained, and the final dimension reduction result is obtained by multiplying the projection matrix with the original data.

The invention firstly embeds high-dimensional data into low-dimensional space by means of graph embedding; secondly, the similarity of different samples is measured by calculating the distance between different samples of the same view; and then the same similarity matrix is shared by different views to fully explore different The relationship between the samples between the views; finally, the similarity is measured to explore the relationship between different views; finally, the projection matrix of each view is obtained, and the final dimension reduction result is obtained by multiplying the projection matrix and the original data, so It can fully explore the correlation of data between different views, and does not ignore the relationship between data and data within a single view, which can highlight the importance of some features.

Preferably, in the step (1),

其中D表示每个样本数据的维度，N表示样本的数量，

是数据X的低维表示矩阵，其中K表示样本降维后的数据维度并且K＜＜，如果原始数据x_i和x_j相似，那么它们的低维表示z_i和z_j也相似，第i个样本与第j个样本之间的相似性用s_ij来表示，那么求解的目标公式(1)表示：Suppose you are given a dataset

where D represents the dimension of each sample data, N represents the number of samples,

is the low-dimensional representation matrix of the data X, where K represents the data dimension after sample dimension reduction and K<<, if the original data x _i and x _j are similar, then their low-dimensional representations z _i and z _j are also similar, the i-th The similarity between the sample and the jth sample is represented by s _ij , then the target formula (1) to be solved is expressed as:

The graph regularization of formula (1) is expressed as:

D是一个对角矩阵，其中

where L represents the normalized graph Laplacian matrix, and

D is a diagonal matrix where

For multi-view data, formula (2) becomes:

where m represents the data of different views, Z ^(m) represents the low-dimensional data of the mth view, L ^(m) represents the normalized graph Laplacian matrix of the mth view, and because the low-dimensional data Z ^(m) is It is obtained by the projection matrix P ^(m) and the original data X ^(m) , so formula (3) is written as:

的目的是避免平凡解。in,

The purpose is to avoid trivial solutions.

Preferably, in the step (3), the formula (4) becomes:

Equation (6) adds a regularization term to the F norm of the S matrix, and Equation (6) becomes:

Preferably, in the step (4), after adding constraints to the original data, the formula (7) becomes:

Equation (8) does not consider the data reconstruction error:

表示把原始数据投影到低维空间，而

表示将投影到低维空间的数据在投影回原始空间，与原始数据计算一个差值，将公式(8)与公式(9)结合得到：in

represents the projection of the original data into a low-dimensional space, and

It means that the data projected to the low-dimensional space is projected back to the original space, and a difference is calculated with the original data, and the formula (8) and the formula (9) are combined to obtain:

Preferably, in the step (5), through iterative update, the projection matrix P ^(m) of each view is finally obtained, and the dimension of the dimension reduced by each view is selected,

和

固定不变，对剩余项构建拉格朗日函数后得：First, the similarity matrix S is constructed from the original data. When the fixed similarity matrix S is unchanged, the formula (10) in

and

Fixed and unchanged, after constructing the Lagrangian function for the remaining terms, we get:

(11)

Through the method of eigenvalue decomposition, find P ^(m) and specify how many dimensions to reduce;

When the projection matrix P ^(m) is fixed, the reconstruction error term does not participate in the update, and formula (10) becomes:

Set the distance between the low-dimensional data in formula (12) and the original data as d _ij :

Formula (12) is written as:

Solving equation (14) becomes:

Thus, get

Those of ordinary skill in the art can understand that all or part of the steps in the method of the above-mentioned embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. During execution, it includes each step of the method in the above embodiment, and the storage medium may be: ROM/RAM, magnetic disk, optical disk, memory card, and the like. Therefore, corresponding to the method of the present invention, the present invention also includes an adaptive multi-view dimensionality reduction device based on graph embedding, which is usually expressed in the form of functional modules corresponding to each step of the method. The device includes:

Embedding module, which embeds high-dimensional data into low-dimensional space by means of graph embedding;

A distance calculation module, which measures the similarity of different samples by calculating the distance between different samples with the same view;

Shared module, which explores the relationship between samples between different views by sharing a same similarity matrix from different views;

A measurement module, which measures similarity to explore the relationship between different views;

Iterative module, which obtains the projection matrix of each view, and obtains the final dimension reduction result by multiplying the projection matrix with the original data.

The technical solutions of the present invention are described in more detail below.

The adaptive multi-view dimensionality reduction method based on graph embedding mainly includes the following steps:

1. Dimensionality reduction based on graph embedding

其中D表示每个样本数据的维度，N表示样本的数量。

是数据X的低维表示矩阵，其中K表示样本降维后的数据维度并且K＜＜N。如果原始数据x_i和x_j相似，那么它们的低维表示z_i和z_j也应该相似。第i个样本与第j个样本之间的相似性用s_ij来表示，那么我们求解的目标可以用一下公式表示：Suppose you are given a dataset

where D represents the dimension of each sample data, and N represents the number of samples.

is a low-dimensional representation matrix of data X, where K represents the data dimension after sample dimension reduction and K<<N. If the original data x _i and x _j are similar, then their low-dimensional representations z _i and z _j should also be similar. The similarity between the i-th sample and the j-th sample is represented by s _ij , then the goal of our solution can be expressed by the following formula:

The graph regularization representation of (1) can be written as the following function:

D是一个对角矩阵，其中

Among them, L in the formula (2) represents the normalized graph Laplacian matrix, and

D is a diagonal matrix where

For multi-view data, formula (2) becomes:

where m represents the data of different views, Z ^(m) represents the low-dimensional data of the mth view, and L ^(m) represents the normalized graph Laplacian matrix of the mth view. And because the low-dimensional data Z ^(m) is obtained through the projection matrix P ^(m) and the original data X ^(m) , the formula (3) can be written in the following form:

的目的是避免平凡解。(4)式没有考虑视与视之间的关联关系。in,

The purpose is to avoid trivial solutions. Equation (4) does not take into account the relationship between vision and vision.

2. Adaptive Local Structure Learning

Since formula (4) does not explore the relationship between views, how to combine the relationship between views will be introduced next. The relationship between views and views can be effectively explored by sharing the similarity matrix of multiple views, because if the similarity matrix between views and views is the same, the Laplacian matrix between views and views will also be the same, then Formula (4) becomes:

And formula (5) is a modification of the following formula:

However, in formula (6), the similarity of the two most similar samples is 1, and the similarity of other samples is 0. In order to solve this problem, a regularization term is added to the F norm of the S matrix, and formula (6) becomes to make:

3. Adaptive multi-view dimensionality reduction based on graph embedding

The similarity in formula (7) only imposes a constraint on the data after dimensionality reduction, and does not constrain the original data before dimensionality reduction, so it cannot be guaranteed that the data structure after dimensionality reduction is the same as that before dimensionality reduction. Therefore, after adding constraints on the original data, formula (7) becomes:

Equation (8) does not consider the data reconstruction error:

表示把原始数据投影到低维空间，而

表示将投影到低维空间的数据在投影回原始空间，与原始数据计算一个差值。将公式(8)与公式(9)结合得到最终的公式：in

represents the projection of the original data into a low-dimensional space, and

Indicates that the data projected to the low-dimensional space is projected back to the original space, and a difference is calculated with the original data. Combine Equation (8) with Equation (9) to get the final formula:

Finally, through the iterative update, P ^(m) and S are solved.

(1) Fix S, update P ^(m)

和

固定不变，对剩余项构建拉格朗日函数后得：First, the similarity matrix S is constructed from the original data. When the fixed similarity matrix S is unchanged, the equation (10) in

and

Fixed and unchanged, after constructing the Lagrangian function for the remaining terms, we get:

Through the eigenvalue decomposition method, we can find P ^(m) and specify how many dimensions to reduce to.

(2) Fix P ^(m) and update S

When the projection matrix P ^(m) is fixed, the reconstruction error term in the formula will not participate in the update, and formula (10) becomes:

Set the distance between the low-dimensional data in formula (12) and the original data as d _ij :

Equation (12) can then be written as:

Solving equation (14) can become:

Thus, it can be obtained

To sum up, through the continuous iterative update of the method, the projection matrix P ^(m) of each view is finally obtained, and the dimension reduced by each view can be selected.

The datasets used in this experiment are text datasets and image datasets, where the text dataset is 3sources and the image dataset is IXMAS.

Among them, 3sources is a multi-view text dataset that collects news from three well-known news websites, BBC News, Reuters, and The Guardian. The dataset covers 416 different news stories from February to April 2009. This experiment selected 169 news stories reported by these three news websites. 3560, 3631, and 3068 dimensional features are extracted for news from three perspectives, and these news categories belong to one of business, entertainment, health, sports, politics, and technology. This dataset contains 169 samples with 6 categories. IXMAS is a multi-view image dataset that collects photos from five different angles. The dataset consists of photos of ten people performing eleven actions of checking their watch, crossing their arms, scratching their head, sitting down, getting up, spinning in circles, walking, waving, hitting, kicking, and picking up, and each action is repeated three times. This dataset contains 339 samples with 11 categories.

In order to verify the dimensionality reduction performance of the proposed scaled-up, the adaptive multi-view dimensionality reduction method (MVDR) based on graph embedding is compared with the classical PCA method, the multi-view non-negative matrix factorization method (MVNMF), and the multi-view dimensionality reduction method that can directly perform multi-view dimensionality reduction. The flexible multi-view dimensionality reduction method (McDR) is compared, and the data dimension is reduced to 20 dimensions uniformly, and the data after dimension reduction is clustered by a unified clustering method, and the clustering performance is compared.

This experiment uses three indicators of normalized mutual information (NMI), correctness (ACC) and purity (Purity) to evaluate its clustering performance. The higher the value of the indicators, the better the clustering performance will be. The specific results are shown in Table 1-Table 2.

Table 1 and Table 2 show the clustering performance after dimensionality reduction in the 3sources and IXMAS databases, respectively. Among them, single view means that the data from one perspective is used for dimensionality reduction and clustering, and multi-view means that the multi-view data is combined for dimensionality reduction and clustering. Clusters are marked in bold in the table with the best single- and multi-view performance results. It can be seen from these two tables that the results of multi-view dimensionality reduction are better than the corresponding single-view dimensionality reduction methods, and the MVDR dimensionality reduction method is better than other methods regardless of single-view or multi-view.

Looking at these tables, the following conclusions can be drawn:

1) The multi-look dimensionality reduction method MVDR is obviously better than other multi-look dimensionality reduction methods, and can achieve the best performance in the clustering effect. This proves that the proposed method can well integrate multi-view information to reduce dimensionality of data, and can extract more characteristic information to obtain better clustering results.

2) In the experiment, the clustering results of multi-view dimensionality reduction are basically better than single-view dimensionality reduction, which proves that multi-view fusion dimensionality reduction is more effective than single-view dimensionality reduction.

In short, compared with other dimensionality reduction methods, MVDR achieves higher clustering accuracy and better dimensionality reduction effect. This bodes well for the future practical application of the MVDR method.

Table 1

Table 2

The above are only preferred embodiments of the present invention, and do not limit the present invention in any form. Any simple modifications, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention still belong to the present invention The protection scope of the technical solution of the invention.

Claims (6)

1. The adaptive multi-view dimensionality reduction method based on graph embedding is characterized in that: the method comprises the following steps: (1) Embed high-dimensional data into low-dimensional space by means of graph embedding; (2) Measure the similarity of different samples by calculating the distance between different samples with the same view; (3) Explore the relationship between samples between different views by sharing a same similarity matrix from different views; (4) Measure similarity, so as to explore the relationship between different views; (5) The projection matrix of each view is obtained, and the final dimension reduction result is obtained by multiplying the projection matrix with the original data. 2. The adaptive multi-view dimensionality reduction method based on graph embedding according to claim 1, wherein: in the step (1), Suppose you are given a dataset

where D represents the dimension of each sample data, N represents the number of samples,

is the low-dimensional representation matrix of data X, where K represents the data dimension after sample dimension reduction and K<<N, if the original data x _i and x _j are similar, then their low-dimensional representations z _i and z _j are also similar, the first The similarity between the i sample and the jth sample is represented by s _ij , then the target formula (1) to be solved is expressed as:

The graph regularization of formula (1) is expressed as:

where L represents the normalized graph Laplacian matrix, and

D is a diagonal matrix where

For multi-view data, formula (2) becomes:

where m represents the data of different views, Z ^(m) represents the low-dimensional data of the mth view, L ^(m) represents the normalized graph Laplacian matrix of the mth view, and because the low-dimensional data Z ^(m) is It is obtained by the projection matrix P ^(m) and the original data X ^(m) , so formula (3) is written as:

in,

The purpose is to avoid trivial solutions. 3. The adaptive multi-view dimensionality reduction method based on graph embedding according to claim 2, wherein: in the step (3), formula (4) becomes:

Equation (6) adds a regularization term to the F norm of the S matrix, and Equation (6) becomes:

4. The adaptive multi-view dimensionality reduction method based on graph embedding according to claim 3, characterized in that: in the step (4), after adding constraints to the original data, the formula (7) becomes:

Equation (8) does not consider the data reconstruction error:

in

represents the projection of the original data into a low-dimensional space, and

It means that the data projected to the low-dimensional space is projected back to the original space, and a difference is calculated with the original data, and the formula (8) and the formula (9) are combined to obtain:

5. The adaptive multi-view dimensionality reduction method based on graph embedding according to claim 4, characterized in that: in the step (5), through iterative update, finally obtain the projection matrix P ^(m) of each view, and Choose the dimension that each view is reduced by, First, the similarity matrix S is constructed from the original data. When the fixed similarity matrix S is unchanged, the formula (10) in

and

Fixed and unchanged, after constructing the Lagrangian function for the remaining terms, we get:

Through the method of eigenvalue decomposition, find P ^(m) and specify how many dimensions to reduce; When the projection matrix P ^(m) is fixed, the reconstruction error term will not participate in the update, and formula (10) becomes:

Set the distance between the low-dimensional data in formula (12) and the original data as d _ij :

Formula (12) is written as:

Solving equation (14) becomes:

Thus, get

6. An adaptive multi-view dimensionality reduction device based on graph embedding, characterized in that: the device comprises: Embedding module, which embeds high-dimensional data into low-dimensional space by means of graph embedding; A distance calculation module, which measures the similarity of different samples by calculating the distance between different samples with the same view; Shared module, which explores the relationship between samples between different views by sharing a same similarity matrix from different views; A measurement module, which measures similarity to explore the relationship between different views; Iterative module, which obtains the projection matrix of each view, and obtains the final dimension reduction result by multiplying the projection matrix with the original data.

Images