CN117409456A - Non-aligned multi-view multi-mark learning method based on graph matching mechanism - Google Patents

Abstract

The invention discloses a non-aligned multi-view multi-mark learning method based on a graph matching mechanism. Based on training data in the sample data set, a feature matrix and an observable mark matrix are constructed. Explicit alignment is carried out on unaligned data through a permutation matrix, namely point-to-point first-order alignment among samples; and carrying out second-order alignment on the graph structure on the view by using distance matrixes of samples in different views, so that the alignment accuracy of the model is further improved. The expression of 'commonality-personality' among aligned data views is mined, and a non-aligned multi-view multi-label learning model based on a graph matching mechanism is constructed by utilizing consistency and complementarity of cross views. Training the model by an alternative optimization method until the model converges to obtain the classification predictor.

Description

Unaligned multi-view multi-label learning method based on graph matching mechanism

Technical field

The invention relates to non-negative matrix decomposition, graph matching technology, non-aligned multi-view learning, and multi-label classification, and specifically relates to a non-aligned multi-view multi-label learning method based on a graph matching mechanism.

Background technique

The large-scale development of the Internet and the popularization of big data and artificial intelligence technology applications have also brought massive multi-view and multi-label data. Multi-view multi-label learning has received widespread attention as the main architecture for solving such problems. In the multi-view multi-label learning task, each sample object is represented by a variety of heterogeneous view information and is marked with several associated labels. However, existing multi-view multi-label learning methods usually explore relevant and complementary information across views. The search for this information is usually based on view alignment relationships (instances in different views describe the same object). However, this view alignment relationship can become a partial view alignment or a view non-alignment relationship due to spatial, temporal, or spatiotemporal asynchrony. For example: in video recommendation, the labeled data comes from different video software, but due to the user's privacy protection principle, these data cannot be matched and aligned with the same user. In the field of face recognition, due to the failure of facial feature detection, multi-view faces cannot be aligned, which makes facial expression recognition impossible. Existing multi-view multi-label learning models cannot directly learn robust multi-label classification models from these non-aligned data.

To this end, the present invention proposes a non-aligned multi-view multi-label classification via graph matching mechanism (MCGM for short) based on a graph matching mechanism to solve the problems in multi-view multi-label learning. Non-alignment issues and semantic comprehensive expression issues. In order to solve the non-alignment problem of multi-view data, by mining the cross-view "instance-instance" and "instance relationship-instance relationship" graph matching relationships, the feature nodes of the same instance in different views can be accurately aligned, and used for subsequent classification tasks; As the existing multi-view multi-label learning algorithm based on shared subspace representation is difficult to describe all the semantic information of multi-view data, a multi-view multi-label classification model based on "community-single" semantic representation is designed, emphasizing the single view. Contribution to specific semantic expressions, promoting semantic expression of few-category samples.

Contents of the invention

The technical problem solved by the present invention is to propose a non-aligned multi-view multi-label learning method based on a graph matching mechanism, realize the classification of non-aligned multi-view multi-label data, and ensure the efficiency and accuracy of the method through comprehensive semantic expression.

The technical solution of the present invention is: a non-aligned multi-view multi-label learning method based on a graph matching mechanism. First, the non-aligned multi-view multi-label data is obtained, and the non-aligned multi-view multi-label data is stored, preprocessed and data set. Divide to form a sample data set. Based on the training data in the sample data set, a feature matrix and an observable label matrix are constructed. Alignment of non-aligned data according to the feature matrix and observable label matrix: 1) Explicitly align the non-aligned data through the permutation matrix, that is, point-to-point first-order alignment between samples; 2) Use the distance matrix of samples in different views to align the views Perform second-order alignment on the graph structure to further improve the alignment accuracy of the model. Mining the expression of "commonality-individuality" between aligned data views, and utilizing the consistency and complementarity across views to build a non-aligned multi-view multi-label learning model based on the graph matching mechanism. The model is trained through the alternating optimization method until the model converges, and a classification predictor is obtained. Predict the test set based on the converged classifier, and use the output probability to obtain the label classification result. The specific steps are as follows:

In the present invention, matrices are represented by bold capital letters, such as X. Vectors are represented by bold lowercase letters, such as x. In addition, (XR) represents the matrix obtained from XR, where · is matrix multiplication. The inverse and transpose of matrix X are represented as X ^-1 and X ^T respectively. X _v represents the feature matrix of the v-th view, and the i-th column and j-th row of X _v are recorded as (X _v ) _:,i and (X _v ) _j,: respectively. (X _v ) _{i, j} is the (i, j) element of X _v , and _xi represents the i-th element of vector x. Additionally, we use Represents the field of real numbers, and the Frobenius norm is expressed as/>

Step S1: Obtain non-aligned multi-view multi-label data, store, pre-process and divide the data set into non-aligned multi-view multi-label data. Since this problem is a completely new one and there is currently no public non-aligned dataset, a synthetic dataset is used. Specifically, based on 6 public multi-view multi-label data sets, instances are randomly shuffled so that instances in different views describe different objects. Obtain non-aligned multi-view multi-label datasets.

Construct a sample data set with V views using the training data Among them/> is the complete feature space of the v-th view, n represents the number of training samples _xi , and d _v represents the dimension of each sample. Represents the label space corresponding to the feature space. Among them, _yi ∈ {0, 1} ^n×q is the tag vector of _xi , and q represents the number of tags.

Step S2: Construct the _first -order and second-order relationship matching across views for the feature matrix A non-aligned multi-view multi-label classification model with "community-single" semantic representation. Specific steps include:

(a) Use the permutation matrix to explicitly align the non-aligned data and perform point-to-point first-order alignment between instances. After first-order alignment, a relatively correct mapping relationship between instances is obtained, and non-negative matrix decomposition is used to extract a common low-dimensional representation of data from different views. Based on the obtained common representation and observable label matrix, the feature mapping matrix W ₀ is introduced to construct a linear mapping relationship from the shared subspace to the label space, and the initial learning model is obtained:

st(M _v ) _{i, j} ∈ {0, 1}, M _v 1＝1, 1 ^T M _v ＝1 ^T , P≥0, H _v ≥0, W ₀ ≥0#(1)

in, is the permutation matrix of the v-th view. The feature matrix is multiplied by the permutation matrix to obtain the aligned multi-view data. P and H _v are obtained by non-negative moment decomposition of the aligned data. /> Represents the individual mapping matrix of the v-th view;/> is a shared subspace, where k is the size of the expected data after reducing the dimension; P ≥ 0 and H _v ≥ 0 are non-negative constraints of the matrix. /> is the coefficient matrix corresponding to P, so W ₀ ≥ 0 is also constrained. W ₀ learns the common information of heterogeneous views by sharing the mapping relationship from subspace P to Y. α and γ are two hyperparameters. The last item is to regularize W ₀ to avoid over-fitting problems and reduce the influence of noise features.

(b) Considering the individual mapping matrix In fact, another set of coefficient matrices is defined for encoding the corresponding characteristics of a single view/> to capture the unique characteristics of a single view, where/> Map the reconstructed feature matrix PH _v with the v-th view personality information to the label space. Utilize the individuality and commonality information of heterogeneous views and further expand formula (1) as follows:

st(M _v ) _{i, j} ∈ {0, 1}, M _v 1＝1, 1 ^T M _v ＝1 ^T , P≥0, H _v ≥0, W ₀ ≥0, W _v ≥0#(2 )

Among them, P can be regarded as a dictionary matrix containing all view information, and H _v represents a single view-specific encoding coefficient. Therefore, PH _v aims to capture the individual information of a specific view, while P captures the shared information of all views. Therefore, PH _v and P can be regarded as the expression of the "commonality-individuality" of multi-view data.

(c) Further consider the alignment and labeling correlation of second-order graph structures between views. Since M _v X _v and M _j X _j represent the correctly aligned views of view X _v and view X _j respectively. Mapping matrices across views to represent the matching degree of cross-view samples between view X _v and view X _j . Based on the common understanding that the graph structure formed after multi-view data alignment should be as consistent as possible, a structure matching loss term is constructed to explore the correct cross-view mapping relationship. A distance matrix S _v between respective samples is established for each view, representing the graph structure of view v. By mapping matrices across views/> Replace the distance matrices S _v and S _j to make the graph connection structure of the two views as similar as possible. /> Represents the tag correlation matrix. It is intended to use the correlation in multi-mark data through matrix A to extract known relevant tag information. The hyperparameter β is introduced to balance the weight of the second-order alignment. Finally, the final unaligned multi-view multi-label learning model based on the graph matching mechanism is expressed as follows:

st(M _v ) _{i, j} ∈ {0, 1}, M _v 1＝1, 1 ^T M _v ＝1 ^T , P≥0, H _v ≥0, W ₀ ≥0, W _v ≥0

in

Step S3: Simplify the expression of the non-aligned multi-view multi-label learning model based on the graph matching mechanism in S2, and conduct alternate optimization training to minimize the model until the model converges to obtain a classification predictor. Specific steps include:

(a) Since the constraints of M _v are non-convex, it is difficult to obtain the optimal solution. And since orthogonal transformations do not change the relationship between vectors, this means that less stringent constraints can also preserve the structure of the data. Relax the constraint of M _v to:

(b) By introducing Lagrange multipliers λ, Φ, Θ, Ω, Ψ, the objective function is transformed into an unconstrained problem. Taking view υ as an example, the objective function (3) on view υ becomes the following form:

(c) Iterative optimization Fixed P,/> A, W ₀ ,/> When , M _v is calculated independently of M _v′ , v′≠v. Therefore, for each view v, M _v is optimized separately.

Standard approach to solving coupled equations Equation (3) and constraints is to use nonlinear methods, such as Newton's method. This nonlinear system of equations is often difficult to solve. Choose to seek an approximate solution. Through this method, the following iterative update rules for M _v can be obtained:

in:

(d) Iterative optimization of P. fixed A, W ₀ ,/> When, in formula (3)/> Taking the derivative of P, we can get:

Using the KKT condition, that is, Φ _{i, j} P _{i, j} = 0, the following iterative update rules for P can be obtained:

(e) Iterative optimization A. fixed P, W ₀ ,/> When, in formula (3)/> Taking the derivative of A, we can get:

Let the derivative equal to 0, we can get the following update rules for A:

(f)Iterative optimization Fixed P,/> A, W ₀ ,/> When , H _v is calculated independently of H _v , v′≠v. Therefore, for each view v, H _v is optimized separately, in formula (3)/> Taking the derivative of H _v , we can get:

Using the KKT condition, that is, Θ _{i, j} (H _v ) _{i, j} = 0, the following iterative update rules for H _v can be obtained:

(g) Iterative optimization of W ₀ . Fixed P, A,/> When, in formula (3)/> Taking the derivative of W ₀ , we can get:

Using the KKT condition, that is, Ω _i,j (W ₀ ) _i,j = 0, the following iterative update rules for W ₀ can be obtained:

(h) Iterative optimization Fixed P,/> A,/> When W ₀ , in formula (3)/> Taking the derivative of W _v , we can get:

Using the KKT condition, that is, Ψ _{i, j} (W _v ) _{i, j} = 0, the following update rules for W _v can be obtained:

(i) Repeat (c) to (h), updating parameters alternately A, W ₀ ,/> ,P until the iteration stop condition is met. The objective function converges, outputs the optimal parameters of the non-aligned multi-view multi-label learning model based on the graph matching mechanism, and obtains a classification predictor.

Step S4, based on the converged non-aligned multi-view multi-label learning model based on the graph matching mechanism, predict the test set, and use the output probability to obtain the label prediction result. The specific steps include: label prediction matrix will be equal to given.

The advantages of the present invention compared with the prior art are:

1. In view of the view non-alignment problem in multi-view multi-label learning tasks, first-order and second-order alignment are proposed to solve this problem. The reordering matrix can adaptively reorder the features in each view and perform first-order alignment to obtain the correct mapping relationship. Therefore, the view non-alignment problem can be simplified to a view alignment problem. In addition, the distance matrices of samples in different views are used to perform structural second-order alignment of the views, thereby improving the efficiency and accuracy of the alignment.

2. Aiming at the problem of comprehensive expression of multi-view and multi-label semantics, the method of the present invention can jointly utilize the consistency and diversity information of multi-view and multi-label data. The model learns a shared subspace from different views, label correlations, an ensemble classifier based on individual and shared feature spaces. Input the aligned data into this multi-view multi-label classification model based on the "community-single" semantic representation, emphasizing the contribution of the single view in specific semantic expression, and promoting the semantic expression of samples with few categories.

3. Learn the invisible correlation existing in the tags by introducing the dynamic tag association matrix A. While a fixed tag correlation matrix can be estimated based on a known tag matrix. However, only using known labeled samples may not be sufficient. The proposed dynamic label association matrix can adaptively measure the correlation between labels and help improve the learning performance of multi-label classification models.

4. The model is simplified to a general situation, and an iterative optimization method for solving the objective function is proposed. Find an approximate solution to the model under a certain time complexity. Afterwards, the effectiveness of the model is verified on six real-world datasets.

Description of the drawings

Figure 1 is a processing flow chart of the method of the present invention.

Figure 2 is a training workflow diagram of the method of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the solutions of the embodiments of the present invention, the embodiments of the present invention will be further described in detail below in conjunction with the drawings and implementation modes.

As shown in Figure 1, the present invention includes the following steps:

1. Obtain non-aligned multi-view multi-label data, store, preprocess and divide the data set. Constructed feature matrix X _v , observable label matrix Y. Since this problem is a completely new one and there are currently no publicly available non-aligned datasets, a synthetic dataset is used. Specifically, based on 6 public multi-view multi-label data sets, instances are randomly shuffled so that instances in different views describe different objects. Obtain non-aligned multi-view multi-label datasets. Construct a sample data set with V views using the training data is the label space corresponding to the feature set.

2. Construct the first-order and second-order relationship matching across views using _the feature matrix A non-aligned multi-view multi-label classification model with semantic representation of “community-single entity”. And perform alternating optimization training to minimize the model until the model converges and obtain the classification predictor. The objective function is as follows:

Specific steps are as follows:

(a) Input feature matrix X _v ; observable label matrix Y; dimension of shared subspace; hyperparameters in formula (6); convergence threshold; number of iterations. The last four input values may be changed due to different data sets to achieve better results.

(b) Randomly initialize M _v , H _v , W _v , P, W ₀ and A. Construct the adjacency matrix S _a of each view through the feature matrix.

(c) According to formulas (5), (7), (9), (11), (13), and (15), M _v , P, A, H _v , W ₀ and W _v are alternately and iteratively optimized respectively. Until the iteration stop condition is met, the above iteration stop condition can be that the difference between the two iterations of the objective function value is less than the convergence threshold, or the maximum number of iterations is reached, and finally the optimal solution of the objective function is output, and a non-aligned multi-step algorithm based on the graph matching mechanism is obtained. View multi-label learning model classifier.

3. Based on the converged non-aligned multi-view multi-label learning model based on the graph matching mechanism, predict the test set and use the output probability to obtain the label prediction results. The specific steps include: label prediction matrix will be equal to given. In order to obtain accurate marking information, set a certain threshold,/> Elements in the vector that are higher than this threshold are set to 1, that is, this mark is the mark of the sample. Below this threshold is set to 0, indicating that the marker is not a marker for this sample. The threshold setting can generally be 0.5, but the threshold often takes different values in different data sets.

The present invention is experimented on six real-world data sets to conduct in-depth experimental studies. And the six multi-view datasets used in the experiments are all publicly available. Their statistical data are summarized in Table 1. For each dataset, Table 1 summarizes the number of samples (n); the number of views (m); the number of different labels (c); the average number of labels per sample (#avg); and the minimum dimensionality of all views (d _min ).

Table 1 Statistics of six multi-view datasets

Emotions is a music data set, and the two views of each example correspond to the rhythm characteristics and timbre characteristics of a piece of music; Yeast is a biological data set, and the two views of each example correspond to the genetic expression and phylogeny of a gene; Corel5k, Pascal07, ESPGame, Mirflicker are four widely used multi-view image datasets. Multiple features of these images were collected, and each image was represented by 6 representative color space views, respectively oriented to different application backgrounds: HUE, SIFT, GIST, HSV, RGB and LAB. By randomly shuffling the instances so that instances in different views describe different objects, six non-aligned multi-view multi-label datasets are obtained. In addition, in order to verify the effectiveness of the MCGM method of the present invention, the MCGM method of the present invention was compared with the following six multi-label methods. Two of the single-view multi-label methods use a concatenation strategy to convert multi-view data sets into single-view data sets before conducting experiments. Other methods are multi-view multi-label learning methods. Comparative methods include single-view multi-label learning methods MLkNN and LLSF, which were published in the top journals in the field of computer vision 2007PR and the top conference in the field of data mining 2016TKDE. The multi-view multi-label learning methods FIMAN, ICM2L, iMvWL and BEMVL were published in the top journal in the field of data mining 2020SIGKDD, the top journal in the field of artificial intelligence 2019TCYB, the top conference in the field of artificial intelligence 2018IJCAI, and the top conference in the field of data mining 2022TKDD. This method uses five evaluation metrics that are widely used in multi-label learning to measure the performance of each algorithm. Specific evaluation indicators include Average Precision, Coverage, Hamming Loss, One Error and rankingLoss. The mean and standard deviation of each metric for each data set are shown in Tables 2 to 7. It should be noted that the present invention displays the value of 1-Ranking Loss in the table.

Table 2 Experimental results of Emotions (mean ± standard deviation)

Table 3 Experimental results of Yeast (mean ± standard deviation)

Table 4 Experimental results of Corel5k (mean ± standard deviation)

Table 5 Experimental results of Pascal07 (mean ± standard deviation)

Table 6 Experimental results of ESPGame (mean ± standard deviation)

Table 7 Experimental results of Mirflicker (mean ± standard deviation)

From the results reported in Table 2 to Table 7, it can be observed that MCGM outperforms other comparison methods in most cases, both in large and small datasets. In 30 experimental settings (6 data sets and 5 evaluation indicators), the ratio of the inventive method ranking first and second in the results is 57% and 40% respectively. And none of the methods is significantly better than the method of the present invention in terms of indicators.

The comparison of MCGM with LLSF and MLkNN shows that the performance of traditional multi-label methods can be seen to be improved by the parallel strategy into multi-view multi-label learning methods which are flawed mainly because they ignore the consistency and complementary information mining of multi-views. . That is, they ignore the physical meaning of individual views in the data set.

Comparison of MCGM with FIMAN, ICM2L, BEMVL and iMvWL shows that the invented method has good performance in dealing with non-aligned view problems. Since other algorithms do not consider the problem of view misalignment, they are flawed when encountering view misalignment. Among them, iMvWL ignores the diversity of views, which leads to its limitations in view information extraction.

It should be noted that the method in the embodiment of the present invention is suitable for non-aligned multi-view multi-label classification problems.

The embodiments of the present invention have been introduced in detail above, and specific implementation modes are used in this article to illustrate the present invention. The description of the above embodiments is only used to help understand the method of the present invention; at the same time, for those of ordinary skill in the art, based on this The idea of the invention will be subject to change in the specific implementation and scope of application. In summary, the contents of this description should not be understood as limiting the invention.

Claims (2)

1. The non-aligned multi-view multi-mark learning method based on the graph matching mechanism is characterized in that firstly, non-aligned multi-view multi-mark data are acquired, and the non-aligned multi-view multi-mark data are stored, preprocessed and data set divided to form a sample data set; constructing a feature matrix and an observable mark matrix based on training data in the sample data set; according to the feature matrix and the observable mark matrix, unaligned data are aligned: 1) Explicit alignment is carried out on unaligned data through a permutation matrix, namely point-to-point first-order alignment among samples; 2) Performing second-order alignment on the view on the graph structure by using distance matrixes of samples in different views, so that the alignment accuracy of the model is improved; mining the expression of commonality-personality among aligned data views, and constructing a non-aligned multi-view multi-mark learning model based on a graph matching mechanism by utilizing consistency and complementarity of cross views; training the model by an alternative optimization method until the model converges to obtain a classification predictor; and predicting the test set based on the converged classifier, and obtaining a mark classification result by using the output probability.

2. The non-aligned multi-view multi-label learning method based on a graph matching mechanism according to claim 1, wherein the method comprises the steps of,

step S1, acquiring non-aligned multi-view multi-mark data, and storing, preprocessing and dividing a data set for the non-aligned multi-view multi-mark data; adopting artificial synthetic data set; obtaining a non-aligned multi-view multi-marker dataset;

constructing a data set with V view samples from training dataWherein-> Is the complete feature space of the v-th view, n represents the training sample x _i Number d of (d) _v Representing the dimension of each sample;representing a markup space corresponding to the feature space; wherein y is _i ∈{0,1} ^n×q Is x _i Q represents the number of marks;

step S2, constructing a feature matrix X for training data in the sample data set acquired in the step S1 _v The observable mark matrix Y is used for constructing first-order and second-order relation matching of cross views, realizing characteristic alignment of a plurality of views, and constructing a non-aligned multi-view multi-mark classification model based on 'common-single' semantic representation; the method comprises the following specific steps:

(a) Explicit alignment is carried out on unaligned data by utilizing a permutation matrix, and point-to-point first-order alignment between examples is carried out; obtaining a relatively correct mapping relation between the examples after first-order alignment, and extracting common low-dimensional representation of different view data by adopting non-negative matrix factorization; introducing a feature mapping matrix W based on the obtained common representation and the observable marking matrix ₀ Constructing a linear mapping relation from a shared subspace to a mark space to obtain an initial learning model:

s.t.(M _v ) _i,j ∈{0,1},M _v 1＝1,1 ^T M _v ＝1 ^T ,P≥0,H _v ≥0,W ₀ ≥0(1)

wherein,is the replacement matrix of the v-th view, and the feature matrix is multiplied by the replacement matrix to obtain aligned multi-view data; p and H _v The aligned data are obtained through non-negative moment decomposition; />An individual mapping matrix representing a v-th view; />Is a shared subspace, where k is the size of the desired data reduced dimension; p is greater than or equal to 0 and H _v 0 is the non-negative constraint of the matrix; />Is a coefficient matrix corresponding to P, thus also constraining W ₀ ≥0；W ₀ Learning the commonality information of the heterogeneous view by sharing the mapping relation from the subspace P to the subspace Y; alpha and gamma are two hyper-parameters;

(b) Taking into account the individual mapping matrixIn fact, for encoding the corresponding characteristics of a single view, another set of coefficient matrices is defined +.>To capture the unique properties of a single view, +.>The reconstructed feature matrix PH with the v-th view personalized information _v Mapping to a markup space; utilizing personality and commonality information of heterogeneous views, andthe expansion formula (1) is as follows:

s.t.(M _v ) _i,j ∈{0,1},M _v 1＝1,1 ^T M _v ＝1 ^T ,P≥0,H _v ≥0,W ₀ ≥0,W _v ≥0(2)

wherein P can be regarded as a dictionary matrix containing all view information, and H _v Representing individual view-specific coding coefficients; PH value _v The goal of (1) is to capture individual information for a particular view, while P captures shared information for all views; PH value _v And P is considered as the expression of "commonality-personality" of the multi-view data;

(c) Considering alignment and mark correlation of second-order diagram structures among views; due to M _v X _v And M _j X _j Respectively represent view X _v And view X _j Is a properly aligned view of (1); mapping matrices with cross-viewTo represent view X _v And view X _j Matching degree of cross-view samples; constructing a structure matching loss item to explore a correct cross-view mapping relation; establishing a distance matrix S between respective samples for each view _v A graph structure representing view v; by mapping matrix across views->Permutation distance matrix S _v And S is _j Making the graph connection structure of the two views as similar as possible; />Representing a marker correlation matrix, and extracting known correlation marker information by utilizing correlation in the multi-marker data through the matrix A; introducing a weight of the second order alignment of the super-parameter beta balance; graph-based matching mechanismThe non-aligned multi-view multi-label learning model of (a) is expressed as follows:

s.t.(M _v ) _i,j ∈{0,1},M _v 1＝1,1 ^T M _v ＝1 ^T ,P≥0,H _v ≥0,W ₀ ≥0,M _v ≥0

wherein the method comprises the steps of

S3, simplifying the expression of the non-aligned multi-view multi-mark learning model based on the graph matching mechanism in S2, and performing alternating optimization training to minimize the model until the model converges to obtain a classification predictor; the method comprises the following specific steps:

(a) Due to M _v Is non-convex and it is difficult to obtain an optimal solution; handle M _v Is relaxed as:

(b) Converting the objective function into an unconstrained problem by introducing Lagrangian multipliers lambda, phi, theta, omega, psi; in view v, the objective function (3) is changed over view v to the following form:

(c) Iterative optimizationFix P->M _v Calculation is independent of M _v′ V' noteqv; thus for each view v, for M _v Optimizing independently;

standard methods solve the coupling equation (3) and constraintsThe following is obtained with respect to M using a nonlinear method _v Is a rule for iterative updating of:

wherein:

(d) Iterative optimization P; fixingIn the formula (3)>Derivative of P to obtain:

using the KKT condition, i.e. phi _i,j P _i,j =0, resulting in the following iterative update rule for P:

(e) Iterative optimization A; fixingIn the formula (3)>Derivative A is obtained by:

let the derivative equal to 0, the following update rule for A is obtained:

(f) Iterative optimizationFix->When H is _v Calculation is independent of H _v V' noteqv; for each view v, for H _v Optimizing alone, in formula (3)>For H _v Derivative is obtained, and the following steps are obtained:

using the KKT condition, i.e. Θ _i,j (H _v ) _i,j =0, giving the following relation to H _v Is a rule for iterative updating of:

(g) Iterative optimization W ₀ The method comprises the steps of carrying out a first treatment on the surface of the FixingIn the formula (3)>For W ₀ Derivative is obtained, and the following steps are obtained:

using the KKT condition, i.e. Ω _i,j (W ₀ ) _i,j =0, giving the following relation to W ₀ Is a rule for iterative updating of:

(h) Iterative optimizationFix->In the formula (3)>For W _v Derivative is obtained, and the following steps are obtained:

using the KKT condition, i.e. ψ _i,j (W _v ) _i,j =0 gets the following about W _v Updating rules:

(i) Repeating (c) to (h) continuously and alternatelyUpdating parameters Until the iteration stop condition is met; converging an objective function, outputting optimal parameters of the non-aligned multi-view multi-mark learning model based on the graph matching mechanism, and obtaining a classification predictor;

step S4, based on the non-aligned multi-view multi-mark learning model based on the graph matching mechanism after convergence, predicting the test set, and obtaining a mark prediction result by using the output probability, wherein the step comprises the following steps: marking prediction matricesWill be given by equationGiven.