CN111581469B - Multi-subspace representation-based partial multi-mark learning method - Google Patents
Multi-subspace representation-based partial multi-mark learning method Download PDFInfo
- Publication number
- CN111581469B CN111581469B CN202010412162.1A CN202010412162A CN111581469B CN 111581469 B CN111581469 B CN 111581469B CN 202010412162 A CN202010412162 A CN 202010412162A CN 111581469 B CN111581469 B CN 111581469B
- Authority
- CN
- China
- Prior art keywords
- subspace
- marking
- matrix
- model
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 239000011159 matrix material Substances 0.000 claims abstract description 63
- 238000013507 mapping Methods 0.000 claims abstract description 24
- 238000005457 optimization Methods 0.000 claims abstract description 21
- 238000012549 training Methods 0.000 claims abstract description 14
- 238000000354 decomposition reaction Methods 0.000 claims abstract description 8
- 239000003550 marker Substances 0.000 claims description 36
- 238000004422 calculation algorithm Methods 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 17
- 230000009467 reduction Effects 0.000 claims description 9
- 238000002372 labelling Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 5
- 102000004169 proteins and genes Human genes 0.000 description 8
- 108090000623 proteins and genes Proteins 0.000 description 8
- 230000008569 process Effects 0.000 description 4
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000000052 comparative effect Effects 0.000 description 3
- 238000002474 experimental method Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 201000010099 disease Diseases 0.000 description 2
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 2
- 230000008451 emotion Effects 0.000 description 2
- 230000002996 emotional effect Effects 0.000 description 2
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 239000002245 particle Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/906—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a multi-subspace representation-based partial multi-label learning method. Constructing a marking subspace by using a real marking matrix, constructing a characteristic subspace by using a characteristic mapping matrix, and obtaining a partial multi-marking learning model based on multi-subspace representation through the marking subspace and the characteristic subspace learning; performing alternating optimization training learning on the multi-label learning model based on the multi-subspace representation, and solving the multi-label learning model based on the multi-subspace representation to obtain an optimal prediction model; and inputting the unknown sample into an optimal prediction model, and outputting the marking information of the unknown sample by the optimal prediction model. The invention solves the problems of noise and redundant marks of the features, and uses the mapping matrix to map the feature space to the subspace, thereby reducing the influence of the feature noise on the prediction model; and reducing the dimension of the marking space to a marking subspace by using a matrix decomposition technology, and using the drawing Laplace constraint marking subspace to eliminate the influence of redundant marking noise on the prediction model.
Description
Technical Field
The invention relates to the technical field of computer application, in particular to a multi-subspace representation-based partial multi-label learning method.
Background
In recent years, multi-label learning is widely applied to the fields of automatic labeling of multimedia contents, bioinformatics, web mining, information retrieval, personalized recommendation and the like, but the prediction accuracy of a multi-label model is reduced due to inaccuracy of data set label information. In the practical application of multi-mark learning, due to the problems of shielding, light and the like, the data characteristics inevitably have noise problems. In order to better solve the problem that the training data marks have redundant marks, students in 2018 separate the multi-mark learning problem that the marks under the weak supervision framework have noise into partial multi-mark learning problems, and more partial multi-mark learning methods are published in international top conferences in recent two years.
The partial multi-mark learning solutions in the prior art can be mainly divided into two categories: unified frame-based partial multi-mark learning method and two-stage-based partial multi-mark learning method.
The multi-label learning based on the unified framework strategy takes the training process of the whole model as a whole, and the process of optimizing the candidate label set (namely the label set containing redundant noise) and the process of learning the prediction model are performed simultaneously, and the learning is performed in a unified framework. There are schemes to measure the probability that each candidate token becomes a true token using token confidence and to derive the true tokens (PML-fp and PML-lc) based on the token's rank. There is also proposed a feature-induced partial multi-label learning algorithm that uses potential dependencies between labels and features to identify noise labels and train a predictive model (fPML). It is also proposed to use a low-rank sparse decomposition model in the framework of partial multi-marker learning to decompose the observed marker matrix into a low-rank real marker matrix and a sparse noise marker matrix, and to obtain the real marker matrix and the prediction model (PML-LRS) simultaneously during training.
The disadvantage of the unified frame-based multi-label learning method in the prior art is that: when the candidate mark set is directly used for prediction model learning, the accuracy of the model can be influenced by the proportion of the redundant marks in the candidate mark set, and when the redundant marks are more, the accuracy of the model can be greatly reduced.
The partial multi-label learning based on the two-stage strategy comprises a selection stage of reliable labels and a prediction model learning stage. The method comprises the steps of obtaining more reliable marks in a candidate mark set by utilizing various disambiguation strategies in the first stage, and learning a prediction model by using the reliable marks as real marks in the second stage. There are schemes to assign different label confidence to each candidate label by a label propagation algorithm in the first stage, and to train a predictive model (part) by virtual label partitioning or Maximum A Posteriori (MAP) using a trusted label with high label confidence in the second stage. Also, in the first stage of the scheme, the label confidence is obtained by utilizing the dependency relationship between the label and the feature, and then a gradient enhancement algorithm of the label confidence is utilized to learn a prediction model (DRAMA) in the second stage.
The two-stage strategy-based partial multi-label learning method in the prior art has the following defects: the reliable mark obtained in the first stage has great influence on the prediction model, and the prediction result of the second stage model can have larger deviation under the conditions that the reliable mark is inaccurate and has larger difference from the real mark. In addition, the existing multi-label learning method does not consider that the characteristics can have a small amount of noise, and the characteristic noise can also influence the accuracy of the prediction model.
Disclosure of Invention
The embodiment of the invention provides a multi-subspace representation-based partial multi-label learning method, which overcomes the defects of the prior art.
In order to achieve the above purpose, the present invention adopts the following technical scheme.
A multi-subspace representation-based partial multi-label learning method, comprising:
constructing a marking subspace by using a real marking matrix, constructing a characteristic subspace by using a characteristic mapping matrix, and obtaining a partial multi-marking learning model based on multi-subspace representation through the marking subspace and the characteristic subspace learning;
performing alternating optimization training learning on the multi-subspace representation-based partial multi-label learning model, and solving the multi-subspace representation-based partial multi-label learning model to obtain an optimal prediction model;
and inputting an unknown sample into the optimal prediction model, and outputting the marking information of the unknown sample by the optimal prediction model.
Preferably, said constructing the markup subspace using the real markup matrix comprises:
true mark matrixObtaining a low-dimensional marking subspace by reducing the marking dimension, and marking the real marking matrix +.>The decomposition is a combination of the following two matrices:
U∈R n×c representing the real markup subspace after dimension reduction, P ε R c×q Representing a marked relationship matrix;
the marker subspace is obtained by minimizing the error between the marker matrix and the subspace reconstruction, as follows:
where R (U, P) is a regularization term used to control the complexity of the entire model;
defining a pairwise similarity matrix S epsilon R n×n ：
I.e. if x i And x j Mutually adjacent then S ij Equal to the calculated similarity between two samples, otherwise 0, the graph Laplace regularization term is introduced by minimizing the following equation:
wherein the method comprises the steps ofIs the matrix of the graph Laplace in this model, where +.>Is a diagonal matrix and the graph Laplace regularization term is used to constrain the marker subspace to have intrinsic consistency with the feature space.
Preferably, the constructing a feature subspace by using the feature mapping matrix, and learning by using the tag subspace and the feature subspace to obtain a partial multi-tag learning model based on multi-subspace representation, including:
setting a feature mapping matrix Q epsilon R d×m And mapping the original feature space into a low-dimensional subspace by using the feature mapping matrix Q to obtain a feature subspace, wherein the feature subspace is expressed as follows:
X T Q∈R n×m
wherein m is a feature dimension of the feature subspace;
using the feature subspace and the mark subspace to learn a prediction model W E R from the feature subspace to the mark subspace m×c ：
s.t.Q T Q＝I
After combining the above items, the following partial multi-label learning model based on multi-subspace representation is obtained:
s.t.Q T Q＝I
this term is the regular term of P, U, W in the present model, used to control the complexity of the model, and α, β, γ are tuning parameters used to maintain the balance of the model.
Preferably, the performing the alternating optimization training learning on the multi-subspace representation-based partial multi-label learning model, solving the multi-subspace representation-based partial multi-label learning model to obtain an optimal prediction model includes:
s3-1: initializing a multi-label learning model based on multi-subspace representation;
s3-2: fixing U, W, Q, updating P, the objective function of the model is equivalent to the following optimization problem:
by deriving zero from the above equation, the solution for P is as follows:
P＝(αU T U+γI) -1 αU T Y
s3-3: fixing P, W and Q, updating U, and enabling the objective function of the model to be equivalent to the following optimization problem:
iterative optimization is carried out on U by using a gradient descent algorithm, and the update rule of U is obtained as follows:
wherein lambda is U The gradient descending step length is obtained through an armijo criterion;
s3-4: fixing P, W, U, updating Q, the objective function of the model is equivalent to the following optimization problem:
s.t.Q T Q＝I
the gradient step-down algorithm is used to update Q, wherein the gradient step-down is also derived using the armijo criterion, the update rule is as follows:
after each iterative update, each row of Q is projected onto a unit sphere, namely:
wherein Q is i，： Is the ith row of the Q matrix.
S3-5: fixing P, U, Q, updating W, the objective function of the model is equivalent to the following optimization problem:
by deriving zero from the above equation, the solution for W is as follows:
W＝(Q T XX T Q+γI) -1 Q T XU
s3-6: repeating S3-2 to S3-5, continuously and alternately updating the parameters W and P until the iteration stop condition is met, converging the multi-subspace representation-based partial multi-label learning model, and outputting an optimal solution (P * ，U * ，Q * ，W * ) Obtaining an optimal prediction model W * 。
Preferably, the iteration stop condition includes the objective function value being smaller than a certain preset threshold, or each bit of W, P, U, Q no longer changes; or a maximum number of iterations is reached.
According to the technical scheme provided by the embodiment of the invention, the multi-subspace representation-based partial multi-mark learning method solves the problems of noise and redundant marks of the features, and the mapping matrix is used for mapping the feature space to the subspace, so that the influence of the feature noise on a prediction model is reduced; and reducing the dimension of the marking space to the marking subspace by using a matrix decomposition technology, and simultaneously, using the drawing Laplace constraint marking subspace to eliminate the influence of redundant marking noise on the prediction model.
Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a workflow diagram of a multi-subspace representation-based partial multi-label learning method embodying the present invention
FIG. 2 is a flowchart of a multi-subspace representation-based partial multi-label model training workflow in accordance with an embodiment of the present invention
FIG. 3 shows the results of a comparative experiment between the method of the present invention and the prior art
FIG. 4 shows the results of the test of Bonferroni-Dunn according to the method of the present invention and the results of the prior art.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for explaining the present invention and are not to be construed as limiting the present invention.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or coupled. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.
It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
For the purpose of facilitating an understanding of the embodiments of the invention, reference will now be made to the drawings of several specific embodiments illustrated in the drawings and in no way should be taken to limit the embodiments of the invention.
The multiple subspaces include a marker subspace and a feature subspace.
Partial multi-label learning is a multi-label learning under a weakly supervised framework, aimed at predicting a label set for a sample using sample feature data. The biggest difference is that the training data markers for partial multi-marker learning are inaccurate, and the true markers of the sample are only a part of the candidate marker set, i.e. redundant markers exist in the training set.
The method uses a learning method based on a characteristic subspace and a mark subspace, the method obtains the mark subspace through matrix decomposition by using a mark space dimension reduction method in mark insertion, obtains the characteristic subspace by using characteristic mapping, and restores the mark of the subspace to the original mark space after obtaining a prediction model by using subspace learning, thereby obtaining the real mark of the sample. The method of the invention simultaneously avoids the influence of noise of the feature and the mark space on the model prediction, and can greatly improve the accuracy of the prediction.
The embodiment of the invention provides a multi-subspace representation-based partial multi-label learning method, the processing flow of which is shown in a figure 1, and specifically comprises the following steps:
s1, constructing a characteristic sample matrix, and carrying out normalization processing on characteristic data in the characteristic sample matrix to obtain a characteristic matrix X epsilon R d×n D and n respectively represent the feature dimension and the number of samples, and are used for recording the feature distribution condition of a certain sample in each feature dimension. And construct a sample marking matrix Y εR n×q Q represents the label dimension, and is labeled 1 if the sample contains the label, and is labeled 0 otherwise.
S2, decomposing a sample marking matrix Y by using a marking space dimension reduction technology (LabelSpaceDimensionReduction, LSDR) in a marking embedding (LabelEmbeding, LE) algorithm to obtain a marking subspace U epsilon R n×c And a marked relationship matrix P E R c×q Where c represents the markup subspace dimension.
While using the graph laplace to ensure that there is internal consistency between the markup subspace and the features. Introducing a feature mapping matrix Q epsilon R d×m M represents a feature subspace dimension, and an orthogonal projection constraint is used to map the original feature space into a low-dimensional subspace, so as to obtain a feature subspace, wherein the feature subspace is represented as follows:
X T Q∈R n×m
where m is the feature dimension of the feature subspace.
Obtaining a multi-subspace representation-based partial multi-label learning model W epsilon R by using label subspace and feature subspace learning m×c 。
Step S3, performing alternating optimization training learning on the multi-subspace representation-based partial multi-label learning model, solving the multi-subspace representation-based partial multi-label learning model, and obtaining an optimal prediction model W * Obtain (P) * ，U * ，Q * ，W * )。
S4, unknown sample X * Input to the optimal predictive model W * The optimal prediction model W * Outputting a signature of the unknown sampleInformation Y * The following is shown:
Y * ＝X *T Q * W * P *
the partial multi-label learning model based on multi-subspace representation in step S2 specifically includes the following steps:
step S2-1: constructing a mark subspace: since the marker matrix of the sample data is low-rank in the multi-marker problem, it is easy to infer the true marker matrix in partial multi-marker learningAlso low rank, the true token matrix can be given a low dimensional token subspace by reducing the token dimension, i.e. the true token matrix can be approximated as a combination of two matrices:
wherein U is E R n×c Representing the label subspace constructed with reduced dimensions.
The U and primary label subspaces are inherently linked. P epsilon R c×q Representing the label relation matrix is equivalent to clustering q labels into c categories, thereby achieving the purpose of dimension reduction.
Since the marker matrix in the partial multi-marker learning contains a small amount of redundant noise, the invention can only obtain the marker subspace by reducing the error between the marker matrix and the subspace reconstruction, and the formula is as follows:
where R (U, P) is a regularization term used to control the complexity of the overall model.
In general, the ideal marker subspace obtained by the dimension reduction of the marker space should maintain the inherent consistency with the sample characteristics, so that the graph Laplace regularization term is introduced into the modelIntrinsic consistency is constrained. Firstly, a pairwise similarity matrix S epsilon R is defined n×n ：
I.e. if x i And x j Mutually adjacent then S ij Equal to the calculated similarity between the two samples, otherwise 0. Subsequently, to maintain inherent consistency, the present invention introduces the graph Laplace regularization term by minimizing the following equation:
wherein the method comprises the steps ofIs the matrix of the graph Laplace in this model, where +.>Is a diagonal matrix and the graph Laplace regularization term is used to constrain the marker subspace to have intrinsic consistency with the feature space.
Step S2-2: constructing a characteristic subspace: most of the existing partial multi-marker learning methods focus on how to solve redundant marker noise in the marker space, but neglect possible noise in the feature space. In practical applications, the characteristic information is often corrupted by outliers and noise, just like the tag space. And data features often have a high dimension and therefore can be time consuming to calculate. Therefore, the embodiment of the invention introduces the feature mapping matrix Q E R d×m Mapping the original feature space to a low-dimensional subspace to obtain a feature subspace, wherein the feature subspace is expressed as follows:
X T Q∈R n×m
wherein m is the feature dimension of the feature subspace, noise feature information is removed by mapping dimension reduction, more accurate feature information with identification capability is combined, and meanwhile, the calculation consumption is reduced.
By constructing the characteristic subspace to the mark subspace, a prediction model W E R from the characteristic subspace to the mark subspace is obtained through learning m×c The following is shown:
s.t.Q T Q＝I
the constraint condition of the above formula is to ensure that the projection matrixes are orthogonal, so that the characteristic subspace is more compact and has resolution.
After merging the above items, the partial multi-label learning model based on multi-subspace representation is as follows as a whole:
s.t.Q T Q＝I
in the present invention we useThis term is a canonical term of P, U, W in the present model that is used to control the complexity of the model. Where β, γ are tuning parameters used to maintain the balance of the model.
The training method of the partial multi-label learning model based on multi-subspace representation in the step S3 comprises the following steps:
s3-1: the model initialization, the optimization problem is that a convex model is easy to solve, so that the optimal solution (P) of the model can be obtained by using a method of alternately optimizing each parameter of the model * ，U * ，Q * ，W * )。
S3-2: fixing U, W, Q, updating P, the objective function of the model is equivalent to the following optimization problem:
by deriving the above equation to be equal to zero, the solution for P can be obtained as follows:
P＝(αU T U+γI) -1 αU T Y
s3-3: fixing P, W and Q, updating U, and enabling the objective function of the model to be equivalent to the following optimization problem:
the objective function is slightly adjustable, so that a standard gradient descent algorithm can be used for carrying out iterative optimization on U, and the U updating rule can be obtained as follows:
wherein lambda is U Is the gradient descent step, which is obtained by the armijo criterion in the present invention.
S3-4: fixing P, W, U, updating Q, the objective function of the model is equivalent to the following optimization problem:
s.t.Q T Q＝I
the problem is the same as step S3-3, and Q is updated by using a gradient descent algorithm, wherein the gradient step size is also obtained by using an armijo criterion, and the Q updating rule is as follows:
to satisfy the orthogonality constraint, each row of Q is projected onto a unit sphere after each iterative update, namely:
wherein Q is i，： Is the ith row of the Q matrix.
S3-5: fixing P, U, Q, updating W, the objective function of the model is equivalent to the following optimization problem:
by deriving the above equation to be equal to zero, the solution for W can be obtained as follows:
W＝(Q T XX T Q+γI) -1 Q T XU
s3-6: repeating S3-2 to S3-5, continuously and alternately updating the parameters W and P until the iteration stop condition is met, converging the multi-subspace representation-based partial multi-label learning model, and outputting an optimal solution (P * ，U * ，Q * ，W * ). The iteration stop condition comprises that the objective function value is smaller than a certain preset threshold value, or each bit of W, P, U and Q is not changed any more; or a maximum number of iterations is reached.
Experiments are carried out on Emotions, genbase, medical, corel5k, bibtex, eurlex-sm and Eurlex-dc data sets, and the method (Partial Multi-Label Learning via Multi-Subspace Representation, MUSER for short) is compared with the six main stream learning methods at present for experimental analysis; the comparison method comprises a classical ML-KNN algorithm and a RankSVM algorithm in the current multi-label learning field; the comparison method also comprises four latest algorithms of the current multi-label learning: partial multi-label learning algorithms (PML-fp and PML-lc) based on label confidence, published in the artificial intelligence field top-level conference AAAI2018; characteristic induction-based partial multi-label learning algorithm (fPML) published in data mining field conference ICDM2018; a partial multi-label learning algorithm (PML-LRS) based on low-rank sparse decomposition and a two-stage partial multi-label learning algorithm (PARTICLE) based on extraction of trusted labels are both published in an artificial intelligence field top-level conference AAAI2019; two-stage partial multiple mark learning algorithm (DRAMA) based on extracting trusted marks is published in IJCAI2019 of the top-level conference in the field of artificial intelligence. The invention uses HammingLoss, rankingLoss, oneError, coverage and AveragePrecision for comparative experimental analysis.
The effects dataset is a small dataset in the field of music consisting of 593 songs, each song having 72-dimensional musical features, separated into 6 emotional markers according to the Tellegen-Watson-Clark emotion model. There are a maximum of 3 tokens per song in the dataset, and the average emotional token per song is 1.87.
The Genbase dataset is a dataset for functional classification of proteins in the biological field. Each instance in the dataset is a protein and each tag is a protein class. The data set is relatively small and the number of markers is high. The dataset consisted of 662 proteins, a total of 27 protein markers, each protein having 1185 dimensional characteristics. There are a maximum of 6 protein markers per protein in the dataset, with an average marker of 1.25.
Medical data sets are one set of text data provided based on the Medical natural language processing challenges of the computing Medical center 2007. The dataset consisted of 978 clinical free text reports with a total of 45 disease code markers, each text report having 1449-dimensional features. There are a maximum of 3 disease code markers per report in the dataset, with an average marker of 1.25.
The Corel5k dataset is a dataset in the image domain that labels 5000 Corel images based on popular image classification and labeling methods. The dataset consisted of 5000 images, a total of 374 image marker markers, each reporting 499-dimensional features. There are a maximum of 5 image markers per image in the dataset, with an average marker of 3.52.
The Bibtex dataset is a text domain dataset based on ECML/PKDD 2008Discovery Challenge data. The dataset contained 7395 bibtex entries from the BibSonomy social tag and publication sharing system, for a total of 159 text tags, each entry having 1836 dimensional features. There are a maximum of 28 markers per entry in the dataset, with an average marker of 2.4.
The Eurlex dataset is a collection of 19348 documents on the european union law. It contains many different types of files. Wherein the Eurlex-sm dataset has a total of 201 markers, each file contains 5000-dimensional features, and each file in the dataset has a maximum of 12 markers, with an average marker of 2.21. The Eurlex-dc dataset has a total of 412 markers, each file contains 5000-dimensional features, and each file in the dataset has a maximum of 7 markers, with an average marker of 1.29.
FIG. 3 shows the results of comparative experiments of the ML-KNN, rankSVM, PMLfp, fPML, PARTICLE, DRAMA and MUSER seven methods in the case that the three redundant markers of the seven data sets are configured to be 1, and the evaluation index is the result after ten cross-validations, wherein the bolded values represent the optimal values; FIG. 4 shows the Bonferroni-Dunn test results of MUSER and other methods, wherein the average rank of each comparison algorithm is arranged along an axis marker, any method with average rank in the heavy line range is comparable to the MUSER method to some extent in each subgraph; the experimental results show that: compared with the current advanced multi-mark learning method, the method has the advantage that the performance is improved to a great extent.
The invention discloses a multi-sub-space representation-based partial multi-mark learning method, which is a collaborative training method for reducing the influence of characteristic noise and redundant mark noise in the process of protecting partial multi-mark learning; in an embodiment, the method of obtaining the feature subspace and the label subspace uses a most basic mapping matrix and matrix factorization model, respectively, and the regularization term uses a basic F-norm as well, and it will be apparent to those skilled in the art that various modifications may be readily made to the above-described embodiments and that the general principles described herein may be applied to other embodiments without the inventive effort. Improvements and modifications that are made in accordance with the claims are therefore intended to be included within the scope of the present invention.
In summary, the present invention provides a multi-subspace representation-based partial multi-label learning method. Mapping the data features from the source feature space to the feature subspace by using a feature mapping matrix, so that noise influence existing in the features is reduced; adopting a mark space dimension reduction technology in a mark insertion algorithm to reduce the dimension of a candidate mark set of data to obtain a mark subspace; the method is based on a unified framework, the feature subspace is more differentiated by using orthogonal projection constraint, and the feature subspace and the data feature have internal consistency by using the drawing Laplace constraint dimension reduction marker.
The partial multi-mark learning method based on multi-subspace representation solves the problems of noise and redundant marks of the features, and the mapping matrix is used for mapping the feature space to the subspace, so that the influence of the feature noise on a prediction model is reduced; and reducing the dimension of the marking space to the marking subspace by using a matrix decomposition technology, and simultaneously, using the drawing Laplace constraint marking subspace to eliminate the influence of redundant marking noise on the prediction model.
Those of ordinary skill in the art will appreciate that: the drawing is a schematic diagram of one embodiment and the modules or flows in the drawing are not necessarily required to practice the invention.
From the above description of embodiments, it will be apparent to those skilled in the art that the present invention may be implemented in software plus a necessary general hardware platform. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the embodiments or some parts of the embodiments of the present invention.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus or system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, with reference to the description of method embodiments in part. The apparatus and system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
The present invention is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present invention are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.
Claims (2)
1. A multi-subspace representation-based partial multi-label learning method, comprising:
constructing a marking subspace by using a real marking matrix, constructing a characteristic subspace by using a characteristic mapping matrix, and obtaining a partial multi-marking learning model based on multi-subspace representation through the marking subspace and the characteristic subspace learning;
performing alternating optimization training learning on the multi-subspace representation-based partial multi-label learning model, and solving the multi-subspace representation-based partial multi-label learning model to obtain an optimal prediction model;
inputting an unknown sample into the optimal prediction model, and outputting the marking information of the unknown sample by the optimal prediction model;
the constructing the marking subspace by using the real marking matrix comprises the following steps:
true mark matrixObtaining a low-dimensional marking subspace by reducing the marking dimension, and marking a real marking matrixThe decomposition is a combination of the following two matrices:
U∈R n×c representing the real markup subspace after dimension reduction, P ε R c×q Representing a marked relationship matrix;
the marker subspace is obtained by minimizing the error between the marker matrix and the subspace reconstruction, as follows:
where R (U, P) is a regularization term used to control the complexity of the entire model;
defining a pairwise similarity matrix S epsilon R n×n ：
I.e. if x i And x j Mutually adjacent then S ij Equal to the calculated similarity between two samples, otherwise 0, the graph Laplace regularization term is introduced by minimizing the following equation:
wherein the method comprises the steps ofIs the matrix of the graph Laplace in the modelWherein->Is a diagonal matrix, and the graph Laplace regularization term is used for constraining the labeling subspace to have intrinsic consistency with the feature space;
the construction of the feature subspace by using the feature mapping matrix, and the acquisition of the partial multi-mark learning model based on multi-subspace representation through the mark subspace and the feature subspace learning, comprises the following steps:
setting a feature mapping matrix Q epsilon R d×m And mapping the original feature space into a low-dimensional subspace by using the feature mapping matrix Q to obtain a feature subspace, wherein the feature subspace is expressed as follows:
Y T Q∈R n×m ,
wherein m is a feature dimension of the feature subspace;
using the feature subspace and the mark subspace to learn a prediction model W E R from the feature subspace to the mark subspace m×c ：
s.t.Q T Q＝I
After combining the above items, the following partial multi-label learning model based on multi-subspace representation is obtained:
s.t.Q T Q＝I
this term is the regular term of P, U, W in the present model, used to control the complexity of the model, alpha, beta, gamma are tuning parameters, used to maintain the balance of the model,
the alternately optimizing training learning is carried out on the multi-subspace representation-based partial multi-label learning model, the multi-subspace representation-based partial multi-label learning model is solved, and an optimal prediction model is obtained, and the method comprises the following steps:
s3-1: initializing a multi-label learning model based on multi-subspace representation;
s3-2: fixing U, W, Q, updating P, the objective function of the model is equivalent to the following optimization problem:
by deriving zero from the above equation, the solution for P is as follows:
P＝(αU T U+γI) -1 αU T Y，
s3-3: fixing P, W and Q, updating U, and enabling the objective function of the model to be equivalent to the following optimization problem:
iterative optimization is carried out on U by using a gradient descent algorithm, and the update rule of U is obtained as follows:
wherein lambda is U The gradient descending step length is obtained through an armijo criterion;
s3-4: fixing P, W, U, updating Q, the objective function of the model is equivalent to the following optimization problem:
s.t.Q T Q＝I，
the gradient step-down algorithm is used to update Q, wherein the gradient step-down is also derived using the armijo criterion, the update rule is as follows:
after each iterative update, each row of Q is projected onto a unit sphere, namely:
Q i，：
wherein Q is i Is the i-th row of the Q matrix,
s3-5: fixing P, U, Q, updating W, the objective function of the model is equivalent to the following optimization problem:
by deriving zero from the above equation, the solution for W is as follows:
W＝(Q T XX T Q+γI) -1 Q T XU，
s3-6: repeating S3-2 to S3-5, and continuously and alternately updating parameters W and P until the iteration stop condition is met, converging the multi-label learning model based on multi-subspace representation, and outputting an optimal solution (P, U, Q and W) of the multi-label learning model based on multi-subspace representation to obtain an optimal prediction model W.
2. The method of claim 1, wherein the iteration stop condition comprises an objective function value less than a certain preset threshold, or each bit of W, P, U, Q no longer changes; or a maximum number of iterations is reached.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010412162.1A CN111581469B (en) | 2020-05-15 | 2020-05-15 | Multi-subspace representation-based partial multi-mark learning method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010412162.1A CN111581469B (en) | 2020-05-15 | 2020-05-15 | Multi-subspace representation-based partial multi-mark learning method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111581469A CN111581469A (en) | 2020-08-25 |
| CN111581469B true CN111581469B (en) | 2024-03-01 |
Family
ID=72125041
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010412162.1A Active CN111581469B (en) | 2020-05-15 | 2020-05-15 | Multi-subspace representation-based partial multi-mark learning method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111581469B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108267451A (en) * | 2017-01-02 | 2018-07-10 | 泰山医学院 | A kind of cloth evenness intelligent checking system |
| CN108388656A (en) * | 2018-03-06 | 2018-08-10 | 南京邮电大学 | A kind of image searching method based on label correlation |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10325220B2 (en) * | 2014-11-17 | 2019-06-18 | Oath Inc. | System and method for large-scale multi-label learning using incomplete label assignments |
-
2020
- 2020-05-15 CN CN202010412162.1A patent/CN111581469B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108267451A (en) * | 2017-01-02 | 2018-07-10 | 泰山医学院 | A kind of cloth evenness intelligent checking system |
| CN108388656A (en) * | 2018-03-06 | 2018-08-10 | 南京邮电大学 | A kind of image searching method based on label correlation |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111581469A (en) | 2020-08-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112214995B (en) | Hierarchical multitasking term embedded learning for synonym prediction | |
| Zhang et al. | Medical image classification based on multi-scale non-negative sparse coding | |
| Varma et al. | Snuba: Automating weak supervision to label training data | |
| Hu et al. | Collective reconstructive embeddings for cross-modal hashing | |
| Zheng et al. | Multi-modal graph learning for disease prediction | |
| US10949456B2 (en) | Method and system for mapping text phrases to a taxonomy | |
| CN112136145A (en) | Attention filtering for multi-instance learning | |
| Leimeister et al. | Skip-gram word embeddings in hyperbolic space | |
| Guan et al. | A unified probabilistic model for global and local unsupervised feature selection | |
| Zhan et al. | Comprehensive distance-preserving autoencoders for cross-modal retrieval | |
| Sadeghi-Tehran et al. | Scalable database indexing and fast image retrieval based on deep learning and hierarchically nested structure applied to remote sensing and plant biology | |
| Chen et al. | Bridging computational features toward multiple semantic features with multi-task regression: A study of CT pulmonary nodules | |
| Kastaniotis et al. | HEp-2 cell classification with vector of hierarchically aggregated residuals | |
| CN111582506A (en) | Multi-label learning method based on global and local label relation | |
| Leng et al. | Bi-level artificial intelligence model for risk classification of acute respiratory diseases based on Chinese clinical data | |
| CN114003726A (en) | Subspace embedding-based academic thesis difference analysis method | |
| Mantoux et al. | Understanding the variability in graph data sets through statistical modeling on the Stiefel manifold | |
| Hong et al. | A mixtures-of-trees framework for multi-label classification | |
| CN111581469B (en) | Multi-subspace representation-based partial multi-mark learning method | |
| Singh et al. | Meta-DZSL: a meta-dictionary learning based approach to zero-shot recognition | |
| Qi et al. | Supervised deep semantics-preserving hashing for real-time pulmonary nodule image retrieval | |
| Zhong et al. | Auto-encoded latent representations of white matter streamlines for quantitative distance analysis | |
| Liu et al. | Large margin and local structure preservation sparse representation classifier for Alzheimer’s magnetic resonance imaging classification | |
| Theodorou et al. | Synthesize extremely high-dimensional longitudinal electronic health records via hierarchical autoregressive language model | |
| Lin et al. | FocAnnot: patch-wise active learning for intensive cell image segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |