CN111027636B - Unsupervised feature selection method and system based on multi-label learning - Google Patents

Unsupervised feature selection method and system based on multi-label learning Download PDF

Info

Publication number
CN111027636B
CN111027636B CN201911312573.7A CN201911312573A CN111027636B CN 111027636 B CN111027636 B CN 111027636B CN 201911312573 A CN201911312573 A CN 201911312573A CN 111027636 B CN111027636 B CN 111027636B
Authority
CN
China
Prior art keywords
matrix
feature selection
label
learning
unsupervised
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911312573.7A
Other languages
Chinese (zh)
Other versions
CN111027636A (en
Inventor
朱磊
石丹
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Normal University
Original Assignee
Shandong Normal University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Normal University filed Critical Shandong Normal University
Priority to CN201911312573.7A priority Critical patent/CN111027636B/en
Publication of CN111027636A publication Critical patent/CN111027636A/en
Application granted granted Critical
Publication of CN111027636B publication Critical patent/CN111027636B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/211—Selection of the most significant subset of features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling

Abstract

The present disclosure provides an unsupervised feature selection method and system based on multi-label learning, including: extracting features of each acquired data sample to obtain a feature data set, learning a binary multi-label matrix and a feature selection matrix for the feature data set, and constructing an unsupervised feature selection objective function based on multi-label learning; solving an unsupervised feature selection target function based on multi-label learning by adopting a discrete optimization method based on an augmented Lagrange multiplier method to obtain a feature selection matrix; and sequencing the feature selection matrix to determine the selected target features. Simultaneously learning multi-label and executing feature selection for semantic guidance, and applying binary constraint in spectrum embedding to obtain multi-label to guide the final feature selection process; in addition, a dynamic sample similarity graph capture data structure is constructed in an adaptive mode, and therefore the discrimination capability of multiple labels is enhanced.

Description

Unsupervised feature selection method and system based on multi-label learning
Technical Field
The disclosure relates to the technical field of feature selection, in particular to an unsupervised feature selection method and system based on multi-label learning.
Background
With the rapid development of information technology, high-dimensional data is emerging in different research fields, such as multimedia computing, data mining, pattern recognition, machine learning and the like. On the one hand, high dimensional data can provide richer information. On the other hand, it also presents a challenging dimensional disaster problem. The high-dimensional data usually contains noise or abnormal values, so that the direct use of such high-dimensional data often has a bad influence on the subsequent learning task and even reduces the performance of the method. To solve this problem, a dimension reduction technique is proposed, which includes two different processing methods: (1) selecting characteristics; (2) and (5) feature extraction.
Feature selection reduces the dimensionality of features by selecting important, discriminative features. Feature selection techniques are mainly divided into two categories depending on whether they rely on data tags or not: (1) supervised feature selection; (2) unsupervised feature selection. Among them, unsupervised feature selection is a more practical, but also more difficult task. For unsupervised feature selection, the most critical issue is how to accurately obtain feature information and use it to guide the feature selection process. In recent years, existing methods employ spectral analysis to explore the intrinsic information structure of the data. These methods include two steps: firstly, creating a sample similarity graph through spectrum analysis; the feature selection matrix is then learned based on spectral embedding.
Although good performance has been achieved, there are still some problems that need to be further solved: (1) the existing unsupervised feature selection method has no label guidance or uses a single label to guide the process of selecting features; the former causes the selected feature semantics to be missing, and the latter causes information loss. (2) The graph created by the existing graph-based feature selection method is not high in quality, a graph is usually constructed by a Gaussian kernel directly on original data, and the graph is kept constant in the whole model learning process. In addition, the process of graph creation and feature selection is split into two separate processes, which also results in a method that is sub-optimal.
Therefore, the key to improve the technical performance is: (1) learning more accurate labels that fit more into the data itself guides feature selection. Real-world data sets, including images, videos, biological data, etc., are often multi-labeled rather than unitary. (2) The quality of the graph is improved, the spectrum analysis is better combined with the feature selection, and the model is more accurately guided to select valuable features.
Disclosure of Invention
In order to overcome the defects of the prior art, the present disclosure provides an unsupervised feature selection method and system based on multi-label learning, which simultaneously learns the multi-labels for semantic guidance and performs feature selection, and applies binary constraint in spectrum embedding to obtain the multi-labels to guide the final feature selection process; in addition, a dynamic sample similarity graph capture data structure is constructed in an adaptive mode, and therefore the discrimination capability of multiple labels is enhanced.
In order to achieve the purpose, the following technical scheme is adopted in the disclosure:
in a first aspect, the present disclosure provides an unsupervised feature selection method based on multi-label learning, including:
extracting features of each acquired data sample to obtain a feature data set, learning a binary multi-label matrix and a feature selection matrix for the feature data set, and constructing an unsupervised feature selection objective function based on multi-label learning;
solving an unsupervised feature selection target function based on multi-label learning by adopting a discrete optimization method based on an augmented Lagrange multiplier method to obtain a feature selection matrix;
the feature selection matrix is ordered to determine the target features to be selected.
As some possible implementations, the unsupervised feature selection objective function based on multi-label learning is:
Figure BDA0002324935830000031
wherein x isi,xj∈R1×dRespectively represent the i, j (1. ltoreq. i, j. ltoreq. n) th sample, G ∈ Rn×nIs a dynamic similarity graph of learning, gijIs an element of row i and column j of G; | | non-woven hairFIs Frobenius norm (F norm for short) of the matrix;
Figure BDA0002324935830000032
is the Laplace matrix of the dynamic graph G, and the degree matrix D is a diagonal matrix with the ith diagonal element of
Figure BDA0002324935830000033
XTIs the transpose of the data matrix X, B is a binary multi-label matrix, B ∈ {0,1}n×lIs the discrete constraint of the binary multi-label matrix, l is the length of the binary multi-label, P ∈ Rd×lIs a feature selection matrix, mu, α is a balance parameter, sigma, β is a regularization parameter, and n is the number of samples.
As some possible implementations, the constructing an unsupervised feature selection objective function based on multi-label learning includes:
learning the feature selection matrix by using a regression model to obtain a low-dimensional feature subspace by using L2,1The norm minimizing item restricts the feature selection matrix to be row sparse;
creating a dynamic sample similarity graph from original characteristic data by a self-adaptive learning method, and obtaining a binary multi-label matrix by adopting spectral analysis;
and constructing the learned binary multi-label matrix and the feature selection matrix into an unsupervised feature selection objective function based on multi-label learning through spectrum embedding.
As some possible implementations, the learned feature selection matrix is represented as:
Figure BDA0002324935830000034
wherein, P ∈ Rd×lIs a feature selection matrix, XTIs the transpose of matrix X, B is a binary multi-label matrix, β is a regularization parameter;
Figure BDA0002324935830000041
is L of the matrix P2,1And (4) norm.
As some possible implementations, the learning process of the dynamic sample similarity graph is represented as:
Figure BDA0002324935830000042
wherein G is a dynamic similarity map, GijIs the element in the ith (1 ≦ i ≦ n) row and jth (1 ≦ j ≦ n) column of G, σ is the regularization parameter, and n is the number of samples.
As some possible implementation manners, the solving process for solving the unsupervised feature selection objective function based on the multi-label learning by using the discrete optimization method based on the augmented lagrange multiplier specifically includes:
fixing any two variables of a feature selection matrix, a binary multi-label matrix variable and a dynamic similarity graph matrix in the objective function, and solving a third variable;
and setting iteration times, and carrying out iterative solution on the solving process to obtain the local optimal solution of the feature selection matrix.
As some possible implementations, the solving process further includes,
spectral embedding Tr (B) for an objective functionTLGB) Item, using an auxiliary discrete variable Z ∈ {0,1}n×lAnd replacing a second binary multi-label matrix B variable, and solving the auxiliary discrete variable by fixing a dynamic similarity graph matrix, a feature selection matrix and a binary multi-label matrix variable in the unsupervised feature selection objective function based on multi-label learning.
In a second aspect, the present disclosure provides an unsupervised feature selection system based on multi-label learning, comprising,
the target function construction module is used for extracting the characteristics of each acquired data sample to obtain a characteristic data set, learning a binary multi-label matrix and a characteristic selection matrix for the characteristic data set, and constructing an unsupervised characteristic selection target function based on multi-label learning;
the solving module is used for solving an unsupervised feature selection target function based on multi-label learning by adopting a discrete optimization method based on an augmented Lagrange multiplier method to obtain a feature selection matrix;
and the selection module is used for sequencing the characteristic selection matrix to determine the target characteristic to be selected.
In a third aspect, the present disclosure provides a computer-readable storage medium having stored therein a plurality of instructions adapted to be loaded by a processor of a terminal device and to perform the steps of the method for unsupervised feature selection based on multi-tag learning.
In a fourth aspect, the present disclosure provides a terminal device comprising a processor and a computer-readable storage medium, the processor being configured to implement instructions; the computer readable storage medium is for storing a plurality of instructions adapted to be loaded by a processor and to perform the steps of the unsupervised feature selection method based on multi-tag learning.
Compared with the prior art, the beneficial effect of this disclosure is:
the method simultaneously learns multiple labels for semantic guidance and performs feature selection, and obtains the multiple labels by applying binary constraint in spectrum embedding to guide the final feature selection process in order to select features with discriminant under the unsupervised condition;
creating a dynamic sample similarity graph and applying a binary constraint in spectrum embedding to generate a binary multi-label, so that the label is more instructive and discriminable, and important features can be accurately selected under the guidance of the multi-label;
the unsupervised feature selection method based on multi-label learning can be extended to multi-view learning and can be used for processing unsupervised feature selection problems on multi-view data.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this disclosure, are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the disclosure and together with the description serve to explain the application and not to limit the disclosure.
FIG. 1 is a flow chart of the disclosed method;
FIG. 2 is a flowchart of the method of example 1;
FIG. 3 is a flowchart of the method of example 2.
Detailed Description
The present disclosure is further described with reference to the following drawings and examples.
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
Example 1
The present disclosure provides an unsupervised feature selection method based on multi-label learning, comprising:
s1: acquiring a real data set, and performing feature extraction on each data sample to acquire a data set in a feature form;
s2: constructing an unsupervised feature selection objective function based on multi-label learning for the feature data set;
s3: solving an unsupervised feature selection objective function based on multi-label learning to obtain a feature selection matrix;
s4: and determining the features to be selected in a sorting mode based on the learned feature selection matrix.
As one or more embodiments, the step S1 includes obtaining a feature data set X ∈ Rd×nWhere d is the feature dimension and n is the number of samples.
As one or more embodiments, the step S2 includes:
the present disclosure proposes learning binary multi-labels and performing feature selection simultaneously. Specifically, binary multitags are learned based on spectral analysis to guide the feature selection process and thus to select features with discriminability. Meanwhile, the method adaptively learns a dynamic sample similarity graph (which can well express the characteristic internal structure) from the original data, and the dynamic graph can further improve the discrimination capability of multiple labels. The regression model is used to learn the feature selection matrix and apply L to the feature selection matrix2,1Norm minimization constrains them to become sparse.
Further, the unsupervised feature selection objective function based on multi-label learning is as follows:
Figure BDA0002324935830000071
wherein x isi,xj∈R1×dRespectively represent the i, j (1. ltoreq. i, j. ltoreq. n) th sample, G ∈ Rn×nIs a dynamic similarity graph of learning, gijIs an element of row i (1. ltoreq. i.ltoreq.n) of G and column j (1. ltoreq. j.ltoreq.n); | | non-woven hairFIs the F-norm of the matrix (e.g.: C | | non-woven cells)FIs a matrix C ∈ Rd×nIs defined as the F norm of
Figure BDA0002324935830000072
Is the Laplace matrix of the dynamic graph G, and the degree matrix D is a diagonal matrix with the ith diagonal element of
Figure BDA0002324935830000073
XTIs the transpose of matrix X, B is a binary multi-label matrix, B ∈ {0,1}n×lIs the discrete constraint of the binary multi-label matrix, l is the length of the binary multi-label, P ∈ Rd×lIs a feature selection matrix, mu, α is a balance parameter, sigma, β is a regularization parameter, and n is the number of samples.
S21: the present disclosure learns a multi-label matrix and a feature selection matrix simultaneously. Learning a feature selection matrix P using a regression model, using L2,1The norm minimization term constrains the feature selection matrix to be row sparse, and the learning of data multi-labels and feature selection are combined into a unified model through spectrum embedding.
The spectral embedding is represented as:
Tr(BTLGB),s.t.B∈{0,1}n×l(2)
wherein Tr () represents the operation of finding the trace of the matrix, B is a binary multi-label matrix, B ∈ {0,1}n×lIs a discrete constraint of a binary multi-label matrix,
Figure BDA0002324935830000081
is the Laplace matrix of the similarity graph G, the degree matrix D is a diagonal matrix, the ith diagonal element of which is
Figure BDA0002324935830000082
The feature selection matrix learning process is represented as:
Figure BDA0002324935830000083
wherein, P ∈ Rd×lIs a matrix of feature choices that is,
Figure BDA0002324935830000084
is L of the matrix P2,1And (4) norm.
S22, in order to accurately acquire the internal structure information of the data and further improve the quality of the binary multi-label, the dynamic similarity graph which can better support the feature selection task is learned by the method, the dynamic sample similarity graph G ∈ R is adaptively learned, and the method and the device are provided with the function of adaptively learningn×nAs a graph model, consider that there are k nearest data points as sample xiIs assumed to be a sample point xiCan pass the similarity probability g with all the sample pointsijTo be associated. In general, for the distance between sample points
Figure BDA0002324935830000085
The smaller the likelihood, the greater the likelihood probability g that should be assignedij。
The process of learning the dynamic sample similarity graph is represented as:
Figure BDA0002324935830000086
as one or more embodiments, the step S3 includes:
because the objective function has a discrete constraint of a multi-label matrix, directly solving the objective function is an NP difficult problem. To solve this problem, the present disclosure proposes a discrete optimization method based on the Augmented Lagrange Multiplier (ALM) to directly solve the multi-label matrix.
The specific iterative steps of the optimization (solving for one variable by fixing other variables) include:
s31: fixing variables P and B in an unsupervised feature selection target function based on multi-label learning, solving a dynamic similarity graph matrix G, and changing the target function into:
Figure BDA0002324935830000091
equation (5) can be written as follows:
Figure BDA0002324935830000092
wherein, bi,bj∈R1×lRespectively are ith and j (i is more than or equal to 1 and j is more than or equal to n) row vectors of the multi-label matrix B.
The row vectors of matrix G are independent of each other, so matrix G is optimized row by row. For the row vector gi：
Figure BDA0002324935830000093
Where 1 denotes a column vector with all elements 1.
Definition of
Figure BDA0002324935830000094
Vector diThe jth element of (a) is
Figure BDA0002324935830000095
Equation (7) can be written in vector form as follows:
Figure BDA0002324935830000096
the lagrange function of equation (8) is:
Figure BDA0002324935830000097
wherein, theta, ηiA value of > 0 is the Lagrangian multiplier. According to KKT (Karush-Kuhn-Tucker) conditions, g is obtainediThe following solutions:
Figure BDA0002324935830000098
s32: fixing variables G and B in an unsupervised feature selection target function based on multi-label learning, solving a feature selection matrix P, and changing the target function into:
Figure BDA0002324935830000099
equation (11) derives P and makes it equal to 0, resulting in a solution for P:
P＝(XXT+βΛ)-1XB (12)
wherein Λ is a diagonal matrix whose elements
Figure BDA0002324935830000101
Is a very small constant.
S33: fixing variables G and P in an unsupervised feature selection target function based on multi-label learning, solving B, wherein the target function is changed into:
Figure BDA0002324935830000102
equation (13) can be converted to the following form:
Figure BDA0002324935830000103
the discrete constraint of the multi-label matrix in the objective function makes it difficult to directly solve the multi-label matrix B (NP-hard problem). The present disclosure proposes to solve B using a discrete optimization method based on the augmented lagrange multiplier method. In particular for BTLGItem B, using an auxiliary discrete variable Z ∈ {0,1}n×lTo replace the second B variable and maintain their equivalence during the optimization process. The following optimization formula is thus obtained:
Figure BDA0002324935830000104
where variable M is used to measure the difference between B and Z. The last term of equation (15) can be simplified as:
Figure BDA0002324935830000105
through the above conversion, the objective function of optimization B becomes:
Figure BDA0002324935830000111
a closed solution for B was obtained:
B＝(sgn(2αXTP-μLGZ+λZ-M)+1)/2 (18)
s34: fixing variables G, P and B in an unsupervised feature selection objective function based on multi-label learning, solving a variable Z, and optimizing the objective function as follows:
Figure BDA0002324935830000112
the closed solution for variable Z is as follows:
Z＝(sgn(-μLGB+λB+M)+1)/2 (20)
s35, according to the ALM theory, the optimization formula of M and lambda is as follows:
Figure BDA0002324935830000113
the method aims to obtain a dynamic graph G, a feature selection matrix P and a binary multi-label B by solving an objective function, and the objective function contains a plurality of unknown variables, so that the objective function cannot be directly solved; and the discrete constraint of the multi-label matrix in the objective function makes it difficult to directly solve the multi-label matrix B. Therefore, a discrete optimization method is provided to solve the algorithm, and a local optimal solution is finally obtained through a finite number of iterations by an iteration solution mode (fixing other variables and solving one variable).
As one or more embodiments, the step S4 specifically includes: selecting matrix P by using learned featuresi||2The importance of the features is measured, and then the features with r maximum values (most important) are selected by sorting.
Finally, a typical unsupervised learning task (clustering) is used to evaluate the performance of the feature selection method. Two general clustering evaluation indexes are adopted: and displaying the clustering result by using clustering Accuracy (ACC) and Normalized Mutual Information (NMI). Both indices range between 0 and 1, with larger numbers indicating better performance. The specific definitions of the two evaluation indexes are respectively:
①ACC
Figure BDA0002324935830000121
Ω＝[w1,...,wc]is a cluster, O ═ o1,...,oc]Is the category true value. biA predicted cluster label, t, representing sample iiIs a genuine label. And if si＝tiThen(s)i,ti) 1, otherwise(s)i,ti)＝0。
②NMI
Figure BDA0002324935830000122
Where C is a set of clusters obtained from the real tags and C' is a set of clusters obtained from the clustering algorithm. H (C) and H (C ') are the entropy of C and C ', respectively, and MI (C, C ') is a mutual information metric, and measures the degree of coincidence of the two data distributions.
The present disclosure proposes to jointly perform multi-label learning and feature selection, learning a binary multi-label for a sample as a supervised information (pseudo label) to support a final feature selection task in order to select important features in an unsupervised environment; in order to improve the quality of multi-label, a sample similarity map is dynamically created by fully exploring the data structure, and then a pseudo label is obtained by a spectrum analysis method. In this invention, multi-label learning, spectral analysis, and feature selection are facilitated in an efficient discrete optimization.
Example 2
As shown in fig. 3, this example also extends the unsupervised feature selection method based on multi-label learning into a multi-view setting, providing an unsupervised multi-view feature selection method based on multi-label learning.
The unsupervised multi-view feature selection method based on multi-label learning comprises the following steps:
s1: acquiring a real data set; the data set includes: a number of samples; extracting a plurality of view features from each sample to obtain a multi-view data set in a feature form;
s2: constructing an unsupervised multi-view feature selection objective function based on multi-label learning for the multi-view feature data set;
s3: solving an unsupervised multi-view feature selection objective function based on multi-label learning to obtain a feature selection matrix;
s4: and determining the features to be selected in a sorting mode based on the learned feature selection matrix.
As one or more embodiments, the step S1 includes:
acquiring a real data set, wherein V view features are extracted from each sample to obtain a multi-view feature data set X;
multi-view feature dataset X ═ X1,X2...,XV]∈Rn×dEach sample contains V views, where n is the number of samples, d is the total feature dimension, and V is the number of views;
Figure BDA0002324935830000131
for the data matrix of the v-th view, dvIs the characteristic dimension of the v-th view.
As one or more embodiments, the step S2 includes:
for each view feature, a sample similarity graph is created in advance by using a Gaussian kernel function;
further, the similarity graph A of the v view featurev：
Figure BDA0002324935830000132
Wherein, the neighbor relation of the sample is measured by a K neighbor algorithm KNN; v ═ 1,2,. V]Is the view index, V is the view number;
Figure BDA0002324935830000133
is a similarity graph of the vth view feature,
Figure BDA0002324935830000134
is the ith (i ═ 1, 2.., n, in the v view]N is the total number of samples) samples;
Figure BDA0002324935830000135
is the jth (j ═ 1, 2.., n in the vth view]) A sample is obtained; exp () represents an exponential function with e as the base,
Figure BDA0002324935830000141
representing a sample
Figure BDA0002324935830000142
The Euclidean distance between; σ is a bandwidth parameter of the function that controls the number of neighbors.
Further, the unsupervised multi-view feature selection objective function based on multi-label learning is as follows:
Figure BDA0002324935830000143
wherein, G ∈ Rn×nIs a dynamic fusion map, gijIs an element of the ith (i is more than or equal to 1 and less than or equal to n) row and the jth (j is more than or equal to 1 and less than or equal to n) column of G, | | | | | Y phosphorFIs the F-norm of the matrix (e.g.: C | | non-woven cells)FIs a matrix C ∈ Rd×nIs defined as the F norm of
Figure BDA0002324935830000144
Is a Laplace matrix of the dynamic fusion graph G, and the degree matrix D is a diagonal matrix with the ith diagonal element of
Figure BDA0002324935830000145
XTIs the transpose of the matrix X, B is the learned binary multi-label matrix, B ∈ {0,1}n×lIs the discrete constraint of the binary multi-label matrix, l is the length of the binary multi-label, P ∈ Rd×lIs the feature selection matrix, μ, α are balance parameters, β are regularization parameters.
Further, the S2 includes:
s21: for a multi-view feature dataset X ═ X1,X2...,XV]∈Rn×d，
Figure BDA0002324935830000146
d is the total feature dimension in the dataset, Av∈Rn×nThe similarity graph corresponding to the V (V is more than or equal to 1 and less than or equal to V) th view feature, a corresponding similarity graph is created in advance for each view feature, a dynamic fusion graph G is learned based on a plurality of pre-created similarity graphs, the aim is to learn the dynamic fusion graph G, and the dynamic graph is consistent with the sample similarity graphs of the plurality of pre-created views; therefore, the creation of a dynamic fusion map is achieved by minimizing the linear combination of reconstruction errors for each view;
weighting a plurality of pre-created similarity graphs in a self-adaptive manner and carrying out linear fusion to realize the learning of a dynamic fusion graph G; the similarity relationship representation of the multi-view data is defined as:
Figure BDA0002324935830000151
equation (24) can also be written in the form of an explicit view weight representation:
Figure BDA0002324935830000152
wherein, γv＝1/2||G-Av||FIs the weight of the v-th view feature.
S22: the multi-label learning and feature selection of data are combined into a unified model through spectrum embedding.
The spectral embedding is represented as:
Tr(BTLGB),s.t.B∈{0,1}n×l(26)
s23: we learn the feature selection matrix with a regression model, and use L2,1The norm minimization term constrains the feature selection matrix to be row sparse. The objective function of the process is:
Figure BDA0002324935830000153
wherein, P ∈ Rd×lIs the feature selection matrix and l is the length of the binary multi-label.
The S3 specifically includes: since a discrete constraint on a multi-label matrix exists in an objective function, directly solving the objective function is an NP difficult problem. In order to solve the problem, a discrete optimization method based on an augmented Lagrange multiplier method (ALM) is provided to directly solve the multi-label matrix.
The specific iterative steps of the optimization (solving for one variable by fixing other variables) include:
s31: fixing variables P and B in an unsupervised multi-view feature selection target function based on multi-label learning, solving variables G of a dynamic fusion graph, and changing the target function into:
Figure BDA0002324935830000154
equation (28) converts to the form:
Figure BDA0002324935830000161
since equation (29) is independent for different i, we can solve equation (29) separately for each i:
Figure BDA0002324935830000162
definition of
Figure BDA0002324935830000163
diIs the jth element as dijEquation (30) can be written as a vector as follows:
Figure BDA0002324935830000164
s32: and fixing variables G and B in the unsupervised multi-view feature selection objective function based on multi-label learning, and solving a feature selection array P. The solution is the same as equation (12) in example 1.
S33: and fixing the unsupervised multi-view feature selection target function variable G and P based on multi-label learning, and solving B. The solution is the same as equation (18) in example 1.
As one or more embodiments, the step S4 includes: selecting matrix P by using learned featuresi||2The importance of the features is measured, and then the features with r maximum values (most important) are selected by sorting.
The method carries out multi-view learning extension on an unsupervised feature selection method based on multi-label learning, carries out feature selection on multi-view data, and combines the multi-label learning and the feature selection into a unified model through spectral analysis; adaptively assigning weights to each view feature to create a fused similarity graph, wherein the graph can accurately capture the internal structural relationship among the multi-view features and improve the quality of the multi-label; the learned multi-labels can well guide the feature selection process to select important and discriminative features.
Example 3
The present disclosure provides an unsupervised feature selection system based on multi-label learning, comprising:
the target function construction module is used for extracting the characteristics of each acquired data sample to obtain a characteristic data set, learning a binary multi-label matrix and a characteristic selection matrix for the characteristic data set, and constructing an unsupervised characteristic selection target function based on multi-label learning;
the solving module is used for solving an unsupervised feature selection target function based on multi-label learning by adopting a discrete optimization method based on an augmented Lagrange multiplier method to obtain a feature selection matrix;
and the selection module is used for sequencing the characteristic selection matrix to determine the selected target characteristics.
Example 4
The present disclosure provides a computer readable storage medium having stored therein a plurality of instructions adapted to be loaded by a processor of a terminal device and to perform the steps of the method for unsupervised feature selection based on multi-tag learning.
Example 5
The present disclosure provides a terminal device comprising a processor and a computer-readable storage medium, the processor configured to implement instructions; the computer readable storage medium is for storing a plurality of instructions adapted to be loaded by a processor and to perform the steps of the unsupervised feature selection method based on multi-tag learning.
Although the present disclosure has been described with reference to specific embodiments, it should be understood that the scope of the present disclosure is not limited thereto, and those skilled in the art will appreciate that various modifications and changes can be made without departing from the spirit and scope of the present disclosure.

Claims (6)

1. The unsupervised feature selection method based on multi-label learning is characterized by comprising the following steps of:
extracting features of each acquired data sample to obtain a feature data set, constructing an unsupervised feature selection target function based on multi-label learning according to the feature data set, and learning a binary multi-label matrix and a feature selection matrix;
solving an unsupervised feature selection target function based on multi-label learning by adopting a discrete optimization method based on an augmented Lagrange multiplier method to obtain a feature selection matrix;
sequencing the feature selection matrix to determine target features to be selected;
the unsupervised feature selection objective function based on multi-label learning is as follows:
Figure FDA0002536506340000011
Figure FDA0002536506340000012
wherein x isi,xj∈R1×dRespectively represent the i, j samples, where 1 ≦ i, j ≦ n, G ∈ Rn×nIs a dynamic similarity graph of learning, gijIs an element in row i and column j of G, where i is greater than or equal to 1 and less than or equal to n, and j is greater than or equal to 1 and less than or equal to n; | | non-woven hairFIs the F norm of the matrix;
Figure FDA0002536506340000013
is the Laplace matrix of the dynamic graph G, and the degree matrix D is a diagonal matrix with the ith diagonal element of
Figure FDA0002536506340000014
XTIs the transpose of matrix X, B ∈ {0,1}n×lIs a binary multi-label matrix, l is the length of the binary multi-label, P ∈ Rd×lIs a feature selection matrix, mu, α is a balance parameter, sigma, β is a regularization parameter, n is the number of samples, | | P | | Y2,1Is L of the matrix P2,1A norm;
the constructing of the unsupervised feature selection objective function based on multi-label learning comprises:
learning the feature selection matrix by using a regression model to obtain a low-dimensional feature subspace by using L2,1The norm minimizing item restricts the feature selection matrix to be row sparse;
establishing a similarity graph corresponding to each feature by adopting a Gaussian kernel function for the feature data set, and obtaining a binary multi-label matrix by learning the dynamic similarity graph and adopting spectral analysis;
constructing a learned binary multi-label matrix and a feature selection matrix into an unsupervised feature selection target function based on multi-label learning through spectrum embedding;
learning the feature selection matrix as:
Figure FDA0002536506340000021
the solving process for solving the unsupervised feature selection target function based on the multi-label learning by adopting the discrete optimization method based on the augmented Lagrange multiplier method specifically comprises the following steps of:
fixing any two variables of a feature selection matrix, a binary multi-label matrix variable and a dynamic similarity graph matrix in the objective function, and solving a third variable;
and setting iteration times, and carrying out iterative solution on the solving process to obtain the local optimal solution of the feature selection matrix.
2. The unsupervised feature selection method based on multi-label learning of claim 1, wherein the learning process of the dynamic similarity graph is expressed as:
Figure FDA0002536506340000022
wherein G is a dynamic similarity map, GijIs the element of the ith row and the jth column of G, i is more than or equal to 1 and less than or equal to n, j is more than or equal to 1 and less than or equal to n, sigma is a regularization parameter, and n is the number of samples.
3. The unsupervised feature selection method based on multi-label learning of claim 1, wherein in the iterative solution:
spectral embedding Tr (B) for an objective functionTLGB) Item, using an auxiliary discrete variable Z ∈ {0,1}n×lAnd replacing a second binary multi-label matrix B variable, and solving the auxiliary discrete variable by fixing a dynamic similarity graph matrix, a feature selection matrix and a binary multi-label matrix variable in the unsupervised feature selection objective function based on multi-label learning.
4. An unsupervised feature selection system based on multi-label learning, comprising:
the target function construction module is used for extracting the characteristics of each acquired data sample to obtain a characteristic data set, constructing an unsupervised characteristic selection target function based on multi-label learning according to the characteristic data set, and learning a binary multi-label matrix and a characteristic selection matrix;
the solving module is used for solving an unsupervised feature selection target function based on multi-label learning by adopting a discrete optimization method based on an augmented Lagrange multiplier method to obtain a feature selection matrix;
the selection module is used for sequencing the feature selection matrix to determine target features to be selected;
the unsupervised feature selection objective function based on multi-label learning is as follows:
Figure FDA0002536506340000031
Figure FDA0002536506340000032
wherein x isi,xj∈R1×dRespectively represent the i, j samples, where 1 ≦ i, j ≦ n, G ∈ Rn×nIs a dynamic similarity graph of learning, gijIs an element in row i and column j of G, where i is greater than or equal to 1 and less than or equal to n, and j is greater than or equal to 1 and less than or equal to n; | | non-woven hairFIs the F norm of the matrix;
Figure FDA0002536506340000033
is the Laplace matrix of the dynamic graph G, and the degree matrix D is a diagonal matrix with the ith diagonal element of
Figure FDA0002536506340000034
XTIs the transpose of matrix X, B ∈ {0,1}n×lIs a binary multi-label matrix, l is the length of the binary multi-label;P∈Rd×lis a feature selection matrix, mu, α is a balance parameter, sigma, β is a regularization parameter, n is the number of samples, | | P | | Y2,1Is L of the matrix P2,1A norm;
the constructing of the unsupervised feature selection objective function based on multi-label learning comprises:
learning the feature selection matrix by using a regression model to obtain a low-dimensional feature subspace by using L2,1The norm minimizing item restricts the feature selection matrix to be row sparse;
establishing a similarity graph corresponding to each feature by adopting a Gaussian kernel function for the feature data set, and obtaining a binary multi-label matrix by learning the dynamic similarity graph and adopting spectral analysis;
constructing a learned binary multi-label matrix and a feature selection matrix into an unsupervised feature selection target function based on multi-label learning through spectrum embedding;
learning the feature selection matrix as:
Figure FDA0002536506340000041
the solving process for solving the unsupervised feature selection target function based on the multi-label learning by adopting the discrete optimization method based on the augmented Lagrange multiplier method specifically comprises the following steps of:
fixing any two variables of a feature selection matrix, a binary multi-label matrix variable and a dynamic similarity graph matrix in the objective function, and solving a third variable;
and setting iteration times, and carrying out iterative solution on the solving process to obtain the local optimal solution of the feature selection matrix.
5. A computer-readable storage medium having stored therein a plurality of instructions adapted to be loaded by a processor of a terminal device and to perform the method according to any one of claims 1-3.
6. A terminal device comprising a processor and a computer readable storage medium, the processor being configured to implement instructions; a computer readable storage medium for storing a plurality of instructions adapted to be loaded by a processor and for performing the method according to any of claims 1-3.
CN201911312573.7A 2019-12-18 2019-12-18 Unsupervised feature selection method and system based on multi-label learning Active CN111027636B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911312573.7A CN111027636B (en) 2019-12-18 2019-12-18 Unsupervised feature selection method and system based on multi-label learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911312573.7A CN111027636B (en) 2019-12-18 2019-12-18 Unsupervised feature selection method and system based on multi-label learning

Publications (2)

Publication Number Publication Date
CN111027636A CN111027636A (en) 2020-04-17
CN111027636B true CN111027636B (en) 2020-09-29

Family

ID=70209821

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911312573.7A Active CN111027636B (en) 2019-12-18 2019-12-18 Unsupervised feature selection method and system based on multi-label learning

Country Status (1)

Country Link
CN (1) CN111027636B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111931562A (en) * 2020-06-28 2020-11-13 山东师范大学 Unsupervised feature selection method and system based on soft label regression
CN112906767A (en) * 2021-02-03 2021-06-04 浙江师范大学 Unsupervised feature selection method based on hidden space learning and popular constraint

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103632138A (en) * 2013-11-20 2014-03-12 南京信息工程大学 Low-rank partitioning sparse representation human face identifying method
CN105046284A (en) * 2015-08-31 2015-11-11 鲁东大学 Feature selection based multi-example multi-tag learning method and system
CN106485096A (en) * 2016-10-20 2017-03-08 中南大学 MiRNA Relationship To Environmental Factors Forecasting Methodology based on random two-way migration and multi-tag study
CN106991447A (en) * 2017-04-06 2017-07-28 哈尔滨理工大学 A kind of embedded multi-class attribute tags dynamic feature selection algorithm
CN109685093A (en) * 2018-09-19 2019-04-26 合肥工业大学 Unsupervised adaptive features select method

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8774513B2 (en) * 2012-01-09 2014-07-08 General Electric Company Image concealing via efficient feature selection
US20180357531A1 (en) * 2015-11-27 2018-12-13 Devanathan GIRIDHARI Method for Text Classification and Feature Selection Using Class Vectors and the System Thereof
CN108537253A (en) * 2018-03-21 2018-09-14 华南理工大学 A kind of adaptive semi-supervised dimension reduction method constrained in pairs based on probability
CN109670418B (en) * 2018-12-04 2021-10-15 厦门理工学院 Unsupervised object identification method combining multi-source feature learning and group sparsity constraint
CN110348287A (en) * 2019-05-24 2019-10-18 中国地质大学（武汉） A kind of unsupervised feature selection approach and device based on dictionary and sample similar diagram

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103632138A (en) * 2013-11-20 2014-03-12 南京信息工程大学 Low-rank partitioning sparse representation human face identifying method
CN105046284A (en) * 2015-08-31 2015-11-11 鲁东大学 Feature selection based multi-example multi-tag learning method and system
CN106485096A (en) * 2016-10-20 2017-03-08 中南大学 MiRNA Relationship To Environmental Factors Forecasting Methodology based on random two-way migration and multi-tag study
CN106991447A (en) * 2017-04-06 2017-07-28 哈尔滨理工大学 A kind of embedded multi-class attribute tags dynamic feature selection algorithm
CN109685093A (en) * 2018-09-19 2019-04-26 合肥工业大学 Unsupervised adaptive features select method

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
A Multi-label Text Classification Framework: Using Supervised and Unsupervised Feature Selection Strategy;Long Ma;《百度学术》;20171231;论文第2-3节 *
Soft-constrained Laplacian score for semi-supervised multi-label feature selection;Abdelouahid Alalga et al;《百度学术》;20150517;全文 *
基于K-均值聚类的无监督的特征选择方法;张莉 等;《计算机应用研究》;20051231(第3期);全文 *
鲁棒的半监督多标签特征选择方法;严菲 等;《智能系统学报》;20190731;第14卷(第4期);全文 *

Also Published As

Publication number Publication date
CN111027636A (en) 2020-04-17

Similar Documents

Publication Publication Date Title
Azadi et al. Auxiliary image regularization for deep cnns with noisy labels
Van Der Maaten Accelerating t-SNE using tree-based algorithms
Wang et al. Unsupervised feature selection via unified trace ratio formulation and k-means clustering (track)
Dong et al. Adaptive collaborative similarity learning for unsupervised multi-view feature selection
US11294624B2 (en) System and method for clustering data
CN109977994B (en) Representative image selection method based on multi-example active learning
CN109784405B (en) Cross-modal retrieval method and system based on pseudo-tag learning and semantic consistency
CN110942091B (en) Semi-supervised few-sample image classification method for searching reliable abnormal data center
Chakraborty et al. Simultaneous variable weighting and determining the number of clusters—A weighted Gaussian means algorithm
Yang et al. A feature-metric-based affinity propagation technique for feature selection in hyperspectral image classification
CN111027636B (en) Unsupervised feature selection method and system based on multi-label learning
CN112232395B (en) Semi-supervised image classification method for generating countermeasure network based on joint training
CN111325264A (en) Multi-label data classification method based on entropy
CN111931562A (en) Unsupervised feature selection method and system based on soft label regression
Yang et al. Unsupervised feature selection based on reconstruction error minimization
Xu et al. Efficient top-k feature selection using coordinate descent method
CN110175631A (en) A kind of multiple view clustering method based on common Learning Subspaces structure and cluster oriental matrix
CN112800138B (en) Big data classification method and system
CN109614581A (en) The Non-negative Matrix Factorization clustering method locally learnt based on antithesis
CN112967755A (en) Cell type identification method for single cell RNA sequencing data
Zhong et al. Selection of diverse features with a diverse regularization
CN113971741A (en) Image labeling method, classification model training method and computer equipment
CN112308097A (en) Sample identification method and device
CN116701907B (en) Multi-label feature selection discriminating method based on self-adaptive graph diffusion
CN114677550B (en) Rapid image pixel screening method based on sparse discrimination K-means

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant