CN103761532B - Label space dimensionality reducing method and system based on feature-related implicit coding - Google Patents
Label space dimensionality reducing method and system based on feature-related implicit coding Download PDFInfo
- Publication number
- CN103761532B CN103761532B CN201410024964.XA CN201410024964A CN103761532B CN 103761532 B CN103761532 B CN 103761532B CN 201410024964 A CN201410024964 A CN 201410024964A CN 103761532 B CN103761532 B CN 103761532B
- Authority
- CN
- China
- Prior art keywords
- matrix
- dimensionality reduction
- function
- optimum
- space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Abstract
The invention provides a label space dimensionality reducing method and system based on feature-related implicit coding. The method comprises the following steps: a training dataset is provided; a feature matrix and a label matrix are constructed according to the training dataset; an optimal related function between a dimensionality reducing matrix and the feature matrix is acquired according to the feature matrix, and an optimal error recovery function between the dimensionality reducing matrix and the label matrix is acquired according to the label matrix; an objective function is constructed according to the optimal related function and the optimal error recovery function; the objective function is used for optimizing dimensionality reducing matrix, and a decoding matrix is acquired through solution according to the optimized dimensionality reducing matrix; the optimized dimensionality reducing matrix is used for learning and training so that a prediction model can be acquired; features of a test case are extracted, and the prediction model is used for predicting expression of the test case in a latent semantic space; the expression of the test case in the latent semantic space is decoded through the decoding matrix so that a classification result of the test case in an original label space can be acquired. The label space dimensionality reducing method is high in compression ratio, good in stability and high in universality.
Description
Technical field
The present invention relates to computer software technology, more particularly to a kind of Label space drop of feature based correlation implicit expression coding
Dimension method and system.
Background technology
Multi-tag sorting technique（Multi-label classification）It is mainly used in for certain example being divided into one
Among individual or multiple classifications, so as to feature that is more complete, meticulously describing example, and the classification belonged to by example also by
The label being referred to as corresponding to which（Label）.Multi-tag sorting technique has quite varied application, such as multi-tag in reality
Text classification, linguistic indexing of pictures, audio frequency sentiment analysis etc..In recent years, emerging in multitude and rapidly sending out with network application
Exhibition, multi-tag classification application starts to face lot of challenges and the difficulty brought by data volume expansion, including Label space
Rapid growth etc..For example, on picture sharing website Flickr, user in uploading pictures can from it is millions of even more
Some contents for describing picture are selected in many vocabularies.For network image semantic tagger etc. is by means of Flickr
For the multi-tag classification application of data, these text vocabulary will be considered different labels, so as to so huge number of tags
Amount will bring significant increase cost on using the Algorithm Learning process of bottom to these.For multi-tag classification, at present
The basic thought of a large amount of methods is remained and is broken down into multiple two classification problems, is each label training and is predicted accordingly
Model（Predictive model）For judging whether an example belongs to the label, most at last the example belonged to it is all
Label is used as its corresponding multiple description.When Label space rapid expansion, i.e., when number of labels is very huge, needed for these methods
Forecast model quantity to be trained also rapidly increases, so as to cause its training cost greatly to rise.
Label space dimensionality reduction appear as solve number of labels it is huge in the case of multi-tag classification problem indicate one
Feasible probing direction, and technical support is provided, progressively became a focus of research circle in recent years, and emerged
Some outstanding dimension reduction methods.For example, using the openness of original tag space, by by compressed sensing（Compressed
sensing）Method carries out the dimensionality reduction of Label space, and is carried out from latent semantic space to original mark using its corresponding decoding algorithm
Sign the recovery in space.On the basis of this scheme, there is learning process of the researcher further by reduction process with forecast model to unify
To under same probabilistic model framework, and then by optimizing the lifting that above-mentioned two process obtains classification performance simultaneously.In addition, having
Study also by principal component analytical method a bit（Principal component analysis）It is applied on Label space dimensionality reduction, claims
For Principal label space transformation methods.Further, there is researcher by feature space and latent language
Dependency between adopted space takes into account, it is proposed that Feature-ware conditional principal label
Space transformation methods, obtain more obvious performance boost.Separately there is researcher to it is also proposed using linear
Gaussian random projecting direction original tag space is mapped, and the value of symbol after reserved mapping is used as dimensionality reduction result, and
Decoding process is then based on KL divergences using a series of（Kullback-Leibler divergence）Hypothesis test come real
It is existing.Also researcher directly carries out Boolean matrix decomposition by the mark matrix to training data（Boolean matrix
decomposition）, dimensionality reduction matrix and decoding matrix are obtained, wherein, dimensionality reduction matrix is dimensionality reduction result, and decoding matrix is then
It is the Linear Mapping that latent semantic space is returned to original tag space.
From the current study, main solution is to presuppose an explicit code function, and is generally taken
For linear function.But due to the complexity of higher dimensional space structure, explicit code function possibly cannot accurately describe original tag
Space to the mapping relations between optimum latent semantic space, so as to affect final dimensionality reduction result.Additionally, despite a small amount of work
Work can not assume that explicit code function, but directly learn dimensionality reduction result, but these work at present will not be latent semantic empty
Between take into account with the dependency of feature space, the dimensionality reduction result for finally giving may be caused to be difficult to be learnt from feature space
Described by the forecast model for arriving, so as to cause final classification performance not good.
The content of the invention
It is contemplated that at least solving one of technical problem in correlation technique to a certain extent.For this purpose, the present invention
One purpose be propose it is a kind of have information consider fully, classification performance conservation degree is high, Label space compression ratio is big, stability
The Label space dimension reduction method of the strong feature based correlation implicit expression coding of good, universality.
Further object is that proposing a kind of Label space dimensionality reduction system of feature based correlation implicit expression coding.
First aspect present invention embodiment proposes a kind of Label space dimension reduction method of feature based correlation implicit expression coding,
Comprise the following steps：Training dataset is provided；According to the training dataset structural features matrix and mark matrix；According to described
Eigenmatrix obtains the optimum correlation function of dimensionality reduction matrix and the eigenmatrix, and obtains the drop according to the mark matrix
Dimension matrix and the optimum restoration errors function for marking matrix；According to the optimum correlation function and the optimum restoration errors
Construction of function object function；Using dimensionality reduction matrix described in the objective function optimization, and according to the dimensionality reduction Matrix Solving after optimization
Go out decoding matrix；Using the dimensionality reduction matrix learning training after the optimization obtaining forecast model；Test case feature is extracted, and
Expression of the test case in latent semantic space is predicted using the forecast model；And using the decoding matrix to institute
State expression of the test case in the latent semantic space to be decoded, to obtain the test case in original tag space
Classification results.
The Label space dimension reduction method of feature based correlation implicit expression coding according to embodiments of the present invention, in study dimensionality reduction knot
Also its dependency with the restoration errors of mark matrix and with feature space has been taken into full account during fruit, by optimization
Process ensure that dimensionality reduction result can return to mark matrix well, while the prediction that also can be learnt on feature space
Described by model such that it is able to preferable multi-tag classification performance is obtained under relatively low training cost.
In some instances, each dimension of the latent semantic space is mutually orthogonal.
In some instances, the classification results to the test case in original tag space carry out binary conversion treatment.
In some instances, dimension of the dimension of the latent semantic space less than the original tag space.
The embodiment of second aspect present invention proposes a kind of Label space dimensionality reduction system of feature based correlation implicit expression coding,
Including：Training module, for carrying out learning training to obtain forecast model according to training dataset；Prediction module, for basis
The forecast model obtains classification results of the test case in original tag space.
The Label space dimensionality reduction system of feature based correlation implicit expression coding according to embodiments of the present invention, in study dimensionality reduction knot
Also its dependency with the restoration errors of mark matrix and with feature space has been taken into full account during fruit, by optimization
Process ensure that dimensionality reduction result can return to mark matrix well, while the prediction that also can be learnt on feature space
Described by model such that it is able to preferable multi-tag classification performance is obtained under relatively low training cost.
In some instances, the training module is specifically included：Constructing module, for according to training data structural features square
Battle array and mark matrix；Optimization module, for the optimum between dimensionality reduction matrix and the eigenmatrix is obtained according to the eigenmatrix
Correlation function, and the optimum restoration errors function between dimensionality reduction matrix and the mark matrix is obtained according to the mark matrix；
MBM, for according to the optimum correlation function and the optimum restoration errors construction of function object function, and applies institute
After stating dimensionality reduction matrix described in objective function optimization, decoding matrix is solved using the dimensionality reduction Matrix Calculating after optimization；Study module, is used for
Using the dimensionality reduction matrix learning training after the optimization obtaining forecast model.
In some instances, each dimension of the latent semantic space is mutually orthogonal.
In some instances, the classification results to the test case in original tag space carry out binary conversion treatment.
In some instances, dimension of the dimension of the latent semantic space less than the original tag space.
The additional aspect of the present invention and advantage will be set forth in part in the description, and partly will become from the following description
Obtain substantially, or recognized by the practice of the present invention.
Description of the drawings
Fig. 1 is the flow process of the Label space dimension reduction method of feature based correlation implicit expression coding according to embodiments of the present invention
Figure；
Fig. 2 is the principle of the Label space dimension reduction method of the feature based correlation implicit expression coding of one embodiment of the invention
Figure；
Fig. 3 is the structural frames of the Label space dimensionality reduction system of feature based correlation implicit expression coding according to embodiments of the present invention
Figure；With
Fig. 4 is the structured flowchart of the training module of one embodiment of the invention.
Specific embodiment
Embodiments of the invention are described below in detail, the example of the embodiment is shown in the drawings, wherein from start to finish
Same or similar label represents same or similar element or the element with same or like function.Below with reference to attached
The embodiment of figure description is exemplary, it is intended to for explaining the present invention, and be not considered as limiting the invention.
In fact, Label space dimensionality reduction, its main purpose is to compress the original tag space of higher-dimension（Original label
space）, on the premise of acceptable algorithm performance is kept, it is encoded into the latent semantic space of a low-dimensional（Latent
semantic space）, so as to by original model training process by from feature space（Feature space）To original tag
The learning process of the forecast model in space, resolve into forecast model from feature space to latent semantic space learning process and
From latent semantic space to the decoding process in original tag space.It is by dimensionality reduction, pre- needed for from feature space to latent semantic space
Model quantity is surveyed, is compared with the quantity needed for before dimensionality reduction, will be greatly reduced.Also, if forecast model is accurate enough, together
When, it is also accurate enough and efficient to the decoding process in original tag space from latent semantic space, then the multi-tag for finally giving
Classification performance should be still acceptable theoretically, and at the same time train cost to be greatly reduced.
A kind of Label space dimensionality reduction side of feature based correlation implicit expression coding is proposed in the embodiment of one aspect of the present invention
Method, comprises the following steps：Training dataset is provided；According to training dataset structural features matrix and mark matrix；According to feature
Matrix obtains the optimum correlation function of dimensionality reduction matrix and eigenmatrix, and obtains dimensionality reduction matrix with mark matrix according to mark matrix
Optimum restoration errors function；According to optimum correlation function and optimum restoration errors construction of function object function；Application target letter
Number optimization dimensionality reduction matrix, and decoding matrix is solved according to the dimensionality reduction Matrix Calculating after optimization；Learnt using the dimensionality reduction matrix after optimization
Train to obtain forecast model；Test case feature is extracted, and predicts test case in latent semantic space using forecast model
Expression；And the expression using decoding matrix to test case in latent semantic space is decoded, to obtain test case
In the classification results in original tag space.
Fig. 1 is the flow process of the Label space dimension reduction method of feature based correlation implicit expression coding according to embodiments of the present invention
Figure, Fig. 2 are the principle framework figures of the Label space dimension reduction method of the feature based correlation implicit expression coding of one embodiment of the invention.
With reference to the Label space dimension reduction method of Fig. 1 implicit expression codings related to the feature based that Fig. 2 specifically describes the present invention.
Step S101：Training dataset is provided.
The principle framework figure of the Label space dimension reduction method of feature based correlation implicit expression coding as shown in Figure 2, the present invention
Method include training process and prediction process.In the training process, need to give a number of training dataset.
Step S102：According to training dataset structural features matrix and mark matrix.
Specifically, it is to the training dataset comprising m test case for giving, suitable according to the Attributions selection of data itself
Characteristic type, and extract corresponding characteristic vector x=[x for each of which test case1,x2,...,xd], wherein, xiIt is
The i-th dimension of characteristic vector x.After the characteristic vector for obtaining all test cases, can be spliced into by which is about to random order
Required eigenmatrix X, X are the matrixes of m × d, wherein, d is the dimension of characteristic vector.
At the same time, to the training dataset comprising m test case, count the quantitative value of the different labels for wherein occurring
K, and situation is belonged to according to the label of each test case, it is which constructs corresponding label vector y=[y1,y2,...,yk],
Wherein, yjRepresent whether the example belongs to j-th label.If it is, value is 1, conversely, value is 0, by that analogy.Together
Sample ground, after the label vector for obtaining all test cases, can be spliced into mark matrix Y by which is about to, here with feature square
The splicing sequence consensus of battle array, Y is the matrix of m × k.
Step S103：The optimum correlation function of dimensionality reduction matrix and eigenmatrix is obtained according to eigenmatrix, and according to mark
Matrix obtains the optimum restoration errors function of dimensionality reduction matrix and mark matrix.
Specifically, the optimum correlation function of dimensionality reduction matrix and eigenmatrix is on the one hand obtained according to eigenmatrix.
In actual mechanical process, with reference to the method for implicit expression coding, it is assumed that there is dimensionality reduction Matrix C.Dimensionality reduction Matrix C and feature
Dependency between matrix X can be decomposed into the dependency sum between each row of dimensionality reduction Matrix C and eigenmatrix X.For
Any one row c in dimensionality reduction Matrix C, which can be described by cosine dependency with the dependency between eigenmatrix X, table
Procured function form is as follows：
Wherein, r is a Linear Mapping of X, for X to be projected to the space at c places.
At the same time, in order to reduce the redundancy in dimensionality reduction result, it is assumed that each row of dimensionality reduction Matrix C are mutually orthogonal,
That is, each dimension of the latent semantic space of dimensionality reduction Matrix C description is mutually orthogonal, its corresponding mathematic(al) representation is CTC=E.By
CTC=E can learn cTC=1, also, r carries out linear scale and do not affect the value of cosine dependency ρ, thus can construct as
Under majorized function be used to solve Linear Mapping r of optimum, and then obtain the optimal correlation of c and X.
ρ*=maxr(Xr)Tc
Constraints：(Xr)T(Xr)=1
The Linear Mapping of optimum can be drawn by method of Lagrange multipliersAgain substitute into
Optimum dependency can be obtained to above-mentioned majorized function isWherein, Δ=X (XTX)-1XT。
Therefore, the optimal correlation between dimensionality reduction Matrix C and eigenmatrix X can be expressed as following functional expression：
Wherein, C.,iRepresent the i-th row of dimensionality reduction Matrix C.
On the other hand the optimum restoration errors function of dimensionality reduction matrix and mark matrix is obtained according to mark matrix.
With reference to the method for implicit expression coding, it is assumed that there is dimensionality reduction Matrix C, C is the matrix of m × l, wherein, l is the dimension after dimensionality reduction
Degree size, therefore l ＜ ＜ k.
On the premise of it is assumed that dimensionality reduction Matrix C is present, the error for returning to mark matrix by C is can be expressed as such as minor function
Formula：
Wherein, D be to ensure decoding efficiency and the linear codec matrix that introduces,What is represented is matrix
Frobenius normal forms square.In the case where dimensionality reduction Matrix C is given, optimum restoration errors are minimum ε.Therefore, lead to
Cross the restoration errors for minimizing decoding matrix D and optimum that ε can obtain optimum.By constructing following majorized function：
The optimum decoding matrix D of decoding matrix D can be solved*=(CTC)-1CTY, due to CTC=E be unit matrix, D*Can
With further abbreviation as D*=CTY, substitutes into above-mentioned majorized function again, obtains optimum restoration errors function：ε*=Tr[YTY-
YTCCTY], the wherein mark of Tr [] representing matrix, that is, diagonal entry sum.
Step S104：According to optimum correlation function and optimum restoration errors construction of function object function.
Can obtain from dimensionality reduction Matrix C returning to the optimum restoration errors function of ε of mark matrix Y by step S103*=Tr
[YTY-YTCCTY], and its optimum correlation function with eigenmatrix XTherefore optimum dimensionality reduction matrix
Above-mentioned optimum restoration errors function should be able to be minimized simultaneously and maximizes above-mentioned optimum correlation function.
From the property of trace of a matrix, ε*=Tr[YTY-YTCCTY]=Tr[YTY]-Tr[YTCCTY], due to Tr [YTY] it is normal
Number, then minimize ε*It is equivalent to maximize Tr [YTCCTY].Additionally, for optimal correlation part, due to maximizing
It is equivalent to maximizeTherefore, maximize P*It is equivalent to maximizeTherefore, in order to same
When minimize ε*And maximize P*, the following object function of equal value of construction can be passed through, to solve the dimensionality reduction Matrix C of optimum.
Ω=maxCTr[YTCCTY]+αTr[CTΔC]
Constraints：CTC=E
Wherein, α is weight parameter, for adjusting the weight relationship between optimum restoration errors and optimal correlation.According to
The property of trace of a matrix, above-mentioned object function can be rewritten as following form：
Ω=maxCTr[CT(YYT+αΔ)C]
Constraints：CTC=E
Step S105：Application target function optimization dimensionality reduction matrix, and decoding square is solved according to the dimensionality reduction Matrix Calculating after optimization
Battle array.
Specifically, the object function Ω for being obtained by step S104, can obtain optimum dimensionality reduction Matrix C.The solution of C can be with
It is decomposed into the optimization problem of each row.For dimensionality reduction Matrix C i-th arranges C.,iOptimization Solution, following optimization can be constructed and asked
Topic：
Constraints：
Using method of Lagrange multipliers, it can be deduced that optimum C., iFollowing optimality condition need to be met：
Wherein, λiIt is the Lagrange multiplier for introducing, and substitutes into the Ω that above-mentioned optimization subproblem can obtain optimumi,
That is λi。
Can be observed by above-mentioned optimality condition and be obtained, optimum C.,iIt is matrix (YYT+ α Δs) a unit character to
Amount（Eigenvector）, and due to the orthogonality of characteristic vector, orthogonality constraint below can meet naturally.So far can send out
Existing, dimensionality reduction Matrix C is actually by matrix (YYT+ α Δs) in l maximum eigenvalue of correspondence（Eigenvalue）Unit it is special
Levy vector to be spliced by row.Therefore, the process for solving dimensionality reduction Matrix C is to matrix (YYT+ α Δs) carry out Eigenvalues Decomposition
（Eigenvalue decomposition）Process, such that it is able to obtain the matrix all eigenvalues and each eigenvalue pair
The unit character vector answered.As the complexity of Eigenvalues Decomposition is not more thanAnd due to (YYT+ α Δs) it is symmetrical square
Battle array, while only needing the unit character vector corresponding to l maximum eigenvalue in one embodiment of the invention, therefore drops
The complexity of dimension Matrix C can be less thanIt is efficient enough so as to ensure that the process for solving dimensionality reduction Matrix C.
In addition, after dimensionality reduction Matrix C is solved by Eigenvalues Decomposition method, can be by minimizing from dimensionality reduction matrix
C returns to optimum restoration errors between original mark matrix Y to try to achieve optimum decoding matrix D.By the calculating of step S103
Cheng Kezhi, optimum decoding matrix D*=CTY。
Step S106：Using the dimensionality reduction matrix learning training after optimization obtaining forecast model.
According to the optimum dimensionality reduction Matrix C that step S105 is obtained, it is each dimension training of the latent semantic space described by which
Corresponding forecast model.Specifically, for i-th dimension（1≤i≤l）, value of j-th test case in the dimension be
Cj,i, so as to the vector that the value of the training data in the dimension of all test cases is constituted is the i-th row C of C.,i.According to
The eigenmatrix X of the training data of all test cases and value vector C in latent semantic space i-th dimension.,i, can learn
Train corresponding forecast model hi:X→C.,i, for predicting value condition of any example in the dimension, its input is survey
The characteristic vector of examination example, and it is then value of the test case in i-th dimension to export.
In actual operation, the selecting of forecast model can be arranged according to the concrete condition of application, conventional including line
Property return（Linear regression）Deng.Through Label space dimensionality reduction, dimension l of latent semantic space is often much smaller than here
Dimension k in original tag space, so that the quantity of required forecast model is greatly lowered, effectively reduces and is trained to
This.
Step S107：Test case feature is extracted, and predicts test case in latent semantic space using forecast model
Represent.
Specifically, when a test case to be sorted is given, need to extract which and training process identical feature,
And obtain the d dimensional feature vectors of the test case
Obtaining the characteristic vector of test caseAfterwards, the l forecast model for being obtained using step S106 learning, in advance
Value of the test case in each dimension of latent semantic space is measured, is shown so as to obtain its l dimension table on latent semantic space
Vector
Step S108：Decoded using expression of the decoding matrix to test case in latent semantic space, to obtain survey
Classification results of the examination example in original tag space.
Using the optimum decoding matrix D obtained by step S105*, to test case latent semantic space l dimension tables show to
Amount is decoded, and the k dimension tables for returning to original tag space show vectorThat is,
The vector for now obtainingValue is real number value, is needed by given threshold（It is usually taken to be 0.5）By its binaryzation.
Specifically, in each dimension value if it exceeds set threshold value then value is 1, otherwise value is 0, it is real so as to represent test
Label ownership situation of the example in original tag space, that is, value is that the label corresponding to 1 each dimension is as to be sorted
The multi-tag classification results of test case.
In one embodiment of the invention, each dimension of latent semantic space is mutually orthogonal, minimizes dimensionality reduction result
In redundancy so that this method can encode the information in more original tag spaces with lower dimension, protect
There is bigger compression ratio to original tag space in having demonstrate,proved reduction process.
In one embodiment of the invention, dimension of the dimension of latent semantic space less than original tag space, and not
Need to pre-set explicit coding function, so as to ensure the forecast model needed for the method for the embodiment of the present invention quantity significantly
Degree is reduced, and effectively reduces training cost so that adaptively can learn the drop of optimum under different data scenes
Dimension result, stability are more preferable.
For example, by the experiment on the standard data set delicious in text classification field, demonstrate enforcement of the present invention
The effectiveness of the Label space dimension reduction method of the feature based correlation implicit expression coding of example.Specifically, by delicious data sets
On Label space dimension fall below original 10%, 20%, 30%, 40% and 50%, and it is real to observe the present invention under different ratios
Apply the classification performance that the method for example can reach, respectively with the accuracy of the mean of average F1 values and Case-based Reasoning based on label come
Weigh（Both of which is that the higher the better）.Table 1 gives the classification performance statistical result of the inventive method, while give also not
When carrying out Label space dimensionality reduction, using the exsertile linear SVM of property（Support vector machine）Can reach
The classification performance for arriving, to carry out the comparison before and after dimensionality reduction.Result from table can be seen that the method for the embodiment of the present invention and exist
Just the 68% of F1 values before non-dimensionality reduction being kept in average F1 values in the case of only retaining original tag Spatial Dimension 10%, while
85% before non-dimensionality reduction is kept on accuracy of the mean.As can be seen here, the method for the embodiment of the present invention can effectively to original tag
Space carries out dimensionality reduction, and while training cost is greatly lowered can ensure acceptable classification performance well.
Experimental result of the method for 1 embodiment of the present invention of table on delicious data sets
| Dimension ratio after dimensionality reduction | 10% | 20% | 30% | 40% | 50% | Before dimensionality reduction |
| Average F1 values based on label | 0.054 | 0.059 | 0.060 | 0.060 | 0.059 | 0.079 |
| The accuracy of the mean of Case-based Reasoning | 0.120 | 0.121 | 0.120 | 0.120 | 0.112 | 0.142 |
The Label space dimension reduction method of feature based correlation implicit expression coding according to embodiments of the present invention, in study dimensionality reduction knot
Also its dependency with the restoration errors of mark matrix and with feature space has been taken into full account during fruit, by optimization
Process ensure that dimensionality reduction result can return to mark matrix well, while the prediction that also can be learnt on feature space
Described by model such that it is able to preferable multi-tag classification performance is obtained under relatively low training cost.
The embodiment of another aspect of the present invention proposes a kind of Label space dimensionality reduction system of feature based correlation implicit expression coding
System, including：Training module 100 and prediction module 200, as shown in Figure 3.
Wherein, training module 100, for carrying out learning training to obtain forecast model according to training dataset.Prediction mould
Block 200, the forecast model for being obtained according to training module 100 obtain classification results of the test case in original tag space.
Specifically, as shown in figure 4, the training module 100 of the embodiment of the present invention is specifically included：Constructing module 10, optimization mould
Block 20, MBM 30 and study module 40.
Wherein, constructing module 100, for according to training dataset structural features matrix and mark matrix.
Specifically, specifically, to the training dataset comprising m test case for giving, according to the attribute of data itself
Suitable characteristic type is selected, and corresponding characteristic vector x=[x is extracted for each of which test case1,x2,...,xd], its
In, xi is the i-th dimension of characteristic vector x.After the characteristic vector for obtaining all test cases, can be with random order by being about to which
Eigenmatrix X needed for being spliced into, X is the matrix of m × d, wherein, d is the dimension of characteristic vector.
At the same time, to the training dataset comprising m test case, count the quantitative value of the different labels for wherein occurring
K, and situation is belonged to according to the label of each test case, it is which constructs corresponding label vector y=[y1,y2,...,yk],
Wherein, yjRepresent whether the example belongs to j-th label.If it is, value is 1, conversely, value is 0, by that analogy.Together
Sample ground, after the label vector for obtaining all test cases, can be spliced into mark matrix Y by which is about to, here with feature square
The splicing sequence consensus of battle array, Y is the matrix of m × k.
Optimization module 20, for the optimum related letter between dimensionality reduction matrix and the eigenmatrix is obtained using eigenmatrix
Number, and the optimum restoration errors function between dimensionality reduction matrix and mark matrix is obtained according to mark matrix.
Specifically, the optimum correlation function of dimensionality reduction matrix and eigenmatrix is on the one hand obtained according to eigenmatrix.
In actual mechanical process, with reference to the method for implicit expression coding, it is assumed that there is dimensionality reduction Matrix C.Dimensionality reduction Matrix C and feature
Dependency between matrix X can be decomposed into the dependency sum between each row of dimensionality reduction Matrix C and eigenmatrix X.For
Any one row c in dimensionality reduction Matrix C, which can be described by cosine dependency with the dependency between eigenmatrix X, table
Procured function form is as follows：
Wherein, r is a Linear Mapping of X, for X to be projected to the space at c places.
At the same time, in order to reduce the redundancy in dimensionality reduction result, it is assumed that each row of dimensionality reduction Matrix C are mutually orthogonal,
That is, each dimension of the latent semantic space of dimensionality reduction Matrix C description is mutually orthogonal, its corresponding mathematic(al) representation is CTC=E.By
CTC=E can learn cTC=1, also, r carries out linear scale and do not affect the value of cosine dependency ρ, thus can construct as
Under majorized function be used to solve Linear Mapping r of optimum, and then obtain the optimal correlation of c and X.
ρ*=maxr(Xr)Tc
Constraints：(Xr)T(Xr)=1
The Linear Mapping of optimum can be drawn by method of Lagrange multipliersAgain substitute into
Optimum dependency can be obtained to above-mentioned majorized function isWherein, Δ=X (XTX)-1XT。
Therefore, the optimal correlation between dimensionality reduction Matrix C and eigenmatrix X can be expressed as following functional expression：
Wherein, C.,iRepresent the i-th row of dimensionality reduction Matrix C.
On the other hand the optimum restoration errors function of dimensionality reduction matrix and mark matrix is obtained according to mark matrix.
With reference to the method for implicit expression coding, it is assumed that there is dimensionality reduction Matrix C, C is the matrix of m × l, wherein, l is the dimension after dimensionality reduction
Degree size, therefore l ＜ ＜ k.
On the premise of it is assumed that dimensionality reduction Matrix C is present, the error for returning to mark matrix by C is can be expressed as such as minor function
Formula：
Wherein, D be to ensure decoding efficiency and the linear codec matrix that introduces,What is represented is matrix
Frobenius normal forms square.In the case where dimensionality reduction Matrix C is given, optimum restoration errors are minimum ε.Therefore, lead to
Cross the restoration errors for minimizing decoding matrix D and optimum that ε can obtain optimum.By constructing following majorized function：
The optimum decoding matrix D of decoding matrix D can be solved*=(CTC)-1CTY, due to CTC=E be unit matrix, D*Can
With further abbreviation as D*=CTY, substitutes into above-mentioned majorized function again, obtains optimum restoration errors function：ε*=Tr[YTY-
YTCCTY], the wherein mark of Tr [] representing matrix, that is, diagonal entry sum.
MBM 30, for according to optimum correlation function and optimum restoration errors construction of function object function, and applies
After objective function optimization dimensionality reduction matrix, decoding matrix is solved using the dimensionality reduction Matrix Calculating after optimization.
Specifically, can obtain from dimensionality reduction Matrix C returning to the optimum restoration errors of mark matrix Y by optimization module 20
Function E*=Tr[YTY-YTCCTY], and its optimum correlation function with eigenmatrix XTherefore it is optimum
Dimensionality reduction matrix should be able to minimize above-mentioned optimum restoration errors function simultaneously and maximize above-mentioned optimum correlation function.
From the property of trace of a matrix, ε*=Tr[YTY-YTCCTY]=Tr[YTY]-Tr[YTCCTY], due to Tr [YTY] it is normal
Number, then minimize ε*It is equivalent to maximize Tr [YTCCTY].Additionally, for optimal correlation part, due to maximizing
It is equivalent to maximizeTherefore, maximize P*It is equivalent to maximizeTherefore, in order to same
When minimize ε*And maximize P*, the following object function of equal value of construction can be passed through, to solve the dimensionality reduction Matrix C of optimum.
Ω=maxCTr[YTCCTY]+αTr[CTΔC]
Constraints：CTC=E
Wherein, α is weight parameter, for adjusting the weight relationship between optimum restoration errors and optimal correlation.According to
The property of trace of a matrix, above-mentioned object function can be rewritten as following form：
Ω=maxCTr[CT(YYT+αΔ)C]
Constraints：CTC=E
By object function Ω, optimum dimensionality reduction Matrix C can be obtained.The solution of optimum dimensionality reduction Matrix C can be decomposed into respectively
The optimization problem of individual row.For dimensionality reduction Matrix C i-th arranges C.,iOptimization Solution, following optimization subproblem can be constructed：
Constraints：
Using method of Lagrange multipliers, it can be deduced that optimum C.,iFollowing optimality condition need to be met：
(YYT+αΔ)C.,i=λiC.,i
Wherein, λiIt is the Lagrange multiplier for introducing, and substitutes into the Ω that above-mentioned optimization subproblem can obtain optimumi,
That is λi。
Can be observed by above-mentioned optimality condition and be obtained, optimum C.,iIt is matrix (YYT+ α Δs) a unit character to
Amount（Eigenvector）, and due to the orthogonality of characteristic vector, orthogonality constraint below can meet naturally.So far can send out
Existing, dimensionality reduction Matrix C is actually by matrix (YYT+ α Δs) in l maximum eigenvalue of correspondence（Eigenvalue）Unit it is special
Levy vector to be spliced by row.Therefore, the process for solving dimensionality reduction Matrix C is to matrix (YYT+ α Δs) carry out Eigenvalues Decomposition
（Eigenvalue decomposition）Process, such that it is able to obtain the matrix all eigenvalues and each eigenvalue pair
The unit character vector answered.As the complexity of Eigenvalues Decomposition is not more thanAnd due to (YYT+ α Δs) it is symmetrical square
Battle array, while only needing the unit character vector corresponding to l maximum eigenvalue in one embodiment of the invention, therefore drops
The complexity of dimension Matrix C can be less thanIt is efficient enough so as to ensure that the process for solving dimensionality reduction Matrix C.
In addition, after dimensionality reduction Matrix C is solved by Eigenvalues Decomposition method, can be by minimizing from dimensionality reduction matrix
C returns to optimum restoration errors between original mark matrix Y to try to achieve optimum decoding matrix D.By the calculating of optimization module 20
Knowable to process, optimum decoding matrix D*=CTY。
Study module 40, for utilizing the dimensionality reduction matrix learning training after optimization to obtain forecast model.
According to the optimum dimensionality reduction Matrix C that MBM 30 is obtained, it is each dimension instruction of the latent semantic space described by which
The corresponding forecast model of white silk.Specifically, for i-th dimension（1≤i≤l）, value of j-th test case in the dimension be
Cj,i, so as to the vector that the value of the training data in the dimension of all test cases is constituted is the i-th row C of C.,i.According to
The eigenmatrix X of the training data of all test cases and value vector C in latent semantic space i-th dimension.,i, can learn
Train corresponding forecast model hi:X→C.,i, for predicting value condition of any example in the dimension, its input is survey
The characteristic vector of examination example, and it is then value of the test case in i-th dimension to export.
In actual operation, the selecting of forecast model can be arranged according to the concrete condition of application, conventional including line
Property return（Linear regression）Deng.Through Label space dimensionality reduction, dimension l of latent semantic space is often much smaller than here
Dimension k in original tag space, so that the quantity of required forecast model is greatly lowered, effectively reduces and is trained to
This.
Additionally, the prediction module 200 of the embodiment of the present invention is specifically included：
（1）When a test case to be sorted is given, need to extract which and training process identical feature, and
To the d dimensional feature vectors of the test case
（2）Obtaining the characteristic vector of test caseAfterwards, the l prediction for being obtained using 100 learning of training module
Model, predicts value of the test case in each dimension of latent semantic space, so as to obtain its l on latent semantic space
Dimension table shows vector
（3）Decoded using expression of the decoding matrix to test case in latent semantic space, to obtain test case
In the classification results in original tag space.
Using the optimum decoding matrix D obtained by optimization module 30*, l dimension table of the test case in latent semantic space is shown
Vector is decoded, and the k dimension tables for returning to original tag space show vectorThat is,
The vector for now obtainingValue is real number value, is needed by given threshold（It is usually taken to be 0.5）By its binaryzation.
Specifically, in each dimension value if it exceeds set threshold value then value is 1, otherwise value is 0, it is real so as to represent test
Label ownership situation of the example in original tag space, that is, value is that the label corresponding to 1 each dimension is as to be sorted
The multi-tag classification results of test case.
In one embodiment of the invention, each dimension of latent semantic space is mutually orthogonal, minimizes dimensionality reduction result
In redundancy so that this method can encode the information in more original tag spaces with lower dimension, protect
There is bigger compression ratio to original tag space in having demonstrate,proved reduction process.
In one embodiment of the invention, dimension of the dimension of latent semantic space less than original tag space, and not
Need to pre-set explicit coding function, so as to ensure that the method for the embodiment of the present invention can be certainly under different data scenes
Adaptively learn the dimensionality reduction result of optimum, stability is more preferable.
The Label space dimensionality reduction system of feature based correlation implicit expression coding according to embodiments of the present invention, in study dimensionality reduction knot
Also its dependency with the restoration errors of mark matrix and with feature space has been taken into full account during fruit, by optimization
Process ensure that dimensionality reduction result can return to mark matrix well, while the prediction that also can be learnt on feature space
Described by model such that it is able to preferable multi-tag classification performance is obtained under relatively low training cost.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
Example ", or the description of " some examples " etc. mean specific features with reference to the embodiment or example description, structure, material or spy
Point is contained at least one embodiment or example of the present invention.In this manual, to the schematic representation of above-mentioned term not
Identical embodiment or example must be directed to.And, the specific features of description, structure, material or feature can be with office
Combined in one or more embodiments or example in an appropriate manner.Additionally, in the case of not conflicting, the skill of this area
The feature of the different embodiments or example described in this specification and different embodiments or example can be tied by art personnel
Close and combine.
Although embodiments of the invention have been shown and described above, it is to be understood that above-described embodiment is example
Property, it is impossible to limitation of the present invention is interpreted as, one of ordinary skill in the art within the scope of the invention can be to above-mentioned
Embodiment is changed, changes, replacing and modification.
Claims (8)
1. the Label space dimension reduction method that a kind of feature based correlation implicit expression is encoded, it is characterised in that comprise the following steps：
Training dataset is provided；
According to the training dataset structural features matrix and mark matrix；
The optimum correlation function of dimensionality reduction matrix and the eigenmatrix is obtained according to the eigenmatrix, and according to the mark square
Battle array obtains the optimum restoration errors function of the dimensionality reduction matrix and the mark matrix, wherein, it is described according to the eigenmatrix
The optimum correlation function that dimensionality reduction matrix is obtained with the eigenmatrix is specifically included：
The mark matrix obtains dimensionality reduction matrix with reference to implicit expression coded method；
Dependency between the dimensionality reduction matrix and the eigenmatrix is resolved into into dependency sum, and passes through cosine dependency
The form of function is expressed as follows：
Wherein, r is a Linear Mapping of eigenmatrix X, for eigenmatrix X is projected to
The space that any one row c is located in dimensionality reduction Matrix C；
Optimal linear mapping r is obtained according to the cosine relevance function, and obtain in the dimensionality reduction Matrix C any one row c with
The optimal correlation of eigenmatrix X；
Linear Mapping r of optimum is obtained by method of Lagrange multipliers*, and according to the optimal linear mapping r*Obtain optimum phase
Close function：Wherein, C·,iRepresent the i-th row of dimensionality reduction Matrix C；
Wherein, the optimum restoration errors function that the dimensionality reduction matrix and the mark matrix are obtained according to the mark matrix
Specifically include：
The mark matrix obtains the dimensionality reduction matrix with reference to implicit expression coding；
The error function expression formula that the dimensionality reduction matrix is returned to the mark matrix is as follows：
Wherein, when ε is minimum, restoration errors are minimum, and D is to ensure decoding efficiency and the linear codec matrix that introduces,Table
What is shown be the Frobenius normal forms of matrix square；
Optimum restoration errors function is obtained by minimizing ε, expression formula is as follows：
According to the optimum correlation function and the optimum restoration errors construction of function object function；
Using dimensionality reduction matrix described in the objective function optimization, and decoding matrix is solved according to the dimensionality reduction Matrix Calculating after optimization；
Using the dimensionality reduction matrix learning training after the optimization obtaining forecast model；
Test case feature is extracted, and expression of the test case in latent semantic space is predicted using the forecast model；
And
Decoded using expression of the decoding matrix to the test case in the latent semantic space, it is described to obtain
Classification results of the test case in original tag space.
2. method according to claim 1, it is characterised in that each dimension of the latent semantic space is mutually orthogonal.
3. method according to claim 1, it is characterised in that classification of the test case in original tag space is tied
Fruit carries out binary conversion treatment.
4. method according to claim 1, it is characterised in that the dimension of the latent semantic space is less than the original tag
The dimension in space.
5. the Label space dimensionality reduction system that a kind of feature based correlation implicit expression is encoded, it is characterised in that include：
Training module, for carrying out learning training to obtain forecast model according to training dataset, wherein, the training module tool
Body includes：
Constructing module, for according to training data structural features matrix and mark matrix；
Optimization module, for the optimum correlation function between dimensionality reduction matrix and the eigenmatrix is obtained according to the eigenmatrix,
And the optimum restoration errors function between dimensionality reduction matrix and the mark matrix is obtained according to the mark matrix, wherein, it is described
Dimensionality reduction matrix is obtained according to the eigenmatrix to specifically include with the optimum correlation function of the eigenmatrix：
The mark matrix obtains dimensionality reduction matrix with reference to implicit expression coded method；
Dependency between the dimensionality reduction matrix and the eigenmatrix is resolved into into dependency sum, and passes through cosine dependency
The form of function is expressed as follows：
Wherein, r is a Linear Mapping of eigenmatrix X, for eigenmatrix X is projected to
The space that any one row c is located in dimensionality reduction Matrix C；
Optimal linear mapping r is obtained according to the cosine relevance function, and obtain in the dimensionality reduction Matrix C any one row c with
The optimal correlation of eigenmatrix X；
Linear Mapping r of optimum is obtained by method of Lagrange multipliers*, and according to the optimal linear mapping r*Obtain optimum phase
Close function：Wherein, C.,iRepresent the i-th row of dimensionality reduction Matrix C；
Wherein, the optimum restoration errors function that the dimensionality reduction matrix and the mark matrix are obtained according to the mark matrix
Specifically include：
The mark matrix obtains the dimensionality reduction matrix with reference to implicit expression coding；
The error function expression formula that the dimensionality reduction matrix is returned to the mark matrix is as follows：
Wherein, when ε is minimum, restoration errors are minimum, and D is to ensure decoding efficiency and the linear codec matrix that introduces,Table
What is shown be the Frobenius normal forms of matrix square；
Optimum restoration errors function is obtained by minimizing ε, expression formula is as follows：
MBM, for according to the optimum correlation function and the optimum restoration errors construction of function object function, and should
After with dimensionality reduction matrix described in the objective function optimization, decoding matrix is solved using the dimensionality reduction Matrix Calculating after optimization；
Study module, for using the dimensionality reduction matrix learning training after the optimization obtaining forecast model；
Prediction module, for obtaining classification results of the test case in original tag space according to the forecast model.
6. system according to claim 5, it is characterised in that each dimension of latent semantic space is mutually orthogonal.
7. system according to claim 5, it is characterised in that classification of the test case in original tag space is tied
Fruit carries out binary conversion treatment.
8. system according to claim 5, it is characterised in that the dimension of latent semantic space is less than the original tag space
Dimension.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410024964.XA CN103761532B (en) | 2014-01-20 | 2014-01-20 | Label space dimensionality reducing method and system based on feature-related implicit coding |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410024964.XA CN103761532B (en) | 2014-01-20 | 2014-01-20 | Label space dimensionality reducing method and system based on feature-related implicit coding |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN103761532A CN103761532A (en) | 2014-04-30 |
| CN103761532B true CN103761532B (en) | 2017-04-19 |
Family
ID=50528767
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410024964.XA Active CN103761532B (en) | 2014-01-20 | 2014-01-20 | Label space dimensionality reducing method and system based on feature-related implicit coding |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103761532B (en) |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106952293B (en) * | 2016-12-26 | 2020-02-28 | 北京影谱科技股份有限公司 | Target tracking method based on nonparametric online clustering |
| CN109036553B (en) * | 2018-08-01 | 2022-03-29 | 北京理工大学 | Disease prediction method based on automatic extraction of medical expert knowledge |
| CN111967501B (en) * | 2020-07-22 | 2023-11-17 | 中国科学院国家空间科学中心 | Method and system for judging load state driven by telemetering original data |
| CN114510518B (en) * | 2022-04-15 | 2022-07-12 | 北京快立方科技有限公司 | Self-adaptive aggregation method and system for massive structured data and electronic equipment |
| US20230409678A1 (en) * | 2022-06-21 | 2023-12-21 | Lemon Inc. | Sample processing based on label mapping |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8189905B2 (en) * | 2007-07-11 | 2012-05-29 | Behavioral Recognition Systems, Inc. | Cognitive model for a machine-learning engine in a video analysis system |
| CN102004774B (en) * | 2010-11-16 | 2012-11-14 | 清华大学 | Personalized user tag modeling and recommendation method based on unified probability model |
| CN102982344B (en) * | 2012-11-12 | 2015-12-16 | 浙江大学 | Based on the support vector machine classification method merging various visual angles feature and many label informations simultaneously |
| CN103176961B (en) * | 2013-03-05 | 2017-02-08 | 哈尔滨工程大学 | Transfer learning method based on latent semantic analysis |
| CN103514456B (en) * | 2013-06-30 | 2017-04-12 | 安科智慧城市技术（中国）有限公司 | Image classification method and device based on compressed sensing multi-core learning |
-
2014
- 2014-01-20 CN CN201410024964.XA patent/CN103761532B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN103761532A (en) | 2014-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103761532B (en) | Label space dimensionality reducing method and system based on feature-related implicit coding | |
| US8787682B2 (en) | Fast image classification by vocabulary tree based image retrieval | |
| Zhang et al. | Active semi-supervised learning based on self-expressive correlation with generative adversarial networks | |
| US20110270604A1 (en) | Systems and methods for semi-supervised relationship extraction | |
| CN112733866B (en) | Network construction method for improving text description correctness of controllable image | |
| CN109948735B (en) | Multi-label classification method, system, device and storage medium | |
| CN112199462A (en) | Cross-modal data processing method and device, storage medium and electronic device | |
| CN109284372A (en) | User's operation behavior analysis method, electronic device and computer readable storage medium | |
| CN113378970B (en) | Sentence similarity detection method and device, electronic equipment and storage medium | |
| CN115146488B (en) | Variable business process intelligent modeling system and method based on big data | |
| CN112711660A (en) | Construction method of text classification sample and training method of text classification model | |
| CN113657425A (en) | Multi-label image classification method based on multi-scale and cross-modal attention mechanism | |
| CN110046356A (en) | Label is embedded in the application study in the classification of microblogging text mood multi-tag | |
| CN111582506A (en) | Multi-label learning method based on global and local label relation | |
| CN113627530A (en) | Similar problem text generation method, device, equipment and medium | |
| CN114925702A (en) | Text similarity recognition method and device, electronic equipment and storage medium | |
| CN113240033B (en) | Visual relation detection method and device based on scene graph high-order semantic structure | |
| CN114399775A (en) | Document title generation method, device, equipment and storage medium | |
| CN113255767A (en) | Bill classification method, device, equipment and storage medium | |
| CN112328655A (en) | Text label mining method, device, equipment and storage medium | |
| CN115456680A (en) | Advertisement click prediction method based on cross feature extraction model and related equipment thereof | |
| CN115098707A (en) | Cross-modal Hash retrieval method and system based on zero sample learning | |
| CN115116557A (en) | Method and related device for predicting molecular label | |
| CN112784838A (en) | Hamming OCR recognition method based on locality sensitive hashing network | |
| Sun et al. | Multi-label active learning with error correcting output codes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |