CN105335756A - Robust learning model and image classification system - Google Patents

Abstract

The invention discloses a robust learning model and an image classification system. The robust learning model comprises the steps that a training set is initialized so that an initial category tag matrix is obtained, and training samples in the training set include the samples of which the categories are known with calibration of category tags corresponding to the categories and the samples of which the categories are unknown without calibration of the category tags; the training samples are processed by a construction method based on neighboring definition and reconstruction weights, a reconstruction coefficient matrix is constructed according to similarity between the samples, and symmetrization and normalization processing is performed; soft tags without calibration samples are determined by utilizing the reconstruction coefficient matrix and the initial category tag matrix, and l2,1 normal regularization is performed on the soft tags of the training samples by adopting an iteration method so that a projection matrix and a soft tag matrix are obtained; mapping is performed on samples under test by utilizing the projection matrix so that the soft tags of the samples are obtained; and the samples under test are the samples of which the categories are unknown without calibration of the categories. Influence of mixed signals in an original space can be effectively reduced by the model so that classification accuracy can be enhanced.

Description

A kind of Robust Learning model and image classification system

Technical field

The present invention relates to pattern-recognition and data mining technology field, more particularly, relate to a kind of Robust Learning model and image classification system.

Background technology

Along with computer technology and intelligentized development, Image Classfication Technology has developed into one of most important research topic in the field such as data mining, machine learning.Sorting technique belonging to Image Classfication Technology is mainly used to the classification of the data judging unknown classification, all be of great importance in fields such as Analysis of Medical Treatment Data, text, webpage and credit card gradings, therefore, accurate sorting technique is come into operation can bring huge social and economic effects.Multinomial research proves that the performance of supervision type learning method is obviously better than without supervision type learning method, but in real world, monitoring data for supervising type learning method is often difficult to obtain, and obtain the classification information of monitoring data by the time of at substantial and manpower by artificial mode of demarcating, thus its practicality is reduced greatly.Thus, semi-supervised learning based on similar diagram structure develops into practicality and one of general classification tool already because of its practicality and classification accuracy, semi-supervised type study is mainly through demarcating the classification of low volume data in each classification in mass data, again it is propagated to the data of unknown classification by similar diagram, and then dope the classification of data of unknown classification.

In recent years, in semi-supervised learning based on the label Law of Communication of label communication theory because of its simply, advantage effectively and rapidly, one of Typical Representative becoming semi-supervised learning.Label Law of Communication is by learning the similarity between sample, by supervision to the classification information of exemplar (i.e. the sample of its classification known) that has propagate to without the exemplar sample of its classification (i.e. the unknown), and then estimate the classification information without exemplar.Label Law of Communication all adopts soft label as the classification results without exemplar mostly at present, but inventor finds, at the soft label of employing as in the label Law of Communication of the classification results without exemplar, the luv space having exemplar corresponding is often containing mixed signal, and these mixed signals can cause harmful effect to the classification without exemplar, and then the classification results without exemplar is caused to be inaccurate.

In sum, the impact existed in prior art due to the mixed signal of luv space causes without the lower problem of the classification results accuracy of exemplar.

Summary of the invention

The object of this invention is to provide a kind of Robust Learning model and image classification system, cause without the lower problem of the classification results accuracy of exemplar with the impact solving the mixed signal due to luv space existed in prior art.

To achieve these goals, the invention provides following technical scheme:

A kind of Robust Learning model, comprising:

Initialization is carried out to the training set obtained in advance, obtain initial category label matrix, wherein, described training set comprises the training sample of predetermined amount, and described training sample comprises its classification known and demarcates has the sample of the class label corresponding with its classification and its classification unknown and the sample not demarcating class label;

Building method based on neighbour's definition and reconstruct power processes described training sample, obtains the similarity measure matrix corresponding with described training sample, carries out presetting process, obtain reconstruction coefficients matrix to described similarity measure matrix;

Based on described reconstruction coefficients matrix and described initial category label matrix, determined the soft label of the training sample not demarcating class label by the level and smooth item of active balance stream shape and label matching item, adopt the soft label of mode to described training sample of iteration to carry out l _2,1norm regularization, obtains projection matrix and the soft label matrix corresponding with the soft label of described training sample;

Utilize described projection matrix to map sample to be tested, obtain the soft label of described sample to be tested; Wherein, described sample to be tested is its classification unknown and does not demarcate the sample of class label.

Preferably, the described building method based on neighbour's definition and reconstruct power processes described training sample, obtains the similarity measure matrix corresponding with described training sample, comprising:

Utilize k nearest neighbor algorithm to process each described training sample, obtain K nearest samples of each training sample, K is positive integer;

The building method of LLE-reconstruct power is adopted to utilize K nearest samples of described training sample and each described training sample to obtain the similarity measure matrix corresponding with described training sample.

Preferably, described to described similarity measure matrix carry out preset process, obtain reconstruction coefficients matrix, comprising:

Described similarity measure matrix is normalized and symmetrization process, obtains described reconstruction coefficients matrix.

Preferably, utilize described projection matrix to map sample to be tested, obtain the soft label of described sample to be tested, comprising:

Obtain sample to be tested x _new;

Utilize P ^tx _newby described sample to be tested x _newbe embedded in projection matrix P, obtain predicted vector;

Determine that the soft label that the element of maximum probability in described predicted vector is corresponding is described sample to be tested x _newsoft label.

Preferably, the described training set to obtaining in advance carries out initialization, obtains initial category label matrix, comprising:

Obtain initialization matrix, use Y ₀=[y ₁, y ₂..., y _l+u] represent, described training set is used represent, wherein, represent that n is multiplied by the space of matrices of l+u, X _lfor demarcating the sample having class label, X _l=[x ₁, x ₂..., x _l], X _ufor not demarcating the sample of class label, X _u=[x _l+1, x _l+2..., x _l+u], n, l and u are positive integer;

Determine the arbitrary y in described initialization matrix _ibe a column vector, i-th training sample x in the corresponding described training set of arbitrary column vector _i, i=1,2 ..., l+u;

Arbitrary demarcation is had to the training sample x of class label _jif, this training sample x _jbelong to the i-th classification, then determine y _{i, j}=1, if this training sample x _jdo not belong to the i-th classification, then determine y _{i, j}=0; For arbitrary training sample x not demarcating class label _j, determine y _{i, j}=0, obtain initial category label matrix Y, wherein, and j=1,2 ..., l+u.

Preferably, the described building method based on neighbour's definition and reconstruct power processes described training sample, obtains the similarity measure matrix corresponding with described training sample, comprising:

Following formula is utilized to obtain described similarity measure matrix:

Wherein, represent training sample x _ik nearest samples, q and K is corresponding, for current training sample x _ireconstruction coefficient vector, characterize its neighbour at reconstruct training sample x _itime percentage contribution, represent corresponding similarity reconstruction coefficients matrix.

Preferably, the soft label of mode to described training sample of described employing iteration carries out l _2,1norm regularization, obtains projection matrix and the soft label matrix corresponding with the soft label of described training sample, comprising:

Following formula is utilized to obtain projection matrix and soft label matrix:

subjf _i≥0,e ^Tf _i＝1fori＝1,2,...,l+u

Wherein, F represents soft label matrix, and P represents projection matrix, and W represents reconstruction coefficients matrix, and D represents diagonal matrix, and D _ii=Σ _jw _i,j, f ⁱrepresent the i-th row vector, X ^tp-F ^trepresent and return discrepance, described recurrence discrepance is used for weighing X ^tp and soft label F ^tdifference degree; μ _i, α, beta, gamma is corresponding balance parameter;

Or, utilize following formula to obtain projection matrix and soft label matrix:

$<F,P>=\arg \underset{F,P}{min}tr\left({\mathrm{F\&Omega;F}}^T\right)+tr\left(\right(F-Y\left)UD{\left(F-Y\right)}^T\right)+\mathrm{\&alpha;}\left|\right|F^T|{|}_{2,1}+\mathrm{\&beta;}\left(\right|\left|P\right|{|}_{2,1}+\mathrm{\&gamma;}\left|\right|X^{T}P-F^T\left|{|}_{2,1}\right)$

subjF≥0,e ^TF＝e ^T

Wherein, Ω=I-W ^t-W+W ^tw, I are unit matrix, and it only has diagonal element non-zero and is 1; U represents a diagonal matrix, and its i-th diagonal element is μ _i, for demarcating the training sample having label, μ _ibe set to+∞, for the training sample not demarcating class label, μ _ibe set to 0.

A kind of image classification system, comprising:

Pretreatment module, for carrying out initialization to the training set obtained in advance, obtain initial category label matrix, wherein, described training set comprises the training sample of predetermined amount, and described training sample comprises its classification known and demarcates has the sample of the class label corresponding with its classification and its classification unknown and the sample not demarcating class label; And for processing described training sample based on the building method of neighbour's definition and reconstruct power, obtain the similarity measure matrix corresponding with described training sample, carry out presetting process to described similarity measure matrix, obtain reconstruction coefficients matrix;

Training module, for based on described reconstruction coefficients matrix and described initial category label matrix, determined the soft label of the training sample not demarcating class label by the level and smooth item of active balance stream shape and label matching item, adopt the soft label of mode to described training sample of iteration to carry out l _2,1norm regularization, obtains projection matrix and the soft label matrix corresponding with the soft label of described training sample;

Prediction module, for utilizing described projection matrix to map sample to be tested, obtains the soft label of described sample to be tested; Wherein, described sample to be tested is the sample not demarcating class label.

Preferably, described pretreatment module comprises:

K nearest neighbor unit, for utilizing k nearest neighbor algorithm to process each described training sample, obtain K nearest samples of each training sample, K is positive integer;

LLE-reconstruct power unit, the building method for adopting LLE-to reconstruct power utilizes K nearest samples of each described training sample to obtain the similarity measure matrix corresponding with described training sample.

Preferably, described pretreatment module comprises:

Presetting processing unit, for being normalized and symmetrization process described similarity measure matrix, obtaining described reconstruction coefficients matrix.

A kind of Robust Learning model provided by the invention and image classification system, comprise: initialization is carried out to the training set obtained in advance, obtain initial category label matrix, wherein, described training set comprises the training sample of predetermined amount, and described training sample comprises its classification known and demarcates has the sample of the class label corresponding with its classification and its classification unknown and the sample not demarcating class label; Building method based on neighbour's definition and reconstruct power processes described training sample, obtains the similarity measure matrix corresponding with described training sample, carries out presetting process, obtain reconstruction coefficients matrix to described similarity measure matrix; Based on described reconstruction coefficients matrix and described initial category label matrix, determined the soft label of the training sample not demarcating class label by the level and smooth item of active balance stream shape and label matching item, adopt the soft label of mode to described training sample of iteration to carry out l _2,1norm regularization, obtains projection matrix and the soft label matrix corresponding with the soft label of described training sample; Utilize described projection matrix to map sample to be tested, obtain the soft label of described sample to be tested; Wherein, described sample to be tested is its classification unknown and does not demarcate the sample of class label.

Compared with prior art, first utilize in the application and demarcate the training sample having the training sample of class label and do not demarcate class label and jointly construct similarity measure matrix, obtain reconstruction coefficients matrix further, and initialization obtains initial category label matrix, initial category label matrix and reconstruction coefficients matrix is utilized to determine not demarcate the soft label of the training sample of class label, by carrying out l to the soft label of training sample _2,1norm regularization, effectively reduce soft label formed luv space in mixed signal on the impact of the accuracy of classification results; By based on l _2,1the tolerance of norm regularization, effectively enhances the robustness embedding the regression residuals between feature and soft label and measure; By calculating based on l _2,1the mapping of the tolerance of norm regularization, enhances and extracts the descriptive of feature.In a word, by introducing l _2,1norm regularization technology, effectively improve the robustness of system for the mixed signal such as noise and heterogeneous data in training sample, the impact avoiding the mixed signal due to luv space mentioned in background technology causes the generation without the lower situation of the classification results accuracy of exemplar, namely, decrease the impact of the mixed signal of luv space, improve the accuracy of the classification results of sample to be tested, enhance classification performance.In addition, the application is the outer Rate Based On The Extended Creep Model of sample, and by embedding the conclusion and prediction that efficiently complete the outer data of sample, without the need to introducing extra restructuring procedure, expansibility can be good.

Accompanying drawing explanation

In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only embodiments of the invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to the accompanying drawing provided.

The process flow diagram of a kind of Robust Learning model that Fig. 1 provides for the embodiment of the present invention;

The structural representation of a kind of image classification system that Fig. 2 provides for the embodiment of the present invention;

The Tag Estimation schematic diagram of the classification of facial image test sample book is obtained in a kind of Robust Learning model that Fig. 3 provides for the embodiment of the present invention and image classification system.

Embodiment

Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.

Refer to Fig. 1, it illustrates the process flow diagram of a kind of Robust Learning model that the embodiment of the present invention provides, can comprise the following steps:

S11: initialization is carried out to the training set obtained in advance, obtain initial category label matrix, wherein, training set comprises the training sample of predetermined amount, and training sample comprises its classification known and demarcates has the sample of the class label corresponding with its classification and its classification unknown and the sample not demarcating class label.

Wherein, in training set, the quantity of training sample can be determined according to actual needs.The class label of training sample is corresponding with the classification of training sample, for the training sample of its classification unknown, it is not carried out to the demarcation of class label, for the training sample of its classification known, utilize the class label corresponding with its classification to demarcate it.

S12: the building method based on neighbour's definition and reconstruct power processes training sample, obtains the similarity measure matrix corresponding with training sample, carries out presetting process, obtain reconstruction coefficients matrix to similarity measure matrix.

It should be noted that, step S11 and step S12 can complete simultaneously, also can be first to complete S11 to complete S12 again, or first completes S12 and complete S11 again, specifically can determine according to actual needs.

S13: based on reconstruction coefficients matrix and initial category label matrix, determines the soft label of the training sample not demarcating class label by the level and smooth item of active balance stream shape and label matching item, adopt the soft label of mode to training sample of iteration to carry out l _2,1norm regularization, obtains projection matrix and the soft label matrix corresponding with the soft label of training sample.

Can be specifically determine the level and smooth item of stream shape based on reconstruction coefficients matrix, based on initial category label matrix determination label matching item, and then determine the soft label corresponding with the stream level and smooth item of shape and label matching item.

The soft label of each training sample is also corresponding with its classification, and soft label matrix is determined according to the soft label of each training sample.The classification of whole training sample can be known by the soft label matrix obtained.

And l _2,1norm regularization is means important in machine learning, in support vector machine learning process, actual is a kind of process cost function being solved to optimum, it by adding norm in cost function, make the result learning to obtain meet rarefaction (sparsity), thus facilitate the mankind to extract feature.

S14: utilize projection matrix to map sample to be tested, obtains the soft label of sample to be tested; Wherein, sample to be tested is its classification unknown and does not demarcate the sample of class label.

Utilize projection matrix to map sample to be tested, obtain the soft label of sample to be tested, determine its classification further.

First utilize in the application and demarcate the training sample having the training sample of class label and do not demarcate class label and jointly construct similarity measure matrix, obtain reconstruction coefficients matrix further, and initialization obtains initial category label matrix, initial category label matrix and reconstruction coefficients matrix is utilized to determine not demarcate the soft label of the training sample of class label, by carrying out l to the soft label of training sample _2,1norm regularization, effectively reduce soft label formed luv space in mixed signal on the impact of the accuracy of classification results; By based on l _2,1the tolerance of norm regularization, effectively enhances the robustness embedding the regression residuals between feature and soft label and measure; By calculating based on l _2,1the mapping of the tolerance of norm regularization, enhances and extracts the descriptive of feature.In a word, by introducing l _2,1norm regularization technology, effectively improve the robustness of system for the mixed signal such as noise and heterogeneous data in training sample, the impact avoiding the mixed signal due to luv space mentioned in background technology causes the generation without the lower situation of the classification results accuracy of exemplar, namely, decrease the impact of the mixed signal of luv space, improve the accuracy of the classification results of sample to be tested, enhance classification performance.In addition, the application is the outer Rate Based On The Extended Creep Model of sample, and by embedding the conclusion and prediction that efficiently complete the outer data of sample, without the need to introducing extra restructuring procedure, expansibility can be good.

In a kind of Robust Learning model that above-described embodiment provides, initialization is carried out to the training set obtained in advance, obtains initial category label matrix, can comprise:

Obtain initialization matrix, use Y ₀=[y ₁, y ₂..., y _l+u] represent, training set is used represent, wherein, represent that n is multiplied by the space of matrices of l+u, X _lfor demarcating the sample of class label, X _l=[x ₁, x ₂..., x _l], X _ufor not demarcating the sample of class label, X _u=[x _l+1, x _l+2..., x _l+u], n, l and u are positive integer;

Determine the arbitrary y in initialization matrix _ibe a column vector, i-th training sample x in the corresponding training set of arbitrary column vector _i, i=1,2 ..., l+u;

Arbitrary demarcation is had to the training sample x of class label _jif, this training sample x _jbelong to the i-th classification, then determine y _{i, j}=1, if this training sample x _jdo not belong to the i-th classification, then determine y _{i, j}=0; For arbitrary training sample x not demarcating class label _j, determine y _{i, j}=0, obtain initial category label matrix Y, wherein, and j=1,2 ..., l+u.

Thus, by above-mentioned steps, based on the class label of each training sample, training sample is changed into the initial category label matrix of the classification that can indicate each training sample, to ensure the smooth realization of subsequent step.

In a kind of Robust Learning model that above-described embodiment provides, the building method based on neighbour's definition and reconstruct power processes training sample, obtains the similarity measure matrix corresponding with training sample, can comprise:

To the training set comprising c class training sample, utilize k nearest neighbor algorithm to process each training sample, obtain K nearest samples of each training sample, K is positive integer;

The building method of LLE-reconstruct power is adopted to utilize the K of training sample and each training sample nearest samples to obtain the similarity measure matrix corresponding with training sample.

Wherein, the thinking of k nearest neighbor algorithm is: if the great majority in the sample of K (namely the most contiguous in feature space) the most similar of a sample in feature space belong to some classifications, then this sample also belongs to this classification.

And the building method of LLE-reconstruct power is the optimization method for non-linear signal features vector dimension, this dimension optimizing is not only quantitatively simple yojan, but when keeping raw data character constant, by the signal map of higher dimensional space on lower dimensional space, i.e. the second extraction of eigenwert.

Concrete, the building method based on neighbour's definition and reconstruct power processes training sample, obtains the similarity measure matrix corresponding with training sample, can comprise:

Following formula is utilized to obtain similarity measure matrix:

Wherein, represent training sample x _ik nearest samples, q and K is corresponding, for current training sample x _ireconstruction coefficient vector, characterize its neighbour at reconstruct training sample x _itime percentage contribution, represent similarity measurements moment matrix.And similarity measurements moment matrix obtain by repeating the above-mentioned step reconstructing the building method weighed to k nearest neighbor algorithm and LLE-relevant to all training samples.

It should be noted that, the building method of LLE-reconstruct power is adopted to utilize the K of training sample and each training sample nearest samples to obtain the similarity measure matrix corresponding with training sample, be specially, adopt the building method of LLE-reconstruct power, calculate, weigh similarity between each summit, the similarity measure matrix of structure similar neighborhoods figure.

In a kind of Robust Learning model that above-described embodiment provides, carry out presetting process to similarity measure matrix, obtain reconstruction coefficients matrix, can comprise:

Similarity measure matrix is normalized and symmetrization process, obtains reconstruction coefficients matrix.

And utilize projection matrix to map sample to be tested, obtain the soft label of sample to be tested, can comprise:

Obtain sample to be tested x _new;

Utilize P ^tx _newby sample to be tested x _newbe embedded in projection matrix P, obtain predicted vector;

Determine that the soft label that in predicted vector, the element of maximum probability is corresponding is sample to be tested x _newsoft label.

Wherein, projection matrix is map classification device, by sample to be tested being embedded in map classification device, predicted vector can be obtained, be the soft label matrix corresponding with sample to be tested, determine that the soft label of the position that the element of maximum probability in soft label matrix is corresponding is the soft label of sample to be tested, and the hard class label of sample to be tested can be summed up as argmax _i≤c(f _new) _i, wherein, represent the predicted vector f of prediction _newi-th element position.

By above-mentioned steps, can according to projection matrix obtain sample to be tested likely belonging to classification, and using the classification of the classification of maximum probability as sample to be tested, ensure that the accuracy of classification results.

Suppose the soft label matrix predicted, namely initialization matrix is F=[f ₁, f ₂... f _l+u], wherein training sample x _jclassification and each arrange f _jmiddle maximal term f _{i, j}position be associated; In a kind of Robust Learning model that above-described embodiment provides, the mode of iteration is adopted to carry out l to soft label _2,1norm regularization, obtains projection matrix and the soft label matrix corresponding with the soft label of training sample, can comprise:

Following formula is utilized to obtain projection matrix and soft label matrix:

subjf _i≥0,e ^Tf _i＝1fori＝1,2,...,l+u

Wherein, F represents soft label matrix, and P represents projection matrix, and W represents reconstruction coefficients matrix, and D represents diagonal matrix, and D _ii=Σ _jw _i,j, f ⁱrepresent the i-th row vector, X ^tp-F ^trepresent and return discrepance, return discrepance and be used for weighing X ^tp and soft label F ^tdifference degree; μ _i, α, beta, gamma is the balance parameter corresponding to above-mentioned every data; Can add row and be 1 constraint condition, i.e. e ^tf _i=1 and Condition of Non-Negative Constrains, i.e. f _i>=0, to ensure in the soft label classification matrix F that exports that each training sample distribution probability meets probability and is 1 and for the distribution probability logic (e is the column vector that an element is 1) that is positive.And projection matrix P, namely map classification device can be expressed as:

And in actual iterative process, usually utilize following formula to obtain projection matrix and soft label matrix:

$<F,P>=\arg \underset{F,P}{min}tr\left({\mathrm{F\&Omega;F}}^T\right)+tr\left(\right(F-Y\left)UD{\left(F-Y\right)}^T\right)+\mathrm{\&alpha;}\left|\right|F^T|{|}_{2,1}+\mathrm{\&beta;}\left(\right|\left|P\right|{|}_{2,1}+\mathrm{\&gamma;}\left|\right|X^{T}P-F^T\left|{|}_{2,1}\right)$

subjF≥0,e ^TF＝e ^T

Wherein, Ω=I-W ^t-W+W ^tw, I are unit matrix, and it only has diagonal element non-zero and is 1; U represents a diagonal matrix, and its i-th diagonal element is μ _i, for demarcating the training sample having label, μ _ibe set to+∞, for the training sample not demarcating class label, μ _ibe set to 0.And the l of projection matrix P _2,1norm, i.e. ‖ P ‖ _2,1can make many row vanishing of projection matrix P, thus in reduction luv space, mixed signal, on the impact of soft label, guarantees that projection matrix P is sparse.

It should be noted that, in actual iterative process, on the basis of above-mentioned formula, can further formula be rewritten as:

Wherein, G, H, V are diagonal matrix, and:

$g_{ii}=\frac{1}{2}\left|\right|p^i\left|\right|,i=1,2,...,l+u$

$h_{ii}=\frac{1}{2}\left|\right|{\hat{f}}^i\left|\right|,i=1,2,...,l+u$

$v_{ii}=\frac{1}{2}\left|\right|r^i\left|\right|,i=1,2,...,l+u$

Wherein, r ⁱr=X ^tp-F ^tthe i-th row vector.

Pass through 0 is set to P partial derivative, following formula can be obtained:

Wherein, $P_V^G=\mathrm{\&gamma;}{(\mathrm{\&gamma;}-{\mathrm{XVX}}^T+G)}^{-1}.$

Pass through 0 is set to F partial derivative, following formula can be obtained:

Wherein,

After each iteration upgrades, initial labels matrix Y, diagonal matrix D, soft label matrix F, and diagonal matrix H, V is updated to:

$F^{k+1}=\&lsqb;F_l^{k+1},F_u^{k+1}\&rsqb;,Y=\&lsqb;Y_l,Y_u\&rsqb;,D=\left[\begin{array}{cc}{D}_l& 0\\ 0& D_u\end{array}\right],H^k=\left[\begin{array}{cc}{H}_l^k& 0\\ 0& H_u^k\end{array}\right],V^k=\left[\begin{array}{cc}{V}_l^k& 0\\ 0& V_u^k\end{array}\right]$

Wherein, d _l, d _u, due to and Y _lbe respectively the original soft label of the prediction of demarcating the training sample of class label and mark matrix, and Y _ufor the original soft label of the prediction of not demarcating the training sample of class label and mark matrix, above-mentioned formula can be rewritten as:

$\begin{array}{c}[F_l^{k+1},F_u^{k+1}]\left[\begin{array}{cc}{\mathrm{\&Omega;}}_{\mathrm{ll}}+\mathrm{\&alpha;}{H}_l^k+U_l{D}_l+\mathrm{\&beta;\&gamma;}{V}_l^k& {\mathrm{\&Omega;}}_{\mathrm{lu}}\\ {\mathrm{\&Omega;}}_{\mathrm{ul}}& {\mathrm{\&Omega;}}_{\mathrm{uu}}+\mathrm{\&alpha;}{H}_u^k+U_u{D}_u+\mathrm{\&beta;\&gamma;}{V}_u^k\end{array}\right]\\ -[Y_l,Y_u]\left[\begin{array}{cc}{U}_l{D}_l& 0\\ 0& U_u{D}_u\end{array}\right]-\mathrm{\&beta;\&gamma;}[{\left(P^k\right)}^T{X}_l,{\left(P^k\right)}^T{X}_u]\left[\begin{array}{cc}{V}_l^k& 0\\ 0& V_u^k\end{array}\right]=0\end{array}$

Wherein, U is according to original having class label and can be divided into 4 parts without the training sample of class label in initial training sample.Above-mentioned formula can be converted into following formula:

$\left\{\begin{array}{c}{F}_u^{k+1}\left({\mathrm{\&Omega;}}_{uu}+{\mathrm{\&alpha;H}}_u^k+U_u{D}_u+{\mathrm{\&beta;\&gamma;V}}_u^k\right)+F_l^{k+1}{\mathrm{\&Omega;}}_{lu}-Y_u{U}_u{D}_u-\mathrm{\&beta;}\mathrm{\&gamma;}{\left(P^k\right)}^T{X}_u{V}_u^k=0\\ F_l^{k+1}\left({\mathrm{\&Omega;}}_{ll}+{\mathrm{\&alpha;H}}_l^k+U_l{D}_l+{\mathrm{\&beta;\&gamma;V}}_l^k\right)+F_u^{k+1}{\mathrm{\&Omega;}}_{ul}-Y_l{U}_l{D}_l-\mathrm{\&beta;}\mathrm{\&gamma;}{\left(P^k\right)}^T{X}_l{V}_l^k=0\end{array}\right.$

After can estimating kth+1 iteration by above-mentioned two equatioies, there is class label in luv space and without the soft tag update of the training sample of class label be:

$\left\{\begin{array}{c}{F}_u^{k+1}=(Y_u{U}_u{D}_u+\mathrm{\&beta;}\mathrm{\&gamma;}{\left(P^k\right)}^T{X}_u{V}_u^k-F_l^{k+1}{\mathrm{\&Omega;}}_{lu}){({\mathrm{\&Omega;}}_{uu}+{\mathrm{\&alpha;H}}_u^k+U_u{D}_u+{\mathrm{\&beta;\&gamma;V}}_u^k)}^{-1}\\ F_l^{k+1}=(Y_l{U}_l{D}_l+\mathrm{\&beta;}\mathrm{\&gamma;}{\left(P^k\right)}^T{X}_l{V}_l^k-F_u^{k+1}{\mathrm{\&Omega;}}_{ul}){({\mathrm{\&Omega;}}_{ll}+{\mathrm{\&alpha;H}}_l^k+U_l{D}_l+{\mathrm{\&beta;\&gamma;V}}_l^k)}^{-1}\end{array}\right.$

Setting ${\mathrm{\&Omega;}}^L={\mathrm{\&Omega;}}_{ll}+{\mathrm{\&alpha;H}}_l^k+U_l{D}_l{\mathrm{\&beta;\&gamma;V}}_l^k,{\mathrm{\&Omega;}}^U={\mathrm{\&Omega;}}_{uu}+U_u{D}_u+{\mathrm{\&beta;\&gamma;V}}_u^k,$ Can obtain by above-mentioned second equation is substituted into first equation:

$\begin{array}{c}{F}_u^{k+1}=\left(Y_u{U}_u{D}_u+\mathrm{\&beta;}\mathrm{\&gamma;}{\left(P^k\right)}^T{X}_u{V}_u^k\right){\left({\mathrm{\&Omega;}}^U\right)}^{-1}\\ -\left(Y_l{U}_l{D}_l+\mathrm{\&beta;}\mathrm{\&gamma;}{\left(P^k\right)}^T{X}_l{V}_l^k-F_u^{k+1}{\mathrm{\&Omega;}}_{ul}\right){\left({\mathrm{\&Omega;}}^L\right)}^{-1}{\mathrm{\&Omega;}}_{lu}{\left({\mathrm{\&Omega;}}^U\right)}^{-1}\end{array}$

Suppose ${\mathrm{\&Omega;}}^A=(Y_u{U}_u{D}_u+{\mathrm{\&beta;\&gamma;P}}^T{X}_u{V}_u^k){\left({\mathrm{\&Omega;}}^U\right)}^{-1},{\mathrm{\&Omega;}}^B={\left({\mathrm{\&Omega;}}^L\right)}^{-1}{\mathrm{\&Omega;}}_{lu}{\left({\mathrm{\&Omega;}}^U\right)}^{-1},$ The soft label of training sample original in current iteration process obtains by following formula:

$\left\{\begin{array}{c}{F}_u^{k+1}=({\mathrm{\&Omega;}}^A-Y_l{U}_l{D}_l{\mathrm{\&Omega;}}^B-\mathrm{\&beta;}\mathrm{\&gamma;}{\left(P^k\right)}^T{X}_l{V}_l^k){(I-{\mathrm{\&Omega;}}_{ul}{\mathrm{\&Omega;}}^B)}^{-1}\\ F_l^{k+1}=(Y_l{U}_l{D}_l+\mathrm{\&beta;}\mathrm{\&gamma;}{\left(P^k\right)}^T{X}_l{V}_l^k-F_u^{k+1}{\mathrm{\&Omega;}}_{ul}){\left({\mathrm{\&Omega;}}^L\right)}^{-1}\end{array}\right.$

Renewable by above-mentioned iterative step with therefore, when original input signal contains noise or even contains error flag, a kind of Robust Learning model provided by the present invention can be upgraded by iterative step thus make identification stronger.By upgrade after with can train and obtain P ^k+1and G ^k+1, with be updated to:

Wherein, be or jth row vector, (r ^j) ^k+1, j=1,2 ..., l is $R_l^{k+1}=X_l^T{P}^{k+1}-{\left(F_l^{k+1}\right)}^T$ Jth row vector, and (r ^j) ^k+1, j=l+1 ..., l+u is $R_u^{k+1}=X_u^T{P}^{k+1}-{\left(F_u^{k+1}\right)}^T$ In row vector.

In actual iterative process, solve for convenience, a kind of Robust Learning model provided by the present invention is established in each iterative process simplifying optimization, therefore, can obtain by addressing the problem

Wherein, if flag data x in original input signal _jstandardization parameter ψ _jbe set to ψ _l, Unlabeled data x _jstandardization parameter ψ _jbe set to ψ _u, work as ψ _lwhen=0, represent identical with original tally to the Tag Estimation result of flag data.I _ψ=(1+U) ^-1, I _ξ=I-I _ψ, respectively by I _ψwith I _ξbe divided into following four parts:

Wherein, with being respectively diagonal element is ψ _lwith 1-ψ _ldiagonal matrix, with being respectively diagonal element is ψ _uwith 1-ψ _udiagonal matrix.If kth+1 iteration can obtain:

Specific algorithm is as follows:

Input:

Raw data matrix initial labels matrix Y, controling parameters ψ, α, beta, gamma, and K.

Export:

1). initialization G ⁰, H ⁰, V ⁰for unit matrix, F ⁰=Y, P ⁰=0, k=0;

2). calculate weight matrix W, structure diagonal matrix D, makes D _ii=Σ _jw _i,j;

3). definition matrix Ω=I-W ^t-W+W ^tw=[Ω _ll, Ω _lu, Ω _ul, Ω _uu].

When not restraining:

Address the problem upgrade soft label matrix F by fixing other values:

Address the problem upgrade projection matrix P by fixing other values:

$F^{k+1}=\&lsqb;Y_l,F_u^{k+1}\&rsqb;,P_V^G=\mathrm{\&gamma;}{({\mathrm{\&gamma;XV}}^k{X}^T+G^k)}^{-1},P^{k+1}={\left(P_V^G\right)}^k{\mathrm{XV}}^k{\left(F^{k+1}\right)}^T$

Upgrading matrix G is:

Calculate $R_l^{k+1}=X_l^T{P}^{k+1}-{\left(F_l^{k+1}\right)}^T,R_u^{k+1}=X_u^T{P}^{k+1}-{\left(F_u^{k+1}\right)}^T,$ Upgraded by the method for above-mentioned renewal H, V and $V^{k+1}=[V_l^{k+1},0;0,V_u^{k+1}];$

If ‖ is F ^k+1-F ^k‖ _f≤ ε, ‖ P ^k+1-P ^k‖ _f≤ ε, end loop; Otherwise K=k+1, continues circulation.

Export:

Optimum solution and optimum projection matrix P ^*=P ^k+1.

Corresponding with said method embodiment, the embodiment of the present invention additionally provides a kind of image classification system, as shown in Figure 2, can comprise:

Pretreatment module 21, for carrying out initialization to the training set obtained in advance, obtain initial category label matrix, wherein, training set comprises the training sample of predetermined amount, and training sample comprises its classification known and demarcates has the sample of the class label corresponding with its classification and its classification unknown and the sample not demarcating class label; And for processing training sample based on the building method of neighbour's definition and reconstruct power, obtain the similarity measure matrix corresponding with training sample, carry out presetting process to similarity measure matrix, obtain reconstruction coefficients matrix;

Training module 22, for based on reconstruction coefficients matrix and initial category label matrix, determines the soft label of the training sample not demarcating class label by the level and smooth item of active balance stream shape and label matching item, adopt the soft label of mode to training sample of iteration to carry out l _2,1norm regularization, obtains projection matrix and the soft label matrix corresponding with the soft label of training sample;

Prediction module 23, for utilizing projection matrix to map sample to be tested, obtains the soft label of sample to be tested; Wherein, sample to be tested is the sample not demarcating class label.

First utilize in the application and demarcate the training sample having the training sample of class label and do not demarcate class label and jointly construct similarity measure matrix, obtain reconstruction coefficients matrix further, and initialization obtains initial category label matrix, initial category label matrix and reconstruction coefficients matrix is utilized to determine not demarcate the soft label of the training sample of class label, by carrying out l to soft label _2,1norm regularization, effectively reduce soft label formed luv space in mixed signal on the impact of the accuracy of classification results; By based on l _2,1the tolerance of norm regularization, effectively enhances the robustness embedding the regression residuals between feature and soft label and measure; By calculating based on l _2,1the mapping of the tolerance of norm regularization, enhances and extracts the descriptive of feature.In a word, by introducing l _2,1norm regularization technology, effectively improve the robustness of system for the mixed signal such as noise and heterogeneous data in training sample, the impact avoiding the mixed signal due to luv space mentioned in background technology causes the generation without the lower situation of the classification results accuracy of exemplar, namely, decrease the impact of the mixed signal of luv space, improve the accuracy of the classification results of sample to be tested, enhance classification performance.In addition, the application is the outer Rate Based On The Extended Creep Model of sample, and by embedding the conclusion and prediction that efficiently complete the outer data of sample, without the need to introducing extra restructuring procedure, expansibility can be good.

In a kind of image classification system that above-described embodiment provides, pretreatment module can comprise:

K nearest neighbor unit, for utilizing k nearest neighbor algorithm to process each training sample, obtain K nearest samples of each training sample, K is positive integer.

LLE-reconstruct power unit, the building method for adopting LLE-to reconstruct power utilizes the K of each training sample nearest samples to obtain the similarity measure matrix corresponding with training sample.

Presetting processing unit, for being normalized and symmetrization process similarity measure matrix, obtaining reconstruction coefficients matrix.

Initialization unit, for: obtain initialization matrix, use Y ₀=[y ₁, y ₂..., y _l+u] represent, training set is used represent, wherein, represent that n is multiplied by the space of matrices of l+u, X _lfor demarcating the sample having class label, X _l=[x ₁, x ₂..., x _l], X _ufor not demarcating the sample of class label, X _u=[x _l+1, x _l+2..., x _l+u], n, l and u are positive integer; Determine the arbitrary y in initialization matrix _ibe a column vector, i-th training sample x in the corresponding training set of arbitrary column vector _i, i=1,2 ..., l+u; Arbitrary demarcation is had to the training sample x of class label _jif, this training sample x _jbelong to the i-th classification, then determine y _{i, j}=1, if this training sample x _jdo not belong to the i-th classification, then determine y _{i, j}=0; For arbitrary training sample x not demarcating class label _j, determine y _{i, j}=0, obtain initial category label matrix Y, wherein, and j=1,2 ..., l+u.

LLE-reconstruct power unit can comprise:

LLE-reconstructs power portion, for: utilize following formula to obtain similarity measure matrix:

Wherein, represent training sample x _ik nearest samples, q and K is corresponding, for current training sample x _ireconstruction coefficient vector, characterize its neighbour at reconstruct training sample x _itime percentage contribution, represent corresponding similarity reconstruction coefficients matrix.

In a kind of image classification system that above-described embodiment provides, prediction module can comprise:

Predicting unit, for: obtain sample to be tested x _new; Utilize P ^tx _newby sample to be tested x _newbe embedded in projection matrix P, obtain predicted vector; Determine that the soft label that in predicted vector, the element of maximum probability is corresponding is sample to be tested x _newsoft label.

In a kind of image classification system that above-described embodiment provides, training module can comprise:

Training unit, for: utilize following formula to obtain projection matrix and soft label matrix:

subjf _i≥0,e ^Tf _i＝1fori＝1,2,...,l+u

Wherein, F represents soft label matrix, and P represents projection matrix, and W represents reconstruction coefficients matrix, and D represents diagonal matrix, and D _ii=Σ _jw _i,j, f ⁱrepresent the i-th row vector, X ^tp-F ^trepresent and return discrepance, return discrepance and be used for weighing X ^tp and soft label F ^tdifference degree; μ _i, α, beta, gamma is corresponding balance parameter;

Or, utilize following formula to obtain projection matrix and soft label matrix:

$<F,P>=\arg \underset{F,P}{min}tr\left({\mathrm{F\&Omega;F}}^T\right)+tr\left(\right(F-Y\left)UD{\left(F-Y\right)}^T\right)+\mathrm{\&alpha;}\left|\right|F^T|{|}_{2,1}+\mathrm{\&beta;}\left(\right|\left|P\right|{|}_{2,1}+\mathrm{\&gamma;}\left|\right|X^{T}P-F^T\left|{|}_{2,1}\right)$

subjF≥0,e ^TF＝e ^T

Wherein, Ω=I-W ^t-W+W ^tw, I are unit matrix, and it only has diagonal element non-zero and is 1; U represents a diagonal matrix, and its i-th diagonal element is μ _i, for demarcating the training sample having label, μ _ibe set to+∞, for the training sample not demarcating class label, μ _ibe set to 0.

For image classification system a kind of disclosed in the embodiment of the present invention, because it is corresponding with Robust Learning model a kind of disclosed in the embodiment of the present invention, therefore, the related content can consulting said method embodiment is illustrated for it.

It should be noted that in addition, a kind of Robust Learning model that the embodiment of the present invention provides 4 True Data set pair embodiment of the present invention is tested, and comprises ExtendedYale-B, ARface, CMUPIEface, CMUposeface.Consider based on calculating high efficiency, the size of all true pictures is compressed to 32x32, therefore the vector of corresponding one 1024 dimension of every pictures in an experiment.In order to method of testing performance, we select the sample of some to form having in training set to demarcate and have the training sample of class label as there being label training sample subset from every class, then the training sample not demarcating class label selecting equal number from every class is as without label training sample subset.These data sets are collected from many aspects, and thus test result has generally illustrative.

Refer to table 1, for a kind of Robust Learning model provided by the present invention and GFHF, LLGC, SLP, LNP, SDA, Lap-LDA, FME and ELP method are at extendedYale-B, ARface, CMUPIE, CMUPose human face data collection test recognition result contrast table, provides average recognition rate and the standard deviation of each methods experiment.This experiment is at the sample composition training set of every class sample selection some, for increasing experiment fairness, the experiment parameter participating in comparative approach is also all carefully chosen, wherein, for class every in training set demarcates the number having the training sample of class label in bracket in table, the unit of recognition result is number percent.

Recognition result contrast table tested by table 1.

In addition, when image is facial image, facial image test sample book is equivalent to the sample to be tested in the application, and facial image training sample is equivalent to the training sample in the application, obtains the Tag Estimation schematic diagram of the classification of facial image test sample book as shown in Figure 3.

Result can be found out by experiment, and the successful of the Robust Learning model that the present invention proposes is better than traditional label propagation algorithm, has higher applicability and robustness.Therefore, the image classification system that the present invention proposes also has same effect.

To the above-mentioned explanation of the disclosed embodiments, those skilled in the art are realized or uses the present invention.To be apparent for a person skilled in the art to the multiple amendment of these embodiments, General Principle as defined herein can without departing from the spirit or scope of the present invention, realize in other embodiments.Therefore, the present invention can not be restricted to these embodiments shown in this article, but will meet the widest scope consistent with principle disclosed herein and features of novelty.

Claims (10)

1. a Robust Learning model, is characterized in that, comprising:

Initialization is carried out to the training set obtained in advance, obtain initial category label matrix, wherein, described training set comprises the training sample of predetermined amount, and described training sample comprises its classification known and demarcates has the sample of the class label corresponding with its classification and its classification unknown and the sample not demarcating class label;

Building method based on neighbour's definition and reconstruct power processes described training sample, obtains the similarity measure matrix corresponding with described training sample, carries out presetting process, obtain reconstruction coefficients matrix to described similarity measure matrix;

Based on described reconstruction coefficients matrix and described initial category label matrix, determined the soft label of the training sample not demarcating class label by the level and smooth item of active balance stream shape and label matching item, adopt the soft label of mode to described training sample of iteration to carry out l _2,1norm regularization, obtains projection matrix and the soft label matrix corresponding with the soft label of described training sample;

Utilize described projection matrix to map sample to be tested, obtain the soft label of described sample to be tested; Wherein, described sample to be tested is its classification unknown and does not demarcate the sample of class label.

2. method according to claim 1, is characterized in that, the described building method based on neighbour's definition and reconstruct power processes described training sample, obtains the similarity measure matrix corresponding with described training sample, comprising:

Utilize k nearest neighbor algorithm to process each described training sample, obtain K nearest samples of each training sample, K is positive integer;

The building method of LLE-reconstruct power is adopted to utilize K nearest samples of described training sample and each described training sample to obtain the similarity measure matrix corresponding with described training sample.

3. method according to claim 2, is characterized in that, described to described similarity measure matrix carry out preset process, obtain reconstruction coefficients matrix, comprising:

Described similarity measure matrix is normalized and symmetrization process, obtains described reconstruction coefficients matrix.

4. method according to claim 3, is characterized in that, utilizes described projection matrix to map sample to be tested, obtains the soft label of described sample to be tested, comprising:

Obtain sample to be tested x _new;

Utilize P ^tx _newby described sample to be tested x _newbe embedded in projection matrix P, obtain predicted vector;

Determine that the soft label that the element of maximum probability in described predicted vector is corresponding is described sample to be tested x _newsoft label.

5. method according to claim 2, is characterized in that, the described training set to obtaining in advance carries out initialization, obtains initial category label matrix, comprising:

Obtain initialization matrix, use Y ₀=[y ₁, y ₂..., y _l+u] represent, described training set is used represent, wherein, represent that n is multiplied by the space of matrices of l+u, X _lfor demarcating the sample having class label, X _l=[x ₁, x ₂..., x _l], X _ufor not demarcating the sample of class label, X _u=[x _l+1, x _l+2..., x _l+u], n, l and u are positive integer;

Determine the arbitrary y in described initialization matrix _ibe a column vector, i-th training sample x in the corresponding described training set of arbitrary column vector _i, i=1,2 ..., l+u;

Arbitrary demarcation is had to the training sample x of class label _jif, this training sample x _jbelong to the i-th classification, then determine y _{i, j}=1, if this training sample x _jdo not belong to the i-th classification, then determine y _{i, j}=0; For arbitrary training sample x not demarcating class label _j, determine y _{i, j}=0, obtain initial category label matrix Y, wherein, and j=1,2 ..., l+u.

6. method according to claim 5, is characterized in that, the described building method based on neighbour's definition and reconstruct power processes described training sample, obtains the similarity measure matrix corresponding with described training sample, comprising:

Following formula is utilized to obtain described similarity measure matrix:

Wherein, represent training sample x _ik nearest samples, q and K is corresponding, for current training sample x _ireconstruction coefficient vector, characterize its neighbour at reconstruct training sample x _itime percentage contribution, represent corresponding similarity reconstruction coefficients matrix.

7. method according to claim 6, is characterized in that, the soft label of mode to described training sample of described employing iteration carries out l _2,1norm regularization, obtains projection matrix and the soft label matrix corresponding with the soft label of described training sample, comprising:

Following formula is utilized to obtain projection matrix and soft label matrix:

subjf _i≥0,e ^Tf _i＝1fori＝1,2,...，l+u

Wherein, F represents soft label matrix, and P represents projection matrix, and W represents reconstruction coefficients matrix, and D represents diagonal matrix, and D _ii=Σ _jw _i,j, f ⁱrepresent the i-th row vector, X ^tp-F ^trepresent and return discrepance, described recurrence discrepance is used for weighing X ^tp and soft label F ^tdifference degree; μ _i, α, beta, gamma is corresponding balance parameter;

Or, utilize following formula to obtain projection matrix and soft label matrix:

$<F,P>=\arg \underset{F,P}{min}tr\left({\mathrm{F\&Omega;F}}^T\right)+tr\left(\right(F-Y\left)UD{\left(F-Y\right)}^T\right)+\mathrm{\&alpha;}\left|\right|F^T|{|}_{2,1}+\mathrm{\&beta;}\left(\right|\left|P\right|{|}_{2,1}+\mathrm{\&gamma;}\left|\right|X^{T}P-F^T\left|{|}_{2,1}\right)$

subjF≥0,e ^TF＝e ^T

Wherein, Ω=I-W ^t-W+W ^tw, I are unit matrix, and it only has diagonal element non-zero and is 1; U represents a diagonal matrix, and its i-th diagonal element is μ _i, for demarcating the training sample having label, μ _ibe set to+∞, for the training sample not demarcating class label, μ _ibe set to 0.

8. an image classification system, is characterized in that, comprising:

Pretreatment module, for carrying out initialization to the training set obtained in advance, obtain initial category label matrix, wherein, described training set comprises the training sample of predetermined amount, and described training sample comprises its classification known and demarcates has the sample of the class label corresponding with its classification and its classification unknown and the sample not demarcating class label; And for processing described training sample based on the building method of neighbour's definition and reconstruct power, obtain the similarity measure matrix corresponding with described training sample, carry out presetting process to described similarity measure matrix, obtain reconstruction coefficients matrix;

Training module, for based on described reconstruction coefficients matrix and described initial category label matrix, determined the soft label of the training sample not demarcating class label by the level and smooth item of active balance stream shape and label matching item, adopt the soft label of mode to described training sample of iteration to carry out l _2,1norm regularization, obtains projection matrix and the soft label matrix corresponding with the soft label of described training sample;

Prediction module, for utilizing described projection matrix to map sample to be tested, obtains the soft label of described sample to be tested; Wherein, described sample to be tested is the sample not demarcating class label.

9. system according to claim 8, is characterized in that, described pretreatment module comprises:

K nearest neighbor unit, for utilizing k nearest neighbor algorithm to process each described training sample, obtain K nearest samples of each training sample, K is positive integer;

LLE-reconstruct power unit, the building method for adopting LLE-to reconstruct power utilizes K nearest samples of each described training sample to obtain the similarity measure matrix corresponding with described training sample.

10. system according to claim 9, is characterized in that, described pretreatment module comprises:

Presetting processing unit, for being normalized and symmetrization process described similarity measure matrix, obtaining described reconstruction coefficients matrix.