CN101520847B - Pattern identification device and method - Google Patents

Abstract

The invention provides a pattern identification device and a pattern identification method; wherein the pattern identification device adopts a semi-supervised learning mechanism, analyzes the structural characteristic of a pattern object sample and carries out linear embedding analysis to the pattern object sample in a nuclear space, thus achieving the purposes of classification and sorting. Particularly, the pattern identification device adopts a local linear embedding mechanism for estimating the flow structure of the pattern object sample, namely, for any pattern object sample, the linear combination of the neighboring object samples thereof are used for carrying out reconstruction, i.e., linear embedding. The reconstruction coefficients of all the object samples are combined to better express the flow structure of the whole input object sample. The invention carries out reconstruction and embedding to the pattern object sample in the nuclear space, thus better reflecting the structural characteristic of a data set.

Description

Pattern recognition device and method

Technical field

The present invention relates to pattern recognition device and method, more particularly, thereby the present invention relates to adopt semi-supervised learning mechanism in nuclear space, schema object to be carried out the linear apparatus and method of describing implementation pattern identification that embed.

Background technology

For purposes such as implementation pattern identification, ordering and retrievals, the object samples that traditional method often relies on some to mark trains, learn obtaining an objective function, thereby realizes above-mentioned function.For instance, in Handwritten Digital Recognition, need collect the image of some handwritten numerals in advance, then these image patterns handled and learnt, thereby obtain a classification function.For example: at first image is carried out binary conversion treatment, extract stroke direction and structural information characteristic then, judge the classification of numeral at last according to these characteristics, promptly generate classification function and carry out discriminator.In this learning process, need inform the actual value of the numeral in each image, promptly need mark these image patterns.Classification function be input as handwritten numeral image (or characteristic of correspondence), be output as the value of the numeral in this image.Like this, when the new handwritten numeral image of one of needs identification, directly this handwritten numeral image is imported above-mentioned classification function, can obtain the value of numeral wherein.

Yet in practical application, obtaining the mark sample often needs expensive time and efforts, and has only the personnel of specialty could accomplish the task of obtaining the mark sample.For example, in biological and computational science field, for protein is classified; Need obtain the shape samples of protein learns in advance; Yet even the crystalline solid analyst of specialty, the time that also need spend some months just can be obtained a protein sample.In contrast, do not mark sample and often exist in a large number, and be easy to obtain.Therefore, how will not mark sample and combine, and therefrom extract Useful Information, just become a vital task in machine learning field with the mark sample.

Mark sample and the difference that does not mark sample are the desired value that whether provides corresponding with it.The so-called sample that do not mark only is one and is processed object, and the mark sample then not only comprises and is processed object, also comprises the desired value that this is processed object.For example, in Handwritten Digital Recognition, not marking sample is exactly a handwriting digital image, and the mark sample also comprises the actual value of the numeral in this image except comprising this image.

The method that relies on the mark sample to carry out pattern classification can be divided into two types, that is: supervised learning and semi-supervised learning.Supervised learning is meant and only relies on the mark sample to learn, train, and therefrom obtains classification function; Semi-supervised learning is meant simultaneously and carries out learning mechanism and method the sample from marking sample and not marking.The basic ideas of semi-supervised learning are: though do not mark the desired value that does not comprise object in the sample; Thereby can not image scale annotate that sample is such directly to be learnt and train; Yet these do not mark and have comprised some useful informations about the object samples space distribution in the sample.If can these useful informations be extracted, and the mark sample combines together, just can help the performance that improves classification, discerns.A given pattern recognition problem, if can provide abundant mark sample to learn, train, the supervised learning method also can reach preferable performance so; Yet if less, the for example above-mentioned protein classification problem of mark sample, the supervised learning method is often failed.In contrast, because the semi-supervised learning method can never mark and extract Useful Information in the sample, therefore, can improve the performance of identification greatly.

In semi-supervised learning mechanism; Given one by the mark sample with do not mark the sample set that sample forms and (be called part and mark sample set; In general; This set comprises a spot of mark sample and a large amount of not mark samples) after, at first analyze the immanent structure of these samples, obtain its spatial distribution characteristic; Then, learn with the mark sample based on the space distribution of sample; At last, do not classify, sort marking sample.

In practical application, the schema object sample of input often comprises a lot of variablees, and promptly these samples are arranged in a higher dimensional space.For example in Handwritten Digital Recognition, the object samples of input is image, and therefore, the dimension of object samples luv space just can be seen the pixel count in the image as.Yet the dimension of the internal space of these image patterns is far below the dimension of its luv space.With digital O is example, if be similar to this numeral with ellipse, the dimension of its internal space is 4, i.e. the radius of centre coordinate and major and minor axis.Consider the distortion and the oval approximate factors such as distortion of handwritten numeral, the inherent dimension of the image pattern corresponding with digital O will be higher than 4, yet this value still will be far smaller than the pixel count in the image.Based on this phenomenon; Semi-supervised learning mechanism is generally all supposed: these data samples in (1) higher dimensional space have inherent lower dimensional space structure, and, in this lower dimensional space, obey flow structure and distribute; So-called flow structure distributes, and the distribution that is meant these samples is level and smooth; (2) on this flow structure, neighbour's sample point generally has identical classification or label.

In recent years, a lot of semi-supervised learning methods have been proposed, thereby to reach the purpose that never marks information extraction raising performance in the sample.Laplce's intrinsic figure (Laplacian Eigenmap) is a kind of more representational semi-supervised learning method, referring to [non-patent literature 1].In the method, at first utilize Euclidean distance and k near neighbor method between sample point (all samples comprise the mark sample and do not mark sample) to make up an adjacent map, obtain its Laplce's matrix thus.So-called adjacent map is exactly to express the relation between sample point with the mode of figure, and the node of figure is corresponding to sample point, and the limit of figure is confirmed by the k near neighbor method.So-called k near neighbor method is exactly to each sample, finds out preceding k the sample nearest with its Euclidean distance, in adjacent map, is this sample and its preceding k neighbour's sample interpolation limit, and the property value on limit is the Euclidean distance between sample.After setting up adjacent map, can obtain its Laplce's matrix at an easy rate, afterwards this matrix is carried out characteristic value decomposition.At last, according to the less proper vector of eigenwert and the sample of mark, the not mark sample that data are concentrated is classified.From seeing in essence, this method has three important characteristics: the data set of (1) this method hypothesis input is to be embedded in the luv space of higher-dimension with flow structure, and promptly the dimension of the internal space of these data is lower; (2) adopt all data (promptly comprise the mark sample and do not mark sample) to estimate its flow structure; (3) utilize Laplacian Beltrami operator to realize the mapping of higher dimensional space, and accomplish the purpose of classification to lower dimensional space.

Local neighborhood Laplce's intrinsic drawing method (referring to [patent documentation 1]) in fact is the extension to the Laplce's intrinsic drawing method in [non-patent literature 1].In the method for [non-patent literature 1], make up Laplce's matrixes according to all sample points in the sample set, and carry out characteristic value decomposition, therefore, when sample point more for a long time, this method is more time-consuming.In order to address this problem, only treat a sub-matrices at classification samples point place in [patent documentation 1] and carry out characteristic value decomposition, thereby, can improve speed significantly.Meanwhile, the method in [patent documentation 1] has also solved the classification problem of new sample point.

The flow pattern ordering is another kind of common semi-supervised learning method, referring to [non-patent literature 2].The main thought of this method is that the immanent structure according to sample point sorts to sample point.Similar with [non-patent literature 1]; This method at first makes up k neighbour figure according to the Euclidean distance between sample point, afterwards, the desired value of mark sample is carried out repetitive propagation along this neighbour figure; All at last sample points all will obtain an evaluation of estimate, and these evaluations of estimate are exactly final sort by.The principle of this method can be explained with a circuit network: as noted earlier, the relation of sample point can represent that the node of adjacent map is corresponding to sample point with adjacent map, and the limit of figure is confirmed by the k near neighbor method; Below, see this adjacent map as a circuit network, link to each other with electrode respectively with mark sample corresponding nodes among the figure; Positive sample connects positive electrode, and negative sample connects negative electrode, and resistor is seen on the limit of figure as; The property value on limit (Euclidean distance between sample) is corresponding to the resistance value of resistor, and in other words, the Euclidean distance between sample is more little; Resistance between corresponding node is just more little, and vice versa; After opening power supply, circuit network reaches equilibrium state, so; Do not mark on the sample corresponding nodes at each, can measure a magnitude of voltage, magnitude of voltage is high more; Explain that this sample approaches positive sample more, magnitude of voltage is low more, explains that this sample approaches negative sample more.In practical implementation, [non-patent literature 2] adopted figure regularization technology to realize said process.

Proposed to utilize Gaussian field harmonic function (Gaussian Fields andHarmonic Functions) to carry out the method for semi-supervised learning in [non-patent literature 3]; This method in fact belongs to same category with the method in [non-patent literature 2]; The realization means of just scheming regularization are different; In the method, push away regularization (Interpolated regularization) in having adopted.[non-patent literature 4] analyzed these two kinds of diverse ways, and drawn its error upper limit theoretically.Under the framework of figure regularization, [non-patent literature 5] revised the objective function of optimizing, and introduces relaxation factor, thereby will scheme the form that regularization changes into the standard SVMs.

Similar with above-mentioned method; Existing semi-supervised learning method is mostly set up neighbour figure (neighbour's matrix) according to the Euclidean distance of sample point; And suppose that this neighbour figure can describe the inherent flow structure of these sample points fully, the Euclidean distance that is about to sample point in the luv space is equal to " measuring distance " on the flow structure.Yet this hypothesis is devious under many circumstances, and is accurate inadequately in other words conj.or perhaps.[non-patent literature 6] and [patent documentation 5] all sets forth this problem.Fig. 1 (a) and Fig. 1 (b) show this problem with concrete example.Fig. 1 (a) is a flow structure of the shape that is similar to " S " in the three dimensions, and Fig. 1 (b) is the corresponding sampled point of this flow structure, i.e. sample set.In the figure, represent the desired value of sampling point with gray scale, promptly the gray-scale value of data sample is close more, represent they similar more (except two end points places of serpentine shape).Two data points A in this space, B, its Euclidean distance shown in the solid line among Fig. 1 (b), and these two points at the measuring distance on the flow structure shown in the dotted line among 1 (b).Visible by figure, it is inaccurate coming the inherent flow structure of expression data point with Euclidean distance.Therefore, this hypothesis of inherent flow structure that the neighbour figure that makes up based on Euclidean distance can describe sample point fully is inaccurate, thereby the existing technical method that is implemented on this hypothesis all is defective, and its performance also is limited.

[patent documentation 1] US patent application publication US 2006/0235812A1

[patent documentation 2] US patent application publication US 2006/0045353A1

[non-patent literature 1] Mikhail Belkin, Partha Niyogi, Semi-SupervisedLearning on Rimanian Manifolds, Machine Learning, Vol.56, pp.209-239,2004

[non-patent literature 2] Dengyong Zhou, Olivier Bousquet, Thomas Navi Lal; Jason Weston, and Bernhard Scholkopf, Learning with Local andGlobal Consistency; Advances in Neural Information ProcessingSystems; Vol.16, pp.321-328,2003

[non-patent literature 3] Xiaojin Zhu; Zoubin Ghahramani; John Lafferty, Semi-Supervised Learning Using Gaussian Fields and HarmonicFunctions, in Proceedings of the twentieth internationalconference on machine learning (ICML2003); Pp.912-919,2003

[non-patent literature 4] Mikhail Belkin; Irina Matveeva; Partha Niyogi; Regularization and Semi Supervised Learning on Large Graphs, inProceedings of annual conference on learning theory (COLT), 2004

[non-patent literature 5] Zhili Wu; Chun-hung Li, Ji Zhu, Jian Huang; ASemi-Supervised SVM for Manifold Learning; In Proceedings of the18th international conference on pattern recognition (ICPR ' 06), Vol.2, pp.490-493

[non-patent literature 6] Joshua B.Tenenbaum, Vin de Silva, John C.Langford; A global geometric framework for nonlineardimensionality reduction, Science, Vol.290; Pp.2319-2323,2000

[non-patent literature 7] Sam T.Roweis, Lawrence K.Saul, Nonlineardimensionality reduction by locally linear embedding, Science, Vol.290, pp.2323-2326,2000

Summary of the invention

In view of the problems referred to above of the prior art the present invention has been proposed.The object of the invention is to provide a kind of at least and comes pattern is carried out the apparatus and method of Classification and Identification based on semi-supervised learning mechanism, and it can overcome the problem of the inherent flow structure that the neighbour figure that makes up based on Euclidean distance can not the perfect representation sample point.

Based on the present invention, the input pattern object comprises mark simultaneously and does not mark sample, and based on these samples, the spatial distribution characteristic of said device analysis sample is also accomplished not marking sample identification (classification and ordering).A characteristic of this device is in nuclear space, to adopt local linear embedding to estimate the flow structure of input object, and based on this, utilizes the figure regularization to realize the function of identification.

At first, calculate input object sample distance between any two, promptly calculate the distance between any two input object samples.Distance calculation is relevant with the form and the kind of input object sample; In pattern recognition problem, generally need at first the input object sample to be handled, extract its principal character; For example: in Handwritten Digital Recognition; Characteristic can be input object sample itself, and promptly the value of image pixel also can be the stroke direction histogram; And in speaker ' s identity identification, characteristic can be the cepstrum coefficient that utilizes voice signal to obtain, fundamental frequency etc.If the characteristic of input object sample is represented as the form of vector, then can be with the distance between Euclidean distance or other distance measure calculating object samples; If the input object sample is represented as structurized characteristic, then need utilize the distance calculation mode corresponding with this characteristic.

Afterwards,, utilize k neighbour mode to set up k neighbour figure, in other words, represent the relation between the input object sample with the mode of figure according to the above-mentioned input object sample distance between any two that obtains.

Next, scheme to make up reproducing kernel Hilbert space (ReproducingKernel Hilbert Space) according to above-mentioned neighbour, for example: Laplce's nuclear space (Laplacian KernelSpace), and in this space, carry out the linear embedding in part.Further saying, is exactly to each sample, all utilize its neighbour's sample in nuclear space to carry out linear reconstruction, and the reconstruction coefficients of all samples just can well be expressed the internal space distribution characteristics of sample.

At last, utilize the desired value of above-mentioned reconstruction coefficients and mark sample, adopt figure regularization technology, give and do not mark evaluation of estimate of sample.And these evaluations of estimate are exactly to not marking the foundation that sample is discerned.

According to an aspect of the present invention; A kind of pattern recognition device is provided; This pattern recognition device utilizes semi-supervised learning mechanism to come the schema object sample of input is discerned, and wherein said schema object sample comprises the mark sample and do not mark sample, and representes with digitalized signature; Said mark sample comprises the digitalized signature and the corresponding desired value of schema object; Said pattern recognition device comprises: non-Euclidean space kernel structure portion, and its schema object sample based on input makes up nuclear matrix in the non-Euclidean space, and this non-Euclidean space is suitable for the flow structure of structural model object; Local linear Embedded Division in the nuclear space, nuclear matrix in its non-Euclidean space that constructs according to said non-Euclidean space kernel structure portion carries out to the schema object sample that the part is linear to embed, with the reconstruction coefficients matrix of generate pattern object samples; And regularization portion, it comes to generate the evaluation and test value for the schema object sample of all inputs according to the reconstruction coefficients matrix of local linear Embedded Division generation in the said nuclear space and the desired value of said mark sample.

According to another aspect of the present invention; A kind of mode identification method is provided; This mode identification method utilizes semi-supervised learning mechanism to come the schema object sample of input is discerned; Wherein said schema object sample comprises the mark sample and does not mark sample, and representes that with digitalized signature said mark sample comprises the digitalized signature and the corresponding desired value of schema object; Said mode identification method may further comprise the steps: the schema object sample based on input makes up nuclear matrix in the non-Euclidean space, and this non-Euclidean space is suitable for the flow structure of structural model object; According to nuclear matrix in the non-Euclidean space that constructs, the schema object sample is carried out the linear embedding in part, with the reconstruction coefficients matrix of generate pattern object samples; And, come to generate the evaluation and test value for the schema object sample of all inputs according to the reconstruction coefficients matrix that generates and the desired value of said mark sample.

Above-mentioned pattern recognition device of the present invention and mode identification method; Owing to adopted in the non-Euclidean space of the inherent flow structure that is suitable for making up the schema object sample nuclear matrix (for example; Nuclear matrix in the reproducing kernel Hilbert space more specifically, is a nuclear matrix in the Laplce space) come the schema object sample is carried out the linear embedding in part; Thereby the inherent flow structure of expression pattern object samples exactly, and finally realize pattern-recognition accurately.

Description of drawings

Fig. 1 (a) illustration an example of flow structure.

Fig. 1 (b) illustration corresponding to the sampled point of the flow structure among Fig. 1 (a), and the example of measuring distance on Euclidean distance and the flow structure.

Fig. 2 is the schematic block diagram according to the pattern recognition device of embodiment of the present invention.

Fig. 3 makes up the process flow diagram of module for Laplce's kernel.

Fig. 4 is the synoptic diagram according to the input object in the embodiments of the invention.

Fig. 5 is the evaluation and test value sketch map based on the input object in the embodiments of the invention.

Fig. 6 is the part input object according to the Handwritten Digital Recognition in the embodiments of the invention.

Embodiment

Fig. 2 is the schematic block diagram according to the pattern recognition device of embodiment of the present invention.

As shown in the figure, this pattern recognition device comprises: Laplce's kernel makes up module, and it makes up Laplce's kernel spacing matrix according to input object sample (comprise the mark sample and do not mark sample); Local linear merge module in the nuclear space, it makes up Laplce's kernel spacing matrix that module construction goes out according to Laplce's kernel, the input object sample is carried out the linear embedding in part, with the reconstruction coefficients matrix of formation object sample; And the regularization module, it is that all input object samples generate the evaluation and test value according to local linear merge module in the nuclear space reconstruction coefficients matrix that generates and the desired value that marks sample.

Except above-mentioned Laplce's kernel spacing, this pattern recognition device can also adopt other reproducing kernel Hilbert space.The purpose that makes up this space is to describe preferably the flow structure of input object.

Input object promptly is processed object set, for example the image in the image recognition, the voice signal in the speech recognition etc.Input object is generally represented with the mode of numerical characteristic; This numerical characteristic is made up of a plurality of variablees; In order to describe this Properties of Objects, for example in image recognition, this characteristic can be for the color histogram of each color of pixel value, image texture features vector, image in the image etc.; In Speaker Identification, this characteristic can be the cepstrum coefficient that obtains based on voice signal, fundamental frequency etc.Input object includes the mark sample simultaneously and does not mark sample.So-called mark sample, promptly except with this sample characteristic of correspondence, also import a desired value corresponding with this sample; For example in two classification image recognitions, for the image of the first kind, its desired value can be set to 1; And for second type image, its desired value can be set to-1; In Speaker Identification, desired value can be numbering of speaker etc.And for non-mark sample, then only input and this sample characteristic of correspondence.

Laplce's kernel makes up module and is used for from the input object sample, making up Laplce's kernel spacing.At first, calculate input object sample distance between any two.Afterwards, make up neighbour figure, calculate Laplce's space matrix according to the k near neighbor method.At last, obtain Laplce's kernel spacing matrix.

Local linear merge module is implemented in the inherent geometry of describing input object in Laplce's kernel spacing in the nuclear space.Further say; Have on the data set of flow structure; Any infinitesimal zone all is linear, therefore, and for each input object sample; Can utilize the linear combination of its adjacent sample to come reconstruct (or approximate), these reconstruction coefficients have then reflected the geometry characteristic of input object collection.

The regularization module is used for calculating the evaluation and test value that does not mark object samples.According to the reconstruction coefficients that local linear merge module in the above-mentioned nuclear space obtains, each object samples can be similar to by its linear combination of neighbour's sample, and therefore, the evaluation and test value of object samples also can be similar to according to these reconstruction coefficients.Based on this principle, under the framework of figure regularization, be optimized calculating, do not marked the evaluation and test value of sample.

The evaluation and test value that obtains from above-mentioned regularization module has reflected the similarity degree that does not mark between sample and the mark sample.For example, in the identification of two quasi-modes, the desired value of the mark sample of first kind pattern is set to 1; And the desired value of the mark sample of second quasi-mode is set to-1, and so, the evaluation and test value that does not mark sample approaches 1 more; It is big more to represent that this sample belongs to the probability of first kind pattern; On the contrary, approach-1 more, it is big more to represent that this sample belongs to the probability of second quasi-mode.

Specifically describe each composition module of pattern recognition device below.

1, Laplce's kernel makes up module

Make that input object is X={X _L, X _U.Wherein, X _L={ (x ₁, y ₁) ..., (x _l, y _l) for marking sample set, X ₁..., X _lExpression marks the numerical characteristic of sample respectively, and l is the number of mark sample, y ₁..., y _lDesired value for these samples; X _U={ x _L+1..., x _nFor not marking sample set, X _L+1..., X _nExpression does not mark the numerical characteristic of sample respectively, total n-1 object samples, and n is the sum of input sample.At the remainder of instructions, all write down the input object sample set by this way.

Fig. 3 is the schematic block diagram of the formation of Laplce's kernel structure module, and it is made up of three modules: neighbour's matrix computations module, Laplce's matrix computations module, and Laplce's kernel computing module.

Neighbour's matrix computations module generates neighbour's matrix according to input object sample distance between any two.The size of neighbour's matrix is n * n (n is a natural number), is used for describing the similarity between any two object samples.

At first, for any two input object samples, calculate the distance between them.The account form of distance is relevant with the expression way of input object sample, for example: if the input object sample represent with vector mode, just can adopt Euclidean distance to estimate and calculate; If object samples representes with structured features, just need be according to the mode of the concrete format design distance calculation of this characteristic.Any given two object samples X _i, X _j, the distance between them be designated as d (i, j).

Next, to each input object sample, find out a nearest with it k object samples.K is for realizing a parameter of pattern recognition device of the present invention, and value is a positive integer, can rule of thumb set.

If neighbour's matrix is A=[a _Ij], i, j=1,2 ..., n.Element a in neighbour's matrix A _IjValue calculate as follows: provide two object samples X arbitrarily _i, X _jIf, X _jBelong to and X _iIf one of nearest k sample is perhaps X _iBelong to and X _jOne of nearest k sample, then a _Ij=a _Ji=exp (d (i, j) ²/ 2 σ ²), otherwise, a _Ij=0; In addition, if the value of i and j is identical, a _Ij=0.σ is for realizing another parameter of pattern recognition device of the present invention, and it has reacted the category of the distance of input object sample, can be set at the mean value of the distance between all object samples, perhaps rule of thumb is provided with.

Obtain after neighbour's matrix, Laplce's matrix computations module generates Laplce's matrix according to neighbour's matrix.Make that D is the diagonal matrix of n * n, the value of each element is on the diagonal line: $D_{\mathrm{Ij}}=\underset{j}{\mathrm{\Σ}}{a}_{\mathrm{Ij}},$

So, Laplce's matrix L can be by computes: L=D-A.Laplce's matrix after the normalization is: $\tilde{L}=D^{-\frac{1}{2}}{\mathrm{LD}}^{-\frac{1}{2}}=I-D^{-\frac{1}{2}}{\mathrm{AD}}^{-\frac{1}{2}},$ Wherein, I is the unit diagonal matrix.

Regularization Laplce matrix is: $P=r\left(\tilde{L}\right)=\tilde{L}+\mathrm{\ϵ}\·I,$ Wherein, I is the unit diagonal matrix, and ε is for realizing a parameter of pattern recognition device of the present invention, and general value is very little, for example gets 10 ^-5

Laplce's kernel computing module promptly obtains Laplce's kernel K to the above-mentioned regularization Laplce matrix inversion that obtains.If P is irreversible for regularization Laplce matrix, then ask its pseudo inverse matrix.

Except Laplce's kernel, pattern recognition device of the present invention can also adopt other reproducing kernel Hilbert space, for example: diffusion kernel, counter-rotating cosine kernel etc.

The building process of diffusion kernel is following:

(1) identical with above-mentioned Laplce's kernel building process, utilize neighbour's matrix computations module to generate neighbour's matrix A;

(2) identical with above-mentioned Laplce's kernel building process, utilize Laplce's matrix computations module to generate normalized Laplce's matrix according to neighbour's matrix

(3) according to above-mentioned normalized Laplce's matrix

Obtain diffusion kernel K _D, computation process is: $K_D=\mathrm{Exp}(-{\mathrm{\τ}}^2/2\tilde{L}),$ τ is for realizing a parameter of pattern recognition device of the present invention.The building process of counter-rotating cosine kernel is following:

(1) identical with above-mentioned Laplce's kernel building process, utilize neighbour's matrix computations module to generate neighbour's matrix A;

(2) identical with above-mentioned Laplce's kernel building process, utilize Laplce's matrix computations module to generate normalized Laplce's matrix

according to neighbour's matrix

(3) according to above-mentioned normalized Laplce's matrix

Cosine kernel K obtains reversing _C,

Computation process is: $K_C=\mathrm{Cos}(\tilde{L}*\mathrm{\π}/4).$

Given regularization Laplce matrix P can define inner product and does<f, f> _H=<f, Pf>, H is exactly a reproducibility Hilbert kernel spacing so, and its kernel be exactly k (i, j)=[P ^-1] _Ij

In order to prove its reproducibility, need satisfy following condition:

f(i)＝<f，k(i，·)> _H

In other words, the f ∈ H to all should have:

F (i)=<f, k (i)> _H=f ^TPK _i,, in other words, f ^T=f ^TPK

Obviously, if K=P ^-1, then above-mentioned condition just can satisfy.Therefore, above-mentioned regularization Laplce inverse of a matrix matrix is exactly Laplce's kernel.

2, local linear merge module in the nuclear space

Local linear embedding (Locally linear embedding is abbreviated as LLE) is proposed by people such as Sam T.Roweis the earliest, referring to [non-patent literature 7].Its basic thought is: to each object samples, utilize its neighbour to put and make up a local linear model.On the data set of flow structure; Any infinitesimal zone all is linear; Therefore, any one data point can be come reconstruct with its linear combination of adjoint point, promptly linear the embedding; And the reconstruction coefficients of all sample points is combined, and just can describe the flow structure of this data set.LLE is applied among the characteristic dimensionality reduction the earliest.People such as Sam T.Roweis notice; Keeping such as rotation, translation, convergent-divergent etc. in the conversion of neighbor relationships; The reconstruction coefficients that obtains from LLE has unchangeability, and therefore, the neighbor relationships of sample point in the flow structure space of low dimension is consistent with the neighbor relationships in its luv space; In other words, the reconstruction coefficients that from luv space, obtains is suitable equally in the flow structure space.Find that based on this LLE can well be used for accomplishing the purpose of characteristic dimensionality reduction.

Local linear embedding of pattern recognition device of the present invention carried out in kernel spacing, specifically, in above-mentioned Laplce's kernel spacing, carries out.

Given set of data samples X={x ₁..., x _n(this sample set comprises the mark sample and do not mark sample) and Laplce's kernel function K, kernel function K has in fact defined a mapping function φ on sample set X, and through this function, data sample is projected among the space F of a higher-dimension, that is: φ: x _i→ φ (x _i), this mapping is satisfied following inner product and is concerned: k (x, x ')=<φ (x), φ (x ')>

Provide two sample φ (x in the F space arbitrarily _m) and φ (x _n), their Euclidean distance is:

‖φ(x _m)-φ(x _n)‖ ²＝k(m，m)+k(n，n)-2k(m，n)

In higher dimensional space F,, all use the linear combination of its neighbour's sample to be similar to i.e. reconstruct for each sample.The total error of this reconstruct is:

ε(W)＝∑ _i‖φ(x _i)-∑ _jw _ijφ(x _j)‖ ²，φ(x _j)∈N(φ(x _i))

In following formula, N (φ (x _i)) expression sample X _iNeighbour's sample in the F of space, W _IjExpression sample X _iReconstruction coefficients.

With sample X _iBe example, suppose sample X in higher dimensional space F _iM neighbour's sample be { x _i ¹..., x _i ^M, by this M neighbour's sample to X _iCoefficient when carrying out reconstruct is W _i=[w _I1..., w _IM] ^T, reconstructed error is so:

$\mathrm{\&epsiv;}={\left|\right|\mathrm{\&phi;}\left(x_i\right)-{\mathrm{\&Sigma;}}_j{w}_{\mathrm{ij}}\mathrm{\&phi;}\left(x_i^1\right)\left|\right|}^2=W_i^T{C}_i{W}_i$

Wherein, C _iBe called as φ (x _i) the Gram matrix, its size is M * M, the value of each element is in the matrix:

$C_i(m,n)={(\mathrm{\&phi;}\left(x_i\right)-\mathrm{\&phi;}\left(x_i^m\right))}^T(\mathrm{\&phi;}\left(x_i\right)-\mathrm{\&phi;}\left(x_i^n\right))$

$=k(x_i^m,x_i^n)-k(x_i,x_i^n)-k(x_i^m,x_i)+k(x_i,x_i)$

Obviously, make the minimum reconstruction coefficients of reconstructed error be:

$W_i=\frac{C_i^{--1}\stackrel{\&OverBar;}{1}}{{\stackrel{\&OverBar;}{1}}^T{C}_i^{-1}\stackrel{\&OverBar;}{1}}$

The input and output of local linear merge module and performed key step are as follows in the nuclear space:

Input: set of data samples X, Laplce's kernel K.

Output: the reconstructed coefficients matrix W of each data sample.

To each data sample x _i, all carry out following operation:

(1) calculates among the higher dimensional space F this sample with the Euclidean distance between other sample, that is:

‖φ(x _i)-φ(x _j)‖ ²＝k(i，i)+k(j，j)-2k(i，j)，j＝1，…，n

(2), find out and x according to above-mentioned distance _iA nearest M sample, { x _i ¹..., x _i ^M.M is another parameter that realizes pattern recognition device of the present invention, and value is a positive integer, can get same value with the parameter k that above-mentioned Laplce's kernel makes up in the module.

(3) calculated size is the Gram Matrix C of M * M _i

(4) according to the Gram Matrix C _iCalculate reconstruction coefficients W _i=[w _I1..., w _IM] ^T

(5) reconstruction coefficients is carried out normalization, that is: make W _iMould be 1.

(6) generate the reconstruction coefficients matrix W.The size of this matrix is n * n, and each row of matrix is corresponding to the reconstruction coefficients of a sample, for preceding M nearest sample, is worth and is W _iIn analog value, otherwise value is 0.

3, regularization module

Given input object sample X={x ₁..., x _l, x _L+1..., x _n, and with mark sample corresponding desired value { y ₁..., y _l, the regularization module is used for generating the evaluation and test value into these samples.In other words, obtain a mapping function f:X → R from the regularization module, this function is mapped as the evaluation and test value of a real number with the input object sample, and this function can be expressed as the form of vector, that is:

f＝{f ₁，…f _n}

Local linear merge module has generated a series of reconstruction coefficients for the input object sample in the above-mentioned nuclear space; According to these coefficients; Each sample can be similar to by its linear combination of neighbour's sample; In addition, these coefficients have also reflected the inherent geometry of object samples, and these coefficients have unchangeability in the conversion that keeps neighbor relationships.Therefore, these coefficients can be used for the evaluation and test value of reconstruct object samples, and in other words, based on these coefficients, the evaluation and test value of each object samples can be similar to by its linear combination of evaluation and test value of neighbour's sample.

The error of the evaluation and test value of sample being carried out reconstruct is: ζ (f)=∑ _i‖ f _i-∑ _jw _Ijf _j‖ ²,

Wherein, W _IjExpression local linear merge module in the above-mentioned nuclear space obtain with object samples X _iRelevant reconstruction coefficients.

In addition, for the mark sample, the evaluation and test value that obtains from the regularization module should be more or less the same with their desired value.Make y=[y ₁..., y _n] ^T, wherein, { y ₁..., y _lFor marking the desired value of sample, { y _L+1..., y _nValue be 0, then this constraint condition can be expressed as: τ (f)=∑ _i(f _i-y _i) ²

The constraint linearity of above-mentioned constraint condition and reconstructed error is combined, is just obtained:

ε(f)＝ζ(f)+μ*τ(f)

＝∑ _i‖f _i-∑ _jw _ijf _j‖ ²+μ∑ _i(f _i-y _i) ²

＝f ^TMf+μ(f-y) ^T(f-y)

Wherein, W representes the reconstruction coefficients matrix that local linear merge module obtains in the above-mentioned nuclear space, and μ generally gets very little real number for realizing another parameter of pattern recognition device of the present invention, for example is taken as 10 ^-3, M=(I-W) ^T(I-W), I is a unit matrix.

The value of mapping function f makes the minimum value of value of above-mentioned ε (f) exactly, that is:

$f=\underset{f:x\&RightArrow;R}{\arg \min }\mathrm{\&epsiv;}\left(f\right)$

ε (f) is carried out differentiate, and the value that obtains f at last is:

f＝μ(M+μI) ^-1y

The key step that the regularization module is carried out is as follows:

Input: the reconstruction coefficients matrix W of the sample of local linear merge module output in the above-mentioned nuclear space, the desired value { y of mark sample ₁..., y _l.

Output: the evaluation and test value vector f of sample.

(1) generates vectorial y=[y ₁..., y _n] ^T, wherein, { y ₁..., y _lFor marking the desired value of sample, { y _L+1..., y _nValue be 0.

(2) compute matrix M=(I-W) ^T(I-W).

(3) calculate evaluation and test value vector f=μ (M+ μ I) ^-1Y.

Come the present invention is further explained through concrete embodiment below.

Embodiment one:

Suppose to have the object samples of 12 inputs, these samples all are expressed as the form of 2 dimensional vectors, and are as follows:

$x=\left[\begin{array}{cccccccccccc}x1& x2& x3& x4& x5& x6& x7& x8& x9& x10& x11& x12\\ 0& 3.1& 5.9& 8.1& .9.5& 10& 1& 1.5& 2.9& 5.1& 7.9& 11\\ 10& 9.5& 8.1& 5.9& 3.1& 0& 2& -1.1& -3.9& -6.1& -7.5& -8\end{array}\right]$

That is: the digitizing expression way of first sample is [0 10], and the digitizing expression way of second sample is [3.1 9.5], by that analogy.

Wherein, first is the mark sample with last object samples, and its label is respectively 1 and-1, and remaining object samples is not for marking sample, that is:

Y＝[1 0 0 0 0 0 0 0 0 0 0 -1]

Fig. 4 is the synoptic diagram of these samples, and wherein, each o'clock, its coordinate was the value of the vector of these object samples corresponding to an input object sample.In the figure, represent two mark samples respectively with solid squares and triangles.

The objective of the invention is to predict that all do not mark the label of sample, to realize the purpose of identification or ordering retrieval.

Step 1: Laplce's kernel makes up module the object samples of input is handled, to make up Laplce's kernel.

At first, calculate distance between any two input object samples.In this embodiment, 12 object samples of input all are represented as the form of 2 dimensional vectors, therefore, calculate the distance between them with Euclidean distance.For example: the Euclidean distance between first and second object samples is:

d(1，2)＝{(3.1-0) ²+(10-9.5) ²} ^0.5＝3.1。

The object samples that obtains in a manner described distance between any two can use a matrix representation to be:

X ₁ X ₂ X ₃ X ₄ X ₅ X ₆ X ₇ X ₈ X ₉ ?X ₁₀ X ₁₁ X ₁₂

X ₁ 0.0 3.1 6.2 9.1 11.8 14.1 8.1 11.2 14.2 16.9 19.2 21.1

X ₂ 3.1 0.0 3.1 6.2 9.1 11.8 7.8 10.7 13.4 15.7 17.7 19.2

X ₃ 6.2 3.1 0.0 3.1 6.2 9.1 7.8 10.2 12.3 14.2 15.7 16.9

X ₄ 9.1 6.2 3.1 0.0 3.1 6.2 8.1 9.6 11.0 12.3 13.4 14.2

X ₅ 11.8 9.1 6.2 3.1 0.0 3.1 8.6 9.0 9.6?10.2?10.7?11.2

X ₆ 14.1?11.8 9.1 6.2 3.1 0.0 9.2 8.6 8.1 7.8 7.8 8.1

X ₇ 8.1 7.8 7.8 8.1 8.6 9.2 0.0 3.1 6.2 9.1?11.8?14.1

X ₈ 11.2?10.7?10.2 9.6 9.0 8.6 3.1 0.0 3.1 6.2 9.1?11.8

X ₉ 14.2?13.4?12.3?11.0 9.6 8.1 6.2 3.1 0.0 3.1 6.2 9.1

X ₁₀?16.9?15.7?14.2?12.3?10.2 7.8 9.1 6.2 3.1 0.0 3.1 6.2

X ₁₁?19.2?17.7?15.7?13.4?10.7?7.81 1.8 9.1 6.2 3.1 0.0 3.1

X ₁₂?21.1?19.2?16.9?14.2?11.2?8.11 4.1?11.8 9.1 6.2 3.1 0.0

According to these distance values,, find out a nearest with it k object samples to each input object sample.In this embodiment, the value with k is made as 3.With object samples X ₁For example describes, these input object samples are to X ₁Distance be respectively: 0,3.1,6.2,9.1,11.8,14.1,8.1,11.2,14.2,16.9,19.2,21.1, therefore, 3 nearest with it object samples are: X ₂, X ₃, X ₇

Afterwards, calculate neighbour's matrix by following principle: for two object samples X _i, X _jIf object samples belongs to and one of nearest k object samples of another object samples, then the respective value in neighbour's matrix is a _Ij=a _Ji=exp (d (i, j) ²/ 2 σ ²), otherwise, a _Ij=0; In addition, the value of the element on the diagonal line is made as 0 in neighbour's matrix.In the present embodiment, σ is set at the mean value of the distance between all object samples, according to the above-mentioned object samples that obtains distance between any two, its value is 8.89.Still be that example describes with first and second object samples, can find out from the calculating of front that second object samples belongs to and one of nearest 3 objects of first object samples, therefore, the respective value in neighbour's matrix is: a ₁₂=exp (3.1 ²/ (2*8.89 ²))=0.94.

According to aforesaid way, calculate the value of all elements in neighbour's matrix, the neighbour's matrix that obtains is following:

A＝ 0?0.94?0.79 0 0 0?0.66 0 0 0 0 0

0.94 0?0.94?0.79 0 0?0.68 0 0 0 0 0

0.79?0.94 0?0.94?0.79?0.00 0 0 0 0 0 0

0?0.79?0.94 0?0.94?0.79 0 0 0 0 0 0

0 0?0.79?0.94 0?0.94 0 0 0 0 0 0

0 0 0?0.79?0.94 0 0 0 0 0?0.68?0.66

0.66?0.68 0 0 0 0 0?0.94?0.79 0 0 0

0 0 0 0 0 0?0.94 0?0.94?0.79 0 0

0 0 0 0 0 0?0.79?0.94 0?0.94?0.79 0

0 0 0 0 0 0 0?0.79?0.94 0?0.94?0.79

0 0 0 0 0?0.68 0 0?0.79?0.94 0?0.94

0 0 0 0 0?0.66 0 0 0?0.79?0.94 0

Obtain after neighbour's matrix, Laplce's matrix computations module generates Laplce's matrix L and normalized Laplce's matrix

according to neighbour's matrix

For generating Laplce's matrix L; At first need generate a diagonal matrix D; Element on this diagonal of a matrix be corresponding row in neighbour's matrix element with; For example, on the diagonal line value of first element be first row in neighbour's matrix A element with, that is: D (1)=0.94+0.79+0.66=2.39.

According to diagonal matrix D and neighbour's matrix A; Can easily obtain Laplce's matrix L, normalized Laplce's matrix

and regularization Laplce matrix P.

At last, to regularization Laplce matrix inversion, obtain Laplce's kernel.In the present embodiment, the value of parameter ε is made as 0.01 in the regularization Laplce matrix, and the Laplce's kernel that obtains is:

K＝?7.75 8.19 8.22 7.89 6.84 7.10 7.54 6.62 7.46 7.33 7.21 6.08

8.19?10.40 9.74 9.55 8.19 8.49 8.80 7.77 8.77 8.64 8.53 7.21

8.22 9.74?10.78 9.89 8.65 8.77 8.65 7.73 8.76 8.70 8.64 7.33

7.89 9.55 9.89?10.71 8.77 9.06 8.53 7.71 8.77 8.76 8.77 7.46

6.84 8.19 8.65 8.77 8.55 8.11 7.42 6.76 7.71 7.73 7.77 6.62

7.10 8.49 8.77 9.06 8.11 9.48 7.96 7.42 8.53 8.65 8.80 7.54

7.54 8.80 8.65 8.53 7.42 7.96 9.48 8.11 9.06 8.77 8.49 7.10

6.62 7.77 7.73 7.71 6.76 7.42 8.11 8.55 ?8.77 ?8.65 ?8.19 6.84

7.46 8.77 8.76 8.77 7.71 8.53 9.06 8.77 10.71 ?9.89 ?9.55 7.89

7.33 8.64 8.70 8.76 7.73 8.65 8.77 8.65 ?9.89 10.78 ?9.74 8.22

7.21 8.53 8.64 8.77 7.77 8.80 8.49 8.19 ?9.55 ?9.74 10.40 8.19

6.08 7.21 7.33 7.46 6.62 7.54 7.10 6.84 ?7.89 ?8.22 ?8.19 7.75

Step 2: in Laplce's kernel spacing, the input object sample is carried out linearity and embed

calculates all object samples distance between any two in Laplce's kernel spacing

For input object sample X _i, X _j, in Laplce's kernel spacing corresponding distance be k (i, i)+k (j, j)-2*k (i, j).Is example with first with second object samples, and their distances in Laplce's kernel spacing are:

k(1，1)+k(2，2)-2*k(1，2)＝7.75+10.40-2*8.19＝1.76。

The distance of any two input object samples in Laplce's kernel spacing that obtains in a manner described is:

0?1.76?2.08?2.68?2.63?3.04?2.16?3.06?3.54?3.87?3.74?3.35

1.76 0?1.70?2.01?2.57?2.91?2.29?3.41?3.57?3.89?3.74?3.74

2.08?1.70 0?1.71?2.03?2.72?2.96?3.86?3.95?4.16?3.89?3.87

2.68?2.01?1.71 0?1.72?2.06?3.14?3.83?3.86?3.95?3.57?3.54

2.63?2.57?2.03?1.72 0?1.80?3.18?3.57?3.83?3.86?3.41?3.06

3.04?2.91?2.72?2.06?1.80 0?3.05?3.18?3.14?2.96?2.29?2.16

2.16?2.29?2.96?3.14?3.18?3.05 0?1.80?2.06?2.72?2.91?3.04

3.06?3.41?3.86?3.83?3.57?3.18?1.80 0?1.72?2.03?2.57?2.63

3.54?3.57?3.95?3.86?3.83?3.14?2.06?1.72 0?1.71?2.01?2.68

3.87?3.89?4.16?3.95?3.86?2.96?2.72?2.03?1.71 0?1.70?2.08

3.74?3.74?3.89?3.57?3.41?2.29?2.91?2.57?2.01?1.70 0?1.76

3.35?3.74?3.87?3.54?3.06?2.16?3.04?2.63?2.68?2.08?1.76 0

nuclear space internal linear embeds

The class of operation that makes up module with Laplce's kernel seemingly according to the above-mentioned distance value that obtains, to each input object sample, is found out a nearest with it M object samples.In the present embodiment, the value of M is identical with the value of k, i.e. M=3.With input object sample X ₁For example describes, these object samples are to X ₁Distance be: 0,1.76,2.08,2.68,2.63,3.04,2.16,3.06,3.54,3.87,3.74,3.35, therefore, 3 nearest with it input object samples are: X ₂, X ₃, X ₇

Next, local linear merge module utilizes neighbour's object samples of each input object sample to come this input object sample of linear-apporximation in the nuclear space, to realize the linear purpose that embeds.

With object samples X ₁For example describes.Obtain from the front, 3 object samples nearest with this object samples are X ₂, X ₃, X ₇, therefore, need in nuclear space, utilize these 3 objects to be similar to object samples X ₁

At first, make up the Gram Matrix C, the size of this matrix is 3 * 3, and the row and column of matrix corresponds respectively to input object sample X ₂, X ₃, X ₇Calculate for example with C (1,2), its value is: C (1,2)=k (X ₂, X ₃)-k (X ₁, X ₂)-k (X ₃, X ₁)+k (X ₁, X ₁)=9.74-8.19-8.22+7.75=1.08; Calculate for example with C (2,3), its value is: C (2,3)=k (X ₃, X ₇)-k (X ₁, X ₇)-k (X ₃, X ₁)+k (X ₁, X ₁)=8.65-7.54-8.22+7.75=0.64.

In a manner described, obtain and input object sample X ₁Corresponding Gram Matrix C, as follows:

$C=\left[\begin{array}{ccc}1.76& 1.08& 0.82\\ 1.08& 2.08& 0.64\\ 0.82& 0.64& 2.16\end{array}\right]$

Then, according to above-mentioned Gram matrix, obtain input object sample X ₁Reconstruction coefficients, its value is 0.29,0.24,0.28, these 3 coefficients correspond respectively to input object sample X ₂, X ₃, X ₇

generates the reconstruction coefficients matrix

For each input object sample, all press aforesaid operations and generate reconstruction coefficients, afterwards, coefficient is carried out normalization, and generate the reconstruction coefficients matrix W.

In the present embodiment, the size of reconstruction coefficients matrix is 12 * 12.Each of this matrix is gone corresponding to an input object sample, wherein, those input object samples nearest with this object samples, corresponding value is set to reconstruction coefficients, otherwise value is 0.With object samples X ₁For example describes, neighbour's object samples of this object samples is X ₂, X ₃, X ₇, corresponding reconstruction coefficients is 0.29,0.24,0.28.Reconstruction coefficients is carried out normalization, obtain 0.35,0.30,0.35, therefore, the content of first row of this matrix is:

0 0.35 0.30 0 0 0 0.35 0 0 0 0 0

According to as above operation, the reconstruction coefficients matrix W that obtains is:

0 0.35?0.30 0 0 0 0.35 0 ?0 ?0 ?0 ?0

0.41 0 ?0.32?0.27 0 0 0 0 0 0 0 0

0 0.43 0 0.31?0.26 0 ?0 ?0 ?0 ?0 ?0 ?0

0 0.27?0.31 0 0.42 0 ?0 ?0 ?0 ?0 ?0 ?0

0 0 0.28?0.31 0 0.41 0 0 0 0 0 0

0 0 0 0.29?0.33 0 0 0 0 0 0 0.38

W＝

0.38 0 0 0 0. ?0 0 ?0.33?0.29 0 0 ?0

0 0 0 0 0 0 0.41 0 0.31?0.28 0 ?0

0 0 0 0 0 0 0 0.42 0 0.31?0.27 0

0 0 0 0 0 0 0 0.26?0.31 0 0.43 0

0 0 0 0 0 0 0 0 0.27?0.32 0 0.41

0 0 0 0 0 0.35 0 0 0 0.30?0.35 0

Step 3: regularization obtains the evaluation and test value of input object sample

In the present embodiment, suppose that first and last input object sample are the mark sample, its label is respectively+and 1 and-1.The initial evaluation and test value that is the input object sample is:

Y＝[1 0 0 0 0 0 0 0 0 0 0 -1]。

At first, the reconstruction coefficients matrix W that obtains according to local linear merge module in the nuclear space is compute matrix M as follows, M=(I-W) ^T(I-W), wherein, I is a unit matrix.The result who obtains is following:

M＝ 1.32?-0.76?-0.17 0.11 ?0 ?0 ?-0.73?0.13 0.11 0 0 0

-0.76 1.38?-0.55?-0.41 0.23 ?0 0.12 0 ?0 0 0 0

-0.17?-0.55 1.37?-0.44?-0.42 0.12 0.10 0 ?0 0 0 0

0.11?-0.41?-0.44 1.35?-0.56?-0.16 ?0 0 ?0 0 0 ?0.11

0 0.23?-0.42?-0.56 1.36?-0.74 ?0 0 ?0 0 0 ?0.13

0 0 0.12?-0.16?-0.74 1.29 0 ?0 ?0 0.10 0.12?-0.73

-0.73?0.12?0.10 0.00 ?0 ?0 ?1.29?-0.74?-0.16 0.12 ?0 ?0

0.13 0 0 0.00 ?0 ?0 -0.74 1.36?-0.56?-0.42 0.23 ?0

0.11 0 0 0.00 ?0 ?0 -0.16?-0.56 1.35?-0.44?-0.41 0.11

0 0 0 ?0 ?0 0.104 0.12?-0.42?-0.44 1.37?-0.55?-0.17

0 0 0 ?0 ?0 0.12 0 0.23?-0.41?-0.55 1.38?-0.76

0 0 0 0.11 0.13?-0.73 0 ?0 0.11?-0.17?-0.76 1.32

In this embodiment, the value with parameter μ is made as 0.01.Then, calculate the evaluation and test value of each input object sample, f=u (M+uI) according to following mode ^-1Y.Wherein, I is a unit matrix.

The evaluation and test value of the input object sample that finally obtains is following:

f＝[0.061?0.074?0.071?0.064?0.044?0.005?-0.005?-0.044?-0.064-0.071?-0.074?-0.061]

Fig. 5 is the synoptic diagram of the input object sample evaluation and test value in the present embodiment.In the figure, each input object sample all uses a point to represent, in addition, the gray scale of each point is corresponding to the evaluation and test value of each object samples, and promptly the gray-scale value of data point is close more, and the input object sample of expression correspondence is similar more.

For the pattern classification problem, if with 0 classification boundaries as the evaluation and test value, can obtain following classification results: preceding 6 input object samples belong to the first kind, and back 6 object samples belong to second type.

Embodiment two:

Pattern recognition device of the present invention can be used for recognition image, below, be that example describes with the handwriting digital image recognition.

Pattern recognition device of the present invention be input as 3000 handwriting digital images, all images is gray level image, the gray-scale value of pixel is between 0 and 255, the size of each image is 16 * 16.3000 images of input are under the jurisdiction of between the numeral 0 to 9, and each numeral comprises 300 images, and therefore, present embodiment in fact is the classification problem of one 10 classification, and each classification is corresponding to a numeral.With 300 corresponding samples of each classification in, suppose that preceding 5 are the mark sample, remaining 295 for not marking sample.

Fig. 6 has provided the sample of part handwriting digital.In the present embodiment, directly as the characteristic quantity of input object sample, therefore, each input object sample can be expressed as the vector of one 256 dimension with the gray-scale value of pixel in the input picture.

Similar with the foregoing description one, the implementation procedure of present embodiment is following:

(1) makes up module by Laplce's kernel the input object sample is handled, make up Laplce's kernel

At first, calculate the distance between any two object samples in the input object sample, obtain size and be 3000 * 3000 matrix.Because the input object sample can be expressed as the vector of one 256 dimension, therefore, directly utilize the Euclidean distance of vector to calculate.

Afterwards, utilize k neighbour principle to generate neighbour's matrix A, in the present embodiment, the value of k is made as 10.

According to above-mentioned neighbour's matrix A, obtain Laplce's matrix and normalized Laplce's matrix, thereby, generating Laplce's kernel K, size of nuclear matrix is 3000 * 3000 in this.

(2) in Laplce's kernel spacing, the input object sample being carried out linearity embeds

This process is very similar with the process of above-mentioned embodiment one.Different is that in nuclear space internal linear embedding operation, the value of parameter M is 10.

(3) regularization

Be similar to the process among the embodiment one, the reconstruction coefficients matrix W compute matrix M=(I-W) that obtains according to local linear merge module in the above-mentioned nuclear space ^T(I-W), wherein, I is a unit matrix.

Next, calculating does not mark the probability that sample belongs to each classification respectively.

With numeral 0 first corresponding classification is that example describes.At first, generate the initial evaluation and test value Y of input object sample.Y is that a length is 3000 vector, wherein, and the corresponding input picture of each element.In this vector, the value of the element corresponding with the mark sample of numeral 0 is made as 1, and remaining value is made as 0; Then, calculate each sample and belong to such other evaluation and test value, f ₀=u (M+uI) ^-1Y, wherein, I is that size is 3000 * 3000 unit matrix, the value of parameters u is 5e-4.f ₀Be that a length is 3000 vector, represent the similarity of each input object sample and numeral 0.

With numeral 1 second corresponding classification is that example describes.At first, generate the initial evaluation and test value Y of input object sample.In this vector, the value of the element corresponding with the mark sample of numeral 1 is made as 1, and remaining value is made as 0; Then, calculate each sample and belong to such other evaluation and test value, f ₁=u (M+uI) ^-1Y, the value of parameters u remains unchanged, and still is 5e-4.f ₁The similarity of representing each input sample and numeral 1.

By that analogy, respectively 10 classifications are handled.

For each input object sample, all obtain 10 evaluation and test values, represent the similarity of this object samples and 10 numerals.At a minute time-like, from 10 evaluation and test values, select that maximum evaluation and test to be worth the classification of pairing classification as this input object sample.

Following table has provided the discrimination that the method in pattern recognition device of the present invention and [non-patent literature 2] obtains:

	0	1	2	3	4	5	6	7	8	9	On average
												The method of [non-patent literature 2]	0.91	0.78	0.87	0.65	0.79	0.92	0.72	0.74	0.54	0.94	0.78
The present invention	0.97	0.87	0.90	0.61	0.73	0.97	0.74	0.79	0.49	0.97	0.80

Therefore the discrimination of pattern recognition device of the present invention is than higher.

Embodiment three:

Pattern recognition device of the present invention can be used for carrying out speaker ' s identity identification, and speaker ' s identity identification is basic identical with handwriting digital image recognition among the above-mentioned embodiment.Different is, speaker ' s identity identification is based on that speaker's voice signal carries out, and, the corresponding classification of each speaker.

At first, from voice signal, extract 4 kinds of characteristics such as cepstrum coefficient, difference cepstrum coefficient, fundamental frequency and difference fundamental frequency, and describe characteristic as the digitizing of input speech signal with the vector that these 4 kinds of characteristics are formed.

Afterwards, carry out processes such as Laplce's kernel structure, the embedding of nuclear space internal linear, regularization respectively, identify speaker's identity.The process that these processes and front are described to Handwritten Digital Recognition is similar, no longer is described in detail at this.

Each composition module of the pattern recognition device of the invention described above can be implemented in the computing machine or single-chip microcomputer equipment with computer program of realizing said function integratedly, also can constitute by each signal conditioning package by separation.

Though, in the above embodiments, the present invention is specified with Handwritten Digital Recognition and speaker ' s identity identification, the invention is not restricted to these, and can be widely used in the various pattern-recognition occasions.

Further, based on embodiments of the invention, the object of the invention can also be realized through the computer program that makes execution aforesaid operations such as computer or single-chip microcomputer.

In addition; Should be understood that in each embodiment, (for example can pass through special circuit or circuit; Interconnection is to carry out the discrete logic gate of dedicated functions), the programmed instruction through carrying out by one or more processor, perhaps carry out said each action through both combinations.Therefore, can implement above-mentioned many aspects through multiple different form, and all these forms considered to be in all in the scope of the content of describing.For in the above-mentioned many aspects each, the embodiment of any this form can refer to " being configured the logic that is used for carrying out said action " at this, perhaps alternatively, is meant " logic of carrying out or can carry out said action ".

Further, according to embodiments of the invention, the object of the invention can also be realized by computer-readable medium, the program that said medium memory is above-mentioned.Computer-readable medium can be can comprise, storage, reception and registration, propagation or convey program, with by executive system, equipment or device any device that use or that and instruction executive system, equipment or device combine.This computer-readable medium for example can be, but be not limited to electronics, magnetic, light, electromagnetism, infrared or semiconductor system, unit or propagation medium.The example more specifically of this computer-readable medium (non-exclusive list) can comprise: have one or more electrical connection, portable computer diskette, random-access memory (ram), ROM (read-only memory) (ROM), Erasable Programmable Read Only Memory EPROM (EPROM or flash memory), the optical fiber of multiple conducting wires, and portable optic disk ROM (read-only memory) (CDROM).

The above-mentioned explanation of the embodiment of the invention only is used for illustration and illustrative purposes, but not is intended to limit the present invention or it is limited to disclosed concrete form.It will be appreciated by those skilled in the art that; Embodiment selected and that describe only is in order to explain principle of the present invention and practical application thereof best; To be suitable for specific intended use; And under situation about not breaking away from, can carry out various modifications and modification to the present invention claim and the invention scope that equivalent limited thereof.

Claims (8)

1. pattern recognition device; This pattern recognition device utilizes semi-supervised learning mechanism to come the schema object sample of input is discerned; Wherein said schema object sample comprises the mark sample and does not mark sample; And represent that with digitalized signature said mark sample comprises the digitalized signature and the corresponding desired value of schema object

Said pattern recognition device comprises:

Non-Euclidean space kernel structure portion, its k neighbour matrix according to the schema object sample of all inputs makes up nuclear matrix in the non-Euclidean space, and this non-Euclidean space is suitable for the flow structure of structural model object;

Local linear Embedded Division in the nuclear space, nuclear matrix in its non-Euclidean space that constructs according to said non-Euclidean space kernel structure portion carries out to the schema object sample that the part is linear to embed, with the reconstruction coefficients matrix of generate pattern object samples; Wherein, nuclear matrix is fallen into a trap and is calculated the schema object sample distance between any two of all inputs in said non-Euclidean space; According to calculating the distance that obtains,, find out M the object samples nearest with the schema object sample of this input to the schema object sample of each input; Utilize neighbour's object of the schema object sample of each input to come the schema object sample of this input of linear-apporximation, to obtain the reconstruction coefficients of each input object sample; Carry out normalization to obtaining reconstruction coefficients, with the reconstruction coefficients matrix of generate pattern object samples; And

Regularization portion, it comes to generate the evaluation and test value for the schema object sample of all inputs according to the reconstruction coefficients matrix of local linear Embedded Division generation in the said nuclear space and the desired value of said mark sample.

2. pattern recognition device as claimed in claim 1 is characterized in that, the inherent geometry of the reconstruction coefficients matrix representation schema object sample of the schema object sample that local linear Embedded Division generates in the said nuclear space.

3. pattern recognition device as claimed in claim 1 is characterized in that, said non-Euclidean space is Laplce space or reproducing kernel Hilbert space.

4. like each the described pattern recognition device among the claim 1-3, this pattern recognition device is used for recognition image or audio frequency, and wherein, schema object is the characteristic quantity that can be used for carrying out pattern-recognition of image or audio frequency.

5. mode identification method; This mode identification method utilizes semi-supervised learning mechanism to come the schema object sample of input is discerned; Wherein said schema object sample comprises the mark sample and does not mark sample; And represent that with digitalized signature said mark sample comprises the digitalized signature and the corresponding desired value of schema object

Said mode identification method may further comprise the steps:

K neighbour matrix according to the schema object sample of all inputs makes up nuclear matrix in the non-Euclidean space, and this non-Euclidean space is suitable for the flow structure of structural model object;

According to nuclear matrix in the non-Euclidean space that constructs, the schema object sample is carried out the linear embedding in part, with the reconstruction coefficients matrix of generate pattern object samples; Comprising: nuclear matrix is fallen into a trap and is calculated the schema object sample distance between any two of all inputs in said non-Euclidean space; According to calculating the distance that obtains,, find out M the object samples nearest with the schema object sample of this input to the schema object sample of each input; Utilize neighbour's object of the schema object sample of each input to come the schema object sample of this input of linear-apporximation, to obtain the reconstruction coefficients of each input object sample; Carry out normalization to obtaining reconstruction coefficients, with the reconstruction coefficients matrix of generate pattern object samples; And

According to the reconstruction coefficients matrix that generates and the desired value of said mark sample, come to generate the evaluation and test value for the schema object sample of all inputs.

6. mode identification method as claimed in claim 5 is characterized in that, the inherent geometry of the reconstruction coefficients matrix representation schema object sample of said schema object sample.

7. mode identification method as claimed in claim 5 is characterized in that, said non-Euclidean space is Laplce space or reproducing kernel Hilbert space.

8. like each the described mode identification method among the claim 5-7, this mode identification method is used for recognition image or audio frequency, and wherein, schema object is the characteristic quantity that can be used for carrying out pattern-recognition of image or audio frequency.

Images