CN101520847A - Pattern identification device and method - Google Patents

Abstract

The invention provides a pattern identification device and a pattern identification method; wherein the pattern identification device adopts a semi-supervised learning mechanism, analyzes the structural characteristic of a pattern object sample and carries out linear embedding analysis to the pattern object sample in a nuclear space, thus achieving the purposes of classification and sorting. Particularly, the pattern identification device adopts a local linear embedding mechanism for estimating the flow structure of the pattern object sample, namely, for any pattern object sample, the linear combination of the neighboring object samples thereof are used for carrying out reconstruction, i.e., linear embedding. The reconstruction coefficients of all the object samples are combined to better express the flow structure of the whole input object sample. The invention carries out reconstruction and embedding to the pattern object sample in the nuclear space, thus better reflecting the structural characteristic of a data set.

Description

Pattern recognition device and method

Technical field

The present invention relates to pattern recognition device and method, more particularly, thereby the present invention relates to adopt semi-supervised learning mechanism in nuclear space, schema object to be carried out the linear apparatus and method of describing implementation pattern identification that embed.

Background technology

For purposes such as implementation pattern identification, ordering and retrievals, the object samples that traditional method often relies on some to mark trains, learn obtaining an objective function, thereby realizes above-mentioned function.For instance, in Handwritten Digital Recognition, need collect the image of some handwritten numerals in advance, then these image patterns be handled and learnt, thereby obtain a classification function.For example: at first image is carried out binary conversion treatment, extract stroke direction and structural information feature then, judge the classification of numeral at last according to these features, promptly generate classification function and carry out discriminator.In this learning process, need inform the actual value of the numeral in each image, promptly need these image patterns are marked.Classification function be input as handwritten numeral image (or characteristic of correspondence), be output as the value of the numeral in this image.Like this, when the new handwritten numeral image of one of needs identification, directly this handwritten numeral image is imported above-mentioned classification function, can obtain the value of numeral wherein.

Yet in actual applications, obtaining the mark sample often needs expensive time and efforts, and has only the personnel of specialty just can finish the task of obtaining the mark sample.For example, in biological and computational science field, for protein is classified, need obtain the shape sample of protein learns in advance, yet even the crystalline solid analyst of specialty, the time that also needs to spend some months just can be obtained a protein sample.In contrast, do not mark sample and often exist in a large number, and be easy to obtain.Therefore, how will not mark sample and combine, and therefrom extract Useful Information, just become a vital task in machine learning field with the mark sample.

Mark sample and the difference that does not mark sample are the desired value that whether provides corresponding with it.The so-called sample that do not mark only is a processed object, and the mark sample then not only comprises processed object, also comprises the desired value of this processed object.For example, in Handwritten Digital Recognition, not marking sample is exactly a handwriting digital image, and the mark sample also comprises the actual value of the numeral in this image except comprising this image.

The method that relies on the mark sample to carry out pattern classification can be divided into two classes, that is: supervised learning and semi-supervised learning.Supervised learning is meant and only relies on the mark sample to learn, train, and therefrom obtains classification function; Semi-supervised learning is meant simultaneously and carries out learning mechanism and method the sample from marking sample and not marking.The basic ideas of semi-supervised learning are: though do not mark the desired value that does not comprise object in the sample, thereby can not image scale annotate sample and directly learn like that and train, yet these do not mark and have comprised some useful informations about the object samples space distribution in the sample.If these useful informations can be extracted, and the mark sample combines together, just can help the performance that improves classification, discerns.A given pattern recognition problem, if can provide abundant mark sample to learn, train, the supervised learning method also can reach preferable performance so; Yet if less, for example above-mentioned protein classification problem of mark sample, the supervised learning method is often failed.In contrast, because the semi-supervised learning method can never mark and extract Useful Information in the sample, therefore, can improve the performance of identification greatly.

In semi-supervised learning mechanism, given one by the mark sample with do not mark the sample set that sample forms and (be called part and mark sample set, in general, this set comprises a spot of mark sample and a large amount of not mark samples) after, at first analyze the immanent structure of these samples, obtain its spatial distribution characteristic; Then, learn based on the space distribution and the mark sample of sample; At last, do not classify, sort marking sample.

In actual applications, the schema object sample of input often comprises a lot of variablees, and promptly these samples are arranged in a higher dimensional space.For example in Handwritten Digital Recognition, the object samples of input is image, and therefore, the dimension of object samples luv space just can be seen the pixel count in the image as.Yet the dimension of the internal space of these image patterns is far below the dimension of its luv space.With numeral 0 is example, if be similar to this numeral with ellipse, the dimension of its internal space is 4, i.e. the radius of centre coordinate and major and minor axis.Consider the distortion and the oval approximate factors such as distortion of handwritten numeral, the inherent dimension of the image pattern corresponding with digital 0 will be higher than 4, yet this value still will be far smaller than the pixel count in the image.Based on this phenomenon, semi-supervised learning mechanism is generally all supposed: these data samples in (1) higher dimensional space have inherent lower dimensional space structure, and, in this lower dimensional space, obey flow structure and distribute, so-called flow structure distributes, and the distribution that is meant these samples is level and smooth; (2) on this flow structure, neighbour's sample point generally has identical classification or label.

In recent years, a lot of semi-supervised learning methods have been proposed, thereby to reach the purpose that never marks information extraction raising performance in the sample.Laplce's intrinsic figure (Laplacian Eigenmap) is a kind of more representational semi-supervised learning method, referring to [non-patent literature 1].In the method, at first utilize Euclidean distance and k near neighbor method between sample point (all samples comprise the mark sample and do not mark sample) to make up an adjacent map, obtain its Laplce's matrix thus.So-called adjacent map is exactly to express relation between sample point with the mode of figure, and the node of figure is corresponding to sample point, and the limit of figure is definite by the k near neighbor method.So-called k near neighbor method is exactly to each sample, finds out preceding k the sample nearest with its Euclidean distance, in adjacent map, is this sample and its preceding k neighbour's sample interpolation limit, and the property value on limit is the Euclidean distance between sample.After setting up adjacent map, can obtain its Laplce's matrix at an easy rate, afterwards this matrix is carried out characteristic value decomposition.At last, according to the less proper vector of eigenwert and the sample of mark, the not mark sample that data are concentrated is classified.From in essence, this method has three important characteristics: the data set of (1) this method hypothesis input is to be embedded in the luv space of higher-dimension with flow structure, and promptly the dimension of the internal space of these data is lower; (2) adopt all data (promptly comprise the mark sample and do not mark sample) to estimate its flow structure; (3) utilize Laplacian Beltrami operator to realize the mapping of higher dimensional space, and finish the purpose of classification to lower dimensional space.

Local neighborhood Laplce's intrinsic drawing method (referring to [patent documentation 1]) in fact is the extension to the Laplce's intrinsic drawing method in [non-patent literature 1].In the method for [non-patent literature 1], make up Laplce's matrixes according to all sample points in the sample set, and carry out characteristic value decomposition, therefore, when sample point more for a long time, this method is more time-consuming.In order to address this problem, a submatrix only treating classification samples point place in [patent documentation 1] carries out characteristic value decomposition, thereby, can improve speed significantly.Meanwhile, the method in [patent documentation 1] has also solved the classification problem of new sample point.

The flow pattern ordering is another kind of common semi-supervised learning method, referring to [non-patent literature 2].The main thought of this method is that the immanent structure according to sample point sorts to sample point.Similar with [non-patent literature 1], this method at first makes up k neighbour figure according to the Euclidean distance between sample point, afterwards, the desired value of mark sample is carried out repetitive propagation along this neighbour figure, all at last sample points all will obtain an evaluation of estimate, and these evaluations of estimate are exactly final sort by.The principle of this method can be explained with a circuit network: as noted earlier, the relation of sample point can represent that the node of adjacent map is corresponding to sample point with adjacent map, and the limit of figure is determined by the k near neighbor method; Below, see this adjacent map as a circuit network, node corresponding with the mark sample among the figure links to each other with electrode respectively, and positive sample connects positive electrode, and negative sample connects negative electrode, resistor is seen on the limit of figure as, the property value on limit (Euclidean distance between sample) is corresponding to the resistance value of resistor, and in other words, the Euclidean distance between sample is more little, resistance between corresponding node is just more little, and vice versa; After opening power supply, circuit network reaches equilibrium state, so, do not mark on the node of sample correspondence at each, can measure a magnitude of voltage, magnitude of voltage is high more, illustrate that this sample approaches positive sample more, magnitude of voltage is low more, illustrates that this sample approaches negative sample more.In concrete enforcement, [non-patent literature 2] adopted figure regularization technology to realize said process.

Proposed to utilize Gaussian field harmonic function (Gaussian Fields andHarmonic Functions) to carry out the method for semi-supervised learning in [non-patent literature 3], this method in fact belongs to same category with the method in [non-patent literature 2], it is the realization means difference of figure regularization, in the method, push away regularization (Interpolated regularizat ion) in having adopted.[non-patent literature 4] analyzed these two kinds of diverse ways, and drawn its error upper limit theoretically.Under the framework of figure regularization, [non-patent literature 5] revised the objective function of optimizing, and introduces relaxation factor, thereby the figure regularization changed into the form of standard support vector machine.

Similar with above-mentioned method, existing semi-supervised learning method is mostly set up neighbour figure (neighbour's matrix) according to the Euclidean distance of sample point, and suppose that this neighbour figure can describe the inherent flow structure of these sample points fully, the Euclidean distance that is about to sample point in the luv space is equal to " measuring distance " on the flow structure.Yet this hypothesis is devious under many circumstances, and is accurate inadequately in other words conj.or perhaps.[non-patent literature 6] and [patent documentation 5] all sets forth this problem.Fig. 1 (a) and Fig. 1 (b) show this problem with concrete example.Fig. 1 (a) is a flow structure of the shape that is similar to " S " in the three dimensions, and Fig. 1 (b) is the corresponding sampled point of this flow structure, i.e. sample set.In the figure, represent the desired value of sampling point with gray scale, promptly the gray-scale value of data sample is close more, represent they similar more (except two end points places of serpentine shape).Two data point A, B in this space, its Euclidean distance shown in the solid line among Fig. 1 (b), and these two points at the measuring distance on the flow structure shown in the dotted line among 1 (b).As seen from the figure, it is inaccurate coming the inherent flow structure of expression data point with Euclidean distance.Therefore, this hypothesis of inherent flow structure that the neighbour figure that makes up based on Euclidean distance can describe sample point fully is inaccurate, thereby the existing technical method that is implemented on this hypothesis all is defective, and its performance also is limited.

[patent documentation 1] US patent application gazette US 2006/0235812 A1

[patent documentation 2] US patent application gazette US 2006/0045353 A1

[non-patent literature 1] Mikhail Belkin, Partha Niyogi, Semi-SupervisedLearning on Rimanian Mani folds, Machine Learning, Vol.56, pp.209-239,2004

[non-patent literature 2] Dengyong Zhou, Olivi er Bousquet, Thomas Navi Lal, Jason Weston, and Bernhard Scholkopf, Learning with Local andGlobal Consistency, Advances in Neural Information ProcessingSystems, Vol.16, pp.321-328,2003

[non-patent literature 3] Xiaojin Zhu, Zoubin Ghahramani, John Lafferty, Semi-Supervised Learning Using Gaussian Fields and HarmonicFunctions, in Proceedings of the twentieth internationalconference on machine learning (ICML2003), pp.912-919,2003

[non-patent literature 4] Mikhail Belkin, Irina Matveeva, Partha Niyogi, Regularization and Semi Supervised Learning on Large Graphs, inProceedings of annual conference on learning theory (COLT), 2004

[non-patent literature 5] Zhili Wu, Chun-hung Li, Ji Zhu, Jian Huang, ASemi-Supervised SVM for Manifold Learning, in Proceedings of the18 ^ThInternational conference on pattern recognition (ICPR ' 06), Vol.2, pp.490-493

[non-patent literature 6] Joshua B.Tenenbaum, Vinde Silva, John C.Langford, A global geometric framework for nonlineardimensionality reduction, Science, Vol.290, pp.2319-2323,2000

[non-patent literature 7] Sam T.Roweis, Lawrence K.Saul, Nonlineardimensionality reduction by locally linear embedding, Science, Vol.290, pp.2323-2326,2000

Summary of the invention

In view of the problems referred to above of the prior art the present invention has been proposed.Purpose of the present invention is to provide a kind of at least and comes pattern is carried out the apparatus and method of Classification and Identification based on semi-supervised learning mechanism, and it can overcome the problem of the inherent flow structure that the neighbour figure that makes up based on Euclidean distance can not the perfect representation sample point.

According to the present invention, the input pattern object comprises mark simultaneously and does not mark sample, and based on these samples, the spatial distribution characteristic of described device analysis sample is also finished not marking sample identification (classification and ordering).A feature of this device is to adopt local linear embedding to estimate the flow structure of input object in nuclear space, and based on this, utilizes the figure regularization to realize the function of identification.

At first, calculate input object sample distance between any two, promptly calculate the distance between any two input object samples.Distance calculation is relevant with the form and the kind of input object sample, in pattern recognition problem, generally need at first the input object sample to be handled, extract its principal character, for example: in Handwritten Digital Recognition, feature can be input object sample itself, and promptly the value of image pixel also can be the stroke direction histogram; And in speaker ' s identity identification, feature can be the cepstrum coefficient that utilizes voice signal to obtain, fundamental frequency etc.If the feature of input object sample is represented as the form of vector, then can be with the distance between Euclidean distance or other distance measure calculating object samples; If the input object sample is represented as structurized feature, then need to utilize the distance calculation mode corresponding with this feature.

Afterwards,, utilize k neighbour mode to set up k neighbour figure, in other words, represent relation between the input object sample with the mode of figure according to the above-mentioned input object sample distance between any two that obtains.

Next, scheme to make up reproducing kernel Hilbert space (ReproducingKernel Hilbert Space) according to above-mentioned neighbour, for example: Laplce's nuclear space (Laplacian KernelSpace), and in this space, carry out the linear embedding in part.Further, be exactly to each sample, all utilize its neighbour's sample in nuclear space to carry out linear reconstruction, and the reconstruction coefficients of all samples just can well be expressed the internal space distribution characteristics of sample.

At last, utilize the desired value of above-mentioned reconstruction coefficients and mark sample, adopt figure regularization technology, give and do not mark evaluation of estimate of sample.And these evaluations of estimate are exactly to not marking the foundation that sample is discerned.

According to an aspect of the present invention, a kind of pattern recognition device is provided, this pattern recognition device utilizes semi-supervised learning mechanism to come the schema object sample of input is discerned, wherein said schema object sample comprises the mark sample and does not mark sample, and represent with digitalized signature, described mark sample comprises the digitalized signature and the corresponding desired value of schema object, described pattern recognition device comprises: non-Euclidean space kernel structure portion, its schema object sample based on input makes up nuclear matrix in the non-Euclidean space, and this non-Euclidean space is suitable for the flow structure of structural model object; Local linear Embedded Division in the nuclear space, nuclear matrix in its non-Euclidean space that constructs according to described non-Euclidean space kernel structure portion is carried out the linear embedding in part to the schema object sample, with the reconstruction coefficients matrix of generate pattern object samples; And regularization portion, it comes to generate the evaluation and test value for the schema object sample of all inputs according to the reconstruction coefficients matrix of local linear Embedded Division generation in the described nuclear space and the desired value of described mark sample.

According to another aspect of the present invention, a kind of mode identification method is provided, this mode identification method utilizes semi-supervised learning mechanism to come the schema object sample of input is discerned, wherein said schema object sample comprises the mark sample and does not mark sample, and represent with digitalized signature, described mark sample comprises the digitalized signature and the corresponding desired value of schema object, described mode identification method may further comprise the steps: the schema object sample based on input makes up nuclear matrix in the non-Euclidean space, and this non-Euclidean space is suitable for the flow structure of structural model object; According to nuclear matrix in the non-Euclidean space that constructs, the schema object sample is carried out the linear embedding in part, with the reconstruction coefficients matrix of generate pattern object samples; And, come to generate the evaluation and test value for the schema object sample of all inputs according to the reconstruction coefficients matrix that generates and the desired value of described mark sample.

Above-mentioned pattern recognition device of the present invention and mode identification method, owing to adopted in the non-Euclidean space of the inherent flow structure that is suitable for making up the schema object sample nuclear matrix (for example, nuclear matrix in the reproducing kernel Hilbert space, more specifically, be nuclear matrix in the Laplce space) come the schema object sample is carried out the linear embedding in part, thereby the inherent flow structure of expression pattern object samples exactly, and finally realize pattern-recognition accurately.

Description of drawings

Fig. 1 (a) illustration an example of flow structure.

Fig. 1 (b) illustration corresponding to the sampled point of the flow structure among Fig. 1 (a), and the example of measuring distance on Euclidean distance and the flow structure.

Fig. 2 is the schematic block diagram according to the pattern recognition device of embodiment of the present invention.

Fig. 3 makes up the process flow diagram of module for Laplce's kernel.

Fig. 4 is the synoptic diagram according to the input object in the embodiments of the invention.

Fig. 5 is the evaluation and test value synoptic diagram according to the input object in the embodiments of the invention.

Fig. 6 is the part input object according to the Handwritten Digital Recognition in the embodiments of the invention.

Embodiment

Fig. 2 is the schematic block diagram according to the pattern recognition device of embodiment of the present invention.

As shown in the figure, this pattern recognition device comprises: Laplce's kernel makes up module, and it makes up Laplce's kernel spacing matrix according to input object sample (comprise the mark sample and do not mark sample); Local linear merge module in the nuclear space, it makes up Laplce's kernel spacing matrix that module construction goes out according to Laplce's kernel, the input object sample is carried out the linear embedding in part, with the reconstruction coefficients matrix of formation object sample; And the regularization module, it is that all input object samples generate the evaluation and test value according to local linear merge module in the nuclear space reconstruction coefficients matrix that generates and the desired value that marks sample.

Except above-mentioned Laplce's kernel spacing, this pattern recognition device can also adopt other reproducing kernel Hilbert space.The purpose that makes up this space is to describe preferably the flow structure of input object.

Input object is processed object set, for example the image in the image recognition, the voice signal in the speech recognition etc.Input object is generally represented in the mode of numerical characteristic, this numerical characteristic is made up of a plurality of variablees, in order to describe this Properties of Objects, for example in image recognition, this feature can be the color histogram of each color of pixel value, image texture features vector, image in the image etc.; In Speaker Identification, this feature can be the cepstrum coefficient that obtains according to voice signal, fundamental frequency etc.Input object includes the mark sample simultaneously and does not mark sample.So-called mark sample, promptly except with this sample characteristic of correspondence, also import a desired value corresponding with this sample, for example in two classification image recognitions, for the image of the first kind, its desired value can be set to 1, and for the image of second class, its desired value can be set to-1; In Speaker Identification, desired value can be numbering of speaker etc.And for non-mark sample, then only input and this sample characteristic of correspondence.

Laplce's kernel makes up module and be used for making up Laplce's kernel spacing from the input object sample.At first, calculate input object sample distance between any two.Afterwards, make up neighbour figure, calculate Laplce's space matrix according to the k near neighbor method.At last, obtain Laplce's kernel spacing matrix.

Local linear merge module is implemented in the inherent geometry of describing input object in Laplce's kernel spacing in the nuclear space.Further, have on the data set of flow structure, any one infinitesimal zone all is linear, therefore, for each input object sample, can utilize the linear combination of its adjacent sample to come reconstruct (or approximate), these reconstruction coefficients have then reflected the geometry feature of input object collection.

The regularization module is used for calculating the evaluation and test value that does not mark object samples.According to the reconstruction coefficients that local linear merge module in the above-mentioned nuclear space obtains, each object samples can be similar to by its linear combination of neighbour's sample, and therefore, the evaluation and test value of object samples also can be similar to according to these reconstruction coefficients.Based on this principle, under the framework of figure regularization, be optimized calculating, do not marked the evaluation and test value of sample.

The evaluation and test value that obtains from above-mentioned regularization module has reflected the similarity degree that does not mark between sample and the mark sample.For example, in the identification of two quasi-modes, the desired value of the mark sample of first kind pattern is set to 1, and the desired value of the mark sample of second quasi-mode is set to-1, and so, the evaluation and test value that does not mark sample approaches 1 more, it is big more to represent that this sample belongs to the probability of first kind pattern, on the contrary, approach-1 more, it is big more to represent that this sample belongs to the probability of second quasi-mode.

Specifically describe each composition module of pattern recognition device below.

1, Laplce's kernel makes up module

Make that input object is X={X _L, X _U.Wherein, X _L={ (x ₁, y ₁) ..., (x ₁, y ₁) for marking sample set, X ₁..., X _lExpression marks the numerical characteristic of sample respectively, 1 number for the mark sample, y ₁..., y ₁Desired value for these samples; X _U={ x _L+1..., x _nFor not marking sample set, X _L+1..., X _nExpression does not mark the numerical characteristic of sample respectively, total n-1 object samples, and n is the sum of input sample.At the remainder of instructions, all write down the input object sample set by this way.

Fig. 3 is the schematic block diagram of the formation of Laplce's kernel structure module, and it is made of three modules: neighbour's matrix computations module, Laplce's matrix computations module, and Laplce's kernel computing module.

Neighbour's matrix computations module generates neighbour's matrix according to input object sample distance between any two.The size of neighbour's matrix is n * n (n is a natural number), is used for describing the similarity between any two object samples.

At first, for any two input object samples, calculate the distance between them.The account form of distance is relevant with the expression way of input object sample, for example: if the input object sample represent with vector mode, just can adopt Euclidean distance to estimate and calculate; If object samples represents with structured features, just need be according to the mode of the concrete format design distance calculation of this feature.Any given two object samples X _i, X _j, the distance between them be designated as d (i, j).

Next, to each input object sample, find out a nearest with it k object samples.K is for realizing a parameter of pattern recognition device of the present invention, and value is a positive integer, can rule of thumb set.

If neighbour's matrix is A=[a _Ij], i, j=1,2 ..., n.Element a in neighbour's matrix A _IjValue calculate as follows: provide two object samples X arbitrarily _i, X _jIf, X _jBelong to and X _iIf one of nearest k sample is perhaps X _iBelong to and X _jOne of nearest k sample, then a _Ij=a _Ji=exp (d (i, j) ²/ 2 σ ²), otherwise, a _Ij=0; In addition, if the value of i and j is identical, a _Ij=0.σ is for realizing another parameter of pattern recognition device of the present invention, and it has reacted the category of the distance of input object sample, can be set at the mean value of the distance between all object samples, perhaps rule of thumb is provided with.

Obtain after neighbour's matrix, Laplce's matrix computations module generates Laplce's matrix according to neighbour's matrix.Make that D is the diagonal matrix of n * n, the value of each element is on the diagonal line: $D_{\mathrm{ii}}=\underset{j}{\mathrm{\Σ}}{a}_{\mathrm{ij}},$ So, Laplce's matrix L can be calculated by following formula: L=D-A.Laplce's matrix after the normalization is: $\tilde{L}=D^{-\frac{1}{2}}{\mathrm{LD}}^{-\frac{1}{2}}=I-D^{-\frac{1}{2}}{\mathrm{AD}}^{-\frac{1}{2}},$ Wherein, I is the unit diagonal matrix.

Regularization Laplce matrix is: $P=r\left(\tilde{L}\right)=\tilde{L}+\mathrm{\ϵ}\·I,$ Wherein, I is the unit diagonal matrix, and ε is for realizing a parameter of pattern recognition device of the present invention, and general value is very little, for example gets 10 ^-5

Laplce's kernel computing module promptly obtains Laplce's kernel K to the above-mentioned regularization Laplce matrix inversion that obtains.If P is irreversible for regularization Laplce matrix, then ask its pseudo inverse matrix.

Except Laplce's kernel, pattern recognition device of the present invention can also adopt other reproducing kernel Hilbert space, for example: diffusion kernel, counter-rotating cosine kernel etc.

The building process of diffusion kernel is as follows:

(1) identical with above-mentioned Laplce's kernel building process, utilize neighbour's matrix computations module to generate neighbour's matrix A;

(2) identical with above-mentioned Laplce's kernel building process, utilize Laplce's matrix computations module to generate normalized Laplce's matrix according to neighbour's matrix

(3) according to above-mentioned normalized Laplce's matrix

, obtain diffusion kernel K _D, computation process is: $K_D=\exp (-{\text{\τ}}^2/2\tilde{L}),$ τ is for realizing a parameter of pattern recognition device of the present invention.

The building process of counter-rotating cosine kernel is as follows:

(1) identical with above-mentioned Laplce's kernel building process, utilize neighbour's matrix computations module to generate neighbour's matrix A;

(2) identical with above-mentioned Laplce's kernel building process, utilize Laplce's matrix computations module to generate normalized Laplce's matrix according to neighbour's matrix

(3) according to above-mentioned normalized Laplce's matrix

, cosine kernel K obtains reversing _C, computation process is: $K_C=\cos (\tilde{L}*\mathrm{\π}/4).$

Given regularization Laplce matrix P can define inner product and be＜f f 〉 _H=＜f, Pf 〉, H is exactly a reproducibility Hilbert kernel spacing so, and its kernel be exactly k (i, j)=[P ^-1] _Ij

In order to prove its reproducibility, need satisfy following condition:

f(i)＝<f，k(i，·)> _H

In other words, the f ∈ H to all should have:

F (i)=＜f, k (i) 〉 _H=f ^TPK _I,In other words, f ^T=f ^TPK

Obviously, if K=P ^-1, then above-mentioned condition just can satisfy.Therefore, above-mentioned regularization Laplce inverse of a matrix matrix is exactly Laplce's kernel.

2, local linear merge module in the nuclear space

Local linear embedding (Locally linear embedding is abbreviated as LLE) is proposed by people such as Sam T.Roweis the earliest, referring to [non-patent literature 7].Its basic thought is: to each object samples, utilize its neighbour to put and make up a local linear model.On the data set of flow structure, any one infinitesimal zone all is linear, therefore, any one data point can be come reconstruct with its linear combination of adjoint point, promptly linear the embedding, and the reconstruction coefficients of all sample points is combined, and just can describe the flow structure of this data set.LLE is applied among the feature dimensionality reduction the earliest.People such as Sam T.Roweis notice, keeping such as rotation, translation, convergent-divergent etc. in the conversion of neighbor relationships, the reconstruction coefficients that obtains from LLE has unchangeability, therefore, the neighbor relationships of sample point in the flow structure space of low-dimensional is consistent with the neighbor relationships in its luv space, in other words, the reconstruction coefficients that obtains from luv space is suitable equally in the flow structure space.Find that based on this LLE can well be used for finishing the purpose of feature dimensionality reduction.

Local linear embedding of pattern recognition device of the present invention carried out in kernel spacing, specifically, carries out in above-mentioned Laplce's kernel spacing.

Given set of data samples X={x ₁..., x _n(this sample set comprises the mark sample and do not mark sample) and Laplce's kernel function K, kernel function K has in fact defined a mapping function φ on sample set X, and by this function, data sample is projected among the space F of a higher-dimension, that is: φ: x _i→ φ (x _i), this mapping is satisfied following inner product and is concerned: k (x, x ')=＜φ (x), φ (x ') 〉.

Provide two sample φ (x in the F space arbitrarily _m) and φ (x _n), their Euclidean distance is:

‖φ(x _m)-φ(x _n)‖ ²＝k(m，m)+k(n，n)-2k(m，n)

In higher dimensional space F,, all use the linear combination of its neighbour's sample to be similar to i.e. reconstruct for each sample.The total error of this reconstruct is:

ε(W)＝∑ _i‖φ(x _i)-∑ _jw _ijφ(x _j)‖ ²，φ(x _j)∈N(φ(x _i))

In following formula, N (φ (x _i)) expression sample X _iNeighbour's sample in the F of space, W _IjExpression sample X _iReconstruction coefficients.

With sample x _iBe example, suppose sample X in higher dimensional space F _iM neighbour's sample be

, by this M neighbour's sample to X _iCoefficient when being reconstructed is W _i=[w _I1..., w _IM] ^T, reconstructed error is so:

$\mathrm{\&epsiv;}={\left|\right|\mathrm{\&phi;}\left(x_i\right)-{\mathrm{\&Sigma;}}_j{w}_{\mathrm{ij}}\mathrm{\&phi;}\left(x_i^l\right)\left|\right|}^2=W_i^T{C}_i{W}_i$

Wherein, C _iBe called as φ (x _i) the Gram matrix, its size is M * M, the value of each element is in the matrix:

$C_i(m,n)={(\mathrm{\&phi;}\left(x_i^{\&prime;}\right)-\mathrm{\&phi;}\left(x_i^m\right))}^T(\mathrm{\&phi;}\left(x_i\right)-\mathrm{\&phi;}\left(x_i^n\right))$

$=k(x_i^m,x_i^n)-k(x_i,x_i^n)-k(x_i^m,x_i)+k(x_i,x_i)$

Obviously, make the reconstruction coefficients of reconstructed error minimum be:

$W_i=\frac{C_i^{-1}\stackrel{\&RightArrow;}{1}}{{\stackrel{\&RightArrow;}{1}}^T{C}_i^{-1}\stackrel{\&RightArrow;}{1}}$

The input and output of the interior local linear merge module of nuclear space and performed key step are as follows:

Input: set of data samples X, Laplce's kernel K.

Output: the reconstructed coefficients matrix W of each data sample.

To each data sample x _i, all carry out following operation:

(1) calculates among the higher dimensional space F this sample with the Euclidean distance between other sample, that is:

‖φ(x _i)-φ(x _j)‖ ²＝k(i，i)+k(j，j)-2k(i，j)，j＝1，...，n

(2), find out and x according to above-mentioned distance _iA nearest M sample,

M is another parameter that realizes pattern recognition device of the present invention, and value is a positive integer, can get same value with the parameter k that above-mentioned Laplce's kernel makes up in the module.

(3) calculating size is the Gram Matrix C of M * M _i

(4) according to the Gram Matrix C _iCalculate reconstruction coefficients W _i=[w _I1..., w _IM] ^T

(5) reconstruction coefficients is carried out normalization, that is: make W _iMould be 1.

(6) generate the reconstruction coefficients matrix W.The size of this matrix is n * n, and each row of matrix is corresponding to the reconstruction coefficients of a sample, for preceding M nearest sample, is worth and is W _iIn analog value, otherwise value is 0.

3, regularization module

Given input object sample X={x ₁..., x _l, x _L+1..., x _n, and with mark sample corresponding desired value { y ₁..., y ₁, the regularization module is used for generating the evaluation and test value into these samples.In other words, obtain a mapping function f:X → R from the regularization module, this function is mapped as the evaluation and test value of a real number with the input object sample, and this function can be expressed as the form of vector, that is:

f＝{f ₁，…f _n}

Local linear merge module has generated a series of reconstruction coefficients for the input object sample in the above-mentioned nuclear space, according to these coefficients, each sample can be similar to by its linear combination of neighbour's sample, in addition, these coefficients have also reflected the inherent geometry of object samples, and these coefficients have unchangeability in the conversion that keeps neighbor relationships.Therefore, these coefficients can be used for the evaluation and test value of reconstruct object samples, and in other words, based on these coefficients, the evaluation and test value of each object samples can be similar to by its linear combination of evaluation and test value of neighbour's sample.

The error that the evaluation and test value of sample is reconstructed is:

Wherein, W _IjExpression local linear merge module in the above-mentioned nuclear space obtain with object samples X _iRelevant reconstruction coefficients.

In addition, for the mark sample, the evaluation and test value that obtains from the regularization module should be more or less the same with their desired value.Make y=[y ₁..., y _n] ^T, wherein, { y ₁..., y ₁For marking the desired value of sample, { y ₀₊₁..., y _nValue be 0, then this constraint condition can be expressed as: τ (f)=∑ _i(f _i-y _i) ²

The constraint linearity of above-mentioned constraint condition and reconstructed error is combined, is just obtained:

$={\mathrm{\&Sigma;}}_i{\left|\right|f_i-{\mathrm{\&Sigma;}}_j{w}_{\mathrm{ij}}{f}_j\left|\right|}^2+\mathrm{\&mu;}{\mathrm{\&Sigma;}}_i{(f_i-y_i)}^2$

$=f^T\mathrm{Mf}+\mathrm{\&mu;}{(f-y)}^T(f-y)$

Wherein, W represents the reconstruction coefficients matrix that local linear merge module obtains in the above-mentioned nuclear space, and μ generally gets very little real number for realizing another parameter of pattern recognition device of the present invention, for example is taken as 10 ^-3, M=(I-W) ^T(I-W), I is a unit matrix.

The value of mapping function f makes the value of the value minimum of above-mentioned ε (f) exactly, that is:

$f=\underset{f:\mathrm{\&chi;}\&RightArrow;R}{\arg \min }\mathrm{\&epsiv;}\left(f\right)$

ε (f) is carried out differentiate, and the value that obtains f at last is:

f＝μ(M+μI) ^-1y

The key step that the regularization module is carried out is as follows:

Input: the reconstruction coefficients matrix W of the sample of local linear merge module output in the above-mentioned nuclear space, the desired value { y of mark sample ₁..., y ₁.

Output: the evaluation and test value vector f of sample.

(1) generates vectorial y=[y ₁..., y _n] ^T, wherein, { y ₁..., y _lFor marking the desired value of sample, { y _L+1..., y _nValue be 0.

(2) compute matrix M=(I-W) ^T(I-W).

(3) calculate evaluation and test value vector f=μ (M+ μ I) ^-1Y.

Come the present invention is further detailed below by specific embodiment.

Embodiment one:

Suppose to have the object samples of 12 inputs, these samples all are expressed as the form of 2 dimensional vectors, and are as follows:

$X=\left[\begin{array}{cccccccccccc}x1& x2& x3& x4& x5& x6& x7& x8& x9& x10& x11& x12\\ 0& 3.1& 5.9& 8.1& 9.5& 10& 1& 1.5& 2.9& 5.1& 7.9& 11\\ 10& 9.5& 8.1& 5.9& 3.1& 0& 2& -1.1& -3.9& -6.1& -7.5& -8\end{array}\right]$

That is: the digitizing expression way of first sample is [0 10], and the digitizing expression way of second sample is [3.1 9.5], by that analogy.

Wherein, first is the mark sample with last object samples, and its label is respectively 1 and-1, and remaining object samples is not for marking sample, that is:

Y＝[1 0 0 0 0 0 0 0 0 0 0 -1]

Fig. 4 is the synoptic diagram of these samples, and wherein, each o'clock, its coordinate was the value of the vector of these object samples corresponding to an input object sample.In the figure, represent two mark samples respectively with solid squares and triangles.

The objective of the invention is to predict that all do not mark the label of sample, to realize the purpose of identification or ordering retrieval.

Step 1: Laplce's kernel makes up module the object samples of input is handled, to make up Laplce's kernel.

At first, calculate distance between any two input object samples.In this embodiment, 12 object samples of input all are represented as the form of 2 dimensional vectors, therefore, calculate distance between them with Euclidean distance.For example: the Euclidean distance between first and second object samples is:

d(1，2)＝{(3.1-0) ²+(10-9.5) ²} ^0.5＝3.1。

The object samples that obtains in a manner described distance between any two can be with a matrix representation:

X ₁ X ₂ X ₃ X ₄ X ₅ X ₆ X ₇ X ₈ X ₉ X ₁₀ X ₁₁ X ₁₂

X ₁ 0.0 3.1 6.2 9.1 11.8 14.1 8.1 11.2 14.2 16.9 19.2 21.1

X ₂ 3.1 0.0 3.1 6.2 9.1 11.8 7.8 10.7 13.4 15.7 17.7 19.2

X ₃ 6.2 3.1 0.0 3.1 6.2 9.1 7.8 10.2 12.3 14.2 15.7 16.9

X ₄ 9.1 6.2 3.1 0.0 3.1 6.2 8.1 9.6 11.0 12.3 13.4 14.2

X ₅ 11.8 9.1 6.2 3.1 0.0 3.1 8.6 9.0 9.6 10.2 10.7 11.2

X ₆ 14.1 11.8 9.1 6.2 3.1 0.0 9.2 8.6 8.1 7.8 7.8 8.1

X ₇ 8.1 7.8 7.8 8.1 8.6 9.2 0.0 3.1 6.2 9.1 11.8 14.1

X ₈ 11.2 10.7 10.2 9.6 9.0 8.6 3.1 0.0 3.1 6.2 9.1 11.8

X ₉ 14.2 13.4 12.3 11.0 9.6 8.1 6.2 3.1 0.0 3.1 6.2 9.1

X ₁₀ 16.9 15.7 14.2 12.3 10.2 7.8 9.1 6.2 3.1 0.0 3.1 6.2

X ₁₁ 19.2 17.7 15.7 13.4 10.7 7.8 11.8 9.1 6.2 3.1 0.0 3.1

X ₁₂ 21.1 19.2 16.9 14.2 11.2 8.1 14.1 11.8 9.1 6.2 3.1 0.0

According to these distance values,, find out a nearest with it k object samples at each input object sample.In this embodiment, the value with k is made as 3.With object samples X ₁For example describes, these input object samples are to X ₁Distance be respectively: 0,3.1,6.2,9.1,11.8,14.1,8.1,11.2,14.2,16.9,19.2,21.1, therefore, 3 nearest with it object samples are: X ₂, X ₃, X ₇

Afterwards, calculate neighbour's matrix by following principle: for two object samples X _i, X _jIf object samples belongs to and one of nearest k object samples of another object samples, then the respective value in neighbour's matrix is a _Ij=a _Ji=exp (d (i, j) ²/ 2 σ ²), otherwise, a _Ij=0; In addition, the value of the element on the diagonal line is made as 0 in neighbour's matrix.In the present embodiment, σ is set at the mean value of the distance between all object samples, according to the above-mentioned object samples that obtains distance between any two, its value is 8.89.Still be that example describes with first and second object samples, from the calculating of front as can be seen, second object samples belongs to and one of nearest 3 objects of first object samples, and therefore, the respective value in neighbour's matrix is: a ₁₂=exp (3.1 ²/ (2*8.89 ²))=0.94.

In the manner described above, calculate the value of all elements in neighbour's matrix, the neighbour's matrix that obtains is as follows:

A＝ 0 0.94 0.79 0 0 0 0.66 0 0 0 0 0

0.94 0 0.94 0.79 0 0 0.68 0 0 0 0 0

0.79 0.94 0 0.94 0.79 0.00 0 0 0 0 0 0

0 0.79 0.94 0 0.94 0.79 0 0 0 0 0 0

0 0 0.79 0.94 0 0.94 0 0 0 0 0 0

0 0 0 0.79 0.94 0 0 0 0 0 0.68 0.66

0.66 0.68 0 0 0 0 0 0.94 0.79 0 0 0

0 0 0 0 0 0 0.94 0 0.94 0.79 0 0

0 0 0 0 0 0 0.79 0.94 0 0.94 0.79 0

0 0 0 0 0 0 0 0.79 0.94 0 0.94 0.79

0 0 0 0 0 0.68 0 0 0.79 0.94 0 0.94

0 0 0 0 0 0.66 0 0 0 0.79 0.94 0

Obtain after neighbour's matrix, Laplce's matrix computations module generates Laplce's matrix L and normalized Laplce's matrix according to neighbour's matrix

For generating Laplce's matrix L, at first need to generate a diagonal matrix D, element on this diagonal of a matrix be corresponding row in neighbour's matrix element and, for example, on the diagonal line value of first element be first row in neighbour's matrix A element and, that is: D (1)=0.94+0.79+0.66=2.39.

According to diagonal matrix D and neighbour's matrix A, can easily obtain Laplce's matrix L, normalized Laplce's matrix

And regularization Laplce matrix P.

At last, to regularization Laplce matrix inversion, obtain Laplce's kernel.In the present embodiment, the value of parameter ε is made as 0.01 in the regularization Laplce matrix, and the Laplce's kernel that obtains is:

K＝7.75 8.19 8.22 7.89 6.84 7.10 7.54 6.62 7.46 7.33 7.21 6.08

8.19 10.40 9.74 9.55 8.19 8.49 8.80 7.77 8.77 8.64 8.53 7.21

8.22 9.74 10.78 9.89 8.65 8.77 8.65 7.73 8.76 8.70 8.64 7.33

7.89 9.55 9.89 10.71 8.77 9.06 8.53 7.71 8.77 8.76 8.77 7.46

6.84 8.19 8.65 8.77 8.55 8.11 7.42 6.76 7.71 7.73 7.77 6.62

7.10 8.49 8.77 9.06 8.11 9.48 7.96 7.42 8.53 8.65 8.80 7.54

7.54 8.80 8.65 8.53 7.42 7.96 9.48 8.11 9.06 8.77 8.49 7.10

6.62 7.77 7.73 7.71 6.76 7.42 8.11 8.55 8.77 8.65 8.19 6.84

7.46 8.77 8.76 8.77 7.71 8.53 9.06 8.77 10.71 9.89 9.55 7.89

7.33 8.64 8.70 8.76 7.73 8.65 8.77 8.65 9.89 10.78 9.74 8.22

7.21 8.53 8.64 8.77 7.77 8.80 8.49 8.19 9.55 9.74 10.40 8.19

6.08 7.21 7.33 7.46 6.62 7.54 7.10 6.84 7.89 8.22 8.19 7.75

Step 2: in Laplce's kernel spacing, the input object sample is carried out linearity and embed

In Laplce's kernel spacing, calculate all object samples distance between any two

For input object sample X _i, X _j, in Laplce's kernel spacing corresponding distance be k (i, i)+k (j, j)-2*k (i, j).With first and second object samples is example, and their distances in Laplce's kernel spacing are:

k(1，1)+k(2，2)-2*k(1，2)＝7.75+10.40-2*8.19＝1.76。

The distance of any two input object samples in Laplce's kernel spacing that obtains in a manner described is:

0 1.76 2.08 2.68 2.63 3.04 2.16 3.06 3.54 3.87 3.74 3.35

1.76 0 1.70 2.01 2.57 2.91 2.29 3.41 3.57 3.89 3.74 3.74

2.08 1.70 0 1.71 2.03 2.72 2.96 3.86 3.95 4.16 3.89 3.87

2.68 2.01 1.71 0 1.72 2.06 3.14 3.83 3.86 3.95 3.57 3.54

2.63 2.57 2.03 1.72 0 1.80 3.18 3.57 3.83 3.86 3.41 3.06

3.04 2.91 2.72 2.06 1.80 0 3.05 3.18 3.14 2.96 2.29 2.16

2.16 2.29 2.96 3.14 3.18 3.05 0 1.80 2.06 2.72 2.91 3.04

3.06 3.41 3.86 3.83 3.57 3.18 1.80 0 1.72 2.03 2.57 2.63

3.54 3.57 3.95 3.86 3.83 3.14 2.06 1.72 0 1.71 2.01 2.68

3.87 3.89 4.16 3.95 3.86 2.96 2.72 2.03 1.71 0 1.70 2.08

3.74 3.74 3.89 3.57 3.41 2.29 2.91 2.57 2.01 1.70 0 1.76

3.35 3.74 3.87 3.54 3.06 2.16 3.04 2.63 2.68 2.08 1.76 0

The nuclear space internal linear embeds

The class of operation that makes up module with Laplce's kernel seemingly according to the above-mentioned distance value that obtains, to each input object sample, is found out a nearest with it M object samples.In the present embodiment, the value of M is identical with the value of k, i.e. M=3.With input object sample X ₁For example describes, these object samples are to X ₁Distance be: 0,1.76,2.08,2.68,2.63,3.04,2.16,3.06,3.54,3.87,3.74,3.35, therefore, 3 nearest with it input object samples are: X ₂, X ₃, X ₇

Next, local linear merge module utilizes neighbour's object samples of each input object sample to come this input object sample of linear-apporximation in the nuclear space, to realize the linear purpose that embeds.

With object samples X ₁For example describes.Obtain from the front, 3 object samples nearest with this object samples are X ₂, X ₃, X ₇, therefore, need in nuclear space, utilize these 3 objects to be similar to object samples X ₁

At first, make up the Gram Matrix C, the size of this matrix is 3x3, and the row and column of matrix corresponds respectively to input object sample X ₂, X ₃, X ₇Calculate for example with C (1,2), its value is: C (1,2)=k (X ₂, X ₃)-k (X ₁, X ₂)-k (X ₃, X ₁)+k (X ₁, X ₁)=9.74-8.19-8.22+7.75=1.08; Calculate for example with C (2,3), its value is: C (2,3)=k (X ₃, X ₇)-k (X ₁, X ₇)-k (X ₃, X ₁)+k (X ₁, X ₁)=8.65-7.54-8.22+7.75=0.64.

In a manner described, obtain and input object sample X ₁Corresponding Gram Matrix C is as follows:

$C=\left[\begin{array}{ccc}1.76& 1.08& 0.82\\ 1.08& 2.08& 0.64\\ 0.82& 0.64& 2.16\end{array}\right]$

Then, according to above-mentioned Gram matrix, obtain input object sample X ₁Reconstruction coefficients, its value is 0.29,0.24,0.28, these 3 coefficients correspond respectively to input object sample X ₂, X ₃, X ₇

Generate the reconstruction coefficients matrix

For each input object sample, all press aforesaid operations and generate reconstruction coefficients, afterwards, coefficient is carried out normalization, and generate the reconstruction coefficients matrix W.

In the present embodiment, the size of reconstruction coefficients matrix is 12x12.Each of this matrix is gone corresponding to an input object sample, wherein, those input object samples nearest with this object samples, corresponding value is set to reconstruction coefficients, otherwise value is 0.With object samples X ₁For example describes, neighbour's object samples of this object samples is X ₂, X ₃, X ₇, corresponding reconstruction coefficients is 0.29,0.24,0.28.Reconstruction coefficients is carried out normalization, obtain 0.35,0.30,0.35, therefore, the content of first row of this matrix is:

0 0.35 0.30 0 0 0 0.35 0 0 0 0 0

According to as above operation, the reconstruction coefficients matrix W that obtains is:

0 0.35 0.30 0 0 0 0.35 0 0 0 0 0

0.41 0 0.32 0.27 0 0 0 0 0 0 0 0

0 0.43 0 0.31 0.26 0 0 0 0 0 0 0

0 0.27 0.31 0 0.42 0 0 0 0 0 0 0

0 0 0.28 0.31 0 0.41 0 0 0 0 0 0

0 0 0 0.29 0.33 0 0 0 0 0 0 0.38

W＝

0.38 0 0 0 0. 0 0 0.33 0.29 0 0 0

0 0 0 0 0 0 0.41 0 0.31 0.28 0 0

0 0 0 0 0 0 0 0.42 0 0.31 0.27 0

0 0 0 0 0 0 0 0.26 0.31 0 0.43 0

0 0 0 0 0 0 0 0 0.27 0.32 0 0.41

0 0 0 0 0 0.35 0 0 0 0.30 0.35 0

Step 3: regularization obtains the evaluation and test value of input object sample

In the present embodiment, suppose that first and last input object sample are the mark sample, its label is respectively+and 1 and-1.The initial evaluation and test value that is the input object sample is:

Y＝[1 0 0 0 0 0 0 0 0 0 0 -1]。

At first, the reconstruction coefficients matrix W that obtains according to local linear merge module in the nuclear space is compute matrix M as follows, M=(I-W) ^T(I-W), wherein, I is a unit matrix.The result who obtains is as follows:

M＝1.32 -0.76 -0.17 0.11 0 0 -0.73 0.13 0.11 0 0 0

-0.76 1.38 -0.55 -0.41 0.23 0 0.12 0 0 0 0 0

-0.17 -0.55 1.37 -0.44 -0.42 0.12 0.10 0 0 0 0 0

0.11 -0.41 -0.44 1.35 -0.56 -0.16 0 0 0 0 0 0.11

0 0.23 -0.42 -0.56 1.36 -0.74 0 0 0 0 0 0.13

0 0 0.12 -0.16 -0.74 1.29 0 0 0 0.10 0.12 -0.73

-0.73 0.12 0.10 0.00 0 0 1.29 -0.74 -0.16 0.12 0 0

0.13 0 0 0.00 0 0 -0.74 1.36 -0.56 -0.42 0.23 0

0.11 0 0 0.00 0 0 -0.16 -0.56 1.35 -0.44 -0.41 0.11

0 0 0 0 0 0.104 0.12 -0.42 -0.44 1.37 -0.55 -0.17

0 0 0 0 0 0.12 0 0.23 -0.41 -0.55 1.38 -0.76

0 0 0 0.11 0.13 -0.73 0 0 0.11 -0.17 -0.76 1.32

In this embodiment, the value with parameter μ is made as 0.01.Then, calculate the evaluation and test value of each input object sample in the following manner, f=u (M+uI) ^-1Y.Wherein, I is a unit matrix.

The evaluation and test value of the input object sample that finally obtains is as follows:

f＝[0.061 0.074 0.071 0.064 0.044 0.005 -0.005 -0.044 -0.064-0.071 -0.074 -0.061]

Fig. 5 is the synoptic diagram of the input object sample evaluation and test value in the present embodiment.In the figure, each input object sample all uses a point to represent, in addition, the gray scale of each point is corresponding to the evaluation and test value of each object samples, and promptly the gray-scale value of data point is close more, and the input object sample of expression correspondence is similar more.

For the pattern classification problem, if with 0 classification boundaries as the evaluation and test value, can obtain following classification results: preceding 6 input object samples belong to the first kind, and back 6 object samples belong to second class.

Embodiment two:

Pattern recognition device of the present invention can be used for recognition image, below, be that example describes with the handwriting digital image recognition.

Pattern recognition device of the present invention be input as 3000 handwriting digital images, all images is gray level image, the gray-scale value of pixel is between 0 and 255, the size of each image is 16x16.3000 images of input are under the jurisdiction of between the numeral 0 to 9, and each numeral comprises 300 images, and therefore, present embodiment in fact is the classification problem of one 10 classification, and each classification is corresponding to a numeral.With 300 corresponding samples of each classification in, suppose that preceding 5 are the mark sample, remaining 295 for not marking sample.

Fig. 6 has provided the sample of part handwriting digital.In the present embodiment, directly as the characteristic quantity of input object sample, therefore, each input object sample can be expressed as the vector of one 256 dimension with the gray-scale value of pixel in the input picture.

Similar with the foregoing description one, the implementation procedure of present embodiment is as follows:

(1) makes up module by Laplce's kernel the input object sample is handled, make up Laplce's kernel

At first, calculate the distance between any two object samples in the input object sample, obtain size and be the matrix of 3000x3000.Because the input object sample can be expressed as the vector of one 256 dimension, therefore, directly utilize the Euclidean distance of vector to calculate.

Afterwards, utilize k neighbour principle to generate neighbour's matrix A, in the present embodiment, the value of k is made as 10.

According to above-mentioned neighbour's matrix A, obtain Laplce's matrix and normalized Laplce's matrix, thereby, generating Laplce's kernel K, size of nuclear matrix is 3000x3000 in this.

(2) in Laplce's kernel spacing the input object sample being carried out linearity embeds

This process is very similar with the process of above-mentioned embodiment one.Different is that in nuclear space internal linear embedding operation, the value of parameter M is 10.

(3) regularization

Be similar to the process among the embodiment one, the reconstruction coefficients matrix W compute matrix M=(I-W) that obtains according to local linear merge module in the above-mentioned nuclear space ^T(I-W), wherein, I is a unit matrix.

Next, calculating does not mark the probability that sample belongs to each classification respectively.

First classification with digital 0 correspondence is that example describes.At first, generate the initial evaluation and test value Y of input object sample.Y is that a length is 3000 vector, wherein, and the corresponding input picture of each element.In this vector, the value of the element corresponding with the mark sample of numeral 0 is made as 1, and remaining value is made as 0; Then, calculate each sample and belong to such other evaluation and test value, f ₀=u (M+uI) ^-1Y, wherein, I is that size is the unit matrix of 3000x3000, the value of parameters u is 5e-4.f ₀Be that a length is 3000 vector, represent the similarity of each input object sample and numeral 0.

Second classification with digital 1 correspondence is that example describes.At first, generate the initial evaluation and test value Y of input object sample.In this vector, the value of the element corresponding with the mark sample of numeral 1 is made as 1, and remaining value is made as 0; Then, calculate each sample and belong to such other evaluation and test value, f ₁=u (M+uI) ^-1Y, the value of parameters u remains unchanged, and still is 5e-4.f ₁The similarity of representing each input sample and numeral 1.

By that analogy, respectively 10 classifications are handled.

For each input object sample, all obtain 10 evaluation and test values, represent the similarity of this object samples and 10 numerals.At a minute time-like, from 10 evaluation and test values, select that maximum evaluation and test to be worth the classification of pairing classification as this input object sample.

Following table has provided the discrimination that the method in pattern recognition device of the present invention and [non-patent literature 2] obtains:

	0	1	2	3	4	5	6	7	8	9	On average
												The method of [non-patent literature 2]	0.91	0.78	0.87	0.65	0.79	0.92	0.72	0.74	0.54	0.94	0.78
The present invention	0.97	0.87	0.90	0.61	0.73	0.97	0.74	0.79	0.49	0.97	0.80

Therefore the discrimination of pattern recognition device of the present invention is than higher.

Embodiment three:

Pattern recognition device of the present invention can be used for carrying out speaker ' s identity identification, and speaker ' s identity identification is basic identical with handwriting digital image recognition among the above-mentioned embodiment.Different is, speaker ' s identity identification is based on that speaker's voice signal carries out, and, the corresponding classification of each speaker.

At first, from voice signal, extract 4 kinds of features such as cepstrum coefficient, difference cepstrum coefficient, fundamental frequency and difference fundamental frequency, and describe feature as the digitizing of input speech signal with the vector that these 4 kinds of features are formed.

Afterwards, carry out processes such as Laplce's kernel structure, the embedding of nuclear space internal linear, regularization respectively, identify speaker's identity.The process that these processes and front are described at Handwritten Digital Recognition is similar, no longer is described in detail at this.

Each composition module of the pattern recognition device of the invention described above can be implemented in the computing machine or single-chip microcomputer equipment with computer program of realizing described function integratedly, also can each be made of discrete signal conditioning package.

Though, in the above embodiments, the present invention is specified with Handwritten Digital Recognition and speaker ' s identity identification, the invention is not restricted to these, and can be widely used in the various pattern-recognition occasions.

Further, according to embodiments of the invention, purpose of the present invention can also realize by the computer program that makes execution aforesaid operations such as computing machine or single-chip microcomputer.

In addition, should be understood that in each embodiment, (for example can pass through special circuit or circuit, interconnection is to carry out the discrete logic gate of dedicated functions), the programmed instruction by carrying out by one or more processor, perhaps carry out described each action by both combinations.Therefore, can implement above-mentioned many aspects by multiple different form, and all these forms considered to be in all in the scope of the content of describing.For in the above-mentioned many aspects each, the embodiment of any this form can refer to " being configured to carry out the logic of described action " at this, perhaps alternatively, is meant " logic of carrying out or can carry out described action ".

Further, according to embodiments of the invention, purpose of the present invention can also be realized by computer-readable medium, the program that described medium memory is above-mentioned.Computer-readable medium can be can comprise, storage, reception and registration, propagation or convey program, with by executive system, equipment or device any device that use or that and instruction executive system, equipment or device combine.This computer-readable medium for example can be, but be not limited to electronics, magnetic, light, electromagnetism, infrared or semiconductor system, unit or propagation medium.The example more specifically of this computer-readable medium (non-exclusive list) can comprise: have one or more electrical connection, portable computer diskette, random-access memory (ram), ROM (read-only memory) (ROM), the Erasable Programmable Read Only Memory EPROM of multiple conducting wires

(EPROM or flash memory), optical fiber, and portable optic disk ROM (read-only memory) (CDROM).

The above-mentioned explanation of the embodiment of the invention only is used for illustration and illustrative purposes, but not is intended to limit the present invention or it is limited to disclosed concrete form.It will be appreciated by those skilled in the art that, embodiment selected and that describe only is in order to explain principle of the present invention and practical application thereof best, use to be suitable for specific expection, and under situation about not breaking away from, can carry out various modifications and variations to the present invention by claim and the invention scope that equivalent limited thereof.

Claims (10)

1, a kind of pattern recognition device, this pattern recognition device utilizes semi-supervised learning mechanism to come the schema object sample of input is discerned, wherein said schema object sample comprises the mark sample and does not mark sample, and represent with digitalized signature, described mark sample comprises the digitalized signature and the corresponding desired value of schema object

Described pattern recognition device comprises:

Non-Euclidean space kernel structure portion, its schema object sample based on input makes up nuclear matrix in the non-Euclidean space, and this non-Euclidean space is suitable for the flow structure of structural model object;

Local linear Embedded Division in the nuclear space, nuclear matrix in its non-Euclidean space that constructs according to described non-Euclidean space kernel structure portion is carried out the linear embedding in part to the schema object sample, with the reconstruction coefficients matrix of generate pattern object samples; And

Regularization portion, it comes to generate the evaluation and test value for the schema object sample of all inputs according to the reconstruction coefficients matrix of local linear Embedded Division generation in the described nuclear space and the desired value of described mark sample.

2, pattern recognition device as claimed in claim 1 is characterized in that, described non-Euclidean space kernel structure portion generates nuclear matrix in the non-Euclidean space according to the k neighbour matrix of the schema object sample of all inputs.

3, pattern recognition device as claimed in claim 1 is characterized in that, the inherent geometry of the reconstruction coefficients matrix representation schema object sample of the schema object sample that local linear Embedded Division generates in the described nuclear space.

4, pattern recognition device as claimed in claim 1 is characterized in that, described non-Euclidean space is Laplce space or reproducing kernel Hilbert space.

5, as each the described pattern recognition device among the claim 1-4, this pattern recognition device is used for recognition image or audio frequency, and wherein, schema object is the characteristic quantity that can be used for carrying out pattern-recognition of image or audio frequency.

6, a kind of mode identification method, this mode identification method utilizes semi-supervised learning mechanism to come the schema object sample of input is discerned, wherein said schema object sample comprises the mark sample and does not mark sample, and represent with digitalized signature, described mark sample comprises the digitalized signature and the corresponding desired value of schema object

Described mode identification method may further comprise the steps:

Schema object sample based on input makes up nuclear matrix in the non-Euclidean space, and this non-Euclidean space is suitable for the flow structure of structural model object;

According to nuclear matrix in the non-Euclidean space that constructs, the schema object sample is carried out the linear embedding in part, with the reconstruction coefficients matrix of generate pattern object samples; And

According to the reconstruction coefficients matrix that generates and the desired value of described mark sample, come to generate the evaluation and test value for the schema object sample of all inputs.

7, mode identification method as claimed in claim 6 is characterized in that, generates nuclear matrix in the non-Euclidean space according to the k neighbour matrix of the schema object sample of all inputs.

8, mode identification method as claimed in claim 6 is characterized in that, the inherent geometry of the reconstruction coefficients matrix representation schema object sample of described schema object sample.

9, mode identification method as claimed in claim 6 is characterized in that, described non-Euclidean space is Laplce space or reproducing kernel Hilbert space.

10, as each the described mode identification method among the claim 6-9, this mode identification method is used for recognition image or audio frequency, and wherein, schema object is the characteristic quantity that can be used for carrying out pattern-recognition of image or audio frequency.

Images