CN108319983A - A kind of nonlinear data dimension reduction method of local nonlinearity alignment - Google Patents

Abstract

The invention discloses a kind of nonlinear data dimension reduction methods based on local nonlinearity alignment.Its Neighbor Points and the corresponding block of composition are taken using KNN algorithms to each sample point in higher dimensional space first.The data point block of these higher dimensional spaces is all dropped to PCA algorithms to the space plane of low-dimensional respectively.By respective translation rotation block is perfectly stitched together each block to lower by PCA on lower dimensional space.The criterion of splicing be each point may include by different several pieces, then in all pieces by comprising this point belong to the same point, they should can be overlapped in the representation theory of low-dimensional；Then these this points in the block can find out the position of its equalization point, and two norms of distance of equalization point the sum of of all points that same point should be represented as in different masses to it should want minimum, so just can guarantee that the same point in different masses overlaps as far as possible.All pieces by the way that, so that sum of the deviations caused by each point is minimum, so different blocks can have been carried out perfect splicing by we in low-dimensional plane after respective translation rotation.To solve the world coordinates that sample point is embedded in low-dimensional, it is achieved the nonlinear data dimensionality reduction of local nonlinearity alignment.

Description

A kind of nonlinear data dimension reduction method of local nonlinearity alignment

Technical field

The invention belongs to machine learning fields, and in particular to a kind of based on the non-of local nonlinearity alignment in manifold learning Linear data dimension reduction method.

Background technology

Data Dimensionality Reduction refers to that sample is mapped to lower dimensional space from higher dimensional space by linear or non-linear method, from And obtain the process that the higher dimensional space indicates in compared with lower dimensional space.The redundancy of legacy data can be reduced by this operation Property, improve the efficiency and specific aim to data processing.The method of Data Dimensionality Reduction is broadly divided into Linear Mapping and Nonlinear Mapping side Method two major classes.The representative method of wherein Linear Mapping method have principal component analysis (Principle Component Analysis, Abbreviation PCA) and linear decision analysis (Linear Discriminant Analysis, abbreviation LDA).Both methods theory at It is ripe, calculate that simple, calculating speed is fast, but its for the high dimensional datas of those nonlinear organizations, you can't get effective answers.

Nonlinear method based on manifold learning then provides a solution route for Data Dimensionality Reduction.Manifold learning is with non- One major class Method of Nonlinear Dimensionality Reduction of monitor mode operation, each method all attempts to retain specific geometric sense, such as distance, angle Degree, proximity or local patch.Since two delivered on Science since 2000 start sex work, Isomap and LLE, manifold learning are always the important topic of data visualization and pattern classification.Come from imaging device, bioinformatics today Mass data with financial application is typically higher-dimension；Therefore there is an urgent need to overcome " dimension curse ".One direct solution party Case is exactly to convert high dimensional data to the dimension reduction method of low-dimensional embedded space.However, the conventional methods such as PCA and MDS can not be found The non-linear or curvilinear structures of input data.On the contrary, manifold learning be suitable for nonlinear organization being launched into it is flat low Tie up embedded space.Therefore, these methods, which have discovered that, is widely applied, such as microarray gene expression, 3 D human body posture Restore, recognition of face and facial expression transfer.Most of pervious manifold learnings are absorbed in a specific visual angle, to protect Hold a single geometric sense.Therefore, how under the premise of ensuring not lose key property, reducing dimension as much as possible becomes A recent research hotspot.

Manifold learning dimension reduction method mainly has：Be locally linear embedding into (Locally Linear Embedding, referred to as LLE), Isometric Maps (Isometric Mapping, abbreviation ISOMAP), local tangent space are aligned (Local tangent Space alignment, abbreviation LTSA) etc..

LLE algorithms stress guarantee local neighborhood structure using based on local linear Conformal Mapping thought.At information In many applications of reason, local message is sometimes more more effective than global information, and advantageous in calculation amount, only comprising multinomial The sparse matrix operation of the formula order of magnitude.In addition, also there is good presentation skills, i.e., when the feelings that global structure is non-Euclidean space Under condition, local geometry is close to Euclidean space.But there is also some application limitations for this method, such as：To parameter and Dimensionality reduction performance failure when outside noise is excessively sensitive, processing is distributed sparse data set.

ISOMAP algorithms are existing by analyzing using the global thought kept based on the geodesic distance before and after dimensionality reduction Manifold of higher dimension, the low-dimensional insertion corresponding to manifold of higher dimension is obtained, to allow the Near-neighbor Structure on manifold of higher dimension between data point to exist It is more completely reappeared in low-dimensional insertion.It is calculated with multi-dimentional scale transformation (Multidimensional Scaling, abbreviation MDS) Method is analysis tool, difference be to calculate on manifold of higher dimension between data point apart from when, abandoned traditional Euclidean distance, power The inherent geometric properties for keeping data point are sought, Euclidean distance is replaced using the geodesic curve distance in Differential Geometry, is by with reality The input data on border estimates a kind of algorithm of its geodesic curve distance.

LTSA algorithms are using the popular learning algorithm based on local tangent space.By approaching cutting for each sample point Space builds the local geometric of low dimensional manifold, then finds out whole low-dimensional embedded coordinate using cutting space arrangement.As one kind Prevalence learning algorithm, LTSA can effectively learn the whole embedded coordinate of embodiment data set low dimensional manifold structure well, But there is also deficiencies for it：It is equal to sample number for the matrix exponent number of Eigenvalues Decomposition in algorithm, when sample set is larger, will be unable to locate Reason.

The present invention proposes another based on the thought locally kept, i.e. the nonlinear data dimensionality reduction side of local nonlinearity alignment Method.This method drops to low-dimensional by PCA to each piece, then exists again by building local domain block structure to input sample point Lower dimensional space is with the sum of two norm errors of distance between the equalization point of each point to the respective point of all blocks put comprising this And each piece of translation and degree of rotation are constrained to minimize the error, to solve the world coordinates that sample point is embedded in low-dimensional, It is achieved the nonlinear data dimensionality reduction of local nonlinearity alignment.Since the dimensionality reduction effect that the method is made is very good, while this The time complexity of method is very low, is relatively more suitable for large-scale high dimensional data dimensionality reduction.

Invention content

It is an object of the invention to propose a kind of nonlinear data dimension reduction method being aligned based on local nonlinearity.The present invention Its domain variability is calculated using KNN algorithms to each sample point of higher dimensional space first and forms corresponding block, then these Block all drops to lower dimensional space using PCA algorithms；On lower dimensional space we by with each point to it is all comprising this point Between the equalization point of the respective point of block the sum of two norm errors of distance and with minimize the error constrain each piece translation and rotation Carryover degree is achieved in mapping of the initial data higher-dimension to lower dimensional space, obtains whole low-dimensional embedded coordinate, is achieved part The nonlinear data dimensionality reduction of non-linear alignment.The particular content of the present invention is as follows：

1, its Neighbor Points and the corresponding block of composition are taken using KNN algorithms to the sample point in higher dimensional space.Take the calculation of block There are many kinds of methods, the present invention mainly to each data point in higher dimensional space using KNN algorithms take its k nearest neighbor point as The field block of this point, has N number of point that can form N number of piece.Such as data point x_pField block be X_p=[x₁ … x_k x_k+1] ∈R^D×(k+1), wherein x₁ … x_kFor x_pK field point, x_k+1For point x_p。

2, the data block of these higher dimensional spaces is all dropped to the process of local homomorphism to the space plane of low-dimensional, coordinate respectively It is expressed as Θ_p∈R^d×(k+1), p=1 ..., N.There are many algorithm of local homomorphism in manifold learning, and the present invention mainly uses PCA to calculate Method carries out Local-projection method, higher dimensional space midpoint x_i, i=1 ..., N, in Θ after PCA dimensionality reductions_pIn coordinate representation be θ_{I, p} =(m_{I, p}, n_{I, p})∈R^1×d。

3, to each block Θ_pBuild unknown spin matrix and translation matrix so that Θ_pIt is flat that low-dimensional can be moved to Any position in face and block Θ_pIt can arbitrarily be rotated around center.Θ_pIt is embedded in Local Coordinate Representations by translating postrotational low-dimensional For Y_p, and Θ_pMiddle certain point θ_{I, p}It is y by translating postrotational low-dimensional coordinate representation_{I, p}.The present invention provides following algorithm and solves This problem：

3.1, by each piece of translation vector ξ_pS and spin matrix v_pS, which is stacked up, constitutes whole translation matrix ξ ∈ R^{N ×d}And rotate integrally matrix V ∈ R^dN×d。

3.2, to each each point θ in the block_{I, p}Structure translation selection vector s_{I, p}∈R^1×PAnd coordinate selection vector l_{I, p} ∈R^1×dP。

3.3, some block Θ_pMiddle certain point θ_{I, p}It is y by translating postrotational low-dimensional coordinate representation_{I, p}=s_{I, p}ξ+l_{I, p}V。

4, it includes data point x to calculate all_iLocal coordinate block Θ_iS all y after respective translation rotation_{I, p}S's Average value y_i, find out y_{I, p}With its average point y_iTwo norm errors of distance simultaneously find out same point y in all pieces_{I, p}S corresponds to y_iThis The global error of a point.

5, to the average translation of each point selection vector s_iAnd averagely rotation selection matrix l_iThe composition that is stacked is overall Mean shift selection matrix S_meanAnd population mean rotation selection matrix L_mean。

6, it calculates all low-dimensionals insertion point and is accumulated in together the object function to be formed with its average point global error and beIt is required that global error is minimum namely object function is minimum, then demand solutionWherein H=(I-S (S^TS)⁺S^T)L。

7, above formula is usually solved with rayleigh quotient, enables V^TV=I_d, by H^TH carries out feature decomposition, obtains minimal eigenvalue Corresponding feature vector finds out spin matrix V, and ξ=- (S^TS)⁺S^TLV。

8, after having found out ξ and V, then the coordinate points of all low-dimensional insertions are：Y=S_meanξ+L_meanV。

The invention has the advantages that：This method to input sample point by building local domain block structure, to every A block drops to low-dimensional by PCA, is then spliced on lower dimensional space again, and whole low-dimensional embedded coordinate is found out, can be effective Ground reduces the sample set dimension of input.Since the dimensionality reduction effect that the method is made is very good, at the same time complexity very It is low, it is relatively more suitable for large-scale high dimensional data drop.

Description of the drawings

Fig. 1 is the operational flowchart for the nonlinear data dimension reduction method of the present invention being aligned based on local nonlinearity.

Specific implementation mode

As shown in the picture, the nonlinear data dimension reduction method based on local nonlinearity alignment, including the following contents：

1, assume that high dimensional data sample point set is X=[x₁ … x_N]∈R^D×N, the sample point set that is mapped in lower dimensional space For Y=[y₁ … y_N]∈R^d×N.Wherein：D is the dimension of higher dimensional space；D (d ＜ ＜ D) is the dimension of lower dimensional space；X is higher-dimension The input of data model is higher dimensional space R^D×NIn N number of D tie up real number column vector.Y is that high dimensional data is mapped in lower dimensional space Output sample set, be lower dimensional space R^d×NIn N number of d tie up real number column vector.

2, its Neighbor Points and the corresponding block of composition are taken using KNN algorithms to the sample point in higher dimensional space.Take the calculation of block There are many kinds of methods, the present invention mainly to each data point in higher dimensional space using KNN algorithms take its k nearest neighbor point as The field block of this point, has N number of point that can form N number of piece.Such as data point x_pField block be X_p=[x₁ … x_k x_k+1] ∈R^D×(k+1), wherein x₁ … x_kFor x_pK field point, x_k+1For point x_p。

3, the data block of these higher dimensional spaces is all dropped to the process of local homomorphism to the space plane of low-dimensional, coordinate respectively It is expressed as Θ_p∈R^d×(k+1), p=1 ..., N.There are many algorithm of local homomorphism in manifold learning, and the present invention mainly uses PCA to calculate Method carries out Local-projection method, higher dimensional space midpoint x_i, i=1 ..., N, in Θ after PCA dimensionality reductions_pIn coordinate representation be θ_{I, p} =(m_{I, p}, n_{I, p})∈R^1×d。

4, assume that p-th piece of unknown translation vector is ξ_p∈R^1×d, unknown spin matrix is v_p∈R^d×d, 1≤p≤P. Θ_pIt is Y by translating postrotational coordinate representation_p, i.e. Y_p=ξ_p+A_pΘ_p, and θ_{I, p}By translating postrotational low-dimensional coordinates table It is shown as

5, l is enabled_{I, p}∈R^1×dPFor θ_{I, p}The coordinate selection vector constituted.Such as θ in first block_{I, 1}Coordinate selection Vector is：l_{I, 1}=(m_{I, 1}, n_{I, 1}, 0,0 ... 0) ∈ R^1×dP.Enable s_{I, p}∈R^1×PP-th piece of Θ_pTranslation selection vector.Such as the The translation of one block select vector for：s_{I, 1}=(1,0,0 ... 0) ∈ R^1×P.So y_{I, p}It is represented by y_{I, p}=s_{I, p}ξ+l_{I, p}V, wherein ξ∈R^N×dAnd V ∈ R^dN×dFor all pieces of translation vector ξ_pS and spin matrix v_pThe matrix that s is constituted, i.e.,

6, so all includes data point x_iLocal coordinate block Θ_iAverage points of the s after respective translation rotation It is set toHere f_iIndicate this point by f_iA block includes.

7, due to x_iProjection appropriate is all y_{I, p}The average value y of s_i.So y_{I, p}With its average point y_iError be：Corresponding to y_iThis point global error beWherein

8, the s each put_i, l_iIt is stacked and constitutes population mean matrix

9, it is possible thereby to which the object function for calculating all low-dimensional insertions and its average point global error isWherein

10, require global error minimum namely object function minimum, then demand solutionThis When be believed that object function minimum value be 0, then S ξ+LV=0 namely S ξ=- LV；It is multiplied by S simultaneously in both sides^TFor S^TS ξ=- S^TLV；To S^TS asks pseudoinverse that can obtain ξ=- (S^TS)⁺S^TLV。

11, at this time Wherein H=(I-S (S^TS)⁺S^T)L。

12, formula V above may have multiple solutions, and in order to avoid trivial solution, above formula is usually solved with rayleigh quotient, even V^TV=I_d, By to H^TH carries out feature decomposition, obtains the feature vector of corresponding minimal eigenvalue to find out spin matrix V, and ξ=- (S^TS)⁺ S^TLV.After having found out ξ and V, then the coordinate points of all low-dimensional insertions are：Y=S_meanξ+L_meanV。

Claims (3)

1. a kind of nonlinear data dimension reduction method of local nonlinearity alignment, it is characterised in that the step of this method is as follows：

A. the field that its k nearest neighbor point is put as this is taken using KNN algorithms to each sample point in higher dimensional space Block has N number of point that can form N number of piece；

B., the data block of these higher dimensional spaces is all dropped to PCA algorithms to the space plane of low-dimensional, coordinate representation Θ respectively_p, p =1 ..., N.Higher dimensional space midpoint x_i, i=1 ..., N, in Θ after PCA dimensionality reductions_pIn coordinate representation be θ_{I, p}；

C. to each block Θ_pBuild unknown spin matrix and translation matrix.Θ_pBy translating postrotational low-dimensional insertion office Portion's coordinate representation is Y_p, and Θ_pMiddle certain point θ_{I, p}It is y by translating postrotational low-dimensional coordinate representation_{I, p}；

D. all low-dimensional insertion points are calculated and the object function of its average point global error isBy to target The solution of function takes the feature vector corresponding to minimal eigenvalue to find out all pieces of spin matrix V, and then finds out all pieces Translation matrix ξ；

E, after having found out ξ and V, the coordinate for further finding out the final low-dimensional insertion of all sample points is：Y=S_meanξ+L_meanV。

2. according to the method described in claim 1, it is characterized in that the step C is specifically included：

C1, by each piece of translation vector ξ_pS and selection matrix v_pS, which is stacked up, constitutes whole translation matrix ξ ∈ R^N×dAnd Rotate integrally matrix V ∈ R^dN×d；

C2, to each each point θ in the block_{I, p}Structure translation selection vector s_{I, p}∈R^1×PAnd coordinate selection vector l_{I, p}∈R^{1 ×dP}；

C3, some block Θ_pMiddle certain point θ_{I, p}It is y by translating postrotational low-dimensional coordinate representation_{I, p}=s_{I, p}ξ+l_{I, p}V。

3. according to the method described in claim 1, it is characterized in that the step D is specifically included：

D1, all calculating include data point x_iLocal coordinate block Θ_iS all y after respective translation rotation_{I, p}S's is flat Mean value y_i, find out y_{I, p}With its average point y_iTwo norm errors of distance simultaneously find out all block midpoint y comprising same point_{I, p}S is corresponding In y_iThe global error of this point；

D2, all block midpoint y comprising same point that each sample point is found out according to D1_{I, p}S corresponds to equalization point y_iInstitute's shape At global error be accumulated in together to form all low-dimensionals insertion point and its average point global error, to construct object function

D3, by the solution to object function, take the feature vector corresponding to minimal eigenvalue to find out all pieces of spin moment Battle array V, and then find out all pieces of translation matrix ξ；

D4, the average translation selection vector s to each point_iAnd averagely rotation selection matrix l_iThe composition that is stacked is overall flat Translation selection matrix S_meanAnd population mean rotation selection matrix L_mean。