CN104063617A

CN104063617A - Multiple linear regression method based on dimensionality reduction hyperplane

Info

Publication number: CN104063617A
Application number: CN201410318782.3A
Authority: CN
Inventors: 许蔚蔚
Original assignee: XU YUYU
Current assignee: XU YUYU
Priority date: 2014-07-07
Filing date: 2014-07-07
Publication date: 2014-09-24

Abstract

The invention relates to the field of probability theory and mathematical statistics, in particular to a multiple linear regression method based on a dimensionality reduction hyperplane. The multiple linear regression method comprises the steps of setting variables x1, x2,..., xn, wherein the variables have a linear dependence relation, setting a matrix of observed values of y as (X, Y), performing principal component analysis on (X, Y) or a normalization matrix (X1, Y1) of the (X, Y), solving a hyperplane equation perpendicular to a (n+1)<th> principal component and passing through the center of gravity of data, arranging the hyperplane equation into the form (as specified in the specification), and obtaining an estimated multiple linear regression equation (as specified in the specification) based on the observation data (X,Y).

Description

A kind of multiple linear regression analysis method based on dimensionality reduction lineoid

Technical field

The present invention relates to Probability Theory and Math Statistics field, particularly relate to a kind of multiple linear regression analysis method based on dimensionality reduction lineoid.

Background technology

Linear regression analysis is one of research method the most basic in mathematical statistics, in order to study the correlationship between variable.In socic-economic field, even if the relation between a lot of variable is not linear in macroscopic view, on microcosmic, still can be similar to and does linearization process.In addition, by the pre-service such as variable being taken the logarithm to, the nonlinear relationship between variable can be transformed to linear relationship sometimes.Statistical study, the numerical evaluation software of main flow all be take matrix operation as basis at present.Therefore, variable is carried out to high-precision linear regression and there is important basic role.

Linear regression can be divided into a heavy monobasic, heavy polynary, multiple several situations such as polynary according to the quantity of independent variable and dependent variable, and wherein, a heavy multiple linear regression is one of wherein basic problem, is summarized as follows:

Be provided with variable x ₁, x ₂..., x _n, y meets linear relation y=β ₀+ β ₁x ₁+ ...+β _nx _n+ ε, wherein β _i(i=0,1 ..., n) being constant, ε is stochastic error.Each variable is carried out to N observation, and observed reading is: y=(y ₁, y ₂..., y _n) ', be x wherein _ijrepresent variable x _jthe i time observed reading.

Above data and loose point set S={ (x _i1..., x _in, y _i) | i ∈ (1 ..., N) } equivalence.The multiple linear regression equations of the estimation based on above observation data is

\hat{y} = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n} .

The matrix form of multiple linear regression equations be Y=(1, X) B+E, wherein, B=(β ₀..., β _n) ', E=(ε ₁..., ε _n) '.

The most frequently used solution of one heavy multiple linear regression is the linear regression method based on least square method: y is considered as to dependent variable, x ₁, x ₂..., x _nbe considered as independent variable, independent variable is not considered as stochastic variable, only has dependent variable to be considered as stochastic variable; The maximum likelihood of parameter matrix B is estimated as

\hat{B} = {({\hat{β}}_{0}, {\hat{β}}_{1})}^{'} = {({(1, X)}^{'} (1, X))}^{- 1} {(1, X)}^{'} Y .

The result of least-squares linear regression does not have coordinate independence.So-called coordinate independence refers to computing place coordinate system to do the result that orthogonal transformation (translation is or/and rotation) does not affect computing.

In socioeconomic variable, seldom there is value not there is " pure " independent variable of randomness.Due to viewing angle, observation instrument, data definition and sum up the difference of method, the observation data of same economic phenomenon may have very big difference in form, but through the even simple coordinate transform of certain linear transformation, between data, just often show obvious equivalence.For the above-mentioned reasons, wishing to have the regression result of data group of relation of equivalence also identical is very natural requirement, and therefore, the linear regression method that development has coordinate invariance is necessary.

Summary of the invention

The invention provides a kind of multiple linear regression analysis method based on dimensionality reduction lineoid, can make regression result there is coordinate independence.

For achieving the above object, the technical solution adopted in the present invention is: a kind of multiple linear regression analysis method based on dimensionality reduction lineoid, and step is as follows:

(1) be provided with variable x ₁, x ₂..., x _n, y meets linear relation y=β ₀+ β ₁x ₁+ ...+β _nx _n+ ε, wherein β _i(i=0,1 ..., n) being constant, ε is stochastic error, and n > 2 carries out N observation to each variable, and observed reading is y=(y ₁, y ₂..., y _n) ', be x wherein _ijrepresent variable x _jthe i time observed reading, above data and loose point set S={ (x _i1..., x _in, y _i) | i ∈ (1 ..., N) } equivalence, the multiple linear regression equations of the estimation based on above observation data is

\hat{y} = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n};

(2) (X, Y) carried out to principal component analysis (PCA), establish major component and be followed successively by G ₁..., G _n+1, corresponding to the vector of unit length of above major component, be expressed as (X, Y) is at n+1 dimension space (g ₁..., g _n) in homography be G=(g _ij) _{(n+1) * N};

(3) at n+1 dimension space (g ₁..., g _n) in, calculate and F _n+1vertical and pass through each sample point geometric center lineoid equation be

g_{n + 1} - \overset{&OverBar;}{g_{n + 1}} = 0,

Wherein

\overset{&OverBar;}{g_{j}} = Σ_{i = 1}^{N} g_{ij};

(4) by formula middle g _n+1use x ₁, x ₂..., x _n, y represents, arranges to be form, multiple linear regression equations for the estimation based on observation data (X, Y).

The present invention also provides the multiple linear regression analysis method of the second based on dimensionality reduction lineoid, and step is as follows:

\hat{y} = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n};

(2) column vector in Y and X is normalized respectively, result is merged into matrix (X ¹, Y ¹), (X ¹, Y ¹) in variable corresponding to each column vector be x ¹ ₁, x ¹ ₂..., x ¹ _n, y ¹, the method for normalized is: establishing Z is column vector, the average of each component that mean (Z) is Z, and the standard deviation of each component that std (Z) is Z, the normalized vector of Z is if Z and Z ¹corresponding variable is respectively z and z ¹, its relational expression is

(3) to (X ¹, Y ¹) carry out principal component analysis (PCA), establish major component and be followed successively by F ₁... F _n+1, corresponding to the vector of unit length of above major component, be expressed as (X ¹, Y ¹) at n+1 dimension space (f ₁..., f _n) in homography be F=(f _ij) _{(n+1) * N};

(4) at n+1 dimension space (f ₁..., f _n) in, calculate and F _n+1vertical and pass through each sample point geometric center lineoid equation be

f_{n + 1} - \overset{&OverBar;}{f_{n + 1}} = 0,

Wherein

\overset{&OverBar;}{f_{j}} = Σ_{i = 1}^{N} f_{ij};

(5) by formula middle f _n+1use x ¹ ₁, x ¹ ₂..., x ¹ _n, y ¹represent, arrange and be form;

(6) by formula in each variable x ₁, x ₂..., x _n, y represents, arranges to be

y = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n}

Form,

\hat{y} = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n}

Multiple linear regression equations for the estimation based on observation data (X, Y).

The beneficial effect that the present invention reaches: make regression result there is coordinate independence, improve regression accuracy.

Accompanying drawing explanation

Embodiment

Homing method concrete steps without normalization link of the present invention are as follows:

\hat{y} = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n} .;

g_{n + 1} - \overset{&OverBar;}{g_{n + 1}} = 0,

Wherein

\overset{&OverBar;}{g_{j}} = Σ_{i = 1}^{N} g_{ij};

Of the present invention have the homing method concrete steps of normalization link as follows:

(1) be provided with variable x ₁, x ₂..., x _n, y meets linear relation y=β ₀+ β ₁x ₁+ ...+β _nx _n+ ε, wherein β _i(i=0,1 .., n) is constant, and ε is stochastic error, and n > 2 carries out N observation to each variable, and observed reading is y=(y ₁, y ₂..., y _n) ', be x wherein _ijrepresent variable x _jthe i time observed reading, above data and loose point set S={ (x _i1..., x _ij, y _i) | i ∈ (1 ..., N) } equivalence, the multiple linear regression equations of the estimation based on above observation data is

\hat{y} = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n};

(2) column vector in Y and X is normalized respectively, result is merged into matrix (X ¹, Y ¹), (X ¹, Y ¹) in variable corresponding to each column vector be x ¹ ₁, x ¹ ₂.., x ¹ _n, y ¹, the method for normalized is: establishing Z is column vector, the average of each component that mean (Z) is Z, and the standard deviation of each component that std (Z) is Z, the normalized vector of Z is if Z and Z ¹corresponding variable is respectively z and z ¹, its relational expression is

f_{n + 1} - \overset{&OverBar;}{f_{n + 1}} = 0,

Wherein

\overset{&OverBar;}{f_{j}} = Σ_{i = 1}^{N} f_{ij};

(6) by formula in each variable x ₁, x ₂.., x _n, y represents, arranges to be

y = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n}

Form,

\hat{y} = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n}

Claims

1. the multiple linear regression analysis method based on dimensionality reduction lineoid, is characterized in that, step is as follows:

\hat{y} = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n};

g_{n + 1} - \overset{&OverBar;}{g_{n + 1}} = 0,

Wherein

\overset{&OverBar;}{g_{j}} = Σ_{i = 1}^{N} g_{ij};

2. the multiple linear regression analysis method based on dimensionality reduction lineoid, is characterized in that, step is as follows:

\hat{y} = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n};

f_{n + 1} - \overset{&OverBar;}{f_{n + 1}} = 0,

Wherein

\overset{&OverBar;}{f_{j}} = Σ_{i = 1}^{N} f_{ij};

y = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n}

Form,

\hat{y} = {\hat{β}}_{0} + {\hat{β}}_{1} x_{1} + . . . + {\hat{β}}_{n} x_{n}