CN113779502A - A Correlation Vector Machine-Based Evidence Function Estimation Method for Image Processing - Google Patents

Abstract

The invention discloses an image processing evidence function estimation method based on a correlation vector machine. Step 1: Prove that the posterior distribution of the weight parameters is a normal distribution by using the mean and covariance in the normal distribution according to the modified prior form of the weight parameters; Step 2: Analyze the data in the image according to multiple The product of the variable Taylor formula, the likelihood function and the prior distribution of the weight is integrated with the weight parameter to obtain the specific expression of the evidence function, that is, the edge likelihood function; Step 3: Based on the edge likelihood function of the data in the image in step 2, Using matrix calculus, matrix algebra and optimization methods to maximize the evidence function containing hyperparameters, the optimal iterative algorithm for each hyperparameter of the image is obtained. The invention is used to solve the problem of complex integration and difficult calculation in the process of integrating the product of the likelihood function and the prior distribution of the weight to obtain the evidence function in image processing.

Description

A Correlation Vector Machine Based Evidence Function Estimation Method for Image Processing

technical field

The invention belongs to the field of image processing, in particular to an image processing evidence function estimation method based on a correlation vector machine.

Background technique

In the process of estimating the evidence function related to the correlation vector machine in the field of image processing, it is necessary to prove that the posterior distribution is a normal distribution, and to find the mean and covariance of the normal distribution. At present, the traditional method has to find the perfect square term, such as The method is not only difficult to operate, but also lacks logic. At the same time, in the current evidence function principle related to correlation vector machines in the field of image processing, the prior distribution of weight parameters is the product of some normal distributions with zero mean, which lacks generality.

In the process of obtaining the evidence function by integrating the weight parameter with the product of the likelihood function and the prior distribution of the weight, it has to face the complex integration, which is difficult to calculate. How to find a new method and logical framework for image processing to obtain the correlation integral of image processing more simply and logically, and to maximize the evidence function more effectively, there are still relatively few researches in this area.

SUMMARY OF THE INVENTION

The present invention provides an image processing evidence function estimation method based on a correlation vector machine, which is used to solve the problem in the process of integrating the product of the likelihood function and the prior distribution of the weight in image processing to obtain the evidence function by integrating the weight parameters. For complex integrals, it is more difficult to calculate.

The present invention is achieved through the following technical solutions:

A correlation vector machine-based image processing evidence function estimation method, the evidence function estimation method comprising the following steps:

Step 1: Prove that the posterior distribution of the weight parameters is a normal distribution by using the mean and covariance in the normal distribution according to the modified prior form of the weight parameters in the data in the image;

Step 2: Integrate the data in the image with the weight parameter according to the multivariate Taylor formula, the product of the likelihood function and the prior distribution of the weight to obtain the specific expression of the evidence function, that is, the edge likelihood function;

Step 3: Based on the edge likelihood function of the data in the image in Step 2, the method of matrix calculus, matrix algebra and optimization is used to maximize the evidence function containing hyperparameters, so as to obtain an optimization iterative algorithm for each hyperparameter of the image.

其中x为标量，A为n×n可逆对称矩阵，Tr(·)为矩阵的迹、

Tr(xy^T)＝x^Ty，其中Tr(·)为矩阵的迹，x为向量，y为向量；算子

定义为

且

其中k∈N；Further, the step 1 is specifically, when

where x is a scalar, A is an n×n invertible symmetric matrix, Tr( ) is the trace of the matrix,

Tr(xy ^T )=x ^T y, where Tr( ) is the trace of the matrix, x is the vector, and y is the vector; the operator

defined as

and

where k∈N;

其中h₁为向量h的第一个分量，h₂为向量h的第二个分量，x₁为向量x的第一个分量，x₂为向量x的第二个分量，Let h=(h ₁ , h ₂ ) ^T and

where h ₁ is the first component of the vector h, h ₂ is the second component of the vector h, x ₁ is the first component of the vector x, and x ₂ is the second component of the vector x,

与

Available

and

作用在x，得到when operator

acting on x, we get

对x作用的公式，可得then according to the operator

The formula acting on x can be obtained

where h=(h ₁ , h ₂ ) ^T and

可得use

Available

的泰勒展开式为If f:B(w,r)→R is defined as f(w)=w ^T Aw+w ^T b+c; where A is an n×n invertible symmetric matrix, b and w are n-dimensional column vectors, and c is a scalar; the Taylor of f(w) in

The Taylor expansion of , is

定义；The (i,j)th element of H is given by

definition;

The general form of a linear regression model in machine learning is

where φ _i (x) is the nonlinear basis function of the input variable, w ₀ is the deviation parameter, and x is the image data vector;

Define φ ₀ (x)=1, so equation (1) can be rewritten as

where w=(w ₀ ,...,w _M-1 ) ^T and φ(x)=(φ ₀ (x),...,φ _M-1 (x)) ^T ;

The objective function is a deterministic function y(x,w) with additive Gaussian noise, i.e.

t=y(x,w)+ε (3)

where ε is a normal random variable with 0 mean and precision β, thus obtaining

p(t|x,w,β)=N(t|y(x,w),β ⁻¹ ) (4).

Further, in the step 1, the weight parameter prior takes the following form

where α is the precision (inverse of variance) vector, α=(α ₁ ,...,α _M ) ^T and γ are the mean vector, γ=(γ ₁ ,...,γ _M ) ^T .

Using formula (4), the likelihood function is obtained

M为所要确定的参数的个数。in,

M is the number of parameters to be determined.

Similarly,

where α=diag(α _i );

The posterior distribution p(w|t,X,α,β,γ)=N(w|m,Σ) for obtaining the weight parameters is also a normal distribution, where

m=(A+βΦ ^T Φ) ^-1 (Aγ+βΦ ^T t) (8)

Σ=(A+βΦ ^T Φ) ^-1 (9)

in

Further, the step 2 is specifically:

p(t|X,α,β,γ)=∫p(t|X,w,β)p(w|α,γ)dw (10)

Using formula (6) and formula (7), we can get

in

得到w＝(A+βΦ^TΦ)^-1(βΦ^Tt+Aγ)＝m，

可得make

Obtain w=(A+βΦ ^T Φ) ^-1 (βΦ ^T t+Aγ)=m,

Available

in

Using formula (11) and formula (12), we can get

X X＝(x₁,x₂,…,x_N)。where m=(A+βΦ ^T Φ) ^-1 (βΦ ^T t+Aγ),

XX=(x ₁ ,x ₂ ,...,x _N ).

Further, the step 3 is specifically,

Taking the logarithm of formula (13), we can get

可得Using Equation (9), Equation (14) and

Available

because

以及公式(15)，可得use

and formula (15), we can get

According to formula (16), we can get

where Σ _ii is the ith element of the main diagonal of the posterior covariance Σ;

From formula (12), we can get

where m _i is the ith component of the posterior mean m;

From formula (17) and formula (18), we can get

从而可得It can be obtained from (19)

thus obtainable

where λ _i =1-α _i Σ _ii ;

可得The definition of Σ according to equation (9) and

Available

与Tr(xy^T)＝x^Ty，可得according to

With Tr(xy ^T )=x ^T y, we can get

Because (A+βΦ ^T Φ)(A+βΦ ^T Φ) ^-1 = _IM , we can get

Φ ^T ΦΣ=β ^-1 (I _M -AΣ) (23)

According to formula (22) and formula (23), we can get

From formula (12), we can get

From formula (24) and formula (25), we can get

thereby getting

令

可得γ＝m，所以According to formula (12), the derivative of γ can be obtained

make

γ=m can be obtained, so

γ _i =m _i (27)

where γ _i is the ith component of γ, and m _i is the ith component of m.

The beneficial effects of the present invention are:

The present invention adopts a more general a priori form of weight parameters, instead of the traditional normal distribution with zero mean for each weight parameter. The optimized method maximizes the evidence function with hyperparameters, which is beneficial to improve the resolution of the image data.

Description of drawings

Fig. 1 is a schematic flow chart of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

A correlation vector machine-based image processing evidence function estimation method, the evidence function estimation method comprising the following steps:

Step 1: Prove that the posterior distribution of the weight parameters is a normal distribution by using the mean and covariance in the normal distribution according to the modified prior form of the weight parameters in the data in the image;

Step 2: Integrate the data in the image with the weight parameter according to the multivariate Taylor formula, the product of the likelihood function and the prior distribution of the weight to obtain the specific expression of the evidence function, that is, the edge likelihood function;

Step 3: Based on the edge likelihood function of the data in the image in step 2, the method of matrix calculus, matrix algebra and optimization is used to maximize the evidence function containing hyperparameters, so as to obtain the optimization iterative algorithm of each hyperparameter of the image.

其中x为标量，A为n×n可逆对称矩阵，Tr(×)为矩阵的迹、

Tr(xy^T)＝x^Ty，其中Tr(·)为矩阵的迹，x为向量，y为向量；算子

定义为

且

其中k∈N；Further, the step 1 is specifically, when

where x is a scalar, A is an n×n invertible symmetric matrix, Tr(×) is the trace of the matrix,

Tr(xy ^T )=x ^T y, where Tr( ) is the trace of the matrix, x is the vector, and y is the vector; the operator

defined as

and

where k∈N;

其中h₁为向量h的第一个分量，h₂为向量h的第二个分量，x₁为向量x的第一个分量，x₂为向量x的第二个分量，Let h=(h ₁ , h ₂ ) ^T and

where h ₁ is the first component of the vector h, h ₂ is the second component of the vector h, x ₁ is the first component of the vector x, and x ₂ is the second component of the vector x,

与

Available

and

作用在x，得到when operator

acting on x, we get

对x作用的公式x(这个x去掉)，可得then according to the operator

The formula x acting on x (this x is removed), we can get

where h=(h ₁ , h ₂ ) ^T and

可得use

Available

的泰勒展开式为If f:B(w,r)→R is defined as f(w)=w ^T Aw+w ^T b+c; where A is an n×n invertible symmetric matrix, b and w are n-dimensional column vectors, and c is a scalar; the Taylor of f(w) in

The Taylor expansion of , is

定义；The (i,j)th element of H is given by

definition;

The general form of a linear regression model in machine learning is

where φ _i (x) is the nonlinear basis function of the input variable, w ₀ is the deviation parameter, and x is the image data vector;

Define φ ₀ (x)=1, so equation (1) can be rewritten as

where w=(w ₀ ,...,w _M-1 ) ^T and φ(x)=(φ ₀ (x),...,φ _M-1 (x)) ^T ;

The objective function is a deterministic function y(x,w) with additive Gaussian noise, i.e.

t=y(x,w)+ε (3)

where ε is a normal random variable with 0 mean and precision β, thus obtaining

p(t|x,w,β)=N(t|y(x,w),β ⁻¹ ) (4).

Further, in the step 1, the weight parameter prior takes the following form

where α is the precision (inverse of variance) vector, α=(α ₁ ,...,α _M ) ^T and γ are the mean vector, γ=(γ ₁ ,...,γ _M ) ^T .

Using formula (4), the likelihood function is obtained

M为所要确定的参数的个数。in,

M is the number of parameters to be determined.

Similarly,

where α=diag(α _i );

The posterior distribution p(w|t,X,α,β,γ)=N(w|m,Σ) for obtaining the weight parameters is also a normal distribution, where

m=(A+βΦ ^T Φ) ^-1 (Aγ+βΦ ^T t) (8)

Σ=(A+βΦ ^T Φ) ^-1 (9)

in

Before proving it, look at the following:

取正态分布的负指数为normal distribution

Take the negative exponent of the normal distribution as

可以获得x＝μ，这暗示f(x)的驻点是该正态分布的均值，同时

f(x)的二阶梯度就是协方差的逆。make

It can be obtained that x = μ, which implies that the stagnation point of f(x) is the mean of the normal distribution, while

The second-order gradient of f(x) is the inverse of the covariance.

The following proves that p(w|t,X,α,β,γ) is a normal distribution;

From formula (6) and formula (7), the negative exponent of the product of p(t|X,w,β)p(w|α,γ) is

因此，make

therefore,

m=w=(A+βΦ ^T Φ) ^-1 (Aγ+βΦ ^T t),

所以Σ＝(A+βΦ^TΦ)^-1，得证。also because

So Σ=(A+βΦ ^T Φ) ^-1 , the proof is obtained.

Further, the step 2 is specifically:

p(t|X,α,β,γ)=∫p(t|X,w,β)p(w|α,γ)dw (10)

Using formula (6) and formula (7), we can get

in

得到w＝(A+βΦ^TΦ)^-1(βΦ^Tt+Aγ)＝m，

可得make

Obtain w=(A+βΦ ^T Φ) ^-1 (βΦ ^T t+Aγ)=m,

Available

in

Using formula (11) and formula (12), we can get

X＝(x₁,x₂,…,x_N)。where m=(A+βΦ ^T Φ) ^-1 (βΦ ^T t+Aγ),

X=(x ₁ , x ₂ , . . . , x _N ).

Further, the step 3 is specifically:

Taking the logarithm of formula (13), we can get

可得Using Equation (9), Equation (14) and

Available

because

以及公式(15)，可得use

and formula (15), we can get

According to formula (16), we can get

where Σ _ii is the ith element of the main diagonal of the posterior covariance Σ;

From formula (12), we can get

where m _i is the ith component of the posterior mean m;

From formula (17) and formula (18), we can get

从而可得It can be obtained from (19)

thus obtainable

where λ _i = 1-α _i Σ _ii ,;

可得The definition of Σ according to equation (9) and

Available

与Tr(xy^T)＝x^Ty，可得according to

With Tr(xy ^T )=x ^T y, we can get

Because (A+βΦ ^T Φ)(A+βΦ ^T Φ) ^-1 = _IM , we can get

Φ ^T ΦΣ=β ^-1 (I _M -AΣ) (23)

According to formula (22) and formula (23), we can get

From formula (12), we can get

From formula (24) and formula (25), we can get

thereby getting

令

可得γ＝m，所以According to formula (12), the derivative of γ can be obtained

make

γ=m can be obtained, so

γ _i =m _i (27)

where γ _i is the ith component of γ, and m _i is the ith component of m.

Claims (5)

1. An image processing evidence function estimation method based on a correlation vector machine is characterized by comprising the following steps:

step 1: the data in the image is proved to be normal distribution by using the mean value and covariance in the normal distribution according to the corrected weight parameter prior form;

step 2: integrating the weight parameters of the data in the image according to a multivariate Taylor formula and a product of a likelihood function and the prior distribution of the weight to obtain a specific expression of an evidence function, namely an edge likelihood function;

and step 3: and (3) based on the edge likelihood function of the data in the image in the step (2), maximizing the evidence function containing the hyper-parameters by utilizing a matrix calculus, a matrix algebra and an optimization method, thereby obtaining the optimization iterative algorithm of each hyper-parameter of the image.

2. The method according to claim 1, wherein the step 1 is specifically when the method comprises

Wherein x is a scalar, A is an nxn invertible symmetric matrix, Tr (-) is a trace of the matrix,

Wherein Tr (·) is the trace of the matrix, and x is the vector and is the vector; operator

Is defined as

And is

wherein k∈N；

let h be (h)₁,h₂)^TAnd

wherein h₁Is the first component of the vector h, h₂Is the second component of the vector h, x₁Is the first component of the vector x, x₂As the second component of the vector x, one obtains

And

when operator

Acting on x to obtain

Then the operator of the basis

Formula for the action of x, can be derived

wherein h＝(h₁,h₂)^TAnd

by using

Can obtain the product

If f is defined as f (w) → R^TAw+w^Tb + c; wherein A is an nxn reversible symmetric matrix, b and w are n-dimensional column vectors, and c is a scalar; taylor of f (w)

Is of Taylor expansion type

The (i, j) th element of H is composed of

Defining;

the general form of a linear regression model in machine learning is

wherein φ_i(x) As a non-linear basis function of the input variable, w₀Is a deviation parameter, x is an image data vector;

definition of phi₀(x) 1, so that formula (1) can be rewritten as

wherein w＝(w₀,…,w_M-1)^TAnd phi (x) to (phi)₀(x),…,φ_M-1(x))^T；

The objective function is a deterministic function y (x, w) with additive Gaussian noise, i.e.

t＝y(x,w)+ε (3)

Where ε is the mean of 0 and the accuracy is the normal random variable of β, thereby obtaining

p(t|x,w,β)＝N(t|y(x,w),β^-1) (4)。

3. The method for estimating an image processing evidence function based on a relevance vector machine according to claim 1, wherein the weight parameter prior in the step 1 is in the form of

Where α is the precision vector, α ═ α₁,…,α_M)^TAnd γ is the mean vector, γ ═ γ₁,…,γ_M)^T。

Using equation (4), a likelihood function is obtained

wherein ,

m is the number of parameters to be determined.

In a similar manner to that described above,

wherein α ═ diag (α)_i)；

The posterior distribution p (w | t, X, α, β, γ) of the obtained weight parameters is also a normal distribution, where N (w | m, Σ) is

m＝(A+βΦ^TΦ)^-1(Aγ+βΦ^Tt) (8)

Σ＝(A+βΦ^TΦ)^-1 (9)

wherein

4. The method for estimating an image processing evidence function based on a correlation vector machine according to claim 1, wherein the step 2 is specifically,

p(t|X,α,β,γ)＝∫p(t|X,w,β)p(w|α,γ)dw (10)

using the formula (6) and the formula (7), it is possible to obtain

wherein

Order to

To obtain

Can obtain the product

wherein

Using the formula (11) and the formula (12), it is possible to obtain

Wherein m ═ a + β Φ^TΦ)^-1(βΦ^Tt+Aγ)，

X＝(x₁,x₂,…,x_N)。

5. The method for estimating an image processing evidence function based on a correlation vector machine according to claim 4, wherein the step 3 is specifically,

logarithm of the formula (13) is obtained

Using equations (9), (14) and

can obtain the product

Because of the fact that

By using

And equation (15), can be derived

From the formula (16), it can be found

wherein Σ_iiIs the ith element of the main diagonal of the a posteriori covariance Σ;

from the formula (12), it can be found

wherein m_iIs the ith component of the posterior mean m;

from the formula (17) and the formula (18), it can be obtained

From (19) to

Thereby obtaining

wherein λ_i＝1-α_iΣ_ii；

Definition of Σ according to equation (9) and

can obtain the product

According to

And Tr (xy)^T)＝x^Ty, is obtained

Because (A + beta. phi)^TΦ)(A+βΦ^TΦ)^-1＝I_MIs obtained by

Φ^TΦΣ＝β^-1(I_M-AΣ) (23)

From the formula (22) and the formula (23), it is possible to obtain

From the formula (12), it can be obtained

From the formula (24) and the formula (25), it can be obtained

Thereby obtaining

According to equation (12), the derivative of γ can be obtained

Order to

Can get gamma as m, so

γ_i＝m_i (27)

wherein γ_iIs the i-th component of γ, m_iIs the ith component of m.

Images