CN113779502B - Image processing evidence function estimation method based on correlation vector machine - Google Patents

Abstract

The invention discloses an image processing evidence function estimation method based on a correlation vector machine. Step 1: Use the mean and covariance in the normal distribution to prove that the posterior distribution of the weight parameters is a normal distribution according to the corrected prior form of the data in the image; Step 2: Use the data in the image according to multiple The product of the variable Taylor formula, the likelihood function and the prior distribution of the weight is integrated with the weight parameter to obtain the specific expression of the evidence function, that is, the marginal likelihood function; step 3: the marginal likelihood function based on the data in the image in step 2, Using matrix calculus, matrix algebra, and optimization methods to maximize the evidence function containing hyperparameters, an iterative algorithm for optimizing each hyperparameter of the image is obtained. The present invention is used to solve the problem that the product of the likelihood function and the prior distribution of the weight is integrated with the weight parameter to obtain the evidence function in the image processing, and the problem of having to face complex integrals and relatively difficult calculations.

Description

A method for estimating evidence function in image processing based on relevance vector machine

Technical Field

The invention belongs to the field of image processing, and in particular relates to an image processing evidence function estimation method based on a correlation vector machine.

Background Art

In the process of estimating the evidence function related to the correlation vector machine in the field of image processing, it is necessary to prove that the posterior distribution is a normal distribution and find the mean and covariance of the normal distribution. The current traditional method has to find the perfect square term, which is not only difficult to operate, but also lacks logic. At the same time, in the evidence function principle related to the correlation vector machine in the field of image processing, the prior distribution of the weight parameter is the product of some normal distributions with a mean of zero, which lacks generality.

In the process of integrating the weight parameter by multiplying the likelihood function by the prior distribution of the weight to obtain the evidence function, we have to face complex integrals that are difficult to calculate. How to find a new image processing method and logical framework to more simply and logically calculate the relevant integrals of image processing and more effectively maximize the evidence function? There is still relatively little research in this area.

Summary of the invention

The present invention provides an image processing evidence function estimation method based on a correlation vector machine, which is used to solve the problem of complex integration and difficult calculation in the process of integrating the weight parameter to obtain the evidence function by multiplying the likelihood function with the prior distribution of the weight in image processing.

The present invention is achieved through the following technical solutions:

An image processing evidence function estimation method based on a relevance vector machine, the evidence function estimation method comprising the following steps:

Step 1: The data in the image are converted into the modified prior form of the weight parameters, and the mean and covariance in the normal distribution are used to prove that the posterior distribution of the weight parameters is a normal distribution;

Step 2: Integrate the weight parameter by multivariate Taylor formula, the likelihood function and the product of the prior distribution of the weight to obtain the specific expression of the evidence function, i.e., the marginal likelihood function.

Step 3: Based on the marginal likelihood function of the data in the image in step 2, matrix calculus, matrix algebra and optimization methods are used to maximize the evidence function containing hyperparameters, thereby obtaining an optimized iterative algorithm for each hyperparameter of the image.

Furthermore, the step 1 is specifically, when where x is a scalar, A is an n×n reversible symmetric matrix, Tr(·) is the trace of the matrix, Tr(xy ^T )＝x ^T y, where Tr(·) is the trace of the matrix, x is a vector, y is a vector; the operator Defined as and where k∈N;

Let h = (h ₁ ,h ₂ ) ^T and Where _h1 is the first component of vector h, _h2 is the second component of vector h, _x1 is the first component of vector x, _x2 is the second component of vector x,

Available and

When the operator Acting on x, we get

According to the operator The formula for the effect on x is:

Where h = (h ₁ ,h ₂ ) ^T and

use Available

If f:B(w,r)→R is defined as f(w)＝w ^T Aw+w ^T b+c; where A is an n×n reversible symmetric matrix, b and w are n-dimensional column vectors, and c is a scalar; the Taylor equation of f(w) is The Taylor expansion of

The (i,j)th element of H is given by definition;

The general form of the linear regression model in machine learning is

Where φ _i (x) is the nonlinear basis function of the input variable, w ₀ is the deviation parameter, and x is the image data vector;

Define φ ₀ (x) = 1, so that equation (1) can be rewritten as

Where w=(w ₀ ,…,w _M-1 ) ^T and φ(x)=(φ ₀ (x),…, φ _M-1 (x)) ^T ;

The objective function is a deterministic function y(x,w) with additive Gaussian noise, that is

t＝y(x,w)+ε (3)

where ε is a normal random variable with mean 0 and precision β, thus obtaining

p(t|x,w,β)=N(t|y(x,w),β ^-1 ) (4).

Furthermore, the weight parameter in step 1 is a priori in the following form:

Where α is the precision (inverse of variance) vector, α = (α ₁ , …, α _M ) ^T and γ is the mean vector, γ = (γ ₁ , …, γ _M ) ^T .

Using formula (4), we can obtain the likelihood function

in, M is the number of parameters to be determined.

Similarly,

where α = diag(α _i );

The posterior distribution of the weight parameters p(w|t,X,α,β,γ)＝N(w|m,Σ) is also a normal distribution, where

m＝(A+βΦ ^T Φ) ^-1 (Aγ+βΦ ^T t) (8)

Σ＝(A+βΦ ^T Φ) ^-1 (9)

in

Furthermore, the step 2 is specifically as follows:

p(t|X,α,β,γ)=∫p(t|X,w,β)p(w|α,γ)dw (10)

Using formula (6) and formula (7), we can get

in

make Obtain w=(A+βΦ ^T Φ) ^-1 (βΦ ^T t+Aγ)=m, Available

in

Using formula (11) and formula (12), we can get

Where m=(A+βΦ ^T Φ) ^-1 (βΦ ^T t+Aγ), XX＝(x ₁ ,x ₂ ,…,x _N ).

Furthermore, the step 3 is specifically as follows:

Taking the logarithm of formula (13) yields

Using formula (9), formula (14) and Available

because

use And formula (15), we can get

From formula (16), we can get

where Σ _ii is the i-th element of the main diagonal of the posterior covariance Σ;

According to formula (12), we can get

where _mi is the i-th component of the posterior mean m;

According to formula (17) and formula (18), we can get

From (19), we can get Thus we can get

Where λ _i =1-α _i Σ _ii ;

According to the definition of Σ in formula (9) and Available

according to and Tr(xy ^T )＝x ^T y, we can get

Because (A+βΦ ^T Φ)(A+βΦ ^T Φ) ^-1 =I _M , we can get

Φ ^T ΦΣ＝β ^-1 (I _M -AΣ) (23)

According to formula (22) and formula (23), we can get

From formula (12), we can get

From formula (24) and formula (25), we can get

Thus we get

According to formula (12), taking the derivative of γ, we can get make We can get γ＝m, so

γ _i =m _i (27)

Where _γi is the i-th component of γ, and _mi is the i-th component of m.

The beneficial effects of the present invention are:

The present invention adopts a more general prior form of weight parameters, rather than the traditional normal distribution with a mean of zero for each weight parameter. The parameters have a larger range of values, and the image data is then used to maximize the evidence function containing hyperparameters according to matrix calculus, matrix algebra and optimization methods, which is beneficial to improving the resolution of the image data.

BRIEF DESCRIPTION OF THE DRAWINGS

Fig. 1 is a schematic diagram of the process of the present invention.

DETAILED DESCRIPTION

The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

An image processing evidence function estimation method based on a relevance vector machine, the evidence function estimation method comprising the following steps:

Step 1: The data in the image is converted into the modified prior form of the weight parameter, and the mean and covariance in the normal distribution are used to prove that the posterior distribution of the weight parameter is a normal distribution;

Step 2: Integrate the weight parameter by multivariate Taylor formula, the likelihood function and the product of the prior distribution of the weight to obtain the specific expression of the evidence function, i.e., the marginal likelihood function.

Step 3: Based on the marginal likelihood function of the data in the image in step 2, matrix calculus, matrix algebra and optimization methods are used to maximize the evidence function containing hyperparameters, thereby obtaining an iterative optimization algorithm for each hyperparameter of the image.

Furthermore, the step 1 is specifically, when Where x is a scalar, A is an n×n reversible symmetric matrix, Tr(×) is the trace of the matrix, Tr(xy ^T )＝x ^T y, where Tr(·) is the trace of the matrix, x is a vector, y is a vector; the operator Defined as and where k∈N;

Let h = (h ₁ ,h ₂ ) ^T and Where _h1 is the first component of vector h, _h2 is the second component of vector h, _x1 is the first component of vector x, _x2 is the second component of vector x,

Available and

When the operator Acting on x, we get

According to the operator The formula x acting on x (remove this x) gives

Where h = (h ₁ ,h ₂ ) ^T and

use Available

If f:B(w,r)→R is defined as f(w)＝w ^T Aw+w ^T b+c; where A is an n×n reversible symmetric matrix, b and w are n-dimensional column vectors, and c is a scalar; the Taylor equation of f(w) is The Taylor expansion of

The (i,j)th element of H is given by definition;

The general form of the linear regression model in machine learning is

Where φ _i (x) is the nonlinear basis function of the input variable, w ₀ is the deviation parameter, and x is the image data vector;

Define φ ₀ (x) = 1, so that equation (1) can be rewritten as

Where w=(w ₀ ,…,w _M-1 ) ^T and φ(x)=(φ ₀ (x),…, φ _M-1 (x)) ^T ;

The objective function is a deterministic function y(x,w) with additive Gaussian noise, that is

t＝y(x,w)+ε (3)

where ε is a normal random variable with mean 0 and precision β, thus obtaining

p(t|x,w,β)=N(t|y(x,w),β ^-1 ) (4).

Furthermore, the weight parameter in step 1 is a priori in the following form:

Where α is the precision (inverse of variance) vector, α = (α ₁ , …, α _M ) ^T and γ is the mean vector, γ = (γ ₁ , …, γ _M ) ^T .

Using formula (4), we can obtain the likelihood function

in, M is the number of parameters to be determined.

Similarly,

where α = diag(α _i );

The posterior distribution of the weight parameters p(w|t,X,α,β,γ)＝N(w|m,Σ) is also a normal distribution, where

m＝(A+βΦ ^T Φ) ^-1 (Aγ+βΦ ^T t) (8)

Σ＝(A+βΦ ^T Φ) ^-1 (9)

in

Before proving this, let’s look at the following:

normal distribution Take the negative exponent of the normal distribution as

make We can obtain x = μ, which implies that the stationary point of f(x) is the mean of the normal distribution, and The second-order gradient of f(x) is the inverse of the covariance.

The following proves that p(w|t,X,α,β,γ) is normally distributed;

From formula (6) and formula (7), we can get the negative exponent of the product p(t|X,w,β)p(w|α,γ) is

make therefore,

m=w=(A+βΦ ^T Φ) ^-1 (Aγ+βΦ ^T t),

Also because Therefore, Σ＝(A+βΦ ^T Φ) ^-1 , proved.

Furthermore, the step 2 is specifically as follows:

p(t|X,α,β,γ)=∫p(t|X,w,β)p(w|α,γ)dw (10)

Using formula (6) and formula (7), we can get

in

make Obtain w=(A+βΦ ^T Φ) ^-1 (βΦ ^T t+Aγ)=m, Available

in

Using formula (11) and formula (12), we can get

Where m=(A+βΦ ^T Φ) ^-1 (βΦ ^T t+Aγ), X＝(x ₁ ,x ₂ ,…,x _N ).

Furthermore, the step 3 is specifically as follows.

Taking the logarithm of formula (13) yields

Using formula (9), formula (14) and Available

because

use And formula (15), we can get

From formula (16), we can get

where Σ _ii is the i-th element of the main diagonal of the posterior covariance Σ;

According to formula (12), we can get

where _mi is the i-th component of the posterior mean m;

According to formula (17) and formula (18), we can get

From (19), we can get Thus we can get

Where λ _i =1-α _i Σ _ii ,;

According to the definition of Σ in formula (9) and Available

according to and Tr(xy ^T )＝x ^T y, we can get

Because (A+βΦ ^T Φ)(A+βΦ ^T Φ) ^-1 =I _M , we can get

Φ ^T ΦΣ＝β ^-1 (I _M -AΣ) (23)

According to formula (22) and formula (23), we can get

From formula (12), we can get

From formula (24) and formula (25), we can get

Thus we get

According to formula (12), taking the derivative of γ, we can get make We can get γ＝m, so

γ _i =m _i (27)

Where _γi is the i-th component of γ, and _mi is the i-th component of m.

Claims (2)

1. A method for estimating an image processing evidence function based on a relevance vector machine, characterized in that the method for estimating an evidence function comprises the following steps: Step 1: The data in the image are converted into the modified prior form of the weight parameters, and the mean and covariance in the normal distribution are used to prove that the posterior distribution of the weight parameters is a normal distribution; Step 2: Integrate the weight parameter by multivariate Taylor formula, the likelihood function and the product of the prior distribution of the weight to obtain the specific expression of the evidence function, i.e., the marginal likelihood function. Step 3: Based on the marginal likelihood function of the data in the image in step 2, matrix calculus, matrix algebra and optimization methods are used to maximize the evidence function containing hyperparameters, thereby obtaining an optimized iterative algorithm for each hyperparameter of the image; The step 1 is specifically as follows: where x is a scalar, A is an n×n reversible symmetric matrix, Tr(·) is the trace of the matrix, Tr(xy ^T )＝x ^T y, where Tr(·) is the trace of the matrix and x is a vector; the operator Defined as and where k∈N; Let h = (h ₁ ,h ₂ ) ^T and Where h ₁ is the first component of vector h, h ₂ is the second component of vector h, x ₁ is the first component of vector x, and x ₂ is the second component of vector x. and When the operator Acting on x, we get According to the operator The formula for the effect on x is: Where h = (h ₁ ,h ₂ ) ^T and use Available If f:B(w,r)→R is defined as f(w)＝w ^T Aw+w ^T b+c; where A is an n×n reversible symmetric matrix, b and w are n-dimensional column vectors, and c is a scalar; the Taylor equation of f(w) is The Taylor expansion of The (i,j)th element of H is given by definition; The general form of a linear regression model in machine learning is Where φ _i (x) is the nonlinear basis function of the input variable, w ₀ is the deviation parameter, and x is the image data vector; Define φ ₀ (x) = 1, so that equation (1) can be rewritten as Where w=(w ₀ ,…,w _M-1 ) ^T and φ(x)=(φ ₀ (x),…, φ _M-1 (x)) ^T ; The objective function is a deterministic function y(x,w) with additive Gaussian noise, that is t＝y(x,w)+ε (3) where ε is a normal random variable with zero mean and precision β, thus obtaining p(t|x,w,β)=N(t|y(x,w),β ^-1 ) (4); The weight parameter in step 1 is a priori in the following form Where α is the precision vector, α = (α ₁ , …, α _M ) ^T and γ is the mean vector, γ = (γ ₁ , …, γ _M ) ^T ; Using formula (4), we can obtain the likelihood function in, M is the number of parameters to be determined; Similarly, where α = diag(α _i ); The posterior distribution of the weight parameters p(w|t,X,α,β,γ)＝N(w|m,Σ) is also a normal distribution, where m＝(A+βΦ ^T Φ) ^-1 (Aγ+βΦ ^T t) (8) Σ＝(A+βΦ ^T Φ) ^-1 (9) in The step 2 is specifically as follows: p(t|X,α,β,γ)=∫p(t|X,w,β)p(w|α,γ)dw (10) Using formula (6) and formula (7), we can get in make get w=(A+βΦ ^T Φ) ^-1 (βΦ ^T t+Aγ)=m, Available in Using formula (11) and formula (12), we can get Where m=(A+βΦ ^T Φ) ^-1 (βΦ ^T t+Aγ), X＝(x ₁ ,x ₂ ,…,x _N ). 2. According to the image processing evidence function estimation method based on relevance vector machine in claim 1, it is characterized in that the step 3 specifically comprises: Taking the logarithm of formula (13) yields Using formula (9), formula (14) and Available because use And formula (15), we can get From formula (16), we can get where Σ _ii is the i-th element of the main diagonal of the posterior covariance Σ; According to formula (12), we can get where _mi is the i-th component of the posterior mean m; According to formula (17) and formula (18), we can get From (19), we can get Thus we can get Where λ _i =1-α _i Σ _ii ; According to the definition of Σ in formula (9) and Available according to and Tr(xy ^T )＝x ^T y, we can get Because (A+βΦ ^T Φ)(A+βΦ ^T Φ) ^-1 =I _M , we can get Φ ^T ΦΣ＝β ^-1 (I _M -AΣ) (23) According to formula (22) and formula (23), we can get From formula (12), we can get From formula (24) and formula (25), we can get Thus we get According to formula (12), taking the derivative of γ, we can get make We can get γ＝m, so γ _i =m _i (27) Where _γi is the i-th component of γ, and _mi is the i-th component of m.