CN109360161B - Multispectral image deblurring method based on gradient domain prior - Google Patents

Abstract

The invention proposes a multispectral image deblurring method based on gradient domain prior, which belongs to the technical field of image processing. This method calculates the reference image corresponding to each channel in the multispectral image by using the Gauss-like function combined with the steering kernel according to the similarity of the adjacent channel images; according to the similarity between the reference image and the target clear image in the gradient domain, the two The norm of the difference in the gradient domain is used as the image prior of the deblurring formula; the image prior is combined with the maximum a posteriori probability estimation method to establish a multispectral image deblurring framework, and the iterative solution finally obtains a clear image. Compared with the existing methods, this method fully considers the similarity of adjacent channel images in the gradient domain, avoids the introduction of redundant image details, improves the quality of multispectral image deblurring, and reduces the deblurring process. amount of calculation.

Description

A Multispectral Image Deblurring Method Based on Gradient Domain Prior

technical field

The invention relates to a multispectral image deblurring method, in particular to a multispectral image deblurring method based on gradient domain prior, and belongs to the technical field of image processing.

Background technique

With the development of multi-spectral imaging technology, more and more multi-spectral imaging technology is applied to all walks of life, involving agriculture, remote sensing, microscopy and aerospace and other aspects. However, due to the limitation of the device's own weight, many lightweight multispectral imaging applications cannot be equipped with complex lens sets, and instead choose imaging systems using simple lenses. A simple lens has a large difference in refractive index for light with different wavelengths, so that these lights form different sizes of circles of confusion on the imaging plane, resulting in different degrees of defocus and blur in the images of each channel. On the one hand, these defocus blurs significantly reduce the imaging quality of the device, and on the other hand, it also affects the visual experience of the image. Therefore, there is a need for an efficient deblurring method for multispectral images that removes these defocus blurs with less computational effort.

Scholars at home and abroad have done a lot of basic research on the problem of defocusing and blurring of multispectral images. The commonly used multispectral defocusing and blurring methods are mainly divided into two types: single-channel image-based deblurring methods and multi-channel image-based deblurring methods.

The single-channel image-based deblurring method is represented by the deblurring method based on outlier handling (Dong J, Pan J, Su Z, et al. Blind image deblurring with outlier handling, ICCV. 2017). According to the influence of outliers and outliers on the deblurring algorithm, this method establishes an efficient data retention item, and then calculates the corresponding clear image of the target. However, this method does not consider the content correlation between multiple spectral images when dealing with the defocus blur of multispectral images, resulting in poor deblurring effect and large amount of computation.

Multispectral image out-of-focus deblurring using interchannel correlation, IEEE Trans.Image Process., vol. .24, no.11, pp.4433–4445, 2015) as the representative. This method mainly obtains the guide image corresponding to the target channel through Tikhonov regularization, and uses the guide image combined with the maximum posterior probability estimation method to obtain the target clear image. This method requires less computation, but because the image information of the edge channel is used in solving the guiding image, some redundant details are introduced in the deblurring process, resulting in a lower accuracy of the final clear image.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the defects of the prior art, in order to effectively solve the problem of defocusing and blurring of multispectral images, and propose a new method for deblurring multispectral images based on gradient domain prior, which can improve the performance of multispectral images. Deblurring quality while reducing the computational complexity of deblurring.

The main principle of the method of the present invention is:

According to the correlation between adjacent spectra in the multispectral image, a Gaussian-like function is used to calculate the reference image corresponding to the target spectral segment. According to the similarity between the reference image and the target clear image in the gradient domain, an image prior is established. The blurred image is preprocessed and deblurred by using the image prior combined with the maximum posterior probability estimation, and finally a clear image after deblurring is obtained.

To achieve the above object, the present invention adopts the following technical solutions:

A multispectral image deblurring method based on gradient domain prior, including the following steps:

Step 1: Calculate the blur kernel corresponding to each channel of the multispectral image.

For the multispectral images to be processed {B ₁ B ₂ ... B _N } (original images), use the blur kernel estimation method based on the normalized cross-correlation matching algorithm (refer to S.-J.Chen and H.- L.Shen, Multispectral image out-of-focus deblurring using interchannel correlation, IEEE Trans.Image Process., vol.24, no.11, pp.4433–4445, 2015), get the blur kernel corresponding to each channel {G ₁ G ₂ ... G _N }. Among them, N is a positive integer.

At the same time, a calculation model of the reference image is established, and the reference image is obtained by calculation.

Let the clear image sequence corresponding to the multispectral image {B ₁ B ₂ ... B _N } be {L ₁ L ₂ ... L _N }, according to the similarity of the adjacent channel images, using the Gauss-like function to calculate the target Spectral corresponding reference image:

Among them, R _i is the reference image, H is the preset window size, and v(i, j) represents the weight function, which determines the weight of the clear image L _j . When obtaining the reference image, some clear images may be unknown. Therefore, the constraint function δ(j) is used to constrain the selection of clear images. If L _j is unknown, the value of δ(j) is 0, otherwise δ( j) The value is 1.

In formula (1), the weight function v(i,j) determines the quality of the reference image, and its form is as follows:

是steering核回归模型，其形式如下：Among them, the functions δ(m) and δ(j) are both constraint functions.

is the steering kernel regression model of the form:

Among them, MSE(B _i ,L _j ) represents the mean square error between the original image B _i and the clear image L _j , MSE(B _i ,L _m ) represents the mean square between the original image B _i and the clear image L _m error. β is the scale operator that controls the weights.

Step 2: Establish an image prior on the gradient domain according to the reference image.

Since the reference image and the target clear image have great similarity in the gradient domain, the gradient similarity between the two is used as the image prior, as follows:

代表梯度算子，L_i和R_i分别为目标清晰图像和步骤一中计算得出的参考图像。in,

represents the gradient operator, and _Li and _Ri are the target clear image and the reference image calculated in step 1, respectively.

Step 3: Establish a deblurring process formula based on the image prior, and iteratively solve the blur kernel to obtain a clear image of the target.

Using the maximum posterior probability estimation method, combined with the image prior obtained in step 2, the formula of the deblurring process is established, which is expressed as follows:

代表梯度算子，*代表卷积操作，k代表通过计算拟得到的模糊核，G代表步骤一中计算得到的模糊核；参数λ和η分别用于控制公式(6)中第二项和第三项的比重，其值根据图像之间的均方差选取，或者人工指定。Among them, L represents the target clear image, B represents the target original image,

Represents the gradient operator, * represents the convolution operation, k represents the fuzzy kernel to be obtained by calculation, G represents the fuzzy kernel calculated in step 1; the parameters λ and η are used to control the second item and the first item in formula (6), respectively. The proportion of the three items, whose value is selected according to the mean square error between the images, or manually specified.

Since it is difficult to directly solve formula (6), it can be further decomposed into two sub-problems to solve, as follows:

与

最终得到目标清晰图像。Finally, using the alternate iterative method, iteratively solves the estimated values of the blur kernel k _i and the target clear image L _i in the frequency domain

and

Finally, a clear image of the target is obtained.

beneficial effect

(1) The traditional single-channel image-based deblurring method treats each channel image in the multispectral image as an independent image for processing, and does not consider the intrinsic relationship between the channels in the multispectral image, resulting in deblurring The resulting image is missing information. At the same time, these methods generally use formulas that are difficult to solve, such as 1-norm or 0-norm, resulting in high computational complexity. The method of the present invention improves the quality and efficiency of multi-spectral image deblurring by taking into account the content correlation between adjacent spectral images in the multi-spectral image and combining with the easy-to-solve 2-norm.

(2) The existing multi-channel image-based deblurring methods have better effects when dealing with images with less blur. However, when dealing with images with a large degree of blur, due to the serious loss of image information in this channel, information from other channels will be introduced to complete it. At this time, there will be a problem of information redundancy, that is, there will be problems that should not appear in the current channel information. This situation greatly reduces the quality of deblurring. In the present invention, the Gauss-like function is used to constrain it to only take values in a small window, thereby avoiding the situation of inconsistent content.

Description of drawings

Figure 1 is a flow chart of the method of the present invention.

Figure 2 shows the comparison between the reference image and the original image.

Detailed ways

The specific embodiments of the method of the present invention will be further described in detail below with reference to the accompanying drawings.

A multispectral image deblurring method based on gradient domain prior, including the following steps:

Step 1: Calculate the blur kernel corresponding to each channel of the multispectral image. At the same time, a calculation model of the reference image is established, and the reference image is obtained by calculation.

For the multispectral image to be processed {B ₁ B ₂ ... B _N } (original image), the image corresponding to the middle spectral segment is regarded as a clear image, and for other images, blur kernels of different sizes are used to blur them. Use a blur kernel estimation method based on a normalized cross-correlation matching algorithm (refer to S.-J.Chen and H.-L.Shen, Multispectral imageout-of-focus deblurring using interchannel correlation, IEEE Trans.ImageProcess., vol. 24, no.11, pp.4433–4445, 2015), and get the blur kernel {G ₁ G ₂ ... G _N } corresponding to each channel. Among them, N is a positive integer.

At the same time, a calculation model of the reference image is established, and the reference image is obtained by calculation.

Let the clear image sequence corresponding to the multispectral image {B ₁ B ₂ ... B _N } be {L ₁ L ₂ ... L _N }, according to the similarity of the adjacent channel images, using the Gauss-like function to calculate the target Spectral corresponding reference image:

Among them, R _i is the reference image, H is the preset window size, and v(i, j) represents the weight function, which determines the weight of the clear image L _j . When obtaining the reference image, some clear images may be unknown. Therefore, the constraint function δ(j) is used to constrain the selection of clear images. If L _j is unknown, the value of δ(j) is 0, otherwise δ( j) The value is 1.

In formula (1), the weight function v(i,j) determines the quality of the reference image, and its form is as follows:

是steering核回归模型，其形式如下：Among them, the functions δ(m) and δ(j) are both constraint functions.

is the steering kernel regression model of the form:

Among them, MSE(B _i ,L _j ) represents the mean square error between the original image B _i and the clear image L _j , MSE(B _i ,L _m ) represents the mean square between the original image B _i and the clear image L _m error. β is the scale operator that controls the weights. In this specific implementation process, β is set to 0.05, and the effect of the obtained reference image is ideal at this time.

Figure 2 shows an example of a reference image. In Figure 2, the left image is a clear image, the middle image is the obtained reference image, and the right image is the original blurred image. It can be seen that the reference image is closer to the clear image, and the similarity is Also high.

Step 2: Establish an image prior on the gradient domain according to the reference image.

Since the reference image and the target clear image have great similarity in the gradient domain, the gradient similarity between the two is used as the image prior, as follows:

代表梯度算子，L_i和R_i分别为目标清晰图像和步骤一中计算得出的参考图像。in,

represents the gradient operator, and _Li and _Ri are the target clear image and the reference image calculated in step 1, respectively.

Step 3: Establish a deblurring process formula based on the image prior, and iteratively solve the blur kernel to obtain a clear image of the target.

Using the maximum posterior probability estimation method, combined with the image prior obtained in step 2, the formula of the deblurring process is established, which is expressed as follows:

代表梯度算子，*代表卷积操作，k代表通过计算拟得到的模糊核，G代表步骤一中计算得到的模糊核；参数λ和η分别用于控制公式(6)中第二项和第三项的比重，其值根据图像之间的均方差选取，或者人工指定。Among them, L and B represent the target clear image and the original image, respectively,

Represents the gradient operator, * represents the convolution operation, k represents the fuzzy kernel to be obtained by calculation, G represents the fuzzy kernel calculated in step 1; the parameters λ and η are used to control the second item and the first item in formula (6), respectively. The proportion of the three items, whose value is selected according to the mean square error between the images, or manually specified.

Since it is difficult to directly solve formula (6), it can be further decomposed into two sub-problems to solve, as follows:

与

最终得到目标清晰图像。Step 4: Use the alternate iteration method to iteratively solve the estimated value of the blur kernel _ki and the target clear image _Li in the frequency domain

and

Finally, a clear image of the target is obtained.

The solution uses the alternate iteration method, first solve the two sub-problems described in step 3 in the frequency domain, and get:

as well as

与

分别代表二维离散傅里叶变换与傅里叶变换的共轭，

代表傅里叶变换的逆变换。

代表梯度算子的离散傅里叶变换，其形式如下：in,

and

represent the conjugate of the two-dimensional discrete Fourier transform and the Fourier transform, respectively,

Represents the inverse of the Fourier transform.

Represents the discrete Fourier transform of the gradient operator, which has the form:

与

分别代表水平方向与垂直方向的梯度算子，即

T代表转置操作。in,

and

respectively represent the gradient operators in the horizontal and vertical directions, namely

T stands for transpose operation.

When solving, since all clear images are unknown at the beginning, a preprocessing process is required to obtain clear images used as intermediate results. The preprocessing process is as follows:

First, the image corresponding to the channel in the middle of the spectrum in the multispectral image is regarded as a clear image, that is, the image corresponding to this channel is clear enough and does not need to be deblurred, and this channel is defined as s.

Then, expand from channel s to both sides, and perform a deblurring calculation on all channels in the order of "s-1s-2...1" or "s+1s+2...N", thus ensuring that There is a clear image available for each channel for subsequent iterative calculations. In practice, combined with Figure 1, the preprocessing and deblurring processes follow the following operations:

First, bring the blur kernel G _i obtained in step 1 into k _i in formula (8) to obtain the first iterative estimated value of the clear image

带入公式(7)中的L_i，计算得到模糊核的第一次迭代估计值

followed by

Bring into Li in formula (7), calculate the first _iterative estimate of the fuzzy kernel

The above process completes the first iteration of the computation.

和

分别带入到公式(8)的k_i以及公式(7)的L_i，计算得到第二次迭代的计算结果

以及

后续迭代过程与第二次迭代相同，都是使用前一次迭代的计算结果作为本次迭代计算的输入进行计算。若计算结果收敛或者迭代次数达到上限，则迭代停止。其中，收敛是指前一次迭代的计算结果与本次迭代的计算结果完全相同。In the second iteration, the calculation result of the first iteration is

and

Bring them into k _i of formula (8) and L _i of formula (7) respectively, and calculate the calculation result of the second iteration

as well as

The subsequent iteration process is the same as the second iteration, and the calculation result of the previous iteration is used as the input of this iteration calculation for calculation. If the calculation result converges or the number of iterations reaches the upper limit, the iteration stops. Among them, convergence means that the calculation result of the previous iteration is exactly the same as the calculation result of this iteration.

Both the preprocessing process and the deblurring process follow the iterative process described above. The upper limit of the number of iterations for both can be set to 3 and 10, respectively.

The resulting clear images are of high quality, and the peak signal-to-noise ratio is improved by 2 to 3 dB in the spectrum with a greater degree of ambiguity compared to the state-of-the-art method.

Claims (4)

1. a multi-spectral image deblurring method based on gradient domain prior, is characterized in that, comprises the following steps: Step 1: Calculate the blur kernel corresponding to each channel of the multispectral image; For the multispectral images to be processed {B ₁ B ₂ ... B _N }, use the blur kernel estimation method to obtain the blur kernel {G ₁ G ₂ ... G _N } corresponding to each channel, where N is a positive integer; At the same time, the calculation model of the reference image is established, and the reference image is obtained by calculation: Let the clear image sequence corresponding to the multispectral image {B ₁ B ₂ ... B _N } be {L ₁ L ₂ ... L _N }, according to the similarity of the adjacent channel images, using the Gauss-like function to calculate the target Spectral corresponding reference image:

Among them, R _i is the reference image, H is the preset window size, v(i, j) represents the weight function, which determines the weight of the clear image L _j ; use the constraint function δ(j) to constrain the selection of the clear image, if If L _j is unknown, then the value of δ(j) is 0, otherwise the value of δ(j) is 1; In formula (1), the weight function v(i,j) determines the quality of the reference image, and its form is as follows:

Among them, the functions δ(m) and δ(j) are both constraint functions;

is the steering kernel regression model of the form:

Among them, MSE(B _i ,L _j ) represents the mean square error between the original image B _i and the clear image L _j , MSE(B _i ,L _m ) represents the mean square between the original image B _i and the clear image L _m error, β is the scale operator that controls the weight; Step 2: Establish an image prior on the gradient domain according to the reference image; The gradient similarity between the reference image and the target clear image is used as the image prior, as follows:

in,

represents the gradient operator, and _Li and _Ri are the target clear image and the reference image calculated in step 1, respectively; Step 3: Establish a deblurring process formula based on the image prior, and iteratively solve the blur kernel to obtain a clear image of the target; Using the maximum posterior probability estimation method, combined with the image prior obtained in step 2, the formula of the deblurring process is established, which is expressed as follows:

Among them, L represents the target clear image, B represents the target original image,

Represents the gradient operator, * represents the convolution operation, k represents the fuzzy kernel to be obtained by calculation, G represents the fuzzy kernel calculated in step 1; the parameters λ and η are used to control the second item and the first item in formula (6), respectively. The proportion of the three items, the value of which is selected according to the mean square error between the images or directly specified manually; Here, the solution formula (6) is decomposed into two sub-problems to solve, as follows:

Finally, using the alternate iterative method, iteratively solves the estimated values of the blur kernel k _i and the target clear image L _i in the frequency domain

and

Finally, a clear image of the target is obtained. 2 . The multispectral image deblurring method based on gradient domain prior according to claim 1 , wherein, in the first step, the scale operator β of the control weight is set to 0.05. 3 . 3. a kind of multispectral image deblurring method based on gradient domain prior as claimed in claim 1, it is characterized in that, in described step 3, use alternate iteration method to iteratively solve blur kernel k _i and target in frequency domain _Estimated value of clear image Li

and

The method is as follows: First, solve the two sub-problems described in step 3 in the frequency domain, we get:

as well as

in,

and

represent the conjugate of the two-dimensional discrete Fourier transform and the Fourier transform, respectively,

represents the inverse transform of the Fourier transform;

Represents the discrete Fourier transform of the gradient operator, which has the form:

in,

and

respectively represent the gradient operators in the horizontal and vertical directions, namely

T stands for transpose operation; When solving, since all clear images are unknown at the beginning, it needs to go through a preprocessing process to obtain clear images used as intermediate results. The preprocessing process is as follows: First, the image corresponding to the channel in the middle of the spectrum in the multispectral image is regarded as a clear image, that is, the image corresponding to this channel is clear enough and does not need to be deblurred, and this channel is defined as s; Then, extend from channel s to both sides, and perform a deblurring calculation on all channels in the order of "s-1s-2...1" or "s+1s+2...N". 4. a kind of multispectral image deblurring method based on gradient domain prior as claimed in claim 3, is characterized in that, described preprocessing process and deblurring process all follow the following operations: First, bring the blur kernel G _i obtained in step 1 into k _i in formula (8) to obtain the first iterative estimated value of the clear image

followed by

Bring into Li in formula (7), calculate the first _iterative estimate of the fuzzy kernel

The above process completes the first iteration of the calculation; In the second iteration, the calculation result of the first iteration is

and

Bring them into k _i of formula (8) and L _i of formula (7) respectively, and calculate the calculation result of the second iteration

as well as

The subsequent iteration process is the same as the second iteration. The calculation result of the previous iteration is used as the input of this iteration calculation; if the calculation result converges or the number of iterations reaches the upper limit, the iteration stops, where convergence refers to the previous iteration. The calculation result of the iteration is exactly the same as the calculation result of this iteration.

Images