CN114820352A - Hyperspectral image denoising method and device and storage medium - Google Patents

Abstract

The invention discloses a hyperspectral image denoising method, a hyperspectral image denoising device and a storage medium, wherein hyperspectral image data to be denoised are input into a hyperspectral image denoising model which is pre-trained and optimized, and an output denoised hyperspectral image result is obtained; the construction method of the hyperspectral image denoising model comprises the following steps: firstly, separating noise from an original image by using low-rank tensor decomposition to obtain a degradation model; then adding an improved weighted total variation regularizer into the model to fully represent the spatial and spectral correlation among the wave bands of the hyperspectral image; then, L1 norm is used for normalizing sparse noise, and F norm is used for normalizing Gaussian noise; and finally, solving the model by using an augmented Lagrange multiplier method, and updating the optimization model by combining with the input training set image with sparse noise and Gaussian noise. The hyperspectral image denoising method disclosed by the invention can obtain a better recovery effect, and is obviously improved in quantitative evaluation and visual comparison.

Description

A kind of hyperspectral image denoising method, device and storage medium

technical field

The invention belongs to the technical field of image processing, and relates to a hyperspectral image denoising method, device and storage medium based on an improved weighted variational tensor decomposition.

Background technique

Hyperspectral images have rich spatial and spectral structure information, and are widely used in military, urban, aerospace and other fields. However, the image will be polluted by various kinds of noise during the acquisition process, such as Gaussian, salt and pepper, band noise, etc., which will seriously degrade the quality of the hyperspectral image. Therefore, it is necessary to denoise the hyperspectral image to restore a clear image close to the original from the degraded image.

A natural approach to denoising hyperspectral images is to treat each band as a grayscale image and then denoise band by band using traditional 2D or 1D denoising methods; subsequent methods exploit the presence of similarities in adjacent image pixels and spatial characteristics, the total variational regularization method is used to achieve smooth spatial segmentation, and the edge information of the image is processed to improve the accuracy of image restoration. These methods are all designed to remove one or two types of noise, namely Gaussian noise, impulse noise, etc. However, the hyperspectral acquisition process is often corrupted by several different types of noise, such as Gaussian noise, impulse noise, dead lines, streaks, etc. Although methods to eliminate noise mixing have been proposed based on low-rank matrix modeling, the recovery effect is not ideal.

SUMMARY OF THE INVENTION

Purpose: Most of the image denoising algorithms in the prior art use low-rank restoration through the similarity of adjacent bands, which are second-order and ignore spatial correlation. In order to solve this problem, the present invention provides an improved weighted variable A hyperspectral image denoising method, device and storage medium based on sub-tensor decomposition, the method aims to solve various mixed noises in the hyperspectral image acquisition process.

Transforming multidimensional hyperspectral data into vectors or matrices usually destroys spectral spatial structural correlations, and tensor-based modeling techniques have advantages over matrixing techniques. Using third-order tensors to simultaneously capture non-local similarity and spectral correlation in spectral space. The tensor-based method basically maintains the inherent structural correlation with better recovery results.

Technical scheme: in order to solve the above-mentioned technical problems, the technical scheme adopted in the present invention is:

In a first aspect, a hyperspectral image denoising method is provided, including:

Obtain the hyperspectral image data to be denoised;

Input the hyperspectral image data to be denoised into the pre-trained optimized hyperspectral image denoising model, and obtain the output denoised hyperspectral image result;

The method for constructing the hyperspectral image denoising model includes:

1), use low-rank tensor decomposition to separate the noise term, and obtain the hyperspectral degradation model;

2), adding an improved weighted total variation regularizer (w-SSTV) to the hyperspectral degradation model obtained in step 1) to fully characterize the spatial and spectral correlations between the hyperspectral image bands;

3) In the model obtained in step 2), use the L1 norm to normalize the sparse noise, and use the F norm to normalize the Gaussian noise, and determine the constraints;

4), use the augmented Lagrange multiplier method to solve the model obtained in step 3), and iteratively optimize the model by inputting the training set images with sparse noise and Gaussian noise, and obtain the optimized hyperspectral image denoising. Model.

In some embodiments, the hyperspectral degradation model includes:

Y=X+N+S

where Y represents a noisy third-order tensor hyperspectral cube Y={Y ¹ , Y ¹ ,…Y ^B }, where Y ⁱ ∈R ^h×w , h represents height, i is frequency band, and width is w, B is the number of bands; X is clean image, N is Gaussian noise, S is sparse noise, X, N, S and Y have the same tensor size.

In some embodiments, the noise terms are separated using a low-rank tensor decomposition, including:

Divide the hyperspectral image into overlapping 3D blocks, for a tensor of order n, decompose it into n factor matrices and a core tensor by Tucker decomposition to take full advantage of the spectral and spatial low-rank properties of the image; factor on each modality The matrix is called the basis matrix or principal component of the tensor; the Tucker decomposition equation is expressed as:

X=C× ₁ U ₁ × ₂ U ₂ ×…× _n U _n ,U _n ^T U _n =I

之间相互作用的核心张量；U是系数矩阵。where C is the control factor matrix

The core tensor of interactions between ; U is the coefficient matrix.

In some embodiments, an improved weighted total variation regularizer is added to the hyperspectral degradation model, including:

||X|| _SSTV =w ₁ ||D _x X||+w ₂ ||D _y X||+w ₃ ||D _z X||

Among them, ||X|| _SSTV is a weighted total variation model, D _x , D _y , D _z represent the first-order forward finite difference operator along the spatial horizontal direction x, the spatial vertical direction y and the spectral direction z, respectively; w ₁ , w ₂ , and w ₃ represent the weighted difference operators in the three directions of x, y, and z, respectively;

D _x =X(i+1,j,k)-X(i,j,k)

D _y =X(i,j+1,k)-X(i,j,k)

D _z =X(i,j,k+1)-X(i,j,k)

In X(i, j, k), i and j represent the horizontal and vertical spatial positions of the image X, respectively, and k represents the kth band of the image X.

In some embodiments, the L1 norm is used to normalize sparse noise and the F norm is used to normalize Gaussian noise, and constraints are determined, including:

s.t.Y=X+S+N,

X=C× ₁ U ₁ × ₂ U ₂ × ₃ U ₃ , U _n ^T U _n =I

where X represents a clean image, N represents Gaussian noise, S represents sparse noise, τ, λ, β represent the respective control coefficient factors of X, S, N, st represents the constraints of the model; C is the control factor matrix U ₁ , The core tensor of the interaction between U ₂ , U ₃ ; U is the coefficient matrix.

In some embodiments, the augmented Lagrangian multiplier method is used to solve, including:

Let X=Z, _Dw (Z)=F, _Dw is the weighted three-dimensional difference operator, there are three first-order difference operators in different directions, Z and F both represent the augmented Lagrange multiplier method The auxiliary variable introduced in ; the augmented Lagrangian function L is expressed as follows:

where μ is the penalty parameter, μ ₁ , μ ₂ , μ ₃ , which represent augmented Lagrange multipliers.

In some embodiments, the model is iteratively optimized by inputting training set images with sparse noise and Gaussian noise, including:

First input the training set image with sparse noise and Gaussian noise, the rank r of the matrix, the weight values w ₁ , w ₂ , w ₃ , initialize the hyperspectral image X, according to the augmented Lagrange multiplier decree X=Z= S=N=0, μ ₁ = μ ₂ = μ ₃ =0, k=0, and then update X, Z, F, S, N and the new generalized Lagrange multipliers μ ₁ , μ ₂ respectively, μ ₃ , the model is iteratively optimized for k+1 times until the iterative stop condition is satisfied.

In a second aspect, the present invention provides a hyperspectral image denoising device, including a processor and a storage medium;

the storage medium is used for storing instructions;

The processor is operable according to the instructions to perform the steps of the method according to the first aspect.

In a third aspect, the present invention provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of the method in the first aspect.

Beneficial effects: The hyperspectral image denoising method and device provided by the present invention have the following advantages: the hyperspectral image denoising method based on weighted variational tensor decomposition is better than the image denoising method in removing a variety of mixed noises The image restoration effect, the structural similarity and the peak signal-to-noise ratio are significantly improved, which can improve the smoothness of the spectral space and obtain better visual effects on the image.

Description of drawings

FIG. 1 is a flowchart of a method for constructing a hyperspectral image denoising model according to an embodiment of the present invention.

Detailed ways

The present invention will be further described below with reference to the accompanying drawings and embodiments. The following examples are only used to illustrate the technical solutions of the present invention more clearly, and cannot be used to limit the protection scope of the present invention.

In the description of the present invention, the meaning of several means one or more, the meaning of multiple means two or more, greater than, less than, exceeding, etc. are understood as not including this number, above, below, within, etc. are understood as including this number. If it is described that the first and the second are only for the purpose of distinguishing technical features, it cannot be understood as indicating or implying relative importance, or indicating the number of the indicated technical features or the order of the indicated technical features. relation.

In the description of the present invention, reference to the terms "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" or the like is meant to be used in conjunction with the embodiment. A particular feature, structure, material or characteristic described or exemplified is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Example 1

Provide a hyperspectral image denoising method, including:

Obtain the hyperspectral image data to be denoised;

Input the hyperspectral image data to be denoised into the pre-trained optimized hyperspectral image denoising model, and obtain the output denoised hyperspectral image result;

The method for constructing the hyperspectral image denoising model includes:

1), use low-rank tensor decomposition to separate the noise term, and obtain the hyperspectral degradation model;

2), adding an improved weighted total variation regularizer (w-SSTV) to the hyperspectral degradation model obtained in step 1) to fully characterize the spatial and spectral correlations between the hyperspectral image bands;

3) In the model obtained in step 2), use the L1 norm to normalize the sparse noise, and use the F norm to normalize the Gaussian noise, and determine the constraints;

4), use the augmented Lagrange multiplier method to solve the model obtained in step 3), and iteratively optimize the model by inputting the training set images with sparse noise and Gaussian noise, and obtain the optimized hyperspectral image denoising. Model.

In some embodiments, an improved hyperspectral image denoising method with weighted total variation tensor decomposition as shown in FIG. 1 includes the following steps:

Step S1, using the low-rank tensor decomposition to separate the noise term to obtain a degenerated hyperspectral model; specifically,

The hyperspectral is first segmented into overlapping 3D blocks, and for a tensor of order n, it is decomposed into n factor matrices and a core tensor by Tucker decomposition to take full advantage of the spectral and spatial low-rank properties of the image. The factor matrix on each mode is called the basis matrix or principal component of the tensor. Tucker decomposition can also be thought of as high-order PCA, used to describe low-rank tensor approximations. The Tucker decomposition equation is expressed as:

X=C× ₁ U ₁ × ₂ U ₂ ×…× _n U _n ,U _n ^T U _n =I

之间相互作用的核心张量。U是系数矩阵，采用经典的高阶正交迭代(HOOI)算法，在不增加计算量的前提下，实现低秩Tucker分解,得到高光谱退化模型：C is the control factor matrix

The core tensor of interactions between . U is the coefficient matrix. The classical high-order orthogonal iterative (HOOI) algorithm is used to achieve low-rank Tucker decomposition without increasing the amount of calculation, and the hyperspectral degradation model is obtained:

Y=X+N+S

Y represents the hyperspectral cube of noisy third-order tensor Y={Y ¹ , Y ¹ ,…Y ^B }, where Y ⁱ ∈ R ^h×w , h represents height, i is frequency band, width is w, and B represents The number of bands; X represents clean image, N and S represent Gaussian noise and sparse noise, respectively, and they have the same tensor size as Y.

Step S2, adding an improved weighted total variation regularizer (w-SSTV); specifically,

Hyperspectral has a strong local smooth structure in its spectral modes. That is, most of the differences between adjacent frequency bands in the spectral domain are close to zero, and the number of zeros is significantly larger than that in the spatial domain. The SSTV regularization algorithm is used to study piecewise smooth structures in the spatial and spectral domains.

Introduce the anisotropic total variation model ||X|| _SSTV , where ||X|| _SSTV =||D _x X||+||D _y X||+||D _z X||,D _x , D _y , D _z represent the first-order forward finite difference operators along the spatial horizontal direction, the spatial vertical direction and the spectral direction, respectively.

D _x =X(i+1,j,k)-X(i,j,k)

D _y =X(i,j+1,k)-X(i,j,k)

D _z =X(i,j,k+1)-X(i,j,k)

In X(i,j,k) i, j represent the spatial position of the image X in the horizontal and vertical directions, respectively, and k represents the kth band of the image X;

The model is weighted w to obtain: ||X|| _SSTV =w ₁ ||D _x X||+w ₂ ||D _y X||+w ₃ ||D _z X||. The weights control how fast X is regularized. w ₁ , w ₂ , and w ₃ represent weighted difference operators in the three directions of x, y, and z, respectively.

Step S3, use the L1 norm to normalize the sparse noise, and the F norm to normalize the Gaussian noise; specifically, the S sparse noise is modeled by the L1 norm, and the N Gaussian noise is modeled by the F norm to obtain the following expression

s.t.Y=X+S+N,

X=C× ₁ U ₁ × ₂ U ₂ × ₃ U ₃ , U _n ^T U _n =I

τ, λ, and β represent the respective control coefficient factors of X, S, and N, respectively, and s.t. represents the constraints of the model, which fully obtains the similarity of the spectral image pixels and the segmental smoothness of the spatial spectrum, reduces artifacts, and removes mixed noise. .

Step S4, performing image denoising by the augmented Lagrange multiplier method, specifically,

The model obtained in the above step S3 is optimized, and X=Z, _Dw (Z)=F, _Dw is a weighted three-dimensional difference operator, there are three first-order difference operators in different directions, and the ALM model expresses The formula is as follows:

μ is the penalty parameter, μ ₁ , μ ₂ , μ ₃ , which represent augmented Lagrange multipliers. Denoising algorithm: first input the image with Gaussian and sparse noise, the rank r of the matrix, the weight value w, the iterative stop criterion, etc., initialize the hyperspectral image X, and according to the augmented Lagrange multiplier law X=Z=S =N=0, μ ₁ =μ ₂ =μ ₃ =0, k=0, then update X, Z, F, S, N and Lagrange multipliers μ ₁ , μ ₂ , μ ₃ respectively, etc., fixing the other variables, and doing k+1 iterations of the model.

The mixed noise experiment is performed on the indian data set, and the final output is the hyperspectral image restoration result.

The following Tables 1-2 show the comparison of the effects of the image denoising method shown in the present invention and the traditional prior art under two experiments:

Experiment 1: Both Gaussian noise and salt and pepper impulse noise are added to all bands of the indian dataset. The variances of Gaussian noise (G) are 0.02, 0.06, and 0.1, respectively. At the same time, salt and pepper impulse noise (P) is also added to all frequency bands to simulate sparse noise. The percentages of impulse noise (P) were 0.04, 0.12 and 0.2, respectively.

Experiment 2: On the basis of Experiment 1, add dead lines noise (deadlines), and other parameters remain unchanged.

Three different simulated observation data were recovered by the method in this paper and the comparison method. Take the average of the SSIM and PSNR of all channels of the restoration result, denoted as MSSIM and MPSNR, and use these two averages as the evaluation standard for the final restoration effect. Peak signal-to-noise ratio (PSNR) is based on error-sensitive image quality assessment. Given a clean image X and a noisy image Y of a size, PSNR is defined as:

M, N represent the spatial dimensions of the hyperspectral image, namely width and height, i, j represent the spatial position, and B represent the number of bands of the hyperspectral image. When the value of the peak signal-to-noise ratio (PNSR) is larger, the distortion of the image is smaller, and the restored image is closer to the real image.

Structural similarity SSIM is defined as:

μ _u , μ _u′ represent the pixel average of image Y and image X, respectively, σ _u , σ _u′ represent the variance of the image, C1, C2 are constants, and B represents the number of bands of the hyperspectral image. The value range of structural similarity SSIM is [0, 1], and the larger the value is, the smaller the image distortion is, and the better the image restoration effect is.

Table 1 Results of experiment 1 on the indian dataset

Table 2 Results of experiment 2 on the indian dataset

Among them, the model in the present invention is replaced by OURS, and the evaluation criteria are MSSIM and MPSNR (structural similarity and peak signal-to-noise ratio). Table 1 shows different image denoising methods. With the enhancement of Gaussian noise and impulse noise, the image restoration effect will decrease. It can be seen that the model of the present invention performs well in the indian data set. Table 2 shows the restoration effects of three kinds of mixed noises on hyperspectral images. It can be seen that the model in this paper performs well, indicating that the influence of various mixed noises on the image is well resolved. Compared with the traditional restoration method, the method provided by the present invention has a significant competitive advantage, so the method performs well in solving the hyperspectral data set with mixed noise, which has certain significance.

Example 2

In a second aspect, this embodiment provides a hyperspectral image denoising device, including a processor and a storage medium;

the storage medium is used for storing instructions;

The processor is configured to operate in accordance with the instructions to perform the steps of the method according to Embodiment 1.

Example 3

In a third aspect, this embodiment provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps of the method in Embodiment 1.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

The above is only the preferred embodiment of the present invention, it should be pointed out that: for those skilled in the art, without departing from the principle of the present invention, several improvements and modifications can also be made, and these improvements and modifications are also It should be regarded as the protection scope of the present invention.

Claims (9)

1. a hyperspectral image denoising method, is characterized in that, comprises: Obtain the hyperspectral image data to be denoised; Input the hyperspectral image data to be denoised into the pre-trained optimized hyperspectral image denoising model, and obtain the output denoised hyperspectral image result; The method for constructing the hyperspectral image denoising model includes: 1), use low-rank tensor decomposition to separate the noise term, and obtain the hyperspectral degradation model; 2), adding an improved weighted total variation regularizer to the hyperspectral degradation model obtained in step 1) to fully characterize the spatial and spectral correlations between the hyperspectral image bands; 3) In the model obtained in step 2), use the L1 norm to normalize the sparse noise, and use the F norm to normalize the Gaussian noise, and determine the constraints; 4), use the augmented Lagrange multiplier method to solve the model obtained in step 3), and iteratively optimize the model by inputting the training set images with sparse noise and Gaussian noise, and obtain the optimized hyperspectral image denoising. Model. 2. The hyperspectral image denoising method according to claim 1, wherein the hyperspectral degradation model comprises: Y=X+N+S where Y represents a noisy third-order tensor hyperspectral cube Y={Y ¹ , Y ¹ ,…Y ^B }, where Y ⁱ ∈R ^h×w , h represents height, i is frequency band, and width is w, B is the number of bands; X is clean image, N is Gaussian noise, S is sparse noise, X, N, S and Y have the same tensor size. 3. The hyperspectral image denoising method according to claim 1 or 2, characterized in that, utilizing low-rank tensor decomposition to separate noise terms, comprising: Divide the hyperspectral image into overlapping 3D blocks, for a tensor of order n, decompose it into n factor matrices and a core tensor by Tucker decomposition to take full advantage of the spectral and spatial low-rank properties of the image; factor on each modality The matrix is called the basis matrix or principal component of the tensor; the Tucker decomposition equation is expressed as: X=C× ₁ U ₁ × ₂ U ₂ ×…× _n U _n ,U _n ^T U _n =I where C is the control factor matrix

The core tensor of interactions between ; U is the coefficient matrix. 4. The hyperspectral image denoising method according to claim 1, wherein an improved weighted total variation regularizer is added to the hyperspectral degradation model, comprising: ||X|| _SSTV =w ₁ ||D _x X||+w ₂ ||D _y X||+w ₃ ||D _z X|| Among them, ||X|| _SSTV is a weighted total variation model, D _x , D _y , D _z represent the first-order forward finite difference operator along the spatial horizontal direction x, the spatial vertical direction y and the spectral direction z, respectively; w ₁ , w ₂ , and w ₃ represent the weighted difference operators in the three directions of x, y, and z, respectively; D _x =X(i+1,j,k)-X(i,j,k) D _y =X(i,j+1,k)-X(i,j,k) D _z =X(i,j,k+1)-X(i,j,k) In X(i, j, k), i and j represent the horizontal and vertical spatial positions of the image X, respectively, and k represents the kth band of the image X. 5. The hyperspectral image denoising method according to claim 1, wherein the L1 norm is used to normalize the sparse noise, the F norm is used to normalize the Gaussian noise, and the constraints are determined, comprising:

s.t.Y=X+S+N, X=C× ₁ U ₁ × ₂ U ₂ × ₃ U ₃ , U _n ^T U _n =I where X represents a clean image, N represents Gaussian noise, S represents sparse noise, τ, λ, β represent the respective control coefficient factors of X, S, N, st represents the constraints of the model; C is the control factor matrix U ₁ , The core tensor of the interaction between U ₂ , U ₃ ; U is the coefficient matrix. 6. The hyperspectral image denoising method according to claim 5, wherein the solution is solved by using the augmented Lagrangian multiplier method, comprising: Let X=Z, _Dw (Z)=F, _Dw is the weighted three-dimensional difference operator, there are three first-order difference operators in different directions, Z and F both represent the augmented Lagrange multiplier method The auxiliary variable introduced in ; the augmented Lagrangian function L is expressed as follows:

where μ is the penalty parameter, μ ₁ , μ ₂ , μ ₃ , which represent augmented Lagrange multipliers. 7. The hyperspectral image denoising method according to claim 6, wherein the model is iteratively optimized by inputting a training set image with sparse noise and Gaussian noise, comprising: First input the training set image with sparse noise and Gaussian noise, the rank r of the matrix, the weight values w ₁ , w ₂ , w ₃ , initialize the hyperspectral image X, according to the augmented Lagrange multiplier decree X=Z= S=N=0, μ ₁ = μ ₂ = μ ₃ =0, k=0, and then update X, Z, F, S, N and the new generalized Lagrange multipliers μ ₁ , μ ₂ respectively, μ ₃ , the model is iteratively optimized for k+1 times until the iterative stop condition is satisfied. 8. A hyperspectral image denoising device, comprising: a processor and a storage medium; the storage medium is used for storing instructions; The processor is adapted to operate in accordance with the instructions to perform the steps of the method according to any one of claims 1 to 7. 9. A storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images