CN113706418A - Long-wave infrared remote sensing image recovery method based on spectral separation - Google Patents
Long-wave infrared remote sensing image recovery method based on spectral separation Download PDFInfo
- Publication number
- CN113706418A CN113706418A CN202111040994.6A CN202111040994A CN113706418A CN 113706418 A CN113706418 A CN 113706418A CN 202111040994 A CN202111040994 A CN 202111040994A CN 113706418 A CN113706418 A CN 113706418A
- Authority
- CN
- China
- Prior art keywords
- wave infrared
- image
- long
- medium
- noise
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000926 separation method Methods 0.000 title claims abstract description 28
- 238000000034 method Methods 0.000 title claims abstract description 26
- 238000011084 recovery Methods 0.000 title claims abstract description 10
- 230000003595 spectral effect Effects 0.000 title claims abstract description 10
- 230000005855 radiation Effects 0.000 claims abstract description 51
- 238000002329 infrared spectrum Methods 0.000 claims abstract description 9
- 238000001228 spectrum Methods 0.000 claims abstract description 4
- 238000004422 calculation algorithm Methods 0.000 claims description 78
- 238000001914 filtration Methods 0.000 claims description 18
- 238000000354 decomposition reaction Methods 0.000 claims description 5
- 239000011159 matrix material Substances 0.000 claims description 5
- 238000005457 optimization Methods 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims description 2
- 230000001186 cumulative effect Effects 0.000 claims description 2
- 238000009795 derivation Methods 0.000 claims description 2
- 239000000126 substance Substances 0.000 claims description 2
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 claims 1
- 238000012216 screening Methods 0.000 claims 1
- 238000000638 solvent extraction Methods 0.000 claims 1
- 230000000694 effects Effects 0.000 abstract description 4
- 238000002474 experimental method Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000001514 detection method Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
Landscapes
- Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
- Image Processing (AREA)
Abstract
A method for recovering a long-wave infrared remote sensing image based on spectral separation relates to the technical field of long-wave infrared remote sensing image recovery. The invention aims to solve the problem that the space signal-to-noise ratio of the existing long-wave infrared remote sensing image is low. The method specifically comprises the following steps: inputting a long-wave infrared remote sensing image, a medium-wave infrared remote sensing image and a corresponding visible light near-infrared remote sensing image which are polluted by noise, and initializing parameters; establishing a medium wave infrared spectrum separation model based on low rank constraint and sparse constraint; introducing auxiliary variables, constructing a cost function of a spectrum separation model, and respectively solving the auxiliary variables and radiation and reflection components of medium wave infrared by utilizing an alternating direction multiplier method in an iterative manner; iteration is carried out circularly until a convergence condition is met; obtaining radiation and reflection components of medium wave infrared; and inputting the radiation component of the medium-wave infrared into the combined denoising model, and outputting the repaired long-wave infrared image. The invention has obvious effect of removing the stripes and the dead lines and can recover the loss information.
Description
Technical Field
The invention relates to the technical field of long-wave infrared remote sensing image recovery in remote sensing digital image processing, in particular to a long-wave infrared remote sensing image recovery method.
Background
The long-wave infrared (thermal infrared) remote sensing image has very wide application in the aspects of night target detection, hidden target detection, forest fire monitoring and the like. However, in a normal temperature state, the intensity of energy emitted by a long-wavelength infrared band is very weak, and the difference between the level of energy reflected by the long-wavelength infrared band and the level of energy reflected by visible light is dozens of times, so that the spatial resolution of an image acquired by the thermal infrared sensor is very low in order to balance the signal-to-noise ratio and the data volume. On the other hand, since weak thermal infrared emission energy penetrates a complex atmosphere, part of the energy is interfered, and the signal to noise ratio of the formed image is low. In satellite-borne long-wave infrared images, the most common form of noise is stripe noise, which can be generally classified into two categories: one is general stripe noise, namely only the visual effect is influenced, but the image information still exists and the loss of the image content is not caused; the other is dead line noise, i.e. resulting in the loss of image content, such stripe noise severely disturbing the interpretation of the image content.
Many scholars have proposed many different algorithms based on these two forms of stripe noise. The stripe noise removal algorithm is generally classified into an inner algorithm and an outer algorithm. The internal algorithm is mainly based on the research on noise characteristics, and only utilizes a noise image to realize denoising, and the algorithm is generally based on a denoising algorithm of wavelet transformation and a variation form thereof, a denoising method in a frequency domain or a total variation algorithm. External algorithms use reference images to remove noise from images by learning noise-free reference images, and therefore such algorithms typically require multiple input variables. However, since the dead line noise in the long wave infrared generally causes a large amount of information loss, the internal algorithm does not have a good denoising effect. On the other hand, the problem of the external algorithm is that the long-wave infrared image is radiation information, and it is difficult to acquire a corresponding radiation image without noise as a reference image, so that it is also difficult to recover the long-wave infrared image by using the external algorithm.
Disclosure of Invention
The invention provides a spectrum separation-based long-wave infrared remote sensing image restoration algorithm for solving the problem that a long-wave infrared image is difficult to restore, auxiliary information is provided for long-wave infrared restoration by separating radiation energy and reflection energy in a medium-wave infrared image, and strip noise and dead lines in the long-wave infrared image are removed by utilizing a combined denoising algorithm, so that the quality of the long-wave infrared image is improved.
The technical scheme adopted by the invention for solving the technical problem is as follows:
a method for recovering a long-wave infrared remote sensing image based on spectral separation comprises the following steps:
step one, acquiring a long-wave infrared remote sensing image x polluted by noiseLWIRAnd acquiring the medium wave infrared image x and the visible light near infrared remote sensing image x which are acquired and registered by different sensors at the same time and on the same platformVNIR(ii) a The three images are m pixels long and n pixels wide in space; subjecting the medium wave infrared image to linear decomposition, i.e. x ═ x1+x2(ii) a Wherein x is1Representing the medium-wave infrared radiation component, x2Represents a medium wave infrared reflection component, and x represents a medium wave infrared image; establishing a medium wave infrared spectrum separation model based on low rank constraint and sparse constraint; initialization parameters, including balance parameter λ1、λ2And a penalty parameter beta1、β2A convergence decision threshold epsilon;
step two, introducing an auxiliary variable z by using a variable separation algorithm1And z2And reconstructing the model; then, respectively and iteratively solving and updating the auxiliary variable z by using an alternating direction multiplier method1And z2And a medium wave infrared radiation component x1And a reflected component x2Calculating a cost function of the spectrum separation model;
judging whether the output of the cost function of the two adjacent iterations is smaller than a preset convergence threshold value or not; if yes, continuing to execute the algorithm downwards; if not, returning to the third step to continue the iterative loop;
step four, outputting the medium wave infrared radiation component x1And a reflected component x2；
Step five, establishing a combined denoising model: the model recovers information lost due to the influence of dead line noise in a long-wave infrared image by utilizing the similarity of the medium-wave infrared radiation component and the gradient of long-wave infrared in a frequency domain space; then, removing the strip noise in the long-wave infrared image by using a one-dimensional guide filtering algorithm in a spatial domain;
and step six, simultaneously inputting the output medium wave infrared radiation component and the noisy long wave infrared image into a combined denoising model to finally obtain a repaired long wave infrared image.
The technical points of the invention are as follows: and inputting the registered near-infrared image, medium-wave infrared image and long-wave infrared image influenced by noise, which are acquired by different sensors on the same time and the same platform. Because the reflection component of the medium wave infrared has structural similarity with the near infrared image in space, the difference value graph of the reflection component of the medium wave infrared has sparse characteristic; meanwhile, the radiation component of the medium wave infrared has strong correlation with the long wave infrared image, so that the difference image of the medium wave infrared has low rank characteristic. Based on the method, an optimization equation is established by introducing sparsity and low-rank regular terms, and convergence of an algorithm is accelerated by using a variable separation algorithm to obtain a reflection component and a radiation component separated by a medium-wave infrared spectrum. And inputting the medium wave infrared radiation component as auxiliary information and the long wave infrared image with noise into a joint denoising algorithm model, and recovering frequency domain information and time domain information to obtain a final long wave infrared image recovery result.
The invention has the following beneficial technical effects: the long-wave infrared image restoration algorithm provided by the invention takes the radiation component image after the separation of the medium-wave infrared spectrum as a reference image, and recovers the information lost due to the strip and dead line noise in the original long-wave infrared image by using a joint denoising algorithm. The recovered long-wave infrared image has high image quality and can meet the requirements of surface temperature inversion, abnormal target detection and the like. Compared with the denoising results of a frequency domain denoising algorithm (FADSF), a one-dimensional Guided filter (1D-Guided filter), a Guided filter (Guided filter), an image block logarithm Expectation (EPLL), a three-dimensional block matching algorithm (BM3D) and a non-local denoising algorithm (NLH _ AWGN), the algorithm can better recover the information of long-wave infrared deletion, and the recovery of long-wave infrared images under the influence of stripe and dead line noise is realized.
Drawings
FIG. 1 is a flowchart of the algorithm of the present invention (a flowchart of a method for recovering a long-wave infrared remote sensing image based on spectral separation).
Fig. 2(a) is a noise-free long-wave infrared image of the XA1 data set.
FIG. 2(b) shows the denoising result of the denoising algorithm of the present invention on XA1 data.
Fig. 2(c) shows the denoising result of the frequency domain denoising algorithm (FADSF) on XA1 data.
Fig. 2(D) shows the denoising result of the XA1 data by the one-dimensional Guided filter denoising algorithm (1D-Guided filter).
Fig. 2(e) shows the denoising result of the three-dimensional block matching denoising algorithm (BM3D) on XA1 data.
Fig. 2(f) shows the denoising result of the image block logarithm expectation denoising algorithm (EPLL) on XA1 data.
Fig. 2(g) shows the denoising result of the Guided filtering denoising algorithm (Guided filter) on XA1 data.
Fig. 2(h) shows the denoising result of the non-local denoising algorithm (NLH _ AWGN) on XA1 data.
Fig. 3(a) is a noise-free long-wave infrared image of the SD1 data set.
FIG. 3(b) shows the denoising result of the denoising algorithm of the present invention on the SD1 data.
Fig. 3(c) shows the denoising result of the frequency domain denoising algorithm (FADSF) on the SD1 data.
Fig. 3(D) shows the denoising result of the SD1 data by the one-dimensional Guided filter denoising algorithm (1D-Guided filter).
Fig. 3(e) shows the denoising result of the SD1 data by the three-dimensional block matching denoising algorithm (BM 3D).
Fig. 3(f) shows the denoising result of the image block logarithm expectation denoising algorithm (EPLL) on the SD1 data.
Fig. 3(g) shows the denoising result of the Guided filtering denoising algorithm (Guided filter) on the SD1 data.
Fig. 3(h) shows the denoising result of the non-local denoising algorithm (NLH _ AWGN) on the SD1 data.
FIG. 4 shows a denoising process of a joint denoising model of SD1 data.
FIG. 5(a) each denoising algorithm recovers the peak signal-to-noise ratio curve of XA1 data.
FIG. 5(b) each denoising algorithm recovers the peak signal-to-noise ratio curve of the SD1 data.
Fig. 5(c) each denoising algorithm recovers the structural similarity curve of XA1 data.
Fig. 5(d) each denoising algorithm recovers the structural similarity curve of the SD1 data.
FIG. 6(a) the denoising of XA2 data according to the present invention.
FIG. 6(b) the denoising result of the present invention on SD2 data.
Detailed description of the invention
The invention is described in detail below with reference to the drawings and the examples.
As shown in fig. 1, the specific implementation steps of the present invention are as follows:
(1) obtaining long-wave infrared remote sensing image x polluted by noiseLWIRAnd acquiring the medium wave infrared image x and the corresponding visible light near infrared remote sensing image x which are acquired and registered by different sensors at the same time and on the same platformVNIR(ii) a The three images are m pixels long and n pixels wide in space; decomposing the medium-wave infrared image, i.e. x ═ x1+x2(ii) a Wherein x is1Representing the medium-wave infrared radiation component, x2Represents a medium wave infrared reflection component, and x represents a medium wave infrared image; establishing a medium wave infrared spectrum separation model based on low rank constraint and sparse constraint; initialization parameters, including balance parameter λ1、λ2And a penalty parameter beta1、β2A convergence decision threshold epsilon;
(2) in order to improve the convergence rate of the medium wave infrared spectrum separation algorithm and solve each variable more efficiently, an auxiliary variable z is introduced by using the variable separation algorithm1And z2And reconstructing the model. Then, respectively and iteratively solving and updating the auxiliary variable z by using an alternating direction multiplier method1And z2And a medium wave infrared radiation component x1And a reflected component x2And calculating a cost function of the spectral separation model.
(3) And judging whether the output of the cost function of the two adjacent iterations is smaller than a preset convergence threshold value. If so, the algorithm continues to execute downward. And if not, returning to the step (3) to continue the iterative loop.
(4) Outputting a medium wave infrared radiation component x1And a reflected component x2。
(5) Establishing a combined denoising model: according to the model, the similarity of the gradient of the medium wave infrared radiation component and the long wave infrared in the frequency domain space is utilized, and the information lost due to the influence of dead line noise in the long wave infrared image is recovered. Then, strip noise in the long-wave infrared image is removed in the space domain by using a one-dimensional guiding filtering algorithm.
(6) And simultaneously inputting the output medium wave infrared radiation component and the noisy long wave infrared image into a joint denoising model to finally obtain a repaired long wave infrared image.
In the step of repairing the long-wave infrared image, the specific operation of the step (1) is as follows:
firstly, linearly decomposing the medium wave infrared image and establishing a linear mixed model of medium wave infrared radiation energy and reflection energy:
x＝x1+x2 (1)
wherein x is1Representing the medium-wave infrared radiation component, x2Representing the medium wave infrared reflectance component and x representing the medium wave infrared image. The two images are m pixels long and n pixels wide in space. Because of the strong correlation between the medium wave infrared radiation component and the long wave infrared band, the difference between the medium wave infrared radiation component and the long wave infrared image containing the strip or dead line noise highlights the noise part and has the low rank characteristic. Based on the method, a low-rank regularization term epsilon of a difference value of a medium wave infrared radiation component and a long wave infrared image constrained by a nuclear norm is constructed1As shown below
Wherein x isLWIRRepresenting a thermal infrared image contaminated by noise, | · | | non-calculation*Represents the kernel norm of the matrix, which is a convex relaxed form of the low rank constraint; y is1Is the difference between the medium wave infrared radiation component and the long wave infrared image，||Y1||*Is defined as a difference image Y1The cumulative sum of singular values of. On the other hand, the medium wave infrared reflection component has strong correlation with the near infrared band image and has similar high-frequency information, so that the difference value of the medium wave infrared reflection component and the near infrared band image has the gradient sparse characteristic, and the full variation can be utilized for constraint. Based on the method, a gradient sparse constraint regular term epsilon of the difference value of the medium wave infrared reflection component and the near infrared band constrained by the total variation is constructed2，ε2Is defined as:
ε2＝||Y||TV＝||x2-xVNIR||TV (3)
wherein Y is the difference between the infrared reflection component of the medium wave and the near infrared image of the visible light, and xVNIRRepresenting a visible light near-infrared remote sensing image, | ·| non-wovenTVRepresents a fully variant constraint, defined as:
wherein D isx，DyRespectively representing the difference operator of the difference image Y in the x, Y directionsi,jThe gray scale value of the ith row and the jth column of the matrix Y is represented. By combining the gradient sparsity of the difference value between the medium wave infrared reflection component and the near infrared band and the low-rank characteristic of the difference value between the medium wave infrared radiation component and the long wave infrared band, a medium wave infrared spectrum separation model is established as follows:
wherein λ is1And λ2A balance parameter that is a balance of the two regularization terms;
the specific operation of the step (2) is as follows:
by efficiently solving for each variable in equation (5) using a variable separation method, equation (5) can be rewritten as
Wherein z is1And z2Is an auxiliary variable introduced in the solution, and is updated and calculated in each iteration, D (x)2-xVNIR)＝[Dx(x2-xVNIR),Dy(x2-xVNIR)]Representing a difference image x2-xVNIRAnd difference operators in the directions of the x and y axes. The cost function for constructing the optimization problem according to equation (6) is as follows:
wherein, beta1And beta2Is a penalty parameter, | ·| luminance1Representing the 1-norm, is the sum of the absolute values of the individual pixel values. I | · | purple wind2Represents the 2-norm, which is the arithmetic square root of the sum of the squares of the individual pixel values;
to solve the optimization problem, the variable x is sequentially updated iteratively by using an alternating direction multiplier method1，x2，z1And z2And when one variable is updated, fixing other three variables, and solving each variable one by one. Defining a variable z1，z2，x1，x2The values after n iterations are respectively z1,n，z2,n，x1,n，x2,nThen, after n iterations, the following four subproblems are solved in the (n + 1) th iteration:
first, a variable z is solved1,n+1Solving equation (8) by using a singular value threshold algorithm:
wherein, taui、ui、
Is x1,n-xLWIRAs a result of singular value decomposition of, i.e.
Then, the threshold shrinkage algorithm is used to solve equation (9):
then, since equations (10) and (11) are quadratic polynomials, x can be solved by solving the partial derivatives of the quadratic polynomials1,n+1And x2,n+1Then, we can get:
wherein E represents an identity matrix;
the specific operation of the step (5) is as follows:
the long wave infrared image and the medium wave infrared radiation component are first blocked. A window of width N x N is defined to slide within the long wave infrared and corresponding mid wave infrared radiation component images, each sliding with a step size δ of half the window width, i.e., δ N/2. The number K of image blocks is calculated to be
Then, the image blocks containing noise are filtered. Calculating the average gray value of each image block in the long-wave infrared and medium-wave infrared radiation components respectively represented as
And
i ∈ K denotes the ith image block.
And
the absolute value of the difference of (D) is defined as DEi：
In the absence of noise in the image block,
and
is small in difference, DEiIs close to 0; when stripe noise is included in an image block, DEiGreater than all DEiAverage of the sum of (a); therefore, the image block having a large gray value difference is extracted by equation (18):
wherein the content of the first and second substances,
representing the mean of all difference image blocks, DiAre extracted image blocks containing noise. For an image block without noise, the long-wave infrared and medium-wave infrared radiation components have similar gradient distribution in a frequency domain, and by utilizing the similarity of the gradients, the gradient of the medium-wave infrared radiation component can be utilized to replace the gradient of the original long-wave infrared component in the frequency domain, so that the information lost due to noise can be recovered in the frequency domain;
the long-wave infrared image repaired by the frequency domain denoising algorithm removes dead line noise, but still contains general band noise, and is represented as a form of superposition of a clean long-wave infrared image and band noise, which is specifically as follows:
I＝q+s (19)
wherein, I is the long-wave infrared image recovered by the frequency domain denoising algorithm, q is the final result after the long-wave infrared image is recovered, and s is the common strip noise. Filtering by one-dimensional guided filtering, in this case with a medium-wave infrared radiation component x1As a guide image. Setting a filter sliding window wkThe guided filtering generally assumes that the guided image and the output image are locally linear, resulting in
qk＝akx1,k+bk (20)
Wherein, akAnd bkIs the weight coefficient of the kth window, qkIs the kth window filter output. x is the number of1,kRepresenting the guide image in the k-th window. For local images within the window, the gradient of the restored image q and the guide image x1There is a linear correlation between the gradients, i.e. d (q) aD (x)1). The loss function S within the filter window is expressed as
Wherein epsilonhIs a regularization parameter to avoid akToo large. Are respectively to akAnd bkDerivation of the deviation, i.e.
Can obtain
Wherein, in the window wkInner, mukAnd
are respectively reference images x1,kThe mean and the variance of (a) is,
representative window wkIntrinsic noise image IkIs the window wkThe number of pixels in. Except for the edge region pixels, for a sliding window of step size 1, each pixel of the original filtered image is contained in | w | sliding windows. Substituting equations (22) and (23) for equation (20) yields | w | q | s for different windowskAnd averaging all the values to obtain a final q value, namely a long-wave infrared image recovery result:
wherein k | i ∈ wkIs the kth overlapping sliding window containing pixel i.
Is the average of the coefficients of the respective windows.
To demonstrate the effectiveness of the present invention, the following experiments were designed:
the invention will be explained in its effectiveness by the following experiments
1. Experimental data:
the experiment is carried out by using four groups of data of China high score five satellites. The full-wave band sensor equipped for the high-resolution five-number satellite comprises 12 wave bands, the wavelength range is 0.45-12.5 mu m, and the full-wave band sensor comprises four visible light near-infrared wave bands, two short-wave infrared wave bands, two medium-wave infrared wave bands and four long-wave infrared wave bands. The spatial resolution of the visible near-infrared band and the short-wave infrared band is 20 m/pixel, and the spatial resolution of the medium-wave infrared band and the long-wave infrared band is 40 m/pixel. On day 4 of 11 months in 2019, two sets of data were collected in west-security (N34.8, E109.5) of china, named XA1 and XA2, respectively, with sizes of 200 × 200 and 340 × 360, respectively. On 7.11.2019, two groups of data were collected in Shandong province (N35.8, E120.1) of China, named SD1 and SD2, respectively, and the sizes were 300 × 300 and 315 × 315, respectively. Of these four data sets, the SD1 data set and the XA1 data set were artificially added with 14 pieces of 1 or 2-pixel wide dead-line noise, and one piece of 4-pixel wide dead-line noise was added as an image containing noise, and the original image was used as a reference image. The two groups of data are used for evaluating the denoising effect of each denoising algorithm. The other two data sets are affected by true banding noise, i.e., SD2 is affected by a single wide dead line noise, XA2 is affected by both dead line noise and banding noise, but the width of the dead line is narrower than in SD 2.
2. Results of the experiment and analysis of the results
In order to verify the effectiveness of the algorithm provided by the invention in removing dead line and stripe noise, the invention is compared with a classical denoising algorithm and the following image denoising algorithms provided in recent years: frequency domain denoising algorithm (FADSF), 1D-Guided filtering (1D-Guided filter), Guided filtering (Guided filter), image block logarithm Expectation (EPLL), three-dimensional block matching algorithm (BM3D), and non-local denoising algorithm (NLH _ AWGN).
In the experiments with simulation data, it can be seen from fig. 2 and 3 that the single input BM3D and NLH _ AWGN algorithms, although classical de-banding algorithms, are not applicable to the banding noise mentioned herein. The EPLL algorithm can recover to some extent the information lost due to the stripe noise, but the recovered information is smoother and differs significantly from the surrounding pixels. The guiding filtering and the 1D-guiding filtering respectively recover lost information from the global direction and the single direction by taking the medium wave infrared as a template, and the lost information is large in amount, so that the lost information cannot be directly recovered through an algorithm of the guiding filtering, and even pixels which are not interfered by noise are influenced. The frequency domain information recovery utilizes the characteristic that medium wave infrared and long wave infrared have similar frequency domain distribution. However, only the disturbed area is processed, so that the processed result recovers the lost information, but has a large difference from the surrounding pixels, and even introduces new stripe noise.
The algorithm provided by the invention has the first step of recovering lost information in a frequency domain, so that dead line noise is converted into stripe noise. In a second step, the strip noise in the spatial domain is removed under guidance of the mid-wave infrared radiation component. In the second image of fig. 4, the dead lines in the original image are removed, but there are many streaks in the second image. To more clearly view the fringes, the local region with fringe noise is scaled. The fringes and texture are then separated using a one-dimensional guided filter under guidance of the mid-wave infrared radiation component. The final de-banding result is the difference image between the second image and the separated fringe image. In the final de-striping result, the same area is scaled, which is the same as the second image, and the striping is clearly observed. The denoising result of the algorithm provided by the invention can better recover the lost information, and has similar temperature distribution compared with a reference image. In order to objectively evaluate the performance of each denoising algorithm, each denoising algorithm calculates the peak signal-to-noise ratio (PSNR) and the Structural Similarity (SSIM) of each long-wavelength infrared band, and plots a graph as shown in fig. 5. As seen from the figure, the algorithm provided by the invention has more stable denoising capability.
In the experiment of real data, the algorithm provided by the invention can better remove dead line noise and stripe noise in the real long-wave infrared image, and can recover lost ground feature information such as airport information, as shown in fig. 6. And lays a foundation for the application of target identification, detection and the like.
Claims (4)
1. A method for recovering a long-wave infrared remote sensing image based on spectral separation is characterized by comprising the following steps:
step one, acquiring a long-wave infrared remote sensing image x polluted by noiseLWIRAnd acquiring the medium wave infrared image x and the visible light near infrared remote sensing image x which are acquired and registered by different sensors at the same time and on the same platformVNIR(ii) a The three images are m pixels long and n pixels wide in space; subjecting the medium wave infrared image to linear decomposition, i.e. x ═ x1+x2(ii) a Wherein x is1Representing the medium-wave infrared radiation component, x2Represents a medium wave infrared reflection component, and x represents a medium wave infrared image; establishing a medium wave infrared spectrum separation model based on low rank constraint and sparse constraint; initialization parameters, including balance parameter λ1、λ2And a penalty parameter beta1、β2A convergence decision threshold epsilon;
step two, introducing an auxiliary variable z by using a variable separation algorithm1And z2And reconstructing the model; then, respectively and iteratively solving and updating the auxiliary variable z by using an alternating direction multiplier method1And z2And a medium wave infrared radiation component x1And a reflected component x2Calculating a cost function of the spectrum separation model;
judging whether the output of the cost function of the two adjacent iterations is smaller than a preset convergence threshold value or not; if yes, continuing to execute the algorithm downwards; if not, returning to the third step to continue the iterative loop;
step four, outputting the medium wave infrared radiation component x1And a reflected component x2；
Step five, establishing a combined denoising model: the model recovers information lost due to the influence of dead line noise in a long-wave infrared image by utilizing the similarity of the medium-wave infrared radiation component and the gradient of long-wave infrared in a frequency domain space; then, removing the strip noise in the long-wave infrared image by using a one-dimensional guide filtering algorithm in a spatial domain;
and step six, simultaneously inputting the output medium wave infrared radiation component and the noisy long wave infrared image into a combined denoising model to finally obtain a repaired long wave infrared image.
2. The method for recovering the long-wave infrared remote sensing image based on the spectral separation as claimed in claim 1, wherein the specific process of the first step is as follows:
the method comprises the following steps of performing linear decomposition on a medium wave infrared image and establishing a linear mixed model of medium wave infrared radiation energy and reflection energy:
x＝x1+x2 (1)
wherein x is1Representing the medium-wave infrared radiation component, x2Represents a medium wave infrared reflection component, and x represents a medium wave infrared image; the two images are m pixels long and n pixels wide in space;
step two, constructing a low-rank regular term epsilon of a difference value of a medium wave infrared radiation component and a long wave infrared image constrained by a nuclear norm1As shown below
Wherein x isLWIRRepresenting a thermal infrared image contaminated by noise, | · | | non-calculation*Represents the kernel norm of the matrix, which is a convex relaxed form of the low rank constraint; y is1Is the difference between the medium wave infrared radiation component and the long wave infrared image, | | Y1||*Is defined as a difference image Y1The cumulative sum of the singular values of;
step three, constructing a gradient sparse regularization term epsilon of a difference value between a medium wave infrared reflection component and a near infrared band image by utilizing total variation constraint2，ε2Is defined as
ε2＝||Y||TV＝||x2-xVNIR||TV (3)
Wherein Y is the difference between the infrared reflection component of the medium wave and the near infrared image of the visible light, and xVNIRRepresenting a visible light near-infrared remote sensing image, | ·| non-wovenTVRepresents a total variation constraint, which is defined as:
wherein D isx，DyRespectively representing the difference operator of the difference image Y in the x, Y directionsi,jGray scale values of ith row and jth column of Y;
step four, combining the gradient sparsity of the difference value between the medium wave infrared reflection component and the near infrared band and the low-rank characteristic of the difference value between the medium wave infrared radiation component and the long wave infrared band, and establishing a medium wave infrared spectrum separation model as follows:
wherein λ is1And λ2To balance the balance parameters of the two regularization terms.
3. The method for recovering the long-wave infrared remote sensing image based on the spectral separation as claimed in claim 2, wherein the specific process of the second step is as follows:
step two, solving each variable in the formula (5) efficiently by using a variable separation method, so that the formula (5) can be rewritten into
Wherein z is1And z2Is an auxiliary variable introduced in the solution, carries out update calculation in each iteration,D(x2-xVNIR)＝[Dx(x2-xVNIR),Dy(x2-xVNIR)]representing a difference image x2-xVNIRDifference operators in the x, y-axis directions;
step two, constructing a cost function of the optimization problem according to the formula (6) as follows:
wherein, beta1And beta2Is a penalty parameter, | ·| luminance1Represents a 1-norm, which is the sum of the absolute values of the individual pixel values; i | · | purple wind2Represents the 2-norm, which is the arithmetic square root of the sum of the squares of the individual pixel values;
step two and step three, in order to solve the optimization problem, the alternative direction multiplier method is utilized to sequentially and iteratively update the four variables x1，x2，z1And z2When one variable is updated, fixing other three variables, and solving each variable one by one; defining a variable z1，z2，x1，x2The values after n iterations are respectively z1,n，z2,n，x1,n，x2,nThen, after n iterations, the following four subproblems are solved in the (n + 1) th iteration:
step two and step four, solving variable z1,n+1Solving the formula (8) by using a singular value threshold algorithm;
wherein, taui、ui、
Is x1,n-xLWIRAs a result of singular value decomposition of, i.e.
Step two and step five, solving the formula (9) by using a threshold shrinkage algorithm
Step two, since the formula (10) and the formula (11) are quadratic polynomials, solving x by solving the partial derivatives of the quadratic polynomials1,n+1And x2,n+1Then can obtain
Wherein E represents an identity matrix.
4. The method for recovering the long-wave infrared remote sensing image based on the spectral separation as claimed in claim 3, wherein the concrete process of the fifth step is as follows:
fifthly, partitioning the long-wave infrared image and the medium-wave infrared radiation component; defining a window of width N × N to slide within the long-wave infrared and corresponding medium-wave infrared radiation component images, the step δ of each slide being half the width of the window, i.e., δ ═ N/2; the number K of image blocks is calculated to be
Step two, screening image blocks containing noise; calculating the average gray value of each image block in the long-wave infrared and medium-wave infrared radiation components respectively represented as
And
representing the ith image block;
and
the absolute value of the difference of (D) is defined as DEi：
Step five and three, under the condition that no noise exists in the image blocks,
and
is small in difference, DEiIs close to 0; when in useWhen the image block includes the slice noise, DEiGreater than all DEiAverage of the sum of (a); therefore, the image block with a large gray value difference is extracted by equation (18);
wherein the content of the first and second substances,
representing the mean of all difference image blocks, DiIs an extracted image block containing noise; then, the similarity of the gradients of the long-wave infrared radiation component and the medium-wave infrared radiation component in the frequency domain is utilized to recover the information lost due to noise in the frequency domain;
fifthly, removing dead line noise from the long-wave infrared image repaired by using a frequency domain denoising algorithm, wherein the long-wave infrared image still contains general band noise, and the general band noise is represented as a form of superposing a clean long-wave infrared image and the band noise, and the specific steps are as follows:
I＝q+s (19)
wherein, I is a long-wave infrared image recovered by a frequency domain denoising algorithm, q is a final result after the long-wave infrared image is recovered, and s is general strip noise;
fifthly, filtering by utilizing one-dimensional guide filtering, wherein the medium wave infrared radiation component x is used1As a guide image; setting a filter sliding window wkThe guided filtering generally assumes that the guided image and the output image are locally linear, resulting in
qk＝akx1,k+bk (20)
Wherein, akAnd bkIs the weight coefficient of the kth window, qkIs the k window filter output; x is the number of1,kRepresenting the guide image in the k-th window; for local images within the window, the gradient of the restored image q and the guide image x1There is a linear correlation between the gradients, i.e. d (q) aD (x)1)；
Step five and six, expressing the loss function S in the filtering window as
Wherein epsilonhIs a regularization parameter to avoid akToo large;
step five and seven, respectively pairing akAnd bkDerivation of the deviation, i.e.
To obtain
Wherein, in the window wkInner, mukAnd
are respectively reference images x1,kThe mean and the variance of (a) is,
representative window wkIntrinsic noise image IkIs the window wkThe number of pixels within;
fifthly, except for the pixels in the edge area, for the sliding window with the step length of 1, each pixel of the original filtering image is contained in | w | sliding windows; substituting equations (22) and (23) for equation (20) yields | w | q | s for different windowskAnd averaging all the values to obtain a final q value, namely a long-wave infrared image recovery result:
wherein k | i ∈ wkIs the kth overlapping sliding window containing pixel i;
is the average of the coefficients of the respective windows.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111040994.6A CN113706418A (en) | 2021-09-07 | 2021-09-07 | Long-wave infrared remote sensing image recovery method based on spectral separation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111040994.6A CN113706418A (en) | 2021-09-07 | 2021-09-07 | Long-wave infrared remote sensing image recovery method based on spectral separation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113706418A true CN113706418A (en) | 2021-11-26 |
Family
ID=78660673
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111040994.6A Pending CN113706418A (en) | 2021-09-07 | 2021-09-07 | Long-wave infrared remote sensing image recovery method based on spectral separation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113706418A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115841435A (en) * | 2023-02-24 | 2023-03-24 | 湖南大学 | Remote sensing image banding removing method based on sorting domain direction gradient histogram constraint |
| CN116012264A (en) * | 2023-03-27 | 2023-04-25 | 山东省工业技术研究院 | Image recovery method based on sparse constraint |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109102477A (en) * | 2018-08-31 | 2018-12-28 | 哈尔滨工业大学 | A kind of high-spectrum remote sensing restoration methods based on the constraint of non-convex low-rank sparse |
| CN111292266A (en) * | 2020-01-22 | 2020-06-16 | 武汉大学 | GF-5 remote sensing image mixed noise removing method based on double low rank matrix decomposition |
| CN111951186A (en) * | 2020-07-16 | 2020-11-17 | 南京邮电大学 | Hyperspectral image denoising method based on low-rank and total variation constraint |
| CN112634167A (en) * | 2020-12-29 | 2021-04-09 | 南京理工大学 | Hyperspectral image filtering method for total variation collaborative norm constraint iterative projection |
-
2021
- 2021-09-07 CN CN202111040994.6A patent/CN113706418A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109102477A (en) * | 2018-08-31 | 2018-12-28 | 哈尔滨工业大学 | A kind of high-spectrum remote sensing restoration methods based on the constraint of non-convex low-rank sparse |
| CN111292266A (en) * | 2020-01-22 | 2020-06-16 | 武汉大学 | GF-5 remote sensing image mixed noise removing method based on double low rank matrix decomposition |
| CN111951186A (en) * | 2020-07-16 | 2020-11-17 | 南京邮电大学 | Hyperspectral image denoising method based on low-rank and total variation constraint |
| CN112634167A (en) * | 2020-12-29 | 2021-04-09 | 南京理工大学 | Hyperspectral image filtering method for total variation collaborative norm constraint iterative projection |
Non-Patent Citations (1)
| Title |
|---|
| 张静妙;高双喜;王晓娜;: "基于低秩字典学习的高光谱遥感图像去噪", 控制工程, no. 06, 20 June 2016 (2016-06-20) * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115841435A (en) * | 2023-02-24 | 2023-03-24 | 湖南大学 | Remote sensing image banding removing method based on sorting domain direction gradient histogram constraint |
| CN115841435B (en) * | 2023-02-24 | 2023-04-25 | 湖南大学 | Remote sensing image strip removing method based on ordering domain direction gradient histogram constraint |
| CN116012264A (en) * | 2023-03-27 | 2023-04-25 | 山东省工业技术研究院 | Image recovery method based on sparse constraint |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109102477B (en) | Hyperspectral remote sensing image recovery method based on non-convex low-rank sparse constraint | |
| CN108133465B (en) | Non-convex low-rank relaxation hyperspectral image recovery method based on spatial spectrum weighted TV | |
| CN106709881B (en) | A kind of high spectrum image denoising method decomposed based on non-convex low-rank matrix | |
| EP2899690B1 (en) | Random imaging | |
| CN112069919A (en) | Hyperspectral image denoising method based on non-convex low-rank matrix approximation and total variation regularization | |
| CN113706418A (en) | Long-wave infrared remote sensing image recovery method based on spectral separation | |
| Li et al. | Coupled segmentation and denoising/deblurring models for hyperspectral material identification | |
| Zeng et al. | Hyperspectral image denoising via global spatial-spectral total variation regularized nonconvex local low-rank tensor approximation | |
| Bourennane et al. | Improvement of target-detection algorithms based on adaptive three-dimensional filtering | |
| CN105551000B (en) | Remote sensing images method for reconstructing based on the constraint of reference image and the constraint of non-convex low-rank | |
| Liu et al. | Hyperspectral image restoration based on low-rank recovery with a local neighborhood weighted spectral–spatial total variation model | |
| CN111738937B (en) | Total variation remote sensing image strip removing method based on tensor decomposition | |
| CN111292266B (en) | GF-5 remote sensing image mixed noise removing method based on double low rank matrix decomposition | |
| Xu et al. | Hyperspectral image denoising by low-rank models with hyper-Laplacian total variation prior | |
| Li et al. | Thin cloud removal fusing full spectral and spatial features for Sentinel-2 imagery | |
| Su et al. | Fast hyperspectral image denoising and destriping method based on graph Laplacian regularization | |
| CN113421198B (en) | Hyperspectral image denoising method based on subspace non-local low-rank tensor decomposition | |
| CN114331976A (en) | Hyperspectral anomaly detection method based on multistage tensor prior constraint | |
| Beitollahi et al. | Using savitsky-golay smoothing filter in hyperspectral data compression by curve fitting | |
| Ely et al. | Exploiting structural complexity for robust and rapid hyperspectral imaging | |
| Liu | Random noise attenuation with weak feature preservation via total variation regularization | |
| CN114359064A (en) | Hyperspectral image recovery method based on dual gradient constraint | |
| Meenakshisundaram | Quality assessment of IKONOS and Quickbird fused images for urban mapping | |
| CN111598797A (en) | Hyperspectral image recovery method based on non-convex tensor norm and total variation | |
| Miao et al. | Thermal hyperspectral image denoising using total variation based on bidirectional estimation and brightness temperature smoothing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |