CN104574268A - Cloud and fog removing method based on non-subsample contourlet transform and non-negative matrix factorization - Google Patents

Abstract

Based on the non-subsampling contourlet transformation and non-negative matrix decomposition method for removing clouds and fog, it involves the field of image processing and remote sensing and telemetry technology, and solves the problem of loss of information on the underlying surface of clouds in the process of cloud and fog removal using existing cloud and fog removal methods , perform NSCT transformation on two remote sensing images containing clouds and fog to obtain their respective low-frequency coefficients and sub-band coefficients in each direction; perform threshold processing on the low-frequency coefficients of the two images and estimate the ground object information in the cloud area, and estimate the The two low-frequency coefficient image matrices are arranged in row-first order to form a new matrix and perform non-negative matrix decomposition to obtain low-frequency coefficients containing the common information of the two images; perform non-negative matrix decomposition and fusion processing on the two respective sub-band coefficients in each direction , to get new sub-bands in each direction; apply the new low-frequency and sub-band coefficients in each direction to perform NSCT inverse transformation, and obtain cloud-free remote sensing images. The invention effectively improves the quality of cloud and fog removal and the speed of algorithm.

Description

Cloud and fog removal method based on non-subsampling contourlet transform and non-negative matrix factorization

technical field

The present invention relates to the technical field of image processing and remote sensing and telemetry, in particular to a cloud removal method based on Non-Subsampled Contourlet Transform (Non-Subsampled Contourlet Transform for short NSCT) and Non-negative Matrix Factorization (Non-negative Matrix Factorization for short NMF).

Background technique

Remote sensing images are of great significance to a country's national defense and economic construction, such as grasping the distribution of vegetation, volcanic activities, land disasters and weather conditions, analyzing atmospheric composition and planetary detection, etc., are widely used. However, due to the coverage of a large number of remote sensing images, the areas that people care about become blurred, so that the remote sensing platform with low time resolution cannot obtain the ground information that is blocked by the clouds, which brings a lot of inconvenience to the subsequent processing of the images. It is impossible to perform image recognition and it is difficult to guarantee its accuracy in image classification. Therefore, it is particularly important to effectively reduce or remove the influence of clouds and increase the utilization rate of remote sensing data in the preprocessing of remote sensing images. Therefore, it is of great significance to find a method to effectively remove cloud cover for remote sensing image processing. The current remote sensing image cloud removal is mainly based on image processing technology, and the technology is becoming more and more mature. Image processing (Image Processing) usually refers to the technology of applying computer to analyze images to achieve the desired results.

The cloud removal method of remote sensing image refers to removing the cloud information contained in the image and recovering the information of the surface under the cloud, so as to improve the utilization rate of remote sensing image.

NSCT was first proposed by Cunha A.L in 2006, and it shows many unique advantages in the multi-resolution analysis of images. NSCT decomposes the image into different scales through the Non-subsampled Pyramid (NSP for short), and the local features of the image can be obtained at different scales; through the Non-subsampled Directional Filter Banks (Non-subsampled Directional Filter Banks, NSDFB for short) will The image is decomposed into sub-bands in each direction, which can capture the edge information of the image in each direction. Through these two transformations, NSCT can decompose the cloud layer information of remote sensing images into finer layers, which is easy to process the cloud layer.

Non-negative Matrix Factorization (NMF for short) is a matrix decomposition algorithm proposed by Lee and Seung in 1999, so that the decomposed elements of the matrix do not contain negative values. By decomposing multiple image matrices with NMF, the features of multiple images can be extracted. Therefore, NMF is widely used in the fusion of multispectral images and hyperspectral images, making the fused image information more abundant.

Contents of the invention

In order to solve the problem of loss of information on the underlying surface of the cloud layer during the cloud removal process using the existing cloud removal method, the present invention provides a cloud removal method based on non-subsampling contourlet transformation and non-negative matrix decomposition.

Based on non-subsampling contourlet transform and non-negative matrix factorization to remove cloud and fog, the method is implemented by the following steps:

Step 1. Perform NSCT transformation on two remote sensing images with different cloud layer distributions, and obtain the low-frequency coefficients of the two remote sensing images and the subband coefficients in each direction of each image respectively;

Step 2. Thresholding the low-frequency coefficients of the two remote sensing images obtained in step 1 to obtain new low-frequency coefficients of the two remote sensing images respectively;

Step 3. Estimate the new low-frequency coefficients of the two remote sensing images in step 2, and obtain the feature information of the cloud layer area in the new low-frequency coefficients; and use the new low-frequency coefficients of the two remote sensing images according to the row priority rule Arrange to form a matrix;

Step 4, carrying out NMF decomposition to the matrix described in step 3 to obtain a feature matrix containing two remote sensing image features, and using the feature matrix as a low-frequency coefficient finally containing two image features;

Step 5. Carry out NMF decomposition to the subband coefficients of each direction of each image described in step 1 to obtain new direction coefficients containing two image features;

Step 6. Perform NSCT inverse transformation on the low-frequency coefficients obtained in step 4 and the sub-band coefficients in each direction obtained in step 5 to obtain a clear cloud-removed image.

Beneficial effects of the present invention: the present invention performs NSCT transformation on two remote sensing images containing clouds and mist to obtain respective low-frequency coefficients and sub-band coefficients in each direction; respectively thresholds the low-frequency coefficients of the two images and uses an estimation method Estimate the ground object information in the cloud area, arrange the estimated two low-frequency coefficient image matrices according to the row priority order to form a new matrix, and perform non-negative matrix decomposition on this matrix to obtain the low-frequency coefficients containing the common information of the two images; The sub-band coefficients in each direction are processed by non-negative matrix decomposition and fusion to obtain new sub-bands in each direction; the new low-frequency and sub-band coefficients in each direction are used for NSCT inverse transformation to obtain cloud-free remote sensing images. The cloud removal method of sampling non-subsampling contourlet transformation and non-negative matrix decomposition in the present invention effectively improves the quality of cloud removal and the speed of the algorithm, and solves the problem that it is difficult to obtain cloud-free images in the current multivariate data fusion cloud removal method As long as the cloud area distribution of the two remote sensing images is different, the cloud removal of remote sensing images can be realized, which provides a new idea for the field of cloud removal of remote sensing images.

Description of drawings

Fig. 1 is cloud and fog flow chart based on non-subsampling contourlet transformation and non-negative matrix decomposition method according to the present invention;

Fig. 2 is that the NSP decomposition of one layer of the test image Zone Plate based on non-subsampling contourlet transformation and non-negative matrix decomposition method according to the present invention is used to obtain the corresponding low-frequency image; wherein Fig. 2a, Fig. 2b and Figure 2c are the original image of the Zone Plate, a layer of low-frequency sub-bands decomposed by NSP, and a schematic diagram of a layer of high-frequency sub-bands decomposed by NSP;

Fig. 3 is the effect diagram of the cloud and fog method based on non-subsampling contourlet transformation and non-negative matrix decomposition method according to the present invention, wherein Fig. 3a and Fig. 3b are two original cloud images A and B respectively, and Fig. 3c and Fig. 3d are respectively are the low-frequency coefficients in Figure 3a and Figure 3b, Figure 3e and Figure 3f are the low-frequency coefficient estimates in Figure 3c and Figure 3d, and Figure 3g is the image after cloud removal.

Detailed ways

Specific embodiments, in conjunction with Fig. 1 to Fig. 3 illustrate this embodiment, based on non-subsampled contourlet transformation and non-negative matrix decomposition method for removing clouds and fog, this method is realized by the following steps:

1. Perform NSCT transformation on two remote sensing cloud images A and cloud image B with different cloud layer distributions to obtain the low-frequency coefficients of cloud image A and the sub-band coefficients in each direction of cloud image A, the low-frequency coefficients of cloud image B and the sub-band coefficients in each direction of cloud image B. band coefficients; combining Figure 3c and Figure 3d in Figure 3;

Step 2, thresholding the low-frequency coefficients of the respective images obtained in step 1 and using existing estimation methods (such as formula 1) to estimate the feature information of the cloud layer area in the low-frequency coefficients, as shown in Figure 3e and Figure 3f;

Step 3, arrange the low-frequency coefficients of cloud image A and cloud image B in step 2 according to the row priority rule to form a new matrix;

Step 4, carry out NMF decomposition to the matrix in the step 3, obtain the feature matrix that contains cloud map A and cloud map B feature, use this feature matrix as final low-frequency coefficient;

Step 5, using the NMF decomposition method, carrying out non-negative matrix decomposition and fusion of the respective direction coefficients in cloud map A and cloud map B to obtain new direction coefficients;

Step 6. Perform NSCT inverse transformation on the low-frequency coefficients and direction coefficients in Step 4 and Step 5 to obtain a clear cloud-removed image, as shown in Figure 3g.

The estimation method adopted in the present embodiment is represented by formula (1):

$\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; mean</span>}}_{\mathrm{<span\; class="notranslate">\; haze</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; mean</span>}}_{\mathrm{<span\; class="notranslate">\; clear</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

in, is the low-frequency coefficient estimated by formula (1), S(i,j) is the low-frequency coefficient before estimation, mean _haze is the mean value of the cloud layer area decomposed by NSCT, and mean _clear is the mean value of the cloud-free area decomposed by NSCT .

The NSCT transformation described in this embodiment is as follows:

Assuming that there is an image matrix with M rows and N columns for NSCT transformation, first use NSP for multi-scale decomposition to obtain low-frequency coefficients and high-frequency coefficients at different scales, and then use NSDFB for direction decomposition on high-frequency coefficients to obtain sub-scales in each direction. With coefficient. Figure 2a in Figure 2 is the original image of the Zone Plate, and Figure 2b and Figure 2c are the NSP decomposition of the Zone Plate with one layer to obtain the corresponding low-frequency image. The NMF decomposition is as follows:

Suppose there are low-frequency coefficients V ₁ and V ₂ of two image matrices with M rows and N columns, and they are arranged according to the row-first rule to become matrices V′ ₁ and V′ ₂ of M×N rows and 1 column, and V′ ₁ and V ′ ₂ form a new matrix V=[V′ ₁ ,V′ ₂ ], perform NMF decomposition of matrix V V≈W _MN×1 H _1×1 , and change matrix W _MN×1 into matrix W with M rows and N columns ′ _M×N , the W′ _M×N is the low-frequency coefficient containing the common features of the two images.

Finally, the new low-frequency coefficients and the fused sub-band coefficients in each direction are subjected to NSCT inverse transformation to obtain a clear cloud-free image.

Claims (2)

1. Based on non-subsampling contourlet transformation and non-negative matrix decomposition method for removing clouds and fog, it is characterized in that the method is realized by the following steps: Step 1. Perform NSCT transformation on two remote sensing images with different cloud layer distributions, and obtain the low-frequency coefficients of the two remote sensing images and the subband coefficients in each direction of each image respectively; Step 2. Thresholding the low-frequency coefficients of the two remote sensing images obtained in step 1 to obtain new low-frequency coefficients of the two remote sensing images respectively; Step 3. Estimate the new low-frequency coefficients of the two remote sensing images in step 2, and obtain the feature information of the cloud layer area in the new low-frequency coefficients; and use the new low-frequency coefficients of the two remote sensing images according to the row priority rule Arrange to form a matrix; Step 4, carrying out NMF decomposition to the matrix described in step 3 to obtain a feature matrix containing two remote sensing image features, and using the feature matrix as a low-frequency coefficient finally containing two image features; Step 5. Carry out NMF decomposition to the subband coefficients of each direction of each image described in step 1 to obtain new direction coefficients containing two image features; Step 6. Perform NSCT inverse transformation on the low-frequency coefficients obtained in step 4 and the sub-band coefficients in each direction obtained in step 5 to obtain a clear cloud-removed image. 2. according to claim 1 based on non-subsampling contourlet transformation and non-negative matrix decomposition method for removing clouds and fog, it is characterized in that, the estimation method adopted in the step 3 obtains new low-frequency coefficients, expressed as: $\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; S</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; S</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; mean</span>}}_{\mathrm{<span\; class="notranslate">\; haze</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; mean</span>}}_{\mathrm{<span\; class="notranslate">\; clear</span>}}<span\; class="notranslate">\; )</span>$ In the formula, is the new low-frequency coefficient estimated by the formula, S(i,j) is the low-frequency coefficient of two remote sensing images, mean _haze is the mean value of the cloud layer area decomposed by NSCT, and mean _clear is the ground feature information of the cloud-free area decomposed by NSCT mean.