CN108198140A - Three-dimensional collaboration filtering and noise reduction method based on NCSR models - Google Patents

Abstract

The invention discloses a three-dimensional collaborative filtering denoising method based on an NCSR model. The specific steps are: using a Canny operator to extract edge pixel points of a noisy image, and storing the coordinates of the edge pixel points in W, thereby realizing pixel classification; Step 2, grouping the noisy images in step 1 by block matching; step 3, performing collaborative filtering and denoising on each grouped image block in step 2; step 4, performing aggregation on the weighted average of the denoised images in step 3 , obtain the final estimated value of the image; process the noisy image in different regions to further improve the clarity of the image details; the present invention's three-dimensional collaborative filtering denoising method based on the NCSR model uses the filtering formula in the NCSR model to filter the noise , making full use of the non-local similarity of the image, significantly reducing the halo and ringing effects in the denoising results.

Description

3D Collaborative Filtering Denoising Method Based on NCSR Model

technical field

The invention belongs to the technical field of image processing, and in particular relates to a three-dimensional collaborative filtering denoising method based on an NCSR model.

Background technique

In recent years, many scholars and researchers have continuously proposed new denoising methods through the study of the characteristics of image noise and image signal itself. The common goal of these methods is to retain more image detail information while achieving denoising. For a natural image, there is a great correlation between its structure and texture features. Therefore, no matter whether the spatial domain method is used to directly average the pixel values, or the frequency domain method is used to transform the image and then perform threshold processing on the transformation coefficients, it makes full use of the local and non-local correlation characteristics of the image itself to obtain a relatively accurate image. Good denoising effect.

The methods of denoising by using local similarity only process the pixels containing noise by comparing the gray value of a single pixel, and the robustness of the algorithm cannot reach the expected effect. Aiming at this defect of the local denoising method, researchers try to mine the non-local correlation of the image to further improve the denoising effect. NL-Means, a non-local denoising method, was first proposed by Buades et al., which provided new ideas for many subsequent non-local denoising methods. Among them, the BM3D algorithm is recognized as a non-local denoising algorithm with relatively good denoising effect, and its algorithm can well overcome the defect that the time complexity of NL-Means is too high. It can not only protect more image details, but also obtain and utilize the non-local similarity of images in a relatively short time. However, halo, ringing and mosaic effects appear in the denoising effect of BM3D. Because it uses a simple hard threshold filtering formula to perform local and non-local filtering on a set of image blocks, and this filtering formula is only suitable for local filtering of images. When filtering non-local image blocks, ringing effects often appear on the edge of the image.

Contents of the invention

The purpose of the present invention is to provide a three-dimensional collaborative filtering denoising method based on the NCSR model, which uses different filtering methods to separately process the edge area and smooth area of the image, and can improve the clarity of the processed image.

The technical scheme adopted in the present invention is a three-dimensional collaborative filtering denoising method based on the NCSR model, which is specifically implemented according to the following steps:

Step 1. Use the Canny operator to extract the edge pixels of the noisy image, and store the coordinates of the edge pixels in the array W;

Step 2, grouping the noisy images in step 1 by block matching to obtain a plurality of grouped image blocks;

Step 3, performing collaborative filtering to denoise each grouped image block in step 2;

Step 4. Denoising the grouped image blocks in step 3 into multiple image blocks, and aggregate the image blocks with the same coordinates in the original image by means of weighted average to obtain the final estimated value of the image.

The specific steps of step 2 are: first determine the position of the reference block, starting from the first 8×8 image block in the upper left corner of the image is the first reference block, and the position of the next reference block is shifted to the right or down by three pixels , the 39×39 area centered on each reference block is the neighborhood where the current reference block is located, and all image blocks with the same size as the reference block in the neighborhood are candidate blocks, traverse all candidate blocks in the neighborhood and find According to the Euclidean distance between it and the current reference block, 16 candidate blocks with the smallest Euclidean distance are selected as a group of similar blocks, and the group of similar blocks forms a group with the corresponding reference block.

The solution method of Euclidean distance is:

In formula (1), d ^noisy represents the Euclidean distance, ||·|| ₂ represents the l-2 paradigm, Z _{x R} represents the pixel value of the reference block, Z _x represents the pixel value of the candidate block in the neighborhood, Represents the number of pixels in an image block.

The specific steps of step 3 are:

Step 3.1, all image blocks first need to perform local two-dimensional wavelet transformation, and then select step a or step b according to whether the position of the central pixel point of the reference block in each group is in W to perform three-dimensional collaborative filtering denoising, and estimate Sparse coefficients for each image block;

a. If the position of the center pixel point of the reference block in the current group is in W, then use the formula (2) in the NCSR model to perform third-dimensional collaborative filtering on the group of image blocks;

b. If the position of the center pixel point of the reference block in the current group is not in W, then use the formula (3) in the NCSR model to perform third-dimensional collaborative filtering on the group of image blocks;

In formula (2), (3), Represents the estimated value of the sparse coefficient of the image block, α represents the sparse coefficient of the image block, β represents the average coefficient value of a group of image blocks, l ₁ =c ₁ λ, l ₂ =c ₂ γ, both c ₁ and c ₂ is a constant;

Step 3.2. After collaborative filtering is performed on the sparse coefficient values of the image blocks through step 3.1, each grouped image block is subjected to two-dimensional wavelet inverse transformation to obtain its pixel value in the spatial domain.

The specific calculation method of the final estimated value of the image in step 4 is:

Step 4.1. Obtain the weights of multiple estimated values at the same position:

In formula (4), N ^xR is the number of multiple estimated values that appear at the same position, and σ _n is the variance of the original image noise;

Step 4.2, using formula (5) to obtain the final denoising result after image block aggregation,

In formula (5), is the estimated pixel value of an image block after non-local filtering, x _R is a reference block in the image, x _m is a similar block in a group of image blocks, is the square characteristic function at x _m , is the estimated value of the final image after denoising.

The beneficial effects of the present invention are:

1) The NCSR model-based three-dimensional collaborative filtering denoising method of the present invention performs regional processing on noisy images (i.e., is divided into edge regions and smooth regions), and further improves the clarity of image details (such as textures, edges, etc.).

2) The 3D collaborative filtering denoising method based on the NCSR model of the present invention uses the filtering formula in the NCSR model to filter the noise, fully utilizes the non-local similarity of the image, and significantly reduces the halo and ringing effects in the denoising results .

Description of drawings

Fig. 1 is the flowchart of the three-dimensional collaborative filtering denoising method based on NCSR model of the present invention;

Fig. 2 is the image that the three-dimensional collaborative filtering denoising method based on NCSR model of the present invention is used for experiment;

Figure 3 is a schematic diagram of a Cameraman (256×256) image polluted by different degrees of Gaussian noise;

Fig. 4 is the schematic diagram of pixel classification using Canny operator in the three-dimensional collaborative filtering denoising method based on NCSR model of the present invention;

Fig. 5 is a schematic diagram of the block matching process in the three-dimensional collaborative filtering denoising method based on the NCSR model of the present invention;

Fig. 6 is a similar grouping diagram of the three-dimensional collaborative filtering denoising method based on the NCSR model of the present invention;

Fig. 7 is the schematic diagram after denoising the Cameraman (256 * 256) image using the three-dimensional collaborative filtering denoising method based on the NCSR model of the present invention;

Fig. 8 is a comparison effect diagram of the denoising results of the three-dimensional collaborative filtering denoising method based on the NCSR model of the present invention and the denoising results of the BM3D algorithm under the same noise intensity;

Fig. 9 is a comparison diagram between the 3D collaborative filtering denoising method based on the NCSR model of the present invention and other excellent denoising algorithms in recent years.

Detailed ways

The present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments.

The three-dimensional collaborative filtering denoising method based on the NCSR model of the present invention, as shown in Figure 1, is specifically implemented according to the following steps:

Step 1. Use the Canny operator to extract the edge pixels of the noisy image, and store the coordinates of the edge pixels in the array W;

The classification of pixels is divided into two categories, one is the pixels in the smooth area, and the other is the pixels in the edge area;

Step 2, grouping the noisy images in step 1 by block matching to obtain a plurality of grouped image blocks;

The specific steps of step 2 are: first determine the position of the reference block, starting from the first 8×8 image block in the upper left corner of the image is the first reference block, and the position of the next reference block is shifted to the right or down by three pixels , the 39×39 area centered on each reference block is the neighborhood where the current reference block is located, and all image blocks with the same size as the reference block in the neighborhood are candidate blocks, traverse all candidate blocks in the neighborhood and find According to the Euclidean distance between it and the current reference block, 16 candidate blocks with the smallest Euclidean distance are selected as a group of similar blocks, and the group of similar blocks forms a group with the corresponding reference block.

The solution method of Euclidean distance is:

In formula (1), d ^noisy represents the Euclidean distance, ||·|| ₂ represents the l-2 paradigm, Z _{x R} represents the pixel value of the reference block, Z _x represents the pixel value of the candidate block in the neighborhood, Represents the number of pixels in an image block.

Step 3, performing collaborative filtering to denoise each grouped image block in step 2;

Step 3.1, all image blocks first need to perform local two-dimensional wavelet transformation, and then select step a or step b according to whether the position of the central pixel point of the reference block in each group is in W to perform three-dimensional collaborative filtering denoising, and estimate Sparse coefficients for each image block;

a. If the position of the center pixel point of the reference block in the current group is in W, then use the formula (2) in the NCSR model to perform third-dimensional collaborative filtering on the group of image blocks;

b. If the position of the center pixel point of the reference block in the current group is not in W, then use the formula (3) in the NCSR model to perform third-dimensional collaborative filtering on the group of image blocks;

In formula (2), (3), Represents the estimated value of the sparse coefficient of the image block, α represents the sparse coefficient of the image block, β represents the average coefficient value of a group of image blocks, l ₁ =c ₁ λ, l ₂ =c ₂ γ, both c ₁ and c ₂ is a constant;

Step 3.2. After collaborative filtering is performed on the sparse coefficient values of the image blocks through step 3.1, each grouped image block is subjected to two-dimensional wavelet inverse transformation to obtain its pixel value in the spatial domain.

Step 4, denoising the grouped image blocks after step 3 denoising into multiple image blocks, each grouped image block needs to return to the position in the original image, and image block estimates from different groups will appear in some positions of the original image, Therefore, these image blocks with the same coordinates need to be aggregated by weighted average to obtain the final image estimation value;

Step 4.1. Obtain the weights of multiple estimated values at the same position:

In formula (4), N ^xR is the number of multiple estimated values that appear at the same position, and σ _n is the variance of the original image noise;

Step 4.2, using formula (5) to obtain the final denoising result after image block aggregation,

In formula (5), is the estimated pixel value of an image block after non-local filtering, x _R is a reference block in the image, x _m is a similar block in a group of image blocks, is the square characteristic function at x _m , is the estimated value of the final image after denoising.

Next, the 3D collaborative filtering denoising method based on the NCSR model is used to process the grayscale image of Cameraman (256×256), as shown in Figure 2. After adding Gaussian noise, as shown in Figure 3, the values of σ _n from left to right are 10, 50, 70, and 100 respectively. The specific denoising process for the third noisy image (σ _n =70) in Figure 3 as follows:

Use the Canny operator to extract the edge pixels of the noisy image, and save the coordinates of the edge pixels to W; as shown in Figure 4, from left to right are Cameraman (256×256), House (256×256) , Lena(512×512), Peppers(256×256), where white pixels represent edge pixels and black pixels represent smooth pixels.

The boxes marked with letters "Ra" and "Rb" in FIG. 5 represent two different reference blocks, and the dotted boxes centered on these two reference blocks respectively represent the neighborhoods of these two reference blocks. The image blocks marked with "a'" and "b'" in these two neighborhoods respectively represent the similar blocks selected from the candidate blocks that are most similar to their corresponding reference blocks, and they will form similar groups with the corresponding reference blocks ;

In the process of block matching and grouping, the first 8×8 image block in the upper left corner of the image is the first reference block, and the position of the next reference block is shifted to the right or down by three pixels. The 39×39 area centered on each reference block is the neighborhood where the current reference block is located, and all image blocks with the same size as the reference block in the neighborhood are candidate blocks, as shown in Fig. 5 . The black frame is the size of the image, and the black dotted frame is the pixel that is symmetrically copied from the image centered on the edge or image vertex, which is the part that provides neighborhood supplement for the edge reference block;

Among them, the block matching group needs to traverse all candidate blocks in the neighborhood and calculate the Euclidean distance between them and the current reference block according to formula (1), and save them together with the coordinates of each candidate block in the array G _m,n . m represents the number of reference blocks, and n represents the number of similar blocks in the group. In the array G _m,n , sort according to the size of the Euclidean distance, and only keep the information of the first 16 similar blocks with the smallest distance. In G _m,n , the grouping formed by block matching is shown in Figure 6.

All image blocks in G _m,n are subjected to local wavelet transform. After transformation, the image block is sparsely represented as A={α _i,j ,i=1,2,h}, where i represents the number of similar blocks in the group, and j represents the number of sparse coefficients in the image block.

For each group of image blocks: use formula (6) to find the non-local mean value β _j at the same position of each group of similar blocks.

In formula (6), ω _i is the weight of each similar block in the group, which is inversely proportional to the Euclidean distance, and its calculation method is:

N is the side length of the image block, N=8.

Calculate the matching error E={α _i,j -β _j ,i=1,2,...,h} of the image block according to the above formula, and the standard deviation σ in set A and the standard deviation δ in set E .

in and are the average values of set A and set E respectively, and h is the number of elements in the set.

Use equations (10) and (11) to find the values of the regularization parameters λ and γ, respectively.

Estimate the sparse coefficient of each image block, the estimation method is as follows:

Determine whether the central coordinates of the reference block in the grouping are in the array W, if in the array W, use the formula (2) derived from the formula (12) to estimate;

If the center coordinates of the reference block are not in W, use formula (13) and its derivation formula (3) for calculation.

In formula (12), (13), is the sparse coefficient estimate, is the two-dimensional wavelet inverse transform, α is the sparse coefficient after wavelet transform, λ and γ are constants, and β is the weighted average coefficient at the same position in a group of similar blocks;

In formulas (2) and (3), l ₁ =c ₁ λ, l ₂ =c ₂ γ, both c ₁ and c ₂ are constants.

All image blocks are subjected to inverse wavelet transform to obtain pixel values in their spatial domain.

All image patches return the position in the original image where other estimates from different groupings are likely to occur, and all estimates for that position are aggregated. The specific calculation method of the final estimated value at this position is:

First get the weights for multiple estimates at the same location:

In formula (4), N ^xR is the number of multiple estimated values appearing at the same position, and σ _n is the noise variance of the original image.

Use formula (5) to obtain the final denoising result after image block aggregation:

In formula (5), is the estimated pixel value of an image block after non-local filtering, x _R is a reference block in the image, x _m is a similar block in a group of image blocks, is the square characteristic function at x _m , is the estimated value of the final image after denoising.

The denoising result of the Cameraman (256×256) image polluted by Gaussian noise (σ _n =70) is shown in FIG. 7 , which is the output result of the present invention after denoising. Figure 8 is a comparison of the denoising results of the present invention and the BM3D algorithm. It can be seen that the denoising results using this algorithm have no great visual difference between the smooth part and BM3D, but the edge details are obviously clearer than BM3D. Ringing and halo effects are visibly reduced. In Figure 8, the left column is the denoising result of BM3D, and the right column is the denoising result of the present invention. It can be seen from the small boxes in each icon that the edge part of the denoising result of the present invention is clearer. Fig. 9 is a comparison diagram of the denoising results of the present invention and the excellent denoising algorithms in recent years, the first image is a clean Cameraman (256×256) image, and the second image is a noisy image (σ _n =100) , the third picture is the denoising result of BM3D algorithm (PSNR=22.81), the fourth picture is the denoising result of LINC algorithm (PSNR=23.28), the fifth picture is the denoising result of PCLR algorithm (PSNR=23.48), the first picture is The six pictures are the denoising results of the present invention (PSNR=23.60), and it can be seen that the present invention still has strong competitiveness in processing edge details.

Table 1 shows the denoised PSNR values (ie peak signal-to-noise ratio) for different images polluted by different degrees of Gaussian noise by the present invention and other algorithms. In most cases, the peak signal-to-noise ratio of the present invention is higher than other algorithms. The average PSNR value of the present invention is 0.3 decibels higher than BM3D, 0.42 decibels higher than LINC, and 0.35 decibels higher than PCLR algorithm. The denoising effect of the present invention is basically the same as that of other algorithms in the smooth area, but it has obvious advantages in detail processing, and can achieve better denoising effect for images with more details.

Table 1 PSNR value comparison of different denoising algorithms

Through the above method, the 3D collaborative filtering denoising method based on the NCSR model of the present invention performs sub-regional processing on noisy images (that is, into edge regions and smooth regions), and further improves the clarity of image details (such as: textures, edges, etc.) degree. The three-dimensional cooperative filtering denoising method based on the NCSR model of the present invention uses the filtering formula in the NCSR model to filter the noise, fully utilizes the non-local similarity of the image, and obviously reduces the halo and ringing effects in the denoising result.

Claims (5)

1. The three-dimensional collaborative filtering denoising method based on the NCSR model is characterized in that, specifically implement according to the following steps: Step 1. Use the Canny operator to extract the edge pixels of the noisy image, and store the coordinates of the edge pixels in the array W; Step 2, grouping the noisy images in step 1 by block matching to obtain a plurality of grouped image blocks; Step 3, performing collaborative filtering to denoise each grouped image block in step 2; Step 4. Denoising the grouped image blocks in step 3 into multiple image blocks, and aggregate the image blocks with the same coordinates in the original image by means of weighted average to obtain the final estimated value of the image. 2. The three-dimensional collaborative filtering denoising method based on NCSR model as claimed in claim 1, is characterized in that, the specific steps described in step 2 are: first determine the position of the reference block, from the first 8 × 8 in the upper left corner of the image The image block starts with the first reference block, and the position of the next reference block is three pixels shifted to the right or down. The 39×39 area centered on each reference block is the neighborhood where the current reference block is located. All image blocks with the same size as the reference block in the neighborhood are candidate blocks, traverse all the candidate blocks in the neighborhood and find the Euclidean distance between them and the current reference block, and select 16 candidate blocks with the smallest Euclidean distance as a group Similar blocks, the group of similar blocks forms a group with the corresponding reference block. 3. the three-dimensional collaborative filtering denoising method based on NCSR model as claimed in claim 2, is characterized in that, the solution method of described Euclidean distance is: In formula (1), d ^noisy represents the Euclidean distance, ||·|| ₂ represents the l-2 paradigm, Z _{x R} represents the pixel value of the reference block, Z _x represents the pixel value of the candidate block in the neighborhood, Represents the number of pixels in an image block. 4. the three-dimensional collaborative filtering denoising method based on NCSR model as claimed in claim 1, is characterized in that, step 3 specific steps are: Step 3.1, all image blocks first need to perform local two-dimensional wavelet transformation, and then select step a or step b according to whether the position of the central pixel point of the reference block in each group is in W to perform three-dimensional collaborative filtering denoising, and estimate Sparse coefficients for each image block; a. If the position of the center pixel point of the reference block in the current group is in W, then use the formula (2) in the NCSR model to perform third-dimensional collaborative filtering on the group of image blocks; b. If the position of the center pixel point of the reference block in the current group is not in W, then use the formula (3) in the NCSR model to perform third-dimensional collaborative filtering on the group of image blocks; In formula (2), (3), Represents the estimated value of the sparse coefficient of the image block, α represents the sparse coefficient of the image block, β represents the average coefficient value of a group of image blocks, l ₁ =c ₁ λ, l ₂ =c ₂ γ, both c ₁ and c ₂ is a constant; Step 3.2. After collaborative filtering is performed on the sparse coefficient values of the image blocks through step 3.1, each grouped image block is subjected to two-dimensional wavelet inverse transformation to obtain its pixel value in the spatial domain. 5. the three-dimensional collaborative filtering denoising method based on NCSR model as claimed in claim 1, is characterized in that, the concrete method for finding the final estimated value of image described in step 4 is: Step 4.1. Obtain the weights of multiple estimated values at the same position: In formula (4), N ^xR is the number of multiple estimated values that appear at the same position, and σ _n is the variance of the original image noise; Step 4.2, using formula (5) to obtain the final denoising result after image block aggregation, In formula (5), is the estimated pixel value of an image block after non-local filtering, x _R is a reference block in the image, x _m is a similar block in a group of image blocks, is the square characteristic function at x _m , is the estimated value of the final image after denoising.