CN104318525A - Space guiding filtering based image detail enhancement method - Google Patents

Abstract

The invention discloses an image detail enhancement method based on space-guided filtering, which is characterized by the following steps: firstly, using an image edge detection operator to extract an edge feature response map from a source image, and performing normalization; then, for Create binarized spatial indication maps for different gray-scale intervals, and perform Gaussian convolution on each spatial indication map to obtain a spatial filter map, and calculate the weight of each spatial filter map; secondly, calculate the accumulation map, and Guided image filtering is performed on the accumulation map to obtain a spatially guided map; finally, the basic image and the residual image are solved, and an image detail enhancement model based on spatially guided filtering is established to enhance the image details of the source image. The invention can effectively improve the enhancement effect on image details.

Description

Image Detail Enhancement Method Based on Spatial Guidance Filtering

technical field

The invention relates to an image detail enhancement method, more specifically, an image detail enhancement method for improving the visual effect of an image and strengthening the interpretation and recognition of detailed images in the image.

Background technique

The 21st century is the information age, with the rapid development of the Internet, portable smart mobile devices such as mobile phones and iPads have spread all over people's daily lives. Almost all portable smart mobile devices have image acquisition functions. Users around the world use mobile phones or iPads to take a large number of pictures every day and share them on the Internet. However, due to the natural environmental factors in which the photos were taken or the limitations of the shooting equipment, the visual effects of many online pictures are not obvious. Therefore, in order to improve the visual effect of pictures, a large number of digital image processing methods have been proposed by researchers. Among a large number of digital image processing methods, image detail enhancement methods have attracted the attention of a large number of researchers in academia and industry in recent years.

The detailed features in images often contain important information, especially in medical images. However, due to the influence of factors such as noise and contrast, the visibility of the detailed features of these images is greatly reduced, which is not conducive to the effective use of image detailed features. As an image processing method, image detail enhancement can not only highlight the detailed feature information in the image, but also weaken or eliminate interference signals.

Currently, several image filtering methods based on edge features are used for image detail enhancement, such as local Laplacian filtering method and guided image filtering method. The main goal of this type of image filtering method is to enhance the details of the image without introducing "artifacts". However, this type of filtering method adopts a uniform filtering strength for pixels in all regions of the image during the image filtering process, without considering the relationship between the filtering strength and the image content of different regions, resulting in the following defects:

(1) When enhancing image details, "artifacts" need to be additionally introduced, which reduces the scope of application of the method;

(2) These filtering methods will not be able to use different filtering strengths for different image regions, reducing the filtering effect.

In the process of image detail enhancement, the user generally enhances certain areas in the image that contain important visual information, while leaving the other common areas unchanged. For example, for a natural landscape photo containing "blue sky", "tree" and "mountain", most users may wish to enhance the details of the "mountain" or "tree" area in the image, while the common The "blue sky" area remains unchanged. If the image regions with different semantics can be accurately identified, and then different filter strengths are used for different regions, the effect of image detail enhancement will be greatly improved. However, due to the "semantic gap" between low-level visual features and high-level semantic content, it becomes difficult to accurately identify image regions semantically. Therefore, the enhancement effects of the current image detail enhancement methods are limited and cannot meet the requirements of users.

Contents of the invention

The purpose of the present invention is to avoid the shortcomings of the above-mentioned prior art, and to provide an image detail enhancement method based on spatial guidance filtering, in order to effectively improve the effect of image detail enhancement.

The present invention adopts following method scheme for solving method problem:

A kind of feature of the image detail enhancement method based on spatial guided filtering of the present invention is to carry out as follows:

Step 1. For a source image I with a resolution of m×n, I∈R ^m×n , use an image edge detection operator to extract the edge response map of the source image I, and divide the edge response map by 255 to perform Normalize to get the normalized edge response matrix m is the length of the source image I; n is the width of the source image I;

Step 2, the normalized marginal response matrix The gray scale interval [0,1] of is evenly divided into k sub-intervals Ω _i , and the corresponding spatial indication map M(i) is established for each sub-interval in the k sub-intervals Ω _i using formula (1), respectively, M(i)∈R ^m×n , 1≤i≤k;

In formula (1): (x, y) represents the normalized marginal response matrix The coordinates of the elements in and correspond to the positions of the pixels in the source image I; 1≤x≤m, 1≤y≤n;

Denotes the normalized marginal response matrix The values of the elements in row x, column _y in the i-th subinterval Ω _i ; value;

Step 3, utilize Gaussian convolution to carry out filter processing to each spatial indicator map in turn, obtain k spatial filter maps I _Gauss (i), I _Gauss (i)∈R ^m×n ;

Step 4, counting the normalized marginal response matrix sequentially The number of elements in each subinterval in which all elements fall in, and use formula (2) to calculate the weight of each spatial filter map;

$\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\mathrm{<span\; class="notranslate">\; mn</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

In formula (2), h _i represents the normalized marginal response matrix The number of elements in the i-th subinterval Ω _i ; w(i) represents the weight of the i-th spatial filter graph;

Step 5, using formula (3) to calculate the cumulative map S _a , S _a ∈ R ^m×n ;

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; Gauss</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Step 6, using the source image I as a guide map, using a guide image filtering method to carry out guide image filtering processing on the accumulated map S _a to obtain a spatial guide map S, ^S∈Rm×n ;

Step 7, using the source image I as a guide image, and using a guide image filtering method to perform guided image filtering processing on the source image I itself to obtain a basic image I _b , I _b ∈ ^{R m×n} ;

Step 8. Using the image detail enhancement model shown in formula (4), perform image detail enhancement processing on the source image I to obtain a detail enhanced image I _o , I _o ∈ ^{R m×n} ;

I _o ＝I _b +S ₀ ·S⊙I _r (4)

In formula (4): S ₀ is the filter strength; S ₀ is a scalar; ⊙ is the Hadamard product symbol, which means that two matrices are multiplied; I _r is the residual image, and I _r =II _b .

Compared with the prior art, the beneficial effects of the present invention are reflected in:

1. The present invention constructs a spatial guidance map, which can approximately estimate different semantic regions in an image, and overcomes the problem of unrecognizable image regions caused by the "semantic gap". When the image details are enhanced, it plays a role of spatial guidance.

2. The present invention proposes an image detail enhancement method based on spatially guided filtering, which combines spatially guided images with guided image filtering to form a spatially guided image filtering method. In the process of space-guided image filtering, different image content areas are treated differently, and different filter strengths are used to overcome the defects of the original guided image filtering method due to the use of uniform filter strength. When image details are enhanced, it can be It makes the enhanced image appear more natural and the visual effect is more obvious.

3. The image detail enhancement method of the present invention only uses simple low-level visual features, so the computational complexity is low, the image processing speed is fast, and good user experience can be obtained.

Description of drawings

Fig. 1 is the source image that the present invention needs to carry out image detail enhancement;

Fig. 2 is the spatial guide map that the present invention establishes according to the source image;

FIG. 3 is an image obtained by using a uniform filter strength to enhance image details of a source image in the prior art;

FIG. 4 is an image after image detail enhancement is performed on a source image using the image detail enhancement method based on spatial guidance filtering in the present invention.

Detailed ways

In this embodiment, an image detail enhancement method based on spatial guidance filtering is mainly used for image detail enhancement of pictures with poor visual effect, so as to improve the visual salience of the image. This method can be made into a software APP and installed on a mobile terminal such as a mobile phone or a PC terminal. The feature of this method is to propose a spatially guided image, and combine it with the original guided image filtering method to form a spatially guided image filtering method, which is used to enhance the details of the image.

The concrete process is as follows when the inventive method carries out image detail enhancement:

Step 1, for the source image I with a resolution of m×n, I∈R ^m×n , use the image edge detection operator to extract the edge response map of the source image I, and divide the edge response map by 255 for normalization, Obtain the normalized marginal response matrix m is the length of the source image I; n is the width of the source image I;

For the convenience of introduction, the source image in step 1 is described using a grayscale image as an example. If the image to be enhanced in detail is a color image, the image matrix of the red, green, and blue color channels in the color image is respectively subjected to detail enhancement processing, and finally the image matrix after detail enhancement of the three color channels is merged into A complete color image, that is, an image after detail enhancement of a color image. Fig. 1 is a source image used in the test of the present invention, mainly an image of a mountain, but the overall visual effect is not very good, and the detailed information of the mountain is not rich enough.

The image edge detection operator in step 1 can be a Sobel edge detection operator or a Laplace edge detection operator, which are more classic edge detection operators, and there are related functions on the matlab software platform that can be called directly. The edge area of an image generally contains image detail information.

Step 2, the normalized marginal response matrix The gray scale interval [0,1] of is evenly divided into k sub-intervals Ω _i , using formula (1) to establish a corresponding space indicator map M(i) for each sub-interval in k sub-intervals Ω _i , M( i)∈Rm ^×n , 1≤i≤k;

In formula (1): (x, y) represents the normalized marginal response matrix The coordinates of the elements in , and correspond to the position of the pixel in the source image I; 1≤x≤m, 1≤y≤n;

Represents the normalized marginal response matrix The values of the elements in row x, column _y in the i-th subinterval Ω _i ; value;

The spatial indication map in step 2 is a binarized map, and step 2 is actually to normalize the edge response matrix in each gray level Do binarization. In fact, the regions with the same semantics in an image are generally in the same gray level, so the spatial indication map reflects the spatial information of the detailed content of the image.

In the test of the method of the present invention, k is 16.

Step 3, utilize Gaussian convolution to carry out filter processing to each spatial indicator map in turn, obtain k spatial filter maps I _Gauss (i), I _Gauss (i)∈R ^m×n ;

In fact, the Gaussian convolution method in step 3 is approximated by three consecutive Boxfilter operations, the purpose of which is to reduce the computational complexity while maintaining the accuracy.

Step 4. Statistically normalize the marginal response matrix in turn The number of elements in each subinterval in which all elements fall in, and use formula (2) to calculate the weight of each spatial filter map;

$\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\frac{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\mathrm{<span\; class="notranslate">\; mn</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

In formula (2), h _i represents the normalized marginal response matrix The number of elements in the i-th subinterval Ω _i where the elements fall into; w(i) represents the weight of the i-th spatial filter graph;

The important details of the image are generally located in the strong edge area of the image, but the proportion of strong edges in the entire edge response is small. Image weak edges often occupy a large proportion in the entire edge response, but weak edges are usually formed by noise. Considering this phenomenon, formula (2) assigns high weights to strong edges with small proportions, and low weights to weak edges with large proportions. Step 4 is actually the normalized marginal response matrix Perform histogram statistics, and the statistical results are transferred to the weights of the spatial filter graph in step 3.

Many previous image detail enhancement methods treat different gray levels equally, and use a uniform filter strength, which leads to a greatly reduced image detail enhancement effect. In step 4, the method of the present invention performs "differential treatment".

Step 5, using formula (3) to calculate the cumulative map S _a , S _a ∈ R ^m×n ;

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; Gauss</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Center\; Dot;</span>\mathrm{<span\; class="notranslate">\; w</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Equation (3) multiplies the spatial filter map obtained by the Gaussian convolution in step 3 by the corresponding weight and accumulates it to obtain the cumulative map.

Step 6, using the source image I as a guide map, using the guide image filtering method to carry out guide image filtering processing on the accumulation map S _a , to obtain a spatial guide map S, S∈R ^m×n ;

The guided image filtering method in step 6 was proposed by Dr. He Yuming of the Visual Computing Group of Microsoft Research Asia at the European Computer Vision Conference in 2010. When the source image is a guided image, guided image filtering is a filtering operation that preserves the edges of the image. In this step, the accumulated image in step 5 is filtered by guided image filtering to obtain the final spatial guided image. The spatial guidance map directly reflects the spatial information of different semantic content of the image, and plays a "guiding" role for enhancing the details of the subsequent image. Figure 2 is the spatial guidance map created based on the source image shown in Figure 1 .

Step 7. Using the source image I as a guide map, use the guide image filtering method to perform guide image filtering on the source image I itself to obtain the basic image I _b , I _b ∈ ^{R m×n} ;

The basic image I _b in this step reflects the content of the basic image in the low frequency band.

Step 8. Using the image detail enhancement model shown in formula (4), perform image detail enhancement processing on the source image I to obtain a detail enhanced image I _o , I _o ∈ ^{R m×n} ;

I _o ＝I _b +S ₀ ·S⊙I _r (4)

In formula (4): S ₀ is the filter strength, S ₀ is a scalar, ⊙ is the Hadamard product symbol, which means that two matrices are multiplied correspondingly; I _r means the residual image, and I _r =II _b . When the method of the present invention is tested, the filter strength _S0 is set to 3. The residual image I _r reflects the image details in the high frequency band.

Equation (4) reflects the image detail enhancement model based on spatial guided filtering proposed by the present invention. This model is actually an improvement of the image detail model shown in Equation (5):

I _o ＝I _b +S ₀ ·I _r (5)

It can be clearly seen that the image detail enhancement model of formula (5) adopts a unified filter strength, while the image detail enhancement model of formula (4) has one more spatial guide map S than formula (5), and the present invention The proposed spatial guidance map S just considers the relationship between image content and filtering strength, and adopts different filtering strengths for different image content regions.

Figure 3 is exactly the image after detail enhancement of the source image shown in Figure 1 according to the image detail enhancement model shown in formula (5). It can be seen that not only the image details of the "mountain" and "tree" areas are enhanced in Figure 3 , and the image in the "sky" area has also been enhanced. This part of the area should not be enhanced. After the enhancement, the image as a whole looks very abrupt. This is the disadvantage of using uniform filter strength.

Fig. 4 is an image after the source image shown in Fig. 1 is enhanced using the image detail enhancement model based on spatial guided filtering shown in formula (4). Compared with Figure 3, Figure 4 has enhanced the mountain and tree areas, but not the "sky" area, which is exactly the desired image enhancement result.

The above is only a preferred implementation mode of the present invention. Anyone familiar with the technical field within the technical scope disclosed in the present invention, according to the technical solution of the present invention and its inventive concept to make equivalent replacements or related parameter changes, all Should be covered within the protection scope of the present invention.

Claims (1)

1. guide an image detail enhancement method for filtering based on space, it is characterized in that carrying out as follows:

Step 1 is the source images I of m × n, I ∈ R for resolution ^{m × n}, utilize the skirt response figure of source images I described in Image Edge-Detection operator extraction, and described skirt response figure be normalized divided by 255, obtain normalization skirt response matrix | ▽ I|, | ▽ I| ∈ R ^{m × n}; M is the length of source images I; N is the width of source images I;

Step 2, by described normalization skirt response matrix | the tonal range interval [0,1] of ▽ I| is evenly divided into k sub-range Ω _i, utilize formula (1) respectively to described k sub-range Ω _iin each sub-range set up corresponding space index map M (i), M (i) ∈ R ^{m × n}, 1≤i≤k;

In formula (1): (x, y) represents described normalization skirt response matrix | the coordinate of element in ▽ I|, and the position corresponding to pixel in described source images I; 1≤x≤m, 1≤y≤n;

| ▽ I (x, y) | ∈ Ω _irepresenting described normalization skirt response matrix | in ▽ I|, the value of xth row y column element is positioned at i-th sub-range Ω _iin; M _{(x, y)}i () represents the value of xth row y column element in i-th space index map M (i);

Step 3, utilize Gaussian convolution to carry out filtering process to each space index map successively, obtain k spatial filtering figure I _gauss(i), I _gauss(i) ∈ R ^{m × n};

Step 4, adding up described normalization skirt response matrix successively | in ▽ I|, all elements drops on the number of element in each sub-range, and utilizes formula (2) to calculate the weight of each spatial filtering figure;

$w\left(i\right)=\frac{1}{k-1}(1-\frac{h_i}{\mathrm{mn}})---\left(2\right)$

In formula (2), h _irepresenting described normalization skirt response matrix | the element in ▽ I| drops on i-th sub-range Ω _ithe number of middle element; W (i) represents the weight dropping on i-th spatial filtering figure;

Step 5, formula (3) is utilized to calculate cumulative figure S _a, S _a∈ R ^{m × n};

$S_a=\underset{i=1}{\overset{k}{\mathrm{\&Sigma;}}}{I}_{\mathrm{Gauss}}\left(i\right)\&CenterDot;w\left(i\right)---\left(3\right)$

Step 6, using described source images I as guiding figure, adopt navigational figure filtering method to described cumulative figure S _aguide image filtering process, obtain space guiding figure S, S ∈ R ^{m × n};

Step 7, using described source images I as guiding figure, adopt navigational figure filtering method self to guide image filtering process to described source images I, obtain base image I _b, I _b∈ R ^{m × n};

Step 8, utilize image detail shown in formula (4) to strengthen model, image detail is carried out to described source images I and strengthens process, obtain details and strengthen image I _o, I _o∈ R ^{m × n};

I _o＝I _b+S ₀·S⊙I _r (4)

In formula (4): S ₀for filtering strength; S ₀for scalar; ⊙ is Hadamard product signs, represents that two matrix correspondences are multiplied; I _rrepresent residual image, and have I _r=I-I _b.