CN105913427A - Machine learning-based noise image saliency detecting method - Google Patents

Abstract

The present invention relates to a noise image saliency detection method based on machine learning, comprising the following steps: 1. Using multiple denoising parameters for noise images of each amplitude respectively to obtain the best denoising parameters for each amplitude; 2. Use the noise evaluation algorithm for each noise image to extract features, obtain the noise value features, and form the noise value feature set; 3. Use the noise value feature set as the feature set of the machine learning algorithm, and pass the machine learning algorithm and quintile cross-validation 4. Use the noise amplitude prediction model to predict the corresponding noise image to obtain the predicted noise amplitude value; 5. Use the predicted noise amplitude value of each noise image and the corresponding best denoising parameters to perform Denoising processing to obtain a denoising image set; 6. Using a saliency detection method on the images in the denoising image set to obtain a final saliency map. This method can improve the detection performance of saliency detection methods in noisy images.

Description

A Noise Image Saliency Detection Method Based on Machine Learning

technical field

The invention relates to the technical fields of image and video processing and computer vision, in particular to a noise image saliency detection method based on machine learning.

Background technique

Human senses mainly include sight, smell, taste, hearing and touch. Humans rely on their senses to receive information from the outside world. Vision plays an important role in human senses. The human visual system can focus on the most important part of the image in a short time, that is, the part that the human eye is most interested in. With the advent of the multimedia era, the popularization of various digital products and the dissemination of digital images in the Internet age, a large number of image resources are generated and transmitted every day. Although massive image data enriches life, it also brings many challenges.

How to efficiently and accurately process these image resources is a key issue. After discovering the selective attention mechanism of the human visual system, researchers tried to simulate the human visual system by computer, thus proposing a saliency detection method. Saliency detection has been applied to image compression and coding, image retrieval, image segmentation, object recognition and content-aware image scaling, etc. For example, in image compression and coding, the salient area is detected first, and then more details are reserved for the salient area, which not only compresses the image, but also retains more important details.

Visual saliency detection has been relatively well studied, however most saliency detection models are proposed for undistorted images, and the experimental data is a collection of undistorted images. Few papers have noticed the impact of distorted images on saliency detection. Zhang et al. found that noise, blur and compression changed the low-level features of images, and proposed a bottom-up saliency detection model based on low-level features of images. At the same time, Zhang et al. found that image quality distortion will cause changes in the saliency map, and there is a certain relationship between the change of the saliency map and the subjective image quality assessment. Gide and Karam evaluated five saliency detection models on the eye-tracking dataset for image quality assessment. The types of artifacts evaluated included blur, noise, and JPEG compression artifacts. Mittal et al. extracted low-level features such as image brightness and contrast, and based on these features, used a machine learning framework to predict the salient regions of JPEG distorted images. Kim and Milanfar proposed a saliency detection model based on a nonparametric regression framework for noisy images.

Most images in real life are distorted, such as distortion caused by peripherals such as camera sensors and image processors, jitter distortion caused by camera shakes, and image compression distortion. In order to improve the application of the saliency detection method on the noise image, the present invention proposes a noise image saliency detection method based on machine learning.

Contents of the invention

The object of the present invention is to provide a noise image saliency detection method based on machine learning, which can improve the detection performance of the saliency detection method in noise images.

In order to achieve the above object, the technical solution of the present invention is: a noise image saliency detection method based on machine learning, comprising the following steps:

Step S1: denoising the noise image of each amplitude using multiple denoising parameters to obtain the best denoising parameters corresponding to each amplitude;

Step S2: use the noise evaluation algorithm for feature extraction for each noise image, and obtain the noise value feature of each noise image, so as to form the noise value feature set P ;

Step S3: use the noise value feature set P as the feature set of the machine learning algorithm, and obtain the noise amplitude prediction model of the noise image through the machine learning algorithm and the quintile cross-validation method;

Step S4: Using the noise amplitude prediction model to predict the corresponding noise image, and obtain the predicted noise amplitude value of each noise image;

Step S5: Perform denoising processing using the predicted noise amplitude value of each noise image and the best denoising parameters corresponding to the amplitude, to obtain a denoising image set;

Step S6: Use a saliency detection method to detect the images in the denoised image set to obtain a final saliency map.

Further, in the step S1, multiple denoising parameters are used to perform denoising processing on the noise image of each amplitude, and the best denoising parameters corresponding to each amplitude are obtained, which specifically includes the following steps:

Step S11: Denoise the noise image of each amplitude by using n kinds of denoising parameters of Gaussian low-pass filter, and obtain a set S of denoised images containing n kinds of denoising parameters for each amplitude;

Step S12: use the saliency detection method VA to calculate the saliency map on the denoised image set S, and obtain the saliency map set T of the denoised image;

Step S13: Use the evaluation index PR-AUC to evaluate the saliency map set T of the denoised image, find the denoising parameters used when the average PR-AUC is the highest value for each amplitude, and obtain the best denoising parameters for each amplitude. Noise parameter.

Further, in the step S2, feature extraction is performed on each noise image using a noise evaluation algorithm to obtain the noise value feature of each noise image, so as to form a noise value feature set P , which specifically includes the following steps:

Step S21: grayscale processing is carried out to noise image, obtains grayscale image I ;

Step S22: Use bilateral filtering to process the grayscale image I to obtain a bilateral filtering result map ;

Step S23: Calculating the grayscale image I and the bilateral filtering result map The difference, get the difference image D ;

Step S24: Use the Canny edge detection method on the grayscale image I to obtain the edge image E , use the dilation operator on the edge image E to expand the edge area, and obtain the enlarged edge image ;

Step S25: Calculate the noise size evaluation value image M , the calculation formula is:

,in

Among them, D _v represents the value of pixel v in the grayscale image, t represents the pixel point, Indicates the enlarged edge image, N indicates the enlarged edge image The set of pixel point t whose value is 0;

Step S26: Divide the noise size evaluation value image M evenly into 3×3 grid areas, respectively calculate the noise size evaluation value image M full image and the noise size evaluation value of each grid area, the calculation formula is:

Among them, M _r represents the corresponding area, r =1, 2, ..., 10 represent the whole image and 9 grid areas respectively, M _{r , v} represent the value of pixel v in the area r ; the noise value feature set is obtained by calculation P = { P ₁ , P ₂ , ..., P ₁₀ }.

Further, in the step S3, the noise value feature set P is used as the feature set of the machine learning algorithm, and the noise amplitude prediction model of the noise image is obtained through the machine learning algorithm and the quintile cross-validation method, which specifically includes the following steps:

Step S31: Arrange the feature values P ₁ , P ₂ , ..., P ₁₀ in the noise value feature set P in ascending order, and use it as the feature set F of the machine learning algorithm, and randomly divide the feature set F into five equal parts: F1, F2, F3, F4, and F5;

Step S32: using F2, F3, F4 and F5 as training data sets for machine learning, using their corresponding image distortion magnitudes in the image quality assessment database as training labels for machine learning, and learning to obtain a noise magnitude prediction model M1;

Step S33: Repeat step S32 to obtain the noise magnitude prediction model M2 when F1, F3, F4 and F5 are used as training data sets, and the noise magnitude prediction models M3, F1, F5 when F1, F2, F4 and F5 are used as training data sets, respectively. F2, F3, and F5 are used as the noise amplitude prediction model M4 when the training data set is used, and the noise amplitude prediction model M5 is used when F1, F2, F3, and F4 are used as the training data set.

Further, in the step S4, the noise amplitude prediction model is used to predict the corresponding noise image, and the predicted noise amplitude value of each noise image is obtained, which specifically includes the following steps:

Step S41: Use the noise amplitude prediction model M1 to predict the image set corresponding to the feature set F1, and obtain the noise amplitude value prediction set V1;

Step S42: Repeat the method of step S41, respectively use the noise amplitude prediction models M2, M3, M4, M5 to predict the image sets corresponding to the feature sets F2, F3, F4, and F5, and obtain noise amplitude value prediction sets V2, V3, V4, V5;

Step S43: Synthesize the noise magnitude prediction set V={V1, V2, V3, V4, V5} to obtain the noise magnitude prediction set V of the complete image set.

Further, in the step S5, the predicted noise amplitude value of each noise image and the optimal denoising parameter corresponding to the amplitude are used for denoising processing to obtain a denoising image set, which specifically includes the following steps:

Step S51: For each noise image, find the noise amplitude value corresponding to the noise image from the noise amplitude value prediction set V;

Step S52: According to the noise amplitude value, the noise image is processed by Gaussian low-pass filter using the corresponding optimal denoising parameters to obtain the denoising image set FI.

Compared with the prior art, the beneficial effects of the present invention are as follows: firstly, machine learning is used to predict the noise amplitude of the noise image, then the optimal denoising parameters suitable for the amplitude are used for denoising processing, and finally the denoising is calculated by using the saliency detection method The saliency map of the post image, because the present invention takes into account the influence of the noise image on the saliency detection method, it can effectively improve the detection performance of the saliency detection method on the noise image, and can be applied to image and video processing, computer vision, etc. Many fields.

Description of drawings

Fig. 1 is a block flow diagram of the method of the present invention.

Fig. 2 is an example picture in step S2 of an embodiment of the present invention (for better display effect, the pixel values of (d), (g) and (h) in Fig. 2 are mapped to [0,1]) .

Fig. 3 is an implementation flow chart of an overall method according to an embodiment of the present invention.

Fig. 4 is an example picture of the original noise image and the final effect after steps S5 and S6 in an embodiment of the present invention.

detailed description

The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

The present invention provides a noise image saliency detection method based on machine learning, as shown in Figure 1 and Figure 3, comprising the following steps:

Step S1: Perform denoising processing on the noise image of each amplitude using various denoising parameters, and obtain the best denoising parameters corresponding to each amplitude. In this embodiment, step S1 specifically includes the following steps:

Step S11: Use 9 Gaussian low-pass filter denoising parameters (template sizes are {3×3, 5×5, 7×7}, standard deviations are {0.5, 0.7, 0.9}) for each amplitude noise The image is denoised to obtain a denoised image set S containing 9 denoising parameters for each amplitude;

Step S12: Use the saliency detection method VA (Saliency detection via absorbing markov chain) to calculate the saliency map on the denoised image set S, and obtain the saliency map set T of the denoised image;

Step S13: Use the evaluation index PR-AUC (the area under precision-recall curve) to evaluate the saliency map set T of the denoised image, and find the denoising parameters used when finding the highest average PR-AUC value for each amplitude , to get the optimal denoising parameters for each magnitude.

Step S2: Use the noise evaluation algorithm to perform feature extraction on each noise image, and obtain the noise value characteristics of each noise image, so as to form the noise value feature set P. In this embodiment, as shown in FIG. 2, step S2 specifically includes the following steps:

Step S21: Perform grayscale processing on the noise image to obtain a grayscale image I (as shown in Figure 2(b));

Step S22: Use bilateral filtering to process the grayscale image I to obtain a bilateral filtering result map (as shown in Figure 2(c));

Step S23: Calculating the grayscale image I and the bilateral filtering result map The difference, get the difference image D (as shown in Figure 2(d));

Step S24: Use the Canny edge detection method on the grayscale image I to obtain the edge image E (as shown in Figure 2(e)), and use the expansion operator to expand the edge area on the edge image E to obtain the enlarged edge image (as shown in Figure 2(f));

Step S25: Calculate the noise size evaluation value image M , the calculation formula is:

,in

Among them, D _v represents the value of pixel v in the grayscale image, t represents the pixel point, Indicates the enlarged edge image, N indicates the enlarged edge image The set of pixel point t whose value is 0;

Step S26: Divide the noise size evaluation value image M (as shown in Figure 2(g)) evenly into 3×3 grid areas (as shown in Figure 2(h)), and calculate the noise size evaluation value image M full image and each The evaluation value of the noise size in the grid area, the calculation formula is:

Among them, M _r represents the corresponding area, r =1, 2, ..., 10 represent the whole image and 9 grid areas respectively, M _{r , v} represent the value of pixel v in the area r ; the noise value feature set is obtained by calculation P = { P ₁ , P ₂ , ..., P ₁₀ }.

Step S3: The noise value feature set P is used as the feature set of the machine learning algorithm, and the noise amplitude prediction model of the noise image is obtained through the machine learning algorithm and the quintile cross-validation method. In this embodiment, step S3 specifically includes the following steps:

Step S31: Arrange the feature values P ₁ , P ₂ , ..., P ₁₀ in the noise value feature set P in ascending order, and use it as the feature set F of the machine learning algorithm, and randomly divide the feature set F into five equal parts: F1, F2, F3, F4, and F5;

Step S32: using F2, F3, F4 and F5 as training data sets for machine learning, using their corresponding image distortion magnitudes in the image quality assessment database TID2013 as training labels for machine learning, and learning to obtain noise magnitude prediction model M1;

Step S33: Repeat step S32 to obtain the noise magnitude prediction model M2 when F1, F3, F4 and F5 are used as training data sets, and the noise magnitude prediction models M3, F1, F5 when F1, F2, F4 and F5 are used as training data sets, respectively. F2, F3, and F5 are used as the noise amplitude prediction model M4 when the training data set is used, and the noise amplitude prediction model M5 is used when F1, F2, F3, and F4 are used as the training data set.

Step S4: Using the noise amplitude prediction model to predict the corresponding noise image, and obtain the predicted noise amplitude value of each noise image. In this embodiment, step S4 specifically includes the following steps:

Step S41: Use the noise amplitude prediction model M1 to predict the image set corresponding to the feature set F1, and obtain the noise amplitude value prediction set V1;

Step S42: Repeat the method of step S41, respectively use the noise amplitude prediction models M2, M3, M4, M5 to predict the image sets corresponding to the feature sets F2, F3, F4, and F5, and obtain noise amplitude value prediction sets V2, V3, V4, V5;

Step S43: Synthesize the noise magnitude prediction set V={V1, V2, V3, V4, V5} to obtain the noise magnitude prediction set V of the complete image set.

Step S4: Using the noise amplitude prediction model to predict the corresponding noise image, and obtain the predicted noise amplitude value of each noise image. Specifically include the following steps:

Step S41: Use the noise amplitude prediction model M1 to predict the image set corresponding to the feature set F1, and obtain the noise amplitude value prediction set V1;

Step S42: Repeat the method of step S41, respectively use the noise amplitude prediction models M2, M3, M4, M5 to predict the image sets corresponding to the feature sets F2, F3, F4, and F5, and obtain noise amplitude value prediction sets V2, V3, V4, V5;

Step S43: Synthesize the noise magnitude prediction set V={V1, V2, V3, V4, V5} to obtain the noise magnitude prediction set V of the complete image set.

Step S5: Perform denoising processing using the predicted noise amplitude value of each noise image and the best denoising parameters corresponding to the amplitude, to obtain a denoising image set. In this embodiment, as shown in FIG. 4, step S5 specifically includes the following steps:

Step S51: For each noise image, find the noise amplitude value corresponding to the noise image from the noise amplitude value prediction set V;

Step S52: According to the noise amplitude value, the noise image is processed by Gaussian low-pass filter using the corresponding optimal denoising parameters to obtain the denoising image set FI.

Step S6: Use the saliency detection method VA to detect the images in the denoised image set FI to obtain the final saliency map.

The noise image saliency detection method based on machine learning provided by the present invention takes into account the influence of the noise image on the saliency detection method, excavates the correlation between the noise size evaluation value feature and the noise amplitude in the image quality evaluation database TID2013, and designs the machine learning The noise prediction model is combined with the denoising method and the parameters set for the amplitude to denoise the image, and finally the saliency detection method VA is used to calculate the saliency map of the denoised image. This method can effectively improve the detection performance of the saliency detection method on noisy images, and can be applied to image and video processing, computer vision and other fields.

The above are the preferred embodiments of the present invention, and all changes made according to the technical solution of the present invention, when the functional effect produced does not exceed the scope of the technical solution of the present invention, all belong to the protection scope of the present invention.

Claims (6)

1. a noise image saliency detection method based on machine learning, is characterized in that, comprises the following steps: Step S1: denoising the noise image of each amplitude using multiple denoising parameters to obtain the best denoising parameters corresponding to each amplitude; Step S2: use the noise evaluation algorithm for feature extraction for each noise image, and obtain the noise value feature of each noise image, so as to form the noise value feature set P ; Step S3: use the noise value feature set P as the feature set of the machine learning algorithm, and obtain the noise amplitude prediction model of the noise image through the machine learning algorithm and the quintile cross-validation method; Step S4: Using the noise amplitude prediction model to predict the corresponding noise image, and obtain the predicted noise amplitude value of each noise image; Step S5: Perform denoising processing using the predicted noise amplitude value of each noise image and the best denoising parameters corresponding to the amplitude, to obtain a denoising image set; Step S6: Use a saliency detection method to detect the images in the denoised image set to obtain a final saliency map. 2. A noise image saliency detection method based on machine learning according to claim 1, characterized in that: in the step S1, the noise image of each amplitude is denoised using multiple denoising parameters , to obtain the optimal denoising parameters corresponding to each amplitude, which specifically includes the following steps: Step S11: Denoise the noise image of each amplitude by using n kinds of denoising parameters of Gaussian low-pass filter, and obtain a set S of denoised images containing n kinds of denoising parameters for each amplitude; Step S12: use the saliency detection method VA to calculate the saliency map on the denoised image set S, and obtain the saliency map set T of the denoised image; Step S13: Use the evaluation index PR-AUC to evaluate the saliency map set T of the denoised image, find the denoising parameters used when the average PR-AUC is the highest value for each amplitude, and obtain the best denoising parameters for each amplitude. Noise parameter. 3. A noise image saliency detection method based on machine learning according to claim 1, characterized in that: in the step S2, each noise image is extracted using a noise evaluation algorithm to obtain each noise image The noise value feature, in order to form the noise value feature set P , specifically includes the following steps: Step S21: grayscale processing is carried out to noise image, obtains grayscale image I ; Step S22: Use bilateral filtering to process the grayscale image I to obtain a bilateral filtering result map ; Step S23: Calculating the grayscale image I and the bilateral filtering result map The difference, get the difference image D ; Step S24: Use the Canny edge detection method on the grayscale image I to obtain the edge image E , use the dilation operator on the edge image E to expand the edge area, and obtain the enlarged edge image ; Step S25: Calculate the noise size evaluation value image M , the calculation formula is: ,in Among them, D _v represents the value of pixel v in the grayscale image, t represents the pixel point, Indicates the enlarged edge image, N indicates the enlarged edge image The set of pixel point t whose value is 0; Step S26: Divide the noise size evaluation value image M evenly into 3×3 grid areas, respectively calculate the noise size evaluation value image M full image and the noise size evaluation value of each grid area, the calculation formula is: Among them, M _r represents the corresponding area, r =1, 2, ..., 10 represent the whole image and 9 grid areas respectively, M _{r , v} represent the value of pixel v in the area r ; the noise value feature set is obtained by calculation P = { P ₁ , P ₂ , ..., P ₁₀ }. 4. A kind of noise image saliency detection method based on machine learning according to claim 3, is characterized in that: in described step S3, noise value feature set P is used as the feature set of machine learning algorithm, and through machine learning Algorithm and quintile cross-validation method to obtain the noise magnitude prediction model of the noise image, which specifically includes the following steps: Step S31: Arrange the feature values P ₁ , P ₂ , ..., P ₁₀ in the noise value feature set P in ascending order, and use it as the feature set F of the machine learning algorithm, and randomly divide the feature set F into five equal parts: F1, F2, F3, F4, and F5; Step S32: using F2, F3, F4 and F5 as training data sets for machine learning, using their corresponding image distortion magnitudes in the image quality assessment database as training labels for machine learning, and learning to obtain a noise magnitude prediction model M1; Step S33: Repeat step S32 to obtain the noise magnitude prediction model M2 when F1, F3, F4 and F5 are used as training data sets, and the noise magnitude prediction models M3, F1, F5 when F1, F2, F4 and F5 are used as training data sets, respectively. F2, F3, and F5 are used as the noise amplitude prediction model M4 when the training data set is used, and the noise amplitude prediction model M5 is used when F1, F2, F3, and F4 are used as the training data set. 5. A noise image saliency detection method based on machine learning according to claim 4, characterized in that: in the step S4, the noise amplitude prediction model is used to predict the corresponding noise image, and each noise image is obtained The predicted noise amplitude value of , specifically includes the following steps: Step S41: Use the noise amplitude prediction model M1 to predict the image set corresponding to the feature set F1, and obtain the noise amplitude value prediction set V1; Step S42: Repeat the method of step S41, respectively use the noise amplitude prediction models M2, M3, M4, M5 to predict the image sets corresponding to the feature sets F2, F3, F4, and F5, and obtain noise amplitude value prediction sets V2, V3, V4, V5; Step S43: Synthesize the noise magnitude prediction set V={V1, V2, V3, V4, V5} to obtain the noise magnitude prediction set V of the complete image set. 6. A noise image saliency detection method based on machine learning according to claim 5, characterized in that, in the step S5, the predicted noise amplitude value of each noise image and the corresponding optimal noise value of the amplitude are used. Noise parameters are used for denoising processing to obtain a denoising image set, which specifically includes the following steps: Step S51: For each noise image, find the noise amplitude value corresponding to the noise image from the noise amplitude value prediction set V; Step S52: According to the noise amplitude value, the noise image is processed by Gaussian low-pass filter using the corresponding optimal denoising parameters to obtain the denoising image set FI.