CN104966085A - Remote sensing image region-of-interest detection method based on multi-significant-feature fusion - Google Patents

Abstract

The invention discloses a remote sensing image interest region detection method based on multi-significant feature fusion, which belongs to the technical field of remote sensing image processing and image recognition. The implementation process includes: 1) Obtain a set of color channels of the input remote sensing image and calculate the color histogram of each channel; 2) Calculate the normalized saliency weight of each color channel from these histograms; 3) Calculate the information saliency feature map; 4) Combine a Group input remote sensing images are converted from RGB color space to CIE Lab color space; 5) Clusters are obtained by clustering algorithm; 6) Salient values of each cluster are calculated to obtain a common salient feature map; The final saliency map is obtained from the figure; 8) The region of interest is extracted by threshold segmentation using the maximum between-class variance method. Compared with the traditional method, the present invention realizes the accurate detection of the region of interest in the remote sensing image without prior knowledge base, and can be widely used in the fields of environmental monitoring, land utilization and agricultural investigation and the like.

Description

A Method for Detection of Regions of Interest in Remote Sensing Images Based on Fusion of Multiple Salient Features

technical field

The invention belongs to the technical field of remote sensing image processing and image recognition, and in particular relates to a method for detecting a region of interest in a remote sensing image based on fusion of multiple salient features.

Background technique

With the rapid development of remote sensing technology, the data scale of remote sensing images is rapidly expanding, and the extraction of regions of interest in remote sensing images can reduce the complexity of remote sensing image analysis and processing, so the extraction of regions of interest in remote sensing images is also a hot spot of attention recently. How to accurately and quickly realize the detection of regions of interest in remote sensing images has become one of the problems to be solved urgently. An effective solution to this problem will be of great significance for alleviating the contradiction between high-speed acquisition and low-speed interpretation of remote sensing images, and will also have important practical application value for related fields such as land use, disaster assessment, town planning, and environmental monitoring.

The ROI detection of traditional remote sensing images is mostly based on the global situation and requires prior knowledge. However, the establishment of the prior knowledge base itself is a very complicated problem, which needs to comprehensively consider the information of the expert knowledge base, the characteristics of the target area, and the characteristics of the background area. Some methods need to introduce training on psychophysical data of color presentation and eye movement, and some methods rely on digital maps of the same area for the detection and classification of regions of interest in remote sensing images. These algorithms all require a priori knowledge base and have high computational complexity.

The visual attention model provides a new perspective for the detection of regions of interest in remote sensing images. Unlike traditional detection methods, the visual attention model is completely driven by data, does not involve the influence of external factors such as knowledge bases, and has the advantages of rapid identification and accurate results. , the visual attention model has received more and more attention, and it is of great significance to introduce the visual attention model into the detection of regions of interest in remote sensing images.

In terms of the visual attention model based on low-level visual features, Itti et al. proposed the Itti visual attention method in the article "A Model of Saliency-Based Visual Attention for Rapid Scene Analysis". This model is close to the human visual system and uses various visual characteristics. Generate a saliency map. In terms of visual attention models based on mathematical methods, Harel et al. proposed a graph-based algorithm (Graph-based visual saliency, GBVS) in the article "Graph-Based Visual Saliency", which uses the traditional Itti model to simulate visual The attention mechanism is used to complete the feature extraction step, and then the graph structure is used to represent the pixel association between images, and finally Markow chains are introduced to calculate the saliency map. In terms of attention models based on frequency domain analysis, Achanta et al. proposed a frequency tuning method (Frequency-tuned, FT) for salient region detection in the article "Frequency-tuned Salient RegionDetection", which converts the input RGB image to CIELab color space After performing Gaussian smoothing, subtracting the arithmetic mean of the image feature vector, and calculating the magnitude by point, a uniform salient map with clear boundaries can be obtained.

The visual attention model based on low-level visual features can better simulate the attention of human eyes, but it does not fully consider the frequency domain characteristics of images, and at the same time, the calculation speed is slow and the efficiency is low, which is difficult to meet the requirements of real-time applications. The visual attention model based on the frequency domain analysis method is simple in form, easy to explain and implement, but when the salient region accounts for too much of the whole image, or the image background is too complex, the saliency map obtained by this method will mistakenly mark part of the background as significant region, and its biological plausibility is not very clear. In recent years, scholars at home and abroad have also proposed new algorithms that apply visual saliency to the detection of regions of interest in remote sensing images. For example, Zhang et al. proposed in the article "Fast Detection of Visual Saliency Regions in Remote Sensing Image based on Region Growing" to reduce image resolution based on wavelet transform and introduce two-dimensional discrete moment transform into visual features to generate saliency maps. However, these algorithms have a common shortcoming, they can only extract salient regions, but cannot distinguish the differences between these salient regions. For a group of remote sensing images with similar ROIs, if their similarity can be used, other areas that interfere with the ROI detection can be excluded.

In terms of calculating the mask of the region of interest, the traditional method often uses a circle with a fixed radius to describe the region of interest, which will bring a lot of redundant information when identifying random regions, and the speed of using a single threshold is very fast, but the region of interest will be There are many small pieces and the area description is not accurate. The method of maximum inter-class variance (Ostu method) is an automatic non-parametric and unsupervised threshold selection method. This method is a simple and efficient method for adaptively calculating a single threshold. This method has the advantages of simple calculation and strong self-adaptation.

Contents of the invention

The object of the present invention is to provide a method for detecting a region of interest in a remote sensing image based on the fusion of multiple salient features, which is used to accurately detect the region of interest in the remote sensing image. Existing ROI detection methods are mainly global-based and require prior knowledge. However, the establishment of the prior knowledge base itself is a very complicated problem, which needs to comprehensively consider the information of the expert knowledge base, the characteristics of the target area, and the characteristics of the background area. Therefore, the method of the present invention mainly focuses on two aspects:

1) There is no need to search and establish a priori knowledge base based on the global;

2) Improve the detection accuracy of the region of interest in the remote sensing image and obtain more accurate information of the region of interest.

The technical solution used in the present invention includes five main processes of generating information-aware feature maps of remote sensing images, generating a total of salient feature maps, generating final saliency maps, generating region-of-interest templates, and generating region-of-interest, specifically including the following steps:

Step 1: Calculate the color histogram, that is, input a set of remote sensing images with a size of M×N, extract each color channel of each image separately, and use f _c (x, y) to represent the color channel c (x, y ) position, construct the intensity histogram H _c (i) of each remote sensing image in different color channels, where M represents the length of the image, N represents the width of the image, and x and y represent the horizontal and vertical coordinates of the image respectively, x=1, 2...M, y=1, 2...N, c represents the color channel, c=1, 2, 3, i represents the pixel intensity value, i=0, 1...255;

Step 2: Calculate the normalized saliency weight of color channel c, that is, calculate the information amount In _c (i) of each pixel intensity value i in the color channel according to the color histogram H _c (i) of color channel c, and calculate the The amount of information is assigned to the pixel point equal to the intensity value of the pixel. After all calculations and assignments are completed, the information amount map LOG _c (x, y) of the color channel c is obtained. Using this information amount map, the salience degree of the color channel c is obtained. h _c , and then use the saliency of each color channel to calculate the normalized saliency weight w _c of each color channel of each image;

Step 3: Calculate the informative feature map, that is, use the standardized saliency weight w _c of each color channel to weight and calculate the preliminary informative feature map of each image, and perform Gaussian smoothing filtering on the initially obtained informative feature map, After filtering the noise, the final informative feature map of each image is obtained;

Step 4: Convert a set of remote sensing images from the RGB color space to the CIE Lab color space, that is, extract the R, G, and B color channel values of each pixel of each image, and convert them to the CIE Lab color space to obtain L, a, b three components, in the RGB color space, R means red red, G means green green, B means blue blue, in CIELab color space, L means brightness, L=0 means black, L=100 means white , a represents the position of the color between red/green, a negative value represents green, a positive value represents red, b represents the position of the color between blue/yellow, b represents blue when negative, and b represents positive Value represents yellow;

Step 5: Use the k-means clustering algorithm to complete the pixel clustering in the CIE Lab color space, that is, use the k-means clustering algorithm to map this group of original remote sensing images to the values of all pixels in the CIE Lab color space for clustering Class, get k clusters;

Step 6: Calculate the common salient feature map, that is, divide the number of pixels contained in the jth cluster by the total number of pixels in the image, and the result of the division is defined as the weight of the jth cluster, where j=1, 2...k, After obtaining the weights of all k clusters, calculate the saliency value of the cluster by using the weight of the cluster and the distance between the clusters, and assign the saliency value of the cluster to each pixel belonging to the cluster, thereby obtaining a set of common saliency feature maps ;

Step 7: Calculate the final saliency map, that is, the informative saliency map obtained by using the histogram information of each color channel is multiplied by the shared saliency map obtained by k-means clustering in the CIE Lab color space, so as to obtain multiple The final saliency map after salient feature fusion;

Step 8: Extracting the region of interest, that is, the segmentation threshold of the final saliency map is obtained by the maximum inter-class variance method, and the final saliency map is segmented into a binary image template by using the threshold, with "1" representing the region of interest, and " 0" represents the non-interest area, and finally the binary image template is multiplied by the original image to obtain the final ROI extraction result.

Description of drawings

Fig. 1 is a flowchart of the present invention.

Fig. 2 is a group of four sample pictures of remote sensing images used in the present invention.

Fig. 3 is the feature map and the final saliency map of the present invention. (a) is the informative saliency map of the example image, (b) is the shared saliency map of the example image, and (c) is the final saliency map of the example image.

Fig. 4 is a comparison of saliency maps generated by the method of the present invention and other methods for an example image. (a) is the saliency map generated by the Itti method, (b) is the saliency map generated by the GBVS method, (c) is the saliency map generated by the FT method, and (d) is the saliency map generated by the method of the present invention.

FIG. 5 is a comparison of ROIs detected by the method of the present invention and other methods in an example picture. (a) is the region of interest figure detected by the Itti method, (b) is the region of interest detected by the GBVS method, (c) is the region of interest detected by the FT method, (d) is the region of interest detected by the method of the present invention area of interest.

Detailed ways

The present invention will be described in further detail below in conjunction with the accompanying drawings. The overall framework of the present invention is shown in Figure 1, and now introduces the implementation details of each step.

Step 1: Calculate the color histogram;

Input a set of remote sensing images of size M×N As shown in Figure 2, each color channel of each image I _p is obtained separately, and f _c (x, y) is used to represent the color intensity of the image I _p in the position (x, y) of the color channel c, and the remote sensing image is constructed The intensity histogram H _c (i) of different color channels, where M represents the length of the image, N represents the width of the image, the total number of remote sensing images in this group is Q, and Indicates a group of remote sensing images with a quantity of Q, I _p represents the pth piece of a group of remote sensing images, p=1, 2...Q, x, y represent the horizontal and vertical coordinates of the image respectively, x=1, 2...M , y=1, 2...N, c represents the color channel, c=1, 2, 3, i represents the pixel intensity value, i=0, 1...255;

The histogram of each color channel of each image in the group of images can be obtained by the following formula:

${\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; \&delta;</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; )</span>$

Among them, δ _c (x, y) represents the binarized image of color channel c, and the calculation formula is:

${\mathrm{<span\; class="notranslate">\; \&delta;</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; ,</span>& {\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; i</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ,</span>& \mathrm{<span\; class="notranslate">\; others\; wie</span>}\end{array}\right.$

Step 2: Calculate the normalized saliency weight of the color channel c;

According to the color histogram H _c (i) of the color channel c of the image I _p , calculate the information amount In _c (i) of each pixel intensity value i in the color channel, use the information amount to calculate and assign, and finally obtain the image The normalized saliency weight w _c of each color channel of I _p is specifically realized by the following four steps;

(1) According to the color histogram H _c (i) in the color channel c of the image I _p , use the following formula to calculate the information amount In _c (i) of each pixel intensity value in the color channel:

In(i) _c ＝-ln(H _c (i))

(2) assign the amount of information to the pixel in the color channel c equal to the pixel intensity value, and obtain the information amount map LOG _c (x, y) of the color channel c, namely:

i = f _c (x, y)

(3) Using the information quantity map LOG _c (x, y) of the color channel c, the saliency degree h _c is calculated, and the calculation formula is as follows:

${\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; LOG</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; LOG</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}$

There are three color channels, h ₁ represents the salience of color channel 1, h ₂ represents the salience of color channel 2, and h ₃ represents the salience of color channel 3;

(4) Divide the saliency of the color channel c by the salience of the three color channels, and take the negative logarithm of the result to obtain the normalized saliency weight w _c of the color channel:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; )</span>\end{array}\right.$

There are three color channels, w ₁ represents the normalized saliency weight of color channel 1, w ₂ represents the normalized saliency weight of color channel 2, and w ₃ represents the normalized saliency weight of color channel 3;

Step 3: Calculate the significant feature map of the amount of information;

Using the normalized saliency weight w _c of each color channel of the image _Ip , the weighted calculation obtains the preliminary information saliency feature map Smap(x, y) of the image, and performs Gaussian smoothing filtering on the preliminary information saliency feature map to filter out the noise Finally, the final informative feature map SS(x, y) is obtained:

$\mathrm{<span\; class="notranslate">\; Smap</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>$

in, Represents a Gaussian smoothing filter;

After the above steps, the remote sensing image group is obtained Informative salient feature map of each remote sensing image.

Step 4: Convert the remote sensing image from the RGB color space to the CIE Lab color space;

Since the color channel of CIELab removes the brightness information to a certain extent, the reflected content is closer to the essence of color perception, so it can better reflect the color smoothness. Based on the obvious advantages of the CIE Lab space in color uniformity, the choice of Clustering is performed on the CIE Lab color space, and the following advanced color space conversion is performed:

Separately extract remote sensing image groups The R, G, B three color channel values of each pixel in each image are converted to the CIE Lab color space, and the L, a, and b three components of each pixel are obtained. The remote sensing image in the CIE Lab color space group as In the RGB color space, R means red, G means green, and B means blue. The three channels of CIE Lab color space represent brightness L, L=0 represents black, L=100 represents white, and the color is red/green The position a between, a negative value represents green, a positive value represents red, the color is between blue/yellow position b, b negative value represents blue, b positive value represents yellow;

Step five: color feature clustering;

Use the k-means clustering algorithm to complete the pixel clustering in the CIE Lab color space, that is, in the CIE Lab color space, cluster the values of all the pixels in this group of images to obtain k clusters. The specific implementation steps are as follows :

(1) Extract remote sensing image group In the three channels of L, a, and b of each image in the CIE Lab color space, adjust the range of the pixel point value in the three channels, so that the range of the pixel point value of the adjusted three channels is the same;

(2) Simultaneously calculate the pixel values of the three channels of all images in the image group, so that the sum of the squares of the distances between each pixel value and the nearest cluster center is the smallest. At this time, all the pixels with the same nearest cluster center are For a cluster, the sum of squared distances W can be calculated using the following formula:

$\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>\underset{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; a</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>$

In the formula, pi _r represents the pixel value, where r=1, 2...n, n is the number of image pixel points, and a _j represents the cluster center, where j=1, 2...k;

Step 6: Calculate the common salient feature map;

After calculating the weights of all k clusters, you can use the weights of the clusters and the distance between the clusters to calculate the saliency value of the cluster, and assign the saliency value of the cluster to each pixel belonging to the cluster, thus obtaining a set of common The salient feature map, the specific implementation requires the following three steps:

(1) Divide the number of pixels contained in the j-th cluster l _j by the total number of pixels in the image group, and the result of the division is defined as the weight ω(l _j ) of the j-th cluster, where j=1, 2... k;

(2) Define D(l _t , l _j ) as the color distance between two clusters l _t , l _j , and the saliency value CL(l _j ) of each cluster can be calculated by the following formula:

$\mathrm{<span\; class="notranslate">\; CL</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&NotEqual;</span>\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; \&omega;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; \&omega;</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span>}$

in,

$\mathrm{<span\; class="notranslate">\; D.</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; ln</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; -</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\frac{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; ts</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; js</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; ts</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; js</span>}}}<span\; class="notranslate">\; )</span>$

In the formula, the values of j and t are both 1, 2...k, q _ts is the probability that the sth color appears in the m colors of the tth cluster, that is, there are m kinds of pixel values in the tth cluster, and s = 1, 2...m;

(3) After clustering, the saliency value of each pixel is equal to the saliency value of the cluster where the pixel is located, thus obtaining the common feature saliency map SM(x, y):

When ILab _p (x, y)∈l _j , where j=1, 2...k, p=1, 2...Q,

SM(x,y)=CL(l _j )

After the above steps, the remote sensing image group is obtained The common salient feature map of each remote sensing image in .

Step 7: Calculate the final saliency map;

Multiply the informative salient feature map obtained through each color channel with the shared salient feature map obtained by k-means clustering in the CIE Lab color space, so as to obtain the multi-significant feature map of each image in this group of remote sensing images. The final saliency map S(x, y) of :

S(x,y)=SS(x,y)×SM(x,y)

Step 8: Region of interest extraction;

The segmentation threshold of the final saliency map is obtained by the method of maximum inter-class variance, and the threshold is used to segment the final saliency map into a binary image template, using "1" to represent the region of interest, and "0" to represent the non-interest region, and finally Multiply the binary image template with the original image to get the final ROI extraction result.

Effect of the present invention can be further illustrated by following experimental results and analysis:

1. Experimental data

The present invention selects a group of visible light remote sensing images of a certain suburb of Beijing from the SPOT5 satellite source image, and intercepts and generates a group of images with a size of 1024×1024 as the source image of this experiment, as shown in Figure 2.

2. Comparative experiment

In order to evaluate the performance of the method of the present invention, we have designed the following comparative experiments, selected existing representative visual attention methods and selected ITTI method, GBVS method, FT method and the method of the present invention to carry out performance comparison. The saliency map and ROI map generated by different methods are compared subjectively, as shown in Figure 4 and Figure 5. In Figure 4, (a) is the saliency map generated by the Itti method, (b) is the saliency map generated by the GBVS method, (c) is the saliency map generated by the FT method, and (d) is the saliency map generated by the method of the present invention. In Fig. 5, (a) is the region of interest map that Itti method generates, (b) is the region of interest map that GBVS method generates, (c) is the region of interest map that FT method generates, (d) is the method of the present invention The resulting region-of-interest map.

After comparison, it can be seen that the resolution of the saliency map obtained by using the Itti model is very low, only 1/256 of the size of the original image. When finally extracting the region of interest, the saliency map should be enlarged. The GBVS model is based on the Itti model, but only uses the Markov chain when obtaining the saliency map. The region of interest obtained by these two models will be larger than the region that needs to be extracted originally, that is, the unnecessary part will be extracted. Using the FT model, better extraction results can be obtained when the background frequency changes little, but when the background frequency changes greatly, it will interfere with the extraction results, while the algorithm in this paper can get better detection results.

Claims (2)

1. A method for detecting regions of interest in remote sensing images based on the fusion of multiple salient features. This method processes a group of remote sensing images. First, it uses the color information of the remote sensing images to construct color histograms of different color channels and perform weighted calculations. Obtain the salient feature map of the amount of information, and then use the k-means clustering algorithm to cluster a group of remote sensing images in the CIE Lab color space and calculate the salient value, so as to obtain a group of common salient feature maps in the CIE Lab color space, and then fuse The above two groups of images obtain the final saliency map, and finally perform threshold segmentation to extract the region of interest through the maximum inter-class variance method, which is characterized in that it includes the following steps: Step 1: Calculate the color histogram, that is, input a set of remote sensing images with a size of M×N, extract each color channel of each image separately, and use f _c (x, y) to represent the color channel c (x, y ) position, construct the intensity histogram H _c (i) of each remote sensing image in different color channels, where M represents the length of the image, N represents the width of the image, and x and y represent the horizontal and vertical coordinates of the image respectively, x=1, 2...M, y=1, 2...N, c represents the color channel, c=1, 2, 3, i represents the pixel intensity value, i=0, 1...255; Step 2: Calculate the normalized saliency weight of color channel c, that is, calculate the information amount In _c (i) of each pixel intensity value i in the color channel according to the color histogram H _c (i) of color channel c, and calculate the The amount of information is assigned to the pixel point equal to the intensity value of the pixel. After all calculations and assignments are completed, the information amount map LOG _c (x, y) of the color channel c is obtained. Using this information amount map, the salience degree of the color channel c is obtained. h _c , and then use the saliency of each color channel to calculate the normalized saliency weight w _c of each color channel of each image; Step 3: Calculate the informative feature map, that is, use the standardized saliency weight w _c of each color channel to weight and calculate the preliminary informative feature map of each image, and perform Gaussian smoothing filtering on the initially obtained informative feature map, After filtering the noise, the final informative feature map of each image is obtained; Step 4: Convert a group of remote sensing images from the RGB color space to the CIE Lab color space, that is, extract the three color channel values of R, G, and B for each pixel of each image, and convert them to the CIE Lab color space to obtain L, a, b three components, in the RGB color space, R means red red, G means green green, B means blue blue, in CIE Lab color space, L means brightness, L=0 means black, L=100 means White, a represents the position of the color between red/green, a is negative for green, a is positive for red, b represents the position of the color between blue/yellow, b is negative for blue, b is Positive values represent yellow; Step 5: Use the k-means clustering algorithm to complete the pixel clustering in the CIE Lab color space, that is, use the k-means clustering algorithm to map this group of original remote sensing images to the values of all pixels in the CIE Lab color space for clustering Class, get k clusters; Step 6: Calculate the common salient feature map, that is, divide the number of pixels contained in the jth cluster by the total number of pixels in the image, and the result of the division is defined as the weight of the jth cluster, where j=1, 2...k, After obtaining the weights of all k clusters, calculate the saliency value of the cluster by using the weight of the cluster and the distance between the clusters, and assign the saliency value of the cluster to each pixel belonging to the cluster, thereby obtaining a set of common saliency feature maps ; Step 7: Calculate the final saliency map, that is, the information saliency feature map obtained by using the histogram information of each color channel is multiplied by the shared saliency map obtained by k-means clustering in the CIE Lab color space, so as to obtain multiple The final saliency map after salient feature fusion; Step 8: Extracting the region of interest, that is, the segmentation threshold of the final saliency map is obtained by the maximum inter-class variance method, and the final saliency map is segmented into a binary image template by using the threshold, with "1" representing the region of interest, and " 0" represents the non-interest area, and finally the binary image template is multiplied by the original image to obtain the final ROI extraction result. 2. a kind of remote sensing image region of interest extraction method based on salient feature clustering according to claim 1, is characterized in that, the specific process of described step 2 is: 1) Calculate the information amount In _c (i) of each pixel intensity value according to the color histogram H _c (i) in the color channel c: In(i) _c ＝-ln(H _c (i)) 2) assign the information amount to the pixel point equal to the pixel intensity value, and obtain the information amount map LOG _c (x, y) of the color channel c, namely: i = f _c (x, y), 3) Calculate the saliency h _c by using the information quantity map LOG _c (x, y) of the color channel c. Since the image contains three color channels, h ₁ represents the saliency of color channel 1, and h ₂ represents the color The salience of channel 2, h ₃ represents the salience of color channel 3: ${\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; LOG</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; 3</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; LOG</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span>}$ 4) Divide the saliency of the color channel c by the sum of the salience of the three color channels, and take the negative logarithm of the result to obtain the normalized saliency weight w _c of the color channel, $\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; )</span>\\ {\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}{{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; )</span>\end{array}\right.$ Since the image contains three color channels, w ₁ represents the normalized saliency weight of color channel 1, w ₂ represents the normalized saliency weight of color channel 2, and w ₃ represents the normalized saliency weight of color channel 3.